2024-05-09T04:28:15.420 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-09T04:28:15.420 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_pdonnell@teuthology --description fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/iozone}} --verbose --archive /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699071 --name pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi -- /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699071/orig.config.yaml 2024-05-09T04:28:15.479 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T04:28:15.568 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699071 branch: wip-pdonnell-testing-20240508.183908-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/iozone}} email: pdonnell@redhat.com first_in_suite: false job_id: '7699071' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240508.183908-debug ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240508.183908-debug sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 1722 sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240508.183908-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi046.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJHavbgeoqf2/rznUP3xvrXZlNPMW2avsJ//WT8uz6sW3sRfbergISG90a+uQr0zHZH3djONqmms4y7GG2+x7GA= smithi072.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOGYdT4dLQAn/Nvcosg/dC9AG+01W/ZltnjfutuebG/nhzvhKH56fY4ihruJyiCn6ib/pEa7AUNonHjRMxkou7I= smithi110.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPYnHibF3/ahdM4STDUfPXcea4exqz+gAvoxHw0z/uKKYit78VFMqrA9A3DfQZjaxMZXVMC/g06TcEydqR20Kk4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - suites/iozone.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_22:06:20 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-09T04:28:15.569 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa; will attempt to use it 2024-05-09T04:28:15.569 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks 2024-05-09T04:28:15.569 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-09T04:28:15.571 INFO:teuthology.task.internal:Checking packages... 2024-05-09T04:28:15.594 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'c9da4049f638c0f84bdd9b4e33c42e86046ac554' 2024-05-09T04:28:15.594 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-09T04:28:15.594 INFO:teuthology.packaging:ref: None 2024-05-09T04:28:15.594 INFO:teuthology.packaging:tag: None 2024-05-09T04:28:15.595 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240508.183908-debug 2024-05-09T04:28:15.595 INFO:teuthology.packaging:sha1: c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:28:15.595 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240508.183908-debug 2024-05-09T04:28:15.750 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3594.gc9da4049 2024-05-09T04:28:15.752 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-09T04:28:15.757 INFO:teuthology.task.internal:no buildpackages task found 2024-05-09T04:28:15.757 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-09T04:28:15.792 INFO:teuthology.task.internal:Saving configuration 2024-05-09T04:28:15.815 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-09T04:28:15.871 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-09T04:28:15.895 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi046.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699071', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 04:22:44.884692', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJHavbgeoqf2/rznUP3xvrXZlNPMW2avsJ//WT8uz6sW3sRfbergISG90a+uQr0zHZH3djONqmms4y7GG2+x7GA='} 2024-05-09T04:28:15.920 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi072.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699071', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 04:22:44.882182', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOGYdT4dLQAn/Nvcosg/dC9AG+01W/ZltnjfutuebG/nhzvhKH56fY4ihruJyiCn6ib/pEa7AUNonHjRMxkou7I='} 2024-05-09T04:28:15.944 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi110.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699071', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-09 04:22:44.883575', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPYnHibF3/ahdM4STDUfPXcea4exqz+gAvoxHw0z/uKKYit78VFMqrA9A3DfQZjaxMZXVMC/g06TcEydqR20Kk4='} 2024-05-09T04:28:15.944 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-09T04:28:15.951 INFO:teuthology.task.internal:roles: ubuntu@smithi046.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-09T04:28:15.951 INFO:teuthology.task.internal:roles: ubuntu@smithi072.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-09T04:28:15.951 INFO:teuthology.task.internal:roles: ubuntu@smithi110.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-09T04:28:15.951 INFO:teuthology.run_tasks:Running task console_log... 2024-05-09T04:28:16.068 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4f0795c0d0>, signals=[15]) 2024-05-09T04:28:16.068 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-09T04:28:16.074 INFO:teuthology.task.internal:Opening connections... 2024-05-09T04:28:16.074 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T04:28:16.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:28:16.154 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi072.front.sepia.ceph.com 2024-05-09T04:28:16.155 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:28:16.226 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi110.front.sepia.ceph.com 2024-05-09T04:28:16.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:28:16.299 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-09T04:28:16.305 DEBUG:teuthology.orchestra.run.smithi046:> uname -m 2024-05-09T04:28:16.323 INFO:teuthology.orchestra.run.smithi046.stdout:x86_64 2024-05-09T04:28:16.323 DEBUG:teuthology.orchestra.run.smithi046:> cat /etc/os-release 2024-05-09T04:28:16.381 INFO:teuthology.orchestra.run.smithi046.stdout:NAME="CentOS Stream" 2024-05-09T04:28:16.381 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION="9" 2024-05-09T04:28:16.381 INFO:teuthology.orchestra.run.smithi046.stdout:ID="centos" 2024-05-09T04:28:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:ID_LIKE="rhel fedora" 2024-05-09T04:28:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:VERSION_ID="9" 2024-05-09T04:28:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:PLATFORM_ID="platform:el9" 2024-05-09T04:28:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T04:28:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:ANSI_COLOR="0;31" 2024-05-09T04:28:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:LOGO="fedora-logo-icon" 2024-05-09T04:28:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T04:28:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:HOME_URL="https://centos.org/" 2024-05-09T04:28:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T04:28:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T04:28:16.382 INFO:teuthology.orchestra.run.smithi046.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T04:28:16.383 INFO:teuthology.lock.ops:Updating smithi046.front.sepia.ceph.com on lock server 2024-05-09T04:28:16.411 DEBUG:teuthology.orchestra.run.smithi072:> uname -m 2024-05-09T04:28:16.430 INFO:teuthology.orchestra.run.smithi072.stdout:x86_64 2024-05-09T04:28:16.431 DEBUG:teuthology.orchestra.run.smithi072:> cat /etc/os-release 2024-05-09T04:28:16.487 INFO:teuthology.orchestra.run.smithi072.stdout:NAME="CentOS Stream" 2024-05-09T04:28:16.487 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION="9" 2024-05-09T04:28:16.487 INFO:teuthology.orchestra.run.smithi072.stdout:ID="centos" 2024-05-09T04:28:16.487 INFO:teuthology.orchestra.run.smithi072.stdout:ID_LIKE="rhel fedora" 2024-05-09T04:28:16.487 INFO:teuthology.orchestra.run.smithi072.stdout:VERSION_ID="9" 2024-05-09T04:28:16.487 INFO:teuthology.orchestra.run.smithi072.stdout:PLATFORM_ID="platform:el9" 2024-05-09T04:28:16.487 INFO:teuthology.orchestra.run.smithi072.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T04:28:16.487 INFO:teuthology.orchestra.run.smithi072.stdout:ANSI_COLOR="0;31" 2024-05-09T04:28:16.487 INFO:teuthology.orchestra.run.smithi072.stdout:LOGO="fedora-logo-icon" 2024-05-09T04:28:16.487 INFO:teuthology.orchestra.run.smithi072.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T04:28:16.488 INFO:teuthology.orchestra.run.smithi072.stdout:HOME_URL="https://centos.org/" 2024-05-09T04:28:16.488 INFO:teuthology.orchestra.run.smithi072.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T04:28:16.488 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T04:28:16.488 INFO:teuthology.orchestra.run.smithi072.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T04:28:16.488 INFO:teuthology.lock.ops:Updating smithi072.front.sepia.ceph.com on lock server 2024-05-09T04:28:16.511 DEBUG:teuthology.orchestra.run.smithi110:> uname -m 2024-05-09T04:28:16.529 INFO:teuthology.orchestra.run.smithi110.stdout:x86_64 2024-05-09T04:28:16.529 DEBUG:teuthology.orchestra.run.smithi110:> cat /etc/os-release 2024-05-09T04:28:16.586 INFO:teuthology.orchestra.run.smithi110.stdout:NAME="CentOS Stream" 2024-05-09T04:28:16.586 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION="9" 2024-05-09T04:28:16.586 INFO:teuthology.orchestra.run.smithi110.stdout:ID="centos" 2024-05-09T04:28:16.586 INFO:teuthology.orchestra.run.smithi110.stdout:ID_LIKE="rhel fedora" 2024-05-09T04:28:16.586 INFO:teuthology.orchestra.run.smithi110.stdout:VERSION_ID="9" 2024-05-09T04:28:16.586 INFO:teuthology.orchestra.run.smithi110.stdout:PLATFORM_ID="platform:el9" 2024-05-09T04:28:16.586 INFO:teuthology.orchestra.run.smithi110.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-09T04:28:16.586 INFO:teuthology.orchestra.run.smithi110.stdout:ANSI_COLOR="0;31" 2024-05-09T04:28:16.586 INFO:teuthology.orchestra.run.smithi110.stdout:LOGO="fedora-logo-icon" 2024-05-09T04:28:16.586 INFO:teuthology.orchestra.run.smithi110.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-09T04:28:16.587 INFO:teuthology.orchestra.run.smithi110.stdout:HOME_URL="https://centos.org/" 2024-05-09T04:28:16.587 INFO:teuthology.orchestra.run.smithi110.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-09T04:28:16.587 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-09T04:28:16.587 INFO:teuthology.orchestra.run.smithi110.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-09T04:28:16.587 INFO:teuthology.lock.ops:Updating smithi110.front.sepia.ceph.com on lock server 2024-05-09T04:28:16.611 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-09T04:28:16.621 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-09T04:28:16.677 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-09T04:28:16.677 DEBUG:teuthology.orchestra.run.smithi046:> test '!' -e /home/ubuntu/cephtest 2024-05-09T04:28:16.680 DEBUG:teuthology.orchestra.run.smithi072:> test '!' -e /home/ubuntu/cephtest 2024-05-09T04:28:16.682 DEBUG:teuthology.orchestra.run.smithi110:> test '!' -e /home/ubuntu/cephtest 2024-05-09T04:28:16.698 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-09T04:28:16.705 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-09T04:28:16.705 DEBUG:teuthology.orchestra.run.smithi046:> test -z $(ls -A /var/lib/ceph) 2024-05-09T04:28:16.738 DEBUG:teuthology.orchestra.run.smithi072:> test -z $(ls -A /var/lib/ceph) 2024-05-09T04:28:16.740 DEBUG:teuthology.orchestra.run.smithi110:> test -z $(ls -A /var/lib/ceph) 2024-05-09T04:28:16.839 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-09T04:28:16.917 INFO:teuthology.run_tasks:Running task kernel... 2024-05-09T04:28:16.931 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-05-09T04:28:16.931 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-09T04:28:16.931 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-09T04:28:16.931 DEBUG:teuthology.orchestra.run.smithi046:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T04:28:16.949 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:28:16.949 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-05-09T04:28:17.007 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-437.el9.x86_64 2024-05-09T04:28:17.007 INFO:teuthology.task.kernel:Running kernel on smithi046: 5.14.0-437.el9.x86_64 2024-05-09T04:28:17.008 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum install -y kernel 2024-05-09T04:28:19.635 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 8.1 MB 00:00 2024-05-09T04:28:22.769 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 9.5 MB/s | 19 MB 00:02 2024-05-09T04:28:27.320 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 7.3 MB/s | 6.2 MB 00:00 2024-05-09T04:28:29.079 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-05-09T04:28:30.555 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-05-09T04:28:35.847 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-09T04:28:37.969 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-09T04:28:37.970 INFO:teuthology.orchestra.run.smithi046.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-09T04:28:38.059 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-09T04:28:38.068 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-09T04:28:38.069 INFO:teuthology.orchestra.run.smithi046.stdout: Package Architecture Version Repository Size 2024-05-09T04:28:38.069 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-09T04:28:38.069 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-09T04:28:38.069 INFO:teuthology.orchestra.run.smithi046.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-09T04:28:38.069 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-09T04:28:38.069 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-09T04:28:38.069 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-09T04:28:38.069 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-09T04:28:38.069 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:28:38.069 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-09T04:28:38.069 INFO:teuthology.orchestra.run.smithi046.stdout:================================================================================ 2024-05-09T04:28:38.069 INFO:teuthology.orchestra.run.smithi046.stdout:Install 4 Packages 2024-05-09T04:28:38.069 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:28:38.070 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 78 M 2024-05-09T04:28:38.070 INFO:teuthology.orchestra.run.smithi046.stdout:Installed size: 123 M 2024-05-09T04:28:38.070 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-09T04:28:38.331 INFO:teuthology.orchestra.run.smithi046.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 2.1 MB/s | 484 kB 00:00 2024-05-09T04:28:38.823 INFO:teuthology.orchestra.run.smithi046.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 22 MB/s | 15 MB 00:00 2024-05-09T04:28:39.407 INFO:teuthology.orchestra.run.smithi046.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 26 MB/s | 28 MB 00:01 2024-05-09T04:28:39.757 INFO:teuthology.orchestra.run.smithi046.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 21 MB/s | 34 MB 00:01 2024-05-09T04:28:39.757 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-09T04:28:39.757 INFO:teuthology.orchestra.run.smithi046.stdout:Total 46 MB/s | 78 MB 00:01 2024-05-09T04:28:40.464 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-09T04:28:40.877 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-09T04:28:40.877 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-09T04:28:41.838 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-09T04:28:41.838 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-09T04:28:44.378 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-09T04:28:45.014 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-09T04:28:45.098 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T04:28:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T04:28:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T04:28:53.120 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T04:28:53.168 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T04:28:58.332 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T04:29:29.554 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T04:29:29.594 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-09T04:29:29.880 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-09T04:29:29.881 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-09T04:29:29.881 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-09T04:29:29.881 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-09T04:29:30.690 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-09T04:29:30.690 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:29:30.691 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-09T04:29:30.691 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-09T04:29:30.691 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-09T04:29:30.691 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-09T04:29:30.691 INFO:teuthology.orchestra.run.smithi046.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-09T04:29:30.691 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:29:30.691 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-09T04:29:31.015 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T04:29:31.102 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T04:29:31.103 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T04:29:31.103 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-09T04:29:31.103 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-09T04:29:31.103 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-09T04:29:31.103 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-09T04:29:31.103 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi046.front.sepia.ceph.com, path=None, version=distro) 2024-05-09T04:29:31.103 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q kernel | sort -rV | head -n 1 2024-05-09T04:29:31.154 INFO:teuthology.orchestra.run.smithi046.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-09T04:29:31.154 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-09T04:29:31.154 DEBUG:teuthology.orchestra.run.smithi046:> sudo rpm -qi grub2-tools 2024-05-09T04:29:31.195 INFO:teuthology.orchestra.run.smithi046.stdout:Name : grub2-tools 2024-05-09T04:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:Epoch : 1 2024-05-09T04:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:Version : 2.06 2024-05-09T04:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:Release : 68.el9 2024-05-09T04:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:Architecture: x86_64 2024-05-09T04:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-09T04:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:Group : Unspecified 2024-05-09T04:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:Size : 8264254 2024-05-09T04:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:License : GPLv3+ 2024-05-09T04:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-09T04:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-09T04:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-09T04:29:31.196 INFO:teuthology.orchestra.run.smithi046.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-09T04:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:Packager : builder@centos.org 2024-05-09T04:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:Vendor : CentOS 2024-05-09T04:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-09T04:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:Summary : Support tools for GRUB. 2024-05-09T04:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:Description : 2024-05-09T04:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-09T04:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-09T04:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-09T04:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:hardware devices. 2024-05-09T04:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:29:31.197 INFO:teuthology.orchestra.run.smithi046.stdout:This subpackage provides tools for support of all platforms. 2024-05-09T04:29:31.198 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-09T04:29:31.198 INFO:teuthology.task.kernel:Updating grub on smithi046 to boot 5.14.0-444.el9.x86_64 2024-05-09T04:29:31.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-09T04:29:31.818 INFO:teuthology.orchestra.run.smithi046.stderr:Generating grub configuration file ... 2024-05-09T04:29:33.230 INFO:teuthology.orchestra.run.smithi046.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-09T04:29:33.267 INFO:teuthology.orchestra.run.smithi046.stderr:done 2024-05-09T04:29:33.269 DEBUG:teuthology.orchestra.run.smithi046:> mktemp 2024-05-09T04:29:33.285 INFO:teuthology.orchestra.run.smithi046.stdout:/tmp/tmp.SONwTpRHEn 2024-05-09T04:29:33.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.SONwTpRHEn 2024-05-09T04:29:33.351 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0666 /tmp/tmp.SONwTpRHEn 2024-05-09T04:29:33.493 DEBUG:teuthology.orchestra.remote:smithi046:/tmp/tmp.SONwTpRHEn is 6KB 2024-05-09T04:29:33.542 DEBUG:teuthology.orchestra.run.smithi046:> rm -fr /tmp/tmp.SONwTpRHEn 2024-05-09T04:29:33.556 DEBUG:teuthology.orchestra.run.smithi046:> sudo /bin/ls /boot/loader/entries || true 2024-05-09T04:29:33.619 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-09T04:29:33.619 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-09T04:29:33.619 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-09T04:29:33.619 INFO:teuthology.orchestra.run.smithi046.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-09T04:29:33.621 DEBUG:teuthology.orchestra.run.smithi046:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-09T04:29:33.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo shutdown -r now 2024-05-09T04:30:03.766 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-09T04:30:03.766 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-09T04:30:03.767 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:30:38.366 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-05-09T04:30:47.368 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-09T04:30:47.369 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:31:05.758 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.46 2024-05-09T04:31:17.760 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-09T04:31:17.761 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:31:19.553 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-05-09T04:31:19.991 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-09T04:31:19.991 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-09T04:31:19.992 DEBUG:teuthology.orchestra.run.smithi046:> uname -r 2024-05-09T04:31:20.011 INFO:teuthology.orchestra.run.smithi046.stdout:5.14.0-444.el9.x86_64 2024-05-09T04:31:20.011 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-09T04:31:20.011 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-09T04:31:20.011 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-09T04:31:21.012 DEBUG:teuthology.parallel:result is None 2024-05-09T04:31:21.012 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-09T04:31:21.020 INFO:teuthology.task.internal:Creating test directory... 2024-05-09T04:31:21.020 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T04:31:21.023 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T04:31:21.026 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-09T04:31:21.044 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-09T04:31:21.050 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-09T04:31:21.098 INFO:teuthology.task.internal:Creating archive directory... 2024-05-09T04:31:21.099 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T04:31:21.101 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T04:31:21.103 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-09T04:31:21.208 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-09T04:31:21.215 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-09T04:31:21.215 DEBUG:teuthology.orchestra.run.smithi072:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T04:31:21.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:31:21.232 DEBUG:teuthology.orchestra.run.smithi110:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-09T04:31:21.248 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:31:21.249 DEBUG:teuthology.orchestra.run.smithi046:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T04:31:21.253 DEBUG:teuthology.orchestra.run.smithi072:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T04:31:21.275 DEBUG:teuthology.orchestra.run.smithi110:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-09T04:31:21.310 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:31:21.330 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:31:21.334 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:31:21.347 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:31:21.362 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:31:21.384 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-09T04:31:21.385 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-09T04:31:21.394 INFO:teuthology.task.internal:Configuring sudo... 2024-05-09T04:31:21.394 DEBUG:teuthology.orchestra.run.smithi046:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T04:31:21.396 DEBUG:teuthology.orchestra.run.smithi072:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T04:31:21.399 DEBUG:teuthology.orchestra.run.smithi110:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-09T04:31:21.427 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-09T04:31:21.436 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-09T04:31:21.436 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T04:31:21.463 DEBUG:teuthology.orchestra.run.smithi072:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T04:31:21.465 DEBUG:teuthology.orchestra.run.smithi110:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-09T04:31:21.483 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:31:21.562 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:31:21.604 DEBUG:teuthology.orchestra.run.smithi046:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:31:21.687 DEBUG:teuthology.orchestra.run.smithi046:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:31:21.729 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:31:21.729 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T04:31:21.795 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:31:21.838 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:31:21.888 DEBUG:teuthology.orchestra.run.smithi072:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:31:21.972 DEBUG:teuthology.orchestra.run.smithi072:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:31:22.012 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-09T04:31:22.012 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T04:31:22.076 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:31:22.117 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-09T04:31:22.181 DEBUG:teuthology.orchestra.run.smithi110:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:31:22.225 DEBUG:teuthology.orchestra.run.smithi110:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-09T04:31:22.267 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-09T04:31:22.267 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-09T04:31:22.333 DEBUG:teuthology.orchestra.run.smithi046:> sudo service rsyslog restart 2024-05-09T04:31:22.335 DEBUG:teuthology.orchestra.run.smithi072:> sudo service rsyslog restart 2024-05-09T04:31:22.338 DEBUG:teuthology.orchestra.run.smithi110:> sudo service rsyslog restart 2024-05-09T04:31:22.376 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T04:31:22.396 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T04:31:22.404 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T04:31:22.665 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-09T04:31:22.673 INFO:teuthology.task.internal:Starting timer... 2024-05-09T04:31:22.673 INFO:teuthology.run_tasks:Running task pcp... 2024-05-09T04:31:22.774 INFO:teuthology.run_tasks:Running task selinux... 2024-05-09T04:31:22.817 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-09T04:31:22.817 DEBUG:teuthology.orchestra.run.smithi046:> sudo service auditd rotate 2024-05-09T04:31:22.884 INFO:teuthology.orchestra.run.smithi046.stdout:Rotating logs: 2024-05-09T04:31:22.886 DEBUG:teuthology.orchestra.run.smithi072:> sudo service auditd rotate 2024-05-09T04:31:22.961 INFO:teuthology.orchestra.run.smithi072.stdout:Rotating logs: 2024-05-09T04:31:22.962 DEBUG:teuthology.orchestra.run.smithi110:> sudo service auditd rotate 2024-05-09T04:31:23.043 INFO:teuthology.orchestra.run.smithi110.stdout:Rotating logs: 2024-05-09T04:31:23.045 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-09T04:31:23.045 DEBUG:teuthology.orchestra.run.smithi046:> /usr/sbin/getenforce 2024-05-09T04:31:23.065 INFO:teuthology.orchestra.run.smithi046.stdout:Permissive 2024-05-09T04:31:23.065 DEBUG:teuthology.orchestra.run.smithi072:> /usr/sbin/getenforce 2024-05-09T04:31:23.091 INFO:teuthology.orchestra.run.smithi072.stdout:Permissive 2024-05-09T04:31:23.091 DEBUG:teuthology.orchestra.run.smithi110:> /usr/sbin/getenforce 2024-05-09T04:31:23.117 INFO:teuthology.orchestra.run.smithi110.stdout:Permissive 2024-05-09T04:31:23.117 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi046.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi072.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi110.front.sepia.ceph.com': 'permissive'} 2024-05-09T04:31:23.118 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T04:31:23.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:31:23.144 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T04:31:23.171 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:31:23.171 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T04:31:23.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:31:23.196 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-09T04:31:23.196 DEBUG:teuthology.orchestra.run.smithi046:> sudo /usr/sbin/setenforce permissive 2024-05-09T04:31:23.290 DEBUG:teuthology.orchestra.run.smithi072:> sudo /usr/sbin/setenforce permissive 2024-05-09T04:31:23.328 DEBUG:teuthology.orchestra.run.smithi110:> sudo /usr/sbin/setenforce permissive 2024-05-09T04:31:23.361 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-09T04:31:23.370 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-09T04:31:23.386 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-09T04:31:23.387 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi046.front.sepia.ceph.com,smithi072.front.sepia.ceph.com,smithi110.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-09T04:35:44.017 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi046.front.sepia.ceph.com'), Remote(name='ubuntu@smithi072.front.sepia.ceph.com'), Remote(name='ubuntu@smithi110.front.sepia.ceph.com')] 2024-05-09T04:35:44.018 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-09T04:35:44.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi046.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:35:44.098 DEBUG:teuthology.orchestra.run.smithi046:> true 2024-05-09T04:35:44.182 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi046.front.sepia.ceph.com' 2024-05-09T04:35:44.183 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-05-09T04:35:44.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi072.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:35:44.257 DEBUG:teuthology.orchestra.run.smithi072:> true 2024-05-09T04:35:44.333 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi072.front.sepia.ceph.com' 2024-05-09T04:35:44.333 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-05-09T04:35:44.334 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi110.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-09T04:35:44.409 DEBUG:teuthology.orchestra.run.smithi110:> true 2024-05-09T04:35:44.491 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi110.front.sepia.ceph.com' 2024-05-09T04:35:44.491 INFO:teuthology.run_tasks:Running task clock... 2024-05-09T04:35:44.501 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-09T04:35:44.501 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T04:35:44.502 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T04:35:44.504 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T04:35:44.504 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T04:35:44.506 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-09T04:35:44.507 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T04:35:44.533 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T04:35:44.538 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T04:35:44.546 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T04:35:44.555 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T04:35:44.562 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-09T04:35:44.568 INFO:teuthology.orchestra.run.smithi072.stderr:sudo: ntpd: command not found 2024-05-09T04:35:44.576 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-09T04:35:44.578 INFO:teuthology.orchestra.run.smithi072.stdout:506 Cannot talk to daemon 2024-05-09T04:35:44.582 INFO:teuthology.orchestra.run.smithi046.stderr:sudo: ntpd: command not found 2024-05-09T04:35:44.591 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T04:35:44.594 INFO:teuthology.orchestra.run.smithi046.stdout:506 Cannot talk to daemon 2024-05-09T04:35:44.600 INFO:teuthology.orchestra.run.smithi110.stderr:sudo: ntpd: command not found 2024-05-09T04:35:44.604 INFO:teuthology.orchestra.run.smithi072.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T04:35:44.610 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T04:35:44.610 INFO:teuthology.orchestra.run.smithi110.stdout:506 Cannot talk to daemon 2024-05-09T04:35:44.624 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-09T04:35:44.625 INFO:teuthology.orchestra.run.smithi046.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T04:35:44.637 INFO:teuthology.orchestra.run.smithi110.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-09T04:35:44.661 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2024-05-09T04:35:44.664 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T04:35:44.664 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2024-05-09T04:35:44.664 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:44.664 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:44.665 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:44.665 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:44.682 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-05-09T04:35:44.686 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T04:35:44.686 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-05-09T04:35:44.686 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:44.686 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:44.686 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:44.686 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:44.700 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-05-09T04:35:44.703 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T04:35:44.703 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-05-09T04:35:44.703 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:44.703 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:44.703 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:44.703 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T04:35:44.704 INFO:teuthology.run_tasks:Running task install... 2024-05-09T04:35:44.712 DEBUG:teuthology.task.install:project ceph 2024-05-09T04:35:44.712 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'}} 2024-05-09T04:35:44.712 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'} 2024-05-09T04:35:44.713 INFO:teuthology.task.install:Using flavor: default 2024-05-09T04:35:44.719 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-09T04:35:44.719 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-09T04:35:44.719 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T04:35:44.720 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:35:44.721 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T04:35:44.722 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:35:44.723 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'tag': None, 'wait_for_package': False} 2024-05-09T04:35:44.723 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:35:44.875 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T04:35:44.875 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T04:35:44.878 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T04:35:44.878 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T04:35:45.059 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/ 2024-05-09T04:35:45.059 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3594.gc9da4049 2024-05-09T04:35:45.144 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T04:35:45.144 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-09T04:35:45.144 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T04:35:45.168 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T04:35:45.169 DEBUG:teuthology.orchestra.run.smithi072:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T04:35:45.213 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T04:35:45.214 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:35:45.214 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T04:35:45.234 DEBUG:teuthology.orchestra.run.smithi072:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T04:35:45.246 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T04:35:45.246 DEBUG:teuthology.orchestra.run.smithi046:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T04:35:45.306 DEBUG:teuthology.orchestra.run.smithi072:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T04:35:45.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T04:35:45.321 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-09T04:35:45.321 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-09T04:35:45.321 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-09T04:35:45.349 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-09T04:35:45.349 DEBUG:teuthology.orchestra.run.smithi110:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/c9da4049f638c0f84bdd9b4e33c42e86046ac554/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-09T04:35:45.370 INFO:teuthology.orchestra.run.smithi072.stdout:check_obsoletes = 1 2024-05-09T04:35:45.371 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum clean all 2024-05-09T04:35:45.394 DEBUG:teuthology.orchestra.run.smithi046:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T04:35:45.416 DEBUG:teuthology.orchestra.run.smithi110:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-09T04:35:45.460 INFO:teuthology.orchestra.run.smithi046.stdout:check_obsoletes = 1 2024-05-09T04:35:45.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum clean all 2024-05-09T04:35:45.492 DEBUG:teuthology.orchestra.run.smithi110:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-09T04:35:45.556 INFO:teuthology.orchestra.run.smithi110.stdout:check_obsoletes = 1 2024-05-09T04:35:45.558 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum clean all 2024-05-09T04:35:45.597 INFO:teuthology.orchestra.run.smithi072.stdout:54 files removed 2024-05-09T04:35:45.625 DEBUG:teuthology.orchestra.run.smithi072:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T04:35:45.718 INFO:teuthology.orchestra.run.smithi046.stdout:54 files removed 2024-05-09T04:35:45.746 DEBUG:teuthology.orchestra.run.smithi046:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T04:35:45.787 INFO:teuthology.orchestra.run.smithi110.stdout:54 files removed 2024-05-09T04:35:45.815 DEBUG:teuthology.orchestra.run.smithi110:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-09T04:35:46.456 INFO:teuthology.orchestra.run.smithi072.stdout:ceph packages for x86_64 146 kB/s | 84 kB 00:00 2024-05-09T04:35:46.484 INFO:teuthology.orchestra.run.smithi046.stdout:ceph packages for x86_64 185 kB/s | 84 kB 00:00 2024-05-09T04:35:46.534 INFO:teuthology.orchestra.run.smithi110.stdout:ceph packages for x86_64 182 kB/s | 84 kB 00:00 2024-05-09T04:35:46.933 INFO:teuthology.orchestra.run.smithi072.stdout:ceph noarch packages 27 kB/s | 12 kB 00:00 2024-05-09T04:35:46.985 INFO:teuthology.orchestra.run.smithi046.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-05-09T04:35:47.169 INFO:teuthology.orchestra.run.smithi110.stdout:ceph noarch packages 20 kB/s | 12 kB 00:00 2024-05-09T04:35:47.478 INFO:teuthology.orchestra.run.smithi110.stdout:ceph source packages 6.9 kB/s | 1.9 kB 00:00 2024-05-09T04:35:47.559 INFO:teuthology.orchestra.run.smithi072.stdout:ceph source packages 3.2 kB/s | 1.9 kB 00:00 2024-05-09T04:35:47.710 INFO:teuthology.orchestra.run.smithi046.stdout:ceph source packages 2.8 kB/s | 1.9 kB 00:00 2024-05-09T04:35:48.203 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.1 MB 00:00 2024-05-09T04:35:48.268 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.1 MB 00:00 2024-05-09T04:35:50.211 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-05-09T04:35:50.361 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - BaseOS 2.8 MB/s | 8.1 MB 00:02 2024-05-09T04:35:52.312 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-05-09T04:35:56.387 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - CRB 2.6 MB/s | 6.2 MB 00:02 2024-05-09T04:35:58.171 INFO:teuthology.orchestra.run.smithi046.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-05-09T04:35:58.897 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - AppStream 2.0 MB/s | 19 MB 00:09 2024-05-09T04:35:59.172 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 6.2 MB 00:03 2024-05-09T04:36:00.431 INFO:teuthology.orchestra.run.smithi046.stdout:Extra Packages for Enterprise Linux 10 MB/s | 21 MB 00:02 2024-05-09T04:36:00.747 INFO:teuthology.orchestra.run.smithi110.stdout:CentOS Stream 9 - Extras packages 40 kB/s | 16 kB 00:00 2024-05-09T04:36:02.302 INFO:teuthology.orchestra.run.smithi110.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-05-09T04:36:02.932 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-05-09T04:36:04.627 INFO:teuthology.orchestra.run.smithi072.stdout:CentOS Stream 9 - Extras packages 30 kB/s | 16 kB 00:00 2024-05-09T04:36:05.914 INFO:teuthology.orchestra.run.smithi046.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2024-05-09T04:36:06.947 INFO:teuthology.orchestra.run.smithi072.stdout:Extra Packages for Enterprise Linux 9.7 MB/s | 21 MB 00:02 2024-05-09T04:36:07.449 INFO:teuthology.orchestra.run.smithi110.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-05-09T04:36:08.350 INFO:teuthology.orchestra.run.smithi046.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:36:08.351 INFO:teuthology.orchestra.run.smithi046.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:36:08.362 INFO:teuthology.orchestra.run.smithi046.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T04:36:08.363 INFO:teuthology.orchestra.run.smithi046.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T04:36:08.364 INFO:teuthology.orchestra.run.smithi046.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T04:36:08.365 INFO:teuthology.orchestra.run.smithi046.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T04:36:08.367 INFO:teuthology.orchestra.run.smithi046.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T04:36:08.369 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T04:36:08.370 INFO:teuthology.orchestra.run.smithi046.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T04:36:08.436 INFO:teuthology.orchestra.run.smithi046.stdout:Dependencies resolved. 2024-05-09T04:36:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-05-09T04:36:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: Package Arch Version Repository Size 2024-05-09T04:36:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-05-09T04:36:08.458 INFO:teuthology.orchestra.run.smithi046.stdout:Installing: 2024-05-09T04:36:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T04:36:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T04:36:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T04:36:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T04:36:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T04:36:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T04:36:08.458 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T04:36:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T04:36:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T04:36:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T04:36:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T04:36:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T04:36:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T04:36:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 763 k 2024-05-09T04:36:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T04:36:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T04:36:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T04:36:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T04:36:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T04:36:08.459 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T04:36:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T04:36:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T04:36:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T04:36:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T04:36:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T04:36:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T04:36:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T04:36:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T04:36:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T04:36:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 162 k 2024-05-09T04:36:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 321 k 2024-05-09T04:36:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 301 k 2024-05-09T04:36:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T04:36:08.460 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T04:36:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T04:36:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T04:36:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T04:36:08.461 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrading: 2024-05-09T04:36:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T04:36:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T04:36:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T04:36:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T04:36:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T04:36:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T04:36:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T04:36:08.461 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T04:36:08.461 INFO:teuthology.orchestra.run.smithi046.stdout:Installing dependencies: 2024-05-09T04:36:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T04:36:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T04:36:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T04:36:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T04:36:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T04:36:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T04:36:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T04:36:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T04:36:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T04:36:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T04:36:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T04:36:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T04:36:08.462 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T04:36:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T04:36:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T04:36:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T04:36:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T04:36:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T04:36:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T04:36:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T04:36:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T04:36:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T04:36:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T04:36:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T04:36:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T04:36:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T04:36:08.463 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T04:36:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T04:36:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T04:36:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T04:36:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T04:36:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T04:36:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T04:36:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T04:36:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T04:36:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T04:36:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T04:36:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T04:36:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T04:36:08.464 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T04:36:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T04:36:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T04:36:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T04:36:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T04:36:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T04:36:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T04:36:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T04:36:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T04:36:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T04:36:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T04:36:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T04:36:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T04:36:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T04:36:08.465 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T04:36:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T04:36:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T04:36:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T04:36:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T04:36:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T04:36:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T04:36:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T04:36:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T04:36:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T04:36:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T04:36:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T04:36:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T04:36:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T04:36:08.466 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T04:36:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T04:36:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T04:36:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T04:36:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T04:36:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T04:36:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T04:36:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T04:36:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T04:36:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T04:36:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T04:36:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T04:36:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T04:36:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T04:36:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T04:36:08.467 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T04:36:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T04:36:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T04:36:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T04:36:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T04:36:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T04:36:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T04:36:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T04:36:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T04:36:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T04:36:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T04:36:08.468 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T04:36:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T04:36:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T04:36:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T04:36:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T04:36:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T04:36:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T04:36:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T04:36:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T04:36:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T04:36:08.469 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T04:36:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T04:36:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T04:36:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T04:36:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T04:36:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T04:36:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T04:36:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:Installing weak dependencies: 2024-05-09T04:36:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T04:36:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:36:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction Summary 2024-05-09T04:36:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:======================================================================================= 2024-05-09T04:36:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:Install 148 Packages 2024-05-09T04:36:08.470 INFO:teuthology.orchestra.run.smithi046.stdout:Upgrade 8 Packages 2024-05-09T04:36:08.470 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:36:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:Total download size: 219 M 2024-05-09T04:36:08.471 INFO:teuthology.orchestra.run.smithi046.stdout:Downloading Packages: 2024-05-09T04:36:09.742 INFO:teuthology.orchestra.run.smithi110.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:36:09.742 INFO:teuthology.orchestra.run.smithi110.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:36:09.753 INFO:teuthology.orchestra.run.smithi110.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T04:36:09.753 INFO:teuthology.orchestra.run.smithi110.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T04:36:09.754 INFO:teuthology.orchestra.run.smithi110.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T04:36:09.755 INFO:teuthology.orchestra.run.smithi110.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T04:36:09.757 INFO:teuthology.orchestra.run.smithi110.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T04:36:09.759 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T04:36:09.760 INFO:teuthology.orchestra.run.smithi110.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T04:36:09.816 INFO:teuthology.orchestra.run.smithi046.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 33 kB/s | 6.6 kB 00:00 2024-05-09T04:36:09.820 INFO:teuthology.orchestra.run.smithi110.stdout:Dependencies resolved. 2024-05-09T04:36:09.845 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-05-09T04:36:09.845 INFO:teuthology.orchestra.run.smithi110.stdout: Package Arch Version Repository Size 2024-05-09T04:36:09.845 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-05-09T04:36:09.845 INFO:teuthology.orchestra.run.smithi110.stdout:Installing: 2024-05-09T04:36:09.845 INFO:teuthology.orchestra.run.smithi110.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T04:36:09.845 INFO:teuthology.orchestra.run.smithi110.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T04:36:09.845 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T04:36:09.845 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T04:36:09.846 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T04:36:09.846 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T04:36:09.846 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T04:36:09.846 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T04:36:09.846 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T04:36:09.846 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T04:36:09.846 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T04:36:09.846 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T04:36:09.846 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T04:36:09.846 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 763 k 2024-05-09T04:36:09.846 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T04:36:09.846 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T04:36:09.846 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T04:36:09.846 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T04:36:09.847 INFO:teuthology.orchestra.run.smithi110.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T04:36:09.847 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T04:36:09.847 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T04:36:09.847 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T04:36:09.847 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T04:36:09.847 INFO:teuthology.orchestra.run.smithi110.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T04:36:09.847 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T04:36:09.847 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T04:36:09.847 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T04:36:09.847 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T04:36:09.847 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T04:36:09.847 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 162 k 2024-05-09T04:36:09.847 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 321 k 2024-05-09T04:36:09.847 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 301 k 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrading: 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout:Installing dependencies: 2024-05-09T04:36:09.848 INFO:teuthology.orchestra.run.smithi110.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T04:36:09.849 INFO:teuthology.orchestra.run.smithi110.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T04:36:09.849 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T04:36:09.849 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T04:36:09.849 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T04:36:09.849 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T04:36:09.849 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T04:36:09.849 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T04:36:09.849 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T04:36:09.849 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T04:36:09.849 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T04:36:09.849 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T04:36:09.849 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T04:36:09.849 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T04:36:09.849 INFO:teuthology.orchestra.run.smithi110.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T04:36:09.850 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T04:36:09.850 INFO:teuthology.orchestra.run.smithi110.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T04:36:09.850 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T04:36:09.850 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T04:36:09.850 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T04:36:09.850 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T04:36:09.850 INFO:teuthology.orchestra.run.smithi110.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T04:36:09.850 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T04:36:09.850 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T04:36:09.850 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T04:36:09.850 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T04:36:09.850 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T04:36:09.850 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T04:36:09.850 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T04:36:09.851 INFO:teuthology.orchestra.run.smithi110.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T04:36:09.851 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T04:36:09.851 INFO:teuthology.orchestra.run.smithi110.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T04:36:09.851 INFO:teuthology.orchestra.run.smithi110.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T04:36:09.851 INFO:teuthology.orchestra.run.smithi110.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T04:36:09.851 INFO:teuthology.orchestra.run.smithi110.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T04:36:09.851 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T04:36:09.851 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T04:36:09.851 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T04:36:09.851 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T04:36:09.851 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T04:36:09.851 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T04:36:09.851 INFO:teuthology.orchestra.run.smithi110.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T04:36:09.851 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T04:36:09.851 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T04:36:09.852 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T04:36:09.853 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T04:36:09.853 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T04:36:09.853 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T04:36:09.853 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T04:36:09.853 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T04:36:09.853 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T04:36:09.853 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T04:36:09.853 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T04:36:09.853 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T04:36:09.853 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T04:36:09.853 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T04:36:09.853 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T04:36:09.853 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T04:36:09.853 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T04:36:09.853 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T04:36:09.854 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T04:36:09.854 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T04:36:09.854 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T04:36:09.854 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T04:36:09.854 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T04:36:09.854 INFO:teuthology.orchestra.run.smithi110.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T04:36:09.854 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T04:36:09.854 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T04:36:09.854 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T04:36:09.854 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T04:36:09.855 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T04:36:09.855 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T04:36:09.856 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T04:36:09.856 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T04:36:09.856 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T04:36:09.856 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T04:36:09.856 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T04:36:09.856 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T04:36:09.856 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T04:36:09.856 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T04:36:09.856 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T04:36:09.856 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T04:36:09.856 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T04:36:09.856 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T04:36:09.856 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T04:36:09.856 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T04:36:09.856 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T04:36:09.857 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T04:36:09.857 INFO:teuthology.orchestra.run.smithi110.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T04:36:09.857 INFO:teuthology.orchestra.run.smithi110.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T04:36:09.857 INFO:teuthology.orchestra.run.smithi110.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T04:36:09.857 INFO:teuthology.orchestra.run.smithi110.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T04:36:09.857 INFO:teuthology.orchestra.run.smithi110.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T04:36:09.857 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T04:36:09.857 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T04:36:09.857 INFO:teuthology.orchestra.run.smithi110.stdout:Installing weak dependencies: 2024-05-09T04:36:09.857 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T04:36:09.857 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:36:09.857 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction Summary 2024-05-09T04:36:09.857 INFO:teuthology.orchestra.run.smithi110.stdout:======================================================================================= 2024-05-09T04:36:09.857 INFO:teuthology.orchestra.run.smithi110.stdout:Install 148 Packages 2024-05-09T04:36:09.857 INFO:teuthology.orchestra.run.smithi110.stdout:Upgrade 8 Packages 2024-05-09T04:36:09.858 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:36:09.858 INFO:teuthology.orchestra.run.smithi110.stdout:Total download size: 219 M 2024-05-09T04:36:09.858 INFO:teuthology.orchestra.run.smithi110.stdout:Downloading Packages: 2024-05-09T04:36:10.651 INFO:teuthology.orchestra.run.smithi110.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 18 kB/s | 6.6 kB 00:00 2024-05-09T04:36:11.241 INFO:teuthology.orchestra.run.smithi046.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 605 kB/s | 863 kB 00:01 2024-05-09T04:36:11.400 INFO:teuthology.orchestra.run.smithi046.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 927 kB/s | 147 kB 00:00 2024-05-09T04:36:11.993 INFO:teuthology.orchestra.run.smithi110.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 643 kB/s | 863 kB 00:01 2024-05-09T04:36:12.132 INFO:teuthology.orchestra.run.smithi072.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-05-09T04:36:12.243 INFO:teuthology.orchestra.run.smithi110.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 587 kB/s | 147 kB 00:00 2024-05-09T04:36:12.775 INFO:teuthology.orchestra.run.smithi046.stdout:(4/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 1.6 MB/s | 2.3 MB 00:01 2024-05-09T04:36:13.350 INFO:teuthology.orchestra.run.smithi046.stdout:(5/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 2.6 MB/s | 1.5 MB 00:00 2024-05-09T04:36:14.459 INFO:teuthology.orchestra.run.smithi072.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:36:14.459 INFO:teuthology.orchestra.run.smithi072.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-09T04:36:14.470 INFO:teuthology.orchestra.run.smithi072.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-09T04:36:14.471 INFO:teuthology.orchestra.run.smithi072.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-09T04:36:14.472 INFO:teuthology.orchestra.run.smithi072.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-09T04:36:14.473 INFO:teuthology.orchestra.run.smithi072.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-09T04:36:14.474 INFO:teuthology.orchestra.run.smithi072.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-09T04:36:14.477 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-09T04:36:14.477 INFO:teuthology.orchestra.run.smithi072.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-09T04:36:14.493 INFO:teuthology.orchestra.run.smithi110.stdout:(4/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 1.0 MB/s | 2.3 MB 00:02 2024-05-09T04:36:14.517 INFO:teuthology.orchestra.run.smithi046.stdout:(6/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 4.2 MB/s | 4.9 MB 00:01 2024-05-09T04:36:14.538 INFO:teuthology.orchestra.run.smithi072.stdout:Dependencies resolved. 2024-05-09T04:36:14.566 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-05-09T04:36:14.566 INFO:teuthology.orchestra.run.smithi072.stdout: Package Arch Version Repository Size 2024-05-09T04:36:14.566 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-05-09T04:36:14.566 INFO:teuthology.orchestra.run.smithi072.stdout:Installing: 2024-05-09T04:36:14.567 INFO:teuthology.orchestra.run.smithi072.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-09T04:36:14.567 INFO:teuthology.orchestra.run.smithi072.stdout: ceph x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 6.6 k 2024-05-09T04:36:14.567 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.4 M 2024-05-09T04:36:14.567 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 863 k 2024-05-09T04:36:14.567 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 147 k 2024-05-09T04:36:14.567 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 1.5 M 2024-05-09T04:36:14.567 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 142 k 2024-05-09T04:36:14.567 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 3.5 M 2024-05-09T04:36:14.567 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 7.4 M 2024-05-09T04:36:14.567 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 49 k 2024-05-09T04:36:14.567 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 10 M 2024-05-09T04:36:14.567 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 43 M 2024-05-09T04:36:14.567 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 273 k 2024-05-09T04:36:14.568 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 763 k 2024-05-09T04:36:14.568 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 224 k 2024-05-09T04:36:14.568 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-shell x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 27 k 2024-05-09T04:36:14.568 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 18 k 2024-05-09T04:36:14.568 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-09T04:36:14.568 INFO:teuthology.orchestra.run.smithi072.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-09T04:36:14.568 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-09T04:36:14.568 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 32 k 2024-05-09T04:36:14.568 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 756 k 2024-05-09T04:36:14.568 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 127 k 2024-05-09T04:36:14.568 INFO:teuthology.orchestra.run.smithi072.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-09T04:36:14.568 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-09T04:36:14.568 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-09T04:36:14.568 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-09T04:36:14.569 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-09T04:36:14.569 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-09T04:36:14.569 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 162 k 2024-05-09T04:36:14.569 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 321 k 2024-05-09T04:36:14.569 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 301 k 2024-05-09T04:36:14.569 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 99 k 2024-05-09T04:36:14.569 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 86 k 2024-05-09T04:36:14.569 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.0 M 2024-05-09T04:36:14.569 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 169 k 2024-05-09T04:36:14.569 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-09T04:36:14.569 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrading: 2024-05-09T04:36:14.569 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-09T04:36:14.569 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-09T04:36:14.569 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-09T04:36:14.570 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-09T04:36:14.570 INFO:teuthology.orchestra.run.smithi072.stdout: librados2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.3 M 2024-05-09T04:36:14.570 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 3.1 M 2024-05-09T04:36:14.570 INFO:teuthology.orchestra.run.smithi072.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-09T04:36:14.570 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-09T04:36:14.570 INFO:teuthology.orchestra.run.smithi072.stdout:Installing dependencies: 2024-05-09T04:36:14.570 INFO:teuthology.orchestra.run.smithi072.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-09T04:36:14.570 INFO:teuthology.orchestra.run.smithi072.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-09T04:36:14.570 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-09T04:36:14.570 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 20 M 2024-05-09T04:36:14.570 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 34 k 2024-05-09T04:36:14.570 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 2.3 M 2024-05-09T04:36:14.570 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 274 k 2024-05-09T04:36:14.570 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 4.9 M 2024-05-09T04:36:14.571 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 18 M 2024-05-09T04:36:14.571 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3594.gc9da4049.el9 ceph-noarch 16 k 2024-05-09T04:36:14.571 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 25 k 2024-05-09T04:36:14.571 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-09T04:36:14.571 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-09T04:36:14.571 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-09T04:36:14.571 INFO:teuthology.orchestra.run.smithi072.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-09T04:36:14.571 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-09T04:36:14.571 INFO:teuthology.orchestra.run.smithi072.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-09T04:36:14.571 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-09T04:36:14.571 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-09T04:36:14.571 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 158 k 2024-05-09T04:36:14.571 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-09T04:36:14.571 INFO:teuthology.orchestra.run.smithi072.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-09T04:36:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-09T04:36:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-09T04:36:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-09T04:36:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 497 k 2024-05-09T04:36:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-09T04:36:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2 x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 5.2 M 2024-05-09T04:36:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-09T04:36:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-09T04:36:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-09T04:36:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-09T04:36:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-09T04:36:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-09T04:36:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-09T04:36:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-09T04:36:14.572 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-09T04:36:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-09T04:36:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-09T04:36:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-09T04:36:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-09T04:36:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-09T04:36:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-09T04:36:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-09T04:36:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-09T04:36:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 45 k 2024-05-09T04:36:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common x86_64 2:19.0.0-3594.gc9da4049.el9 ceph 135 k 2024-05-09T04:36:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-09T04:36:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-09T04:36:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-09T04:36:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-09T04:36:14.573 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-09T04:36:14.574 INFO:teuthology.orchestra.run.smithi072.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-09T04:36:14.574 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-09T04:36:14.574 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-09T04:36:14.574 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-09T04:36:14.574 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-09T04:36:14.574 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-09T04:36:14.574 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-09T04:36:14.574 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-09T04:36:14.574 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-09T04:36:14.574 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-09T04:36:14.574 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-09T04:36:14.574 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-09T04:36:14.574 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-09T04:36:14.574 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-09T04:36:14.575 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-09T04:36:14.575 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-09T04:36:14.575 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-09T04:36:14.575 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-09T04:36:14.575 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-09T04:36:14.575 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-09T04:36:14.575 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-09T04:36:14.575 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-09T04:36:14.575 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-09T04:36:14.575 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-09T04:36:14.575 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-09T04:36:14.575 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-09T04:36:14.575 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-09T04:36:14.576 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-09T04:36:14.576 INFO:teuthology.orchestra.run.smithi072.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-09T04:36:14.576 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-09T04:36:14.576 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-09T04:36:14.576 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-09T04:36:14.576 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-09T04:36:14.576 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-09T04:36:14.576 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-09T04:36:14.576 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-09T04:36:14.576 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-09T04:36:14.576 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-09T04:36:14.576 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-09T04:36:14.576 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-09T04:36:14.576 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-09T04:36:14.576 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-09T04:36:14.577 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-09T04:36:14.577 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-09T04:36:14.577 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-09T04:36:14.577 INFO:teuthology.orchestra.run.smithi072.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-09T04:36:14.577 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-09T04:36:14.577 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-09T04:36:14.577 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-09T04:36:14.577 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-09T04:36:14.577 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-09T04:36:14.577 INFO:teuthology.orchestra.run.smithi072.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-09T04:36:14.577 INFO:teuthology.orchestra.run.smithi072.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-09T04:36:14.577 INFO:teuthology.orchestra.run.smithi072.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-09T04:36:14.577 INFO:teuthology.orchestra.run.smithi072.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-09T04:36:14.577 INFO:teuthology.orchestra.run.smithi072.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-09T04:36:14.577 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-09T04:36:14.578 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-09T04:36:14.578 INFO:teuthology.orchestra.run.smithi072.stdout:Installing weak dependencies: 2024-05-09T04:36:14.578 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-09T04:36:14.578 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-09T04:36:14.578 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction Summary 2024-05-09T04:36:14.578 INFO:teuthology.orchestra.run.smithi072.stdout:======================================================================================= 2024-05-09T04:36:14.578 INFO:teuthology.orchestra.run.smithi072.stdout:Install 148 Packages 2024-05-09T04:36:14.578 INFO:teuthology.orchestra.run.smithi072.stdout:Upgrade 8 Packages 2024-05-09T04:36:14.578 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-09T04:36:14.579 INFO:teuthology.orchestra.run.smithi072.stdout:Total download size: 219 M 2024-05-09T04:36:14.579 INFO:teuthology.orchestra.run.smithi072.stdout:Downloading Packages: 2024-05-09T04:36:15.402 INFO:teuthology.orchestra.run.smithi110.stdout:(5/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 1.7 MB/s | 1.5 MB 00:00 2024-05-09T04:36:15.469 INFO:teuthology.orchestra.run.smithi110.stdout:(6/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 1.0 MB/s | 5.4 MB 00:05 2024-05-09T04:36:15.528 INFO:teuthology.orchestra.run.smithi072.stdout:(1/156): ceph-19.0.0-3594.gc9da4049.el9.x86_64. 27 kB/s | 6.6 kB 00:00 2024-05-09T04:36:15.984 INFO:teuthology.orchestra.run.smithi046.stdout:(7/156): ceph-base-19.0.0-3594.gc9da4049.el9.x8 868 kB/s | 5.4 MB 00:06 2024-05-09T04:36:17.669 INFO:teuthology.orchestra.run.smithi110.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 2.2 MB/s | 4.9 MB 00:02 2024-05-09T04:36:17.754 INFO:teuthology.orchestra.run.smithi072.stdout:(2/156): ceph-fuse-19.0.0-3594.gc9da4049.el9.x8 388 kB/s | 863 kB 00:02 2024-05-09T04:36:18.029 INFO:teuthology.orchestra.run.smithi072.stdout:(3/156): ceph-immutable-object-cache-19.0.0-359 533 kB/s | 147 kB 00:00 2024-05-09T04:36:18.858 INFO:teuthology.orchestra.run.smithi046.stdout:(8/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x86 4.1 MB/s | 18 MB 00:04 2024-05-09T04:36:18.886 INFO:teuthology.orchestra.run.smithi110.stdout:(8/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x86 5.2 MB/s | 18 MB 00:03 2024-05-09T04:36:18.934 INFO:teuthology.orchestra.run.smithi046.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 336 kB/s | 25 kB 00:00 2024-05-09T04:36:19.069 INFO:teuthology.orchestra.run.smithi110.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 137 kB/s | 25 kB 00:00 2024-05-09T04:36:20.238 INFO:teuthology.orchestra.run.smithi072.stdout:(4/156): ceph-common-19.0.0-3594.gc9da4049.el9. 4.0 MB/s | 20 MB 00:04 2024-05-09T04:36:20.313 INFO:teuthology.orchestra.run.smithi072.stdout:(5/156): ceph-mds-19.0.0-3594.gc9da4049.el9.x86 1.0 MB/s | 2.3 MB 00:02 2024-05-09T04:36:20.522 INFO:teuthology.orchestra.run.smithi072.stdout:(6/156): ceph-mgr-19.0.0-3594.gc9da4049.el9.x86 5.3 MB/s | 1.5 MB 00:00 2024-05-09T04:36:21.811 INFO:teuthology.orchestra.run.smithi110.stdout:(10/156): ceph-radosgw-19.0.0-3594.gc9da4049.el 2.4 MB/s | 10 MB 00:04 2024-05-09T04:36:21.859 INFO:teuthology.orchestra.run.smithi046.stdout:(10/156): ceph-radosgw-19.0.0-3594.gc9da4049.el 1.7 MB/s | 10 MB 00:05 2024-05-09T04:36:22.003 INFO:teuthology.orchestra.run.smithi110.stdout:(11/156): ceph-common-19.0.0-3594.gc9da4049.el9 1.7 MB/s | 20 MB 00:11 2024-05-09T04:36:22.009 INFO:teuthology.orchestra.run.smithi046.stdout:(11/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 1.5 MB/s | 224 kB 00:00 2024-05-09T04:36:22.028 INFO:teuthology.orchestra.run.smithi110.stdout:(12/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 1.0 MB/s | 224 kB 00:00 2024-05-09T04:36:22.086 INFO:teuthology.orchestra.run.smithi110.stdout:(13/156): cephfs-shell-19.0.0-3594.gc9da4049.el 325 kB/s | 27 kB 00:00 2024-05-09T04:36:22.093 INFO:teuthology.orchestra.run.smithi046.stdout:(12/156): cephfs-shell-19.0.0-3594.gc9da4049.el 324 kB/s | 27 kB 00:00 2024-05-09T04:36:22.112 INFO:teuthology.orchestra.run.smithi110.stdout:(14/156): libcephfs-devel-19.0.0-3594.gc9da4049 379 kB/s | 32 kB 00:00 2024-05-09T04:36:22.176 INFO:teuthology.orchestra.run.smithi046.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 381 kB/s | 32 kB 00:00 2024-05-09T04:36:22.204 INFO:teuthology.orchestra.run.smithi110.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 1.7 MB/s | 158 kB 00:00 2024-05-09T04:36:22.304 INFO:teuthology.orchestra.run.smithi110.stdout:(16/156): librados-devel-19.0.0-3594.gc9da4049. 1.2 MB/s | 127 kB 00:00 2024-05-09T04:36:22.396 INFO:teuthology.orchestra.run.smithi110.stdout:(17/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 2.4 MB/s | 756 kB 00:00 2024-05-09T04:36:22.563 INFO:teuthology.orchestra.run.smithi110.stdout:(18/156): libradosstriper1-19.0.0-3594.gc9da404 1.9 MB/s | 497 kB 00:00 2024-05-09T04:36:22.640 INFO:teuthology.orchestra.run.smithi072.stdout:(7/156): ceph-mon-19.0.0-3594.gc9da4049.el9.x86 2.1 MB/s | 4.9 MB 00:02 2024-05-09T04:36:22.660 INFO:teuthology.orchestra.run.smithi046.stdout:(14/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 1.5 MB/s | 756 kB 00:00 2024-05-09T04:36:22.680 INFO:teuthology.orchestra.run.smithi110.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 387 kB/s | 45 kB 00:00 2024-05-09T04:36:22.814 INFO:teuthology.orchestra.run.smithi110.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 1.0 MB/s | 135 kB 00:00 2024-05-09T04:36:22.818 INFO:teuthology.orchestra.run.smithi046.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 998 kB/s | 158 kB 00:00 2024-05-09T04:36:22.914 INFO:teuthology.orchestra.run.smithi110.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 1.6 MB/s | 162 kB 00:00 2024-05-09T04:36:22.986 INFO:teuthology.orchestra.run.smithi046.stdout:(16/156): librados-devel-19.0.0-3594.gc9da4049. 760 kB/s | 127 kB 00:00 2024-05-09T04:36:23.098 INFO:teuthology.orchestra.run.smithi110.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 1.7 MB/s | 321 kB 00:00 2024-05-09T04:36:23.232 INFO:teuthology.orchestra.run.smithi110.stdout:(23/156): python3-rbd-19.0.0-3594.gc9da4049.el9 2.2 MB/s | 301 kB 00:00 2024-05-09T04:36:23.286 INFO:teuthology.orchestra.run.smithi046.stdout:(17/156): libradosstriper1-19.0.0-3594.gc9da404 1.6 MB/s | 497 kB 00:00 2024-05-09T04:36:23.290 INFO:teuthology.orchestra.run.smithi110.stdout:(24/156): python3-rgw-19.0.0-3594.gc9da4049.el9 1.7 MB/s | 99 kB 00:00 2024-05-09T04:36:23.365 INFO:teuthology.orchestra.run.smithi110.stdout:(25/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 1.1 MB/s | 86 kB 00:00 2024-05-09T04:36:23.390 INFO:teuthology.orchestra.run.smithi072.stdout:(8/156): ceph-osd-19.0.0-3594.gc9da4049.el9.x86 6.2 MB/s | 18 MB 00:02 2024-05-09T04:36:23.465 INFO:teuthology.orchestra.run.smithi072.stdout:(9/156): ceph-selinux-19.0.0-3594.gc9da4049.el9 336 kB/s | 25 kB 00:00 2024-05-09T04:36:23.516 INFO:teuthology.orchestra.run.smithi110.stdout:(26/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 4.6 MB/s | 5.2 MB 00:01 2024-05-09T04:36:23.599 INFO:teuthology.orchestra.run.smithi110.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 2.0 MB/s | 169 kB 00:00 2024-05-09T04:36:23.691 INFO:teuthology.orchestra.run.smithi110.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 372 kB/s | 34 kB 00:00 2024-05-09T04:36:23.792 INFO:teuthology.orchestra.run.smithi110.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 1.4 MB/s | 142 kB 00:00 2024-05-09T04:36:24.667 INFO:teuthology.orchestra.run.smithi110.stdout:(30/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 4.0 MB/s | 3.5 MB 00:00 2024-05-09T04:36:24.776 INFO:teuthology.orchestra.run.smithi110.stdout:(31/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 2.2 MB/s | 3.0 MB 00:01 2024-05-09T04:36:25.293 INFO:teuthology.orchestra.run.smithi110.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 531 kB/s | 274 kB 00:00 2024-05-09T04:36:25.393 INFO:teuthology.orchestra.run.smithi110.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 495 kB/s | 49 kB 00:00 2024-05-09T04:36:25.501 INFO:teuthology.orchestra.run.smithi110.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3594.gc 152 kB/s | 16 kB 00:00 2024-05-09T04:36:25.777 INFO:teuthology.orchestra.run.smithi110.stdout:(35/156): ceph-volume-19.0.0-3594.gc9da4049.el9 993 kB/s | 273 kB 00:00 2024-05-09T04:36:26.294 INFO:teuthology.orchestra.run.smithi110.stdout:(36/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 1.4 MB/s | 763 kB 00:00 2024-05-09T04:36:26.294 INFO:teuthology.orchestra.run.smithi046.stdout:(18/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 1.7 MB/s | 5.2 MB 00:03 2024-05-09T04:36:26.378 INFO:teuthology.orchestra.run.smithi110.stdout:(37/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 219 kB/s | 18 kB 00:00 2024-05-09T04:36:26.386 INFO:teuthology.orchestra.run.smithi046.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 495 kB/s | 45 kB 00:00 2024-05-09T04:36:26.478 INFO:teuthology.orchestra.run.smithi046.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 1.4 MB/s | 135 kB 00:00 2024-05-09T04:36:26.721 INFO:teuthology.orchestra.run.smithi046.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 669 kB/s | 162 kB 00:00 2024-05-09T04:36:26.736 INFO:teuthology.orchestra.run.smithi110.stdout:(38/156): ceph-mgr-diskprediction-local-19.0.0- 3.6 MB/s | 7.4 MB 00:02 2024-05-09T04:36:26.921 INFO:teuthology.orchestra.run.smithi046.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 1.6 MB/s | 321 kB 00:00 2024-05-09T04:36:27.007 INFO:teuthology.orchestra.run.smithi072.stdout:(10/156): ceph-base-19.0.0-3594.gc9da4049.el9.x 471 kB/s | 5.4 MB 00:11 2024-05-09T04:36:27.088 INFO:teuthology.orchestra.run.smithi046.stdout:(23/156): python3-rbd-19.0.0-3594.gc9da4049.el9 1.8 MB/s | 301 kB 00:00 2024-05-09T04:36:27.188 INFO:teuthology.orchestra.run.smithi046.stdout:(24/156): python3-rgw-19.0.0-3594.gc9da4049.el9 993 kB/s | 99 kB 00:00 2024-05-09T04:36:27.228 INFO:teuthology.orchestra.run.smithi110.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 94 kB/s | 80 kB 00:00 2024-05-09T04:36:27.280 INFO:teuthology.orchestra.run.smithi046.stdout:(25/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 938 kB/s | 86 kB 00:00 2024-05-09T04:36:27.366 INFO:teuthology.orchestra.run.smithi072.stdout:(11/156): ceph-radosgw-19.0.0-3594.gc9da4049.el 2.1 MB/s | 10 MB 00:04 2024-05-09T04:36:27.433 INFO:teuthology.orchestra.run.smithi072.stdout:(12/156): cephfs-mirror-19.0.0-3594.gc9da4049.e 526 kB/s | 224 kB 00:00 2024-05-09T04:36:27.576 INFO:teuthology.orchestra.run.smithi072.stdout:(13/156): libcephfs-devel-19.0.0-3594.gc9da4049 223 kB/s | 32 kB 00:00 2024-05-09T04:36:27.795 INFO:teuthology.orchestra.run.smithi110.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 779 kB/s | 803 kB 00:01 2024-05-09T04:36:27.894 INFO:teuthology.orchestra.run.smithi072.stdout:(14/156): cephfs-shell-19.0.0-3594.gc9da4049.el 51 kB/s | 27 kB 00:00 2024-05-09T04:36:28.095 INFO:teuthology.orchestra.run.smithi072.stdout:(15/156): libcephsqlite-19.0.0-3594.gc9da4049.e 787 kB/s | 158 kB 00:00 2024-05-09T04:36:28.120 INFO:teuthology.orchestra.run.smithi110.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 219 kB/s | 195 kB 00:00 2024-05-09T04:36:28.221 INFO:teuthology.orchestra.run.smithi072.stdout:(16/156): librados-devel-19.0.0-3594.gc9da4049. 1.0 MB/s | 127 kB 00:00 2024-05-09T04:36:28.638 INFO:teuthology.orchestra.run.smithi072.stdout:(17/156): libradosstriper1-19.0.0-3594.gc9da404 1.2 MB/s | 497 kB 00:00 2024-05-09T04:36:28.679 INFO:teuthology.orchestra.run.smithi110.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 287 kB/s | 253 kB 00:00 2024-05-09T04:36:28.788 INFO:teuthology.orchestra.run.smithi072.stdout:(18/156): libcephfs2-19.0.0-3594.gc9da4049.el9. 624 kB/s | 756 kB 00:01 2024-05-09T04:36:28.863 INFO:teuthology.orchestra.run.smithi046.stdout:(26/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 1.9 MB/s | 3.0 MB 00:01 2024-05-09T04:36:28.897 INFO:teuthology.orchestra.run.smithi072.stdout:(19/156): python3-ceph-argparse-19.0.0-3594.gc9 416 kB/s | 45 kB 00:00 2024-05-09T04:36:29.038 INFO:teuthology.orchestra.run.smithi046.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 967 kB/s | 169 kB 00:00 2024-05-09T04:36:29.196 INFO:teuthology.orchestra.run.smithi110.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 1.1 MB/s | 1.2 MB 00:01 2024-05-09T04:36:29.256 INFO:teuthology.orchestra.run.smithi072.stdout:(20/156): python3-ceph-common-19.0.0-3594.gc9da 378 kB/s | 135 kB 00:00 2024-05-09T04:36:29.414 INFO:teuthology.orchestra.run.smithi046.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3594.g 91 kB/s | 34 kB 00:00 2024-05-09T04:36:29.504 INFO:teuthology.orchestra.run.smithi110.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 129 kB/s | 106 kB 00:00 2024-05-09T04:36:29.589 INFO:teuthology.orchestra.run.smithi046.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 813 kB/s | 142 kB 00:00 2024-05-09T04:36:30.013 INFO:teuthology.orchestra.run.smithi110.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 166 kB/s | 135 kB 00:00 2024-05-09T04:36:30.040 INFO:teuthology.orchestra.run.smithi072.stdout:(21/156): python3-cephfs-19.0.0-3594.gc9da4049. 207 kB/s | 162 kB 00:00 2024-05-09T04:36:30.338 INFO:teuthology.orchestra.run.smithi110.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 150 kB/s | 125 kB 00:00 2024-05-09T04:36:30.439 INFO:teuthology.orchestra.run.smithi110.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 6.8 MB/s | 689 kB 00:00 2024-05-09T04:36:30.547 INFO:teuthology.orchestra.run.smithi110.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 8.5 MB/s | 943 kB 00:00 2024-05-09T04:36:30.614 INFO:teuthology.orchestra.run.smithi110.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-05-09T04:36:30.664 INFO:teuthology.orchestra.run.smithi110.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-05-09T04:36:30.740 INFO:teuthology.orchestra.run.smithi110.stdout:(51/156): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-05-09T04:36:30.790 INFO:teuthology.orchestra.run.smithi110.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-05-09T04:36:30.940 INFO:teuthology.orchestra.run.smithi110.stdout:(53/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2024-05-09T04:36:30.982 INFO:teuthology.orchestra.run.smithi110.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 439 kB/s | 18 kB 00:00 2024-05-09T04:36:31.007 INFO:teuthology.orchestra.run.smithi110.stdout:(55/156): python3-urllib3-1.26.5-5.el9.noarch.r 216 kB/s | 215 kB 00:00 2024-05-09T04:36:31.033 INFO:teuthology.orchestra.run.smithi110.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 228 kB/s | 11 kB 00:00 2024-05-09T04:36:31.083 INFO:teuthology.orchestra.run.smithi110.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-05-09T04:36:31.100 INFO:teuthology.orchestra.run.smithi110.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 673 kB/s | 45 kB 00:00 2024-05-09T04:36:31.175 INFO:teuthology.orchestra.run.smithi110.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 662 kB 00:00 2024-05-09T04:36:31.191 INFO:teuthology.orchestra.run.smithi072.stdout:(22/156): python3-rados-19.0.0-3594.gc9da4049.e 279 kB/s | 321 kB 00:01 2024-05-09T04:36:31.209 INFO:teuthology.orchestra.run.smithi110.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-05-09T04:36:31.243 INFO:teuthology.orchestra.run.smithi110.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 381 kB/s | 25 kB 00:00 2024-05-09T04:36:31.260 INFO:teuthology.orchestra.run.smithi110.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 948 kB/s | 47 kB 00:00 2024-05-09T04:36:31.327 INFO:teuthology.orchestra.run.smithi110.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-05-09T04:36:31.347 INFO:teuthology.orchestra.run.smithi046.stdout:(30/156): ceph-common-19.0.0-3594.gc9da4049.el9 922 kB/s | 20 MB 00:21 2024-05-09T04:36:31.352 INFO:teuthology.orchestra.run.smithi110.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-09T04:36:31.385 INFO:teuthology.orchestra.run.smithi110.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 603 kB/s | 35 kB 00:00 2024-05-09T04:36:31.561 INFO:teuthology.orchestra.run.smithi110.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-05-09T04:36:31.620 INFO:teuthology.orchestra.run.smithi110.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-05-09T04:36:31.720 INFO:teuthology.orchestra.run.smithi110.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 8.9 MB/s | 911 kB 00:00 2024-05-09T04:36:31.753 INFO:teuthology.orchestra.run.smithi110.stdout:(69/156): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-05-09T04:36:31.812 INFO:teuthology.orchestra.run.smithi110.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-09T04:36:31.921 INFO:teuthology.orchestra.run.smithi110.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-05-09T04:36:32.029 INFO:teuthology.orchestra.run.smithi110.stdout:(72/156): postgresql-server-13.14-1.el9.x86_64. 21 MB/s | 5.8 MB 00:00 2024-05-09T04:36:32.055 INFO:teuthology.orchestra.run.smithi110.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 654 kB/s | 87 kB 00:00 2024-05-09T04:36:32.166 INFO:teuthology.orchestra.run.smithi072.stdout:(23/156): python3-rbd-19.0.0-3594.gc9da4049.el9 308 kB/s | 301 kB 00:00 2024-05-09T04:36:32.280 INFO:teuthology.orchestra.run.smithi110.stdout:(74/156): python3-babel-2.9.1-2.el9.noarch.rpm 24 MB/s | 6.0 MB 00:00 2024-05-09T04:36:32.281 INFO:teuthology.orchestra.run.smithi046.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 1.3 MB/s | 3.5 MB 00:02 2024-05-09T04:36:32.355 INFO:teuthology.orchestra.run.smithi110.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-05-09T04:36:32.405 INFO:teuthology.orchestra.run.smithi110.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 511 kB/s | 26 kB 00:00 2024-05-09T04:36:32.431 INFO:teuthology.orchestra.run.smithi110.stdout:(77/156): python3-devel-3.9.18-3.el9.x86_64.rpm 650 kB/s | 244 kB 00:00 2024-05-09T04:36:32.517 INFO:teuthology.orchestra.run.smithi072.stdout:(24/156): python3-rgw-19.0.0-3594.gc9da4049.el9 283 kB/s | 99 kB 00:00 2024-05-09T04:36:32.523 INFO:teuthology.orchestra.run.smithi110.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 163 kB/s | 19 kB 00:00 2024-05-09T04:36:32.548 INFO:teuthology.orchestra.run.smithi046.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 1.0 MB/s | 274 kB 00:00 2024-05-09T04:36:32.573 INFO:teuthology.orchestra.run.smithi110.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 1.2 MB/s | 172 kB 00:00 2024-05-09T04:36:32.598 INFO:teuthology.orchestra.run.smithi110.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 462 kB/s | 35 kB 00:00 2024-05-09T04:36:32.648 INFO:teuthology.orchestra.run.smithi046.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 493 kB/s | 49 kB 00:00 2024-05-09T04:36:32.674 INFO:teuthology.orchestra.run.smithi110.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-05-09T04:36:32.740 INFO:teuthology.orchestra.run.smithi046.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3594.gc 179 kB/s | 16 kB 00:00 2024-05-09T04:36:32.749 INFO:teuthology.orchestra.run.smithi110.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-05-09T04:36:32.824 INFO:teuthology.orchestra.run.smithi110.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-05-09T04:36:32.842 INFO:teuthology.orchestra.run.smithi072.stdout:(25/156): rbd-fuse-19.0.0-3594.gc9da4049.el9.x8 263 kB/s | 86 kB 00:00 2024-05-09T04:36:32.849 INFO:teuthology.orchestra.run.smithi110.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 770 kB/s | 77 kB 00:00 2024-05-09T04:36:32.900 INFO:teuthology.orchestra.run.smithi110.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 559 kB/s | 42 kB 00:00 2024-05-09T04:36:32.950 INFO:teuthology.orchestra.run.smithi110.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-05-09T04:36:32.992 INFO:teuthology.orchestra.run.smithi110.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-05-09T04:36:33.043 INFO:teuthology.orchestra.run.smithi110.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.0 MB/s | 279 kB 00:00 2024-05-09T04:36:33.059 INFO:teuthology.orchestra.run.smithi110.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 761 kB/s | 51 kB 00:00 2024-05-09T04:36:33.065 INFO:teuthology.orchestra.run.smithi046.stdout:(35/156): ceph-volume-19.0.0-3594.gc9da4049.el9 841 kB/s | 273 kB 00:00 2024-05-09T04:36:33.101 INFO:teuthology.orchestra.run.smithi110.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 914 kB/s | 54 kB 00:00 2024-05-09T04:36:33.152 INFO:teuthology.orchestra.run.smithi110.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 833 kB/s | 42 kB 00:00 2024-05-09T04:36:33.202 INFO:teuthology.orchestra.run.smithi110.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 879 kB/s | 44 kB 00:00 2024-05-09T04:36:33.277 INFO:teuthology.orchestra.run.smithi110.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-05-09T04:36:33.327 INFO:teuthology.orchestra.run.smithi110.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-05-09T04:36:33.378 INFO:teuthology.orchestra.run.smithi110.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 944 kB/s | 47 kB 00:00 2024-05-09T04:36:33.470 INFO:teuthology.orchestra.run.smithi110.stdout:(96/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.3 MB/s | 305 kB 00:00 2024-05-09T04:36:33.645 INFO:teuthology.orchestra.run.smithi110.stdout:(97/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-05-09T04:36:33.662 INFO:teuthology.orchestra.run.smithi110.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 301 kB/s | 58 kB 00:00 2024-05-09T04:36:33.712 INFO:teuthology.orchestra.run.smithi110.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.4 MB/s | 94 kB 00:00 2024-05-09T04:36:33.738 INFO:teuthology.orchestra.run.smithi110.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 845 kB/s | 64 kB 00:00 2024-05-09T04:36:33.757 INFO:teuthology.orchestra.run.smithi046.stdout:(36/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 1.1 MB/s | 763 kB 00:00 2024-05-09T04:36:33.780 INFO:teuthology.orchestra.run.smithi110.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 670 kB/s | 45 kB 00:00 2024-05-09T04:36:33.849 INFO:teuthology.orchestra.run.smithi046.stdout:(37/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 199 kB/s | 18 kB 00:00 2024-05-09T04:36:33.908 INFO:teuthology.orchestra.run.smithi046.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 1.3 MB/s | 80 kB 00:00 2024-05-09T04:36:33.960 INFO:teuthology.orchestra.run.smithi072.stdout:(26/156): librgw2-19.0.0-3594.gc9da4049.el9.x86 993 kB/s | 5.2 MB 00:05 2024-05-09T04:36:33.997 INFO:teuthology.orchestra.run.smithi110.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 86 kB/s | 22 kB 00:00 2024-05-09T04:36:34.008 INFO:teuthology.orchestra.run.smithi046.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 7.8 MB/s | 803 kB 00:00 2024-05-09T04:36:34.083 INFO:teuthology.orchestra.run.smithi046.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 195 kB 00:00 2024-05-09T04:36:34.159 INFO:teuthology.orchestra.run.smithi046.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.3 MB/s | 253 kB 00:00 2024-05-09T04:36:34.259 INFO:teuthology.orchestra.run.smithi046.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 12 MB/s | 1.2 MB 00:00 2024-05-09T04:36:34.326 INFO:teuthology.orchestra.run.smithi046.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.6 MB/s | 106 kB 00:00 2024-05-09T04:36:34.393 INFO:teuthology.orchestra.run.smithi046.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-05-09T04:36:34.420 INFO:teuthology.orchestra.run.smithi072.stdout:(27/156): rbd-nbd-19.0.0-3594.gc9da4049.el9.x86 367 kB/s | 169 kB 00:00 2024-05-09T04:36:34.460 INFO:teuthology.orchestra.run.smithi046.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 1.8 MB/s | 125 kB 00:00 2024-05-09T04:36:34.535 INFO:teuthology.orchestra.run.smithi046.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-05-09T04:36:34.589 INFO:teuthology.orchestra.run.smithi110.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 1.0 MB/s | 863 kB 00:00 2024-05-09T04:36:34.619 INFO:teuthology.orchestra.run.smithi046.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 8.1 MB/s | 689 kB 00:00 2024-05-09T04:36:34.727 INFO:teuthology.orchestra.run.smithi046.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 8.5 MB/s | 943 kB 00:00 2024-05-09T04:36:35.129 INFO:teuthology.orchestra.run.smithi046.stdout:(49/156): ceph-test-19.0.0-3594.gc9da4049.el9.x 2.7 MB/s | 43 MB 00:16 2024-05-09T04:36:35.178 INFO:teuthology.orchestra.run.smithi046.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 433 kB/s | 21 kB 00:00 2024-05-09T04:36:35.203 INFO:teuthology.orchestra.run.smithi046.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 225 kB/s | 107 kB 00:00 2024-05-09T04:36:35.254 INFO:teuthology.orchestra.run.smithi046.stdout:(52/156): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-05-09T04:36:35.287 INFO:teuthology.orchestra.run.smithi046.stdout:(53/156): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-05-09T04:36:35.338 INFO:teuthology.orchestra.run.smithi046.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-05-09T04:36:35.346 INFO:teuthology.orchestra.run.smithi072.stdout:(28/156): ceph-test-19.0.0-3594.gc9da4049.el9.x 3.6 MB/s | 43 MB 00:11 2024-05-09T04:36:35.373 INFO:teuthology.orchestra.run.smithi072.stdout:(29/156): ceph-grafana-dashboards-19.0.0-3594.g 36 kB/s | 34 kB 00:00 2024-05-09T04:36:35.380 INFO:teuthology.orchestra.run.smithi046.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 269 kB/s | 11 kB 00:00 2024-05-09T04:36:35.438 INFO:teuthology.orchestra.run.smithi046.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 16 MB/s | 3.0 MB 00:00 2024-05-09T04:36:35.480 INFO:teuthology.orchestra.run.smithi046.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-05-09T04:36:35.522 INFO:teuthology.orchestra.run.smithi046.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 542 kB/s | 45 kB 00:00 2024-05-09T04:36:35.547 INFO:teuthology.orchestra.run.smithi072.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3594.gc9da404 710 kB/s | 142 kB 00:00 2024-05-09T04:36:35.606 INFO:teuthology.orchestra.run.smithi046.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 5.2 MB/s | 662 kB 00:00 2024-05-09T04:36:35.631 INFO:teuthology.orchestra.run.smithi046.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 582 kB 00:00 2024-05-09T04:36:35.673 INFO:teuthology.orchestra.run.smithi046.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 380 kB/s | 25 kB 00:00 2024-05-09T04:36:35.698 INFO:teuthology.orchestra.run.smithi046.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 708 kB/s | 47 kB 00:00 2024-05-09T04:36:35.765 INFO:teuthology.orchestra.run.smithi046.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-05-09T04:36:35.790 INFO:teuthology.orchestra.run.smithi046.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-05-09T04:36:35.832 INFO:teuthology.orchestra.run.smithi046.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 526 kB/s | 35 kB 00:00 2024-05-09T04:36:35.889 INFO:teuthology.orchestra.run.smithi110.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 238 kB/s | 309 kB 00:01 2024-05-09T04:36:35.947 INFO:teuthology.orchestra.run.smithi110.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 281 kB/s | 548 kB 00:01 2024-05-09T04:36:35.999 INFO:teuthology.orchestra.run.smithi046.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 23 MB/s | 4.7 MB 00:00 2024-05-09T04:36:36.014 INFO:teuthology.orchestra.run.smithi110.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 155 kB/s | 19 kB 00:00 2024-05-09T04:36:36.074 INFO:teuthology.orchestra.run.smithi046.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-05-09T04:36:36.157 INFO:teuthology.orchestra.run.smithi110.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 183 kB/s | 26 kB 00:00 2024-05-09T04:36:36.166 INFO:teuthology.orchestra.run.smithi046.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2024-05-09T04:36:36.208 INFO:teuthology.orchestra.run.smithi046.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 6.7 MB/s | 911 kB 00:00 2024-05-09T04:36:36.267 INFO:teuthology.orchestra.run.smithi046.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.3 MB/s | 133 kB 00:00 2024-05-09T04:36:36.375 INFO:teuthology.orchestra.run.smithi046.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-05-09T04:36:36.407 INFO:teuthology.orchestra.run.smithi110.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 194 kB/s | 49 kB 00:00 2024-05-09T04:36:36.435 INFO:teuthology.orchestra.run.smithi046.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-05-09T04:36:36.527 INFO:teuthology.orchestra.run.smithi046.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-05-09T04:36:36.601 INFO:teuthology.orchestra.run.smithi046.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-05-09T04:36:36.685 INFO:teuthology.orchestra.run.smithi046.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.9 MB/s | 248 kB 00:00 2024-05-09T04:36:36.777 INFO:teuthology.orchestra.run.smithi046.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-05-09T04:36:36.802 INFO:teuthology.orchestra.run.smithi046.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 219 kB/s | 26 kB 00:00 2024-05-09T04:36:36.911 INFO:teuthology.orchestra.run.smithi046.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 143 kB/s | 19 kB 00:00 2024-05-09T04:36:37.061 INFO:teuthology.orchestra.run.smithi046.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 665 kB/s | 172 kB 00:00 2024-05-09T04:36:37.078 INFO:teuthology.orchestra.run.smithi046.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 207 kB/s | 35 kB 00:00 2024-05-09T04:36:37.154 INFO:teuthology.orchestra.run.smithi046.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-05-09T04:36:37.229 INFO:teuthology.orchestra.run.smithi046.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.9 MB/s | 222 kB 00:00 2024-05-09T04:36:37.288 INFO:teuthology.orchestra.run.smithi046.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-09T04:36:37.397 INFO:teuthology.orchestra.run.smithi046.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 389 kB/s | 42 kB 00:00 2024-05-09T04:36:37.471 INFO:teuthology.orchestra.run.smithi046.stdout:(85/156): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-05-09T04:36:37.538 INFO:teuthology.orchestra.run.smithi046.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-09T04:36:37.614 INFO:teuthology.orchestra.run.smithi046.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-05-09T04:36:37.672 INFO:teuthology.orchestra.run.smithi046.stdout:(88/156): python3-pytz-2021.1-5.el9.noarch.rpm 876 kB/s | 51 kB 00:00 2024-05-09T04:36:37.731 INFO:teuthology.orchestra.run.smithi046.stdout:(89/156): python3-requests-oauthlib-1.3.0-12.el 919 kB/s | 54 kB 00:00 2024-05-09T04:36:37.849 INFO:teuthology.orchestra.run.smithi110.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 47 kB/s | 67 kB 00:01 2024-05-09T04:36:38.190 INFO:teuthology.orchestra.run.smithi046.stdout:(90/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-05-09T04:36:38.248 INFO:teuthology.orchestra.run.smithi046.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 717 kB/s | 42 kB 00:00 2024-05-09T04:36:38.307 INFO:teuthology.orchestra.run.smithi046.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 755 kB/s | 44 kB 00:00 2024-05-09T04:36:38.590 INFO:teuthology.orchestra.run.smithi046.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 780 kB/s | 221 kB 00:00 2024-05-09T04:36:38.641 INFO:teuthology.orchestra.run.smithi046.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-05-09T04:36:38.708 INFO:teuthology.orchestra.run.smithi046.stdout:(95/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.1 MB/s | 5.1 MB 00:01 2024-05-09T04:36:38.739 INFO:teuthology.orchestra.run.smithi072.stdout:(31/156): ceph-mgr-diskprediction-local-19.0.0- 2.3 MB/s | 7.4 MB 00:03 2024-05-09T04:36:38.758 INFO:teuthology.orchestra.run.smithi046.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 405 kB/s | 47 kB 00:00 2024-05-09T04:36:38.808 INFO:teuthology.orchestra.run.smithi046.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-09T04:36:38.832 INFO:teuthology.orchestra.run.smithi110.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 154 kB/s | 151 kB 00:00 2024-05-09T04:36:38.833 INFO:teuthology.orchestra.run.smithi046.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 771 kB/s | 58 kB 00:00 2024-05-09T04:36:38.881 INFO:teuthology.orchestra.run.smithi072.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3594.gc9 1.9 MB/s | 274 kB 00:00 2024-05-09T04:36:38.884 INFO:teuthology.orchestra.run.smithi046.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2024-05-09T04:36:38.909 INFO:teuthology.orchestra.run.smithi046.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 846 kB/s | 64 kB 00:00 2024-05-09T04:36:38.948 INFO:teuthology.orchestra.run.smithi072.stdout:(33/156): ceph-mgr-rook-19.0.0-3594.gc9da4049.e 737 kB/s | 49 kB 00:00 2024-05-09T04:36:38.951 INFO:teuthology.orchestra.run.smithi046.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 670 kB/s | 45 kB 00:00 2024-05-09T04:36:39.007 INFO:teuthology.orchestra.run.smithi072.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3594.gc 282 kB/s | 16 kB 00:00 2024-05-09T04:36:39.166 INFO:teuthology.orchestra.run.smithi072.stdout:(35/156): ceph-volume-19.0.0-3594.gc9da4049.el9 1.7 MB/s | 273 kB 00:00 2024-05-09T04:36:39.508 INFO:teuthology.orchestra.run.smithi072.stdout:(36/156): rbd-mirror-19.0.0-3594.gc9da4049.el9. 467 kB/s | 3.0 MB 00:06 2024-05-09T04:36:39.601 INFO:teuthology.orchestra.run.smithi072.stdout:(37/156): cephadm-19.0.0-3594.gc9da4049.el9.noa 1.7 MB/s | 763 kB 00:00 2024-05-09T04:36:39.618 INFO:teuthology.orchestra.run.smithi046.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-05-09T04:36:39.626 INFO:teuthology.orchestra.run.smithi072.stdout:(38/156): cephfs-top-19.0.0-3594.gc9da4049.el9. 155 kB/s | 18 kB 00:00 2024-05-09T04:36:40.102 INFO:teuthology.orchestra.run.smithi046.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 751 kB/s | 863 kB 00:01 2024-05-09T04:36:40.418 INFO:teuthology.orchestra.run.smithi072.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 98 kB/s | 80 kB 00:00 2024-05-09T04:36:40.468 INFO:teuthology.orchestra.run.smithi046.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 843 kB/s | 309 kB 00:00 2024-05-09T04:36:40.493 INFO:teuthology.orchestra.run.smithi046.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 626 kB/s | 548 kB 00:00 2024-05-09T04:36:40.535 INFO:teuthology.orchestra.run.smithi046.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 290 kB/s | 19 kB 00:00 2024-05-09T04:36:40.611 INFO:teuthology.orchestra.run.smithi046.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 345 kB/s | 26 kB 00:00 2024-05-09T04:36:40.703 INFO:teuthology.orchestra.run.smithi046.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 530 kB/s | 49 kB 00:00 2024-05-09T04:36:40.787 INFO:teuthology.orchestra.run.smithi046.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 807 kB/s | 67 kB 00:00 2024-05-09T04:36:40.820 INFO:teuthology.orchestra.run.smithi072.stdout:(40/156): libgfortran-11.4.1-3.el9.x86_64.rpm 673 kB/s | 803 kB 00:01 2024-05-09T04:36:40.879 INFO:teuthology.orchestra.run.smithi046.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 1.6 MB/s | 151 kB 00:00 2024-05-09T04:36:41.146 INFO:teuthology.orchestra.run.smithi046.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 3.1 MB/s | 839 kB 00:00 2024-05-09T04:36:41.288 INFO:teuthology.orchestra.run.smithi046.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 3.8 MB/s | 548 kB 00:00 2024-05-09T04:36:41.362 INFO:teuthology.orchestra.run.smithi046.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 581 kB/s | 43 kB 00:00 2024-05-09T04:36:41.363 INFO:teuthology.orchestra.run.smithi072.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 207 kB/s | 195 kB 00:00 2024-05-09T04:36:41.439 INFO:teuthology.orchestra.run.smithi046.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 429 kB/s | 32 kB 00:00 2024-05-09T04:36:41.514 INFO:teuthology.orchestra.run.smithi046.stdout:(115/156): python3-certifi-2021.10.8-2.el9.noar 200 kB/s | 15 kB 00:00 2024-05-09T04:36:41.605 INFO:teuthology.orchestra.run.smithi046.stdout:(116/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-05-09T04:36:41.689 INFO:teuthology.orchestra.run.smithi046.stdout:(117/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-05-09T04:36:41.773 INFO:teuthology.orchestra.run.smithi046.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.6 MB/s | 303 kB 00:00 2024-05-09T04:36:41.814 INFO:teuthology.orchestra.run.smithi072.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 255 kB/s | 253 kB 00:00 2024-05-09T04:36:41.848 INFO:teuthology.orchestra.run.smithi046.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 472 kB/s | 35 kB 00:00 2024-05-09T04:36:41.889 INFO:teuthology.orchestra.run.smithi072.stdout:(43/156): ceph-mgr-dashboard-19.0.0-3594.gc9da4 556 kB/s | 3.5 MB 00:06 2024-05-09T04:36:41.923 INFO:teuthology.orchestra.run.smithi046.stdout:(120/156): python3-google-auth-2.29.0-1.el9.noa 2.7 MB/s | 206 kB 00:00 2024-05-09T04:36:42.007 INFO:teuthology.orchestra.run.smithi110.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 264 kB/s | 839 kB 00:03 2024-05-09T04:36:42.257 INFO:teuthology.orchestra.run.smithi046.stdout:(121/156): python3-grpcio-1.46.7-10.el9.x86_64. 6.1 MB/s | 2.0 MB 00:00 2024-05-09T04:36:42.324 INFO:teuthology.orchestra.run.smithi046.stdout:(122/156): libarrow-9.0.0-11.el9.x86_64.rpm 2.4 MB/s | 4.4 MB 00:01 2024-05-09T04:36:42.349 INFO:teuthology.orchestra.run.smithi046.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 1.5 MB/s | 144 kB 00:00 2024-05-09T04:36:42.416 INFO:teuthology.orchestra.run.smithi046.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 116 kB/s | 11 kB 00:00 2024-05-09T04:36:42.441 INFO:teuthology.orchestra.run.smithi046.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-05-09T04:36:42.516 INFO:teuthology.orchestra.run.smithi046.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 231 kB/s | 23 kB 00:00 2024-05-09T04:36:42.542 INFO:teuthology.orchestra.run.smithi046.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 194 kB/s | 19 kB 00:00 2024-05-09T04:36:42.556 INFO:teuthology.orchestra.run.smithi072.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 1.0 MB/s | 1.2 MB 00:01 2024-05-09T04:36:42.600 INFO:teuthology.orchestra.run.smithi046.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 236 kB/s | 20 kB 00:00 2024-05-09T04:36:42.690 INFO:teuthology.orchestra.run.smithi072.stdout:(45/156): python3-ply-3.11-14.el9.noarch.rpm 122 kB/s | 106 kB 00:00 2024-05-09T04:36:42.700 INFO:teuthology.orchestra.run.smithi046.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 463 kB/s | 46 kB 00:00 2024-05-09T04:36:42.768 INFO:teuthology.orchestra.run.smithi046.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 4.5 MB/s | 1.0 MB 00:00 2024-05-09T04:36:42.793 INFO:teuthology.orchestra.run.smithi046.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 858 kB/s | 79 kB 00:00 2024-05-09T04:36:42.815 INFO:teuthology.orchestra.run.smithi072.stdout:(46/156): python3-pycparser-2.20-6.el9.noarch.r 146 kB/s | 135 kB 00:00 2024-05-09T04:36:42.851 INFO:teuthology.orchestra.run.smithi046.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-05-09T04:36:42.885 INFO:teuthology.orchestra.run.smithi046.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2024-05-09T04:36:42.935 INFO:teuthology.orchestra.run.smithi046.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 197 kB/s | 16 kB 00:00 2024-05-09T04:36:42.969 INFO:teuthology.orchestra.run.smithi046.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-05-09T04:36:43.019 INFO:teuthology.orchestra.run.smithi046.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 337 kB/s | 28 kB 00:00 2024-05-09T04:36:43.053 INFO:teuthology.orchestra.run.smithi046.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-05-09T04:36:43.120 INFO:teuthology.orchestra.run.smithi046.stdout:(138/156): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-05-09T04:36:43.145 INFO:teuthology.orchestra.run.smithi046.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 643 kB/s | 59 kB 00:00 2024-05-09T04:36:43.195 INFO:teuthology.orchestra.run.smithi046.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 474 kB/s | 36 kB 00:00 2024-05-09T04:36:43.199 INFO:teuthology.orchestra.run.smithi072.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-05-09T04:36:43.212 INFO:teuthology.orchestra.run.smithi046.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 762 kB/s | 51 kB 00:00 2024-05-09T04:36:43.271 INFO:teuthology.orchestra.run.smithi046.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-09T04:36:43.296 INFO:teuthology.orchestra.run.smithi046.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-05-09T04:36:43.346 INFO:teuthology.orchestra.run.smithi046.stdout:(144/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-05-09T04:36:43.363 INFO:teuthology.orchestra.run.smithi046.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 298 kB/s | 20 kB 00:00 2024-05-09T04:36:43.422 INFO:teuthology.orchestra.run.smithi046.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-09T04:36:43.442 INFO:teuthology.orchestra.run.smithi072.stdout:(48/156): python3-requests-2.25.1-8.el9.noarch. 141 kB/s | 125 kB 00:00 2024-05-09T04:36:43.500 INFO:teuthology.orchestra.run.smithi110.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 367 kB/s | 548 kB 00:01 2024-05-09T04:36:43.579 INFO:teuthology.orchestra.run.smithi072.stdout:(49/156): python3-urllib3-1.26.5-5.el9.noarch.r 242 kB/s | 215 kB 00:00 2024-05-09T04:36:43.609 INFO:teuthology.orchestra.run.smithi072.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 639 kB/s | 107 kB 00:00 2024-05-09T04:36:43.624 INFO:teuthology.orchestra.run.smithi110.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 348 kB/s | 43 kB 00:00 2024-05-09T04:36:43.660 INFO:teuthology.orchestra.run.smithi072.stdout:(51/156): bison-3.7.4-5.el9.x86_64.rpm 2.0 MB/s | 943 kB 00:00 2024-05-09T04:36:43.672 INFO:teuthology.orchestra.run.smithi046.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 5.1 MB/s | 1.6 MB 00:00 2024-05-09T04:36:43.685 INFO:teuthology.orchestra.run.smithi072.stdout:(52/156): elfutils-libelf-devel-0.190-2.el9.x86 197 kB/s | 21 kB 00:00 2024-05-09T04:36:43.719 INFO:teuthology.orchestra.run.smithi072.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 553 kB/s | 33 kB 00:00 2024-05-09T04:36:43.742 INFO:teuthology.orchestra.run.smithi110.stdout:(114/156): python3-cachetools-4.2.4-1.el9.noarc 275 kB/s | 32 kB 00:00 2024-05-09T04:36:43.753 INFO:teuthology.orchestra.run.smithi072.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.1 MB/s | 313 kB 00:00 2024-05-09T04:36:43.780 INFO:teuthology.orchestra.run.smithi072.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 301 kB/s | 18 kB 00:00 2024-05-09T04:36:43.814 INFO:teuthology.orchestra.run.smithi072.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 184 kB/s | 11 kB 00:00 2024-05-09T04:36:43.859 INFO:teuthology.orchestra.run.smithi110.stdout:(115/156): python3-certifi-2021.10.8-2.el9.noar 129 kB/s | 15 kB 00:00 2024-05-09T04:36:43.873 INFO:teuthology.orchestra.run.smithi072.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-05-09T04:36:43.907 INFO:teuthology.orchestra.run.smithi072.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-05-09T04:36:44.074 INFO:teuthology.orchestra.run.smithi072.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-05-09T04:36:44.108 INFO:teuthology.orchestra.run.smithi072.stdout:(60/156): libtool-2.4.6-45.el9.x86_64.rpm 2.8 MB/s | 582 kB 00:00 2024-05-09T04:36:44.134 INFO:teuthology.orchestra.run.smithi072.stdout:(61/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 429 kB/s | 25 kB 00:00 2024-05-09T04:36:44.176 INFO:teuthology.orchestra.run.smithi072.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 702 kB/s | 47 kB 00:00 2024-05-09T04:36:44.189 INFO:teuthology.orchestra.run.smithi046.stdout:(148/156): ceph-mgr-diskprediction-local-19.0.0 591 kB/s | 7.4 MB 00:12 2024-05-09T04:36:44.243 INFO:teuthology.orchestra.run.smithi072.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-09T04:36:44.273 INFO:teuthology.orchestra.run.smithi046.stdout:(149/156): librados2-19.0.0-3594.gc9da4049.el9. 3.9 MB/s | 3.3 MB 00:00 2024-05-09T04:36:44.277 INFO:teuthology.orchestra.run.smithi072.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-05-09T04:36:44.284 INFO:teuthology.orchestra.run.smithi110.stdout:(116/156): python3-cheroot-8.6.0-4.el9.noarch.r 404 kB/s | 172 kB 00:00 2024-05-09T04:36:44.311 INFO:teuthology.orchestra.run.smithi072.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 521 kB/s | 35 kB 00:00 2024-05-09T04:36:44.424 INFO:teuthology.orchestra.run.smithi046.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 9.9 MB/s | 2.3 MB 00:00 2024-05-09T04:36:44.465 INFO:teuthology.orchestra.run.smithi046.stdout:(151/156): NetworkManager-libnm-1.47.5-1.el9.x8 9.5 MB/s | 1.8 MB 00:00 2024-05-09T04:36:44.487 INFO:teuthology.orchestra.run.smithi072.stdout:(66/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.7 MB/s | 3.0 MB 00:00 2024-05-09T04:36:44.516 INFO:teuthology.orchestra.run.smithi046.stdout:(152/156): NetworkManager-team-1.47.5-1.el9.x86 455 kB/s | 42 kB 00:00 2024-05-09T04:36:44.566 INFO:teuthology.orchestra.run.smithi046.stdout:(153/156): NetworkManager-tui-1.47.5-1.el9.x86_ 2.4 MB/s | 248 kB 00:00 2024-05-09T04:36:44.674 INFO:teuthology.orchestra.run.smithi046.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 8.5 MB/s | 1.3 MB 00:00 2024-05-09T04:36:44.783 INFO:teuthology.orchestra.run.smithi046.stdout:(155/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 11 MB/s | 2.4 MB 00:00 2024-05-09T04:36:44.871 INFO:teuthology.orchestra.run.smithi072.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 4.1 MB/s | 1.6 MB 00:00 2024-05-09T04:36:45.043 INFO:teuthology.orchestra.run.smithi110.stdout:(117/156): python3-cherrypy-18.6.1-2.el9.noarch 473 kB/s | 358 kB 00:00 2024-05-09T04:36:45.154 INFO:teuthology.orchestra.run.smithi072.stdout:(68/156): postgresql-contrib-13.14-1.el9.x86_64 3.1 MB/s | 911 kB 00:00 2024-05-09T04:36:45.158 INFO:teuthology.orchestra.run.smithi046.stdout:(156/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 2.1 MB/s | 3.1 MB 00:01 2024-05-09T04:36:45.161 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-09T04:36:45.161 INFO:teuthology.orchestra.run.smithi046.stdout:Total 6.0 MB/s | 219 MB 00:36 2024-05-09T04:36:45.248 INFO:teuthology.orchestra.run.smithi072.stdout:(69/156): postgresql-private-libs-13.14-1.el9.x 1.4 MB/s | 133 kB 00:00 2024-05-09T04:36:45.574 INFO:teuthology.orchestra.run.smithi072.stdout:(70/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 3.5 MB/s | 4.4 MB 00:01 2024-05-09T04:36:45.750 INFO:teuthology.orchestra.run.smithi072.stdout:(71/156): openblas-openmp-0.3.21-2.el9.x86_64.r 3.2 MB/s | 4.7 MB 00:01 2024-05-09T04:36:45.785 INFO:teuthology.orchestra.run.smithi110.stdout:(118/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 409 kB/s | 303 kB 00:00 2024-05-09T04:36:45.842 INFO:teuthology.orchestra.run.smithi072.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 952 kB/s | 87 kB 00:00 2024-05-09T04:36:45.892 INFO:teuthology.orchestra.run.smithi072.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 3.2 MB/s | 1.0 MB 00:00 2024-05-09T04:36:45.910 INFO:teuthology.orchestra.run.smithi110.stdout:(119/156): python3-colorama-0.4.4-12.el9.noarch 282 kB/s | 35 kB 00:00 2024-05-09T04:36:45.976 INFO:teuthology.orchestra.run.smithi072.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.9 MB/s | 244 kB 00:00 2024-05-09T04:36:46.052 INFO:teuthology.orchestra.run.smithi072.stdout:(75/156): python3-jinja2-2.11.3-5.el9.noarch.rp 3.2 MB/s | 248 kB 00:00 2024-05-09T04:36:46.128 INFO:teuthology.orchestra.run.smithi072.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 339 kB/s | 26 kB 00:00 2024-05-09T04:36:46.186 INFO:teuthology.orchestra.run.smithi072.stdout:(77/156): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-05-09T04:36:46.262 INFO:teuthology.orchestra.run.smithi072.stdout:(78/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-05-09T04:36:46.369 INFO:teuthology.orchestra.run.smithi110.stdout:(120/156): python3-google-auth-2.29.0-1.el9.noa 449 kB/s | 206 kB 00:00 2024-05-09T04:36:46.396 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction check 2024-05-09T04:36:46.521 INFO:teuthology.orchestra.run.smithi072.stdout:(79/156): postgresql-server-13.14-1.el9.x86_64. 4.5 MB/s | 5.8 MB 00:01 2024-05-09T04:36:47.218 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction check succeeded. 2024-05-09T04:36:47.218 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction test 2024-05-09T04:36:47.219 INFO:teuthology.orchestra.run.smithi072.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 122 kB/s | 35 kB 00:00 2024-05-09T04:36:47.219 INFO:teuthology.orchestra.run.smithi072.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.1 MB/s | 250 kB 00:00 2024-05-09T04:36:47.219 INFO:teuthology.orchestra.run.smithi072.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-05-09T04:36:47.219 INFO:teuthology.orchestra.run.smithi072.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 825 kB/s | 77 kB 00:00 2024-05-09T04:36:47.219 INFO:teuthology.orchestra.run.smithi072.stdout:(84/156): python3-prettytable-0.7.2-27.el9.noar 460 kB/s | 42 kB 00:00 2024-05-09T04:36:47.219 INFO:teuthology.orchestra.run.smithi072.stdout:(85/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.4 MB/s | 6.0 MB 00:01 2024-05-09T04:36:47.227 INFO:teuthology.orchestra.run.smithi072.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 1.4 MB/s | 269 kB 00:00 2024-05-09T04:36:47.277 INFO:teuthology.orchestra.run.smithi072.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2024-05-09T04:36:47.370 INFO:teuthology.orchestra.run.smithi072.stdout:(88/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 6.0 MB/s | 5.1 MB 00:00 2024-05-09T04:36:47.395 INFO:teuthology.orchestra.run.smithi072.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 433 kB/s | 51 kB 00:00 2024-05-09T04:36:47.420 INFO:teuthology.orchestra.run.smithi072.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 1.4 MB/s | 279 kB 00:00 2024-05-09T04:36:47.455 INFO:teuthology.orchestra.run.smithi072.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 634 kB/s | 54 kB 00:00 2024-05-09T04:36:47.505 INFO:teuthology.orchestra.run.smithi072.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 493 kB/s | 42 kB 00:00 2024-05-09T04:36:47.539 INFO:teuthology.orchestra.run.smithi072.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 523 kB/s | 44 kB 00:00 2024-05-09T04:36:47.589 INFO:teuthology.orchestra.run.smithi072.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.6 MB/s | 221 kB 00:00 2024-05-09T04:36:47.623 INFO:teuthology.orchestra.run.smithi072.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 426 kB/s | 36 kB 00:00 2024-05-09T04:36:47.682 INFO:teuthology.orchestra.run.smithi072.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 512 kB/s | 47 kB 00:00 2024-05-09T04:36:47.718 INFO:teuthology.orchestra.run.smithi072.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.2 MB/s | 305 kB 00:00 2024-05-09T04:36:47.768 INFO:teuthology.orchestra.run.smithi072.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 673 kB/s | 58 kB 00:00 2024-05-09T04:36:47.802 INFO:teuthology.orchestra.run.smithi072.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-05-09T04:36:47.853 INFO:teuthology.orchestra.run.smithi072.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 756 kB/s | 64 kB 00:00 2024-05-09T04:36:47.887 INFO:teuthology.orchestra.run.smithi072.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 532 kB/s | 45 kB 00:00 2024-05-09T04:36:48.212 INFO:teuthology.orchestra.run.smithi072.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 62 kB/s | 22 kB 00:00 2024-05-09T04:36:48.469 INFO:teuthology.orchestra.run.smithi110.stdout:(121/156): libarrow-9.0.0-11.el9.x86_64.rpm 361 kB/s | 4.4 MB 00:12 2024-05-09T04:36:48.471 INFO:teuthology.orchestra.run.smithi072.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 2.1 MB/s | 548 kB 00:00 2024-05-09T04:36:48.530 INFO:teuthology.orchestra.run.smithi072.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 5.2 MB/s | 309 kB 00:00 2024-05-09T04:36:48.597 INFO:teuthology.orchestra.run.smithi072.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 289 kB/s | 19 kB 00:00 2024-05-09T04:36:48.744 INFO:teuthology.orchestra.run.smithi110.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 526 kB/s | 144 kB 00:00 2024-05-09T04:36:48.797 INFO:teuthology.orchestra.run.smithi046.stdout:Transaction test succeeded. 2024-05-09T04:36:48.798 INFO:teuthology.orchestra.run.smithi046.stdout:Running transaction 2024-05-09T04:36:48.953 INFO:teuthology.orchestra.run.smithi110.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 51 kB/s | 11 kB 00:00 2024-05-09T04:36:49.081 INFO:teuthology.orchestra.run.smithi072.stdout:(106/156): libarrow-9.0.0-11.el9.x86_64.rpm 9.1 MB/s | 4.4 MB 00:00 2024-05-09T04:36:49.095 INFO:teuthology.orchestra.run.smithi110.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 125 kB/s | 18 kB 00:00 2024-05-09T04:36:49.115 INFO:teuthology.orchestra.run.smithi072.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 703 kB/s | 863 kB 00:01 2024-05-09T04:36:49.149 INFO:teuthology.orchestra.run.smithi072.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 384 kB/s | 26 kB 00:00 2024-05-09T04:36:49.199 INFO:teuthology.orchestra.run.smithi072.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 575 kB/s | 49 kB 00:00 2024-05-09T04:36:49.225 INFO:teuthology.orchestra.run.smithi072.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 888 kB/s | 67 kB 00:00 2024-05-09T04:36:49.245 INFO:teuthology.orchestra.run.smithi110.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 154 kB/s | 23 kB 00:00 2024-05-09T04:36:49.277 INFO:teuthology.orchestra.run.smithi072.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 1.9 MB/s | 151 kB 00:00 2024-05-09T04:36:49.311 INFO:teuthology.orchestra.run.smithi072.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.6 MB/s | 839 kB 00:00 2024-05-09T04:36:49.345 INFO:teuthology.orchestra.run.smithi072.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 8.1 MB/s | 548 kB 00:00 2024-05-09T04:36:49.379 INFO:teuthology.orchestra.run.smithi110.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 146 kB/s | 19 kB 00:00 2024-05-09T04:36:49.529 INFO:teuthology.orchestra.run.smithi110.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 131 kB/s | 20 kB 00:00 2024-05-09T04:36:49.604 INFO:teuthology.orchestra.run.smithi072.stdout:(114/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 7.4 MB/s | 16 MB 00:02 2024-05-09T04:36:49.630 INFO:teuthology.orchestra.run.smithi072.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 137 kB/s | 43 kB 00:00 2024-05-09T04:36:49.655 INFO:teuthology.orchestra.run.smithi072.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 104 kB/s | 32 kB 00:00 2024-05-09T04:36:49.680 INFO:teuthology.orchestra.run.smithi072.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 198 kB/s | 15 kB 00:00 2024-05-09T04:36:49.748 INFO:teuthology.orchestra.run.smithi072.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.8 MB/s | 172 kB 00:00 2024-05-09T04:36:49.781 INFO:teuthology.orchestra.run.smithi072.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-05-09T04:36:49.815 INFO:teuthology.orchestra.run.smithi072.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.2 MB/s | 303 kB 00:00 2024-05-09T04:36:49.841 INFO:teuthology.orchestra.run.smithi072.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 380 kB/s | 35 kB 00:00 2024-05-09T04:36:49.883 INFO:teuthology.orchestra.run.smithi072.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-09T04:36:50.013 INFO:teuthology.orchestra.run.smithi110.stdout:(128/156): ceph-test-19.0.0-3594.gc9da4049.el9. 1.4 MB/s | 43 MB 00:30 2024-05-09T04:36:50.100 INFO:teuthology.orchestra.run.smithi072.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.2 MB/s | 2.0 MB 00:00 2024-05-09T04:36:50.134 INFO:teuthology.orchestra.run.smithi072.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 493 kB/s | 144 kB 00:00 2024-05-09T04:36:50.154 INFO:teuthology.orchestra.run.smithi110.stdout:(129/156): python3-grpcio-1.46.7-10.el9.x86_64. 551 kB/s | 2.0 MB 00:03 2024-05-09T04:36:50.260 INFO:teuthology.orchestra.run.smithi072.stdout:(125/156): python3-jaraco-8.2.1-3.el9.noarch.rp 28 kB/s | 11 kB 00:00 2024-05-09T04:36:50.285 INFO:teuthology.orchestra.run.smithi072.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 96 kB/s | 18 kB 00:00 2024-05-09T04:36:50.312 INFO:teuthology.orchestra.run.smithi072.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 130 kB/s | 23 kB 00:00 2024-05-09T04:36:50.354 INFO:teuthology.orchestra.run.smithi072.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 207 kB/s | 19 kB 00:00 2024-05-09T04:36:50.380 INFO:teuthology.orchestra.run.smithi072.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 292 kB/s | 20 kB 00:00 2024-05-09T04:36:50.405 INFO:teuthology.orchestra.run.smithi110.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 315 kB/s | 79 kB 00:00 2024-05-09T04:36:50.472 INFO:teuthology.orchestra.run.smithi072.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-05-09T04:36:50.489 INFO:teuthology.orchestra.run.smithi110.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 97 kB/s | 46 kB 00:00 2024-05-09T04:36:50.498 INFO:teuthology.orchestra.run.smithi072.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 324 kB/s | 46 kB 00:00 2024-05-09T04:36:50.522 INFO:teuthology.orchestra.run.smithi110.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 494 kB/s | 58 kB 00:00 2024-05-09T04:36:50.523 INFO:teuthology.orchestra.run.smithi072.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 551 kB/s | 79 kB 00:00 2024-05-09T04:36:50.548 INFO:teuthology.orchestra.run.smithi072.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 761 kB/s | 58 kB 00:00 2024-05-09T04:36:50.574 INFO:teuthology.orchestra.run.smithi072.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-09T04:36:50.641 INFO:teuthology.orchestra.run.smithi072.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 179 kB/s | 16 kB 00:00 2024-05-09T04:36:50.664 INFO:teuthology.orchestra.run.smithi110.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 116 kB/s | 16 kB 00:00 2024-05-09T04:36:50.692 INFO:teuthology.orchestra.run.smithi072.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 632 kB/s | 90 kB 00:00 2024-05-09T04:36:50.717 INFO:teuthology.orchestra.run.smithi072.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 197 kB/s | 28 kB 00:00 2024-05-09T04:36:50.743 INFO:teuthology.orchestra.run.smithi072.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 304 kB/s | 31 kB 00:00 2024-05-09T04:36:50.777 INFO:teuthology.orchestra.run.smithi072.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-05-09T04:36:50.810 INFO:teuthology.orchestra.run.smithi072.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 880 kB/s | 59 kB 00:00 2024-05-09T04:36:50.836 INFO:teuthology.orchestra.run.smithi072.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 386 kB/s | 36 kB 00:00 2024-05-09T04:36:50.862 INFO:teuthology.orchestra.run.smithi072.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 602 kB/s | 51 kB 00:00 2024-05-09T04:36:50.897 INFO:teuthology.orchestra.run.smithi072.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 2.6 MB/s | 230 kB 00:00 2024-05-09T04:36:50.914 INFO:teuthology.orchestra.run.smithi110.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 359 kB/s | 90 kB 00:00 2024-05-09T04:36:50.922 INFO:teuthology.orchestra.run.smithi072.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-05-09T04:36:50.948 INFO:teuthology.orchestra.run.smithi072.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.8 MB/s | 427 kB 00:00 2024-05-09T04:36:50.982 INFO:teuthology.orchestra.run.smithi072.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 240 kB/s | 20 kB 00:00 2024-05-09T04:36:51.040 INFO:teuthology.orchestra.run.smithi110.stdout:(135/156): python3-kubernetes-26.1.0-2.el9.noar 692 kB/s | 1.0 MB 00:01 2024-05-09T04:36:51.057 INFO:teuthology.orchestra.run.smithi110.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 198 kB/s | 28 kB 00:00 2024-05-09T04:36:51.057 INFO:teuthology.orchestra.run.smithi072.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-05-09T04:36:51.174 INFO:teuthology.orchestra.run.smithi110.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 230 kB/s | 31 kB 00:00 2024-05-09T04:36:51.299 INFO:teuthology.orchestra.run.smithi110.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 473 kB/s | 59 kB 00:00 2024-05-09T04:36:51.383 INFO:teuthology.orchestra.run.smithi072.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-05-09T04:36:51.433 INFO:teuthology.orchestra.run.smithi110.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 268 kB/s | 36 kB 00:00 2024-05-09T04:36:51.483 INFO:teuthology.orchestra.run.smithi110.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 443 kB/s | 188 kB 00:00 2024-05-09T04:36:51.575 INFO:teuthology.orchestra.run.smithi110.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 359 kB/s | 51 kB 00:00 2024-05-09T04:36:51.750 INFO:teuthology.orchestra.run.smithi110.stdout:(142/156): python3-pecan-1.4.2-3.el9.noarch.rpm 216 kB/s | 272 kB 00:01 2024-05-09T04:36:51.826 INFO:teuthology.orchestra.run.smithi110.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 358 kB/s | 90 kB 00:00 2024-05-09T04:36:51.851 INFO:teuthology.orchestra.run.smithi110.stdout:(144/156): python3-webob-1.8.7-6.el9.noarch.rpm 625 kB/s | 230 kB 00:00 2024-05-09T04:36:51.968 INFO:teuthology.orchestra.run.smithi110.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 140 kB/s | 20 kB 00:00 2024-05-09T04:36:52.252 INFO:teuthology.orchestra.run.smithi110.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 479 kB/s | 192 kB 00:00 2024-05-09T04:36:52.942 INFO:teuthology.orchestra.run.smithi072.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 1.5 MB/s | 2.3 MB 00:01 2024-05-09T04:36:52.952 INFO:teuthology.orchestra.run.smithi110.stdout:(147/156): librados2-19.0.0-3594.gc9da4049.el9. 4.7 MB/s | 3.3 MB 00:00 2024-05-09T04:36:53.269 INFO:teuthology.orchestra.run.smithi110.stdout:(148/156): python3-werkzeug-2.0.3-3.el9.1.noarc 281 kB/s | 427 kB 00:01 2024-05-09T04:36:53.528 INFO:teuthology.orchestra.run.smithi110.stdout:(149/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 5.4 MB/s | 3.1 MB 00:00 2024-05-09T04:36:53.534 INFO:teuthology.orchestra.run.smithi072.stdout:(150/156): librados2-19.0.0-3594.gc9da4049.el9. 1.3 MB/s | 3.3 MB 00:02 2024-05-09T04:36:53.801 INFO:teuthology.orchestra.run.smithi072.stdout:(151/156): librbd1-19.0.0-3594.gc9da4049.el9.x8 1.1 MB/s | 3.1 MB 00:02 2024-05-09T04:36:54.028 INFO:teuthology.orchestra.run.smithi110.stdout:(150/156): thrift-0.15.0-2.el9.x86_64.rpm 786 kB/s | 1.6 MB 00:02 2024-05-09T04:36:54.377 INFO:teuthology.orchestra.run.smithi072.stdout:(152/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.3 MB/s | 1.8 MB 00:01 2024-05-09T04:36:54.402 INFO:teuthology.orchestra.run.smithi072.stdout:(153/156): NetworkManager-team-1.47.5-1.el9.x86 48 kB/s | 42 kB 00:00 2024-05-09T04:36:54.786 INFO:teuthology.orchestra.run.smithi110.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 55 kB/s | 42 kB 00:00 2024-05-09T04:36:54.836 INFO:teuthology.orchestra.run.smithi110.stdout:(152/156): NetworkManager-1.47.5-1.el9.x86_64.r 1.5 MB/s | 2.3 MB 00:01 2024-05-09T04:36:54.844 INFO:teuthology.orchestra.run.smithi072.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 238 kB/s | 248 kB 00:01 2024-05-09T04:36:54.878 INFO:teuthology.orchestra.run.smithi110.stdout:(153/156): NetworkManager-libnm-1.47.5-1.el9.x8 1.4 MB/s | 1.8 MB 00:01 2024-05-09T04:36:54.987 INFO:teuthology.orchestra.run.smithi046.stdout: Preparing : 1/1 2024-05-09T04:36:55.064 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T04:36:55.102 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T04:36:55.161 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T04:36:55.197 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T04:36:55.222 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T04:36:55.279 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T04:36:55.428 INFO:teuthology.orchestra.run.smithi072.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-05-09T04:36:55.570 INFO:teuthology.orchestra.run.smithi072.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.1 MB/s | 2.4 MB 00:01 2024-05-09T04:36:55.577 INFO:teuthology.orchestra.run.smithi072.stdout:-------------------------------------------------------------------------------- 2024-05-09T04:36:55.578 INFO:teuthology.orchestra.run.smithi072.stdout:Total 5.3 MB/s | 219 MB 00:41 2024-05-09T04:36:55.637 INFO:teuthology.orchestra.run.smithi110.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 292 kB/s | 248 kB 00:00 2024-05-09T04:36:55.702 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T04:36:55.756 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:36:55.862 INFO:teuthology.orchestra.run.smithi110.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 1.3 MB/s | 1.3 MB 00:01 2024-05-09T04:36:55.893 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:36:55.924 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:36:56.017 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:36:56.046 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T04:36:56.054 INFO:teuthology.orchestra.run.smithi110.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-05-09T04:36:56.058 INFO:teuthology.orchestra.run.smithi110.stdout:-------------------------------------------------------------------------------- 2024-05-09T04:36:56.058 INFO:teuthology.orchestra.run.smithi110.stdout:Total 4.7 MB/s | 219 MB 00:46 2024-05-09T04:36:56.233 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T04:36:56.307 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T04:36:56.337 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:36:56.435 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:36:56.462 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T04:36:56.509 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T04:36:56.539 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:36:56.564 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:36:56.758 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:36:56.783 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction check 2024-05-09T04:36:56.926 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:36:56.935 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction check succeeded. 2024-05-09T04:36:56.935 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction test 2024-05-09T04:36:57.121 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:36:57.179 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:36:57.244 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction check 2024-05-09T04:36:57.302 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:36:57.380 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:36:57.397 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction check succeeded. 2024-05-09T04:36:57.397 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction test 2024-05-09T04:36:57.589 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:36:57.697 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T04:36:57.743 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T04:36:57.778 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T04:36:57.854 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T04:36:57.894 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T04:36:57.938 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T04:36:57.971 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T04:36:57.998 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T04:36:58.062 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T04:36:58.113 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T04:36:58.142 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T04:36:58.187 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T04:36:58.265 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T04:36:58.297 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T04:36:58.341 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T04:36:58.369 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T04:36:58.441 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T04:36:58.492 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T04:36:58.571 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T04:36:58.697 INFO:teuthology.orchestra.run.smithi072.stdout:Transaction test succeeded. 2024-05-09T04:36:58.698 INFO:teuthology.orchestra.run.smithi072.stdout:Running transaction 2024-05-09T04:36:58.716 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T04:36:58.771 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T04:36:59.208 INFO:teuthology.orchestra.run.smithi110.stdout:Transaction test succeeded. 2024-05-09T04:36:59.208 INFO:teuthology.orchestra.run.smithi110.stdout:Running transaction 2024-05-09T04:36:59.327 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T04:36:59.410 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T04:36:59.444 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T04:36:59.487 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T04:36:59.525 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T04:36:59.568 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T04:36:59.730 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T04:36:59.786 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T04:36:59.882 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T04:36:59.944 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T04:37:00.016 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T04:37:00.054 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T04:37:00.091 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T04:37:00.139 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T04:37:00.182 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T04:37:00.225 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T04:37:00.267 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T04:37:00.411 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T04:37:00.456 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T04:37:00.486 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T04:37:00.528 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T04:37:00.627 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T04:37:00.660 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T04:37:00.703 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T04:37:00.759 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T04:37:00.783 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T04:37:00.898 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T04:37:01.032 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T04:37:01.066 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T04:37:01.112 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T04:37:01.151 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T04:37:01.193 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T04:37:01.232 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T04:37:01.262 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T04:37:01.288 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T04:37:01.321 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T04:37:01.364 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T04:37:01.397 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T04:37:01.488 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T04:37:01.541 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T04:37:01.575 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T04:37:01.623 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T04:37:01.735 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T04:37:01.795 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T04:37:02.563 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T04:37:02.607 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T04:37:02.641 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T04:37:02.744 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T04:37:02.808 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T04:37:02.899 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T04:37:02.934 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T04:37:03.118 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T04:37:03.149 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T04:37:03.229 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T04:37:03.922 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T04:37:04.043 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T04:37:04.780 INFO:teuthology.orchestra.run.smithi072.stdout: Preparing : 1/1 2024-05-09T04:37:04.887 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T04:37:04.934 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T04:37:04.997 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T04:37:05.031 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T04:37:05.059 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T04:37:05.101 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T04:37:05.275 INFO:teuthology.orchestra.run.smithi110.stdout: Preparing : 1/1 2024-05-09T04:37:05.365 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T04:37:05.367 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-09T04:37:05.405 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-09T04:37:05.435 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T04:37:05.437 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T04:37:05.454 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T04:37:05.467 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-09T04:37:05.479 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T04:37:05.501 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-09T04:37:05.527 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-09T04:37:05.578 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-09T04:37:05.637 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:37:05.763 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:37:05.789 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:37:05.813 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T04:37:05.850 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T04:37:05.874 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:37:05.904 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T04:37:05.913 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-09T04:37:05.942 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T04:37:05.972 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T04:37:06.011 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T04:37:06.019 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:37:06.055 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T04:37:06.146 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T04:37:06.152 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:37:06.174 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:37:06.187 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:37:06.279 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:37:06.280 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:37:06.308 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T04:37:06.311 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T04:37:06.374 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T04:37:06.405 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:37:06.426 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:37:06.462 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-09T04:37:06.514 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-09T04:37:06.522 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T04:37:06.542 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:37:06.553 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T04:37:06.615 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:37:06.633 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T04:37:06.646 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:37:06.663 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T04:37:06.676 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T04:37:06.722 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 14/164 2024-05-09T04:37:06.744 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:37:06.766 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-09T04:37:06.776 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:37:06.951 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:37:06.952 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:37:07.003 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:37:07.080 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-09T04:37:07.115 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:37:07.217 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:37:07.250 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:37:07.260 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-09T04:37:07.383 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:37:07.445 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:37:07.452 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:37:07.564 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T04:37:07.597 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T04:37:07.633 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T04:37:07.639 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-09T04:37:07.708 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T04:37:07.749 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-09T04:37:07.752 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T04:37:07.774 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-09T04:37:07.805 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T04:37:07.809 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-09T04:37:07.838 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T04:37:07.875 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-09T04:37:07.875 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T04:37:07.917 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-09T04:37:07.953 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T04:37:07.960 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-09T04:37:07.993 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-09T04:37:08.007 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T04:37:08.021 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-09T04:37:08.048 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T04:37:08.091 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-09T04:37:08.092 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T04:37:08.144 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-09T04:37:08.168 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T04:37:08.174 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-09T04:37:08.204 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T04:37:08.209 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-09T04:37:08.248 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T04:37:08.277 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T04:37:08.285 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-09T04:37:08.320 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 32/164 2024-05-09T04:37:08.348 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T04:37:08.363 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 33/164 2024-05-09T04:37:08.392 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 34/164 2024-05-09T04:37:08.395 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T04:37:08.465 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-09T04:37:08.469 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T04:37:08.516 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-09T04:37:08.591 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-09T04:37:08.611 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T04:37:08.666 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T04:37:08.726 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-09T04:37:08.777 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-09T04:37:09.035 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:09.067 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:09.193 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T04:37:09.290 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T04:37:09.329 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T04:37:09.341 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-09T04:37:09.372 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T04:37:09.413 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T04:37:09.423 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-09T04:37:09.451 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-09T04:37:09.455 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T04:37:09.502 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-09T04:37:09.540 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 44/164 2024-05-09T04:37:09.582 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-09T04:37:09.623 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T04:37:09.662 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:09.677 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T04:37:09.704 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T04:37:09.733 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-09T04:37:09.770 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T04:37:09.788 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T04:37:09.788 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T04:37:09.788 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T04:37:09.788 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:37:09.793 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-09T04:37:09.830 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T04:37:09.835 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T04:37:09.886 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-09T04:37:09.909 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T04:37:09.942 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T04:37:09.958 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-09T04:37:09.979 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T04:37:10.031 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-09T04:37:10.034 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T04:37:10.061 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-09T04:37:10.072 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T04:37:10.098 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-09T04:37:10.116 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T04:37:10.145 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-09T04:37:10.159 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T04:37:10.181 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-09T04:37:10.224 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-09T04:37:10.267 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-09T04:37:10.293 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T04:37:10.341 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T04:37:10.382 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T04:37:10.404 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-09T04:37:10.424 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T04:37:10.447 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-09T04:37:10.477 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-09T04:37:10.518 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-09T04:37:10.519 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T04:37:10.559 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T04:37:10.600 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T04:37:10.614 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-09T04:37:10.626 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T04:37:10.643 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-09T04:37:10.651 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T04:37:10.683 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-09T04:37:10.709 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-09T04:37:10.734 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-09T04:37:10.769 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T04:37:10.810 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-09T04:37:10.899 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T04:37:10.936 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-09T04:37:10.942 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T04:37:10.978 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T04:37:10.982 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-09T04:37:11.011 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-09T04:37:11.013 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T04:37:11.045 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-09T04:37:11.061 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T04:37:11.084 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-09T04:37:11.101 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T04:37:11.116 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-09T04:37:11.141 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T04:37:11.146 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-09T04:37:11.177 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T04:37:11.182 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-09T04:37:11.211 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T04:37:11.214 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-09T04:37:11.247 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T04:37:11.261 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-09T04:37:11.281 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T04:37:11.297 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-09T04:37:11.369 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T04:37:11.386 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-09T04:37:11.420 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T04:37:11.436 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-09T04:37:11.451 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T04:37:11.477 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-09T04:37:11.499 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T04:37:11.526 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-09T04:37:11.598 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T04:37:11.632 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-09T04:37:11.653 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T04:37:11.691 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-09T04:37:12.401 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T04:37:12.406 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-09T04:37:12.450 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T04:37:12.459 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-09T04:37:12.484 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T04:37:12.494 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-09T04:37:12.585 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T04:37:12.592 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-09T04:37:12.642 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T04:37:12.655 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-09T04:37:12.731 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T04:37:12.741 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-09T04:37:12.776 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T04:37:12.777 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-09T04:37:12.944 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T04:37:12.954 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-09T04:37:12.974 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T04:37:12.985 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-09T04:37:13.052 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T04:37:13.063 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-09T04:37:13.692 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T04:37:13.722 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-09T04:37:13.814 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T04:37:13.852 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-09T04:37:15.027 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T04:37:15.095 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-09T04:37:15.128 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T04:37:15.156 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T04:37:15.172 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-09T04:37:15.191 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T04:37:15.192 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-09T04:37:15.217 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-09T04:37:15.507 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T04:37:15.560 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-09T04:37:15.571 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T04:37:15.605 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T04:37:15.662 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T04:37:15.694 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T04:37:15.695 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 101/164 2024-05-09T04:37:15.726 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 102/164 2024-05-09T04:37:15.741 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T04:37:15.762 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-09T04:37:16.229 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T04:37:16.255 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-09T04:37:16.262 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T04:37:16.282 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T04:37:16.342 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T04:37:16.345 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 105/164 2024-05-09T04:37:16.367 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T04:37:16.373 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 106/164 2024-05-09T04:37:18.584 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:18.631 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:18.651 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:18.697 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:19.192 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:19.235 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T04:37:19.253 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 107/164 2024-05-09T04:37:19.295 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T04:37:19.309 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T04:37:19.309 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T04:37:19.309 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T04:37:19.309 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:37:19.336 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T04:37:19.383 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 108/164 2024-05-09T04:37:19.383 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-09T04:37:19.383 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-09T04:37:19.383 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-09T04:37:19.415 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T04:37:21.795 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T04:37:21.795 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /sys 2024-05-09T04:37:21.796 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /proc 2024-05-09T04:37:21.796 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /mnt 2024-05-09T04:37:21.796 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /var/tmp 2024-05-09T04:37:21.796 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /home 2024-05-09T04:37:21.796 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /root 2024-05-09T04:37:21.796 INFO:teuthology.orchestra.run.smithi046.stdout:skipping the directory /tmp 2024-05-09T04:37:21.796 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:37:21.969 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T04:37:21.985 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T04:37:21.986 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:21.986 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T04:37:21.986 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:37:23.632 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T04:37:23.660 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T04:37:23.660 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:23.660 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T04:37:23.660 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:37:23.660 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:37:23.660 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:37:23.899 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T04:37:23.924 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T04:37:23.924 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:23.924 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T04:37:23.924 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:37:23.924 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:37:23.924 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:37:24.408 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T04:37:24.433 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T04:37:24.433 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:24.433 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T04:37:24.433 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:37:24.433 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:37:24.433 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:37:24.539 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T04:37:24.597 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T04:37:24.660 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T04:37:24.856 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T04:37:24.909 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T04:37:24.998 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T04:37:25.096 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T04:37:25.241 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T04:37:25.293 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T04:37:25.324 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T04:37:25.351 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T04:37:25.806 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T04:37:25.833 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T04:37:25.866 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T04:37:25.954 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T04:37:25.981 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T04:37:27.074 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T04:37:27.108 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T04:37:28.121 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T04:37:28.141 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T04:37:28.262 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T04:37:28.440 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T04:37:28.477 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T04:37:28.502 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T04:37:28.502 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:28.502 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T04:37:28.502 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:37:28.502 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:37:28.502 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:37:28.552 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T04:37:28.564 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T04:37:28.666 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T04:37:28.728 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T04:37:29.712 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T04:37:29.738 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T04:37:29.763 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T04:37:29.763 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:29.763 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T04:37:29.764 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:37:29.764 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:37:29.764 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:37:29.805 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T04:37:29.830 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T04:37:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:29.830 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T04:37:29.830 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:37:29.881 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T04:37:29.906 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T04:37:29.906 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:29.906 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T04:37:29.906 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:37:29.906 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:37:29.906 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:37:30.232 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T04:37:30.256 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T04:37:30.257 INFO:teuthology.orchestra.run.smithi046.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:30.257 INFO:teuthology.orchestra.run.smithi046.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T04:37:30.257 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:37:30.257 INFO:teuthology.orchestra.run.smithi046.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:37:30.257 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:37:31.152 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T04:37:31.153 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /sys 2024-05-09T04:37:31.153 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /proc 2024-05-09T04:37:31.153 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /mnt 2024-05-09T04:37:31.153 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /var/tmp 2024-05-09T04:37:31.153 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /home 2024-05-09T04:37:31.153 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /root 2024-05-09T04:37:31.153 INFO:teuthology.orchestra.run.smithi110.stdout:skipping the directory /tmp 2024-05-09T04:37:31.153 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:37:31.332 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T04:37:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T04:37:31.347 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:31.347 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T04:37:31.347 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:37:32.901 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T04:37:32.926 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T04:37:32.926 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:32.927 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T04:37:32.927 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:37:32.927 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:37:32.927 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:37:33.161 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T04:37:33.184 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T04:37:33.184 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:33.184 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T04:37:33.184 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:37:33.184 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:37:33.184 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:37:33.645 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T04:37:33.668 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T04:37:33.668 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:33.668 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T04:37:33.668 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:37:33.668 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:37:33.668 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:37:33.783 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T04:37:33.840 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T04:37:33.896 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T04:37:34.084 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T04:37:34.130 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T04:37:34.207 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T04:37:34.212 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 109/164 2024-05-09T04:37:34.212 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /sys 2024-05-09T04:37:34.213 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /proc 2024-05-09T04:37:34.213 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /mnt 2024-05-09T04:37:34.213 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /var/tmp 2024-05-09T04:37:34.213 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /home 2024-05-09T04:37:34.213 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /root 2024-05-09T04:37:34.213 INFO:teuthology.orchestra.run.smithi072.stdout:skipping the directory /tmp 2024-05-09T04:37:34.213 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-09T04:37:34.296 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T04:37:34.388 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T04:37:34.406 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 110/164 2024-05-09T04:37:34.406 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:34.407 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-09T04:37:34.407 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-09T04:37:34.444 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T04:37:34.496 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T04:37:34.528 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T04:37:34.564 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T04:37:34.777 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T04:37:34.819 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T04:37:34.860 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T04:37:34.900 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T04:37:34.927 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T04:37:35.016 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T04:37:35.046 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T04:37:35.078 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T04:37:35.138 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T04:37:35.139 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T04:37:35.169 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T04:37:35.189 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T04:37:35.219 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T04:37:35.374 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T04:37:35.420 INFO:teuthology.orchestra.run.smithi046.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T04:37:35.996 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T04:37:36.022 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:37:36.022 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 111/164 2024-05-09T04:37:36.022 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:36.022 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-09T04:37:36.023 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:37:36.023 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-09T04:37:36.023 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-09T04:37:36.059 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:37:36.081 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:37:36.213 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T04:37:36.247 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T04:37:36.253 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T04:37:36.270 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 112/164 2024-05-09T04:37:36.270 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:36.270 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-09T04:37:36.270 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:37:36.270 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-09T04:37:36.270 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-09T04:37:36.731 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T04:37:36.754 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 113/164 2024-05-09T04:37:36.754 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:36.754 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-09T04:37:36.754 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:37:36.754 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-09T04:37:36.755 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-09T04:37:36.854 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-09T04:37:36.911 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-09T04:37:36.977 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-09T04:37:37.011 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T04:37:37.089 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T04:37:37.141 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T04:37:37.162 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-09T04:37:37.213 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T04:37:37.221 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-09T04:37:37.238 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T04:37:37.245 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T04:37:37.275 INFO:teuthology.orchestra.run.smithi046.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T04:37:37.292 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-09T04:37:37.301 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T04:37:37.301 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:37:37.357 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T04:37:37.385 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:37:37.390 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-09T04:37:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T04:37:37.419 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:37:37.503 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:37:37.519 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T04:37:37.528 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T04:37:37.530 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-09T04:37:37.542 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:37:37.542 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:37:37.548 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T04:37:37.570 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T04:37:37.570 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:37.571 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T04:37:37.571 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:37:37.571 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:37:37.571 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:37:37.585 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-09T04:37:37.615 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T04:37:37.626 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-09T04:37:37.626 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T04:37:37.654 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-09T04:37:37.723 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T04:37:37.729 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:37:37.729 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:37:37.771 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T04:37:37.822 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:37:37.890 INFO:teuthology.orchestra.run.smithi046.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:37:38.102 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T04:37:38.133 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 125/164 2024-05-09T04:37:38.165 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 126/164 2024-05-09T04:37:38.227 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 127/164 2024-05-09T04:37:38.259 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T04:37:38.703 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T04:37:38.734 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T04:37:38.757 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T04:37:38.757 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:38.757 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T04:37:38.757 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:37:38.757 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:37:38.758 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:37:38.801 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T04:37:38.824 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T04:37:38.824 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:38.824 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T04:37:38.824 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:37:38.869 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T04:37:38.891 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T04:37:38.891 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:38.891 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T04:37:38.891 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:37:38.892 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:37:38.892 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:37:39.202 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T04:37:39.225 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T04:37:39.225 INFO:teuthology.orchestra.run.smithi110.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:39.226 INFO:teuthology.orchestra.run.smithi110.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T04:37:39.226 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:37:39.226 INFO:teuthology.orchestra.run.smithi110.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:37:39.226 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:37:39.296 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 128/164 2024-05-09T04:37:39.373 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T04:37:40.339 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 129/164 2024-05-09T04:37:40.371 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T04:37:40.481 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 130/164 2024-05-09T04:37:40.648 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 131/164 2024-05-09T04:37:40.677 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T04:37:40.700 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 132/164 2024-05-09T04:37:40.700 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:40.700 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-09T04:37:40.700 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:37:40.700 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-09T04:37:40.701 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-09T04:37:40.753 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T04:37:40.765 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 133/164 2024-05-09T04:37:40.871 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 134/164 2024-05-09T04:37:40.927 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 135/164 2024-05-09T04:37:41.859 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-09T04:37:41.900 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T04:37:41.923 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 137/164 2024-05-09T04:37:41.923 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:41.923 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-09T04:37:41.923 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:37:41.923 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-09T04:37:41.923 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-09T04:37:41.967 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T04:37:41.989 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 138/164 2024-05-09T04:37:41.989 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:41.989 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-09T04:37:41.989 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-09T04:37:42.045 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T04:37:42.068 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 139/164 2024-05-09T04:37:42.068 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:42.068 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-09T04:37:42.068 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:37:42.068 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-09T04:37:42.068 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-09T04:37:42.379 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T04:37:42.402 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 140/164 2024-05-09T04:37:42.402 INFO:teuthology.orchestra.run.smithi072.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-09T04:37:42.402 INFO:teuthology.orchestra.run.smithi072.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-09T04:37:42.402 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:37:42.403 INFO:teuthology.orchestra.run.smithi072.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-09T04:37:42.403 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-09T04:37:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:37:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T04:37:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T04:37:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T04:37:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T04:37:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T04:37:43.426 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T04:37:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T04:37:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:37:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:37:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T04:37:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T04:37:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T04:37:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:37:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T04:37:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T04:37:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T04:37:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T04:37:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T04:37:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T04:37:43.427 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T04:37:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T04:37:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T04:37:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T04:37:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T04:37:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T04:37:43.428 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T04:37:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T04:37:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T04:37:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T04:37:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T04:37:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T04:37:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T04:37:43.442 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T04:37:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T04:37:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T04:37:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T04:37:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T04:37:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T04:37:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T04:37:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T04:37:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T04:37:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T04:37:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T04:37:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T04:37:43.443 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T04:37:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T04:37:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T04:37:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T04:37:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T04:37:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T04:37:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T04:37:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T04:37:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T04:37:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T04:37:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T04:37:43.444 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T04:37:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T04:37:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T04:37:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T04:37:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T04:37:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T04:37:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T04:37:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T04:37:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T04:37:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T04:37:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T04:37:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T04:37:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T04:37:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T04:37:43.445 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T04:37:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T04:37:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T04:37:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T04:37:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T04:37:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T04:37:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T04:37:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T04:37:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T04:37:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T04:37:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T04:37:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T04:37:43.446 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T04:37:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T04:37:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T04:37:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T04:37:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T04:37:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T04:37:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T04:37:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T04:37:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T04:37:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T04:37:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T04:37:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T04:37:43.447 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T04:37:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T04:37:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T04:37:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T04:37:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T04:37:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T04:37:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T04:37:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T04:37:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T04:37:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T04:37:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T04:37:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T04:37:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T04:37:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T04:37:43.448 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T04:37:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T04:37:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T04:37:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T04:37:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T04:37:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T04:37:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T04:37:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T04:37:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T04:37:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T04:37:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T04:37:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T04:37:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T04:37:43.449 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T04:37:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T04:37:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T04:37:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T04:37:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T04:37:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T04:37:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T04:37:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T04:37:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T04:37:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T04:37:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T04:37:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T04:37:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T04:37:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T04:37:43.450 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T04:37:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T04:37:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T04:37:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T04:37:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T04:37:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T04:37:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T04:37:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T04:37:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T04:37:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T04:37:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T04:37:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T04:37:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T04:37:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T04:37:43.451 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T04:37:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T04:37:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T04:37:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T04:37:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T04:37:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T04:37:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T04:37:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T04:37:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T04:37:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T04:37:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T04:37:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T04:37:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T04:37:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T04:37:43.452 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T04:37:43.473 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T04:37:43.514 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T04:37:43.555 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T04:37:43.587 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T04:37:43.614 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T04:37:43.799 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T04:37:43.835 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T04:37:43.866 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T04:37:43.895 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T04:37:43.941 INFO:teuthology.orchestra.run.smithi110.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T04:37:44.540 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:37:44.581 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:37:44.601 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:37:45.425 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T04:37:45.518 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T04:37:45.559 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T04:37:45.642 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T04:37:45.699 INFO:teuthology.orchestra.run.smithi110.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T04:37:45.731 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T04:37:45.731 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:37:45.815 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:37:45.848 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T04:37:45.848 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:37:45.950 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:37:45.974 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T04:37:45.980 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:37:45.980 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:37:46.104 INFO:teuthology.orchestra.run.smithi046.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:37:46.104 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:37:46.105 INFO:teuthology.orchestra.run.smithi046.stdout:Upgraded: 2024-05-09T04:37:46.105 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:46.105 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:46.105 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:46.105 INFO:teuthology.orchestra.run.smithi046.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:46.105 INFO:teuthology.orchestra.run.smithi046.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.105 INFO:teuthology.orchestra.run.smithi046.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.105 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T04:37:46.105 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T04:37:46.105 INFO:teuthology.orchestra.run.smithi046.stdout:Installed: 2024-05-09T04:37:46.105 INFO:teuthology.orchestra.run.smithi046.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T04:37:46.105 INFO:teuthology.orchestra.run.smithi046.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T04:37:46.105 INFO:teuthology.orchestra.run.smithi046.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T04:37:46.106 INFO:teuthology.orchestra.run.smithi046.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T04:37:46.106 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.106 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.106 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.106 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.106 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:46.106 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.106 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.106 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.106 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:46.106 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:46.106 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:46.106 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:46.106 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:46.107 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.107 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.107 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:46.107 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.107 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.107 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.107 INFO:teuthology.orchestra.run.smithi046.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:46.107 INFO:teuthology.orchestra.run.smithi046.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:46.107 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.107 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.107 INFO:teuthology.orchestra.run.smithi046.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:46.108 INFO:teuthology.orchestra.run.smithi046.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T04:37:46.108 INFO:teuthology.orchestra.run.smithi046.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T04:37:46.108 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T04:37:46.108 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T04:37:46.108 INFO:teuthology.orchestra.run.smithi046.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T04:37:46.108 INFO:teuthology.orchestra.run.smithi046.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T04:37:46.108 INFO:teuthology.orchestra.run.smithi046.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T04:37:46.108 INFO:teuthology.orchestra.run.smithi046.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T04:37:46.109 INFO:teuthology.orchestra.run.smithi046.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T04:37:46.109 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T04:37:46.109 INFO:teuthology.orchestra.run.smithi046.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T04:37:46.109 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.110 INFO:teuthology.orchestra.run.smithi046.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.110 INFO:teuthology.orchestra.run.smithi046.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.110 INFO:teuthology.orchestra.run.smithi046.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T04:37:46.110 INFO:teuthology.orchestra.run.smithi046.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T04:37:46.110 INFO:teuthology.orchestra.run.smithi046.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T04:37:46.110 INFO:teuthology.orchestra.run.smithi046.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T04:37:46.110 INFO:teuthology.orchestra.run.smithi046.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T04:37:46.111 INFO:teuthology.orchestra.run.smithi046.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.111 INFO:teuthology.orchestra.run.smithi046.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.111 INFO:teuthology.orchestra.run.smithi046.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T04:37:46.111 INFO:teuthology.orchestra.run.smithi046.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.111 INFO:teuthology.orchestra.run.smithi046.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T04:37:46.111 INFO:teuthology.orchestra.run.smithi046.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T04:37:46.111 INFO:teuthology.orchestra.run.smithi046.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T04:37:46.111 INFO:teuthology.orchestra.run.smithi046.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T04:37:46.111 INFO:teuthology.orchestra.run.smithi046.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T04:37:46.111 INFO:teuthology.orchestra.run.smithi046.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T04:37:46.111 INFO:teuthology.orchestra.run.smithi046.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T04:37:46.111 INFO:teuthology.orchestra.run.smithi046.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T04:37:46.111 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T04:37:46.112 INFO:teuthology.orchestra.run.smithi046.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T04:37:46.112 INFO:teuthology.orchestra.run.smithi046.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T04:37:46.112 INFO:teuthology.orchestra.run.smithi046.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T04:37:46.112 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T04:37:46.112 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T04:37:46.112 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T04:37:46.112 INFO:teuthology.orchestra.run.smithi046.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T04:37:46.112 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T04:37:46.112 INFO:teuthology.orchestra.run.smithi046.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T04:37:46.112 INFO:teuthology.orchestra.run.smithi046.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T04:37:46.112 INFO:teuthology.orchestra.run.smithi046.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T04:37:46.112 INFO:teuthology.orchestra.run.smithi046.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T04:37:46.112 INFO:teuthology.orchestra.run.smithi046.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T04:37:46.113 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T04:37:46.113 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.113 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.113 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.113 INFO:teuthology.orchestra.run.smithi046.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T04:37:46.113 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T04:37:46.113 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T04:37:46.113 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T04:37:46.114 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T04:37:46.114 INFO:teuthology.orchestra.run.smithi046.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T04:37:46.114 INFO:teuthology.orchestra.run.smithi046.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T04:37:46.114 INFO:teuthology.orchestra.run.smithi046.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T04:37:46.114 INFO:teuthology.orchestra.run.smithi046.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T04:37:46.114 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T04:37:46.114 INFO:teuthology.orchestra.run.smithi046.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T04:37:46.114 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T04:37:46.115 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T04:37:46.115 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T04:37:46.115 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T04:37:46.115 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T04:37:46.115 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T04:37:46.115 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T04:37:46.115 INFO:teuthology.orchestra.run.smithi046.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T04:37:46.115 INFO:teuthology.orchestra.run.smithi046.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T04:37:46.115 INFO:teuthology.orchestra.run.smithi046.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T04:37:46.115 INFO:teuthology.orchestra.run.smithi046.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T04:37:46.115 INFO:teuthology.orchestra.run.smithi046.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T04:37:46.115 INFO:teuthology.orchestra.run.smithi046.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T04:37:46.115 INFO:teuthology.orchestra.run.smithi046.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T04:37:46.116 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T04:37:46.116 INFO:teuthology.orchestra.run.smithi046.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T04:37:46.116 INFO:teuthology.orchestra.run.smithi046.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T04:37:46.116 INFO:teuthology.orchestra.run.smithi046.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T04:37:46.116 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T04:37:46.116 INFO:teuthology.orchestra.run.smithi046.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T04:37:46.116 INFO:teuthology.orchestra.run.smithi046.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T04:37:46.116 INFO:teuthology.orchestra.run.smithi046.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T04:37:46.116 INFO:teuthology.orchestra.run.smithi046.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T04:37:46.116 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T04:37:46.116 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T04:37:46.117 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T04:37:46.117 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T04:37:46.117 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T04:37:46.117 INFO:teuthology.orchestra.run.smithi046.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T04:37:46.117 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.117 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.117 INFO:teuthology.orchestra.run.smithi046.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T04:37:46.117 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T04:37:46.117 INFO:teuthology.orchestra.run.smithi046.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T04:37:46.117 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.117 INFO:teuthology.orchestra.run.smithi046.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T04:37:46.118 INFO:teuthology.orchestra.run.smithi046.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T04:37:46.118 INFO:teuthology.orchestra.run.smithi046.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T04:37:46.118 INFO:teuthology.orchestra.run.smithi046.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T04:37:46.118 INFO:teuthology.orchestra.run.smithi046.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T04:37:46.118 INFO:teuthology.orchestra.run.smithi046.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T04:37:46.118 INFO:teuthology.orchestra.run.smithi046.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T04:37:46.118 INFO:teuthology.orchestra.run.smithi046.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T04:37:46.118 INFO:teuthology.orchestra.run.smithi046.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T04:37:46.118 INFO:teuthology.orchestra.run.smithi046.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T04:37:46.118 INFO:teuthology.orchestra.run.smithi046.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T04:37:46.118 INFO:teuthology.orchestra.run.smithi046.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T04:37:46.119 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T04:37:46.119 INFO:teuthology.orchestra.run.smithi046.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T04:37:46.119 INFO:teuthology.orchestra.run.smithi046.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T04:37:46.119 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.119 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.119 INFO:teuthology.orchestra.run.smithi046.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:46.119 INFO:teuthology.orchestra.run.smithi046.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T04:37:46.119 INFO:teuthology.orchestra.run.smithi046.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T04:37:46.119 INFO:teuthology.orchestra.run.smithi046.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T04:37:46.119 INFO:teuthology.orchestra.run.smithi046.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T04:37:46.119 INFO:teuthology.orchestra.run.smithi046.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T04:37:46.119 INFO:teuthology.orchestra.run.smithi046.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T04:37:46.120 INFO:teuthology.orchestra.run.smithi046.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T04:37:46.120 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:37:46.120 INFO:teuthology.orchestra.run.smithi046.stdout:Complete! 2024-05-09T04:37:46.168 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:37:46.168 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:37:46.260 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:37:46.339 INFO:teuthology.orchestra.run.smithi110.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:37:46.518 DEBUG:teuthology.parallel:result is None 2024-05-09T04:37:46.664 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 141/164 2024-05-09T04:37:46.714 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 142/164 2024-05-09T04:37:46.755 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 143/164 2024-05-09T04:37:46.789 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-09T04:37:46.816 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-09T04:37:46.997 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 146/164 2024-05-09T04:37:47.052 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-09T04:37:47.084 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 148/164 2024-05-09T04:37:47.114 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-09T04:37:47.161 INFO:teuthology.orchestra.run.smithi072.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-09T04:37:47.811 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:37:47.861 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:37:47.886 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-09T04:37:48.727 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 152/164 2024-05-09T04:37:48.837 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-09T04:37:48.887 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-09T04:37:48.979 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-09T04:37:49.020 INFO:teuthology.orchestra.run.smithi072.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-09T04:37:49.055 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-09T04:37:49.055 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:37:49.149 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-09T04:37:49.192 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-09T04:37:49.192 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:37:49.286 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-09T04:37:49.311 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-09T04:37:49.316 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:37:49.316 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:37:49.505 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-09T04:37:49.505 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:37:49.604 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-09T04:37:49.676 INFO:teuthology.orchestra.run.smithi072.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:37:51.984 INFO:teuthology.orchestra.run.smithi110.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:37:51.985 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T04:37:51.985 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T04:37:51.985 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T04:37:51.985 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T04:37:51.985 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T04:37:51.985 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T04:37:51.985 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T04:37:51.985 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:37:51.985 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:37:51.985 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T04:37:51.986 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T04:37:51.986 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T04:37:51.986 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:37:51.986 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T04:37:51.986 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T04:37:51.986 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T04:37:51.986 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T04:37:51.986 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T04:37:51.986 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T04:37:51.986 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T04:37:51.986 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T04:37:51.986 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T04:37:51.987 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T04:37:51.987 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T04:37:51.987 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T04:37:51.987 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T04:37:51.987 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T04:37:51.987 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T04:37:51.987 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T04:37:51.987 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T04:37:52.000 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T04:37:52.000 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T04:37:52.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T04:37:52.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T04:37:52.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T04:37:52.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T04:37:52.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T04:37:52.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T04:37:52.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T04:37:52.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T04:37:52.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T04:37:52.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T04:37:52.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T04:37:52.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T04:37:52.001 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T04:37:52.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T04:37:52.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T04:37:52.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T04:37:52.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T04:37:52.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T04:37:52.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T04:37:52.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T04:37:52.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T04:37:52.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T04:37:52.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T04:37:52.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T04:37:52.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T04:37:52.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T04:37:52.002 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T04:37:52.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T04:37:52.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T04:37:52.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T04:37:52.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T04:37:52.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T04:37:52.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T04:37:52.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T04:37:52.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T04:37:52.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T04:37:52.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T04:37:52.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T04:37:52.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T04:37:52.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T04:37:52.003 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T04:37:52.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T04:37:52.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T04:37:52.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T04:37:52.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T04:37:52.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T04:37:52.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T04:37:52.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T04:37:52.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T04:37:52.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T04:37:52.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T04:37:52.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T04:37:52.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T04:37:52.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T04:37:52.004 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T04:37:52.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T04:37:52.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T04:37:52.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T04:37:52.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T04:37:52.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T04:37:52.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T04:37:52.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T04:37:52.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T04:37:52.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T04:37:52.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T04:37:52.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T04:37:52.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T04:37:52.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T04:37:52.005 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T04:37:52.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T04:37:52.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T04:37:52.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T04:37:52.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T04:37:52.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T04:37:52.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T04:37:52.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T04:37:52.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T04:37:52.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T04:37:52.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T04:37:52.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T04:37:52.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T04:37:52.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T04:37:52.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T04:37:52.006 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T04:37:52.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T04:37:52.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T04:37:52.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T04:37:52.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T04:37:52.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T04:37:52.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T04:37:52.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T04:37:52.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T04:37:52.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T04:37:52.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T04:37:52.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T04:37:52.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T04:37:52.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T04:37:52.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T04:37:52.007 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T04:37:52.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T04:37:52.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T04:37:52.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T04:37:52.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T04:37:52.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T04:37:52.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T04:37:52.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T04:37:52.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T04:37:52.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T04:37:52.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T04:37:52.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T04:37:52.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T04:37:52.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T04:37:52.008 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T04:37:52.009 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T04:37:52.010 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T04:37:52.010 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T04:37:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:37:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:37:54.660 INFO:teuthology.orchestra.run.smithi110.stdout:Upgraded: 2024-05-09T04:37:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:54.660 INFO:teuthology.orchestra.run.smithi110.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T04:37:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T04:37:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T04:37:54.661 INFO:teuthology.orchestra.run.smithi110.stdout:Installed: 2024-05-09T04:37:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T04:37:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T04:37:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T04:37:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T04:37:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.661 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.662 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:37:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T04:37:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T04:37:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T04:37:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T04:37:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T04:37:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T04:37:54.663 INFO:teuthology.orchestra.run.smithi110.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T04:37:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T04:37:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T04:37:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T04:37:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T04:37:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T04:37:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T04:37:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T04:37:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T04:37:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T04:37:54.664 INFO:teuthology.orchestra.run.smithi110.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T04:37:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T04:37:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T04:37:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T04:37:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T04:37:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T04:37:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T04:37:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T04:37:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T04:37:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T04:37:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T04:37:54.665 INFO:teuthology.orchestra.run.smithi110.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T04:37:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T04:37:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T04:37:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T04:37:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T04:37:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T04:37:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T04:37:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T04:37:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T04:37:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T04:37:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T04:37:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T04:37:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T04:37:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.666 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T04:37:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T04:37:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T04:37:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T04:37:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T04:37:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T04:37:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T04:37:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T04:37:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T04:37:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T04:37:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T04:37:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T04:37:54.667 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T04:37:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T04:37:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T04:37:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T04:37:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T04:37:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T04:37:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T04:37:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T04:37:54.668 INFO:teuthology.orchestra.run.smithi110.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T04:37:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T04:37:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T04:37:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T04:37:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T04:37:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T04:37:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T04:37:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T04:37:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T04:37:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T04:37:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T04:37:54.669 INFO:teuthology.orchestra.run.smithi110.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T04:37:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T04:37:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T04:37:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T04:37:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T04:37:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T04:37:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T04:37:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T04:37:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T04:37:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T04:37:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T04:37:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T04:37:54.670 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T04:37:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T04:37:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T04:37:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T04:37:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T04:37:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T04:37:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T04:37:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T04:37:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T04:37:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T04:37:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T04:37:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T04:37:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T04:37:54.671 INFO:teuthology.orchestra.run.smithi110.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T04:37:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T04:37:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:37:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T04:37:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T04:37:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T04:37:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T04:37:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T04:37:54.672 INFO:teuthology.orchestra.run.smithi110.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T04:37:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T04:37:54.673 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:37:54.673 INFO:teuthology.orchestra.run.smithi110.stdout:Complete! 2024-05-09T04:37:55.094 DEBUG:teuthology.parallel:result is None 2024-05-09T04:37:59.632 INFO:teuthology.orchestra.run.smithi072.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:37:59.632 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 1/164 2024-05-09T04:37:59.632 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2/164 2024-05-09T04:37:59.632 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 3/164 2024-05-09T04:37:59.632 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 4/164 2024-05-09T04:37:59.632 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3594.gc9da4 5/164 2024-05-09T04:37:59.632 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 6/164 2024-05-09T04:37:59.632 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 7/164 2024-05-09T04:37:59.633 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 8/164 2024-05-09T04:37:59.633 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 9/164 2024-05-09T04:37:59.633 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 10/164 2024-05-09T04:37:59.633 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 11/164 2024-05-09T04:37:59.633 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 12/164 2024-05-09T04:37:59.633 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 13/164 2024-05-09T04:37:59.633 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 14/164 2024-05-09T04:37:59.633 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_ 15/164 2024-05-09T04:37:59.633 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 16/164 2024-05-09T04:37:59.633 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 17/164 2024-05-09T04:37:59.633 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_6 18/164 2024-05-09T04:37:59.633 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86 19/164 2024-05-09T04:37:59.633 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 20/164 2024-05-09T04:37:59.633 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el 21/164 2024-05-09T04:37:59.634 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ceph-common-2:19.0.0-3594.gc9da4049.el9. 22/164 2024-05-09T04:37:59.634 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_6 23/164 2024-05-09T04:37:59.634 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 24/164 2024-05-09T04:37:59.634 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 25/164 2024-05-09T04:37:59.634 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 26/164 2024-05-09T04:37:59.634 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 27/164 2024-05-09T04:37:59.634 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 28/164 2024-05-09T04:37:59.634 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 29/164 2024-05-09T04:37:59.634 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049. 30/164 2024-05-09T04:37:59.647 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noa 31/164 2024-05-09T04:37:59.647 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.n 32/164 2024-05-09T04:37:59.647 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9d 33/164 2024-05-09T04:37:59.647 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el 34/164 2024-05-09T04:37:59.647 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 35/164 2024-05-09T04:37:59.647 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.e 36/164 2024-05-09T04:37:59.647 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 37/164 2024-05-09T04:37:59.647 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 38/164 2024-05-09T04:37:59.648 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 39/164 2024-05-09T04:37:59.648 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-09T04:37:59.648 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-09T04:37:59.648 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-09T04:37:59.648 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-09T04:37:59.648 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-09T04:37:59.648 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-09T04:37:59.648 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-09T04:37:59.648 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-09T04:37:59.648 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-09T04:37:59.648 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-09T04:37:59.648 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-09T04:37:59.648 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-09T04:37:59.648 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-09T04:37:59.649 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-09T04:37:59.649 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-09T04:37:59.649 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-09T04:37:59.649 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-09T04:37:59.649 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-09T04:37:59.649 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-09T04:37:59.649 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-09T04:37:59.649 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-09T04:37:59.649 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-09T04:37:59.649 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-09T04:37:59.649 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-09T04:37:59.649 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-09T04:37:59.649 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-09T04:37:59.649 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-09T04:37:59.650 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-09T04:37:59.650 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-09T04:37:59.650 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-09T04:37:59.650 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-09T04:37:59.650 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-09T04:37:59.650 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-09T04:37:59.650 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-09T04:37:59.650 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-09T04:37:59.650 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-09T04:37:59.650 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-09T04:37:59.650 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-09T04:37:59.650 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-09T04:37:59.650 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-09T04:37:59.651 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-09T04:37:59.651 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-09T04:37:59.651 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-09T04:37:59.651 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-09T04:37:59.651 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-09T04:37:59.651 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-09T04:37:59.651 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-09T04:37:59.651 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-09T04:37:59.651 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-09T04:37:59.651 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-09T04:37:59.651 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-09T04:37:59.651 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-09T04:37:59.651 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-09T04:37:59.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-09T04:37:59.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-09T04:37:59.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-09T04:37:59.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-09T04:37:59.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-09T04:37:59.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-09T04:37:59.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-09T04:37:59.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-09T04:37:59.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-09T04:37:59.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-09T04:37:59.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-09T04:37:59.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-09T04:37:59.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-09T04:37:59.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-09T04:37:59.652 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-09T04:37:59.653 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-09T04:37:59.653 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-09T04:37:59.653 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-09T04:37:59.653 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-09T04:37:59.653 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-09T04:37:59.653 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-09T04:37:59.653 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-09T04:37:59.653 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-09T04:37:59.653 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-09T04:37:59.653 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-09T04:37:59.653 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-09T04:37:59.653 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-09T04:37:59.653 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-09T04:37:59.653 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-09T04:37:59.653 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-09T04:37:59.654 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-09T04:37:59.654 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-09T04:37:59.654 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-09T04:37:59.654 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-09T04:37:59.654 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-09T04:37:59.654 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-09T04:37:59.654 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-09T04:37:59.654 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-09T04:37:59.654 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-09T04:37:59.654 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-09T04:37:59.654 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-09T04:37:59.654 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-09T04:37:59.654 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-09T04:37:59.655 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-09T04:37:59.655 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-09T04:37:59.655 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-09T04:37:59.655 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-09T04:37:59.655 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-09T04:37:59.655 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-09T04:37:59.655 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-09T04:37:59.655 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-09T04:37:59.655 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-09T04:37:59.655 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-09T04:37:59.655 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-09T04:37:59.655 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-09T04:37:59.655 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-09T04:37:59.655 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 149/164 2024-05-09T04:37:59.656 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-09T04:37:59.656 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 151/164 2024-05-09T04:37:59.656 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-09T04:37:59.656 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-09T04:37:59.656 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-09T04:37:59.656 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-09T04:37:59.656 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-09T04:37:59.656 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-09T04:37:59.656 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-09T04:37:59.656 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-09T04:37:59.656 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-09T04:37:59.656 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-09T04:37:59.656 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-09T04:37:59.657 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-09T04:38:02.203 INFO:teuthology.orchestra.run.smithi072.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-09T04:38:02.203 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-09T04:38:02.203 INFO:teuthology.orchestra.run.smithi072.stdout:Upgraded: 2024-05-09T04:38:02.203 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-09T04:38:02.203 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-09T04:38:02.203 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-09T04:38:02.203 INFO:teuthology.orchestra.run.smithi072.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-09T04:38:02.203 INFO:teuthology.orchestra.run.smithi072.stdout: librados2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.203 INFO:teuthology.orchestra.run.smithi072.stdout: librbd1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.204 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-09T04:38:02.204 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-09T04:38:02.204 INFO:teuthology.orchestra.run.smithi072.stdout:Installed: 2024-05-09T04:38:02.204 INFO:teuthology.orchestra.run.smithi072.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-09T04:38:02.204 INFO:teuthology.orchestra.run.smithi072.stdout: automake-1.16.2-8.el9.noarch 2024-05-09T04:38:02.204 INFO:teuthology.orchestra.run.smithi072.stdout: bison-3.7.4-5.el9.x86_64 2024-05-09T04:38:02.204 INFO:teuthology.orchestra.run.smithi072.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-09T04:38:02.204 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.204 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-base-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.204 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.204 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.204 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-grafana-dashboards-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:02.204 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-immutable-object-cache-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.205 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mds-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.205 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.205 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:02.205 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-dashboard-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:02.205 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:02.205 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-modules-core-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:02.205 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mgr-rook-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:02.205 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-mon-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.205 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-osd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.205 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-prometheus-alerts-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:02.205 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-radosgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.205 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-selinux-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.205 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-test-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.206 INFO:teuthology.orchestra.run.smithi072.stdout: ceph-volume-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:02.206 INFO:teuthology.orchestra.run.smithi072.stdout: cephadm-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:02.206 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.206 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-shell-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.206 INFO:teuthology.orchestra.run.smithi072.stdout: cephfs-top-2:19.0.0-3594.gc9da4049.el9.noarch 2024-05-09T04:38:02.206 INFO:teuthology.orchestra.run.smithi072.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-09T04:38:02.206 INFO:teuthology.orchestra.run.smithi072.stdout: flex-2.6.4-9.el9.x86_64 2024-05-09T04:38:02.206 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-09T04:38:02.206 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-09T04:38:02.206 INFO:teuthology.orchestra.run.smithi072.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-09T04:38:02.206 INFO:teuthology.orchestra.run.smithi072.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-09T04:38:02.206 INFO:teuthology.orchestra.run.smithi072.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-09T04:38:02.206 INFO:teuthology.orchestra.run.smithi072.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-09T04:38:02.206 INFO:teuthology.orchestra.run.smithi072.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-09T04:38:02.207 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-09T04:38:02.207 INFO:teuthology.orchestra.run.smithi072.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-09T04:38:02.207 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.207 INFO:teuthology.orchestra.run.smithi072.stdout: libcephfs2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.207 INFO:teuthology.orchestra.run.smithi072.stdout: libcephsqlite-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.207 INFO:teuthology.orchestra.run.smithi072.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-09T04:38:02.207 INFO:teuthology.orchestra.run.smithi072.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-09T04:38:02.207 INFO:teuthology.orchestra.run.smithi072.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-09T04:38:02.207 INFO:teuthology.orchestra.run.smithi072.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-09T04:38:02.207 INFO:teuthology.orchestra.run.smithi072.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-09T04:38:02.207 INFO:teuthology.orchestra.run.smithi072.stdout: librados-devel-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.207 INFO:teuthology.orchestra.run.smithi072.stdout: libradosstriper1-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.207 INFO:teuthology.orchestra.run.smithi072.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-09T04:38:02.207 INFO:teuthology.orchestra.run.smithi072.stdout: librgw2-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.208 INFO:teuthology.orchestra.run.smithi072.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-09T04:38:02.208 INFO:teuthology.orchestra.run.smithi072.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-09T04:38:02.208 INFO:teuthology.orchestra.run.smithi072.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-09T04:38:02.208 INFO:teuthology.orchestra.run.smithi072.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-09T04:38:02.208 INFO:teuthology.orchestra.run.smithi072.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-09T04:38:02.208 INFO:teuthology.orchestra.run.smithi072.stdout: lua-5.4.4-4.el9.x86_64 2024-05-09T04:38:02.208 INFO:teuthology.orchestra.run.smithi072.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-09T04:38:02.208 INFO:teuthology.orchestra.run.smithi072.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-09T04:38:02.208 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-09T04:38:02.208 INFO:teuthology.orchestra.run.smithi072.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-09T04:38:02.208 INFO:teuthology.orchestra.run.smithi072.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-09T04:38:02.208 INFO:teuthology.orchestra.run.smithi072.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-09T04:38:02.208 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-13.14-1.el9.x86_64 2024-05-09T04:38:02.208 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-09T04:38:02.208 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-09T04:38:02.209 INFO:teuthology.orchestra.run.smithi072.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-09T04:38:02.209 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-09T04:38:02.209 INFO:teuthology.orchestra.run.smithi072.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-09T04:38:02.209 INFO:teuthology.orchestra.run.smithi072.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-09T04:38:02.209 INFO:teuthology.orchestra.run.smithi072.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-09T04:38:02.209 INFO:teuthology.orchestra.run.smithi072.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-09T04:38:02.209 INFO:teuthology.orchestra.run.smithi072.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-09T04:38:02.209 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-09T04:38:02.209 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-argparse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.209 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ceph-common-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.209 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cephfs-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.209 INFO:teuthology.orchestra.run.smithi072.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-09T04:38:02.209 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-09T04:38:02.210 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-09T04:38:02.210 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-09T04:38:02.210 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-09T04:38:02.210 INFO:teuthology.orchestra.run.smithi072.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-09T04:38:02.210 INFO:teuthology.orchestra.run.smithi072.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-09T04:38:02.210 INFO:teuthology.orchestra.run.smithi072.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-09T04:38:02.210 INFO:teuthology.orchestra.run.smithi072.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-09T04:38:02.210 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-09T04:38:02.210 INFO:teuthology.orchestra.run.smithi072.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-09T04:38:02.210 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-09T04:38:02.210 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-09T04:38:02.210 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-09T04:38:02.210 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-09T04:38:02.210 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-09T04:38:02.211 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-09T04:38:02.211 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-09T04:38:02.211 INFO:teuthology.orchestra.run.smithi072.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-09T04:38:02.211 INFO:teuthology.orchestra.run.smithi072.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-09T04:38:02.211 INFO:teuthology.orchestra.run.smithi072.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-09T04:38:02.211 INFO:teuthology.orchestra.run.smithi072.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-09T04:38:02.211 INFO:teuthology.orchestra.run.smithi072.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-09T04:38:02.211 INFO:teuthology.orchestra.run.smithi072.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-09T04:38:02.211 INFO:teuthology.orchestra.run.smithi072.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-09T04:38:02.211 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-09T04:38:02.211 INFO:teuthology.orchestra.run.smithi072.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-09T04:38:02.211 INFO:teuthology.orchestra.run.smithi072.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-09T04:38:02.211 INFO:teuthology.orchestra.run.smithi072.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-09T04:38:02.211 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-09T04:38:02.212 INFO:teuthology.orchestra.run.smithi072.stdout: python3-ply-3.11-14.el9.noarch 2024-05-09T04:38:02.212 INFO:teuthology.orchestra.run.smithi072.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-09T04:38:02.212 INFO:teuthology.orchestra.run.smithi072.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-09T04:38:02.212 INFO:teuthology.orchestra.run.smithi072.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-09T04:38:02.212 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-09T04:38:02.212 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-09T04:38:02.212 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-09T04:38:02.212 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-09T04:38:02.212 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-09T04:38:02.212 INFO:teuthology.orchestra.run.smithi072.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-09T04:38:02.212 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rados-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.212 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.212 INFO:teuthology.orchestra.run.smithi072.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-09T04:38:02.212 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-09T04:38:02.213 INFO:teuthology.orchestra.run.smithi072.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-09T04:38:02.213 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rgw-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.213 INFO:teuthology.orchestra.run.smithi072.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-09T04:38:02.213 INFO:teuthology.orchestra.run.smithi072.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-09T04:38:02.213 INFO:teuthology.orchestra.run.smithi072.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-09T04:38:02.213 INFO:teuthology.orchestra.run.smithi072.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-09T04:38:02.213 INFO:teuthology.orchestra.run.smithi072.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-09T04:38:02.213 INFO:teuthology.orchestra.run.smithi072.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-09T04:38:02.213 INFO:teuthology.orchestra.run.smithi072.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-09T04:38:02.213 INFO:teuthology.orchestra.run.smithi072.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-09T04:38:02.213 INFO:teuthology.orchestra.run.smithi072.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-09T04:38:02.213 INFO:teuthology.orchestra.run.smithi072.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-09T04:38:02.213 INFO:teuthology.orchestra.run.smithi072.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-09T04:38:02.213 INFO:teuthology.orchestra.run.smithi072.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-09T04:38:02.214 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-09T04:38:02.214 INFO:teuthology.orchestra.run.smithi072.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-09T04:38:02.214 INFO:teuthology.orchestra.run.smithi072.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-09T04:38:02.214 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-fuse-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.214 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-mirror-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.214 INFO:teuthology.orchestra.run.smithi072.stdout: rbd-nbd-2:19.0.0-3594.gc9da4049.el9.x86_64 2024-05-09T04:38:02.214 INFO:teuthology.orchestra.run.smithi072.stdout: re2-1:20211101-3.el9.x86_64 2024-05-09T04:38:02.214 INFO:teuthology.orchestra.run.smithi072.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-09T04:38:02.214 INFO:teuthology.orchestra.run.smithi072.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-09T04:38:02.214 INFO:teuthology.orchestra.run.smithi072.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-09T04:38:02.214 INFO:teuthology.orchestra.run.smithi072.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-09T04:38:02.214 INFO:teuthology.orchestra.run.smithi072.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-09T04:38:02.215 INFO:teuthology.orchestra.run.smithi072.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-09T04:38:02.215 INFO:teuthology.orchestra.run.smithi072.stdout: 2024-05-09T04:38:02.215 INFO:teuthology.orchestra.run.smithi072.stdout:Complete! 2024-05-09T04:38:02.742 DEBUG:teuthology.parallel:result is None 2024-05-09T04:38:02.743 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:38:03.099 DEBUG:teuthology.orchestra.run.smithi046:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T04:38:03.127 INFO:teuthology.orchestra.run.smithi046.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T04:38:03.127 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T04:38:03.127 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T04:38:03.129 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:38:03.283 DEBUG:teuthology.orchestra.run.smithi072:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T04:38:03.308 INFO:teuthology.orchestra.run.smithi072.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T04:38:03.308 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T04:38:03.308 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T04:38:03.310 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:38:03.662 DEBUG:teuthology.orchestra.run.smithi110:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-09T04:38:03.687 INFO:teuthology.orchestra.run.smithi110.stdout:19.0.0-3594.gc9da4049.el9 2024-05-09T04:38:03.688 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3594.gc9da4049.el9 2024-05-09T04:38:03.688 INFO:teuthology.task.install:The correct ceph version 19.0.0-3594.gc9da4049 is installed. 2024-05-09T04:38:03.690 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-09T04:38:03.691 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:38:03.691 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T04:38:03.724 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-09T04:38:03.725 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T04:38:03.755 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-09T04:38:03.755 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-09T04:38:03.785 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-09T04:38:03.785 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:38:03.785 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T04:38:03.816 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T04:38:03.885 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-09T04:38:03.886 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T04:38:03.914 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T04:38:03.981 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-09T04:38:03.981 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/daemon-helper 2024-05-09T04:38:04.009 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-09T04:38:04.076 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-09T04:38:04.077 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:38:04.077 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T04:38:04.103 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T04:38:04.169 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-09T04:38:04.170 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T04:38:04.197 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T04:38:04.264 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-09T04:38:04.264 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-09T04:38:04.290 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-09T04:38:04.358 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-09T04:38:04.358 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:38:04.359 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T04:38:04.387 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T04:38:04.454 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-09T04:38:04.454 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T04:38:04.481 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T04:38:04.547 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-09T04:38:04.547 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/usr/bin/stdin-killer 2024-05-09T04:38:04.575 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-09T04:38:04.643 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-09T04:38:04.753 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'subvols': {'create': 2}} 2024-05-09T04:38:04.753 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:38:04.754 INFO:tasks.cephadm:Cluster fsid is ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:38:04.754 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-09T04:38:04.754 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.46', 'mon.b': '172.21.15.72', 'mon.c': '172.21.15.110'} 2024-05-09T04:38:04.754 INFO:tasks.cephadm:First mon is mon.a on smithi046 2024-05-09T04:38:04.754 INFO:tasks.cephadm:First mgr is x 2024-05-09T04:38:04.754 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-09T04:38:04.754 DEBUG:teuthology.orchestra.run.smithi046:> sudo hostname $(hostname -s) 2024-05-09T04:38:04.781 DEBUG:teuthology.orchestra.run.smithi072:> sudo hostname $(hostname -s) 2024-05-09T04:38:04.807 DEBUG:teuthology.orchestra.run.smithi110:> sudo hostname $(hostname -s) 2024-05-09T04:38:04.835 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-09T04:38:04.836 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:38:04.990 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79226/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3594-gc9da4049', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3594.gc9da4049'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-09 00:57:42.125322', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.183908-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': 'c9da4049f638c0f84bdd9b4e33c42e86046ac554', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79224/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3594-gc9da4049', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3594.gc9da4049'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-08 19:52:58.989794', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240508.183908-debug', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-09T04:38:05.219 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240508.183908-debug, sha1 c9da4049f638c0f84bdd9b4e33c42e86046ac554 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:38:05.221 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm 2024-05-09T04:38:05.221 DEBUG:teuthology.orchestra.run.smithi046:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:05.697 INFO:teuthology.orchestra.run.smithi046.stdout:-rw-r--r--. 1 ubuntu ubuntu 782864 May 9 04:38 /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:05.697 DEBUG:teuthology.orchestra.run.smithi072:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:06.158 INFO:teuthology.orchestra.run.smithi072.stdout:-rw-r--r--. 1 ubuntu ubuntu 782864 May 9 04:38 /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:06.159 DEBUG:teuthology.orchestra.run.smithi110:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240508.183908-debug/c9da4049f638c0f84bdd9b4e33c42e86046ac554/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:06.875 INFO:teuthology.orchestra.run.smithi110.stdout:-rw-r--r--. 1 ubuntu ubuntu 782864 May 9 04:38 /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:06.876 DEBUG:teuthology.orchestra.run.smithi046:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:06.896 DEBUG:teuthology.orchestra.run.smithi072:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:06.915 DEBUG:teuthology.orchestra.run.smithi110:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-09T04:38:06.950 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 on all hosts... 2024-05-09T04:38:06.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T04:38:06.953 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T04:38:06.958 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 pull 2024-05-09T04:38:08.561 INFO:teuthology.orchestra.run.smithi110.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T04:38:08.574 INFO:teuthology.orchestra.run.smithi046.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T04:38:09.750 INFO:teuthology.orchestra.run.smithi072.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T04:38:47.973 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-05-09T04:38:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T04:38:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T04:38:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "repo_digests": [ 2024-05-09T04:38:47.973 INFO:teuthology.orchestra.run.smithi072.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T04:38:47.974 INFO:teuthology.orchestra.run.smithi072.stdout: ] 2024-05-09T04:38:47.974 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-05-09T04:39:16.593 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:39:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T04:39:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T04:39:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "repo_digests": [ 2024-05-09T04:39:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T04:39:16.594 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T04:39:16.594 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:39:16.636 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-05-09T04:39:16.637 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_version": "ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev)", 2024-05-09T04:39:16.637 INFO:teuthology.orchestra.run.smithi110.stdout: "image_id": "db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7", 2024-05-09T04:39:16.637 INFO:teuthology.orchestra.run.smithi110.stdout: "repo_digests": [ 2024-05-09T04:39:16.637 INFO:teuthology.orchestra.run.smithi110.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584" 2024-05-09T04:39:16.637 INFO:teuthology.orchestra.run.smithi110.stdout: ] 2024-05-09T04:39:16.637 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-05-09T04:39:16.661 DEBUG:teuthology.orchestra.run.smithi046:> sudo mkdir -p /etc/ceph 2024-05-09T04:39:16.691 DEBUG:teuthology.orchestra.run.smithi072:> sudo mkdir -p /etc/ceph 2024-05-09T04:39:16.724 DEBUG:teuthology.orchestra.run.smithi110:> sudo mkdir -p /etc/ceph 2024-05-09T04:39:16.750 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 777 /etc/ceph 2024-05-09T04:39:16.777 DEBUG:teuthology.orchestra.run.smithi072:> sudo chmod 777 /etc/ceph 2024-05-09T04:39:16.804 DEBUG:teuthology.orchestra.run.smithi110:> sudo chmod 777 /etc/ceph 2024-05-09T04:39:16.828 INFO:tasks.cephadm:Writing seed config... 2024-05-09T04:39:16.829 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-09T04:39:16.829 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-09T04:39:16.829 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-09T04:39:16.829 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-09T04:39:16.829 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-09T04:39:16.829 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-09T04:39:16.830 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-09T04:39:16.830 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-09T04:39:16.830 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-09T04:39:16.830 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-09T04:39:16.830 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-09T04:39:16.830 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-09T04:39:16.830 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-09T04:39:16.830 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-09T04:39:16.830 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-09T04:39:16.830 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-09T04:39:16.830 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-09T04:39:16.830 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-05-09T04:39:16.830 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-09T04:39:16.831 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-09T04:39:16.831 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-09T04:39:16.831 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-09T04:39:16.831 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-09T04:39:16.831 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-09T04:39:16.831 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-09T04:39:16.831 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-09T04:39:16.831 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-09T04:39:16.831 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-09T04:39:16.831 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-09T04:39:16.831 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-09T04:39:16.831 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-09T04:39:16.831 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-09T04:39:16.831 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-09T04:39:16.832 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-09T04:39:16.832 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-09T04:39:16.832 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-09T04:39:16.832 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-09T04:39:16.832 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-09T04:39:16.832 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-09T04:39:16.832 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-09T04:39:16.832 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-09T04:39:16.832 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-09T04:39:16.832 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-09T04:39:16.832 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-09T04:39:16.832 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-09T04:39:16.832 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-09T04:39:16.832 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-09T04:39:16.833 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:39:16.833 DEBUG:teuthology.orchestra.run.smithi046:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-09T04:39:16.850 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ecf9ae2c-0dbd-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-09T04:39:16.851 DEBUG:teuthology.orchestra.run.smithi046:mon.a> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mon.a.service 2024-05-09T04:39:16.893 DEBUG:teuthology.orchestra.run.smithi046:mgr.x> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.x.service 2024-05-09T04:39:16.936 INFO:tasks.cephadm:Bootstrapping... 2024-05-09T04:39:16.936 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 -v bootstrap --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.46 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-09T04:39:17.134 INFO:teuthology.orchestra.run.smithi046.stdout:-------------------------------------------------------------------------------- 2024-05-09T04:39:17.134 INFO:teuthology.orchestra.run.smithi046.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554', '-v', 'bootstrap', '--fsid', 'ecf9ae2c-0dbd-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.46', '--skip-admin-label'] 2024-05-09T04:39:17.135 INFO:teuthology.orchestra.run.smithi046.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-09T04:39:17.135 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying podman|docker is present... 2024-05-09T04:39:17.161 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T04:39:17.161 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying lvm2 is present... 2024-05-09T04:39:17.162 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying time synchronization is in place... 2024-05-09T04:39:17.169 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T04:39:17.169 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T04:39:17.176 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T04:39:17.176 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-09T04:39:17.183 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-05-09T04:39:17.189 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-05-09T04:39:17.190 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-05-09T04:39:17.190 INFO:teuthology.orchestra.run.smithi046.stdout:Repeating the final host check... 2024-05-09T04:39:17.235 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-09T04:39:17.235 INFO:teuthology.orchestra.run.smithi046.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-09T04:39:17.235 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl is present 2024-05-09T04:39:17.235 INFO:teuthology.orchestra.run.smithi046.stdout:lvcreate is present 2024-05-09T04:39:17.242 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-09T04:39:17.242 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-09T04:39:17.248 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-09T04:39:17.248 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-09T04:39:17.255 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout enabled 2024-05-09T04:39:17.262 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout active 2024-05-09T04:39:17.262 INFO:teuthology.orchestra.run.smithi046.stdout:Unit chronyd.service is enabled and running 2024-05-09T04:39:17.262 INFO:teuthology.orchestra.run.smithi046.stdout:Host looks OK 2024-05-09T04:39:17.262 INFO:teuthology.orchestra.run.smithi046.stdout:Cluster fsid: ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:17.262 INFO:teuthology.orchestra.run.smithi046.stdout:Acquiring lock 139819426991840 on /run/cephadm/ecf9ae2c-0dbd-11ef-bc97-c7b262605968.lock 2024-05-09T04:39:17.262 INFO:teuthology.orchestra.run.smithi046.stdout:Lock 139819426991840 acquired on /run/cephadm/ecf9ae2c-0dbd-11ef-bc97-c7b262605968.lock 2024-05-09T04:39:17.263 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 3300 ... 2024-05-09T04:39:17.263 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying IP 172.21.15.46 port 6789 ... 2024-05-09T04:39:17.263 INFO:teuthology.orchestra.run.smithi046.stdout:Base mon IP(s) is [172.21.15.46:3300, 172.21.15.46:6789], mon addrv is [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-05-09T04:39:17.266 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.46 metric 100 2024-05-09T04:39:17.267 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.46 metric 100 2024-05-09T04:39:17.269 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-09T04:39:17.269 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-09T04:39:17.269 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-09T04:39:17.272 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-09T04:39:17.272 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-09T04:39:17.272 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T04:39:17.272 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-09T04:39:17.272 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-05-09T04:39:17.272 INFO:teuthology.orchestra.run.smithi046.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-09T04:39:17.273 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-05-09T04:39:17.273 INFO:teuthology.orchestra.run.smithi046.stdout:Mon IP `172.21.15.46` is in CIDR network `172.21.0.0/20` 2024-05-09T04:39:17.273 INFO:teuthology.orchestra.run.smithi046.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-09T04:39:17.273 INFO:teuthology.orchestra.run.smithi046.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-09T04:39:17.274 INFO:teuthology.orchestra.run.smithi046.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T04:39:17.764 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stdout db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 2024-05-09T04:39:17.764 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554... 2024-05-09T04:39:17.764 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Getting image source signatures 2024-05-09T04:39:17.764 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:0411e0fd7a5d45f8ab31b23cec89f97c70494de39461f644c7cdc77de3b9cbca 2024-05-09T04:39:17.764 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-09T04:39:17.764 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Copying config sha256:db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 2024-05-09T04:39:17.764 INFO:teuthology.orchestra.run.smithi046.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-09T04:39:18.593 INFO:teuthology.orchestra.run.smithi046.stdout:ceph: stdout ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T04:39:18.594 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph version: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev) 2024-05-09T04:39:18.594 INFO:teuthology.orchestra.run.smithi046.stdout:Extracting ceph user uid/gid from container image... 2024-05-09T04:39:19.653 INFO:teuthology.orchestra.run.smithi046.stdout:stat: stdout 167 167 2024-05-09T04:39:19.653 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial keys... 2024-05-09T04:39:20.567 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQB4UzxmVPs6ABAAHUvDi+KQExZbiRcnMVLTxA== 2024-05-09T04:39:21.514 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQB4UzxmcECXNBAAbNb1me57LXqEqN5P3SZdUg== 2024-05-09T04:39:22.411 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph-authtool: stdout AQB5UzxmtOCLMxAAltnP25Is5sUd8N0OsKkW2w== 2024-05-09T04:39:22.412 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial monmap... 2024-05-09T04:39:23.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T04:39:23.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-09T04:39:23.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:23.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T04:39:23.268 INFO:teuthology.orchestra.run.smithi046.stdout:monmaptool for a [v2:172.21.15.46:3300,v1:172.21.15.46:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-09T04:39:23.268 INFO:teuthology.orchestra.run.smithi046.stdout:setting min_mon_release = quincy 2024-05-09T04:39:23.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: set fsid to ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:23.268 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-09T04:39:23.268 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:39:23.268 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mon... 2024-05-09T04:39:24.115 INFO:teuthology.orchestra.run.smithi046.stdout:create mon.a on 2024-05-09T04:39:24.325 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-09T04:39:24.470 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-09T04:39:24.657 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968.target. 2024-05-09T04:39:24.658 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968.target. 2024-05-09T04:39:24.891 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mon.a 2024-05-09T04:39:24.891 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-09T04:39:25.125 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968.target.wants/ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@.service. 2024-05-09T04:39:25.177 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:25 smithi046 systemd[1]: Starting Ceph mon.a for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:39:25.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:25 smithi046 podman[29736]: 2024-05-09 04:39:25.365849579 +0000 UTC m=+0.111507355 container create b6c116744b46ec481e1df56b2a079c5ea79a9e3429d188c19998e3297dbdd967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-09T04:39:25.474 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:25 smithi046 podman[29736]: 2024-05-09 04:39:25.272848085 +0000 UTC m=+0.018505875 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:39:25.649 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T04:39:25.650 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-05-09T04:39:25.657 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T04:39:25.657 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-09T04:39:25.657 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-05-09T04:39:25.657 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T04:39:25.658 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon to start... 2024-05-09T04:39:25.658 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mon... 2024-05-09T04:39:25.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:25 smithi046 podman[29736]: 2024-05-09 04:39:25.57652639 +0000 UTC m=+0.322184177 container init b6c116744b46ec481e1df56b2a079c5ea79a9e3429d188c19998e3297dbdd967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T04:39:25.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:25 smithi046 podman[29736]: 2024-05-09 04:39:25.580541715 +0000 UTC m=+0.326199494 container start b6c116744b46ec481e1df56b2a079c5ea79a9e3429d188c19998e3297dbdd967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-05-09T04:39:25.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:25 smithi046 bash[29736]: b6c116744b46ec481e1df56b2a079c5ea79a9e3429d188c19998e3297dbdd967 2024-05-09T04:39:25.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:25 smithi046 ceph-mon[29761]: mkfs ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:25.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:25 smithi046 systemd[1]: Started Ceph mon.a for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:39:25.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:25 smithi046 ceph-mon[29761]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T04:39:26.682 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:26 smithi046 ceph-mon[29761]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T04:39:26.682 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:26 smithi046 ceph-mon[29761]: monmap epoch 1 2024-05-09T04:39:26.682 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:26 smithi046 ceph-mon[29761]: fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:26.682 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:26 smithi046 ceph-mon[29761]: last_changed 2024-05-09T04:39:22.726361+0000 2024-05-09T04:39:26.682 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:26 smithi046 ceph-mon[29761]: created 2024-05-09T04:39:22.726361+0000 2024-05-09T04:39:26.682 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:26 smithi046 ceph-mon[29761]: min_mon_release 19 (squid) 2024-05-09T04:39:26.683 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:26 smithi046 ceph-mon[29761]: election_strategy: 1 2024-05-09T04:39:26.683 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:26 smithi046 ceph-mon[29761]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T04:39:26.683 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:26 smithi046 ceph-mon[29761]: fsmap 2024-05-09T04:39:26.683 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:26 smithi046 ceph-mon[29761]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T04:39:26.683 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:26 smithi046 ceph-mon[29761]: mgrmap e1: no daemons active 2024-05-09T04:39:26.683 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:26 smithi046 ceph-mon[29761]: from='client.? 172.21.15.46:0/1892101732' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-09T04:39:26.684 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout cluster: 2024-05-09T04:39:26.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout id: ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:26.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-09T04:39:26.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:26.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout services: 2024-05-09T04:39:26.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.509206s) 2024-05-09T04:39:26.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-09T04:39:26.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-09T04:39:26.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:26.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout data: 2024-05-09T04:39:26.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-09T04:39:26.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-09T04:39:26.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-09T04:39:26.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout pgs: 2024-05-09T04:39:26.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:26.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.059+0000 7f84d58be700 1 Processor -- start 2024-05-09T04:39:26.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.059+0000 7f84d58be700 1 -- start start 2024-05-09T04:39:26.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.059+0000 7f84d58be700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d007cce0 0x7f84d010c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:26.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.059+0000 7f84d58be700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84d010ccd0 con 0x7f84d007cce0 2024-05-09T04:39:26.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.060+0000 7f84ceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d007cce0 0x7f84d010c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:26.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.060+0000 7f84ceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d007cce0 0x7f84d010c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35878/0 (socket says 172.21.15.46:35878) 2024-05-09T04:39:26.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.060+0000 7f84ceffd700 1 -- 172.21.15.46:0/3640233001 learned_addr learned my addr 172.21.15.46:0/3640233001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.060+0000 7f84ceffd700 1 -- 172.21.15.46:0/3640233001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d010ce50 con 0x7f84d007cce0 2024-05-09T04:39:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.060+0000 7f84ceffd700 1 --2- 172.21.15.46:0/3640233001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d007cce0 0x7f84d010c6c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f84c00098a0 tx=0x7f84c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1700f4a42376fca7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.060+0000 7f84cdffb700 1 -- 172.21.15.46:0/3640233001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84c0004160 con 0x7f84d007cce0 2024-05-09T04:39:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.060+0000 7f84cdffb700 1 -- 172.21.15.46:0/3640233001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c0004300 con 0x7f84d007cce0 2024-05-09T04:39:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.060+0000 7f84cdffb700 1 -- 172.21.15.46:0/3640233001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84c00036a0 con 0x7f84d007cce0 2024-05-09T04:39:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.061+0000 7f84d58be700 1 -- 172.21.15.46:0/3640233001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d007cce0 msgr2=0x7f84d010c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.061+0000 7f84d58be700 1 --2- 172.21.15.46:0/3640233001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d007cce0 0x7f84d010c6c0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f84c00098a0 tx=0x7f84c0009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.061+0000 7f84d58be700 1 -- 172.21.15.46:0/3640233001 shutdown_connections 2024-05-09T04:39:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.061+0000 7f84d58be700 1 --2- 172.21.15.46:0/3640233001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d007cce0 0x7f84d010c6c0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.061+0000 7f84d58be700 1 -- 172.21.15.46:0/3640233001 >> 172.21.15.46:0/3640233001 conn(0x7f84d01040f0 msgr2=0x7f84d0106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.061+0000 7f84d58be700 1 -- 172.21.15.46:0/3640233001 shutdown_connections 2024-05-09T04:39:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.061+0000 7f84d58be700 1 -- 172.21.15.46:0/3640233001 wait complete. 2024-05-09T04:39:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.062+0000 7f84d58be700 1 Processor -- start 2024-05-09T04:39:26.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.062+0000 7f84d58be700 1 -- start start 2024-05-09T04:39:26.693 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.063+0000 7f84d58be700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d007cce0 0x7f84d019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:26.693 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.063+0000 7f84d58be700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84d010d380 con 0x7f84d007cce0 2024-05-09T04:39:26.693 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.063+0000 7f84ceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d007cce0 0x7f84d019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:26.693 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.063+0000 7f84ceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d007cce0 0x7f84d019d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35884/0 (socket says 172.21.15.46:35884) 2024-05-09T04:39:26.693 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.063+0000 7f84ceffd700 1 -- 172.21.15.46:0/1892101732 learned_addr learned my addr 172.21.15.46:0/1892101732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:26.693 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.063+0000 7f84ceffd700 1 -- 172.21.15.46:0/1892101732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d019dbb0 con 0x7f84d007cce0 2024-05-09T04:39:26.693 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.063+0000 7f84ceffd700 1 --2- 172.21.15.46:0/1892101732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d007cce0 0x7f84d019d670 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f84c0000c00 tx=0x7f84c0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:26.693 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.064+0000 7f84d48bc700 1 -- 172.21.15.46:0/1892101732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84c0003db0 con 0x7f84d007cce0 2024-05-09T04:39:26.693 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.064+0000 7f84d48bc700 1 -- 172.21.15.46:0/1892101732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f84c0011630 con 0x7f84d007cce0 2024-05-09T04:39:26.693 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.064+0000 7f84d58be700 1 -- 172.21.15.46:0/1892101732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d019de40 con 0x7f84d007cce0 2024-05-09T04:39:26.693 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.064+0000 7f84d48bc700 1 -- 172.21.15.46:0/1892101732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f84c0011950 con 0x7f84d007cce0 2024-05-09T04:39:26.693 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.064+0000 7f84d58be700 1 -- 172.21.15.46:0/1892101732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d01a0a50 con 0x7f84d007cce0 2024-05-09T04:39:26.693 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.064+0000 7f84d48bc700 1 -- 172.21.15.46:0/1892101732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f84c0011b80 con 0x7f84d007cce0 2024-05-09T04:39:26.693 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.064+0000 7f84d48bc700 1 -- 172.21.15.46:0/1892101732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f84c001c450 con 0x7f84d007cce0 2024-05-09T04:39:26.694 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.065+0000 7f84d58be700 1 -- 172.21.15.46:0/1892101732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d0064e20 con 0x7f84d007cce0 2024-05-09T04:39:26.694 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.067+0000 7f84d48bc700 1 -- 172.21.15.46:0/1892101732 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f84c0032020 con 0x7f84d007cce0 2024-05-09T04:39:26.694 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.141+0000 7f84d58be700 1 -- 172.21.15.46:0/1892101732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7f84d002e8f0 con 0x7f84d007cce0 2024-05-09T04:39:26.694 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.141+0000 7f84d48bc700 1 -- 172.21.15.46:0/1892101732 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7f84c0004260 con 0x7f84d007cce0 2024-05-09T04:39:26.694 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.142+0000 7f84d58be700 1 -- 172.21.15.46:0/1892101732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d007cce0 msgr2=0x7f84d019d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:26.694 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.142+0000 7f84d58be700 1 --2- 172.21.15.46:0/1892101732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d007cce0 0x7f84d019d670 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f84c0000c00 tx=0x7f84c0004750 comp rx=0 tx=0).stop 2024-05-09T04:39:26.694 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.142+0000 7f84d58be700 1 -- 172.21.15.46:0/1892101732 shutdown_connections 2024-05-09T04:39:26.694 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.142+0000 7f84d58be700 1 --2- 172.21.15.46:0/1892101732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84d007cce0 0x7f84d019d670 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:26.694 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.142+0000 7f84d58be700 1 -- 172.21.15.46:0/1892101732 >> 172.21.15.46:0/1892101732 conn(0x7f84d01040f0 msgr2=0x7f84d01063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:26.694 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.142+0000 7f84d58be700 1 -- 172.21.15.46:0/1892101732 shutdown_connections 2024-05-09T04:39:26.694 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:26.142+0000 7f84d58be700 1 -- 172.21.15.46:0/1892101732 wait complete. 2024-05-09T04:39:26.694 INFO:teuthology.orchestra.run.smithi046.stdout:mon is available 2024-05-09T04:39:26.694 INFO:teuthology.orchestra.run.smithi046.stdout:Assimilating anything we can from ceph.conf... 2024-05-09T04:39:27.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:27 smithi046 ceph-mon[29761]: from='client.? 172.21.15.46:0/1686614685' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T04:39:27.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:27 smithi046 ceph-mon[29761]: from='client.? 172.21.15.46:0/1686614685' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-09T04:39:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:27.914 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T04:39:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-09T04:39:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.46:3300,v1:172.21.15.46:6789] 2024-05-09T04:39:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T04:39:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T04:39:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T04:39:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T04:39:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T04:39:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T04:39:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T04:39:27.915 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T04:39:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T04:39:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T04:39:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T04:39:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T04:39:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.141+0000 7f458c61c700 1 Processor -- start 2024-05-09T04:39:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.141+0000 7f458c61c700 1 -- start start 2024-05-09T04:39:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.141+0000 7f458c61c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f458407ccc0 0x7f458410c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.141+0000 7f458c61c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f458410ccd0 con 0x7f458407ccc0 2024-05-09T04:39:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.142+0000 7f458a3b8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f458407ccc0 0x7f458410c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.142+0000 7f458a3b8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f458407ccc0 0x7f458410c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35892/0 (socket says 172.21.15.46:35892) 2024-05-09T04:39:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.142+0000 7f458a3b8700 1 -- 172.21.15.46:0/2892599448 learned_addr learned my addr 172.21.15.46:0/2892599448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.142+0000 7f458a3b8700 1 -- 172.21.15.46:0/2892599448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458410ce50 con 0x7f458407ccc0 2024-05-09T04:39:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.142+0000 7f458a3b8700 1 --2- 172.21.15.46:0/2892599448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f458407ccc0 0x7f458410c6e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f45800098a0 tx=0x7f4580009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb2a0e0c1f5b424a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:27.916 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.142+0000 7f45893b6700 1 -- 172.21.15.46:0/2892599448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4580004160 con 0x7f458407ccc0 2024-05-09T04:39:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.142+0000 7f45893b6700 1 -- 172.21.15.46:0/2892599448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4580004300 con 0x7f458407ccc0 2024-05-09T04:39:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.142+0000 7f45893b6700 1 -- 172.21.15.46:0/2892599448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45800036a0 con 0x7f458407ccc0 2024-05-09T04:39:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.143+0000 7f458c61c700 1 -- 172.21.15.46:0/2892599448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f458407ccc0 msgr2=0x7f458410c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.143+0000 7f458c61c700 1 --2- 172.21.15.46:0/2892599448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f458407ccc0 0x7f458410c6e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f45800098a0 tx=0x7f4580009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.143+0000 7f458c61c700 1 -- 172.21.15.46:0/2892599448 shutdown_connections 2024-05-09T04:39:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.143+0000 7f458c61c700 1 --2- 172.21.15.46:0/2892599448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f458407ccc0 0x7f458410c6e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.143+0000 7f458c61c700 1 -- 172.21.15.46:0/2892599448 >> 172.21.15.46:0/2892599448 conn(0x7f45841040f0 msgr2=0x7f4584106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.143+0000 7f458c61c700 1 -- 172.21.15.46:0/2892599448 shutdown_connections 2024-05-09T04:39:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.143+0000 7f458c61c700 1 -- 172.21.15.46:0/2892599448 wait complete. 2024-05-09T04:39:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.144+0000 7f458c61c700 1 Processor -- start 2024-05-09T04:39:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.144+0000 7f458c61c700 1 -- start start 2024-05-09T04:39:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.144+0000 7f458c61c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f458407ccc0 0x7f458419d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.144+0000 7f458c61c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f458410d380 con 0x7f458407ccc0 2024-05-09T04:39:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.144+0000 7f458a3b8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f458407ccc0 0x7f458419d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:27.917 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.144+0000 7f458a3b8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f458407ccc0 0x7f458419d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35900/0 (socket says 172.21.15.46:35900) 2024-05-09T04:39:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.144+0000 7f458a3b8700 1 -- 172.21.15.46:0/1686614685 learned_addr learned my addr 172.21.15.46:0/1686614685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.145+0000 7f458a3b8700 1 -- 172.21.15.46:0/1686614685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458419dbd0 con 0x7f458407ccc0 2024-05-09T04:39:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.145+0000 7f458a3b8700 1 --2- 172.21.15.46:0/1686614685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f458407ccc0 0x7f458419d690 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4580000c00 tx=0x7f4580004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.145+0000 7f457b7fe700 1 -- 172.21.15.46:0/1686614685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4580003db0 con 0x7f458407ccc0 2024-05-09T04:39:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.145+0000 7f457b7fe700 1 -- 172.21.15.46:0/1686614685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4580011630 con 0x7f458407ccc0 2024-05-09T04:39:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.145+0000 7f458c61c700 1 -- 172.21.15.46:0/1686614685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458419de60 con 0x7f458407ccc0 2024-05-09T04:39:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.145+0000 7f457b7fe700 1 -- 172.21.15.46:0/1686614685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4580011950 con 0x7f458407ccc0 2024-05-09T04:39:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.145+0000 7f458c61c700 1 -- 172.21.15.46:0/1686614685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45841a0a70 con 0x7f458407ccc0 2024-05-09T04:39:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.145+0000 7f457b7fe700 1 -- 172.21.15.46:0/1686614685 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4580011af0 con 0x7f458407ccc0 2024-05-09T04:39:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.145+0000 7f457b7fe700 1 -- 172.21.15.46:0/1686614685 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f458001c450 con 0x7f458407ccc0 2024-05-09T04:39:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.146+0000 7f458c61c700 1 -- 172.21.15.46:0/1686614685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4584064e20 con 0x7f458407ccc0 2024-05-09T04:39:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.148+0000 7f457b7fe700 1 -- 172.21.15.46:0/1686614685 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f4580032020 con 0x7f458407ccc0 2024-05-09T04:39:27.918 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.220+0000 7f458c61c700 1 -- 172.21.15.46:0/1686614685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f458402e8f0 con 0x7f458407ccc0 2024-05-09T04:39:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.224+0000 7f457b7fe700 1 -- 172.21.15.46:0/1686614685 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7f4580004260 con 0x7f458407ccc0 2024-05-09T04:39:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.224+0000 7f458c61c700 1 -- 172.21.15.46:0/1686614685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f458407ccc0 msgr2=0x7f458419d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.224+0000 7f458c61c700 1 --2- 172.21.15.46:0/1686614685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f458407ccc0 0x7f458419d690 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f4580000c00 tx=0x7f4580004750 comp rx=0 tx=0).stop 2024-05-09T04:39:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.224+0000 7f458c61c700 1 -- 172.21.15.46:0/1686614685 shutdown_connections 2024-05-09T04:39:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.224+0000 7f458c61c700 1 --2- 172.21.15.46:0/1686614685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f458407ccc0 0x7f458419d690 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.225+0000 7f458c61c700 1 -- 172.21.15.46:0/1686614685 >> 172.21.15.46:0/1686614685 conn(0x7f45841040f0 msgr2=0x7f4584106350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.225+0000 7f458c61c700 1 -- 172.21.15.46:0/1686614685 shutdown_connections 2024-05-09T04:39:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:27.225+0000 7f458c61c700 1 -- 172.21.15.46:0/1686614685 wait complete. 2024-05-09T04:39:27.919 INFO:teuthology.orchestra.run.smithi046.stdout:Generating new minimal ceph.conf... 2024-05-09T04:39:28.969 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:28 smithi046 ceph-mon[29761]: from='client.? 172.21.15.46:0/2084916700' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"}]: dispatch 2024-05-09T04:39:28.976 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.355+0000 7f72443dc700 1 Processor -- start 2024-05-09T04:39:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.355+0000 7f72443dc700 1 -- start start 2024-05-09T04:39:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.355+0000 7f72443dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c108a90 0x7f723c108ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.355+0000 7f72443dc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f723c109490 con 0x7f723c108a90 2024-05-09T04:39:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.355+0000 7f7242178700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c108a90 0x7f723c108ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.355+0000 7f7242178700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c108a90 0x7f723c108ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35910/0 (socket says 172.21.15.46:35910) 2024-05-09T04:39:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.355+0000 7f7242178700 1 -- 172.21.15.46:0/4046931183 learned_addr learned my addr 172.21.15.46:0/4046931183 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.356+0000 7f7242178700 1 -- 172.21.15.46:0/4046931183 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c109bc0 con 0x7f723c108a90 2024-05-09T04:39:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.356+0000 7f7242178700 1 --2- 172.21.15.46:0/4046931183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c108a90 0x7f723c108ea0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f72380098a0 tx=0x7f7238009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3710aa76f20ce72e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.356+0000 7f7241176700 1 -- 172.21.15.46:0/4046931183 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7238004160 con 0x7f723c108a90 2024-05-09T04:39:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.356+0000 7f7241176700 1 -- 172.21.15.46:0/4046931183 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7238004300 con 0x7f723c108a90 2024-05-09T04:39:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.356+0000 7f7241176700 1 -- 172.21.15.46:0/4046931183 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7238011af0 con 0x7f723c108a90 2024-05-09T04:39:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.357+0000 7f72443dc700 1 -- 172.21.15.46:0/4046931183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c108a90 msgr2=0x7f723c108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:28.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.357+0000 7f72443dc700 1 --2- 172.21.15.46:0/4046931183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c108a90 0x7f723c108ea0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f72380098a0 tx=0x7f7238009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.357+0000 7f72443dc700 1 -- 172.21.15.46:0/4046931183 shutdown_connections 2024-05-09T04:39:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.357+0000 7f72443dc700 1 --2- 172.21.15.46:0/4046931183 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c108a90 0x7f723c108ea0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.357+0000 7f72443dc700 1 -- 172.21.15.46:0/4046931183 >> 172.21.15.46:0/4046931183 conn(0x7f723c1040f0 msgr2=0x7f723c106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.357+0000 7f72443dc700 1 -- 172.21.15.46:0/4046931183 shutdown_connections 2024-05-09T04:39:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.357+0000 7f72443dc700 1 -- 172.21.15.46:0/4046931183 wait complete. 2024-05-09T04:39:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.358+0000 7f72443dc700 1 Processor -- start 2024-05-09T04:39:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.358+0000 7f72443dc700 1 -- start start 2024-05-09T04:39:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.358+0000 7f72443dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c108a90 0x7f723c19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.358+0000 7f72443dc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f723c07c160 con 0x7f723c108a90 2024-05-09T04:39:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.358+0000 7f7242178700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c108a90 0x7f723c19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.358+0000 7f7242178700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c108a90 0x7f723c19bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35920/0 (socket says 172.21.15.46:35920) 2024-05-09T04:39:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.358+0000 7f7242178700 1 -- 172.21.15.46:0/2084916700 learned_addr learned my addr 172.21.15.46:0/2084916700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.358+0000 7f7242178700 1 -- 172.21.15.46:0/2084916700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f723c19c2b0 con 0x7f723c108a90 2024-05-09T04:39:28.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.359+0000 7f7242178700 1 --2- 172.21.15.46:0/2084916700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c108a90 0x7f723c19bd70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7238009fb0 tx=0x7f7238011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:28.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.359+0000 7f72337fe700 1 -- 172.21.15.46:0/2084916700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72380036a0 con 0x7f723c108a90 2024-05-09T04:39:28.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.359+0000 7f72337fe700 1 -- 172.21.15.46:0/2084916700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f7238024440 con 0x7f723c108a90 2024-05-09T04:39:28.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.359+0000 7f72443dc700 1 -- 172.21.15.46:0/2084916700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f723c19c540 con 0x7f723c108a90 2024-05-09T04:39:28.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.359+0000 7f72337fe700 1 -- 172.21.15.46:0/2084916700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f723801a5c0 con 0x7f723c108a90 2024-05-09T04:39:28.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.359+0000 7f72443dc700 1 -- 172.21.15.46:0/2084916700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f723c19f150 con 0x7f723c108a90 2024-05-09T04:39:28.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.359+0000 7f72337fe700 1 -- 172.21.15.46:0/2084916700 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f723801abc0 con 0x7f723c108a90 2024-05-09T04:39:28.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.359+0000 7f72337fe700 1 -- 172.21.15.46:0/2084916700 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7238019700 con 0x7f723c108a90 2024-05-09T04:39:28.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.360+0000 7f72443dc700 1 -- 172.21.15.46:0/2084916700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f723c064e20 con 0x7f723c108a90 2024-05-09T04:39:28.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.362+0000 7f72337fe700 1 -- 172.21.15.46:0/2084916700 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f72380245f0 con 0x7f723c108a90 2024-05-09T04:39:28.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.433+0000 7f72443dc700 1 -- 172.21.15.46:0/2084916700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"} v 0) -- 0x7f723c194f80 con 0x7f723c108a90 2024-05-09T04:39:28.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.433+0000 7f72337fe700 1 -- 172.21.15.46:0/2084916700 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf", "output-file": "/var/lib/ceph/mon/ceph-a/config"}]=0 v2) ==== 126+0+175 (secure 0 0 0) 0x7f723802dce0 con 0x7f723c108a90 2024-05-09T04:39:28.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.434+0000 7f72443dc700 1 -- 172.21.15.46:0/2084916700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c108a90 msgr2=0x7f723c19bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:28.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.434+0000 7f72443dc700 1 --2- 172.21.15.46:0/2084916700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c108a90 0x7f723c19bd70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7238009fb0 tx=0x7f7238011890 comp rx=0 tx=0).stop 2024-05-09T04:39:28.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.434+0000 7f72443dc700 1 -- 172.21.15.46:0/2084916700 shutdown_connections 2024-05-09T04:39:28.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.434+0000 7f72443dc700 1 --2- 172.21.15.46:0/2084916700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f723c108a90 0x7f723c19bd70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:28.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.434+0000 7f72443dc700 1 -- 172.21.15.46:0/2084916700 >> 172.21.15.46:0/2084916700 conn(0x7f723c1040f0 msgr2=0x7f723c192040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:28.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.434+0000 7f72443dc700 1 -- 172.21.15.46:0/2084916700 shutdown_connections 2024-05-09T04:39:28.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:28.434+0000 7f72443dc700 1 -- 172.21.15.46:0/2084916700 wait complete. 2024-05-09T04:39:28.980 INFO:teuthology.orchestra.run.smithi046.stdout:Restarting the monitor... 2024-05-09T04:39:29.248 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:28 smithi046 systemd[1]: Stopping Ceph mon.a for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:39:29.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:29 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a[29750]: 2024-05-09T04:39:29.248+0000 7f8fd793e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T04:39:29.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:29 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a[29750]: 2024-05-09T04:39:29.248+0000 7f8fd793e700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-09T04:39:29.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:29 smithi046 podman[30010]: 2024-05-09 04:39:29.412805789 +0000 UTC m=+0.271157461 container died b6c116744b46ec481e1df56b2a079c5ea79a9e3429d188c19998e3297dbdd967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-09T04:39:30.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:29 smithi046 podman[30010]: 2024-05-09 04:39:29.79007673 +0000 UTC m=+0.648428402 container cleanup b6c116744b46ec481e1df56b2a079c5ea79a9e3429d188c19998e3297dbdd967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T04:39:30.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:29 smithi046 bash[30010]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a 2024-05-09T04:39:30.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 podman[30024]: 2024-05-09 04:39:30.166019226 +0000 UTC m=+0.749184345 container remove b6c116744b46ec481e1df56b2a079c5ea79a9e3429d188c19998e3297dbdd967 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-09T04:39:30.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 systemd[1]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-09T04:39:30.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 systemd[1]: Stopped Ceph mon.a for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:39:30.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 systemd[1]: Starting Ceph mon.a for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:39:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 podman[30117]: 2024-05-09 04:39:30.591900938 +0000 UTC m=+0.100356101 container create 0809ad9b5ec45012eb937b11f1b3003ca18bd02884a956db0d2e187377516c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T04:39:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 podman[30117]: 2024-05-09 04:39:30.509703321 +0000 UTC m=+0.018158504 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:39:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 podman[30117]: 2024-05-09 04:39:30.777592793 +0000 UTC m=+0.286047959 container init 0809ad9b5ec45012eb937b11f1b3003ca18bd02884a956db0d2e187377516c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T04:39:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 podman[30117]: 2024-05-09 04:39:30.781078106 +0000 UTC m=+0.289533275 container start 0809ad9b5ec45012eb937b11f1b3003ca18bd02884a956db0d2e187377516c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T04:39:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T04:39:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev), process ceph-mon, pid 2 2024-05-09T04:39:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: pidfile_write: ignore empty --pid-file 2024-05-09T04:39:30.860 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: load: jerasure load: lrc 2024-05-09T04:39:30.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: RocksDB version: 7.9.2 2024-05-09T04:39:30.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Git sha 0 2024-05-09T04:39:30.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Compile date 2024-05-08 19:33:14 2024-05-09T04:39:30.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: DB SUMMARY 2024-05-09T04:39:30.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: DB Session ID: R2S1JWZLAW97M40ZBRZB 2024-05-09T04:39:30.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: CURRENT file: CURRENT 2024-05-09T04:39:30.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T04:39:30.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-09T04:39:30.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-09T04:39:30.861 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105270 ; 2024-05-09T04:39:30.862 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.error_if_exists: 0 2024-05-09T04:39:30.862 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.create_if_missing: 0 2024-05-09T04:39:30.862 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.paranoid_checks: 1 2024-05-09T04:39:30.862 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T04:39:30.862 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T04:39:30.862 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T04:39:30.863 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.env: 0x564ec0d91500 2024-05-09T04:39:30.863 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T04:39:30.863 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.info_log: 0x564ec2eee020 2024-05-09T04:39:30.863 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T04:39:30.863 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.statistics: (nil) 2024-05-09T04:39:30.863 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.use_fsync: 0 2024-05-09T04:39:30.863 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_log_file_size: 0 2024-05-09T04:39:30.863 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T04:39:30.863 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T04:39:30.863 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T04:39:30.863 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T04:39:30.863 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.allow_fallocate: 1 2024-05-09T04:39:30.863 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T04:39:30.863 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T04:39:30.864 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.use_direct_reads: 0 2024-05-09T04:39:30.864 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T04:39:30.864 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T04:39:30.864 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.db_log_dir: 2024-05-09T04:39:30.864 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.wal_dir: 2024-05-09T04:39:30.864 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T04:39:30.864 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T04:39:30.864 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T04:39:30.864 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T04:39:30.864 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T04:39:30.864 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T04:39:30.864 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T04:39:30.864 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T04:39:30.864 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.write_buffer_manager: 0x564ec2db88c0 2024-05-09T04:39:30.865 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T04:39:30.865 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T04:39:30.865 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T04:39:30.865 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T04:39:30.865 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T04:39:30.865 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T04:39:30.865 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T04:39:30.865 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T04:39:30.865 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.unordered_write: 0 2024-05-09T04:39:30.865 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T04:39:30.865 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T04:39:30.865 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T04:39:30.865 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T04:39:30.865 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.row_cache: None 2024-05-09T04:39:30.865 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.wal_filter: None 2024-05-09T04:39:30.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T04:39:30.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T04:39:30.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.two_write_queues: 0 2024-05-09T04:39:30.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T04:39:30.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.wal_compression: 0 2024-05-09T04:39:30.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.atomic_flush: 0 2024-05-09T04:39:30.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T04:39:30.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T04:39:30.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T04:39:30.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.log_readahead_size: 0 2024-05-09T04:39:30.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T04:39:30.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T04:39:30.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T04:39:30.866 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T04:39:30.867 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T04:39:30.867 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T04:39:30.867 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T04:39:30.867 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_background_jobs: 2 2024-05-09T04:39:30.867 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_background_compactions: -1 2024-05-09T04:39:30.867 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_subcompactions: 1 2024-05-09T04:39:30.867 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T04:39:30.867 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T04:39:30.867 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T04:39:30.867 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T04:39:30.867 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T04:39:30.867 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T04:39:30.867 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T04:39:30.867 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T04:39:30.867 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_open_files: -1 2024-05-09T04:39:30.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T04:39:30.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T04:39:30.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T04:39:30.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T04:39:30.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_background_flushes: -1 2024-05-09T04:39:30.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Compression algorithms supported: 2024-05-09T04:39:30.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T04:39:30.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: kZSTD supported: 0 2024-05-09T04:39:30.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: kXpressCompression supported: 0 2024-05-09T04:39:30.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T04:39:30.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: kZlibCompression supported: 1 2024-05-09T04:39:30.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: kSnappyCompression supported: 1 2024-05-09T04:39:30.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: kLZ4Compression supported: 1 2024-05-09T04:39:30.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: kBZip2Compression supported: 0 2024-05-09T04:39:30.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T04:39:30.869 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T04:39:30.869 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-09T04:39:30.869 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T04:39:30.869 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T04:39:30.869 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.merge_operator: 2024-05-09T04:39:30.869 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compaction_filter: None 2024-05-09T04:39:30.869 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T04:39:30.869 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T04:39:30.869 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T04:39:30.869 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T04:39:30.869 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564ec2eee1e0) 2024-05-09T04:39:30.869 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks: 1 2024-05-09T04:39:30.869 INFO:journalctl@ceph.mon.a.smithi046.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T04:39:30.869 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T04:39:30.870 INFO:journalctl@ceph.mon.a.smithi046.stdout: pin_top_level_index_and_filter: 1 2024-05-09T04:39:30.870 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_type: 0 2024-05-09T04:39:30.870 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_index_type: 0 2024-05-09T04:39:30.870 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_shortening: 1 2024-05-09T04:39:30.870 INFO:journalctl@ceph.mon.a.smithi046.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T04:39:30.870 INFO:journalctl@ceph.mon.a.smithi046.stdout: checksum: 4 2024-05-09T04:39:30.870 INFO:journalctl@ceph.mon.a.smithi046.stdout: no_block_cache: 0 2024-05-09T04:39:30.870 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache: 0x564ec2deb1f0 2024-05-09T04:39:30.870 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_name: BinnedLRUCache 2024-05-09T04:39:30.870 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_options: 2024-05-09T04:39:30.870 INFO:journalctl@ceph.mon.a.smithi046.stdout: capacity : 536870912 2024-05-09T04:39:30.870 INFO:journalctl@ceph.mon.a.smithi046.stdout: num_shard_bits : 4 2024-05-09T04:39:30.870 INFO:journalctl@ceph.mon.a.smithi046.stdout: strict_capacity_limit : 0 2024-05-09T04:39:30.870 INFO:journalctl@ceph.mon.a.smithi046.stdout: high_pri_pool_ratio: 0.000 2024-05-09T04:39:30.871 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_cache_compressed: (nil) 2024-05-09T04:39:30.871 INFO:journalctl@ceph.mon.a.smithi046.stdout: persistent_cache: (nil) 2024-05-09T04:39:30.871 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size: 4096 2024-05-09T04:39:30.871 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_size_deviation: 10 2024-05-09T04:39:30.871 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_restart_interval: 16 2024-05-09T04:39:30.871 INFO:journalctl@ceph.mon.a.smithi046.stdout: index_block_restart_interval: 1 2024-05-09T04:39:30.871 INFO:journalctl@ceph.mon.a.smithi046.stdout: metadata_block_size: 4096 2024-05-09T04:39:30.871 INFO:journalctl@ceph.mon.a.smithi046.stdout: partition_filters: 0 2024-05-09T04:39:30.871 INFO:journalctl@ceph.mon.a.smithi046.stdout: use_delta_encoding: 1 2024-05-09T04:39:30.871 INFO:journalctl@ceph.mon.a.smithi046.stdout: filter_policy: bloomfilter 2024-05-09T04:39:30.871 INFO:journalctl@ceph.mon.a.smithi046.stdout: whole_key_filtering: 1 2024-05-09T04:39:30.871 INFO:journalctl@ceph.mon.a.smithi046.stdout: verify_compression: 0 2024-05-09T04:39:30.871 INFO:journalctl@ceph.mon.a.smithi046.stdout: read_amp_bytes_per_bit: 0 2024-05-09T04:39:30.871 INFO:journalctl@ceph.mon.a.smithi046.stdout: format_version: 5 2024-05-09T04:39:30.872 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_index_compression: 1 2024-05-09T04:39:30.872 INFO:journalctl@ceph.mon.a.smithi046.stdout: block_align: 0 2024-05-09T04:39:30.872 INFO:journalctl@ceph.mon.a.smithi046.stdout: max_auto_readahead_size: 262144 2024-05-09T04:39:30.872 INFO:journalctl@ceph.mon.a.smithi046.stdout: prepopulate_block_cache: 0 2024-05-09T04:39:30.872 INFO:journalctl@ceph.mon.a.smithi046.stdout: initial_auto_readahead_size: 8192 2024-05-09T04:39:30.872 INFO:journalctl@ceph.mon.a.smithi046.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T04:39:30.872 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T04:39:30.872 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T04:39:30.872 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compression: NoCompression 2024-05-09T04:39:30.872 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T04:39:30.872 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T04:39:30.872 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T04:39:30.872 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.num_levels: 7 2024-05-09T04:39:30.872 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T04:39:30.872 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T04:39:30.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T04:39:30.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T04:39:30.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T04:39:30.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T04:39:30.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T04:39:30.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T04:39:30.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T04:39:30.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T04:39:30.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T04:39:30.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T04:39:30.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T04:39:30.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T04:39:30.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T04:39:30.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T04:39:30.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T04:39:30.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T04:39:30.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T04:39:30.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T04:39:30.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T04:39:30.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T04:39:30.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T04:39:30.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T04:39:30.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T04:39:30.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T04:39:30.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T04:39:30.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T04:39:30.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T04:39:30.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T04:39:30.874 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T04:39:30.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T04:39:30.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T04:39:30.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T04:39:30.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T04:39:30.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T04:39:30.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T04:39:30.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T04:39:30.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T04:39:30.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T04:39:30.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T04:39:30.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T04:39:30.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T04:39:30.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T04:39:30.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T04:39:30.876 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T04:39:30.876 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T04:39:30.876 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T04:39:30.876 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T04:39:30.876 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T04:39:30.876 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T04:39:30.876 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T04:39:30.876 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T04:39:30.876 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-09T04:39:30.876 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.inplace_update_support: 0 2024-05-09T04:39:30.876 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T04:39:30.876 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T04:39:30.877 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T04:39:30.877 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T04:39:30.877 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.bloom_locality: 0 2024-05-09T04:39:30.877 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.max_successive_merges: 0 2024-05-09T04:39:30.877 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T04:39:30.877 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T04:39:30.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T04:39:30.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T04:39:30.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.ttl: 2592000 2024-05-09T04:39:30.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T04:39:30.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T04:39:30.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T04:39:30.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.enable_blob_files: false 2024-05-09T04:39:30.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.min_blob_size: 0 2024-05-09T04:39:30.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T04:39:30.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T04:39:30.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T04:39:30.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T04:39:30.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T04:39:30.878 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T04:39:30.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T04:39:30.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T04:39:30.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-09T04:39:30.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-09T04:39:30.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: bf63f7ad-730c-4c2d-a666-bfba42e5cfa7 2024-05-09T04:39:30.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715229570821233, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-09T04:39:30.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-09T04:39:30.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715229570822431, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100431, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98315, "index_size": 308, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14161, "raw_average_key_size": 49, "raw_value_size": 90614, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715229570, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "bf63f7ad-730c-4c2d-a666-bfba42e5cfa7", "db_session_id": "R2S1JWZLAW97M40ZBRZB", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-09T04:39:30.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715229570822516, "job": 1, "event": "recovery_finished"} 2024-05-09T04:39:30.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-09T04:39:30.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T04:39:30.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564ec2dcfc00 2024-05-09T04:39:30.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: DB pointer 0x564ec7bc2000 2024-05-09T04:39:30.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T04:39:30.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T04:39:30.880 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** DB Stats ** 2024-05-09T04:39:30.880 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T04:39:30.880 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T04:39:30.880 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T04:39:30.880 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T04:39:30.880 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T04:39:30.880 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T04:39:30.880 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T04:39:30.880 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-09T04:39:30.880 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-05-09T04:39:30.880 INFO:journalctl@ceph.mon.a.smithi046.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T04:39:30.880 INFO:journalctl@ceph.mon.a.smithi046.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T04:39:30.880 INFO:journalctl@ceph.mon.a.smithi046.stdout: L0 2/0 99.94 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T04:39:30.880 INFO:journalctl@ceph.mon.a.smithi046.stdout: Sum 2/0 99.94 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T04:39:30.880 INFO:journalctl@ceph.mon.a.smithi046.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T04:39:30.881 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-09T04:39:30.881 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** Compaction Stats [default] ** 2024-05-09T04:39:30.881 INFO:journalctl@ceph.mon.a.smithi046.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T04:39:30.881 INFO:journalctl@ceph.mon.a.smithi046.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T04:39:30.881 INFO:journalctl@ceph.mon.a.smithi046.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 157.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-09T04:39:30.881 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-09T04:39:30.881 INFO:journalctl@ceph.mon.a.smithi046.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T04:39:30.881 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-09T04:39:30.881 INFO:journalctl@ceph.mon.a.smithi046.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T04:39:30.881 INFO:journalctl@ceph.mon.a.smithi046.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T04:39:30.881 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T04:39:30.881 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T04:39:30.881 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T04:39:30.881 INFO:journalctl@ceph.mon.a.smithi046.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T04:39:30.882 INFO:journalctl@ceph.mon.a.smithi046.stdout: Cumulative compaction: 0.00 GB write, 28.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T04:39:30.882 INFO:journalctl@ceph.mon.a.smithi046.stdout: Interval compaction: 0.00 GB write, 28.14 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T04:39:30.882 INFO:journalctl@ceph.mon.a.smithi046.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T04:39:30.882 INFO:journalctl@ceph.mon.a.smithi046.stdout: Block cache BinnedLRUCache@0x564ec2deb1f0#2 capacity: 512.00 MB usage: 1.53 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-05-09T04:39:30.882 INFO:journalctl@ceph.mon.a.smithi046.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.03 KB,0.000196695%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-09T04:39:30.882 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-09T04:39:30.882 INFO:journalctl@ceph.mon.a.smithi046.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T04:39:30.882 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: starting mon.a rank 0 at public addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] at bind addrs [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:30.882 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: mon.a@-1(???) e1 preinit fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:30.882 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: mon.a@-1(???).mds e1 new map 2024-05-09T04:39:30.882 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: mon.a@-1(???).mds e1 print_map 2024-05-09T04:39:30.882 INFO:journalctl@ceph.mon.a.smithi046.stdout: e1 2024-05-09T04:39:30.882 INFO:journalctl@ceph.mon.a.smithi046.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T04:39:30.882 INFO:journalctl@ceph.mon.a.smithi046.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T04:39:30.883 INFO:journalctl@ceph.mon.a.smithi046.stdout: legacy client fscid: -1 2024-05-09T04:39:30.883 INFO:journalctl@ceph.mon.a.smithi046.stdout: 2024-05-09T04:39:30.883 INFO:journalctl@ceph.mon.a.smithi046.stdout: No filesystems configured 2024-05-09T04:39:30.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T04:39:30.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T04:39:30.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T04:39:30.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T04:39:30.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 ceph-mon[30136]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-09T04:39:30.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 bash[30117]: 0809ad9b5ec45012eb937b11f1b3003ca18bd02884a956db0d2e187377516c3f 2024-05-09T04:39:30.883 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:30 smithi046 systemd[1]: Started Ceph mon.a for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:39:30.883 INFO:teuthology.orchestra.run.smithi046.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-09T04:39:32.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:31 smithi046 ceph-mon[30136]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-09T04:39:32.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:31 smithi046 ceph-mon[30136]: monmap epoch 1 2024-05-09T04:39:32.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:31 smithi046 ceph-mon[30136]: fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:32.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:31 smithi046 ceph-mon[30136]: last_changed 2024-05-09T04:39:22.726361+0000 2024-05-09T04:39:32.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:31 smithi046 ceph-mon[30136]: created 2024-05-09T04:39:22.726361+0000 2024-05-09T04:39:32.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:31 smithi046 ceph-mon[30136]: min_mon_release 19 (squid) 2024-05-09T04:39:32.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:31 smithi046 ceph-mon[30136]: election_strategy: 1 2024-05-09T04:39:32.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:31 smithi046 ceph-mon[30136]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T04:39:32.066 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:31 smithi046 ceph-mon[30136]: fsmap 2024-05-09T04:39:32.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:31 smithi046 ceph-mon[30136]: osdmap e1: 0 total, 0 up, 0 in 2024-05-09T04:39:32.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:31 smithi046 ceph-mon[30136]: mgrmap e1: no daemons active 2024-05-09T04:39:32.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:31 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1720631872' entity='client.admin' 2024-05-09T04:39:32.069 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.331+0000 7f15bf0ec700 1 Processor -- start 2024-05-09T04:39:32.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.331+0000 7f15bf0ec700 1 -- start start 2024-05-09T04:39:32.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.331+0000 7f15bf0ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15b8109c20 0x7f15b810a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:32.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.331+0000 7f15bf0ec700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15b810a620 con 0x7f15b8109c20 2024-05-09T04:39:32.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.332+0000 7f15bce88700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15b8109c20 0x7f15b810a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:32.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.332+0000 7f15bce88700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15b8109c20 0x7f15b810a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35928/0 (socket says 172.21.15.46:35928) 2024-05-09T04:39:32.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.332+0000 7f15bce88700 1 -- 172.21.15.46:0/3393429892 learned_addr learned my addr 172.21.15.46:0/3393429892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:32.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.332+0000 7f15bce88700 1 -- 172.21.15.46:0/3393429892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b810ad50 con 0x7f15b8109c20 2024-05-09T04:39:32.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.332+0000 7f15bce88700 1 --2- 172.21.15.46:0/3393429892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15b8109c20 0x7f15b810a030 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0098a0 tx=0x7f15ac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76a72ebd59403d3b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:32.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.332+0000 7f15b77fe700 1 -- 172.21.15.46:0/3393429892 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15ac004160 con 0x7f15b8109c20 2024-05-09T04:39:32.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.332+0000 7f15b77fe700 1 -- 172.21.15.46:0/3393429892 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f15ac004300 con 0x7f15b8109c20 2024-05-09T04:39:32.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.332+0000 7f15b77fe700 1 -- 172.21.15.46:0/3393429892 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15ac011af0 con 0x7f15b8109c20 2024-05-09T04:39:32.074 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.333+0000 7f15bf0ec700 1 -- 172.21.15.46:0/3393429892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15b8109c20 msgr2=0x7f15b810a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:32.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.333+0000 7f15bf0ec700 1 --2- 172.21.15.46:0/3393429892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15b8109c20 0x7f15b810a030 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f15ac0098a0 tx=0x7f15ac009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:32.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.333+0000 7f15bf0ec700 1 -- 172.21.15.46:0/3393429892 shutdown_connections 2024-05-09T04:39:32.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.333+0000 7f15bf0ec700 1 --2- 172.21.15.46:0/3393429892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15b8109c20 0x7f15b810a030 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:32.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.333+0000 7f15bf0ec700 1 -- 172.21.15.46:0/3393429892 >> 172.21.15.46:0/3393429892 conn(0x7f15b81052e0 msgr2=0x7f15b8107730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:32.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.333+0000 7f15bf0ec700 1 -- 172.21.15.46:0/3393429892 shutdown_connections 2024-05-09T04:39:32.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.333+0000 7f15bf0ec700 1 -- 172.21.15.46:0/3393429892 wait complete. 2024-05-09T04:39:32.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.334+0000 7f15bf0ec700 1 Processor -- start 2024-05-09T04:39:32.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.334+0000 7f15bf0ec700 1 -- start start 2024-05-09T04:39:32.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.334+0000 7f15bf0ec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15b8109c20 0x7f15b819e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:32.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.334+0000 7f15bf0ec700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15b810e100 con 0x7f15b8109c20 2024-05-09T04:39:32.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.334+0000 7f15bce88700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15b8109c20 0x7f15b819e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:32.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.334+0000 7f15bce88700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15b8109c20 0x7f15b819e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35934/0 (socket says 172.21.15.46:35934) 2024-05-09T04:39:32.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.334+0000 7f15bce88700 1 -- 172.21.15.46:0/1720631872 learned_addr learned my addr 172.21.15.46:0/1720631872 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:32.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.335+0000 7f15bce88700 1 -- 172.21.15.46:0/1720631872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b819e580 con 0x7f15b8109c20 2024-05-09T04:39:32.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.335+0000 7f15bce88700 1 --2- 172.21.15.46:0/1720631872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15b8109c20 0x7f15b819e040 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f15ac011670 tx=0x7f15ac004280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:32.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.335+0000 7f15b5ffb700 1 -- 172.21.15.46:0/1720631872 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15ac0036a0 con 0x7f15b8109c20 2024-05-09T04:39:32.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.335+0000 7f15b5ffb700 1 -- 172.21.15.46:0/1720631872 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f15ac003cd0 con 0x7f15b8109c20 2024-05-09T04:39:32.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.335+0000 7f15b5ffb700 1 -- 172.21.15.46:0/1720631872 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f15ac0193c0 con 0x7f15b8109c20 2024-05-09T04:39:32.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.335+0000 7f15bf0ec700 1 -- 172.21.15.46:0/1720631872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b819e810 con 0x7f15b8109c20 2024-05-09T04:39:32.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.335+0000 7f15bf0ec700 1 -- 172.21.15.46:0/1720631872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b81a1420 con 0x7f15b8109c20 2024-05-09T04:39:32.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.335+0000 7f15b5ffb700 1 -- 172.21.15.46:0/1720631872 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f15ac0199c0 con 0x7f15b8109c20 2024-05-09T04:39:32.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.335+0000 7f15b5ffb700 1 -- 172.21.15.46:0/1720631872 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f15ac0243f0 con 0x7f15b8109c20 2024-05-09T04:39:32.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.336+0000 7f15bf0ec700 1 -- 172.21.15.46:0/1720631872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b8064e20 con 0x7f15b8109c20 2024-05-09T04:39:32.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.339+0000 7f15b5ffb700 1 -- 172.21.15.46:0/1720631872 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f15ac043ec0 con 0x7f15b8109c20 2024-05-09T04:39:32.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.409+0000 7f15bf0ec700 1 -- 172.21.15.46:0/1720631872 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f15b81a15f0 con 0x7f15b8109c20 2024-05-09T04:39:32.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.411+0000 7f15b5ffb700 1 -- 172.21.15.46:0/1720631872 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f15ac032020 con 0x7f15b8109c20 2024-05-09T04:39:32.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.412+0000 7f15bf0ec700 1 -- 172.21.15.46:0/1720631872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15b8109c20 msgr2=0x7f15b819e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:32.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.412+0000 7f15bf0ec700 1 --2- 172.21.15.46:0/1720631872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15b8109c20 0x7f15b819e040 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f15ac011670 tx=0x7f15ac004280 comp rx=0 tx=0).stop 2024-05-09T04:39:32.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.412+0000 7f15bf0ec700 1 -- 172.21.15.46:0/1720631872 shutdown_connections 2024-05-09T04:39:32.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.412+0000 7f15bf0ec700 1 --2- 172.21.15.46:0/1720631872 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15b8109c20 0x7f15b819e040 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:32.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.412+0000 7f15bf0ec700 1 -- 172.21.15.46:0/1720631872 >> 172.21.15.46:0/1720631872 conn(0x7f15b81052e0 msgr2=0x7f15b8106ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:32.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.412+0000 7f15bf0ec700 1 -- 172.21.15.46:0/1720631872 shutdown_connections 2024-05-09T04:39:32.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:31.412+0000 7f15bf0ec700 1 -- 172.21.15.46:0/1720631872 wait complete. 2024-05-09T04:39:32.077 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-09T04:39:32.099 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-09T04:39:32.099 INFO:teuthology.orchestra.run.smithi046.stdout:Creating mgr... 2024-05-09T04:39:32.099 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-09T04:39:32.100 INFO:teuthology.orchestra.run.smithi046.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-09T04:39:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.x 2024-05-09T04:39:32.339 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-09T04:39:32.490 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968.target.wants/ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@.service. 2024-05-09T04:39:32.718 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:32 smithi046 systemd[1]: Starting Ceph mgr.x for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:39:33.021 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:32 smithi046 podman[30352]: 2024-05-09 04:39:32.71948158 +0000 UTC m=+0.102896042 container create a3cb336f79308678ad9c1bbd97f083feea349e6b49314c83601697bea93399a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-05-09T04:39:33.021 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:32 smithi046 podman[30352]: 2024-05-09 04:39:32.634703611 +0000 UTC m=+0.018118085 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:39:33.021 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:32 smithi046 podman[30352]: 2024-05-09 04:39:32.930064615 +0000 UTC m=+0.313479080 container init a3cb336f79308678ad9c1bbd97f083feea349e6b49314c83601697bea93399a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-09T04:39:33.021 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:32 smithi046 podman[30352]: 2024-05-09 04:39:32.932929819 +0000 UTC m=+0.316344284 container start a3cb336f79308678ad9c1bbd97f083feea349e6b49314c83601697bea93399a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-05-09T04:39:33.021 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:33 smithi046 bash[30352]: a3cb336f79308678ad9c1bbd97f083feea349e6b49314c83601697bea93399a3 2024-05-09T04:39:33.035 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T04:39:33.035 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-05-09T04:39:33.043 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T04:39:33.043 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-09T04:39:33.043 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-05-09T04:39:33.043 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to enable service . firewalld.service is not available 2024-05-09T04:39:33.052 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T04:39:33.052 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-05-09T04:39:33.060 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T04:39:33.060 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-09T04:39:33.060 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-05-09T04:39:33.060 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-09T04:39:33.060 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr to start... 2024-05-09T04:39:33.060 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr... 2024-05-09T04:39:33.296 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:33 smithi046 systemd[1]: Started Ceph mgr.x for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:39:33.296 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:33 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:33.103+0000 7fcaad685200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T04:39:33.296 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:33 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:33.296+0000 7fcaad685200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:39:33.581 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:33 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:33.403+0000 7fcaad685200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T04:39:33.960 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:33 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1273453743' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T04:39:33.961 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:33 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:33.740+0000 7fcaad685200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T04:39:33.961 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:33 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:33.832+0000 7fcaad685200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T04:39:34.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:34.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:39:34.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "ecf9ae2c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:39:34.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T04:39:34.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T04:39:34.277 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T04:39:34.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T04:39:34.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:34.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T04:39:34.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T04:39:34.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-05-09T04:39:34.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:39:34.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T04:39:34.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T04:39:34.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:39:34.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-09T04:39:34.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T04:39:34.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:34.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T04:39:34.278 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T04:39:34.279 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:34.279 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T04:39:34.279 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:34.279 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T04:39:34.279 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T04:39:34.279 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T04:39:34.279 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T04:39:34.279 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T04:39:34.279 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T04:39:34.279 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:34.279 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T04:39:34.279 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T04:39:34.279 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T04:39:34.279 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T04:39:34.280 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T04:39:34.280 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T04:39:34.280 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T04:39:34.280 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T04:39:34.280 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T04:39:34.280 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:34.280 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T04:39:34.280 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:34.280 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T04:39:34.280 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T04:39:34.280 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:34.282 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T04:39:34.282 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T04:39:34.282 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T04:39:34.282 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T04:39:34.283 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T04:39:34.283 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T04:39:34.283 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T04:39:34.283 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:39:34.283 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:39:34.283 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:34.283 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T04:39:34.283 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:34.283 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T04:39:25.634513+0000", 2024-05-09T04:39:34.283 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:39:34.283 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:34.283 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T04:39:34.283 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:39:34.288 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.641+0000 7f841797f700 1 Processor -- start 2024-05-09T04:39:34.288 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.641+0000 7f841797f700 1 -- start start 2024-05-09T04:39:34.288 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.641+0000 7f841797f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410108460 0x7f8410108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:34.288 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.641+0000 7f841797f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f841007cdf0 con 0x7f8410108460 2024-05-09T04:39:34.288 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.642+0000 7f841571b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410108460 0x7f8410108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:34.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.642+0000 7f841571b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410108460 0x7f8410108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35964/0 (socket says 172.21.15.46:35964) 2024-05-09T04:39:34.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.642+0000 7f841571b700 1 -- 172.21.15.46:0/3624100534 learned_addr learned my addr 172.21.15.46:0/3624100534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:34.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.642+0000 7f841571b700 1 -- 172.21.15.46:0/3624100534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8410108db0 con 0x7f8410108460 2024-05-09T04:39:34.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.642+0000 7f841571b700 1 --2- 172.21.15.46:0/3624100534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410108460 0x7f8410108870 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f840c0098a0 tx=0x7f840c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9989d9a1ef85ad server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:34.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.643+0000 7f8407fff700 1 -- 172.21.15.46:0/3624100534 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f840c004160 con 0x7f8410108460 2024-05-09T04:39:34.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.643+0000 7f8407fff700 1 -- 172.21.15.46:0/3624100534 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f840c004300 con 0x7f8410108460 2024-05-09T04:39:34.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.643+0000 7f8407fff700 1 -- 172.21.15.46:0/3624100534 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f840c011af0 con 0x7f8410108460 2024-05-09T04:39:34.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.643+0000 7f841797f700 1 -- 172.21.15.46:0/3624100534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410108460 msgr2=0x7f8410108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:34.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.644+0000 7f841797f700 1 --2- 172.21.15.46:0/3624100534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410108460 0x7f8410108870 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f840c0098a0 tx=0x7f840c009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:34.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.644+0000 7f841797f700 1 -- 172.21.15.46:0/3624100534 shutdown_connections 2024-05-09T04:39:34.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.644+0000 7f841797f700 1 --2- 172.21.15.46:0/3624100534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410108460 0x7f8410108870 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:34.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.644+0000 7f841797f700 1 -- 172.21.15.46:0/3624100534 >> 172.21.15.46:0/3624100534 conn(0x7f84101040f0 msgr2=0x7f8410106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:34.289 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.644+0000 7f841797f700 1 -- 172.21.15.46:0/3624100534 shutdown_connections 2024-05-09T04:39:34.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.644+0000 7f841797f700 1 -- 172.21.15.46:0/3624100534 wait complete. 2024-05-09T04:39:34.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.645+0000 7f841797f700 1 Processor -- start 2024-05-09T04:39:34.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.645+0000 7f841797f700 1 -- start start 2024-05-09T04:39:34.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.645+0000 7f841797f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410108460 0x7f84101a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:34.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.645+0000 7f841797f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8410109cb0 con 0x7f8410108460 2024-05-09T04:39:34.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.645+0000 7f841571b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410108460 0x7f84101a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:34.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.645+0000 7f841571b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410108460 0x7f84101a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35974/0 (socket says 172.21.15.46:35974) 2024-05-09T04:39:34.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.645+0000 7f841571b700 1 -- 172.21.15.46:0/1273453743 learned_addr learned my addr 172.21.15.46:0/1273453743 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:34.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.645+0000 7f841571b700 1 -- 172.21.15.46:0/1273453743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84101a0670 con 0x7f8410108460 2024-05-09T04:39:34.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.646+0000 7f841571b700 1 --2- 172.21.15.46:0/1273453743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410108460 0x7f84101a0130 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f840c0117b0 tx=0x7f840c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:34.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.646+0000 7f84067fc700 1 -- 172.21.15.46:0/1273453743 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f840c003af0 con 0x7f8410108460 2024-05-09T04:39:34.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.646+0000 7f841797f700 1 -- 172.21.15.46:0/1273453743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84101a0900 con 0x7f8410108460 2024-05-09T04:39:34.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.646+0000 7f84067fc700 1 -- 172.21.15.46:0/1273453743 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f840c020070 con 0x7f8410108460 2024-05-09T04:39:34.290 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.646+0000 7f84067fc700 1 -- 172.21.15.46:0/1273453743 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f840c019630 con 0x7f8410108460 2024-05-09T04:39:34.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.646+0000 7f841797f700 1 -- 172.21.15.46:0/1273453743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84101a3510 con 0x7f8410108460 2024-05-09T04:39:34.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.646+0000 7f84067fc700 1 -- 172.21.15.46:0/1273453743 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f840c019c80 con 0x7f8410108460 2024-05-09T04:39:34.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.646+0000 7f84067fc700 1 -- 172.21.15.46:0/1273453743 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f840c00b720 con 0x7f8410108460 2024-05-09T04:39:34.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.647+0000 7f841797f700 1 -- 172.21.15.46:0/1273453743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8410064e20 con 0x7f8410108460 2024-05-09T04:39:34.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.649+0000 7f84067fc700 1 -- 172.21.15.46:0/1273453743 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f840c0042a0 con 0x7f8410108460 2024-05-09T04:39:34.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.724+0000 7f841797f700 1 -- 172.21.15.46:0/1273453743 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f841002e9a0 con 0x7f8410108460 2024-05-09T04:39:34.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.724+0000 7f84067fc700 1 -- 172.21.15.46:0/1273453743 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f840c01a9c0 con 0x7f8410108460 2024-05-09T04:39:34.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.725+0000 7f841797f700 1 -- 172.21.15.46:0/1273453743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410108460 msgr2=0x7f84101a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:34.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.725+0000 7f841797f700 1 --2- 172.21.15.46:0/1273453743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410108460 0x7f84101a0130 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f840c0117b0 tx=0x7f840c011b70 comp rx=0 tx=0).stop 2024-05-09T04:39:34.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.725+0000 7f841797f700 1 -- 172.21.15.46:0/1273453743 shutdown_connections 2024-05-09T04:39:34.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.725+0000 7f841797f700 1 --2- 172.21.15.46:0/1273453743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8410108460 0x7f84101a0130 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:34.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.725+0000 7f841797f700 1 -- 172.21.15.46:0/1273453743 >> 172.21.15.46:0/1273453743 conn(0x7f84101040f0 msgr2=0x7f8410196820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:34.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.726+0000 7f841797f700 1 -- 172.21.15.46:0/1273453743 shutdown_connections 2024-05-09T04:39:34.291 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:33.726+0000 7f841797f700 1 -- 172.21.15.46:0/1273453743 wait complete. 2024-05-09T04:39:34.292 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (1/15)... 2024-05-09T04:39:35.466 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:35 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:35.107+0000 7fcaad685200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T04:39:35.466 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:35 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:35.286+0000 7fcaad685200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T04:39:35.466 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:35 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:35.466+0000 7fcaad685200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T04:39:36.074 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:35 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:35.766+0000 7fcaad685200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T04:39:36.075 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:35 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:35.862+0000 7fcaad685200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T04:39:36.075 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:36 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:36.075+0000 7fcaad685200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T04:39:36.329 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:36 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:36.275+0000 7fcaad685200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:39:37.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:36 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3787219600' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T04:39:37.079 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:36 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:36.930+0000 7fcaad685200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T04:39:37.454 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:37 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:37.162+0000 7fcaad685200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T04:39:37.487 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:37.487 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:39:37.487 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "ecf9ae2c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:39:37.487 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T04:39:37.487 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T04:39:37.487 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T04:39:37.487 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T04:39:37.487 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:37.487 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T04:39:37.487 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T04:39:37.487 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-05-09T04:39:37.487 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:39:37.487 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T04:39:37.488 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T04:39:37.488 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:39:37.488 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-05-09T04:39:37.488 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T04:39:37.488 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:37.488 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T04:39:37.488 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T04:39:37.488 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:37.488 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T04:39:37.488 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:37.488 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T04:39:37.488 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T04:39:37.488 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T04:39:37.488 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T04:39:37.488 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T04:39:37.489 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T04:39:37.489 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:37.489 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T04:39:37.489 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T04:39:37.489 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T04:39:37.489 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T04:39:37.489 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T04:39:37.489 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T04:39:37.489 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T04:39:37.489 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T04:39:37.489 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T04:39:37.489 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:37.489 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T04:39:37.489 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:37.489 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T04:39:37.490 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T04:39:37.490 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:37.490 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T04:39:37.490 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T04:39:37.490 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T04:39:37.490 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T04:39:37.490 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T04:39:37.490 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T04:39:37.490 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T04:39:37.490 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:39:37.490 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:39:37.490 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:37.490 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T04:39:37.490 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T04:39:25.634513+0000", 2024-05-09T04:39:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:39:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T04:39:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:39:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.769+0000 7fa63acd0700 1 Processor -- start 2024-05-09T04:39:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.769+0000 7fa63acd0700 1 -- start start 2024-05-09T04:39:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.769+0000 7fa63acd0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa634109040 0x7fa634109450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.769+0000 7fa63acd0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa634109a40 con 0x7fa634109040 2024-05-09T04:39:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.769+0000 7fa638a6c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa634109040 0x7fa634109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.769+0000 7fa638a6c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa634109040 0x7fa634109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59372/0 (socket says 172.21.15.46:59372) 2024-05-09T04:39:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.769+0000 7fa638a6c700 1 -- 172.21.15.46:0/2779301525 learned_addr learned my addr 172.21.15.46:0/2779301525 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.770+0000 7fa638a6c700 1 -- 172.21.15.46:0/2779301525 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa634109bc0 con 0x7fa634109040 2024-05-09T04:39:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.770+0000 7fa638a6c700 1 --2- 172.21.15.46:0/2779301525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa634109040 0x7fa634109450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6280098a0 tx=0x7fa628009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f1835364077e06 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:37.491 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.770+0000 7fa6337fe700 1 -- 172.21.15.46:0/2779301525 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa628004160 con 0x7fa634109040 2024-05-09T04:39:37.492 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.770+0000 7fa6337fe700 1 -- 172.21.15.46:0/2779301525 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa628004300 con 0x7fa634109040 2024-05-09T04:39:37.492 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.770+0000 7fa6337fe700 1 -- 172.21.15.46:0/2779301525 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa628011af0 con 0x7fa634109040 2024-05-09T04:39:37.492 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.771+0000 7fa63acd0700 1 -- 172.21.15.46:0/2779301525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa634109040 msgr2=0x7fa634109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:37.492 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.771+0000 7fa63acd0700 1 --2- 172.21.15.46:0/2779301525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa634109040 0x7fa634109450 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa6280098a0 tx=0x7fa628009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:37.492 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.771+0000 7fa63acd0700 1 -- 172.21.15.46:0/2779301525 shutdown_connections 2024-05-09T04:39:37.492 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.771+0000 7fa63acd0700 1 --2- 172.21.15.46:0/2779301525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa634109040 0x7fa634109450 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:37.492 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.771+0000 7fa63acd0700 1 -- 172.21.15.46:0/2779301525 >> 172.21.15.46:0/2779301525 conn(0x7fa6341039b0 msgr2=0x7fa634105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:37.492 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.771+0000 7fa63acd0700 1 -- 172.21.15.46:0/2779301525 shutdown_connections 2024-05-09T04:39:37.492 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.771+0000 7fa63acd0700 1 -- 172.21.15.46:0/2779301525 wait complete. 2024-05-09T04:39:37.492 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.772+0000 7fa63acd0700 1 Processor -- start 2024-05-09T04:39:37.492 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.772+0000 7fa63acd0700 1 -- start start 2024-05-09T04:39:37.492 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.772+0000 7fa63acd0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa634109040 0x7fa634108640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:37.492 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.772+0000 7fa63acd0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa63410aac0 con 0x7fa634109040 2024-05-09T04:39:37.492 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.772+0000 7fa638a6c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa634109040 0x7fa634108640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:37.492 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.772+0000 7fa638a6c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa634109040 0x7fa634108640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59388/0 (socket says 172.21.15.46:59388) 2024-05-09T04:39:37.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.772+0000 7fa638a6c700 1 -- 172.21.15.46:0/3787219600 learned_addr learned my addr 172.21.15.46:0/3787219600 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:37.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.773+0000 7fa638a6c700 1 -- 172.21.15.46:0/3787219600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa634108b80 con 0x7fa634109040 2024-05-09T04:39:37.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.773+0000 7fa638a6c700 1 --2- 172.21.15.46:0/3787219600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa634109040 0x7fa634108640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6280090e0 tx=0x7fa628011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:37.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.773+0000 7fa631ffb700 1 -- 172.21.15.46:0/3787219600 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6280039e0 con 0x7fa634109040 2024-05-09T04:39:37.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.773+0000 7fa631ffb700 1 -- 172.21.15.46:0/3787219600 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa628003b80 con 0x7fa634109040 2024-05-09T04:39:37.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.773+0000 7fa631ffb700 1 -- 172.21.15.46:0/3787219600 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa6280246e0 con 0x7fa634109040 2024-05-09T04:39:37.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.773+0000 7fa63acd0700 1 -- 172.21.15.46:0/3787219600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6341a0d80 con 0x7fa634109040 2024-05-09T04:39:37.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.773+0000 7fa63acd0700 1 -- 172.21.15.46:0/3787219600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6341a1200 con 0x7fa634109040 2024-05-09T04:39:37.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.773+0000 7fa631ffb700 1 -- 172.21.15.46:0/3787219600 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa628022020 con 0x7fa634109040 2024-05-09T04:39:37.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.774+0000 7fa631ffb700 1 -- 172.21.15.46:0/3787219600 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa62802dcd0 con 0x7fa634109040 2024-05-09T04:39:37.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.774+0000 7fa63acd0700 1 -- 172.21.15.46:0/3787219600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa634064e20 con 0x7fa634109040 2024-05-09T04:39:37.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.777+0000 7fa631ffb700 1 -- 172.21.15.46:0/3787219600 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fa62801aae0 con 0x7fa634109040 2024-05-09T04:39:37.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.852+0000 7fa63acd0700 1 -- 172.21.15.46:0/3787219600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fa6341a15c0 con 0x7fa634109040 2024-05-09T04:39:37.493 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.852+0000 7fa631ffb700 1 -- 172.21.15.46:0/3787219600 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7fa6341a15c0 con 0x7fa634109040 2024-05-09T04:39:37.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.853+0000 7fa63acd0700 1 -- 172.21.15.46:0/3787219600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa634109040 msgr2=0x7fa634108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:37.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.853+0000 7fa63acd0700 1 --2- 172.21.15.46:0/3787219600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa634109040 0x7fa634108640 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6280090e0 tx=0x7fa628011940 comp rx=0 tx=0).stop 2024-05-09T04:39:37.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.853+0000 7fa63acd0700 1 -- 172.21.15.46:0/3787219600 shutdown_connections 2024-05-09T04:39:37.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.853+0000 7fa63acd0700 1 --2- 172.21.15.46:0/3787219600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa634109040 0x7fa634108640 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:37.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.853+0000 7fa63acd0700 1 -- 172.21.15.46:0/3787219600 >> 172.21.15.46:0/3787219600 conn(0x7fa6341039b0 msgr2=0x7fa634105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:37.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.853+0000 7fa63acd0700 1 -- 172.21.15.46:0/3787219600 shutdown_connections 2024-05-09T04:39:37.494 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:36.853+0000 7fa63acd0700 1 -- 172.21.15.46:0/3787219600 wait complete. 2024-05-09T04:39:37.494 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (2/15)... 2024-05-09T04:39:37.708 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:37 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:37.608+0000 7fcaad685200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T04:39:37.708 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:37 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:37.709+0000 7fcaad685200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T04:39:38.467 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:38 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:38.094+0000 7fcaad685200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T04:39:38.467 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:38 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:38.468+0000 7fcaad685200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T04:39:38.817 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:38 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:38.564+0000 7fcaad685200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T04:39:39.079 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:38 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:38.818+0000 7fcaad685200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T04:39:39.079 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:38 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:38.958+0000 7fcaad685200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T04:39:39.079 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:39 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:39.057+0000 7fcaad685200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T04:39:39.694 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:39 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:39.398+0000 7fcaad685200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T04:39:39.694 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:39 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:39.497+0000 7fcaad685200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T04:39:40.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:40 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/208891245' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T04:39:40.329 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:40 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:40.150+0000 7fcaad685200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T04:39:40.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:40.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:39:40.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "ecf9ae2c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:39:40.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T04:39:40.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T04:39:40.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T04:39:40.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T04:39:40.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:40.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T04:39:40.569 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T04:39:40.569 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-05-09T04:39:40.569 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:39:40.569 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T04:39:40.569 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T04:39:40.569 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:39:40.569 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-05-09T04:39:40.569 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T04:39:40.569 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:40.569 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T04:39:40.569 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T04:39:40.569 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:40.569 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T04:39:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T04:39:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T04:39:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T04:39:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T04:39:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T04:39:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T04:39:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T04:39:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T04:39:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T04:39:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T04:39:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T04:39:40.570 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T04:39:40.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T04:39:40.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T04:39:40.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T04:39:40.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:40.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T04:39:40.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:40.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T04:39:40.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T04:39:40.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:40.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T04:39:40.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-09T04:39:40.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T04:39:40.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T04:39:40.571 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T04:39:25.634513+0000", 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.897+0000 7febd7af5700 1 Processor -- start 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.897+0000 7febd7af5700 1 -- start start 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.897+0000 7febd7af5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd0108af0 0x7febd0108f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:40.572 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.897+0000 7febd7af5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7febd01094f0 con 0x7febd0108af0 2024-05-09T04:39:40.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.898+0000 7febd5891700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd0108af0 0x7febd0108f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:40.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.898+0000 7febd5891700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd0108af0 0x7febd0108f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59400/0 (socket says 172.21.15.46:59400) 2024-05-09T04:39:40.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.898+0000 7febd5891700 1 -- 172.21.15.46:0/759072601 learned_addr learned my addr 172.21.15.46:0/759072601 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:40.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.898+0000 7febd5891700 1 -- 172.21.15.46:0/759072601 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd0109c20 con 0x7febd0108af0 2024-05-09T04:39:40.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.898+0000 7febd5891700 1 --2- 172.21.15.46:0/759072601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd0108af0 0x7febd0108f00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7febcc0098a0 tx=0x7febcc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a477e7ae6639b830 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:40.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.898+0000 7febd488f700 1 -- 172.21.15.46:0/759072601 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febcc004160 con 0x7febd0108af0 2024-05-09T04:39:40.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.898+0000 7febd488f700 1 -- 172.21.15.46:0/759072601 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7febcc004300 con 0x7febd0108af0 2024-05-09T04:39:40.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.898+0000 7febd488f700 1 -- 172.21.15.46:0/759072601 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febcc011af0 con 0x7febd0108af0 2024-05-09T04:39:40.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.899+0000 7febd7af5700 1 -- 172.21.15.46:0/759072601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd0108af0 msgr2=0x7febd0108f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:40.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.899+0000 7febd7af5700 1 --2- 172.21.15.46:0/759072601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd0108af0 0x7febd0108f00 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7febcc0098a0 tx=0x7febcc009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:40.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.899+0000 7febd7af5700 1 -- 172.21.15.46:0/759072601 shutdown_connections 2024-05-09T04:39:40.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.899+0000 7febd7af5700 1 --2- 172.21.15.46:0/759072601 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd0108af0 0x7febd0108f00 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:40.573 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.899+0000 7febd7af5700 1 -- 172.21.15.46:0/759072601 >> 172.21.15.46:0/759072601 conn(0x7febd0104150 msgr2=0x7febd01065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:40.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.899+0000 7febd7af5700 1 -- 172.21.15.46:0/759072601 shutdown_connections 2024-05-09T04:39:40.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.899+0000 7febd7af5700 1 -- 172.21.15.46:0/759072601 wait complete. 2024-05-09T04:39:40.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.900+0000 7febd7af5700 1 Processor -- start 2024-05-09T04:39:40.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.900+0000 7febd7af5700 1 -- start start 2024-05-09T04:39:40.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.900+0000 7febd7af5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd0108af0 0x7febd007d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:40.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.900+0000 7febd7af5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7febd010cfd0 con 0x7febd0108af0 2024-05-09T04:39:40.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.900+0000 7febd5891700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd0108af0 0x7febd007d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:40.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.900+0000 7febd5891700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd0108af0 0x7febd007d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59416/0 (socket says 172.21.15.46:59416) 2024-05-09T04:39:40.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.900+0000 7febd5891700 1 -- 172.21.15.46:0/208891245 learned_addr learned my addr 172.21.15.46:0/208891245 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:40.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.901+0000 7febd5891700 1 -- 172.21.15.46:0/208891245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febd007d750 con 0x7febd0108af0 2024-05-09T04:39:40.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.901+0000 7febd5891700 1 --2- 172.21.15.46:0/208891245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd0108af0 0x7febd007d210 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7febcc011890 tx=0x7febcc011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:40.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.901+0000 7febc6ffd700 1 -- 172.21.15.46:0/208891245 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febcc0039e0 con 0x7febd0108af0 2024-05-09T04:39:40.574 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.901+0000 7febc6ffd700 1 -- 172.21.15.46:0/208891245 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7febcc003b80 con 0x7febd0108af0 2024-05-09T04:39:40.575 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.901+0000 7febd7af5700 1 -- 172.21.15.46:0/208891245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febd01a0d80 con 0x7febd0108af0 2024-05-09T04:39:40.575 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.901+0000 7febc6ffd700 1 -- 172.21.15.46:0/208891245 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7febcc0246e0 con 0x7febd0108af0 2024-05-09T04:39:40.575 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.901+0000 7febd7af5700 1 -- 172.21.15.46:0/208891245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febd01a1200 con 0x7febd0108af0 2024-05-09T04:39:40.575 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.902+0000 7febc6ffd700 1 -- 172.21.15.46:0/208891245 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7febcc022020 con 0x7febd0108af0 2024-05-09T04:39:40.575 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.902+0000 7febc6ffd700 1 -- 172.21.15.46:0/208891245 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7febcc02dca0 con 0x7febd0108af0 2024-05-09T04:39:40.575 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.902+0000 7febd7af5700 1 -- 172.21.15.46:0/208891245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febd0064e20 con 0x7febd0108af0 2024-05-09T04:39:40.575 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.904+0000 7febc6ffd700 1 -- 172.21.15.46:0/208891245 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7febcc01aa70 con 0x7febd0108af0 2024-05-09T04:39:40.575 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.979+0000 7febd7af5700 1 -- 172.21.15.46:0/208891245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7febd01a15a0 con 0x7febd0108af0 2024-05-09T04:39:40.575 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.980+0000 7febc6ffd700 1 -- 172.21.15.46:0/208891245 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7febcc02d3f0 con 0x7febd0108af0 2024-05-09T04:39:40.575 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.981+0000 7febd7af5700 1 -- 172.21.15.46:0/208891245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd0108af0 msgr2=0x7febd007d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:40.575 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.981+0000 7febd7af5700 1 --2- 172.21.15.46:0/208891245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd0108af0 0x7febd007d210 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7febcc011890 tx=0x7febcc011970 comp rx=0 tx=0).stop 2024-05-09T04:39:40.575 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.981+0000 7febd7af5700 1 -- 172.21.15.46:0/208891245 shutdown_connections 2024-05-09T04:39:40.575 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.981+0000 7febd7af5700 1 --2- 172.21.15.46:0/208891245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febd0108af0 0x7febd007d210 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:40.576 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.981+0000 7febd7af5700 1 -- 172.21.15.46:0/208891245 >> 172.21.15.46:0/208891245 conn(0x7febd0104150 msgr2=0x7febd0104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:40.576 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.981+0000 7febd7af5700 1 -- 172.21.15.46:0/208891245 shutdown_connections 2024-05-09T04:39:40.576 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:39.981+0000 7febd7af5700 1 -- 172.21.15.46:0/208891245 wait complete. 2024-05-09T04:39:40.576 INFO:teuthology.orchestra.run.smithi046.stdout:mgr not available, waiting (3/15)... 2024-05-09T04:39:40.829 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:40 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:40.377+0000 7fcaad685200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T04:39:41.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:41 smithi046 ceph-mon[30136]: Activating manager daemon x 2024-05-09T04:39:41.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:41 smithi046 ceph-mon[30136]: mgrmap e2: x(active, starting, since 0.00415293s) 2024-05-09T04:39:41.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:41 smithi046 ceph-mon[30136]: from='mgr.14100 172.21.15.46:0/3380744528' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T04:39:41.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:41 smithi046 ceph-mon[30136]: from='mgr.14100 172.21.15.46:0/3380744528' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T04:39:41.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:41 smithi046 ceph-mon[30136]: from='mgr.14100 172.21.15.46:0/3380744528' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T04:39:41.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:41 smithi046 ceph-mon[30136]: from='mgr.14100 172.21.15.46:0/3380744528' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:39:41.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:41 smithi046 ceph-mon[30136]: from='mgr.14100 172.21.15.46:0/3380744528' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T04:39:41.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:41 smithi046 ceph-mon[30136]: Manager daemon x is now available 2024-05-09T04:39:41.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:41 smithi046 ceph-mon[30136]: from='mgr.14100 172.21.15.46:0/3380744528' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T04:39:41.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:41 smithi046 ceph-mon[30136]: from='mgr.14100 172.21.15.46:0/3380744528' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T04:39:41.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:41 smithi046 ceph-mon[30136]: from='mgr.14100 172.21.15.46:0/3380744528' entity='mgr.x' 2024-05-09T04:39:41.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:41 smithi046 ceph-mon[30136]: from='mgr.14100 172.21.15.46:0/3380744528' entity='mgr.x' 2024-05-09T04:39:41.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:41 smithi046 ceph-mon[30136]: from='mgr.14100 172.21.15.46:0/3380744528' entity='mgr.x' 2024-05-09T04:39:42.691 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:42 smithi046 ceph-mon[30136]: mgrmap e3: x(active, since 1.00883s) 2024-05-09T04:39:43.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:43 smithi046 ceph-mon[30136]: mgrmap e4: x(active, since 2s) 2024-05-09T04:39:46.366 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:46 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/261661848' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-09T04:39:50.059 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:50.062 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:39:50.062 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsid": "ecf9ae2c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:39:50.063 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "health": { 2024-05-09T04:39:50.063 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-09T04:39:50.063 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-09T04:39:50.063 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-09T04:39:50.063 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:50.063 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-09T04:39:50.063 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-09T04:39:50.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 0 2024-05-09T04:39:50.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:39:50.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-09T04:39:50.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "a" 2024-05-09T04:39:50.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:39:50.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "quorum_age": 15, 2024-05-09T04:39:50.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-09T04:39:50.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:50.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-09T04:39:50.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-09T04:39:50.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:50.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-09T04:39:50.064 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:50.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-09T04:39:50.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-09T04:39:50.065 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-09T04:39:50.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-09T04:39:50.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-09T04:39:50.075 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-09T04:39:50.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:50.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-09T04:39:50.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-09T04:39:50.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-09T04:39:50.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-09T04:39:50.076 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-09T04:39:50.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-09T04:39:50.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-09T04:39:50.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-09T04:39:50.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-09T04:39:50.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:50.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-09T04:39:50.077 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:50.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-09T04:39:50.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-09T04:39:50.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:50.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-09T04:39:50.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T04:39:50.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-09T04:39:50.078 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-09T04:39:50.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-09T04:39:50.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-09T04:39:50.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "restful" 2024-05-09T04:39:50.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ], 2024-05-09T04:39:50.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:39:50.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:50.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-09T04:39:50.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-09T04:39:50.079 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "modified": "2024-05-09T04:39:25.634513+0000", 2024-05-09T04:39:50.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-09T04:39:50.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout }, 2024-05-09T04:39:50.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-09T04:39:50.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:39:50.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.747+0000 7f29f6ba7700 1 Processor -- start 2024-05-09T04:39:50.080 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.747+0000 7f29f6ba7700 1 -- start start 2024-05-09T04:39:50.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.747+0000 7f29f6ba7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29f010a060 0x7f29f010a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:50.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.747+0000 7f29f6ba7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29f010aa40 con 0x7f29f010a060 2024-05-09T04:39:50.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.747+0000 7f29f4943700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29f010a060 0x7f29f010a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:50.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.747+0000 7f29f4943700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29f010a060 0x7f29f010a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51248/0 (socket says 172.21.15.46:51248) 2024-05-09T04:39:50.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.747+0000 7f29f4943700 1 -- 172.21.15.46:0/180360227 learned_addr learned my addr 172.21.15.46:0/180360227 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:50.081 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.747+0000 7f29f4943700 1 -- 172.21.15.46:0/180360227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f010abc0 con 0x7f29f010a060 2024-05-09T04:39:50.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.748+0000 7f29f4943700 1 --2- 172.21.15.46:0/180360227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29f010a060 0x7f29f010a470 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f29e40098a0 tx=0x7f29e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666c6976154f8f6a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:50.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.748+0000 7f29ef7fe700 1 -- 172.21.15.46:0/180360227 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29e4004160 con 0x7f29f010a060 2024-05-09T04:39:50.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.748+0000 7f29ef7fe700 1 -- 172.21.15.46:0/180360227 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f29e4004300 con 0x7f29f010a060 2024-05-09T04:39:50.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.748+0000 7f29ef7fe700 1 -- 172.21.15.46:0/180360227 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29e4011af0 con 0x7f29f010a060 2024-05-09T04:39:50.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.749+0000 7f29f6ba7700 1 -- 172.21.15.46:0/180360227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29f010a060 msgr2=0x7f29f010a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:50.082 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.749+0000 7f29f6ba7700 1 --2- 172.21.15.46:0/180360227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29f010a060 0x7f29f010a470 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f29e40098a0 tx=0x7f29e4009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:50.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.749+0000 7f29f6ba7700 1 -- 172.21.15.46:0/180360227 shutdown_connections 2024-05-09T04:39:50.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.749+0000 7f29f6ba7700 1 --2- 172.21.15.46:0/180360227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29f010a060 0x7f29f010a470 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:50.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.749+0000 7f29f6ba7700 1 -- 172.21.15.46:0/180360227 >> 172.21.15.46:0/180360227 conn(0x7f29f0103870 msgr2=0x7f29f0105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:50.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.749+0000 7f29f6ba7700 1 -- 172.21.15.46:0/180360227 shutdown_connections 2024-05-09T04:39:50.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.749+0000 7f29f6ba7700 1 -- 172.21.15.46:0/180360227 wait complete. 2024-05-09T04:39:50.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.749+0000 7f29f6ba7700 1 Processor -- start 2024-05-09T04:39:50.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.750+0000 7f29f6ba7700 1 -- start start 2024-05-09T04:39:50.083 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.750+0000 7f29f6ba7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29f010a060 0x7f29f01a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.750+0000 7f29f6ba7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29f0110110 con 0x7f29f010a060 2024-05-09T04:39:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.750+0000 7f29f4943700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29f010a060 0x7f29f01a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.750+0000 7f29f4943700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29f010a060 0x7f29f01a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51254/0 (socket says 172.21.15.46:51254) 2024-05-09T04:39:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.750+0000 7f29f4943700 1 -- 172.21.15.46:0/261661848 learned_addr learned my addr 172.21.15.46:0/261661848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.750+0000 7f29f4943700 1 -- 172.21.15.46:0/261661848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f01a06d0 con 0x7f29f010a060 2024-05-09T04:39:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.750+0000 7f29f4943700 1 --2- 172.21.15.46:0/261661848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29f010a060 0x7f29f01a0190 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f29e4011890 tx=0x7f29e4011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.750+0000 7f29edffb700 1 -- 172.21.15.46:0/261661848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29e40039e0 con 0x7f29f010a060 2024-05-09T04:39:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.750+0000 7f29edffb700 1 -- 172.21.15.46:0/261661848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f29e4003b80 con 0x7f29f010a060 2024-05-09T04:39:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.750+0000 7f29edffb700 1 -- 172.21.15.46:0/261661848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f29e4024750 con 0x7f29f010a060 2024-05-09T04:39:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.750+0000 7f29f6ba7700 1 -- 172.21.15.46:0/261661848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f01a0960 con 0x7f29f010a060 2024-05-09T04:39:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.751+0000 7f29f6ba7700 1 -- 172.21.15.46:0/261661848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f01a3570 con 0x7f29f010a060 2024-05-09T04:39:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.752+0000 7f29f6ba7700 1 -- 172.21.15.46:0/261661848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f0064e20 con 0x7f29f010a060 2024-05-09T04:39:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.752+0000 7f29edffb700 1 -- 172.21.15.46:0/261661848 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f29e4022020 con 0x7f29f010a060 2024-05-09T04:39:50.084 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.752+0000 7f29edffb700 1 --2- 172.21.15.46:0/261661848 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f29e003e420 0x7f29e0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.752+0000 7f29edffb700 1 -- 172.21.15.46:0/261661848 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f29e4053480 con 0x7f29f010a060 2024-05-09T04:39:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.752+0000 7f29effff700 1 --2- 172.21.15.46:0/261661848 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f29e003e420 0x7f29e0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.753+0000 7f29effff700 1 --2- 172.21.15.46:0/261661848 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f29e003e420 0x7f29e0040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f29dc006fd0 tx=0x7f29dc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:45.758+0000 7f29edffb700 1 -- 172.21.15.46:0/261661848 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f29e40212c0 con 0x7f29f010a060 2024-05-09T04:39:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.084+0000 7f29f6ba7700 1 -- 172.21.15.46:0/261661848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f29f01a3bf0 con 0x7f29f010a060 2024-05-09T04:39:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.085+0000 7f29edffb700 1 -- 172.21.15.46:0/261661848 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f29e402c350 con 0x7f29f010a060 2024-05-09T04:39:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.087+0000 7f29f6ba7700 1 -- 172.21.15.46:0/261661848 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f29e003e420 msgr2=0x7f29e0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.087+0000 7f29f6ba7700 1 --2- 172.21.15.46:0/261661848 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f29e003e420 0x7f29e0040930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f29dc006fd0 tx=0x7f29dc006e40 comp rx=0 tx=0).stop 2024-05-09T04:39:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.087+0000 7f29f6ba7700 1 -- 172.21.15.46:0/261661848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29f010a060 msgr2=0x7f29f01a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.087+0000 7f29f6ba7700 1 --2- 172.21.15.46:0/261661848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29f010a060 0x7f29f01a0190 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f29e4011890 tx=0x7f29e4011970 comp rx=0 tx=0).stop 2024-05-09T04:39:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.087+0000 7f29f6ba7700 1 -- 172.21.15.46:0/261661848 shutdown_connections 2024-05-09T04:39:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.087+0000 7f29f6ba7700 1 --2- 172.21.15.46:0/261661848 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f29e003e420 0x7f29e0040930 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:50.086 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.087+0000 7f29f6ba7700 1 --2- 172.21.15.46:0/261661848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29f010a060 0x7f29f01a0190 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:50.086 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.087+0000 7f29f6ba7700 1 -- 172.21.15.46:0/261661848 >> 172.21.15.46:0/261661848 conn(0x7f29f0103870 msgr2=0x7f29f0104290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:50.086 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.087+0000 7f29f6ba7700 1 -- 172.21.15.46:0/261661848 shutdown_connections 2024-05-09T04:39:50.086 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:46.088+0000 7f29f6ba7700 1 -- 172.21.15.46:0/261661848 wait complete. 2024-05-09T04:39:50.086 INFO:teuthology.orchestra.run.smithi046.stdout:mgr is available 2024-05-09T04:39:51.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:50 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3181519095' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-09T04:39:51.205 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:51.205 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [global] 2024-05-09T04:39:51.205 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout fsid = ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:39:51.205 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-09T04:39:51.205 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-09T04:39:51.205 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-09T04:39:51.205 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-09T04:39:51.205 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-09T04:39:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-09T04:39:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-09T04:39:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 2024-05-09T04:39:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout [osd] 2024-05-09T04:39:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-09T04:39:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-09T04:39:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-09T04:39:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-09T04:39:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-09T04:39:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.430+0000 7fce8db32700 1 Processor -- start 2024-05-09T04:39:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.430+0000 7fce8db32700 1 -- start start 2024-05-09T04:39:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.430+0000 7fce8db32700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce88108a90 0x7fce88108ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.430+0000 7fce8db32700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce88109490 con 0x7fce88108a90 2024-05-09T04:39:51.206 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.430+0000 7fce877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce88108a90 0x7fce88108ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:51.207 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.430+0000 7fce877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce88108a90 0x7fce88108ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51268/0 (socket says 172.21.15.46:51268) 2024-05-09T04:39:51.207 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.430+0000 7fce877fe700 1 -- 172.21.15.46:0/271367729 learned_addr learned my addr 172.21.15.46:0/271367729 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:51.207 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.430+0000 7fce877fe700 1 -- 172.21.15.46:0/271367729 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce88109bc0 con 0x7fce88108a90 2024-05-09T04:39:51.207 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.431+0000 7fce877fe700 1 --2- 172.21.15.46:0/271367729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce88108a90 0x7fce88108ea0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fce780098a0 tx=0x7fce78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1995f9d5245ff64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:51.207 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.431+0000 7fce86ffd700 1 -- 172.21.15.46:0/271367729 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce78004160 con 0x7fce88108a90 2024-05-09T04:39:51.207 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.431+0000 7fce86ffd700 1 -- 172.21.15.46:0/271367729 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fce78004300 con 0x7fce88108a90 2024-05-09T04:39:51.207 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.431+0000 7fce86ffd700 1 -- 172.21.15.46:0/271367729 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce78011af0 con 0x7fce88108a90 2024-05-09T04:39:51.207 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.432+0000 7fce8db32700 1 -- 172.21.15.46:0/271367729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce88108a90 msgr2=0x7fce88108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:51.207 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.432+0000 7fce8db32700 1 --2- 172.21.15.46:0/271367729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce88108a90 0x7fce88108ea0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fce780098a0 tx=0x7fce78009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:51.207 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.432+0000 7fce8db32700 1 -- 172.21.15.46:0/271367729 shutdown_connections 2024-05-09T04:39:51.207 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.432+0000 7fce8db32700 1 --2- 172.21.15.46:0/271367729 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce88108a90 0x7fce88108ea0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:51.207 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.432+0000 7fce8db32700 1 -- 172.21.15.46:0/271367729 >> 172.21.15.46:0/271367729 conn(0x7fce881040f0 msgr2=0x7fce88106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:51.207 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.432+0000 7fce8db32700 1 -- 172.21.15.46:0/271367729 shutdown_connections 2024-05-09T04:39:51.207 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.432+0000 7fce8db32700 1 -- 172.21.15.46:0/271367729 wait complete. 2024-05-09T04:39:51.207 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.432+0000 7fce8db32700 1 Processor -- start 2024-05-09T04:39:51.208 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.432+0000 7fce8db32700 1 -- start start 2024-05-09T04:39:51.208 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.433+0000 7fce8db32700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce88108a90 0x7fce8819dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:51.208 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.433+0000 7fce8db32700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fce8810cf70 con 0x7fce88108a90 2024-05-09T04:39:51.208 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.433+0000 7fce877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce88108a90 0x7fce8819dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:51.208 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.433+0000 7fce877fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce88108a90 0x7fce8819dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51270/0 (socket says 172.21.15.46:51270) 2024-05-09T04:39:51.208 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.433+0000 7fce877fe700 1 -- 172.21.15.46:0/3181519095 learned_addr learned my addr 172.21.15.46:0/3181519095 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:51.208 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.433+0000 7fce877fe700 1 -- 172.21.15.46:0/3181519095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce8819e500 con 0x7fce88108a90 2024-05-09T04:39:51.208 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.433+0000 7fce877fe700 1 --2- 172.21.15.46:0/3181519095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce88108a90 0x7fce8819dfc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fce78009fb0 tx=0x7fce78011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:51.208 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.434+0000 7fce857fa700 1 -- 172.21.15.46:0/3181519095 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce780036a0 con 0x7fce88108a90 2024-05-09T04:39:51.208 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.434+0000 7fce857fa700 1 -- 172.21.15.46:0/3181519095 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fce78003cd0 con 0x7fce88108a90 2024-05-09T04:39:51.208 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.434+0000 7fce8db32700 1 -- 172.21.15.46:0/3181519095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce8819e790 con 0x7fce88108a90 2024-05-09T04:39:51.208 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.434+0000 7fce857fa700 1 -- 172.21.15.46:0/3181519095 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fce7801a430 con 0x7fce88108a90 2024-05-09T04:39:51.208 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.434+0000 7fce8db32700 1 -- 172.21.15.46:0/3181519095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce881a13a0 con 0x7fce88108a90 2024-05-09T04:39:51.208 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.435+0000 7fce857fa700 1 -- 172.21.15.46:0/3181519095 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fce7801a6f0 con 0x7fce88108a90 2024-05-09T04:39:51.209 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.435+0000 7fce8db32700 1 -- 172.21.15.46:0/3181519095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce88064e20 con 0x7fce88108a90 2024-05-09T04:39:51.209 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.435+0000 7fce857fa700 1 --2- 172.21.15.46:0/3181519095 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7fce7403e3d0 0x7fce740408e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:51.209 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.435+0000 7fce857fa700 1 -- 172.21.15.46:0/3181519095 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fce78053660 con 0x7fce88108a90 2024-05-09T04:39:51.209 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.435+0000 7fce7effd700 1 --2- 172.21.15.46:0/3181519095 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7fce7403e3d0 0x7fce740408e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:51.209 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.435+0000 7fce7effd700 1 --2- 172.21.15.46:0/3181519095 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7fce7403e3d0 0x7fce740408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fce70006fd0 tx=0x7fce70006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:51.209 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.440+0000 7fce857fa700 1 -- 172.21.15.46:0/3181519095 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fce78017e90 con 0x7fce88108a90 2024-05-09T04:39:51.209 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.652+0000 7fce8db32700 1 -- 172.21.15.46:0/3181519095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fce8802e850 con 0x7fce88108a90 2024-05-09T04:39:51.209 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.653+0000 7fce857fa700 1 -- 172.21.15.46:0/3181519095 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fce7802c350 con 0x7fce88108a90 2024-05-09T04:39:51.209 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.656+0000 7fce8db32700 1 -- 172.21.15.46:0/3181519095 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7fce7403e3d0 msgr2=0x7fce740408e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:51.209 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.656+0000 7fce8db32700 1 --2- 172.21.15.46:0/3181519095 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7fce7403e3d0 0x7fce740408e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fce70006fd0 tx=0x7fce70006e40 comp rx=0 tx=0).stop 2024-05-09T04:39:51.209 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.656+0000 7fce8db32700 1 -- 172.21.15.46:0/3181519095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce88108a90 msgr2=0x7fce8819dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:51.209 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.656+0000 7fce8db32700 1 --2- 172.21.15.46:0/3181519095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce88108a90 0x7fce8819dfc0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fce78009fb0 tx=0x7fce78011890 comp rx=0 tx=0).stop 2024-05-09T04:39:51.209 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.656+0000 7fce8db32700 1 -- 172.21.15.46:0/3181519095 shutdown_connections 2024-05-09T04:39:51.209 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.656+0000 7fce8db32700 1 --2- 172.21.15.46:0/3181519095 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7fce7403e3d0 0x7fce740408e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:51.209 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.656+0000 7fce8db32700 1 --2- 172.21.15.46:0/3181519095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fce88108a90 0x7fce8819dfc0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:51.210 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.656+0000 7fce8db32700 1 -- 172.21.15.46:0/3181519095 >> 172.21.15.46:0/3181519095 conn(0x7fce881040f0 msgr2=0x7fce88194510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:51.210 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.656+0000 7fce8db32700 1 -- 172.21.15.46:0/3181519095 shutdown_connections 2024-05-09T04:39:51.210 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:50.656+0000 7fce8db32700 1 -- 172.21.15.46:0/3181519095 wait complete. 2024-05-09T04:39:51.210 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling cephadm module... 2024-05-09T04:39:52.156 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:51 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1783570318' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-09T04:39:52.156 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:52 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: ignoring --setuser ceph since I am not root 2024-05-09T04:39:52.157 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:52 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: ignoring --setgroup ceph since I am not root 2024-05-09T04:39:52.157 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:52 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:52.155+0000 7f8620df2200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T04:39:52.460 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:52 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:52.352+0000 7f8620df2200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:39:52.460 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:52 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:52.460+0000 7f8620df2200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T04:39:52.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.614+0000 7f2ce6f7e700 1 Processor -- start 2024-05-09T04:39:52.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.614+0000 7f2ce6f7e700 1 -- start start 2024-05-09T04:39:52.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.614+0000 7f2ce6f7e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce007d6f0 0x7f2ce010eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:52.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.614+0000 7f2ce6f7e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce010f090 con 0x7f2ce007d6f0 2024-05-09T04:39:52.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.615+0000 7f2ce4d1a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce007d6f0 0x7f2ce010eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:52.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.615+0000 7f2ce4d1a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce007d6f0 0x7f2ce010eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51282/0 (socket says 172.21.15.46:51282) 2024-05-09T04:39:52.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.615+0000 7f2ce4d1a700 1 -- 172.21.15.46:0/189938664 learned_addr learned my addr 172.21.15.46:0/189938664 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:52.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.615+0000 7f2ce4d1a700 1 -- 172.21.15.46:0/189938664 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce010f210 con 0x7f2ce007d6f0 2024-05-09T04:39:52.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.615+0000 7f2ce4d1a700 1 --2- 172.21.15.46:0/189938664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce007d6f0 0x7f2ce010eb50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40098a0 tx=0x7f2cd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f604541f54503e18 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:52.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.615+0000 7f2cdf7fe700 1 -- 172.21.15.46:0/189938664 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cd4004160 con 0x7f2ce007d6f0 2024-05-09T04:39:52.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.615+0000 7f2cdf7fe700 1 -- 172.21.15.46:0/189938664 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2cd4004300 con 0x7f2ce007d6f0 2024-05-09T04:39:52.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.615+0000 7f2cdf7fe700 1 -- 172.21.15.46:0/189938664 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cd4011af0 con 0x7f2ce007d6f0 2024-05-09T04:39:52.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.616+0000 7f2ce6f7e700 1 -- 172.21.15.46:0/189938664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce007d6f0 msgr2=0x7f2ce010eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:52.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.616+0000 7f2ce6f7e700 1 --2- 172.21.15.46:0/189938664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce007d6f0 0x7f2ce010eb50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40098a0 tx=0x7f2cd4009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:52.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.616+0000 7f2ce6f7e700 1 -- 172.21.15.46:0/189938664 shutdown_connections 2024-05-09T04:39:52.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.616+0000 7f2ce6f7e700 1 --2- 172.21.15.46:0/189938664 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce007d6f0 0x7f2ce010eb50 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:52.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.616+0000 7f2ce6f7e700 1 -- 172.21.15.46:0/189938664 >> 172.21.15.46:0/189938664 conn(0x7f2ce0104160 msgr2=0x7f2ce01065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:52.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.616+0000 7f2ce6f7e700 1 -- 172.21.15.46:0/189938664 shutdown_connections 2024-05-09T04:39:52.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.616+0000 7f2ce6f7e700 1 -- 172.21.15.46:0/189938664 wait complete. 2024-05-09T04:39:52.565 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.617+0000 7f2ce6f7e700 1 Processor -- start 2024-05-09T04:39:52.566 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.617+0000 7f2ce6f7e700 1 -- start start 2024-05-09T04:39:52.566 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.617+0000 7f2ce6f7e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce007d6f0 0x7f2ce01a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:52.566 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.617+0000 7f2ce6f7e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2ce0110170 con 0x7f2ce007d6f0 2024-05-09T04:39:52.566 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.617+0000 7f2ce4d1a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce007d6f0 0x7f2ce01a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:52.566 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.617+0000 7f2ce4d1a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce007d6f0 0x7f2ce01a0160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51290/0 (socket says 172.21.15.46:51290) 2024-05-09T04:39:52.566 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.617+0000 7f2ce4d1a700 1 -- 172.21.15.46:0/1783570318 learned_addr learned my addr 172.21.15.46:0/1783570318 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:52.566 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.617+0000 7f2ce4d1a700 1 -- 172.21.15.46:0/1783570318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ce01a06a0 con 0x7f2ce007d6f0 2024-05-09T04:39:52.566 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.618+0000 7f2ce4d1a700 1 --2- 172.21.15.46:0/1783570318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce007d6f0 0x7f2ce01a0160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40117b0 tx=0x7f2cd4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:52.566 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.618+0000 7f2cddffb700 1 -- 172.21.15.46:0/1783570318 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cd4003af0 con 0x7f2ce007d6f0 2024-05-09T04:39:52.566 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.618+0000 7f2cddffb700 1 -- 172.21.15.46:0/1783570318 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2cd4020070 con 0x7f2ce007d6f0 2024-05-09T04:39:52.566 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.618+0000 7f2cddffb700 1 -- 172.21.15.46:0/1783570318 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2cd4019630 con 0x7f2ce007d6f0 2024-05-09T04:39:52.566 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.618+0000 7f2ce6f7e700 1 -- 172.21.15.46:0/1783570318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ce01a0930 con 0x7f2ce007d6f0 2024-05-09T04:39:52.566 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.618+0000 7f2ce6f7e700 1 -- 172.21.15.46:0/1783570318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ce01a3540 con 0x7f2ce007d6f0 2024-05-09T04:39:52.566 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.619+0000 7f2cddffb700 1 -- 172.21.15.46:0/1783570318 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f2cd401a9c0 con 0x7f2ce007d6f0 2024-05-09T04:39:52.567 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.619+0000 7f2ce6f7e700 1 -- 172.21.15.46:0/1783570318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ce0064e20 con 0x7f2ce007d6f0 2024-05-09T04:39:52.567 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.619+0000 7f2cddffb700 1 --2- 172.21.15.46:0/1783570318 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f2cd003e420 0x7f2cd0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:52.567 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.619+0000 7f2cddffb700 1 -- 172.21.15.46:0/1783570318 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2cd4053aa0 con 0x7f2ce007d6f0 2024-05-09T04:39:52.567 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.620+0000 7f2cdffff700 1 --2- 172.21.15.46:0/1783570318 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f2cd003e420 0x7f2cd0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:52.567 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.620+0000 7f2cdffff700 1 --2- 172.21.15.46:0/1783570318 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f2cd003e420 0x7f2cd0040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc006fd0 tx=0x7f2ccc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:52.567 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.625+0000 7f2cddffb700 1 -- 172.21.15.46:0/1783570318 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2cd40172b0 con 0x7f2ce007d6f0 2024-05-09T04:39:52.567 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.900+0000 7f2ce6f7e700 1 -- 172.21.15.46:0/1783570318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f2ce01a3850 con 0x7f2ce007d6f0 2024-05-09T04:39:52.567 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.952+0000 7f2cddffb700 1 -- 172.21.15.46:0/1783570318 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f2cd40042a0 con 0x7f2ce007d6f0 2024-05-09T04:39:52.567 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.954+0000 7f2cddffb700 1 -- 172.21.15.46:0/1783570318 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f2cd401a430 con 0x7f2ce007d6f0 2024-05-09T04:39:52.567 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.955+0000 7f2ce6f7e700 1 -- 172.21.15.46:0/1783570318 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f2cd003e420 msgr2=0x7f2cd0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:52.567 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.955+0000 7f2ce6f7e700 1 --2- 172.21.15.46:0/1783570318 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f2cd003e420 0x7f2cd0040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2ccc006fd0 tx=0x7f2ccc006e40 comp rx=0 tx=0).stop 2024-05-09T04:39:52.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.955+0000 7f2ce6f7e700 1 -- 172.21.15.46:0/1783570318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce007d6f0 msgr2=0x7f2ce01a0160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:52.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.955+0000 7f2ce6f7e700 1 --2- 172.21.15.46:0/1783570318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce007d6f0 0x7f2ce01a0160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2cd40117b0 tx=0x7f2cd4011b70 comp rx=0 tx=0).stop 2024-05-09T04:39:52.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.955+0000 7f2ce6f7e700 1 -- 172.21.15.46:0/1783570318 shutdown_connections 2024-05-09T04:39:52.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.955+0000 7f2ce6f7e700 1 --2- 172.21.15.46:0/1783570318 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f2cd003e420 0x7f2cd0040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:52.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.955+0000 7f2ce6f7e700 1 --2- 172.21.15.46:0/1783570318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2ce007d6f0 0x7f2ce01a0160 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:52.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.955+0000 7f2ce6f7e700 1 -- 172.21.15.46:0/1783570318 >> 172.21.15.46:0/1783570318 conn(0x7f2ce0104160 msgr2=0x7f2ce0104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:52.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.955+0000 7f2ce6f7e700 1 -- 172.21.15.46:0/1783570318 shutdown_connections 2024-05-09T04:39:52.568 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:51.955+0000 7f2ce6f7e700 1 -- 172.21.15.46:0/1783570318 wait complete. 2024-05-09T04:39:53.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:52 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1783570318' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-09T04:39:53.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:52 smithi046 ceph-mon[30136]: mgrmap e5: x(active, since 11s) 2024-05-09T04:39:53.080 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:52 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:52.798+0000 7f8620df2200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T04:39:53.080 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:52 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:52.889+0000 7f8620df2200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T04:39:53.901 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:39:53.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-09T04:39:53.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T04:39:53.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T04:39:53.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T04:39:53.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:39:53.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.011+0000 7fc8dc65d700 1 Processor -- start 2024-05-09T04:39:53.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.011+0000 7fc8dc65d700 1 -- start start 2024-05-09T04:39:53.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.012+0000 7fc8dc65d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d407cce0 0x7fc8d410eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:53.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.012+0000 7fc8dc65d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d410f000 con 0x7fc8d407cce0 2024-05-09T04:39:53.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.012+0000 7fc8da3f9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d407cce0 0x7fc8d410eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:53.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.012+0000 7fc8da3f9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d407cce0 0x7fc8d410eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51304/0 (socket says 172.21.15.46:51304) 2024-05-09T04:39:53.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.012+0000 7fc8da3f9700 1 -- 172.21.15.46:0/1452815585 learned_addr learned my addr 172.21.15.46:0/1452815585 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:53.902 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.012+0000 7fc8da3f9700 1 -- 172.21.15.46:0/1452815585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d410f180 con 0x7fc8d407cce0 2024-05-09T04:39:53.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.013+0000 7fc8da3f9700 1 --2- 172.21.15.46:0/1452815585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d407cce0 0x7fc8d410eac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00098a0 tx=0x7fc8d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4114a5c46104eab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:53.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.013+0000 7fc8d93f7700 1 -- 172.21.15.46:0/1452815585 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8d0004160 con 0x7fc8d407cce0 2024-05-09T04:39:53.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.013+0000 7fc8d93f7700 1 -- 172.21.15.46:0/1452815585 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc8d0004300 con 0x7fc8d407cce0 2024-05-09T04:39:53.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.013+0000 7fc8d93f7700 1 -- 172.21.15.46:0/1452815585 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8d0011af0 con 0x7fc8d407cce0 2024-05-09T04:39:53.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.014+0000 7fc8dc65d700 1 -- 172.21.15.46:0/1452815585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d407cce0 msgr2=0x7fc8d410eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:53.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.014+0000 7fc8dc65d700 1 --2- 172.21.15.46:0/1452815585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d407cce0 0x7fc8d410eac0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc8d00098a0 tx=0x7fc8d0009bb0 comp rx=0 tx=0).stop 2024-05-09T04:39:53.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.014+0000 7fc8dc65d700 1 -- 172.21.15.46:0/1452815585 shutdown_connections 2024-05-09T04:39:53.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.014+0000 7fc8dc65d700 1 --2- 172.21.15.46:0/1452815585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d407cce0 0x7fc8d410eac0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:53.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.014+0000 7fc8dc65d700 1 -- 172.21.15.46:0/1452815585 >> 172.21.15.46:0/1452815585 conn(0x7fc8d41040f0 msgr2=0x7fc8d4106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:53.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.014+0000 7fc8dc65d700 1 -- 172.21.15.46:0/1452815585 shutdown_connections 2024-05-09T04:39:53.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.014+0000 7fc8dc65d700 1 -- 172.21.15.46:0/1452815585 wait complete. 2024-05-09T04:39:53.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.015+0000 7fc8dc65d700 1 Processor -- start 2024-05-09T04:39:53.903 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.015+0000 7fc8dc65d700 1 -- start start 2024-05-09T04:39:53.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.015+0000 7fc8dc65d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d407cce0 0x7fc8d41a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:53.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.015+0000 7fc8dc65d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8d41100e0 con 0x7fc8d407cce0 2024-05-09T04:39:53.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.015+0000 7fc8da3f9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d407cce0 0x7fc8d41a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:39:53.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.015+0000 7fc8da3f9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d407cce0 0x7fc8d41a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51310/0 (socket says 172.21.15.46:51310) 2024-05-09T04:39:53.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.015+0000 7fc8da3f9700 1 -- 172.21.15.46:0/3517229173 learned_addr learned my addr 172.21.15.46:0/3517229173 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:39:53.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.015+0000 7fc8da3f9700 1 -- 172.21.15.46:0/3517229173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8d41a06c0 con 0x7fc8d407cce0 2024-05-09T04:39:53.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.016+0000 7fc8da3f9700 1 --2- 172.21.15.46:0/3517229173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d407cce0 0x7fc8d41a0180 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009fb0 tx=0x7fc8d0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:39:53.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.016+0000 7fc8cb7fe700 1 -- 172.21.15.46:0/3517229173 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8d00036a0 con 0x7fc8d407cce0 2024-05-09T04:39:53.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.016+0000 7fc8dc65d700 1 -- 172.21.15.46:0/3517229173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8d41a0950 con 0x7fc8d407cce0 2024-05-09T04:39:53.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.016+0000 7fc8cb7fe700 1 -- 172.21.15.46:0/3517229173 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc8d001a430 con 0x7fc8d407cce0 2024-05-09T04:39:53.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.016+0000 7fc8cb7fe700 1 -- 172.21.15.46:0/3517229173 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8d0019550 con 0x7fc8d407cce0 2024-05-09T04:39:53.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.016+0000 7fc8dc65d700 1 -- 172.21.15.46:0/3517229173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8d41a3560 con 0x7fc8d407cce0 2024-05-09T04:39:53.904 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.017+0000 7fc8dc65d700 1 -- 172.21.15.46:0/3517229173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8d4064e20 con 0x7fc8d407cce0 2024-05-09T04:39:53.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.017+0000 7fc8cb7fe700 1 -- 172.21.15.46:0/3517229173 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fc8d0019800 con 0x7fc8d407cce0 2024-05-09T04:39:53.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.017+0000 7fc8cb7fe700 1 --2- 172.21.15.46:0/3517229173 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7fc8c003e470 0x7fc8c0040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:39:53.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.017+0000 7fc8cb7fe700 1 -- 172.21.15.46:0/3517229173 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc8d0053a60 con 0x7fc8d407cce0 2024-05-09T04:39:53.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.017+0000 7fc8d9bf8700 1 -- 172.21.15.46:0/3517229173 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7fc8c003e470 msgr2=0x7fc8c0040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2738158981 2024-05-09T04:39:53.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.017+0000 7fc8d9bf8700 1 --2- 172.21.15.46:0/3517229173 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7fc8c003e470 0x7fc8c0040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:39:53.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.027+0000 7fc8cb7fe700 1 -- 172.21.15.46:0/3517229173 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc8d0023390 con 0x7fc8d407cce0 2024-05-09T04:39:53.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.218+0000 7fc8d9bf8700 1 -- 172.21.15.46:0/3517229173 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7fc8c003e470 msgr2=0x7fc8c0040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2738158981 2024-05-09T04:39:53.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.218+0000 7fc8d9bf8700 1 --2- 172.21.15.46:0/3517229173 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7fc8c003e470 0x7fc8c0040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:39:53.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.328+0000 7fc8dc65d700 1 -- 172.21.15.46:0/3517229173 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fc8d41a3be0 con 0x7fc8d407cce0 2024-05-09T04:39:53.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.329+0000 7fc8cb7fe700 1 -- 172.21.15.46:0/3517229173 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fc8d0053480 con 0x7fc8d407cce0 2024-05-09T04:39:53.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.333+0000 7fc8dc65d700 1 -- 172.21.15.46:0/3517229173 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7fc8c003e470 msgr2=0x7fc8c0040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:39:53.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.333+0000 7fc8dc65d700 1 --2- 172.21.15.46:0/3517229173 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7fc8c003e470 0x7fc8c0040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:53.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.333+0000 7fc8dc65d700 1 -- 172.21.15.46:0/3517229173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d407cce0 msgr2=0x7fc8d41a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:39:53.905 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.333+0000 7fc8dc65d700 1 --2- 172.21.15.46:0/3517229173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d407cce0 0x7fc8d41a0180 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009fb0 tx=0x7fc8d0011940 comp rx=0 tx=0).stop 2024-05-09T04:39:53.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.334+0000 7fc8dc65d700 1 -- 172.21.15.46:0/3517229173 shutdown_connections 2024-05-09T04:39:53.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.334+0000 7fc8dc65d700 1 --2- 172.21.15.46:0/3517229173 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7fc8c003e470 0x7fc8c0040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:53.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.334+0000 7fc8dc65d700 1 --2- 172.21.15.46:0/3517229173 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc8d407cce0 0x7fc8d41a0180 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:39:53.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.334+0000 7fc8dc65d700 1 -- 172.21.15.46:0/3517229173 >> 172.21.15.46:0/3517229173 conn(0x7fc8d41040f0 msgr2=0x7fc8d4104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:39:53.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.335+0000 7fc8dc65d700 1 -- 172.21.15.46:0/3517229173 shutdown_connections 2024-05-09T04:39:53.906 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:53.336+0000 7fc8dc65d700 1 -- 172.21.15.46:0/3517229173 wait complete. 2024-05-09T04:39:53.906 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-05-09T04:39:53.906 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 5... 2024-05-09T04:39:54.229 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:53 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3517229173' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T04:39:54.229 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:54 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:54.166+0000 7f8620df2200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T04:39:54.531 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:54 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:54.352+0000 7f8620df2200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T04:39:54.829 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:54 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:54.531+0000 7f8620df2200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T04:39:55.147 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:54 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:54.836+0000 7f8620df2200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T04:39:55.147 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:54 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:54.934+0000 7f8620df2200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T04:39:55.147 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:55 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:55.146+0000 7f8620df2200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T04:39:55.579 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:55 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:55.352+0000 7f8620df2200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:39:56.079 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:55 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:55.985+0000 7f8620df2200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T04:39:56.579 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:56 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:56.220+0000 7f8620df2200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T04:39:57.079 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:56 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:56.678+0000 7f8620df2200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T04:39:57.079 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:56 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:56.780+0000 7f8620df2200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T04:39:57.568 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:57.189+0000 7f8620df2200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T04:39:57.568 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:57.567+0000 7f8620df2200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T04:39:57.829 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:57.665+0000 7f8620df2200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T04:39:58.329 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:57.925+0000 7f8620df2200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T04:39:58.329 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:58 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:58.067+0000 7f8620df2200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T04:39:58.329 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:58 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:58.171+0000 7f8620df2200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T04:39:58.829 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:58 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:58.543+0000 7f8620df2200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T04:39:58.829 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:58 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:58.643+0000 7f8620df2200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T04:39:59.566 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:59 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:59.279+0000 7f8620df2200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T04:39:59.566 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:39:59 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:39:59.512+0000 7f8620df2200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T04:39:59.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:59 smithi046 ceph-mon[30136]: Active manager daemon x restarted 2024-05-09T04:39:59.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:59 smithi046 ceph-mon[30136]: Activating manager daemon x 2024-05-09T04:39:59.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:59 smithi046 ceph-mon[30136]: osdmap e2: 0 total, 0 up, 0 in 2024-05-09T04:39:59.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:59 smithi046 ceph-mon[30136]: mgrmap e6: x(active, starting, since 0.00394902s) 2024-05-09T04:39:59.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:59 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:39:59.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:59 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T04:39:59.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:59 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T04:39:59.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:59 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T04:39:59.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:59 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T04:39:59.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:59 smithi046 ceph-mon[30136]: Manager daemon x is now available 2024-05-09T04:39:59.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:59 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' 2024-05-09T04:39:59.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:59 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' 2024-05-09T04:39:59.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:59 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:39:59.832 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:39:59 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:40:00.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:00 smithi046 ceph-mon[30136]: Found migration_current of "None". Setting to last migration. 2024-05-09T04:40:00.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:00 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T04:40:00.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:00 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T04:40:00.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:00 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' 2024-05-09T04:40:00.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:00 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' 2024-05-09T04:40:00.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:00 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:40:00.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:00 smithi046 ceph-mon[30136]: mgrmap e7: x(active, since 1.0073s) 2024-05-09T04:40:01.094 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:40:01.094 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-09T04:40:01.094 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T04:40:01.094 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:40:01.094 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.344+0000 7f89d5fac700 1 Processor -- start 2024-05-09T04:40:01.094 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.344+0000 7f89d5fac700 1 -- start start 2024-05-09T04:40:01.095 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.344+0000 7f89d5fac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d007cca0 0x7f89d010c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:01.095 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.344+0000 7f89d5fac700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f89d010ccf0 con 0x7f89d007cca0 2024-05-09T04:40:01.095 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.345+0000 7f89cf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d007cca0 0x7f89d010c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:01.095 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.345+0000 7f89cf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d007cca0 0x7f89d010c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50202/0 (socket says 172.21.15.46:50202) 2024-05-09T04:40:01.095 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.345+0000 7f89cf7fe700 1 -- 172.21.15.46:0/2807843503 learned_addr learned my addr 172.21.15.46:0/2807843503 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:01.095 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.345+0000 7f89cf7fe700 1 -- 172.21.15.46:0/2807843503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d010ce70 con 0x7f89d007cca0 2024-05-09T04:40:01.095 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.345+0000 7f89cf7fe700 1 --2- 172.21.15.46:0/2807843503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d007cca0 0x7f89d010c720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f89c00098a0 tx=0x7f89c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40272d6d9793728e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:01.095 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.345+0000 7f89ce7fc700 1 -- 172.21.15.46:0/2807843503 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89c0004160 con 0x7f89d007cca0 2024-05-09T04:40:01.095 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.345+0000 7f89ce7fc700 1 -- 172.21.15.46:0/2807843503 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f89c0004300 con 0x7f89d007cca0 2024-05-09T04:40:01.095 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.345+0000 7f89ce7fc700 1 -- 172.21.15.46:0/2807843503 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89c0011af0 con 0x7f89d007cca0 2024-05-09T04:40:01.095 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.346+0000 7f89d5fac700 1 -- 172.21.15.46:0/2807843503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d007cca0 msgr2=0x7f89d010c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:01.095 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.346+0000 7f89d5fac700 1 --2- 172.21.15.46:0/2807843503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d007cca0 0x7f89d010c720 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f89c00098a0 tx=0x7f89c0009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:01.095 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.346+0000 7f89d5fac700 1 -- 172.21.15.46:0/2807843503 shutdown_connections 2024-05-09T04:40:01.096 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.346+0000 7f89d5fac700 1 --2- 172.21.15.46:0/2807843503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d007cca0 0x7f89d010c720 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:01.096 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.346+0000 7f89d5fac700 1 -- 172.21.15.46:0/2807843503 >> 172.21.15.46:0/2807843503 conn(0x7f89d01040f0 msgr2=0x7f89d01065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:01.096 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.346+0000 7f89d5fac700 1 -- 172.21.15.46:0/2807843503 shutdown_connections 2024-05-09T04:40:01.096 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.346+0000 7f89d5fac700 1 -- 172.21.15.46:0/2807843503 wait complete. 2024-05-09T04:40:01.096 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.347+0000 7f89d5fac700 1 Processor -- start 2024-05-09T04:40:01.096 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.347+0000 7f89d5fac700 1 -- start start 2024-05-09T04:40:01.096 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.347+0000 7f89d5fac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d007cca0 0x7f89d019df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:01.096 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.347+0000 7f89d5fac700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f89d010def0 con 0x7f89d007cca0 2024-05-09T04:40:01.096 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.347+0000 7f89cf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d007cca0 0x7f89d019df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:01.096 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.347+0000 7f89cf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d007cca0 0x7f89d019df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50208/0 (socket says 172.21.15.46:50208) 2024-05-09T04:40:01.096 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.348+0000 7f89cf7fe700 1 -- 172.21.15.46:0/3678002061 learned_addr learned my addr 172.21.15.46:0/3678002061 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:01.096 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.348+0000 7f89cf7fe700 1 -- 172.21.15.46:0/3678002061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89d019e4d0 con 0x7f89d007cca0 2024-05-09T04:40:01.096 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.348+0000 7f89cf7fe700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d007cca0 0x7f89d019df90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009fb0 tx=0x7f89c0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:01.096 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.348+0000 7f89ccff9700 1 -- 172.21.15.46:0/3678002061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89c00036a0 con 0x7f89d007cca0 2024-05-09T04:40:01.097 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.348+0000 7f89ccff9700 1 -- 172.21.15.46:0/3678002061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f89c001a430 con 0x7f89d007cca0 2024-05-09T04:40:01.097 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.348+0000 7f89d5fac700 1 -- 172.21.15.46:0/3678002061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89d019e760 con 0x7f89d007cca0 2024-05-09T04:40:01.097 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.348+0000 7f89d5fac700 1 -- 172.21.15.46:0/3678002061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89d01a1370 con 0x7f89d007cca0 2024-05-09T04:40:01.097 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.348+0000 7f89ccff9700 1 -- 172.21.15.46:0/3678002061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f89c0019550 con 0x7f89d007cca0 2024-05-09T04:40:01.097 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.349+0000 7f89ccff9700 1 -- 172.21.15.46:0/3678002061 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f89c0019800 con 0x7f89d007cca0 2024-05-09T04:40:01.097 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.350+0000 7f89ccff9700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f89b803e470 0x7f89b8040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:01.097 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.350+0000 7f89ccff9700 1 -- 172.21.15.46:0/3678002061 --> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f89b8041090 con 0x7f89b803e470 2024-05-09T04:40:01.097 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.350+0000 7f89ceffd700 1 -- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f89b803e470 msgr2=0x7f89b8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2738158981 2024-05-09T04:40:01.097 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.350+0000 7f89ccff9700 1 -- 172.21.15.46:0/3678002061 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f89c0053850 con 0x7f89d007cca0 2024-05-09T04:40:01.097 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.350+0000 7f89ceffd700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f89b803e470 0x7f89b8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:40:01.097 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.551+0000 7f89ceffd700 1 -- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f89b803e470 msgr2=0x7f89b8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2738158981 2024-05-09T04:40:01.097 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.551+0000 7f89ceffd700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f89b803e470 0x7f89b8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:40:01.098 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.952+0000 7f89ceffd700 1 -- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f89b803e470 msgr2=0x7f89b8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2738158981 2024-05-09T04:40:01.098 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:54.952+0000 7f89ceffd700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f89b803e470 0x7f89b8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T04:40:01.098 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.753+0000 7f89ceffd700 1 -- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f89b803e470 msgr2=0x7f89b8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2738158981 2024-05-09T04:40:01.098 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:55.753+0000 7f89ceffd700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f89b803e470 0x7f89b8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T04:40:01.098 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:57.354+0000 7f89ceffd700 1 -- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f89b803e470 msgr2=0x7f89b8040980 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/2738158981 2024-05-09T04:40:01.098 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:57.354+0000 7f89ceffd700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f89b803e470 0x7f89b8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T04:40:01.098 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.516+0000 7f89ccff9700 1 -- 172.21.15.46:0/3678002061 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f89c000fd80 con 0x7f89d007cca0 2024-05-09T04:40:01.098 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.516+0000 7f89ccff9700 1 -- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f89b803e470 msgr2=0x7f89b8040980 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:40:01.098 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:39:59.516+0000 7f89ccff9700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f89b803e470 0x7f89b8040980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:01.098 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.518+0000 7f89ccff9700 1 -- 172.21.15.46:0/3678002061 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f89c0018350 con 0x7f89d007cca0 2024-05-09T04:40:01.098 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.518+0000 7f89ccff9700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f89b8041db0 0x7f89b80441d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:01.098 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.518+0000 7f89ccff9700 1 -- 172.21.15.46:0/3678002061 --> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f89b8041090 con 0x7f89b8041db0 2024-05-09T04:40:01.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.519+0000 7f89ceffd700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f89b8041db0 0x7f89b80441d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:01.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.520+0000 7f89ceffd700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f89b8041db0 0x7f89b80441d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f89c4003a10 tx=0x7f89c40092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:01.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.520+0000 7f89ccff9700 1 -- 172.21.15.46:0/3678002061 <== mgr.14120 v2:172.21.15.46:6800/123334554 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f89b8041090 con 0x7f89b8041db0 2024-05-09T04:40:01.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.529+0000 7f89d5fac700 1 -- 172.21.15.46:0/3678002061 --> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f89bc002630 con 0x7f89b8041db0 2024-05-09T04:40:01.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.529+0000 7f89ccff9700 1 -- 172.21.15.46:0/3678002061 <== mgr.14120 v2:172.21.15.46:6800/123334554 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f89bc002630 con 0x7f89b8041db0 2024-05-09T04:40:01.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.530+0000 7f89d5fac700 1 -- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f89b8041db0 msgr2=0x7f89b80441d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:01.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.530+0000 7f89d5fac700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f89b8041db0 0x7f89b80441d0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f89c4003a10 tx=0x7f89c40092b0 comp rx=0 tx=0).stop 2024-05-09T04:40:01.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.530+0000 7f89d5fac700 1 -- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d007cca0 msgr2=0x7f89d019df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:01.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.530+0000 7f89d5fac700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d007cca0 0x7f89d019df90 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f89c0009fb0 tx=0x7f89c0011940 comp rx=0 tx=0).stop 2024-05-09T04:40:01.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.530+0000 7f89d5fac700 1 -- 172.21.15.46:0/3678002061 shutdown_connections 2024-05-09T04:40:01.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.530+0000 7f89d5fac700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f89b8041db0 0x7f89b80441d0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:01.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.530+0000 7f89d5fac700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f89d007cca0 0x7f89d019df90 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:01.099 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.530+0000 7f89d5fac700 1 --2- 172.21.15.46:0/3678002061 >> [v2:172.21.15.46:6800/2738158981,v1:172.21.15.46:6801/2738158981] conn(0x7f89b803e470 0x7f89b8040980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:01.100 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.530+0000 7f89d5fac700 1 -- 172.21.15.46:0/3678002061 >> 172.21.15.46:0/3678002061 conn(0x7f89d01040f0 msgr2=0x7f89d0104a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:01.100 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.530+0000 7f89d5fac700 1 -- 172.21.15.46:0/3678002061 shutdown_connections 2024-05-09T04:40:01.100 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:00.530+0000 7f89d5fac700 1 -- 172.21.15.46:0/3678002061 wait complete. 2024-05-09T04:40:01.100 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 5 is available 2024-05-09T04:40:01.100 INFO:teuthology.orchestra.run.smithi046.stdout:Setting orchestrator backend to cephadm... 2024-05-09T04:40:01.811 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:01 smithi046 ceph-mon[30136]: [09/May/2024:04:40:00] ENGINE Bus STARTING 2024-05-09T04:40:01.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:01 smithi046 ceph-mon[30136]: [09/May/2024:04:40:00] ENGINE Serving on http://172.21.15.46:8765 2024-05-09T04:40:01.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:01 smithi046 ceph-mon[30136]: [09/May/2024:04:40:00] ENGINE Serving on https://172.21.15.46:7150 2024-05-09T04:40:01.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:01 smithi046 ceph-mon[30136]: [09/May/2024:04:40:00] ENGINE Bus STARTED 2024-05-09T04:40:01.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:01 smithi046 ceph-mon[30136]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T04:40:01.812 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:01 smithi046 ceph-mon[30136]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T04:40:02.364 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.565+0000 7f265759b700 1 Processor -- start 2024-05-09T04:40:02.364 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.565+0000 7f265759b700 1 -- start start 2024-05-09T04:40:02.364 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.566+0000 7f265759b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2650109c80 0x7f265010a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:02.364 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.566+0000 7f265759b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f265010a680 con 0x7f2650109c80 2024-05-09T04:40:02.364 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.566+0000 7f2655337700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2650109c80 0x7f265010a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.566+0000 7f2655337700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2650109c80 0x7f265010a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50274/0 (socket says 172.21.15.46:50274) 2024-05-09T04:40:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.566+0000 7f2655337700 1 -- 172.21.15.46:0/3638634636 learned_addr learned my addr 172.21.15.46:0/3638634636 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.566+0000 7f2655337700 1 -- 172.21.15.46:0/3638634636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f265010ad60 con 0x7f2650109c80 2024-05-09T04:40:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.566+0000 7f2655337700 1 --2- 172.21.15.46:0/3638634636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2650109c80 0x7f265010a090 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f264c0098a0 tx=0x7f264c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63ba1a310f69ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.569+0000 7f2647fff700 1 -- 172.21.15.46:0/3638634636 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f264c004160 con 0x7f2650109c80 2024-05-09T04:40:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.569+0000 7f2647fff700 1 -- 172.21.15.46:0/3638634636 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f264c004300 con 0x7f2650109c80 2024-05-09T04:40:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.569+0000 7f2647fff700 1 -- 172.21.15.46:0/3638634636 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f264c011af0 con 0x7f2650109c80 2024-05-09T04:40:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.570+0000 7f265759b700 1 -- 172.21.15.46:0/3638634636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2650109c80 msgr2=0x7f265010a090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.570+0000 7f265759b700 1 --2- 172.21.15.46:0/3638634636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2650109c80 0x7f265010a090 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f264c0098a0 tx=0x7f264c009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.570+0000 7f265759b700 1 -- 172.21.15.46:0/3638634636 shutdown_connections 2024-05-09T04:40:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.570+0000 7f265759b700 1 --2- 172.21.15.46:0/3638634636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2650109c80 0x7f265010a090 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.570+0000 7f265759b700 1 -- 172.21.15.46:0/3638634636 >> 172.21.15.46:0/3638634636 conn(0x7f26501052e0 msgr2=0x7f2650107770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.570+0000 7f265759b700 1 -- 172.21.15.46:0/3638634636 shutdown_connections 2024-05-09T04:40:02.365 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.570+0000 7f265759b700 1 -- 172.21.15.46:0/3638634636 wait complete. 2024-05-09T04:40:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.571+0000 7f265759b700 1 Processor -- start 2024-05-09T04:40:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.571+0000 7f265759b700 1 -- start start 2024-05-09T04:40:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.572+0000 7f265759b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2650109c80 0x7f26501a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.572+0000 7f265759b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f265010df90 con 0x7f2650109c80 2024-05-09T04:40:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.572+0000 7f2655337700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2650109c80 0x7f26501a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.572+0000 7f2655337700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2650109c80 0x7f26501a6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50282/0 (socket says 172.21.15.46:50282) 2024-05-09T04:40:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.572+0000 7f2655337700 1 -- 172.21.15.46:0/4141925223 learned_addr learned my addr 172.21.15.46:0/4141925223 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.572+0000 7f2655337700 1 -- 172.21.15.46:0/4141925223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26501a6cc0 con 0x7f2650109c80 2024-05-09T04:40:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.572+0000 7f2655337700 1 --2- 172.21.15.46:0/4141925223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2650109c80 0x7f26501a6780 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f264c011890 tx=0x7f264c011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.573+0000 7f26467fc700 1 -- 172.21.15.46:0/4141925223 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f264c0039e0 con 0x7f2650109c80 2024-05-09T04:40:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.573+0000 7f26467fc700 1 -- 172.21.15.46:0/4141925223 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f264c003b80 con 0x7f2650109c80 2024-05-09T04:40:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.573+0000 7f265759b700 1 -- 172.21.15.46:0/4141925223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26501a6f50 con 0x7f2650109c80 2024-05-09T04:40:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.573+0000 7f26467fc700 1 -- 172.21.15.46:0/4141925223 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f264c0246e0 con 0x7f2650109c80 2024-05-09T04:40:02.366 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.573+0000 7f265759b700 1 -- 172.21.15.46:0/4141925223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26501a9b60 con 0x7f2650109c80 2024-05-09T04:40:02.367 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.574+0000 7f26467fc700 1 -- 172.21.15.46:0/4141925223 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f264c022020 con 0x7f2650109c80 2024-05-09T04:40:02.367 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.574+0000 7f265759b700 1 -- 172.21.15.46:0/4141925223 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2650064e20 con 0x7f2650109c80 2024-05-09T04:40:02.367 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.574+0000 7f26467fc700 1 --2- 172.21.15.46:0/4141925223 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f263c03e4c0 0x7f263c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:02.367 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.575+0000 7f26467fc700 1 -- 172.21.15.46:0/4141925223 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f264c053720 con 0x7f2650109c80 2024-05-09T04:40:02.367 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.575+0000 7f2654b36700 1 --2- 172.21.15.46:0/4141925223 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f263c03e4c0 0x7f263c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:02.367 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.575+0000 7f2654b36700 1 --2- 172.21.15.46:0/4141925223 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f263c03e4c0 0x7f263c0409d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2640006fd0 tx=0x7f2640006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:02.367 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.580+0000 7f26467fc700 1 -- 172.21.15.46:0/4141925223 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f264c02c350 con 0x7f2650109c80 2024-05-09T04:40:02.367 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.811+0000 7f265759b700 1 -- 172.21.15.46:0/4141925223 --> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f26501063c0 con 0x7f263c03e4c0 2024-05-09T04:40:02.367 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.817+0000 7f26467fc700 1 -- 172.21.15.46:0/4141925223 <== mgr.14120 v2:172.21.15.46:6800/123334554 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f26501063c0 con 0x7f263c03e4c0 2024-05-09T04:40:02.367 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.820+0000 7f265759b700 1 -- 172.21.15.46:0/4141925223 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f263c03e4c0 msgr2=0x7f263c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:02.367 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.820+0000 7f265759b700 1 --2- 172.21.15.46:0/4141925223 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f263c03e4c0 0x7f263c0409d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2640006fd0 tx=0x7f2640006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:02.367 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.820+0000 7f265759b700 1 -- 172.21.15.46:0/4141925223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2650109c80 msgr2=0x7f26501a6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:02.367 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.820+0000 7f265759b700 1 --2- 172.21.15.46:0/4141925223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2650109c80 0x7f26501a6780 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f264c011890 tx=0x7f264c011970 comp rx=0 tx=0).stop 2024-05-09T04:40:02.368 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.820+0000 7f265759b700 1 -- 172.21.15.46:0/4141925223 shutdown_connections 2024-05-09T04:40:02.368 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.820+0000 7f265759b700 1 --2- 172.21.15.46:0/4141925223 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f263c03e4c0 0x7f263c0409d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:02.368 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.820+0000 7f265759b700 1 --2- 172.21.15.46:0/4141925223 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2650109c80 0x7f26501a6780 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:02.368 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.820+0000 7f265759b700 1 -- 172.21.15.46:0/4141925223 >> 172.21.15.46:0/4141925223 conn(0x7f26501052e0 msgr2=0x7f2650105c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:02.368 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.820+0000 7f265759b700 1 -- 172.21.15.46:0/4141925223 shutdown_connections 2024-05-09T04:40:02.368 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:01.820+0000 7f265759b700 1 -- 172.21.15.46:0/4141925223 wait complete. 2024-05-09T04:40:02.675 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:02 smithi046 ceph-mon[30136]: mgrmap e8: x(active, since 2s) 2024-05-09T04:40:02.675 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:02 smithi046 ceph-mon[30136]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:02.675 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:02 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' 2024-05-09T04:40:02.675 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:02 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:40:03.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-09T04:40:03.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.793+0000 7fe88c8be700 1 Processor -- start 2024-05-09T04:40:03.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.793+0000 7fe88c8be700 1 -- start start 2024-05-09T04:40:03.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.793+0000 7fe88c8be700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe88807d570 0x7fe88810c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:03.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.793+0000 7fe88c8be700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe88810cd30 con 0x7fe88807d570 2024-05-09T04:40:03.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.793+0000 7fe88659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe88807d570 0x7fe88810c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:03.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.793+0000 7fe88659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe88807d570 0x7fe88810c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50286/0 (socket says 172.21.15.46:50286) 2024-05-09T04:40:03.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.793+0000 7fe88659c700 1 -- 172.21.15.46:0/1675977716 learned_addr learned my addr 172.21.15.46:0/1675977716 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:03.621 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.794+0000 7fe88659c700 1 -- 172.21.15.46:0/1675977716 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88810ceb0 con 0x7fe88807d570 2024-05-09T04:40:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.794+0000 7fe88659c700 1 --2- 172.21.15.46:0/1675977716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe88807d570 0x7fe88810c720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe8780098a0 tx=0x7fe878009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1442f552206fe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.794+0000 7fe88559a700 1 -- 172.21.15.46:0/1675977716 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe878004160 con 0x7fe88807d570 2024-05-09T04:40:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.794+0000 7fe88559a700 1 -- 172.21.15.46:0/1675977716 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe878004300 con 0x7fe88807d570 2024-05-09T04:40:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.794+0000 7fe88559a700 1 -- 172.21.15.46:0/1675977716 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe878011af0 con 0x7fe88807d570 2024-05-09T04:40:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.795+0000 7fe88c8be700 1 -- 172.21.15.46:0/1675977716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe88807d570 msgr2=0x7fe88810c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.795+0000 7fe88c8be700 1 --2- 172.21.15.46:0/1675977716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe88807d570 0x7fe88810c720 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe8780098a0 tx=0x7fe878009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.795+0000 7fe88c8be700 1 -- 172.21.15.46:0/1675977716 shutdown_connections 2024-05-09T04:40:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.795+0000 7fe88c8be700 1 --2- 172.21.15.46:0/1675977716 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe88807d570 0x7fe88810c720 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.795+0000 7fe88c8be700 1 -- 172.21.15.46:0/1675977716 >> 172.21.15.46:0/1675977716 conn(0x7fe888104150 msgr2=0x7fe8881065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.795+0000 7fe88c8be700 1 -- 172.21.15.46:0/1675977716 shutdown_connections 2024-05-09T04:40:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.795+0000 7fe88c8be700 1 -- 172.21.15.46:0/1675977716 wait complete. 2024-05-09T04:40:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.796+0000 7fe88c8be700 1 Processor -- start 2024-05-09T04:40:03.622 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.796+0000 7fe88c8be700 1 -- start start 2024-05-09T04:40:03.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.796+0000 7fe88c8be700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe88807d570 0x7fe88819df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:03.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.796+0000 7fe88c8be700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe88810df30 con 0x7fe88807d570 2024-05-09T04:40:03.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.796+0000 7fe88659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe88807d570 0x7fe88819df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:03.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.796+0000 7fe88659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe88807d570 0x7fe88819df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50294/0 (socket says 172.21.15.46:50294) 2024-05-09T04:40:03.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.796+0000 7fe88659c700 1 -- 172.21.15.46:0/324234234 learned_addr learned my addr 172.21.15.46:0/324234234 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:03.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.796+0000 7fe88659c700 1 -- 172.21.15.46:0/324234234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88819e470 con 0x7fe88807d570 2024-05-09T04:40:03.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.797+0000 7fe88659c700 1 --2- 172.21.15.46:0/324234234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe88807d570 0x7fe88819df30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe878009fb0 tx=0x7fe878011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:03.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.797+0000 7fe8777fe700 1 -- 172.21.15.46:0/324234234 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe8780036a0 con 0x7fe88807d570 2024-05-09T04:40:03.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.797+0000 7fe8777fe700 1 -- 172.21.15.46:0/324234234 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe87801a430 con 0x7fe88807d570 2024-05-09T04:40:03.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.797+0000 7fe88c8be700 1 -- 172.21.15.46:0/324234234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88819e700 con 0x7fe88807d570 2024-05-09T04:40:03.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.797+0000 7fe8777fe700 1 -- 172.21.15.46:0/324234234 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe878019550 con 0x7fe88807d570 2024-05-09T04:40:03.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.797+0000 7fe88c8be700 1 -- 172.21.15.46:0/324234234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8881a1310 con 0x7fe88807d570 2024-05-09T04:40:03.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.798+0000 7fe88c8be700 1 -- 172.21.15.46:0/324234234 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe888064e20 con 0x7fe88807d570 2024-05-09T04:40:03.623 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.798+0000 7fe8777fe700 1 -- 172.21.15.46:0/324234234 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fe878019850 con 0x7fe88807d570 2024-05-09T04:40:03.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.798+0000 7fe8777fe700 1 --2- 172.21.15.46:0/324234234 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fe87003e420 0x7fe870040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:03.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.798+0000 7fe8777fe700 1 -- 172.21.15.46:0/324234234 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe8780538d0 con 0x7fe88807d570 2024-05-09T04:40:03.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.798+0000 7fe885d9b700 1 --2- 172.21.15.46:0/324234234 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fe87003e420 0x7fe870040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:03.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.799+0000 7fe885d9b700 1 --2- 172.21.15.46:0/324234234 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fe87003e420 0x7fe870040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe87c006fd0 tx=0x7fe87c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:03.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:02.804+0000 7fe8777fe700 1 -- 172.21.15.46:0/324234234 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe878017e90 con 0x7fe88807d570 2024-05-09T04:40:03.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:03.021+0000 7fe88c8be700 1 -- 172.21.15.46:0/324234234 --> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7fe8881051a0 con 0x7fe87003e420 2024-05-09T04:40:03.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:03.022+0000 7fe8777fe700 1 -- 172.21.15.46:0/324234234 <== mgr.14120 v2:172.21.15.46:6800/123334554 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7fe8881051a0 con 0x7fe87003e420 2024-05-09T04:40:03.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:03.024+0000 7fe88c8be700 1 -- 172.21.15.46:0/324234234 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fe87003e420 msgr2=0x7fe870040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:03.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:03.024+0000 7fe88c8be700 1 --2- 172.21.15.46:0/324234234 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fe87003e420 0x7fe870040930 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe87c006fd0 tx=0x7fe87c006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:03.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:03.024+0000 7fe88c8be700 1 -- 172.21.15.46:0/324234234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe88807d570 msgr2=0x7fe88819df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:03.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:03.024+0000 7fe88c8be700 1 --2- 172.21.15.46:0/324234234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe88807d570 0x7fe88819df30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe878009fb0 tx=0x7fe878011940 comp rx=0 tx=0).stop 2024-05-09T04:40:03.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:03.025+0000 7fe88c8be700 1 -- 172.21.15.46:0/324234234 shutdown_connections 2024-05-09T04:40:03.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:03.025+0000 7fe88c8be700 1 --2- 172.21.15.46:0/324234234 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fe87003e420 0x7fe870040930 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:03.624 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:03.025+0000 7fe88c8be700 1 --2- 172.21.15.46:0/324234234 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe88807d570 0x7fe88819df30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:03.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:03.025+0000 7fe88c8be700 1 -- 172.21.15.46:0/324234234 >> 172.21.15.46:0/324234234 conn(0x7fe888104150 msgr2=0x7fe888104a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:03.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:03.025+0000 7fe88c8be700 1 -- 172.21.15.46:0/324234234 shutdown_connections 2024-05-09T04:40:03.625 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:03.025+0000 7fe88c8be700 1 -- 172.21.15.46:0/324234234 wait complete. 2024-05-09T04:40:03.625 INFO:teuthology.orchestra.run.smithi046.stdout:Generating ssh key... 2024-05-09T04:40:03.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:03 smithi046 ceph-mon[30136]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:05.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: Generating public/private rsa key pair. 2024-05-09T04:40:05.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: Your identification has been saved in /tmp/tmp_v88rfd1/key. 2024-05-09T04:40:05.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: Your public key has been saved in /tmp/tmp_v88rfd1/key.pub. 2024-05-09T04:40:05.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: The key fingerprint is: 2024-05-09T04:40:05.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: SHA256:VzJUoLD/EhbT4y7OuYOCFp2fdnfOGhFy6puoVJv0HUA ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:40:05.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: The key's randomart image is: 2024-05-09T04:40:05.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: +---[RSA 3072]----+ 2024-05-09T04:40:05.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: | . E oo. | 2024-05-09T04:40:05.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: | + + | 2024-05-09T04:40:05.277 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: | . * B . | 2024-05-09T04:40:05.278 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: | . O * | 2024-05-09T04:40:05.278 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: | . .o S = | 2024-05-09T04:40:05.278 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: | . oo * * o | 2024-05-09T04:40:05.278 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: | o..oo= = | 2024-05-09T04:40:05.278 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: | o.. =+o*.o. | 2024-05-09T04:40:05.278 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: | . .+..B+.+o | 2024-05-09T04:40:05.278 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: +----[SHA256]-----+ 2024-05-09T04:40:05.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.228+0000 7f1c724d2700 1 Processor -- start 2024-05-09T04:40:05.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.228+0000 7f1c724d2700 1 -- start start 2024-05-09T04:40:05.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.228+0000 7f1c724d2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c07d570 0x7f1c6c10c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:05.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.228+0000 7f1c724d2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c10cd60 con 0x7f1c6c07d570 2024-05-09T04:40:05.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.229+0000 7f1c6bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c07d570 0x7f1c6c10c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:05.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.229+0000 7f1c6bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c07d570 0x7f1c6c10c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59340/0 (socket says 172.21.15.46:59340) 2024-05-09T04:40:05.560 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.229+0000 7f1c6bfff700 1 -- 172.21.15.46:0/3471956274 learned_addr learned my addr 172.21.15.46:0/3471956274 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:05.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.229+0000 7f1c6bfff700 1 -- 172.21.15.46:0/3471956274 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c10cee0 con 0x7f1c6c07d570 2024-05-09T04:40:05.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.229+0000 7f1c6bfff700 1 --2- 172.21.15.46:0/3471956274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c07d570 0x7f1c6c10c750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0098a0 tx=0x7f1c5c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c4406237ad5bf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:05.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.229+0000 7f1c6b7fe700 1 -- 172.21.15.46:0/3471956274 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c5c004160 con 0x7f1c6c07d570 2024-05-09T04:40:05.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.229+0000 7f1c6b7fe700 1 -- 172.21.15.46:0/3471956274 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1c5c004300 con 0x7f1c6c07d570 2024-05-09T04:40:05.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.229+0000 7f1c6b7fe700 1 -- 172.21.15.46:0/3471956274 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c5c011af0 con 0x7f1c6c07d570 2024-05-09T04:40:05.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.230+0000 7f1c724d2700 1 -- 172.21.15.46:0/3471956274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c07d570 msgr2=0x7f1c6c10c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:05.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.230+0000 7f1c724d2700 1 --2- 172.21.15.46:0/3471956274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c07d570 0x7f1c6c10c750 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0098a0 tx=0x7f1c5c009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:05.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.230+0000 7f1c724d2700 1 -- 172.21.15.46:0/3471956274 shutdown_connections 2024-05-09T04:40:05.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.230+0000 7f1c724d2700 1 --2- 172.21.15.46:0/3471956274 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c07d570 0x7f1c6c10c750 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:05.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.230+0000 7f1c724d2700 1 -- 172.21.15.46:0/3471956274 >> 172.21.15.46:0/3471956274 conn(0x7f1c6c1041a0 msgr2=0x7f1c6c106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:05.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.230+0000 7f1c724d2700 1 -- 172.21.15.46:0/3471956274 shutdown_connections 2024-05-09T04:40:05.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.230+0000 7f1c724d2700 1 -- 172.21.15.46:0/3471956274 wait complete. 2024-05-09T04:40:05.561 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.231+0000 7f1c724d2700 1 Processor -- start 2024-05-09T04:40:05.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.231+0000 7f1c724d2700 1 -- start start 2024-05-09T04:40:05.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.231+0000 7f1c724d2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c07d570 0x7f1c6c19def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:05.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.231+0000 7f1c724d2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1c6c10def0 con 0x7f1c6c07d570 2024-05-09T04:40:05.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.231+0000 7f1c6bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c07d570 0x7f1c6c19def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:05.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.231+0000 7f1c6bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c07d570 0x7f1c6c19def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59352/0 (socket says 172.21.15.46:59352) 2024-05-09T04:40:05.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.231+0000 7f1c6bfff700 1 -- 172.21.15.46:0/1968226186 learned_addr learned my addr 172.21.15.46:0/1968226186 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:05.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.232+0000 7f1c6bfff700 1 -- 172.21.15.46:0/1968226186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c6c19e430 con 0x7f1c6c07d570 2024-05-09T04:40:05.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.232+0000 7f1c6bfff700 1 --2- 172.21.15.46:0/1968226186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c07d570 0x7f1c6c19def0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0117b0 tx=0x7f1c5c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:05.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.232+0000 7f1c69ffb700 1 -- 172.21.15.46:0/1968226186 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c5c003af0 con 0x7f1c6c07d570 2024-05-09T04:40:05.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.232+0000 7f1c69ffb700 1 -- 172.21.15.46:0/1968226186 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f1c5c020070 con 0x7f1c6c07d570 2024-05-09T04:40:05.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.232+0000 7f1c724d2700 1 -- 172.21.15.46:0/1968226186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c6c19e6c0 con 0x7f1c6c07d570 2024-05-09T04:40:05.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.232+0000 7f1c69ffb700 1 -- 172.21.15.46:0/1968226186 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1c5c019630 con 0x7f1c6c07d570 2024-05-09T04:40:05.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.232+0000 7f1c724d2700 1 -- 172.21.15.46:0/1968226186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c6c1a12d0 con 0x7f1c6c07d570 2024-05-09T04:40:05.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.233+0000 7f1c724d2700 1 -- 172.21.15.46:0/1968226186 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c6c064e20 con 0x7f1c6c07d570 2024-05-09T04:40:05.562 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.233+0000 7f1c69ffb700 1 -- 172.21.15.46:0/1968226186 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f1c5c0198a0 con 0x7f1c6c07d570 2024-05-09T04:40:05.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.233+0000 7f1c69ffb700 1 --2- 172.21.15.46:0/1968226186 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f1c5803e4c0 0x7f1c580409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:05.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.233+0000 7f1c69ffb700 1 -- 172.21.15.46:0/1968226186 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f1c5c053aa0 con 0x7f1c6c07d570 2024-05-09T04:40:05.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.233+0000 7f1c63fff700 1 --2- 172.21.15.46:0/1968226186 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f1c5803e4c0 0x7f1c580409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:05.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.234+0000 7f1c63fff700 1 --2- 172.21.15.46:0/1968226186 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f1c5803e4c0 0x7f1c580409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c54006fd0 tx=0x7f1c54006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:05.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.239+0000 7f1c69ffb700 1 -- 172.21.15.46:0/1968226186 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1c5c016260 con 0x7f1c6c07d570 2024-05-09T04:40:05.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.452+0000 7f1c724d2700 1 -- 172.21.15.46:0/1968226186 --> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f1c6c105150 con 0x7f1c5803e4c0 2024-05-09T04:40:05.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.969+0000 7f1c69ffb700 1 -- 172.21.15.46:0/1968226186 <== mgr.14120 v2:172.21.15.46:6800/123334554 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f1c6c105150 con 0x7f1c5803e4c0 2024-05-09T04:40:05.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.971+0000 7f1c724d2700 1 -- 172.21.15.46:0/1968226186 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f1c5803e4c0 msgr2=0x7f1c580409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:05.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.971+0000 7f1c724d2700 1 --2- 172.21.15.46:0/1968226186 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f1c5803e4c0 0x7f1c580409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1c54006fd0 tx=0x7f1c54006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:05.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.971+0000 7f1c724d2700 1 -- 172.21.15.46:0/1968226186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c07d570 msgr2=0x7f1c6c19def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:05.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.971+0000 7f1c724d2700 1 --2- 172.21.15.46:0/1968226186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c07d570 0x7f1c6c19def0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c0117b0 tx=0x7f1c5c011b70 comp rx=0 tx=0).stop 2024-05-09T04:40:05.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.971+0000 7f1c724d2700 1 -- 172.21.15.46:0/1968226186 shutdown_connections 2024-05-09T04:40:05.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.971+0000 7f1c724d2700 1 --2- 172.21.15.46:0/1968226186 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f1c5803e4c0 0x7f1c580409d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:05.563 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.971+0000 7f1c724d2700 1 --2- 172.21.15.46:0/1968226186 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c6c07d570 0x7f1c6c19def0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:05.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.971+0000 7f1c724d2700 1 -- 172.21.15.46:0/1968226186 >> 172.21.15.46:0/1968226186 conn(0x7f1c6c1041a0 msgr2=0x7f1c6c104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:05.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.971+0000 7f1c724d2700 1 -- 172.21.15.46:0/1968226186 shutdown_connections 2024-05-09T04:40:05.564 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:04.971+0000 7f1c724d2700 1 -- 172.21.15.46:0/1968226186 wait complete. 2024-05-09T04:40:06.213 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:05 smithi046 ceph-mon[30136]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:06.213 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:05 smithi046 ceph-mon[30136]: Generating ssh key... 2024-05-09T04:40:06.213 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:05 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' 2024-05-09T04:40:06.213 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:05 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' 2024-05-09T04:40:06.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk4Fe4IpocKCoHeDQCM+pC9CSjPYdzd9hrME1fWXf++9cR05pC9ZB1WAP5uC0wfyrpLrW8aiGzI7nY+jRPOTu135ofr2ikGCUDNdYhNtbW8XCAkENZXqqTm368KeEJ3Rz1Ou1avaUfwVkfECh8MDiZbpO9NPt0IQzp9zBeHAy+cF3bn1lrEWVmXmSngyVXWsElC0ptQdv62ZjouiOAd6hyvSjh3i662wwtx0uaNwl87OAkkg84j8b6K7GsojWd68NTF82/WDN0NA+BqN4nth+gQwee858X8xexmTsBvdEPbBxrCuf8J5AllzIOEl5AdUMMZdpBc3yA9UdY0omtZeja/dOaTQsnFqeHrwsjnmkvfC70PcHWo8jkw6k0Ua9AK2dV7hcoVHxsgvqIxMz05Z50i5P+lhCPoDPXaavR1cnmuX7LIGITvBqi6M3sb9/lrPIqJ+m0pFhB5AhKovscrLZXPxDUavTxPf2dSbx/vwTnVnFaxwcm5pTtggV0J7dZeXc= ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:40:06.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.984+0000 7f401b093700 1 Processor -- start 2024-05-09T04:40:06.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.984+0000 7f401b093700 1 -- start start 2024-05-09T04:40:06.742 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.984+0000 7f401b093700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4014108b00 0x7f4014108f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:06.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.984+0000 7f401b093700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4014109500 con 0x7f4014108b00 2024-05-09T04:40:06.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.984+0000 7f4018e2f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4014108b00 0x7f4014108f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:06.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.984+0000 7f4018e2f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4014108b00 0x7f4014108f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59364/0 (socket says 172.21.15.46:59364) 2024-05-09T04:40:06.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.984+0000 7f4018e2f700 1 -- 172.21.15.46:0/114605708 learned_addr learned my addr 172.21.15.46:0/114605708 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:06.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.985+0000 7f4018e2f700 1 -- 172.21.15.46:0/114605708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4014109c30 con 0x7f4014108b00 2024-05-09T04:40:06.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.985+0000 7f4018e2f700 1 --2- 172.21.15.46:0/114605708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4014108b00 0x7f4014108f10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f40080098a0 tx=0x7f4008009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5134bc6775de5ee server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:06.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.985+0000 7f40137fe700 1 -- 172.21.15.46:0/114605708 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4008004160 con 0x7f4014108b00 2024-05-09T04:40:06.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.985+0000 7f40137fe700 1 -- 172.21.15.46:0/114605708 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4008004300 con 0x7f4014108b00 2024-05-09T04:40:06.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.985+0000 7f40137fe700 1 -- 172.21.15.46:0/114605708 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4008011af0 con 0x7f4014108b00 2024-05-09T04:40:06.743 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.986+0000 7f401b093700 1 -- 172.21.15.46:0/114605708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4014108b00 msgr2=0x7f4014108f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:06.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.986+0000 7f401b093700 1 --2- 172.21.15.46:0/114605708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4014108b00 0x7f4014108f10 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f40080098a0 tx=0x7f4008009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:06.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.986+0000 7f401b093700 1 -- 172.21.15.46:0/114605708 shutdown_connections 2024-05-09T04:40:06.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.986+0000 7f401b093700 1 --2- 172.21.15.46:0/114605708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4014108b00 0x7f4014108f10 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:06.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.986+0000 7f401b093700 1 -- 172.21.15.46:0/114605708 >> 172.21.15.46:0/114605708 conn(0x7f4014104140 msgr2=0x7f40141065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:06.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.986+0000 7f401b093700 1 -- 172.21.15.46:0/114605708 shutdown_connections 2024-05-09T04:40:06.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.986+0000 7f401b093700 1 -- 172.21.15.46:0/114605708 wait complete. 2024-05-09T04:40:06.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.987+0000 7f401b093700 1 Processor -- start 2024-05-09T04:40:06.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.987+0000 7f401b093700 1 -- start start 2024-05-09T04:40:06.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.987+0000 7f401b093700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4014108b00 0x7f401407d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:06.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.987+0000 7f401b093700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f401410cfe0 con 0x7f4014108b00 2024-05-09T04:40:06.744 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.987+0000 7f4018e2f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4014108b00 0x7f401407d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:06.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.987+0000 7f4018e2f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4014108b00 0x7f401407d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59368/0 (socket says 172.21.15.46:59368) 2024-05-09T04:40:06.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.987+0000 7f4018e2f700 1 -- 172.21.15.46:0/1672867343 learned_addr learned my addr 172.21.15.46:0/1672867343 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:06.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.987+0000 7f4018e2f700 1 -- 172.21.15.46:0/1672867343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f401407d790 con 0x7f4014108b00 2024-05-09T04:40:06.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.987+0000 7f4018e2f700 1 --2- 172.21.15.46:0/1672867343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4014108b00 0x7f401407d250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4008009fb0 tx=0x7f4008011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:06.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.988+0000 7f4011ffb700 1 -- 172.21.15.46:0/1672867343 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40080036a0 con 0x7f4014108b00 2024-05-09T04:40:06.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.988+0000 7f4011ffb700 1 -- 172.21.15.46:0/1672867343 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4008003cd0 con 0x7f4014108b00 2024-05-09T04:40:06.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.988+0000 7f4011ffb700 1 -- 172.21.15.46:0/1672867343 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f400801a430 con 0x7f4014108b00 2024-05-09T04:40:06.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.988+0000 7f401b093700 1 -- 172.21.15.46:0/1672867343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40141a0d10 con 0x7f4014108b00 2024-05-09T04:40:06.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.988+0000 7f401b093700 1 -- 172.21.15.46:0/1672867343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40141a11f0 con 0x7f4014108b00 2024-05-09T04:40:06.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.989+0000 7f4011ffb700 1 -- 172.21.15.46:0/1672867343 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f4008003840 con 0x7f4014108b00 2024-05-09T04:40:06.745 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.989+0000 7f401b093700 1 -- 172.21.15.46:0/1672867343 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4014064e20 con 0x7f4014108b00 2024-05-09T04:40:06.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.989+0000 7f4011ffb700 1 --2- 172.21.15.46:0/1672867343 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f400403e470 0x7f4004040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:06.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.989+0000 7f4011ffb700 1 -- 172.21.15.46:0/1672867343 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4008053600 con 0x7f4014108b00 2024-05-09T04:40:06.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.989+0000 7f4013fff700 1 --2- 172.21.15.46:0/1672867343 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f400403e470 0x7f4004040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:06.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.990+0000 7f4013fff700 1 --2- 172.21.15.46:0/1672867343 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f400403e470 0x7f4004040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4000006fd0 tx=0x7f4000006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:06.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:05.995+0000 7f4011ffb700 1 -- 172.21.15.46:0/1672867343 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4008024440 con 0x7f4014108b00 2024-05-09T04:40:06.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.212+0000 7f401b093700 1 -- 172.21.15.46:0/1672867343 --> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f4014105110 con 0x7f400403e470 2024-05-09T04:40:06.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.213+0000 7f4011ffb700 1 -- 172.21.15.46:0/1672867343 <== mgr.14120 v2:172.21.15.46:6800/123334554 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f4014105110 con 0x7f400403e470 2024-05-09T04:40:06.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.216+0000 7f401b093700 1 -- 172.21.15.46:0/1672867343 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f400403e470 msgr2=0x7f4004040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:06.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.216+0000 7f401b093700 1 --2- 172.21.15.46:0/1672867343 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f400403e470 0x7f4004040980 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4000006fd0 tx=0x7f4000006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:06.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.216+0000 7f401b093700 1 -- 172.21.15.46:0/1672867343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4014108b00 msgr2=0x7f401407d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:06.746 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.216+0000 7f401b093700 1 --2- 172.21.15.46:0/1672867343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4014108b00 0x7f401407d250 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4008009fb0 tx=0x7f4008011890 comp rx=0 tx=0).stop 2024-05-09T04:40:06.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.216+0000 7f401b093700 1 -- 172.21.15.46:0/1672867343 shutdown_connections 2024-05-09T04:40:06.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.216+0000 7f401b093700 1 --2- 172.21.15.46:0/1672867343 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f400403e470 0x7f4004040980 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:06.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.216+0000 7f401b093700 1 --2- 172.21.15.46:0/1672867343 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4014108b00 0x7f401407d250 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:06.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.216+0000 7f401b093700 1 -- 172.21.15.46:0/1672867343 >> 172.21.15.46:0/1672867343 conn(0x7f4014104140 msgr2=0x7f4014104a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:06.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.216+0000 7f401b093700 1 -- 172.21.15.46:0/1672867343 shutdown_connections 2024-05-09T04:40:06.747 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:06.216+0000 7f401b093700 1 -- 172.21.15.46:0/1672867343 wait complete. 2024-05-09T04:40:06.747 INFO:teuthology.orchestra.run.smithi046.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-09T04:40:06.747 INFO:teuthology.orchestra.run.smithi046.stdout:Adding key to root@localhost authorized_keys... 2024-05-09T04:40:06.747 INFO:teuthology.orchestra.run.smithi046.stdout:Adding host smithi046... 2024-05-09T04:40:07.170 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:07 smithi046 ceph-mon[30136]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:08.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:08 smithi046 ceph-mon[30136]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:08.308 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:08 smithi046 ceph-mon[30136]: Deploying cephadm binary to smithi046 2024-05-09T04:40:12.183 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Added host 'smithi046' with addr '172.21.15.46' 2024-05-09T04:40:12.183 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.193+0000 7f61ecee9700 1 Processor -- start 2024-05-09T04:40:12.183 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.193+0000 7f61ecee9700 1 -- start start 2024-05-09T04:40:12.183 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.194+0000 7f61ecee9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61e807cca0 0x7f61e810eb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:12.183 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.194+0000 7f61ecee9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f61e810f040 con 0x7f61e807cca0 2024-05-09T04:40:12.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.194+0000 7f61e659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61e807cca0 0x7f61e810eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:12.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.194+0000 7f61e659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61e807cca0 0x7f61e810eb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59370/0 (socket says 172.21.15.46:59370) 2024-05-09T04:40:12.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.194+0000 7f61e659c700 1 -- 172.21.15.46:0/462978937 learned_addr learned my addr 172.21.15.46:0/462978937 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:12.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.194+0000 7f61e659c700 1 -- 172.21.15.46:0/462978937 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e810f1c0 con 0x7f61e807cca0 2024-05-09T04:40:12.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.195+0000 7f61e659c700 1 --2- 172.21.15.46:0/462978937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61e807cca0 0x7f61e810eb00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f61d80098a0 tx=0x7f61d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1943dd6f87ee6c4c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:12.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.195+0000 7f61e559a700 1 -- 172.21.15.46:0/462978937 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61d8004160 con 0x7f61e807cca0 2024-05-09T04:40:12.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.195+0000 7f61e559a700 1 -- 172.21.15.46:0/462978937 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f61d8004300 con 0x7f61e807cca0 2024-05-09T04:40:12.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.195+0000 7f61e559a700 1 -- 172.21.15.46:0/462978937 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61d8011af0 con 0x7f61e807cca0 2024-05-09T04:40:12.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.195+0000 7f61ecee9700 1 -- 172.21.15.46:0/462978937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61e807cca0 msgr2=0x7f61e810eb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:12.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.195+0000 7f61ecee9700 1 --2- 172.21.15.46:0/462978937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61e807cca0 0x7f61e810eb00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f61d80098a0 tx=0x7f61d8009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:12.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.196+0000 7f61ecee9700 1 -- 172.21.15.46:0/462978937 shutdown_connections 2024-05-09T04:40:12.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.196+0000 7f61ecee9700 1 --2- 172.21.15.46:0/462978937 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61e807cca0 0x7f61e810eb00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:12.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.196+0000 7f61ecee9700 1 -- 172.21.15.46:0/462978937 >> 172.21.15.46:0/462978937 conn(0x7f61e81040f0 msgr2=0x7f61e8106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:12.184 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.196+0000 7f61ecee9700 1 -- 172.21.15.46:0/462978937 shutdown_connections 2024-05-09T04:40:12.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.196+0000 7f61ecee9700 1 -- 172.21.15.46:0/462978937 wait complete. 2024-05-09T04:40:12.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.196+0000 7f61ecee9700 1 Processor -- start 2024-05-09T04:40:12.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.197+0000 7f61ecee9700 1 -- start start 2024-05-09T04:40:12.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.197+0000 7f61ecee9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61e807cca0 0x7f61e81a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:12.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.197+0000 7f61ecee9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f61e81100d0 con 0x7f61e807cca0 2024-05-09T04:40:12.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.197+0000 7f61e659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61e807cca0 0x7f61e81a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:12.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.197+0000 7f61e659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61e807cca0 0x7f61e81a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59382/0 (socket says 172.21.15.46:59382) 2024-05-09T04:40:12.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.197+0000 7f61e659c700 1 -- 172.21.15.46:0/2414390211 learned_addr learned my addr 172.21.15.46:0/2414390211 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:12.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.197+0000 7f61e659c700 1 -- 172.21.15.46:0/2414390211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61e81a06d0 con 0x7f61e807cca0 2024-05-09T04:40:12.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.197+0000 7f61e659c700 1 --2- 172.21.15.46:0/2414390211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61e807cca0 0x7f61e81a0190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009fb0 tx=0x7f61d8011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:12.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.198+0000 7f61d77fe700 1 -- 172.21.15.46:0/2414390211 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61d80036a0 con 0x7f61e807cca0 2024-05-09T04:40:12.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.198+0000 7f61d77fe700 1 -- 172.21.15.46:0/2414390211 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f61d801a430 con 0x7f61e807cca0 2024-05-09T04:40:12.185 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.198+0000 7f61ecee9700 1 -- 172.21.15.46:0/2414390211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61e81a0960 con 0x7f61e807cca0 2024-05-09T04:40:12.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.198+0000 7f61d77fe700 1 -- 172.21.15.46:0/2414390211 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61d8019550 con 0x7f61e807cca0 2024-05-09T04:40:12.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.198+0000 7f61ecee9700 1 -- 172.21.15.46:0/2414390211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61e81a3570 con 0x7f61e807cca0 2024-05-09T04:40:12.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.198+0000 7f61d77fe700 1 -- 172.21.15.46:0/2414390211 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f61d8019850 con 0x7f61e807cca0 2024-05-09T04:40:12.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.199+0000 7f61ecee9700 1 -- 172.21.15.46:0/2414390211 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61e8064e20 con 0x7f61e807cca0 2024-05-09T04:40:12.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.199+0000 7f61d77fe700 1 --2- 172.21.15.46:0/2414390211 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f61d003e420 0x7f61d0040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:12.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.199+0000 7f61d77fe700 1 -- 172.21.15.46:0/2414390211 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f61d80538b0 con 0x7f61e807cca0 2024-05-09T04:40:12.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.199+0000 7f61e5d9b700 1 --2- 172.21.15.46:0/2414390211 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f61d003e420 0x7f61d0040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:12.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.199+0000 7f61e5d9b700 1 --2- 172.21.15.46:0/2414390211 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f61d003e420 0x7f61d0040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f61dc006fd0 tx=0x7f61dc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:12.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.204+0000 7f61d77fe700 1 -- 172.21.15.46:0/2414390211 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61d8017e90 con 0x7f61e807cca0 2024-05-09T04:40:12.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:07.422+0000 7f61ecee9700 1 -- 172.21.15.46:0/2414390211 --> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi046", "addr": "172.21.15.46", "target": ["mon-mgr", ""]}) -- 0x7f61e8105270 con 0x7f61d003e420 2024-05-09T04:40:12.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.654+0000 7f61d77fe700 1 -- 172.21.15.46:0/2414390211 <== mgr.14120 v2:172.21.15.46:6800/123334554 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f61e8105270 con 0x7f61d003e420 2024-05-09T04:40:12.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.657+0000 7f61ecee9700 1 -- 172.21.15.46:0/2414390211 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f61d003e420 msgr2=0x7f61d0040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:12.186 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.657+0000 7f61ecee9700 1 --2- 172.21.15.46:0/2414390211 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f61d003e420 0x7f61d0040930 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f61dc006fd0 tx=0x7f61dc006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:12.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.657+0000 7f61ecee9700 1 -- 172.21.15.46:0/2414390211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61e807cca0 msgr2=0x7f61e81a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:12.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.657+0000 7f61ecee9700 1 --2- 172.21.15.46:0/2414390211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61e807cca0 0x7f61e81a0190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f61d8009fb0 tx=0x7f61d8011940 comp rx=0 tx=0).stop 2024-05-09T04:40:12.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.657+0000 7f61ecee9700 1 -- 172.21.15.46:0/2414390211 shutdown_connections 2024-05-09T04:40:12.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.657+0000 7f61ecee9700 1 --2- 172.21.15.46:0/2414390211 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f61d003e420 0x7f61d0040930 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:12.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.657+0000 7f61ecee9700 1 --2- 172.21.15.46:0/2414390211 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f61e807cca0 0x7f61e81a0190 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:12.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.657+0000 7f61ecee9700 1 -- 172.21.15.46:0/2414390211 >> 172.21.15.46:0/2414390211 conn(0x7f61e81040f0 msgr2=0x7f61e8104b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:12.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.657+0000 7f61ecee9700 1 -- 172.21.15.46:0/2414390211 shutdown_connections 2024-05-09T04:40:12.187 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:11.657+0000 7f61ecee9700 1 -- 172.21.15.46:0/2414390211 wait complete. 2024-05-09T04:40:12.187 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mon service... 2024-05-09T04:40:12.697 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:12 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' 2024-05-09T04:40:12.697 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:12 smithi046 ceph-mon[30136]: Added host smithi046 2024-05-09T04:40:12.697 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:12 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:40:14.014 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-09T04:40:14.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.697+0000 7f2aa5e96700 1 Processor -- start 2024-05-09T04:40:14.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.697+0000 7f2aa5e96700 1 -- start start 2024-05-09T04:40:14.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.697+0000 7f2aa5e96700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aa0108a90 0x7f2aa0108ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:14.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.697+0000 7f2aa5e96700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa0109490 con 0x7f2aa0108a90 2024-05-09T04:40:14.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.697+0000 7f2a9f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aa0108a90 0x7f2aa0108ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:14.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.697+0000 7f2a9f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aa0108a90 0x7f2aa0108ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59396/0 (socket says 172.21.15.46:59396) 2024-05-09T04:40:14.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.697+0000 7f2a9f7fe700 1 -- 172.21.15.46:0/2097452722 learned_addr learned my addr 172.21.15.46:0/2097452722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:14.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.698+0000 7f2a9f7fe700 1 -- 172.21.15.46:0/2097452722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa0109bc0 con 0x7f2aa0108a90 2024-05-09T04:40:14.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.698+0000 7f2a9f7fe700 1 --2- 172.21.15.46:0/2097452722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aa0108a90 0x7f2aa0108ea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2a900098a0 tx=0x7f2a90009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2063fc70873f7e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:14.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.698+0000 7f2a9e7fc700 1 -- 172.21.15.46:0/2097452722 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a90004160 con 0x7f2aa0108a90 2024-05-09T04:40:14.015 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.698+0000 7f2a9e7fc700 1 -- 172.21.15.46:0/2097452722 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2a90004300 con 0x7f2aa0108a90 2024-05-09T04:40:14.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.698+0000 7f2a9e7fc700 1 -- 172.21.15.46:0/2097452722 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a90011af0 con 0x7f2aa0108a90 2024-05-09T04:40:14.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.699+0000 7f2aa5e96700 1 -- 172.21.15.46:0/2097452722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aa0108a90 msgr2=0x7f2aa0108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:14.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.699+0000 7f2aa5e96700 1 --2- 172.21.15.46:0/2097452722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aa0108a90 0x7f2aa0108ea0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2a900098a0 tx=0x7f2a90009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:14.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.699+0000 7f2aa5e96700 1 -- 172.21.15.46:0/2097452722 shutdown_connections 2024-05-09T04:40:14.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.699+0000 7f2aa5e96700 1 --2- 172.21.15.46:0/2097452722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aa0108a90 0x7f2aa0108ea0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:14.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.699+0000 7f2aa5e96700 1 -- 172.21.15.46:0/2097452722 >> 172.21.15.46:0/2097452722 conn(0x7f2aa01040f0 msgr2=0x7f2aa0106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:14.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.699+0000 7f2aa5e96700 1 -- 172.21.15.46:0/2097452722 shutdown_connections 2024-05-09T04:40:14.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.699+0000 7f2aa5e96700 1 -- 172.21.15.46:0/2097452722 wait complete. 2024-05-09T04:40:14.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.700+0000 7f2aa5e96700 1 Processor -- start 2024-05-09T04:40:14.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.700+0000 7f2aa5e96700 1 -- start start 2024-05-09T04:40:14.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.700+0000 7f2aa5e96700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aa0108a90 0x7f2aa019ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:14.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.700+0000 7f2aa5e96700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa010cf70 con 0x7f2aa0108a90 2024-05-09T04:40:14.016 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.700+0000 7f2a9f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aa0108a90 0x7f2aa019ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:14.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.700+0000 7f2a9f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aa0108a90 0x7f2aa019ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59410/0 (socket says 172.21.15.46:59410) 2024-05-09T04:40:14.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.700+0000 7f2a9f7fe700 1 -- 172.21.15.46:0/3138718356 learned_addr learned my addr 172.21.15.46:0/3138718356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:14.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.700+0000 7f2a9f7fe700 1 -- 172.21.15.46:0/3138718356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa019d3f0 con 0x7f2aa0108a90 2024-05-09T04:40:14.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.700+0000 7f2a9f7fe700 1 --2- 172.21.15.46:0/3138718356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aa0108a90 0x7f2aa019ceb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2a90011890 tx=0x7f2a90011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:14.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.701+0000 7f2a9cff9700 1 -- 172.21.15.46:0/3138718356 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a900039e0 con 0x7f2aa0108a90 2024-05-09T04:40:14.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.701+0000 7f2aa5e96700 1 -- 172.21.15.46:0/3138718356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa019d680 con 0x7f2aa0108a90 2024-05-09T04:40:14.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.701+0000 7f2a9cff9700 1 -- 172.21.15.46:0/3138718356 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2a90003b80 con 0x7f2aa0108a90 2024-05-09T04:40:14.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.701+0000 7f2a9cff9700 1 -- 172.21.15.46:0/3138718356 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a900246e0 con 0x7f2aa0108a90 2024-05-09T04:40:14.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.701+0000 7f2aa5e96700 1 -- 172.21.15.46:0/3138718356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa01a0290 con 0x7f2aa0108a90 2024-05-09T04:40:14.017 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.702+0000 7f2a9cff9700 1 -- 172.21.15.46:0/3138718356 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f2a90022020 con 0x7f2aa0108a90 2024-05-09T04:40:14.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.702+0000 7f2aa5e96700 1 -- 172.21.15.46:0/3138718356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2aa0064e20 con 0x7f2aa0108a90 2024-05-09T04:40:14.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.702+0000 7f2a9cff9700 1 --2- 172.21.15.46:0/3138718356 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f2a8803e470 0x7f2a88040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:14.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.702+0000 7f2a9cff9700 1 -- 172.21.15.46:0/3138718356 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2a90053760 con 0x7f2aa0108a90 2024-05-09T04:40:14.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.702+0000 7f2a9effd700 1 --2- 172.21.15.46:0/3138718356 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f2a8803e470 0x7f2a88040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:14.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.703+0000 7f2a9effd700 1 --2- 172.21.15.46:0/3138718356 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f2a8803e470 0x7f2a88040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2a94006fd0 tx=0x7f2a94006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:14.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.707+0000 7f2a9cff9700 1 -- 172.21.15.46:0/3138718356 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2a90020e00 con 0x7f2aa0108a90 2024-05-09T04:40:14.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.951+0000 7f2aa5e96700 1 -- 172.21.15.46:0/3138718356 --> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f2aa0105190 con 0x7f2a8803e470 2024-05-09T04:40:14.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.957+0000 7f2a9cff9700 1 -- 172.21.15.46:0/3138718356 <== mgr.14120 v2:172.21.15.46:6800/123334554 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2aa0105190 con 0x7f2a8803e470 2024-05-09T04:40:14.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.959+0000 7f2aa5e96700 1 -- 172.21.15.46:0/3138718356 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f2a8803e470 msgr2=0x7f2a88040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:14.018 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.959+0000 7f2aa5e96700 1 --2- 172.21.15.46:0/3138718356 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f2a8803e470 0x7f2a88040980 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2a94006fd0 tx=0x7f2a94006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:14.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.959+0000 7f2aa5e96700 1 -- 172.21.15.46:0/3138718356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aa0108a90 msgr2=0x7f2aa019ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:14.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.959+0000 7f2aa5e96700 1 --2- 172.21.15.46:0/3138718356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aa0108a90 0x7f2aa019ceb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2a90011890 tx=0x7f2a90011970 comp rx=0 tx=0).stop 2024-05-09T04:40:14.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.959+0000 7f2aa5e96700 1 -- 172.21.15.46:0/3138718356 shutdown_connections 2024-05-09T04:40:14.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.959+0000 7f2aa5e96700 1 --2- 172.21.15.46:0/3138718356 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f2a8803e470 0x7f2a88040980 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:14.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.959+0000 7f2aa5e96700 1 --2- 172.21.15.46:0/3138718356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2aa0108a90 0x7f2aa019ceb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:14.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.959+0000 7f2aa5e96700 1 -- 172.21.15.46:0/3138718356 >> 172.21.15.46:0/3138718356 conn(0x7f2aa01040f0 msgr2=0x7f2aa0104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:14.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.959+0000 7f2aa5e96700 1 -- 172.21.15.46:0/3138718356 shutdown_connections 2024-05-09T04:40:14.019 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:12.959+0000 7f2aa5e96700 1 -- 172.21.15.46:0/3138718356 wait complete. 2024-05-09T04:40:14.019 INFO:teuthology.orchestra.run.smithi046.stdout:Deploying unmanaged mgr service... 2024-05-09T04:40:14.123 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:13 smithi046 ceph-mon[30136]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:14.123 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:13 smithi046 ceph-mon[30136]: Saving service mon spec with placement count:5 2024-05-09T04:40:14.123 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:13 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' 2024-05-09T04:40:15.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-09T04:40:15.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.407+0000 7f3637dd8700 1 Processor -- start 2024-05-09T04:40:15.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.407+0000 7f3637dd8700 1 -- start start 2024-05-09T04:40:15.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.407+0000 7f3637dd8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f363007f7f0 0x7f363007dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:15.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.407+0000 7f3637dd8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f363007e200 con 0x7f363007f7f0 2024-05-09T04:40:15.448 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.407+0000 7f3635b74700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f363007f7f0 0x7f363007dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:15.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.407+0000 7f3635b74700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f363007f7f0 0x7f363007dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49242/0 (socket says 172.21.15.46:49242) 2024-05-09T04:40:15.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.407+0000 7f3635b74700 1 -- 172.21.15.46:0/56743431 learned_addr learned my addr 172.21.15.46:0/56743431 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:15.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.408+0000 7f3635b74700 1 -- 172.21.15.46:0/56743431 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f363007e380 con 0x7f363007f7f0 2024-05-09T04:40:15.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.408+0000 7f3635b74700 1 --2- 172.21.15.46:0/56743431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f363007f7f0 0x7f363007dcc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f362c0098a0 tx=0x7f362c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928ea70edcbb5936 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:15.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.408+0000 7f3634b72700 1 -- 172.21.15.46:0/56743431 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f362c004160 con 0x7f363007f7f0 2024-05-09T04:40:15.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.408+0000 7f3634b72700 1 -- 172.21.15.46:0/56743431 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f362c004300 con 0x7f363007f7f0 2024-05-09T04:40:15.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.408+0000 7f3634b72700 1 -- 172.21.15.46:0/56743431 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f362c011af0 con 0x7f363007f7f0 2024-05-09T04:40:15.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.409+0000 7f3637dd8700 1 -- 172.21.15.46:0/56743431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f363007f7f0 msgr2=0x7f363007dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:15.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.409+0000 7f3637dd8700 1 --2- 172.21.15.46:0/56743431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f363007f7f0 0x7f363007dcc0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f362c0098a0 tx=0x7f362c009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:15.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.409+0000 7f3637dd8700 1 -- 172.21.15.46:0/56743431 shutdown_connections 2024-05-09T04:40:15.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.409+0000 7f3637dd8700 1 --2- 172.21.15.46:0/56743431 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f363007f7f0 0x7f363007dcc0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:15.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.409+0000 7f3637dd8700 1 -- 172.21.15.46:0/56743431 >> 172.21.15.46:0/56743431 conn(0x7f3630105270 msgr2=0x7f36301076e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:15.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.409+0000 7f3637dd8700 1 -- 172.21.15.46:0/56743431 shutdown_connections 2024-05-09T04:40:15.449 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.409+0000 7f3637dd8700 1 -- 172.21.15.46:0/56743431 wait complete. 2024-05-09T04:40:15.450 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.410+0000 7f3637dd8700 1 Processor -- start 2024-05-09T04:40:15.450 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.410+0000 7f3637dd8700 1 -- start start 2024-05-09T04:40:15.450 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.410+0000 7f3637dd8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f363007f7f0 0x7f36301a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:15.450 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.410+0000 7f3637dd8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3630110130 con 0x7f363007f7f0 2024-05-09T04:40:15.450 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.410+0000 7f3635b74700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f363007f7f0 0x7f36301a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:15.450 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.410+0000 7f3635b74700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f363007f7f0 0x7f36301a0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49256/0 (socket says 172.21.15.46:49256) 2024-05-09T04:40:15.450 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.410+0000 7f3635b74700 1 -- 172.21.15.46:0/3667045914 learned_addr learned my addr 172.21.15.46:0/3667045914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:15.450 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.411+0000 7f3635b74700 1 -- 172.21.15.46:0/3667045914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36301a0690 con 0x7f363007f7f0 2024-05-09T04:40:15.450 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.411+0000 7f3635b74700 1 --2- 172.21.15.46:0/3667045914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f363007f7f0 0x7f36301a0150 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f362c0117b0 tx=0x7f362c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:15.450 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.411+0000 7f3626ffd700 1 -- 172.21.15.46:0/3667045914 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f362c003af0 con 0x7f363007f7f0 2024-05-09T04:40:15.450 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.411+0000 7f3626ffd700 1 -- 172.21.15.46:0/3667045914 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f362c020070 con 0x7f363007f7f0 2024-05-09T04:40:15.450 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.411+0000 7f3637dd8700 1 -- 172.21.15.46:0/3667045914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36301a0920 con 0x7f363007f7f0 2024-05-09T04:40:15.450 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.411+0000 7f3626ffd700 1 -- 172.21.15.46:0/3667045914 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f362c019630 con 0x7f363007f7f0 2024-05-09T04:40:15.451 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.411+0000 7f3637dd8700 1 -- 172.21.15.46:0/3667045914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36301a3530 con 0x7f363007f7f0 2024-05-09T04:40:15.451 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.412+0000 7f3626ffd700 1 -- 172.21.15.46:0/3667045914 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f362c0198a0 con 0x7f363007f7f0 2024-05-09T04:40:15.451 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.412+0000 7f3626ffd700 1 --2- 172.21.15.46:0/3667045914 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f361c03e4c0 0x7f361c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:15.451 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.412+0000 7f3637dd8700 1 -- 172.21.15.46:0/3667045914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3630064e20 con 0x7f363007f7f0 2024-05-09T04:40:15.451 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.412+0000 7f3626ffd700 1 -- 172.21.15.46:0/3667045914 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f362c01a9c0 con 0x7f363007f7f0 2024-05-09T04:40:15.451 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.412+0000 7f3635373700 1 --2- 172.21.15.46:0/3667045914 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f361c03e4c0 0x7f361c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:15.451 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.413+0000 7f3635373700 1 --2- 172.21.15.46:0/3667045914 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f361c03e4c0 0x7f361c0409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3620006fd0 tx=0x7f3620006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:15.451 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.418+0000 7f3626ffd700 1 -- 172.21.15.46:0/3667045914 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f362c016260 con 0x7f363007f7f0 2024-05-09T04:40:15.451 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.650+0000 7f3637dd8700 1 -- 172.21.15.46:0/3667045914 --> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3630106350 con 0x7f361c03e4c0 2024-05-09T04:40:15.451 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.655+0000 7f3626ffd700 1 -- 172.21.15.46:0/3667045914 <== mgr.14120 v2:172.21.15.46:6800/123334554 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3630106350 con 0x7f361c03e4c0 2024-05-09T04:40:15.451 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.657+0000 7f3637dd8700 1 -- 172.21.15.46:0/3667045914 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f361c03e4c0 msgr2=0x7f361c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:15.451 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.657+0000 7f3637dd8700 1 --2- 172.21.15.46:0/3667045914 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f361c03e4c0 0x7f361c0409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3620006fd0 tx=0x7f3620006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:15.451 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.657+0000 7f3637dd8700 1 -- 172.21.15.46:0/3667045914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f363007f7f0 msgr2=0x7f36301a0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:15.451 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.657+0000 7f3637dd8700 1 --2- 172.21.15.46:0/3667045914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f363007f7f0 0x7f36301a0150 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f362c0117b0 tx=0x7f362c011b70 comp rx=0 tx=0).stop 2024-05-09T04:40:15.452 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.657+0000 7f3637dd8700 1 -- 172.21.15.46:0/3667045914 shutdown_connections 2024-05-09T04:40:15.452 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.657+0000 7f3637dd8700 1 --2- 172.21.15.46:0/3667045914 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f361c03e4c0 0x7f361c0409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:15.452 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.657+0000 7f3637dd8700 1 --2- 172.21.15.46:0/3667045914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f363007f7f0 0x7f36301a0150 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:15.452 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.657+0000 7f3637dd8700 1 -- 172.21.15.46:0/3667045914 >> 172.21.15.46:0/3667045914 conn(0x7f3630105270 msgr2=0x7f3630105bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:15.452 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.657+0000 7f3637dd8700 1 -- 172.21.15.46:0/3667045914 shutdown_connections 2024-05-09T04:40:15.452 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:14.657+0000 7f3637dd8700 1 -- 172.21.15.46:0/3667045914 wait complete. 2024-05-09T04:40:15.691 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:15 smithi046 ceph-mon[30136]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:15.691 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:15 smithi046 ceph-mon[30136]: Saving service mgr spec with placement count:2 2024-05-09T04:40:15.691 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:15 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' 2024-05-09T04:40:15.691 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:15 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' 2024-05-09T04:40:15.691 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:15 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' 2024-05-09T04:40:17.158 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.904+0000 7f1705858700 1 Processor -- start 2024-05-09T04:40:17.158 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.904+0000 7f1705858700 1 -- start start 2024-05-09T04:40:17.158 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.905+0000 7f1705858700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1700109c00 0x7f170010a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:17.158 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.905+0000 7f1705858700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f170010a600 con 0x7f1700109c00 2024-05-09T04:40:17.158 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.905+0000 7f16feffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1700109c00 0x7f170010a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:17.158 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.905+0000 7f16feffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1700109c00 0x7f170010a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49268/0 (socket says 172.21.15.46:49268) 2024-05-09T04:40:17.158 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.905+0000 7f16feffd700 1 -- 172.21.15.46:0/2078639770 learned_addr learned my addr 172.21.15.46:0/2078639770 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:17.158 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.905+0000 7f16feffd700 1 -- 172.21.15.46:0/2078639770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170010ad30 con 0x7f1700109c00 2024-05-09T04:40:17.159 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.905+0000 7f16feffd700 1 --2- 172.21.15.46:0/2078639770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1700109c00 0x7f170010a010 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f16f00098a0 tx=0x7f16f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d64efa8e5d0ec502 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:17.159 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.906+0000 7f16fdffb700 1 -- 172.21.15.46:0/2078639770 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f0004160 con 0x7f1700109c00 2024-05-09T04:40:17.159 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.906+0000 7f16fdffb700 1 -- 172.21.15.46:0/2078639770 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f0004300 con 0x7f1700109c00 2024-05-09T04:40:17.159 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.906+0000 7f16fdffb700 1 -- 172.21.15.46:0/2078639770 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f0011a20 con 0x7f1700109c00 2024-05-09T04:40:17.159 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.907+0000 7f1705858700 1 -- 172.21.15.46:0/2078639770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1700109c00 msgr2=0x7f170010a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:17.159 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.907+0000 7f1705858700 1 --2- 172.21.15.46:0/2078639770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1700109c00 0x7f170010a010 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f16f00098a0 tx=0x7f16f0009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:17.159 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.907+0000 7f1705858700 1 -- 172.21.15.46:0/2078639770 shutdown_connections 2024-05-09T04:40:17.159 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.907+0000 7f1705858700 1 --2- 172.21.15.46:0/2078639770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1700109c00 0x7f170010a010 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:17.159 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.907+0000 7f1705858700 1 -- 172.21.15.46:0/2078639770 >> 172.21.15.46:0/2078639770 conn(0x7f1700105220 msgr2=0x7f17001076f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:17.159 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.907+0000 7f1705858700 1 -- 172.21.15.46:0/2078639770 shutdown_connections 2024-05-09T04:40:17.160 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.907+0000 7f1705858700 1 -- 172.21.15.46:0/2078639770 wait complete. 2024-05-09T04:40:17.160 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.907+0000 7f1705858700 1 Processor -- start 2024-05-09T04:40:17.160 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.908+0000 7f1705858700 1 -- start start 2024-05-09T04:40:17.160 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.908+0000 7f1705858700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1700109c00 0x7f170019df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:17.160 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.908+0000 7f1705858700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f170010e0e0 con 0x7f1700109c00 2024-05-09T04:40:17.160 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.908+0000 7f16feffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1700109c00 0x7f170019df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:17.160 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.908+0000 7f16feffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1700109c00 0x7f170019df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49272/0 (socket says 172.21.15.46:49272) 2024-05-09T04:40:17.160 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.908+0000 7f16feffd700 1 -- 172.21.15.46:0/1298428387 learned_addr learned my addr 172.21.15.46:0/1298428387 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:17.160 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.908+0000 7f16feffd700 1 -- 172.21.15.46:0/1298428387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f170019e4b0 con 0x7f1700109c00 2024-05-09T04:40:17.160 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.908+0000 7f16feffd700 1 --2- 172.21.15.46:0/1298428387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1700109c00 0x7f170019df70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f16f0003e70 tx=0x7f16f0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:17.160 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.909+0000 7f1704856700 1 -- 172.21.15.46:0/1298428387 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f0003af0 con 0x7f1700109c00 2024-05-09T04:40:17.160 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.909+0000 7f1704856700 1 -- 172.21.15.46:0/1298428387 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16f001a8e0 con 0x7f1700109c00 2024-05-09T04:40:17.160 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.909+0000 7f1705858700 1 -- 172.21.15.46:0/1298428387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f170019e740 con 0x7f1700109c00 2024-05-09T04:40:17.160 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.909+0000 7f1704856700 1 -- 172.21.15.46:0/1298428387 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f16f0024870 con 0x7f1700109c00 2024-05-09T04:40:17.161 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.909+0000 7f1705858700 1 -- 172.21.15.46:0/1298428387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17001a1350 con 0x7f1700109c00 2024-05-09T04:40:17.161 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.910+0000 7f1705858700 1 -- 172.21.15.46:0/1298428387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1700064e20 con 0x7f1700109c00 2024-05-09T04:40:17.161 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.910+0000 7f1704856700 1 -- 172.21.15.46:0/1298428387 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f16f0022020 con 0x7f1700109c00 2024-05-09T04:40:17.161 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.910+0000 7f1704856700 1 --2- 172.21.15.46:0/1298428387 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f16ec03e470 0x7f16ec040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:17.161 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.910+0000 7f1704856700 1 -- 172.21.15.46:0/1298428387 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f16f00538a0 con 0x7f1700109c00 2024-05-09T04:40:17.161 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.911+0000 7f16fe7fc700 1 --2- 172.21.15.46:0/1298428387 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f16ec03e470 0x7f16ec040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:17.161 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.911+0000 7f16fe7fc700 1 --2- 172.21.15.46:0/1298428387 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f16ec03e470 0x7f16ec040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f16e8006fd0 tx=0x7f16e8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:17.161 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:15.916+0000 7f1704856700 1 -- 172.21.15.46:0/1298428387 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f16f0029df0 con 0x7f1700109c00 2024-05-09T04:40:17.161 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:16.148+0000 7f1705858700 1 -- 172.21.15.46:0/1298428387 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f17001a1940 con 0x7f1700109c00 2024-05-09T04:40:17.161 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:16.152+0000 7f1704856700 1 -- 172.21.15.46:0/1298428387 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f16f001ad90 con 0x7f1700109c00 2024-05-09T04:40:17.161 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:16.155+0000 7f1705858700 1 -- 172.21.15.46:0/1298428387 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f16ec03e470 msgr2=0x7f16ec040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:17.161 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:16.155+0000 7f1705858700 1 --2- 172.21.15.46:0/1298428387 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f16ec03e470 0x7f16ec040980 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f16e8006fd0 tx=0x7f16e8006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:17.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:16.155+0000 7f1705858700 1 -- 172.21.15.46:0/1298428387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1700109c00 msgr2=0x7f170019df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:17.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:16.155+0000 7f1705858700 1 --2- 172.21.15.46:0/1298428387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1700109c00 0x7f170019df70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f16f0003e70 tx=0x7f16f0011aa0 comp rx=0 tx=0).stop 2024-05-09T04:40:17.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:16.155+0000 7f1705858700 1 -- 172.21.15.46:0/1298428387 shutdown_connections 2024-05-09T04:40:17.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:16.155+0000 7f1705858700 1 --2- 172.21.15.46:0/1298428387 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f16ec03e470 0x7f16ec040980 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:17.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:16.155+0000 7f1705858700 1 --2- 172.21.15.46:0/1298428387 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1700109c00 0x7f170019df70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:17.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:16.155+0000 7f1705858700 1 -- 172.21.15.46:0/1298428387 >> 172.21.15.46:0/1298428387 conn(0x7f1700105220 msgr2=0x7f1700105a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:17.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:16.156+0000 7f1705858700 1 -- 172.21.15.46:0/1298428387 shutdown_connections 2024-05-09T04:40:17.162 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:16.156+0000 7f1705858700 1 -- 172.21.15.46:0/1298428387 wait complete. 2024-05-09T04:40:17.284 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:17 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1298428387' entity='client.admin' 2024-05-09T04:40:18.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:18 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' 2024-05-09T04:40:18.545 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:18 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2163162321' entity='client.admin' 2024-05-09T04:40:18.553 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.674+0000 7fd585151700 1 Processor -- start 2024-05-09T04:40:18.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.674+0000 7fd585151700 1 -- start start 2024-05-09T04:40:18.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.674+0000 7fd585151700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd580108a90 0x7fd580108ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:18.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.674+0000 7fd585151700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd580109490 con 0x7fd580108a90 2024-05-09T04:40:18.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.674+0000 7fd57ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd580108a90 0x7fd580108ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:18.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.674+0000 7fd57ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd580108a90 0x7fd580108ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49284/0 (socket says 172.21.15.46:49284) 2024-05-09T04:40:18.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.674+0000 7fd57ed9d700 1 -- 172.21.15.46:0/4077741613 learned_addr learned my addr 172.21.15.46:0/4077741613 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:18.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.675+0000 7fd57ed9d700 1 -- 172.21.15.46:0/4077741613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd580109bc0 con 0x7fd580108a90 2024-05-09T04:40:18.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.675+0000 7fd57ed9d700 1 --2- 172.21.15.46:0/4077741613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd580108a90 0x7fd580108ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5700098a0 tx=0x7fd570009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef4a5b763869803 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:18.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.675+0000 7fd57dd9b700 1 -- 172.21.15.46:0/4077741613 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd570004160 con 0x7fd580108a90 2024-05-09T04:40:18.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.675+0000 7fd57dd9b700 1 -- 172.21.15.46:0/4077741613 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd570004300 con 0x7fd580108a90 2024-05-09T04:40:18.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.675+0000 7fd57dd9b700 1 -- 172.21.15.46:0/4077741613 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd570011a20 con 0x7fd580108a90 2024-05-09T04:40:18.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.676+0000 7fd585151700 1 -- 172.21.15.46:0/4077741613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd580108a90 msgr2=0x7fd580108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:18.554 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.676+0000 7fd585151700 1 --2- 172.21.15.46:0/4077741613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd580108a90 0x7fd580108ea0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd5700098a0 tx=0x7fd570009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:18.555 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.676+0000 7fd585151700 1 -- 172.21.15.46:0/4077741613 shutdown_connections 2024-05-09T04:40:18.555 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.676+0000 7fd585151700 1 --2- 172.21.15.46:0/4077741613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd580108a90 0x7fd580108ea0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:18.555 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.676+0000 7fd585151700 1 -- 172.21.15.46:0/4077741613 >> 172.21.15.46:0/4077741613 conn(0x7fd5801040f0 msgr2=0x7fd580106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:18.555 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.676+0000 7fd585151700 1 -- 172.21.15.46:0/4077741613 shutdown_connections 2024-05-09T04:40:18.555 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.676+0000 7fd585151700 1 -- 172.21.15.46:0/4077741613 wait complete. 2024-05-09T04:40:18.555 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.677+0000 7fd585151700 1 Processor -- start 2024-05-09T04:40:18.555 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.677+0000 7fd585151700 1 -- start start 2024-05-09T04:40:18.555 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.677+0000 7fd585151700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd580108a90 0x7fd58019bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:18.555 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.677+0000 7fd585151700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd58007c160 con 0x7fd580108a90 2024-05-09T04:40:18.555 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.678+0000 7fd57ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd580108a90 0x7fd58019bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:18.555 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.678+0000 7fd57ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd580108a90 0x7fd58019bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49300/0 (socket says 172.21.15.46:49300) 2024-05-09T04:40:18.555 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.678+0000 7fd57ed9d700 1 -- 172.21.15.46:0/2163162321 learned_addr learned my addr 172.21.15.46:0/2163162321 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:18.555 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.678+0000 7fd57ed9d700 1 -- 172.21.15.46:0/2163162321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd58019c270 con 0x7fd580108a90 2024-05-09T04:40:18.555 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.678+0000 7fd57ed9d700 1 --2- 172.21.15.46:0/2163162321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd580108a90 0x7fd58019bd30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd570003f00 tx=0x7fd570011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:18.556 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.678+0000 7fd567fff700 1 -- 172.21.15.46:0/2163162321 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd570003aa0 con 0x7fd580108a90 2024-05-09T04:40:18.556 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.678+0000 7fd567fff700 1 -- 172.21.15.46:0/2163162321 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd57001b8f0 con 0x7fd580108a90 2024-05-09T04:40:18.556 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.678+0000 7fd585151700 1 -- 172.21.15.46:0/2163162321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd58019c500 con 0x7fd580108a90 2024-05-09T04:40:18.556 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.678+0000 7fd567fff700 1 -- 172.21.15.46:0/2163162321 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd570023890 con 0x7fd580108a90 2024-05-09T04:40:18.556 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.678+0000 7fd585151700 1 -- 172.21.15.46:0/2163162321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd58019f100 con 0x7fd580108a90 2024-05-09T04:40:18.556 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.679+0000 7fd567fff700 1 -- 172.21.15.46:0/2163162321 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd57002c3f0 con 0x7fd580108a90 2024-05-09T04:40:18.556 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.680+0000 7fd585151700 1 -- 172.21.15.46:0/2163162321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd580064e20 con 0x7fd580108a90 2024-05-09T04:40:18.556 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.680+0000 7fd567fff700 1 --2- 172.21.15.46:0/2163162321 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fd568040720 0x7fd568042c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:18.556 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.680+0000 7fd567fff700 1 -- 172.21.15.46:0/2163162321 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd570054d20 con 0x7fd580108a90 2024-05-09T04:40:18.556 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.680+0000 7fd57e59c700 1 --2- 172.21.15.46:0/2163162321 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fd568040720 0x7fd568042c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:18.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.680+0000 7fd57e59c700 1 --2- 172.21.15.46:0/2163162321 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fd568040720 0x7fd568042c30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd574006fd0 tx=0x7fd574006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:18.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.685+0000 7fd567fff700 1 -- 172.21.15.46:0/2163162321 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd57001f030 con 0x7fd580108a90 2024-05-09T04:40:18.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.926+0000 7fd585151700 1 -- 172.21.15.46:0/2163162321 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fd580194f40 con 0x7fd580108a90 2024-05-09T04:40:18.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.929+0000 7fd567fff700 1 -- 172.21.15.46:0/2163162321 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fd57000f2e0 con 0x7fd580108a90 2024-05-09T04:40:18.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.932+0000 7fd585151700 1 -- 172.21.15.46:0/2163162321 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fd568040720 msgr2=0x7fd568042c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:18.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.932+0000 7fd585151700 1 --2- 172.21.15.46:0/2163162321 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fd568040720 0x7fd568042c30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd574006fd0 tx=0x7fd574006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:18.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.932+0000 7fd585151700 1 -- 172.21.15.46:0/2163162321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd580108a90 msgr2=0x7fd58019bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:18.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.932+0000 7fd585151700 1 --2- 172.21.15.46:0/2163162321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd580108a90 0x7fd58019bd30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd570003f00 tx=0x7fd570011aa0 comp rx=0 tx=0).stop 2024-05-09T04:40:18.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.932+0000 7fd585151700 1 -- 172.21.15.46:0/2163162321 shutdown_connections 2024-05-09T04:40:18.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.932+0000 7fd585151700 1 --2- 172.21.15.46:0/2163162321 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fd568040720 0x7fd568042c30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:18.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.932+0000 7fd585151700 1 --2- 172.21.15.46:0/2163162321 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd580108a90 0x7fd58019bd30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:18.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.932+0000 7fd585151700 1 -- 172.21.15.46:0/2163162321 >> 172.21.15.46:0/2163162321 conn(0x7fd5801040f0 msgr2=0x7fd580192000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:18.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.933+0000 7fd585151700 1 -- 172.21.15.46:0/2163162321 shutdown_connections 2024-05-09T04:40:18.557 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:17.933+0000 7fd585151700 1 -- 172.21.15.46:0/2163162321 wait complete. 2024-05-09T04:40:18.558 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling the dashboard module... 2024-05-09T04:40:19.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:19 smithi046 ceph-mon[30136]: from='mgr.14120 172.21.15.46:0/3892777608' entity='mgr.x' 2024-05-09T04:40:19.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:19 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2913546829' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-09T04:40:19.830 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:19 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: ignoring --setuser ceph since I am not root 2024-05-09T04:40:19.830 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:19 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: ignoring --setgroup ceph since I am not root 2024-05-09T04:40:20.155 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:19 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:19.854+0000 7eff162f1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T04:40:20.155 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:20 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:20.050+0000 7eff162f1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:40:20.310 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.013+0000 7f6a017b3700 1 Processor -- start 2024-05-09T04:40:20.310 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.013+0000 7f6a017b3700 1 -- start start 2024-05-09T04:40:20.310 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.013+0000 7f6a017b3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69fc10a6e0 0x7f69fc10cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:20.310 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.013+0000 7f6a017b3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc07d820 con 0x7f69fc10a6e0 2024-05-09T04:40:20.310 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.013+0000 7f69faffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69fc10a6e0 0x7f69fc10cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:20.310 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.013+0000 7f69faffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69fc10a6e0 0x7f69fc10cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49304/0 (socket says 172.21.15.46:49304) 2024-05-09T04:40:20.310 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.013+0000 7f69faffd700 1 -- 172.21.15.46:0/3301617829 learned_addr learned my addr 172.21.15.46:0/3301617829 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:20.310 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.013+0000 7f69faffd700 1 -- 172.21.15.46:0/3301617829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc10d040 con 0x7f69fc10a6e0 2024-05-09T04:40:20.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.014+0000 7f69faffd700 1 --2- 172.21.15.46:0/3301617829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69fc10a6e0 0x7f69fc10cb00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0098a0 tx=0x7f69ec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c395c9802f8f7ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:20.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.014+0000 7f69f9ffb700 1 -- 172.21.15.46:0/3301617829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ec004160 con 0x7f69fc10a6e0 2024-05-09T04:40:20.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.014+0000 7f69f9ffb700 1 -- 172.21.15.46:0/3301617829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69ec004300 con 0x7f69fc10a6e0 2024-05-09T04:40:20.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.014+0000 7f69f9ffb700 1 -- 172.21.15.46:0/3301617829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ec011a20 con 0x7f69fc10a6e0 2024-05-09T04:40:20.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.015+0000 7f6a017b3700 1 -- 172.21.15.46:0/3301617829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69fc10a6e0 msgr2=0x7f69fc10cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:20.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.015+0000 7f6a017b3700 1 --2- 172.21.15.46:0/3301617829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69fc10a6e0 0x7f69fc10cb00 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0098a0 tx=0x7f69ec009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:20.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.015+0000 7f6a017b3700 1 -- 172.21.15.46:0/3301617829 shutdown_connections 2024-05-09T04:40:20.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.015+0000 7f6a017b3700 1 --2- 172.21.15.46:0/3301617829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69fc10a6e0 0x7f69fc10cb00 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:20.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.015+0000 7f6a017b3700 1 -- 172.21.15.46:0/3301617829 >> 172.21.15.46:0/3301617829 conn(0x7f69fc104160 msgr2=0x7f69fc1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:20.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.015+0000 7f6a017b3700 1 -- 172.21.15.46:0/3301617829 shutdown_connections 2024-05-09T04:40:20.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.015+0000 7f6a017b3700 1 -- 172.21.15.46:0/3301617829 wait complete. 2024-05-09T04:40:20.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.015+0000 7f6a017b3700 1 Processor -- start 2024-05-09T04:40:20.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.016+0000 7f6a017b3700 1 -- start start 2024-05-09T04:40:20.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.016+0000 7f6a017b3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69fc10a6e0 0x7f69fc1a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:20.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.016+0000 7f6a017b3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f69fc109240 con 0x7f69fc10a6e0 2024-05-09T04:40:20.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.016+0000 7f69faffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69fc10a6e0 0x7f69fc1a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:20.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.016+0000 7f69faffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69fc10a6e0 0x7f69fc1a0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49320/0 (socket says 172.21.15.46:49320) 2024-05-09T04:40:20.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.016+0000 7f69faffd700 1 -- 172.21.15.46:0/2913546829 learned_addr learned my addr 172.21.15.46:0/2913546829 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:20.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.016+0000 7f69faffd700 1 -- 172.21.15.46:0/2913546829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69fc1a0680 con 0x7f69fc10a6e0 2024-05-09T04:40:20.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.016+0000 7f69faffd700 1 --2- 172.21.15.46:0/2913546829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69fc10a6e0 0x7f69fc1a0140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0119f0 tx=0x7f69ec011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:20.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.017+0000 7f69e3fff700 1 -- 172.21.15.46:0/2913546829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ec0039e0 con 0x7f69fc10a6e0 2024-05-09T04:40:20.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.017+0000 7f69e3fff700 1 -- 172.21.15.46:0/2913546829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f69ec01b8f0 con 0x7f69fc10a6e0 2024-05-09T04:40:20.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.017+0000 7f6a017b3700 1 -- 172.21.15.46:0/2913546829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69fc1a0910 con 0x7f69fc10a6e0 2024-05-09T04:40:20.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.017+0000 7f69e3fff700 1 -- 172.21.15.46:0/2913546829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f69ec0237a0 con 0x7f69fc10a6e0 2024-05-09T04:40:20.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.017+0000 7f6a017b3700 1 -- 172.21.15.46:0/2913546829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69fc1a3520 con 0x7f69fc10a6e0 2024-05-09T04:40:20.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.018+0000 7f6a017b3700 1 -- 172.21.15.46:0/2913546829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69fc064e20 con 0x7f69fc10a6e0 2024-05-09T04:40:20.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.018+0000 7f69e3fff700 1 -- 172.21.15.46:0/2913546829 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f69ec023a50 con 0x7f69fc10a6e0 2024-05-09T04:40:20.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.018+0000 7f69e3fff700 1 --2- 172.21.15.46:0/2913546829 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f69e403e470 0x7f69e4040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:20.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.018+0000 7f69e3fff700 1 -- 172.21.15.46:0/2913546829 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f69ec053ee0 con 0x7f69fc10a6e0 2024-05-09T04:40:20.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.018+0000 7f69fa7fc700 1 --2- 172.21.15.46:0/2913546829 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f69e403e470 0x7f69e4040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:20.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.019+0000 7f69fa7fc700 1 --2- 172.21.15.46:0/2913546829 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f69e403e470 0x7f69e4040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f69f0006fd0 tx=0x7f69f0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:20.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.023+0000 7f69e3fff700 1 -- 172.21.15.46:0/2913546829 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f69ec01f030 con 0x7f69fc10a6e0 2024-05-09T04:40:20.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.319+0000 7f6a017b3700 1 -- 172.21.15.46:0/2913546829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f69fc02e860 con 0x7f69fc10a6e0 2024-05-09T04:40:20.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.638+0000 7f69e3fff700 1 -- 172.21.15.46:0/2913546829 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f69ec00b3e0 con 0x7f69fc10a6e0 2024-05-09T04:40:20.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.640+0000 7f69e3fff700 1 -- 172.21.15.46:0/2913546829 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f69ec02c3e0 con 0x7f69fc10a6e0 2024-05-09T04:40:20.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.641+0000 7f6a017b3700 1 -- 172.21.15.46:0/2913546829 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f69e403e470 msgr2=0x7f69e4040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:20.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.641+0000 7f6a017b3700 1 --2- 172.21.15.46:0/2913546829 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f69e403e470 0x7f69e4040980 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f69f0006fd0 tx=0x7f69f0006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:20.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.641+0000 7f6a017b3700 1 -- 172.21.15.46:0/2913546829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69fc10a6e0 msgr2=0x7f69fc1a0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:20.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.641+0000 7f6a017b3700 1 --2- 172.21.15.46:0/2913546829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69fc10a6e0 0x7f69fc1a0140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f69ec0119f0 tx=0x7f69ec011ad0 comp rx=0 tx=0).stop 2024-05-09T04:40:20.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.641+0000 7f6a017b3700 1 -- 172.21.15.46:0/2913546829 shutdown_connections 2024-05-09T04:40:20.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.641+0000 7f6a017b3700 1 --2- 172.21.15.46:0/2913546829 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f69e403e470 0x7f69e4040980 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:20.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.641+0000 7f6a017b3700 1 --2- 172.21.15.46:0/2913546829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f69fc10a6e0 0x7f69fc1a0140 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:20.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.641+0000 7f6a017b3700 1 -- 172.21.15.46:0/2913546829 >> 172.21.15.46:0/2913546829 conn(0x7f69fc104160 msgr2=0x7f69fc104ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:20.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.642+0000 7f6a017b3700 1 -- 172.21.15.46:0/2913546829 shutdown_connections 2024-05-09T04:40:20.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:19.642+0000 7f6a017b3700 1 -- 172.21.15.46:0/2913546829 wait complete. 2024-05-09T04:40:20.428 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:20 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:20.159+0000 7eff162f1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T04:40:20.714 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:20 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2913546829' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-09T04:40:20.715 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:20 smithi046 ceph-mon[30136]: mgrmap e9: x(active, since 20s) 2024-05-09T04:40:20.715 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:20 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:20.490+0000 7eff162f1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T04:40:20.715 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:20 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:20.586+0000 7eff162f1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T04:40:21.677 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:21 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/329865722' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-09T04:40:21.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:40:21.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-09T04:40:21.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-09T04:40:21.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-09T04:40:21.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-09T04:40:21.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:40:21.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.783+0000 7fd9b5db3700 1 Processor -- start 2024-05-09T04:40:21.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.783+0000 7fd9b5db3700 1 -- start start 2024-05-09T04:40:21.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.783+0000 7fd9b5db3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9b0108b20 0x7fd9b0108f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:21.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.783+0000 7fd9b5db3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b0109520 con 0x7fd9b0108b20 2024-05-09T04:40:21.687 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.783+0000 7fd9af7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9b0108b20 0x7fd9b0108f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:21.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.783+0000 7fd9af7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9b0108b20 0x7fd9b0108f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49352/0 (socket says 172.21.15.46:49352) 2024-05-09T04:40:21.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.783+0000 7fd9af7fe700 1 -- 172.21.15.46:0/1646038818 learned_addr learned my addr 172.21.15.46:0/1646038818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:21.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.784+0000 7fd9af7fe700 1 -- 172.21.15.46:0/1646038818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b0109c50 con 0x7fd9b0108b20 2024-05-09T04:40:21.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.784+0000 7fd9af7fe700 1 --2- 172.21.15.46:0/1646038818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9b0108b20 0x7fd9b0108f30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00098a0 tx=0x7fd9a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c76c2c58000c308 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:21.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.784+0000 7fd9ae7fc700 1 -- 172.21.15.46:0/1646038818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9a0004160 con 0x7fd9b0108b20 2024-05-09T04:40:21.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.784+0000 7fd9ae7fc700 1 -- 172.21.15.46:0/1646038818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a0004300 con 0x7fd9b0108b20 2024-05-09T04:40:21.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.784+0000 7fd9ae7fc700 1 -- 172.21.15.46:0/1646038818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9a0011a20 con 0x7fd9b0108b20 2024-05-09T04:40:21.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.785+0000 7fd9b5db3700 1 -- 172.21.15.46:0/1646038818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9b0108b20 msgr2=0x7fd9b0108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:21.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.785+0000 7fd9b5db3700 1 --2- 172.21.15.46:0/1646038818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9b0108b20 0x7fd9b0108f30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00098a0 tx=0x7fd9a0009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:21.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.785+0000 7fd9b5db3700 1 -- 172.21.15.46:0/1646038818 shutdown_connections 2024-05-09T04:40:21.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.785+0000 7fd9b5db3700 1 --2- 172.21.15.46:0/1646038818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9b0108b20 0x7fd9b0108f30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:21.688 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.785+0000 7fd9b5db3700 1 -- 172.21.15.46:0/1646038818 >> 172.21.15.46:0/1646038818 conn(0x7fd9b01041c0 msgr2=0x7fd9b0106610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:21.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.785+0000 7fd9b5db3700 1 -- 172.21.15.46:0/1646038818 shutdown_connections 2024-05-09T04:40:21.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.785+0000 7fd9b5db3700 1 -- 172.21.15.46:0/1646038818 wait complete. 2024-05-09T04:40:21.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.786+0000 7fd9b5db3700 1 Processor -- start 2024-05-09T04:40:21.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.786+0000 7fd9b5db3700 1 -- start start 2024-05-09T04:40:21.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.786+0000 7fd9b5db3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9b0108b20 0x7fd9b01a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:21.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.786+0000 7fd9b5db3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b007c9f0 con 0x7fd9b0108b20 2024-05-09T04:40:21.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.787+0000 7fd9af7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9b0108b20 0x7fd9b01a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:21.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.787+0000 7fd9af7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9b0108b20 0x7fd9b01a0190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49360/0 (socket says 172.21.15.46:49360) 2024-05-09T04:40:21.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.787+0000 7fd9af7fe700 1 -- 172.21.15.46:0/329865722 learned_addr learned my addr 172.21.15.46:0/329865722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:21.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.787+0000 7fd9af7fe700 1 -- 172.21.15.46:0/329865722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b01a06d0 con 0x7fd9b0108b20 2024-05-09T04:40:21.689 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.787+0000 7fd9af7fe700 1 --2- 172.21.15.46:0/329865722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9b0108b20 0x7fd9b01a0190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00119f0 tx=0x7fd9a0011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:21.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.787+0000 7fd9acff9700 1 -- 172.21.15.46:0/329865722 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9a0003a60 con 0x7fd9b0108b20 2024-05-09T04:40:21.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.787+0000 7fd9acff9700 1 -- 172.21.15.46:0/329865722 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9a001b8f0 con 0x7fd9b0108b20 2024-05-09T04:40:21.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.787+0000 7fd9acff9700 1 -- 172.21.15.46:0/329865722 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd9a0023880 con 0x7fd9b0108b20 2024-05-09T04:40:21.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.787+0000 7fd9b5db3700 1 -- 172.21.15.46:0/329865722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b01a0960 con 0x7fd9b0108b20 2024-05-09T04:40:21.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.787+0000 7fd9b5db3700 1 -- 172.21.15.46:0/329865722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b01a3570 con 0x7fd9b0108b20 2024-05-09T04:40:21.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.788+0000 7fd9b5db3700 1 -- 172.21.15.46:0/329865722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9b0064e20 con 0x7fd9b0108b20 2024-05-09T04:40:21.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.789+0000 7fd9acff9700 1 -- 172.21.15.46:0/329865722 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fd9a002c3f0 con 0x7fd9b0108b20 2024-05-09T04:40:21.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.789+0000 7fd9acff9700 1 --2- 172.21.15.46:0/329865722 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fd99803e4c0 0x7fd9980409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:21.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.789+0000 7fd9acff9700 1 -- 172.21.15.46:0/329865722 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd9a0053fd0 con 0x7fd9b0108b20 2024-05-09T04:40:21.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.789+0000 7fd9aeffd700 1 -- 172.21.15.46:0/329865722 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fd99803e4c0 msgr2=0x7fd9980409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/123334554 2024-05-09T04:40:21.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.789+0000 7fd9aeffd700 1 --2- 172.21.15.46:0/329865722 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fd99803e4c0 0x7fd9980409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:40:21.690 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.794+0000 7fd9acff9700 1 -- 172.21.15.46:0/329865722 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd9a001f030 con 0x7fd9b0108b20 2024-05-09T04:40:21.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.990+0000 7fd9aeffd700 1 -- 172.21.15.46:0/329865722 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fd99803e4c0 msgr2=0x7fd9980409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/123334554 2024-05-09T04:40:21.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:20.990+0000 7fd9aeffd700 1 --2- 172.21.15.46:0/329865722 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fd99803e4c0 0x7fd9980409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:40:21.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:21.116+0000 7fd9b5db3700 1 -- 172.21.15.46:0/329865722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fd9b01a3740 con 0x7fd9b0108b20 2024-05-09T04:40:21.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:21.117+0000 7fd9acff9700 1 -- 172.21.15.46:0/329865722 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fd9b01a3740 con 0x7fd9b0108b20 2024-05-09T04:40:21.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:21.120+0000 7fd9b5db3700 1 -- 172.21.15.46:0/329865722 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fd99803e4c0 msgr2=0x7fd9980409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:40:21.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:21.120+0000 7fd9b5db3700 1 --2- 172.21.15.46:0/329865722 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fd99803e4c0 0x7fd9980409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:21.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:21.120+0000 7fd9b5db3700 1 -- 172.21.15.46:0/329865722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9b0108b20 msgr2=0x7fd9b01a0190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:21.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:21.120+0000 7fd9b5db3700 1 --2- 172.21.15.46:0/329865722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9b0108b20 0x7fd9b01a0190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00119f0 tx=0x7fd9a0011ad0 comp rx=0 tx=0).stop 2024-05-09T04:40:21.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:21.120+0000 7fd9b5db3700 1 -- 172.21.15.46:0/329865722 shutdown_connections 2024-05-09T04:40:21.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:21.120+0000 7fd9b5db3700 1 --2- 172.21.15.46:0/329865722 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7fd99803e4c0 0x7fd9980409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:21.691 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:21.121+0000 7fd9b5db3700 1 --2- 172.21.15.46:0/329865722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd9b0108b20 0x7fd9b01a0190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:21.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:21.121+0000 7fd9b5db3700 1 -- 172.21.15.46:0/329865722 >> 172.21.15.46:0/329865722 conn(0x7fd9b01041c0 msgr2=0x7fd9b0192050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:21.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:21.121+0000 7fd9b5db3700 1 -- 172.21.15.46:0/329865722 shutdown_connections 2024-05-09T04:40:21.692 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:21.121+0000 7fd9b5db3700 1 -- 172.21.15.46:0/329865722 wait complete. 2024-05-09T04:40:21.692 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for the mgr to restart... 2024-05-09T04:40:21.692 INFO:teuthology.orchestra.run.smithi046.stdout:Waiting for mgr epoch 9... 2024-05-09T04:40:22.005 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:21 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:21.890+0000 7eff162f1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T04:40:22.260 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:22 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:22.078+0000 7eff162f1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T04:40:22.260 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:22 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:22.260+0000 7eff162f1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T04:40:22.829 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:22 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:22.570+0000 7eff162f1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T04:40:22.829 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:22 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:22.668+0000 7eff162f1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T04:40:23.329 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:22 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:22.881+0000 7eff162f1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T04:40:23.329 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:23 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:23.086+0000 7eff162f1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:40:23.829 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:23 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:23.722+0000 7eff162f1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T04:40:24.329 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:23 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:23.957+0000 7eff162f1200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T04:40:24.829 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:24 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:24.408+0000 7eff162f1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T04:40:24.829 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:24 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:24.511+0000 7eff162f1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T04:40:25.279 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:24 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:24.900+0000 7eff162f1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T04:40:25.279 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:25 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:25.280+0000 7eff162f1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T04:40:25.579 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:25 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:25.379+0000 7eff162f1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T04:40:26.079 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:25 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:25.640+0000 7eff162f1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T04:40:26.079 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:25 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:25.781+0000 7eff162f1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T04:40:26.079 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:25 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:25.884+0000 7eff162f1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T04:40:26.579 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:26 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:26.225+0000 7eff162f1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T04:40:26.579 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:26 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:26.324+0000 7eff162f1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T04:40:27.198 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:26 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:26.966+0000 7eff162f1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T04:40:27.198 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:40:27 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:40:27.198+0000 7eff162f1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T04:40:27.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:27 smithi046 ceph-mon[30136]: Active manager daemon x restarted 2024-05-09T04:40:27.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:27 smithi046 ceph-mon[30136]: Activating manager daemon x 2024-05-09T04:40:27.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:27 smithi046 ceph-mon[30136]: osdmap e3: 0 total, 0 up, 0 in 2024-05-09T04:40:27.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:27 smithi046 ceph-mon[30136]: mgrmap e10: x(active, starting, since 0.0041114s) 2024-05-09T04:40:27.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:40:27.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T04:40:27.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T04:40:27.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T04:40:27.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T04:40:27.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:27 smithi046 ceph-mon[30136]: Manager daemon x is now available 2024-05-09T04:40:27.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:40:28.511 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T04:40:28.511 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T04:40:28.511 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:28 smithi046 ceph-mon[30136]: [09/May/2024:04:40:27] ENGINE Bus STARTING 2024-05-09T04:40:28.511 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:28 smithi046 ceph-mon[30136]: mgrmap e11: x(active, since 1.00698s) 2024-05-09T04:40:29.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout { 2024-05-09T04:40:29.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-09T04:40:29.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-09T04:40:29.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout } 2024-05-09T04:40:29.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.122+0000 7f4aa168b700 1 Processor -- start 2024-05-09T04:40:29.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.122+0000 7f4aa168b700 1 -- start start 2024-05-09T04:40:29.164 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.122+0000 7f4aa168b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9c10a690 0x7f4a9c10cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:29.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.122+0000 7f4aa168b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c07cdf0 con 0x7f4a9c10a690 2024-05-09T04:40:29.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.123+0000 7f4a9affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9c10a690 0x7f4a9c10cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:29.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.123+0000 7f4a9affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9c10a690 0x7f4a9c10cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49376/0 (socket says 172.21.15.46:49376) 2024-05-09T04:40:29.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.123+0000 7f4a9affd700 1 -- 172.21.15.46:0/2895765944 learned_addr learned my addr 172.21.15.46:0/2895765944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:29.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.123+0000 7f4a9affd700 1 -- 172.21.15.46:0/2895765944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c10cff0 con 0x7f4a9c10a690 2024-05-09T04:40:29.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.123+0000 7f4a9affd700 1 --2- 172.21.15.46:0/2895765944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9c10a690 0x7f4a9c10cab0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0098a0 tx=0x7f4a8c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397a6d26c33f4088 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:29.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.123+0000 7f4a99ffb700 1 -- 172.21.15.46:0/2895765944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a8c004160 con 0x7f4a9c10a690 2024-05-09T04:40:29.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.123+0000 7f4a99ffb700 1 -- 172.21.15.46:0/2895765944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a8c004300 con 0x7f4a9c10a690 2024-05-09T04:40:29.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.123+0000 7f4a99ffb700 1 -- 172.21.15.46:0/2895765944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a8c011a20 con 0x7f4a9c10a690 2024-05-09T04:40:29.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.125+0000 7f4aa168b700 1 -- 172.21.15.46:0/2895765944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9c10a690 msgr2=0x7f4a9c10cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:29.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.125+0000 7f4aa168b700 1 --2- 172.21.15.46:0/2895765944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9c10a690 0x7f4a9c10cab0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0098a0 tx=0x7f4a8c009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:29.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.125+0000 7f4aa168b700 1 -- 172.21.15.46:0/2895765944 shutdown_connections 2024-05-09T04:40:29.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.125+0000 7f4aa168b700 1 --2- 172.21.15.46:0/2895765944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9c10a690 0x7f4a9c10cab0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:29.165 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.125+0000 7f4aa168b700 1 -- 172.21.15.46:0/2895765944 >> 172.21.15.46:0/2895765944 conn(0x7f4a9c1040f0 msgr2=0x7f4a9c1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:29.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.125+0000 7f4aa168b700 1 -- 172.21.15.46:0/2895765944 shutdown_connections 2024-05-09T04:40:29.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.125+0000 7f4aa168b700 1 -- 172.21.15.46:0/2895765944 wait complete. 2024-05-09T04:40:29.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.126+0000 7f4aa168b700 1 Processor -- start 2024-05-09T04:40:29.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.126+0000 7f4aa168b700 1 -- start start 2024-05-09T04:40:29.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.126+0000 7f4aa168b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9c10a690 0x7f4a9c19def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:29.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.126+0000 7f4aa168b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a9c10df30 con 0x7f4a9c10a690 2024-05-09T04:40:29.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.126+0000 7f4a9affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9c10a690 0x7f4a9c19def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:29.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.126+0000 7f4a9affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9c10a690 0x7f4a9c19def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49378/0 (socket says 172.21.15.46:49378) 2024-05-09T04:40:29.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.126+0000 7f4a9affd700 1 -- 172.21.15.46:0/3878067720 learned_addr learned my addr 172.21.15.46:0/3878067720 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:29.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.127+0000 7f4a9affd700 1 -- 172.21.15.46:0/3878067720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a9c19e430 con 0x7f4a9c10a690 2024-05-09T04:40:29.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.127+0000 7f4a9affd700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9c10a690 0x7f4a9c19def0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c000c00 tx=0x7f4a8c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:29.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.127+0000 7f4a83fff700 1 -- 172.21.15.46:0/3878067720 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a8c003aa0 con 0x7f4a9c10a690 2024-05-09T04:40:29.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.127+0000 7f4a83fff700 1 -- 172.21.15.46:0/3878067720 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a8c01b8f0 con 0x7f4a9c10a690 2024-05-09T04:40:29.166 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.127+0000 7f4aa168b700 1 -- 172.21.15.46:0/3878067720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a9c19e6c0 con 0x7f4a9c10a690 2024-05-09T04:40:29.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.127+0000 7f4a83fff700 1 -- 172.21.15.46:0/3878067720 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a8c023890 con 0x7f4a9c10a690 2024-05-09T04:40:29.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.127+0000 7f4aa168b700 1 -- 172.21.15.46:0/3878067720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a9c1a12d0 con 0x7f4a9c10a690 2024-05-09T04:40:29.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.128+0000 7f4a83fff700 1 -- 172.21.15.46:0/3878067720 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f4a8c02c3f0 con 0x7f4a9c10a690 2024-05-09T04:40:29.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.129+0000 7f4a83fff700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f4a8403e4c0 0x7f4a840409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:29.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.129+0000 7f4a83fff700 1 -- 172.21.15.46:0/3878067720 --> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4a840410e0 con 0x7f4a8403e4c0 2024-05-09T04:40:29.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.129+0000 7f4a83fff700 1 -- 172.21.15.46:0/3878067720 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4a8c053e70 con 0x7f4a9c10a690 2024-05-09T04:40:29.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.129+0000 7f4a9a7fc700 1 -- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f4a8403e4c0 msgr2=0x7f4a840409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/123334554 2024-05-09T04:40:29.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.129+0000 7f4a9a7fc700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f4a8403e4c0 0x7f4a840409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:40:29.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.330+0000 7f4a9a7fc700 1 -- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f4a8403e4c0 msgr2=0x7f4a840409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/123334554 2024-05-09T04:40:29.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.330+0000 7f4a9a7fc700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f4a8403e4c0 0x7f4a840409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:40:29.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.731+0000 7f4a9a7fc700 1 -- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f4a8403e4c0 msgr2=0x7f4a840409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/123334554 2024-05-09T04:40:29.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:22.731+0000 7f4a9a7fc700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f4a8403e4c0 0x7f4a840409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-09T04:40:29.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.532+0000 7f4a9a7fc700 1 -- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f4a8403e4c0 msgr2=0x7f4a840409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/123334554 2024-05-09T04:40:29.167 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:23.532+0000 7f4a9a7fc700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f4a8403e4c0 0x7f4a840409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-09T04:40:29.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.133+0000 7f4a9a7fc700 1 -- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f4a8403e4c0 msgr2=0x7f4a840409d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/123334554 2024-05-09T04:40:29.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:25.133+0000 7f4a9a7fc700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f4a8403e4c0 0x7f4a840409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-09T04:40:29.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.202+0000 7f4a83fff700 1 -- 172.21.15.46:0/3878067720 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f4a8c02c780 con 0x7f4a9c10a690 2024-05-09T04:40:29.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.202+0000 7f4a83fff700 1 -- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f4a8403e4c0 msgr2=0x7f4a840409d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:40:29.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:27.202+0000 7f4a83fff700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f4a8403e4c0 0x7f4a840409d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:29.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.204+0000 7f4a83fff700 1 -- 172.21.15.46:0/3878067720 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f4a8c055a60 con 0x7f4a9c10a690 2024-05-09T04:40:29.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.204+0000 7f4a83fff700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4a8c053a00 0x7f4a84043fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:29.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.204+0000 7f4a83fff700 1 -- 172.21.15.46:0/3878067720 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f4a840410e0 con 0x7f4a8c053a00 2024-05-09T04:40:29.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.205+0000 7f4a9a7fc700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4a8c053a00 0x7f4a84043fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:29.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.206+0000 7f4a9a7fc700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4a8c053a00 0x7f4a84043fc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4a90003a10 tx=0x7f4a900092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:29.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.207+0000 7f4a83fff700 1 -- 172.21.15.46:0/3878067720 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f4a840410e0 con 0x7f4a8c053a00 2024-05-09T04:40:29.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.215+0000 7f4aa168b700 1 -- 172.21.15.46:0/3878067720 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f4a9c064e20 con 0x7f4a8c053a00 2024-05-09T04:40:29.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.215+0000 7f4a83fff700 1 -- 172.21.15.46:0/3878067720 <== mgr.14152 v2:172.21.15.46:6800/3691531627 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f4a9c064e20 con 0x7f4a8c053a00 2024-05-09T04:40:29.168 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.216+0000 7f4aa168b700 1 -- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4a8c053a00 msgr2=0x7f4a84043fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.216+0000 7f4aa168b700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4a8c053a00 0x7f4a84043fc0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f4a90003a10 tx=0x7f4a900092b0 comp rx=0 tx=0).stop 2024-05-09T04:40:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.216+0000 7f4aa168b700 1 -- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9c10a690 msgr2=0x7f4a9c19def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.216+0000 7f4aa168b700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9c10a690 0x7f4a9c19def0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c000c00 tx=0x7f4a8c011aa0 comp rx=0 tx=0).stop 2024-05-09T04:40:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.216+0000 7f4aa168b700 1 -- 172.21.15.46:0/3878067720 shutdown_connections 2024-05-09T04:40:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.216+0000 7f4aa168b700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4a8c053a00 0x7f4a84043fc0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.216+0000 7f4aa168b700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a9c10a690 0x7f4a9c19def0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.216+0000 7f4aa168b700 1 --2- 172.21.15.46:0/3878067720 >> [v2:172.21.15.46:6800/123334554,v1:172.21.15.46:6801/123334554] conn(0x7f4a8403e4c0 0x7f4a840409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.216+0000 7f4aa168b700 1 -- 172.21.15.46:0/3878067720 >> 172.21.15.46:0/3878067720 conn(0x7f4a9c1040f0 msgr2=0x7f4a9c1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.216+0000 7f4aa168b700 1 -- 172.21.15.46:0/3878067720 shutdown_connections 2024-05-09T04:40:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:28.216+0000 7f4aa168b700 1 -- 172.21.15.46:0/3878067720 wait complete. 2024-05-09T04:40:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:mgr epoch 9 is available 2024-05-09T04:40:29.169 INFO:teuthology.orchestra.run.smithi046.stdout:Generating a dashboard self-signed certificate... 2024-05-09T04:40:29.416 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:29 smithi046 ceph-mon[30136]: [09/May/2024:04:40:27] ENGINE Serving on https://172.21.15.46:7150 2024-05-09T04:40:29.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:29 smithi046 ceph-mon[30136]: [09/May/2024:04:40:27] ENGINE Serving on http://172.21.15.46:8765 2024-05-09T04:40:29.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:29 smithi046 ceph-mon[30136]: [09/May/2024:04:40:27] ENGINE Bus STARTED 2024-05-09T04:40:29.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:29 smithi046 ceph-mon[30136]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-09T04:40:29.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:29 smithi046 ceph-mon[30136]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-09T04:40:29.417 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:30.519 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-09T04:40:30.519 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.627+0000 7f439f1f5700 1 Processor -- start 2024-05-09T04:40:30.519 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.627+0000 7f439f1f5700 1 -- start start 2024-05-09T04:40:30.519 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.627+0000 7f439f1f5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f439807f810 0x7f439807dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:30.519 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.627+0000 7f439f1f5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f439807e220 con 0x7f439807f810 2024-05-09T04:40:30.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.627+0000 7f439cf91700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f439807f810 0x7f439807dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:30.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.628+0000 7f439cf91700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f439807f810 0x7f439807dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38820/0 (socket says 172.21.15.46:38820) 2024-05-09T04:40:30.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.628+0000 7f439cf91700 1 -- 172.21.15.46:0/2100059519 learned_addr learned my addr 172.21.15.46:0/2100059519 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:30.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.628+0000 7f439cf91700 1 -- 172.21.15.46:0/2100059519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439807e3a0 con 0x7f439807f810 2024-05-09T04:40:30.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.628+0000 7f439cf91700 1 --2- 172.21.15.46:0/2100059519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f439807f810 0x7f439807dce0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f43940098a0 tx=0x7f4394009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1769ca5c67222de8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:30.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.628+0000 7f438f7fe700 1 -- 172.21.15.46:0/2100059519 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4394004160 con 0x7f439807f810 2024-05-09T04:40:30.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.628+0000 7f438f7fe700 1 -- 172.21.15.46:0/2100059519 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4394004300 con 0x7f439807f810 2024-05-09T04:40:30.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.628+0000 7f438f7fe700 1 -- 172.21.15.46:0/2100059519 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4394011a20 con 0x7f439807f810 2024-05-09T04:40:30.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.629+0000 7f439f1f5700 1 -- 172.21.15.46:0/2100059519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f439807f810 msgr2=0x7f439807dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:30.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.629+0000 7f439f1f5700 1 --2- 172.21.15.46:0/2100059519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f439807f810 0x7f439807dce0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f43940098a0 tx=0x7f4394009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:30.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.629+0000 7f439f1f5700 1 -- 172.21.15.46:0/2100059519 shutdown_connections 2024-05-09T04:40:30.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.629+0000 7f439f1f5700 1 --2- 172.21.15.46:0/2100059519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f439807f810 0x7f439807dce0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:30.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.629+0000 7f439f1f5700 1 -- 172.21.15.46:0/2100059519 >> 172.21.15.46:0/2100059519 conn(0x7f4398105220 msgr2=0x7f43981076f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:30.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.629+0000 7f439f1f5700 1 -- 172.21.15.46:0/2100059519 shutdown_connections 2024-05-09T04:40:30.520 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.629+0000 7f439f1f5700 1 -- 172.21.15.46:0/2100059519 wait complete. 2024-05-09T04:40:30.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.630+0000 7f439f1f5700 1 Processor -- start 2024-05-09T04:40:30.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.630+0000 7f439f1f5700 1 -- start start 2024-05-09T04:40:30.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.630+0000 7f439f1f5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f439807f810 0x7f43981a0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:30.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.630+0000 7f439f1f5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43981101c0 con 0x7f439807f810 2024-05-09T04:40:30.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.630+0000 7f439cf91700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f439807f810 0x7f43981a0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:30.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.630+0000 7f439cf91700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f439807f810 0x7f43981a0100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38822/0 (socket says 172.21.15.46:38822) 2024-05-09T04:40:30.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.630+0000 7f439cf91700 1 -- 172.21.15.46:0/2563030858 learned_addr learned my addr 172.21.15.46:0/2563030858 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:30.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.631+0000 7f439cf91700 1 -- 172.21.15.46:0/2563030858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43981a0640 con 0x7f439807f810 2024-05-09T04:40:30.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.631+0000 7f439cf91700 1 --2- 172.21.15.46:0/2563030858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f439807f810 0x7f43981a0100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f43940119f0 tx=0x7f4394011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:30.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.631+0000 7f438dffb700 1 -- 172.21.15.46:0/2563030858 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43940039e0 con 0x7f439807f810 2024-05-09T04:40:30.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.631+0000 7f439f1f5700 1 -- 172.21.15.46:0/2563030858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43981a08d0 con 0x7f439807f810 2024-05-09T04:40:30.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.631+0000 7f438dffb700 1 -- 172.21.15.46:0/2563030858 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f439401b8f0 con 0x7f439807f810 2024-05-09T04:40:30.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.631+0000 7f438dffb700 1 -- 172.21.15.46:0/2563030858 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43940237a0 con 0x7f439807f810 2024-05-09T04:40:30.521 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.631+0000 7f439f1f5700 1 -- 172.21.15.46:0/2563030858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43981a34e0 con 0x7f439807f810 2024-05-09T04:40:30.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.632+0000 7f438dffb700 1 -- 172.21.15.46:0/2563030858 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f439402c3f0 con 0x7f439807f810 2024-05-09T04:40:30.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.632+0000 7f439f1f5700 1 -- 172.21.15.46:0/2563030858 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4398064e20 con 0x7f439807f810 2024-05-09T04:40:30.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.633+0000 7f438dffb700 1 --2- 172.21.15.46:0/2563030858 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f438003e360 0x7f4380040870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:30.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.633+0000 7f438dffb700 1 -- 172.21.15.46:0/2563030858 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4394054cd0 con 0x7f439807f810 2024-05-09T04:40:30.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.633+0000 7f438ffff700 1 --2- 172.21.15.46:0/2563030858 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f438003e360 0x7f4380040870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:30.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.633+0000 7f438ffff700 1 --2- 172.21.15.46:0/2563030858 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f438003e360 0x7f4380040870 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4388006fd0 tx=0x7f4388006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:30.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.638+0000 7f438dffb700 1 -- 172.21.15.46:0/2563030858 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f439401f030 con 0x7f439807f810 2024-05-09T04:40:30.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.813+0000 7f438dffb700 1 -- 172.21.15.46:0/2563030858 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f4394053800 con 0x7f439807f810 2024-05-09T04:40:30.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:29.862+0000 7f439f1f5700 1 -- 172.21.15.46:0/2563030858 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f4398106230 con 0x7f438003e360 2024-05-09T04:40:30.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.023+0000 7f438dffb700 1 -- 172.21.15.46:0/2563030858 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f4398106230 con 0x7f438003e360 2024-05-09T04:40:30.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.025+0000 7f439f1f5700 1 -- 172.21.15.46:0/2563030858 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f438003e360 msgr2=0x7f4380040870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:30.522 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.025+0000 7f439f1f5700 1 --2- 172.21.15.46:0/2563030858 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f438003e360 0x7f4380040870 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f4388006fd0 tx=0x7f4388006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.025+0000 7f439f1f5700 1 -- 172.21.15.46:0/2563030858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f439807f810 msgr2=0x7f43981a0100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.025+0000 7f439f1f5700 1 --2- 172.21.15.46:0/2563030858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f439807f810 0x7f43981a0100 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f43940119f0 tx=0x7f4394011ad0 comp rx=0 tx=0).stop 2024-05-09T04:40:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.025+0000 7f439f1f5700 1 -- 172.21.15.46:0/2563030858 shutdown_connections 2024-05-09T04:40:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.025+0000 7f439f1f5700 1 --2- 172.21.15.46:0/2563030858 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f438003e360 0x7f4380040870 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.025+0000 7f439f1f5700 1 --2- 172.21.15.46:0/2563030858 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f439807f810 0x7f43981a0100 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.025+0000 7f439f1f5700 1 -- 172.21.15.46:0/2563030858 >> 172.21.15.46:0/2563030858 conn(0x7f4398105220 msgr2=0x7f4398105b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.025+0000 7f439f1f5700 1 -- 172.21.15.46:0/2563030858 shutdown_connections 2024-05-09T04:40:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.025+0000 7f439f1f5700 1 -- 172.21.15.46:0/2563030858 wait complete. 2024-05-09T04:40:30.523 INFO:teuthology.orchestra.run.smithi046.stdout:Creating initial admin user... 2024-05-09T04:40:30.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:30 smithi046 ceph-mon[30136]: mgrmap e12: x(active, since 2s) 2024-05-09T04:40:30.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:30 smithi046 ceph-mon[30136]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:30.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:30 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:30.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:30 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:31.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$iVAN4T1ziQHlpkLpJmxhIOandds2xy.M3W4o6eLtpTO/jqiHLT.ra", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715229631, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-09T04:40:31.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.955+0000 7f792c828700 1 Processor -- start 2024-05-09T04:40:31.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.955+0000 7f792c828700 1 -- start start 2024-05-09T04:40:31.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.956+0000 7f792c828700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792807d670 0x7f792810c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:31.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.956+0000 7f792c828700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f792810cda0 con 0x7f792807d670 2024-05-09T04:40:31.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.956+0000 7f792659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792807d670 0x7f792810c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:31.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.956+0000 7f792659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792807d670 0x7f792810c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38824/0 (socket says 172.21.15.46:38824) 2024-05-09T04:40:31.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.956+0000 7f792659c700 1 -- 172.21.15.46:0/3732020604 learned_addr learned my addr 172.21.15.46:0/3732020604 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:31.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.956+0000 7f792659c700 1 -- 172.21.15.46:0/3732020604 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792810cf20 con 0x7f792807d670 2024-05-09T04:40:31.975 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.957+0000 7f792659c700 1 --2- 172.21.15.46:0/3732020604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792807d670 0x7f792810c7d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79180098a0 tx=0x7f7918009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba7917315de1d546 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:31.977 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.957+0000 7f792559a700 1 -- 172.21.15.46:0/3732020604 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7918004160 con 0x7f792807d670 2024-05-09T04:40:31.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.957+0000 7f792559a700 1 -- 172.21.15.46:0/3732020604 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7918004300 con 0x7f792807d670 2024-05-09T04:40:31.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.957+0000 7f792559a700 1 -- 172.21.15.46:0/3732020604 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7918011a20 con 0x7f792807d670 2024-05-09T04:40:31.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.958+0000 7f792c828700 1 -- 172.21.15.46:0/3732020604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792807d670 msgr2=0x7f792810c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:31.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.958+0000 7f792c828700 1 --2- 172.21.15.46:0/3732020604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792807d670 0x7f792810c7d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f79180098a0 tx=0x7f7918009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:31.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.958+0000 7f792c828700 1 -- 172.21.15.46:0/3732020604 shutdown_connections 2024-05-09T04:40:31.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.958+0000 7f792c828700 1 --2- 172.21.15.46:0/3732020604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792807d670 0x7f792810c7d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:31.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.958+0000 7f792c828700 1 -- 172.21.15.46:0/3732020604 >> 172.21.15.46:0/3732020604 conn(0x7f7928104160 msgr2=0x7f79281065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:31.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.958+0000 7f792c828700 1 -- 172.21.15.46:0/3732020604 shutdown_connections 2024-05-09T04:40:31.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.958+0000 7f792c828700 1 -- 172.21.15.46:0/3732020604 wait complete. 2024-05-09T04:40:31.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.959+0000 7f792c828700 1 Processor -- start 2024-05-09T04:40:31.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.959+0000 7f792c828700 1 -- start start 2024-05-09T04:40:31.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.959+0000 7f792c828700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792807d670 0x7f792819df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:31.978 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.959+0000 7f792c828700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f792810df80 con 0x7f792807d670 2024-05-09T04:40:31.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.959+0000 7f792659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792807d670 0x7f792819df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:31.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.959+0000 7f792659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792807d670 0x7f792819df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38838/0 (socket says 172.21.15.46:38838) 2024-05-09T04:40:31.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.959+0000 7f792659c700 1 -- 172.21.15.46:0/1266677519 learned_addr learned my addr 172.21.15.46:0/1266677519 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:31.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.959+0000 7f792659c700 1 -- 172.21.15.46:0/1266677519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f792819e460 con 0x7f792807d670 2024-05-09T04:40:31.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.959+0000 7f792659c700 1 --2- 172.21.15.46:0/1266677519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792807d670 0x7f792819df20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f79180119f0 tx=0x7f7918011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:31.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.959+0000 7f79177fe700 1 -- 172.21.15.46:0/1266677519 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79180039e0 con 0x7f792807d670 2024-05-09T04:40:31.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.960+0000 7f792c828700 1 -- 172.21.15.46:0/1266677519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f792819e6f0 con 0x7f792807d670 2024-05-09T04:40:31.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.960+0000 7f79177fe700 1 -- 172.21.15.46:0/1266677519 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f791801b8f0 con 0x7f792807d670 2024-05-09T04:40:31.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.960+0000 7f79177fe700 1 -- 172.21.15.46:0/1266677519 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f79180237a0 con 0x7f792807d670 2024-05-09T04:40:31.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.960+0000 7f792c828700 1 -- 172.21.15.46:0/1266677519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79281a1300 con 0x7f792807d670 2024-05-09T04:40:31.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.961+0000 7f79177fe700 1 -- 172.21.15.46:0/1266677519 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f791802c3f0 con 0x7f792807d670 2024-05-09T04:40:31.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.961+0000 7f792c828700 1 -- 172.21.15.46:0/1266677519 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7928064e20 con 0x7f792807d670 2024-05-09T04:40:31.979 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.961+0000 7f79177fe700 1 --2- 172.21.15.46:0/1266677519 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f791003e470 0x7f7910040980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:31.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.961+0000 7f79177fe700 1 -- 172.21.15.46:0/1266677519 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7918053f70 con 0x7f792807d670 2024-05-09T04:40:31.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.961+0000 7f7925d9b700 1 --2- 172.21.15.46:0/1266677519 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f791003e470 0x7f7910040980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:31.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.961+0000 7f7925d9b700 1 --2- 172.21.15.46:0/1266677519 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f791003e470 0x7f7910040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f791c006fd0 tx=0x7f791c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:31.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:30.966+0000 7f79177fe700 1 -- 172.21.15.46:0/1266677519 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f791801f030 con 0x7f792807d670 2024-05-09T04:40:31.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:31.198+0000 7f792c828700 1 -- 172.21.15.46:0/1266677519 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f79281a1590 con 0x7f791003e470 2024-05-09T04:40:31.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:31.445+0000 7f79177fe700 1 -- 172.21.15.46:0/1266677519 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f79281a1590 con 0x7f791003e470 2024-05-09T04:40:31.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:31.447+0000 7f792c828700 1 -- 172.21.15.46:0/1266677519 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f791003e470 msgr2=0x7f7910040980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:31.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:31.447+0000 7f792c828700 1 --2- 172.21.15.46:0/1266677519 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f791003e470 0x7f7910040980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f791c006fd0 tx=0x7f791c006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:31.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:31.447+0000 7f792c828700 1 -- 172.21.15.46:0/1266677519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792807d670 msgr2=0x7f792819df20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:31.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:31.447+0000 7f792c828700 1 --2- 172.21.15.46:0/1266677519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792807d670 0x7f792819df20 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f79180119f0 tx=0x7f7918011ad0 comp rx=0 tx=0).stop 2024-05-09T04:40:31.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:31.447+0000 7f792c828700 1 -- 172.21.15.46:0/1266677519 shutdown_connections 2024-05-09T04:40:31.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:31.447+0000 7f792c828700 1 --2- 172.21.15.46:0/1266677519 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f791003e470 0x7f7910040980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:31.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:31.447+0000 7f792c828700 1 --2- 172.21.15.46:0/1266677519 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f792807d670 0x7f792819df20 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:31.980 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:31.447+0000 7f792c828700 1 -- 172.21.15.46:0/1266677519 >> 172.21.15.46:0/1266677519 conn(0x7f7928104160 msgr2=0x7f79281049c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:31.981 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:31.447+0000 7f792c828700 1 -- 172.21.15.46:0/1266677519 shutdown_connections 2024-05-09T04:40:31.981 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:31.447+0000 7f792c828700 1 -- 172.21.15.46:0/1266677519 wait complete. 2024-05-09T04:40:31.981 INFO:teuthology.orchestra.run.smithi046.stdout:Fetching dashboard port number... 2024-05-09T04:40:32.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:32 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:32.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:32 smithi046 ceph-mon[30136]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:32.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:32 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:33.209 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:33 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2382694367' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-09T04:40:33.317 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stdout 8443 2024-05-09T04:40:33.317 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.396+0000 7fca95e76700 1 Processor -- start 2024-05-09T04:40:33.317 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.396+0000 7fca95e76700 1 -- start start 2024-05-09T04:40:33.317 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.396+0000 7fca95e76700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90108a90 0x7fca90108ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:33.317 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.396+0000 7fca95e76700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fca90109490 con 0x7fca90108a90 2024-05-09T04:40:33.317 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.396+0000 7fca8f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90108a90 0x7fca90108ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:33.317 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.397+0000 7fca8f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90108a90 0x7fca90108ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38844/0 (socket says 172.21.15.46:38844) 2024-05-09T04:40:33.317 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.397+0000 7fca8f7fe700 1 -- 172.21.15.46:0/2945481376 learned_addr learned my addr 172.21.15.46:0/2945481376 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:33.317 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.397+0000 7fca8f7fe700 1 -- 172.21.15.46:0/2945481376 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca90109bc0 con 0x7fca90108a90 2024-05-09T04:40:33.317 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.397+0000 7fca8f7fe700 1 --2- 172.21.15.46:0/2945481376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90108a90 0x7fca90108ea0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fca800098a0 tx=0x7fca80009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e8e10d8c7b1b3c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:33.317 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.397+0000 7fca8e7fc700 1 -- 172.21.15.46:0/2945481376 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca80004160 con 0x7fca90108a90 2024-05-09T04:40:33.317 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.397+0000 7fca8e7fc700 1 -- 172.21.15.46:0/2945481376 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca80004300 con 0x7fca90108a90 2024-05-09T04:40:33.318 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.397+0000 7fca8e7fc700 1 -- 172.21.15.46:0/2945481376 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca80011a20 con 0x7fca90108a90 2024-05-09T04:40:33.318 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.398+0000 7fca95e76700 1 -- 172.21.15.46:0/2945481376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90108a90 msgr2=0x7fca90108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:33.318 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.398+0000 7fca95e76700 1 --2- 172.21.15.46:0/2945481376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90108a90 0x7fca90108ea0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fca800098a0 tx=0x7fca80009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:33.318 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.398+0000 7fca95e76700 1 -- 172.21.15.46:0/2945481376 shutdown_connections 2024-05-09T04:40:33.318 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.398+0000 7fca95e76700 1 --2- 172.21.15.46:0/2945481376 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90108a90 0x7fca90108ea0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:33.318 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.398+0000 7fca95e76700 1 -- 172.21.15.46:0/2945481376 >> 172.21.15.46:0/2945481376 conn(0x7fca901040f0 msgr2=0x7fca90106580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:33.318 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.398+0000 7fca95e76700 1 -- 172.21.15.46:0/2945481376 shutdown_connections 2024-05-09T04:40:33.318 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.398+0000 7fca95e76700 1 -- 172.21.15.46:0/2945481376 wait complete. 2024-05-09T04:40:33.318 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.399+0000 7fca95e76700 1 Processor -- start 2024-05-09T04:40:33.318 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.399+0000 7fca95e76700 1 -- start start 2024-05-09T04:40:33.318 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.399+0000 7fca95e76700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90108a90 0x7fca9007c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:33.318 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.399+0000 7fca95e76700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fca9010cf70 con 0x7fca90108a90 2024-05-09T04:40:33.318 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.399+0000 7fca8f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90108a90 0x7fca9007c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:33.319 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.399+0000 7fca8f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90108a90 0x7fca9007c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:38856/0 (socket says 172.21.15.46:38856) 2024-05-09T04:40:33.319 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.399+0000 7fca8f7fe700 1 -- 172.21.15.46:0/2382694367 learned_addr learned my addr 172.21.15.46:0/2382694367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:33.319 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.400+0000 7fca8f7fe700 1 -- 172.21.15.46:0/2382694367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca9007cea0 con 0x7fca90108a90 2024-05-09T04:40:33.319 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.400+0000 7fca8f7fe700 1 --2- 172.21.15.46:0/2382694367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90108a90 0x7fca9007c960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fca80003e70 tx=0x7fca80011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:33.319 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.400+0000 7fca8cff9700 1 -- 172.21.15.46:0/2382694367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca80003aa0 con 0x7fca90108a90 2024-05-09T04:40:33.319 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.400+0000 7fca8cff9700 1 -- 172.21.15.46:0/2382694367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca8001a8e0 con 0x7fca90108a90 2024-05-09T04:40:33.319 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.400+0000 7fca95e76700 1 -- 172.21.15.46:0/2382694367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca901a0d00 con 0x7fca90108a90 2024-05-09T04:40:33.319 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.400+0000 7fca8cff9700 1 -- 172.21.15.46:0/2382694367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca80024870 con 0x7fca90108a90 2024-05-09T04:40:33.319 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.400+0000 7fca95e76700 1 -- 172.21.15.46:0/2382694367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca901a1180 con 0x7fca90108a90 2024-05-09T04:40:33.319 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.401+0000 7fca8cff9700 1 -- 172.21.15.46:0/2382694367 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7fca80022020 con 0x7fca90108a90 2024-05-09T04:40:33.319 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.401+0000 7fca95e76700 1 -- 172.21.15.46:0/2382694367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca90064e20 con 0x7fca90108a90 2024-05-09T04:40:33.319 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.401+0000 7fca8cff9700 1 --2- 172.21.15.46:0/2382694367 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fca7c03e4c0 0x7fca7c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:33.319 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.401+0000 7fca8cff9700 1 -- 172.21.15.46:0/2382694367 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fca80053a10 con 0x7fca90108a90 2024-05-09T04:40:33.319 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.402+0000 7fca8effd700 1 --2- 172.21.15.46:0/2382694367 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fca7c03e4c0 0x7fca7c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.402+0000 7fca8effd700 1 --2- 172.21.15.46:0/2382694367 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fca7c03e4c0 0x7fca7c0409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fca78006fd0 tx=0x7fca78006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.407+0000 7fca8cff9700 1 -- 172.21.15.46:0/2382694367 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca8000e260 con 0x7fca90108a90 2024-05-09T04:40:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.636+0000 7fca95e76700 1 -- 172.21.15.46:0/2382694367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fca901a1400 con 0x7fca90108a90 2024-05-09T04:40:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.636+0000 7fca8cff9700 1 -- 172.21.15.46:0/2382694367 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fca8001aa90 con 0x7fca90108a90 2024-05-09T04:40:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.639+0000 7fca95e76700 1 -- 172.21.15.46:0/2382694367 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fca7c03e4c0 msgr2=0x7fca7c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.639+0000 7fca95e76700 1 --2- 172.21.15.46:0/2382694367 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fca7c03e4c0 0x7fca7c0409d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fca78006fd0 tx=0x7fca78006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.639+0000 7fca95e76700 1 -- 172.21.15.46:0/2382694367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90108a90 msgr2=0x7fca9007c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.639+0000 7fca95e76700 1 --2- 172.21.15.46:0/2382694367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90108a90 0x7fca9007c960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fca80003e70 tx=0x7fca80011aa0 comp rx=0 tx=0).stop 2024-05-09T04:40:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.639+0000 7fca95e76700 1 -- 172.21.15.46:0/2382694367 shutdown_connections 2024-05-09T04:40:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.639+0000 7fca95e76700 1 --2- 172.21.15.46:0/2382694367 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fca7c03e4c0 0x7fca7c0409d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.639+0000 7fca95e76700 1 --2- 172.21.15.46:0/2382694367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca90108a90 0x7fca9007c960 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.639+0000 7fca95e76700 1 -- 172.21.15.46:0/2382694367 >> 172.21.15.46:0/2382694367 conn(0x7fca901040f0 msgr2=0x7fca90104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.639+0000 7fca95e76700 1 -- 172.21.15.46:0/2382694367 shutdown_connections 2024-05-09T04:40:33.320 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:32.639+0000 7fca95e76700 1 -- 172.21.15.46:0/2382694367 wait complete. 2024-05-09T04:40:33.324 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-09T04:40:33.324 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout disabled 2024-05-09T04:40:33.332 INFO:teuthology.orchestra.run.smithi046.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-09T04:40:33.332 INFO:teuthology.orchestra.run.smithi046.stdout:systemctl: stdout inactive 2024-05-09T04:40:33.332 INFO:teuthology.orchestra.run.smithi046.stdout:firewalld.service is not enabled 2024-05-09T04:40:33.332 INFO:teuthology.orchestra.run.smithi046.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-09T04:40:33.334 INFO:teuthology.orchestra.run.smithi046.stdout:Ceph Dashboard is now available at: 2024-05-09T04:40:33.334 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:40:33.334 INFO:teuthology.orchestra.run.smithi046.stdout: URL: https://smithi046.front.sepia.ceph.com:8443/ 2024-05-09T04:40:33.334 INFO:teuthology.orchestra.run.smithi046.stdout: User: admin 2024-05-09T04:40:33.335 INFO:teuthology.orchestra.run.smithi046.stdout: Password: lcl95pvile 2024-05-09T04:40:33.335 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:40:33.335 INFO:teuthology.orchestra.run.smithi046.stdout:Saving cluster configuration to /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config directory 2024-05-09T04:40:33.335 INFO:teuthology.orchestra.run.smithi046.stdout:Enabling autotune for osd_memory_target 2024-05-09T04:40:35.303 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:35 smithi046 ceph-mon[30136]: mgrmap e13: x(active, since 6s) 2024-05-09T04:40:35.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.495+0000 7ff9e0c32700 1 Processor -- start 2024-05-09T04:40:35.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.495+0000 7ff9e0c32700 1 -- start start 2024-05-09T04:40:35.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.495+0000 7ff9e0c32700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc10a6d0 0x7ff9dc10caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:35.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.495+0000 7ff9e0c32700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc07d680 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.495+0000 7ff9da59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc10a6d0 0x7ff9dc10caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:35.311 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.495+0000 7ff9da59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc10a6d0 0x7ff9dc10caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46186/0 (socket says 172.21.15.46:46186) 2024-05-09T04:40:35.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.495+0000 7ff9da59c700 1 -- 172.21.15.46:0/1692183866 learned_addr learned my addr 172.21.15.46:0/1692183866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:35.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.496+0000 7ff9da59c700 1 -- 172.21.15.46:0/1692183866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc10d030 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.496+0000 7ff9da59c700 1 --2- 172.21.15.46:0/1692183866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc10a6d0 0x7ff9dc10caf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0098a0 tx=0x7ff9cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee2b4e0c8178bb8d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:35.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.496+0000 7ff9d959a700 1 -- 172.21.15.46:0/1692183866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9cc004160 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.496+0000 7ff9d959a700 1 -- 172.21.15.46:0/1692183866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9cc004300 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.496+0000 7ff9d959a700 1 -- 172.21.15.46:0/1692183866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9cc011a20 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.497+0000 7ff9e0c32700 1 -- 172.21.15.46:0/1692183866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc10a6d0 msgr2=0x7ff9dc10caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:35.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.497+0000 7ff9e0c32700 1 --2- 172.21.15.46:0/1692183866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc10a6d0 0x7ff9dc10caf0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc0098a0 tx=0x7ff9cc009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:35.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.497+0000 7ff9e0c32700 1 -- 172.21.15.46:0/1692183866 shutdown_connections 2024-05-09T04:40:35.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.497+0000 7ff9e0c32700 1 --2- 172.21.15.46:0/1692183866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc10a6d0 0x7ff9dc10caf0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:35.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.497+0000 7ff9e0c32700 1 -- 172.21.15.46:0/1692183866 >> 172.21.15.46:0/1692183866 conn(0x7ff9dc104150 msgr2=0x7ff9dc1065e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:35.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.497+0000 7ff9e0c32700 1 -- 172.21.15.46:0/1692183866 shutdown_connections 2024-05-09T04:40:35.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.497+0000 7ff9e0c32700 1 -- 172.21.15.46:0/1692183866 wait complete. 2024-05-09T04:40:35.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.498+0000 7ff9e0c32700 1 Processor -- start 2024-05-09T04:40:35.312 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.498+0000 7ff9e0c32700 1 -- start start 2024-05-09T04:40:35.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.498+0000 7ff9e0c32700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc10a6d0 0x7ff9dc19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:35.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.498+0000 7ff9e0c32700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff9dc10df70 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.498+0000 7ff9da59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc10a6d0 0x7ff9dc19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:35.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.498+0000 7ff9da59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc10a6d0 0x7ff9dc19df10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46198/0 (socket says 172.21.15.46:46198) 2024-05-09T04:40:35.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.498+0000 7ff9da59c700 1 -- 172.21.15.46:0/3706800978 learned_addr learned my addr 172.21.15.46:0/3706800978 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:35.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.499+0000 7ff9da59c700 1 -- 172.21.15.46:0/3706800978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9dc19e450 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.499+0000 7ff9da59c700 1 --2- 172.21.15.46:0/3706800978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc10a6d0 0x7ff9dc19df10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc000c00 tx=0x7ff9cc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:35.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.499+0000 7ff9cb7fe700 1 -- 172.21.15.46:0/3706800978 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9cc003aa0 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.499+0000 7ff9cb7fe700 1 -- 172.21.15.46:0/3706800978 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9cc01b8f0 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.499+0000 7ff9e0c32700 1 -- 172.21.15.46:0/3706800978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9dc19e6e0 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.499+0000 7ff9cb7fe700 1 -- 172.21.15.46:0/3706800978 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff9cc023890 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.499+0000 7ff9e0c32700 1 -- 172.21.15.46:0/3706800978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9dc1a12f0 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.500+0000 7ff9cb7fe700 1 -- 172.21.15.46:0/3706800978 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff9cc02c3f0 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.313 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.500+0000 7ff9cb7fe700 1 --2- 172.21.15.46:0/3706800978 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff9c403e5c0 0x7ff9c4040ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:35.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.500+0000 7ff9e0c32700 1 -- 172.21.15.46:0/3706800978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9dc064e20 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.500+0000 7ff9cb7fe700 1 -- 172.21.15.46:0/3706800978 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff9cc054f90 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.500+0000 7ff9d9d9b700 1 --2- 172.21.15.46:0/3706800978 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff9c403e5c0 0x7ff9c4040ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:35.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.501+0000 7ff9d9d9b700 1 --2- 172.21.15.46:0/3706800978 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff9c403e5c0 0x7ff9c4040ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0006fd0 tx=0x7ff9d0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:35.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.506+0000 7ff9cb7fe700 1 -- 172.21.15.46:0/3706800978 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff9cc01f030 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.717+0000 7ff9e0c32700 1 -- 172.21.15.46:0/3706800978 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7ff9dc1a14c0 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.718+0000 7ff9cb7fe700 1 -- 172.21.15.46:0/3706800978 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7ff9cc017e90 con 0x7ff9dc10a6d0 2024-05-09T04:40:35.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.720+0000 7ff9e0c32700 1 -- 172.21.15.46:0/3706800978 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff9c403e5c0 msgr2=0x7ff9c4040ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:35.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.720+0000 7ff9e0c32700 1 --2- 172.21.15.46:0/3706800978 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff9c403e5c0 0x7ff9c4040ad0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff9d0006fd0 tx=0x7ff9d0006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:35.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.720+0000 7ff9e0c32700 1 -- 172.21.15.46:0/3706800978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc10a6d0 msgr2=0x7ff9dc19df10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:35.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.720+0000 7ff9e0c32700 1 --2- 172.21.15.46:0/3706800978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc10a6d0 0x7ff9dc19df10 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc000c00 tx=0x7ff9cc011aa0 comp rx=0 tx=0).stop 2024-05-09T04:40:35.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.720+0000 7ff9e0c32700 1 -- 172.21.15.46:0/3706800978 shutdown_connections 2024-05-09T04:40:35.314 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.720+0000 7ff9e0c32700 1 --2- 172.21.15.46:0/3706800978 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff9c403e5c0 0x7ff9c4040ad0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:35.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.720+0000 7ff9e0c32700 1 --2- 172.21.15.46:0/3706800978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9dc10a6d0 0x7ff9dc19df10 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:35.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.720+0000 7ff9e0c32700 1 -- 172.21.15.46:0/3706800978 >> 172.21.15.46:0/3706800978 conn(0x7ff9dc104150 msgr2=0x7ff9dc1049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:35.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.721+0000 7ff9e0c32700 1 -- 172.21.15.46:0/3706800978 shutdown_connections 2024-05-09T04:40:35.315 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:34.721+0000 7ff9e0c32700 1 -- 172.21.15.46:0/3706800978 wait complete. 2024-05-09T04:40:36.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.735+0000 7f56f2710700 1 Processor -- start 2024-05-09T04:40:36.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.735+0000 7f56f2710700 1 -- start start 2024-05-09T04:40:36.651 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.735+0000 7f56f2710700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec10a6e0 0x7f56ec10cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:36.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.735+0000 7f56f2710700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec07d820 con 0x7f56ec10a6e0 2024-05-09T04:40:36.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.736+0000 7f56ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec10a6e0 0x7f56ec10cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:36.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.736+0000 7f56ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec10a6e0 0x7f56ec10cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46202/0 (socket says 172.21.15.46:46202) 2024-05-09T04:40:36.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.736+0000 7f56ebfff700 1 -- 172.21.15.46:0/1073145007 learned_addr learned my addr 172.21.15.46:0/1073145007 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:36.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.736+0000 7f56ebfff700 1 -- 172.21.15.46:0/1073145007 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ec10d040 con 0x7f56ec10a6e0 2024-05-09T04:40:36.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.736+0000 7f56ebfff700 1 --2- 172.21.15.46:0/1073145007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec10a6e0 0x7f56ec10cb00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0098a0 tx=0x7f56dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e6bd16ff783152 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:36.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.736+0000 7f56eb7fe700 1 -- 172.21.15.46:0/1073145007 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56dc004160 con 0x7f56ec10a6e0 2024-05-09T04:40:36.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.736+0000 7f56eb7fe700 1 -- 172.21.15.46:0/1073145007 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56dc004300 con 0x7f56ec10a6e0 2024-05-09T04:40:36.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.736+0000 7f56eb7fe700 1 -- 172.21.15.46:0/1073145007 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56dc011a20 con 0x7f56ec10a6e0 2024-05-09T04:40:36.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.737+0000 7f56f2710700 1 -- 172.21.15.46:0/1073145007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec10a6e0 msgr2=0x7f56ec10cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:36.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.737+0000 7f56f2710700 1 --2- 172.21.15.46:0/1073145007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec10a6e0 0x7f56ec10cb00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0098a0 tx=0x7f56dc009bb0 comp rx=0 tx=0).stop 2024-05-09T04:40:36.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.737+0000 7f56f2710700 1 -- 172.21.15.46:0/1073145007 shutdown_connections 2024-05-09T04:40:36.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.737+0000 7f56f2710700 1 --2- 172.21.15.46:0/1073145007 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec10a6e0 0x7f56ec10cb00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:36.652 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.737+0000 7f56f2710700 1 -- 172.21.15.46:0/1073145007 >> 172.21.15.46:0/1073145007 conn(0x7f56ec104160 msgr2=0x7f56ec1065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.737+0000 7f56f2710700 1 -- 172.21.15.46:0/1073145007 shutdown_connections 2024-05-09T04:40:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.737+0000 7f56f2710700 1 -- 172.21.15.46:0/1073145007 wait complete. 2024-05-09T04:40:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.738+0000 7f56f2710700 1 Processor -- start 2024-05-09T04:40:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.738+0000 7f56f2710700 1 -- start start 2024-05-09T04:40:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.738+0000 7f56f2710700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec10a6e0 0x7f56ec19df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.738+0000 7f56f2710700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec10df80 con 0x7f56ec10a6e0 2024-05-09T04:40:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.738+0000 7f56ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec10a6e0 0x7f56ec19df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.738+0000 7f56ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec10a6e0 0x7f56ec19df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46218/0 (socket says 172.21.15.46:46218) 2024-05-09T04:40:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.738+0000 7f56ebfff700 1 -- 172.21.15.46:0/2572152142 learned_addr learned my addr 172.21.15.46:0/2572152142 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.739+0000 7f56ebfff700 1 -- 172.21.15.46:0/2572152142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ec19e480 con 0x7f56ec10a6e0 2024-05-09T04:40:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.739+0000 7f56ebfff700 1 --2- 172.21.15.46:0/2572152142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec10a6e0 0x7f56ec19df40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f56dc000c00 tx=0x7f56dc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.739+0000 7f56e9ffb700 1 -- 172.21.15.46:0/2572152142 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56dc003aa0 con 0x7f56ec10a6e0 2024-05-09T04:40:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.739+0000 7f56e9ffb700 1 -- 172.21.15.46:0/2572152142 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56dc01b8f0 con 0x7f56ec10a6e0 2024-05-09T04:40:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.739+0000 7f56e9ffb700 1 -- 172.21.15.46:0/2572152142 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56dc023890 con 0x7f56ec10a6e0 2024-05-09T04:40:36.653 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.739+0000 7f56f2710700 1 -- 172.21.15.46:0/2572152142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ec19e710 con 0x7f56ec10a6e0 2024-05-09T04:40:36.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.739+0000 7f56f2710700 1 -- 172.21.15.46:0/2572152142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ec1a1320 con 0x7f56ec10a6e0 2024-05-09T04:40:36.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.740+0000 7f56f2710700 1 -- 172.21.15.46:0/2572152142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56ec064e20 con 0x7f56ec10a6e0 2024-05-09T04:40:36.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.740+0000 7f56e9ffb700 1 -- 172.21.15.46:0/2572152142 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f56dc02c3f0 con 0x7f56ec10a6e0 2024-05-09T04:40:36.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.740+0000 7f56e9ffb700 1 --2- 172.21.15.46:0/2572152142 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f56d803e5c0 0x7f56d8040ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:36.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.740+0000 7f56e9ffb700 1 -- 172.21.15.46:0/2572152142 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f56dc0542a0 con 0x7f56ec10a6e0 2024-05-09T04:40:36.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.740+0000 7f56e3fff700 1 --2- 172.21.15.46:0/2572152142 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f56d803e5c0 0x7f56d8040ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:36.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.741+0000 7f56e3fff700 1 --2- 172.21.15.46:0/2572152142 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f56d803e5c0 0x7f56d8040ad0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f56d4006fd0 tx=0x7f56d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:36.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:35.746+0000 7f56e9ffb700 1 -- 172.21.15.46:0/2572152142 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f56dc01f030 con 0x7f56ec10a6e0 2024-05-09T04:40:36.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.080+0000 7f56f2710700 1 -- 172.21.15.46:0/2572152142 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f56ec1a14f0 con 0x7f56ec10a6e0 2024-05-09T04:40:36.654 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.083+0000 7f56e9ffb700 1 -- 172.21.15.46:0/2572152142 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7f56dc02b3a0 con 0x7f56ec10a6e0 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.085+0000 7f56f2710700 1 -- 172.21.15.46:0/2572152142 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f56d803e5c0 msgr2=0x7f56d8040ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.085+0000 7f56f2710700 1 --2- 172.21.15.46:0/2572152142 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f56d803e5c0 0x7f56d8040ad0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f56d4006fd0 tx=0x7f56d4006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.085+0000 7f56f2710700 1 -- 172.21.15.46:0/2572152142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec10a6e0 msgr2=0x7f56ec19df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.085+0000 7f56f2710700 1 --2- 172.21.15.46:0/2572152142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec10a6e0 0x7f56ec19df40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f56dc000c00 tx=0x7f56dc011aa0 comp rx=0 tx=0).stop 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.085+0000 7f56f2710700 1 -- 172.21.15.46:0/2572152142 shutdown_connections 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.085+0000 7f56f2710700 1 --2- 172.21.15.46:0/2572152142 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f56d803e5c0 0x7f56d8040ad0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.085+0000 7f56f2710700 1 --2- 172.21.15.46:0/2572152142 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f56ec10a6e0 0x7f56ec19df40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.085+0000 7f56f2710700 1 -- 172.21.15.46:0/2572152142 >> 172.21.15.46:0/2572152142 conn(0x7f56ec104160 msgr2=0x7f56ec104a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.086+0000 7f56f2710700 1 -- 172.21.15.46:0/2572152142 shutdown_connections 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr 2024-05-09T04:40:36.086+0000 7f56f2710700 1 -- 172.21.15.46:0/2572152142 wait complete. 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:40:36.655 INFO:teuthology.orchestra.run.smithi046.stdout:Or, if you are only running a single cluster on this host: 2024-05-09T04:40:36.656 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:40:36.656 INFO:teuthology.orchestra.run.smithi046.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-09T04:40:36.656 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:40:36.656 INFO:teuthology.orchestra.run.smithi046.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-09T04:40:36.656 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:40:36.656 INFO:teuthology.orchestra.run.smithi046.stdout: ceph telemetry on 2024-05-09T04:40:36.656 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:40:36.656 INFO:teuthology.orchestra.run.smithi046.stdout:For more information see: 2024-05-09T04:40:36.656 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:40:36.656 INFO:teuthology.orchestra.run.smithi046.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-09T04:40:36.656 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:40:36.656 INFO:teuthology.orchestra.run.smithi046.stdout:Bootstrap complete. 2024-05-09T04:40:36.690 INFO:tasks.cephadm:Fetching config... 2024-05-09T04:40:36.690 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:40:36.690 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-09T04:40:36.707 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-09T04:40:36.707 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:40:36.707 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-09T04:40:36.763 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-09T04:40:36.763 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:40:36.763 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-09T04:40:36.829 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-09T04:40:36.829 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:40:36.829 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-09T04:40:36.885 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-09T04:40:36.885 DEBUG:teuthology.orchestra.run.smithi046:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk4Fe4IpocKCoHeDQCM+pC9CSjPYdzd9hrME1fWXf++9cR05pC9ZB1WAP5uC0wfyrpLrW8aiGzI7nY+jRPOTu135ofr2ikGCUDNdYhNtbW8XCAkENZXqqTm368KeEJ3Rz1Ou1avaUfwVkfECh8MDiZbpO9NPt0IQzp9zBeHAy+cF3bn1lrEWVmXmSngyVXWsElC0ptQdv62ZjouiOAd6hyvSjh3i662wwtx0uaNwl87OAkkg84j8b6K7GsojWd68NTF82/WDN0NA+BqN4nth+gQwee858X8xexmTsBvdEPbBxrCuf8J5AllzIOEl5AdUMMZdpBc3yA9UdY0omtZeja/dOaTQsnFqeHrwsjnmkvfC70PcHWo8jkw6k0Ua9AK2dV7hcoVHxsgvqIxMz05Z50i5P+lhCPoDPXaavR1cnmuX7LIGITvBqi6M3sb9/lrPIqJ+m0pFhB5AhKovscrLZXPxDUavTxPf2dSbx/vwTnVnFaxwcm5pTtggV0J7dZeXc= ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T04:40:36.981 INFO:teuthology.orchestra.run.smithi046.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk4Fe4IpocKCoHeDQCM+pC9CSjPYdzd9hrME1fWXf++9cR05pC9ZB1WAP5uC0wfyrpLrW8aiGzI7nY+jRPOTu135ofr2ikGCUDNdYhNtbW8XCAkENZXqqTm368KeEJ3Rz1Ou1avaUfwVkfECh8MDiZbpO9NPt0IQzp9zBeHAy+cF3bn1lrEWVmXmSngyVXWsElC0ptQdv62ZjouiOAd6hyvSjh3i662wwtx0uaNwl87OAkkg84j8b6K7GsojWd68NTF82/WDN0NA+BqN4nth+gQwee858X8xexmTsBvdEPbBxrCuf8J5AllzIOEl5AdUMMZdpBc3yA9UdY0omtZeja/dOaTQsnFqeHrwsjnmkvfC70PcHWo8jkw6k0Ua9AK2dV7hcoVHxsgvqIxMz05Z50i5P+lhCPoDPXaavR1cnmuX7LIGITvBqi6M3sb9/lrPIqJ+m0pFhB5AhKovscrLZXPxDUavTxPf2dSbx/vwTnVnFaxwcm5pTtggV0J7dZeXc= ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:40:36.992 DEBUG:teuthology.orchestra.run.smithi072:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk4Fe4IpocKCoHeDQCM+pC9CSjPYdzd9hrME1fWXf++9cR05pC9ZB1WAP5uC0wfyrpLrW8aiGzI7nY+jRPOTu135ofr2ikGCUDNdYhNtbW8XCAkENZXqqTm368KeEJ3Rz1Ou1avaUfwVkfECh8MDiZbpO9NPt0IQzp9zBeHAy+cF3bn1lrEWVmXmSngyVXWsElC0ptQdv62ZjouiOAd6hyvSjh3i662wwtx0uaNwl87OAkkg84j8b6K7GsojWd68NTF82/WDN0NA+BqN4nth+gQwee858X8xexmTsBvdEPbBxrCuf8J5AllzIOEl5AdUMMZdpBc3yA9UdY0omtZeja/dOaTQsnFqeHrwsjnmkvfC70PcHWo8jkw6k0Ua9AK2dV7hcoVHxsgvqIxMz05Z50i5P+lhCPoDPXaavR1cnmuX7LIGITvBqi6M3sb9/lrPIqJ+m0pFhB5AhKovscrLZXPxDUavTxPf2dSbx/vwTnVnFaxwcm5pTtggV0J7dZeXc= ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T04:40:37.033 INFO:teuthology.orchestra.run.smithi072.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk4Fe4IpocKCoHeDQCM+pC9CSjPYdzd9hrME1fWXf++9cR05pC9ZB1WAP5uC0wfyrpLrW8aiGzI7nY+jRPOTu135ofr2ikGCUDNdYhNtbW8XCAkENZXqqTm368KeEJ3Rz1Ou1avaUfwVkfECh8MDiZbpO9NPt0IQzp9zBeHAy+cF3bn1lrEWVmXmSngyVXWsElC0ptQdv62ZjouiOAd6hyvSjh3i662wwtx0uaNwl87OAkkg84j8b6K7GsojWd68NTF82/WDN0NA+BqN4nth+gQwee858X8xexmTsBvdEPbBxrCuf8J5AllzIOEl5AdUMMZdpBc3yA9UdY0omtZeja/dOaTQsnFqeHrwsjnmkvfC70PcHWo8jkw6k0Ua9AK2dV7hcoVHxsgvqIxMz05Z50i5P+lhCPoDPXaavR1cnmuX7LIGITvBqi6M3sb9/lrPIqJ+m0pFhB5AhKovscrLZXPxDUavTxPf2dSbx/vwTnVnFaxwcm5pTtggV0J7dZeXc= ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:40:37.045 DEBUG:teuthology.orchestra.run.smithi110:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk4Fe4IpocKCoHeDQCM+pC9CSjPYdzd9hrME1fWXf++9cR05pC9ZB1WAP5uC0wfyrpLrW8aiGzI7nY+jRPOTu135ofr2ikGCUDNdYhNtbW8XCAkENZXqqTm368KeEJ3Rz1Ou1avaUfwVkfECh8MDiZbpO9NPt0IQzp9zBeHAy+cF3bn1lrEWVmXmSngyVXWsElC0ptQdv62ZjouiOAd6hyvSjh3i662wwtx0uaNwl87OAkkg84j8b6K7GsojWd68NTF82/WDN0NA+BqN4nth+gQwee858X8xexmTsBvdEPbBxrCuf8J5AllzIOEl5AdUMMZdpBc3yA9UdY0omtZeja/dOaTQsnFqeHrwsjnmkvfC70PcHWo8jkw6k0Ua9AK2dV7hcoVHxsgvqIxMz05Z50i5P+lhCPoDPXaavR1cnmuX7LIGITvBqi6M3sb9/lrPIqJ+m0pFhB5AhKovscrLZXPxDUavTxPf2dSbx/vwTnVnFaxwcm5pTtggV0J7dZeXc= ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-09T04:40:37.085 INFO:teuthology.orchestra.run.smithi110.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCk4Fe4IpocKCoHeDQCM+pC9CSjPYdzd9hrME1fWXf++9cR05pC9ZB1WAP5uC0wfyrpLrW8aiGzI7nY+jRPOTu135ofr2ikGCUDNdYhNtbW8XCAkENZXqqTm368KeEJ3Rz1Ou1avaUfwVkfECh8MDiZbpO9NPt0IQzp9zBeHAy+cF3bn1lrEWVmXmSngyVXWsElC0ptQdv62ZjouiOAd6hyvSjh3i662wwtx0uaNwl87OAkkg84j8b6K7GsojWd68NTF82/WDN0NA+BqN4nth+gQwee858X8xexmTsBvdEPbBxrCuf8J5AllzIOEl5AdUMMZdpBc3yA9UdY0omtZeja/dOaTQsnFqeHrwsjnmkvfC70PcHWo8jkw6k0Ua9AK2dV7hcoVHxsgvqIxMz05Z50i5P+lhCPoDPXaavR1cnmuX7LIGITvBqi6M3sb9/lrPIqJ+m0pFhB5AhKovscrLZXPxDUavTxPf2dSbx/vwTnVnFaxwcm5pTtggV0J7dZeXc= ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:40:37.097 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-09T04:40:37.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:37 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2572152142' entity='client.admin' 2024-05-09T04:40:37.350 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:40:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.809+0000 7fe31a9dc700 1 -- 172.21.15.46:0/984827017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe314102ad0 msgr2=0x7fe314102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.809+0000 7fe31a9dc700 1 --2- 172.21.15.46:0/984827017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe314102ad0 0x7fe314102ee0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe304009910 tx=0x7fe304009c20 comp rx=0 tx=0).stop 2024-05-09T04:40:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.809+0000 7fe31a9dc700 1 -- 172.21.15.46:0/984827017 shutdown_connections 2024-05-09T04:40:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.809+0000 7fe31a9dc700 1 --2- 172.21.15.46:0/984827017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe314102ad0 0x7fe314102ee0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.809+0000 7fe31a9dc700 1 -- 172.21.15.46:0/984827017 >> 172.21.15.46:0/984827017 conn(0x7fe3140fe110 msgr2=0x7fe3141005c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.809+0000 7fe31a9dc700 1 -- 172.21.15.46:0/984827017 shutdown_connections 2024-05-09T04:40:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.809+0000 7fe31a9dc700 1 -- 172.21.15.46:0/984827017 wait complete. 2024-05-09T04:40:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.809+0000 7fe31a9dc700 1 Processor -- start 2024-05-09T04:40:39.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.809+0000 7fe31a9dc700 1 -- start start 2024-05-09T04:40:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.810+0000 7fe31a9dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe314102ad0 0x7fe31406c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.810+0000 7fe31a9dc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe314108a90 con 0x7fe314102ad0 2024-05-09T04:40:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.810+0000 7fe313fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe314102ad0 0x7fe31406c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.810+0000 7fe313fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe314102ad0 0x7fe31406c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46238/0 (socket says 172.21.15.46:46238) 2024-05-09T04:40:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.810+0000 7fe313fff700 1 -- 172.21.15.46:0/3717408333 learned_addr learned my addr 172.21.15.46:0/3717408333 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.810+0000 7fe313fff700 1 -- 172.21.15.46:0/3717408333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe31406c6e0 con 0x7fe314102ad0 2024-05-09T04:40:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.811+0000 7fe313fff700 1 --2- 172.21.15.46:0/3717408333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe314102ad0 0x7fe31406c1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe304006010 tx=0x7fe304004fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.811+0000 7fe3117fa700 1 -- 172.21.15.46:0/3717408333 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe304026070 con 0x7fe314102ad0 2024-05-09T04:40:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.811+0000 7fe31a9dc700 1 -- 172.21.15.46:0/3717408333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe31419c4a0 con 0x7fe314102ad0 2024-05-09T04:40:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.811+0000 7fe3117fa700 1 -- 172.21.15.46:0/3717408333 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe30400bc00 con 0x7fe314102ad0 2024-05-09T04:40:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.811+0000 7fe31a9dc700 1 -- 172.21.15.46:0/3717408333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe31419c920 con 0x7fe314102ad0 2024-05-09T04:40:39.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.811+0000 7fe3117fa700 1 -- 172.21.15.46:0/3717408333 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe304003970 con 0x7fe314102ad0 2024-05-09T04:40:39.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.812+0000 7fe31a9dc700 1 -- 172.21.15.46:0/3717408333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe314109210 con 0x7fe314102ad0 2024-05-09T04:40:39.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.812+0000 7fe3117fa700 1 -- 172.21.15.46:0/3717408333 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe304029070 con 0x7fe314102ad0 2024-05-09T04:40:39.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.813+0000 7fe3117fa700 1 --2- 172.21.15.46:0/3717408333 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe2fc03e510 0x7fe2fc040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.813+0000 7fe3117fa700 1 -- 172.21.15.46:0/3717408333 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe30401e070 con 0x7fe314102ad0 2024-05-09T04:40:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.813+0000 7fe3137fe700 1 --2- 172.21.15.46:0/3717408333 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe2fc03e510 0x7fe2fc040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:39.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.813+0000 7fe3137fe700 1 --2- 172.21.15.46:0/3717408333 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe2fc03e510 0x7fe2fc040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe308006fd0 tx=0x7fe308006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:39.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:39.818+0000 7fe3117fa700 1 -- 172.21.15.46:0/3717408333 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe304013070 con 0x7fe314102ad0 2024-05-09T04:40:40.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:40.046+0000 7fe31a9dc700 1 -- 172.21.15.46:0/3717408333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fe31402e7e0 con 0x7fe314102ad0 2024-05-09T04:40:40.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:40.049+0000 7fe3117fa700 1 -- 172.21.15.46:0/3717408333 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fe3040300c0 con 0x7fe314102ad0 2024-05-09T04:40:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:40.052+0000 7fe31a9dc700 1 -- 172.21.15.46:0/3717408333 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe2fc03e510 msgr2=0x7fe2fc040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:40.052+0000 7fe31a9dc700 1 --2- 172.21.15.46:0/3717408333 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe2fc03e510 0x7fe2fc040a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe308006fd0 tx=0x7fe308006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:40.052+0000 7fe31a9dc700 1 -- 172.21.15.46:0/3717408333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe314102ad0 msgr2=0x7fe31406c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:40.052+0000 7fe31a9dc700 1 --2- 172.21.15.46:0/3717408333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe314102ad0 0x7fe31406c1a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe304006010 tx=0x7fe304004fa0 comp rx=0 tx=0).stop 2024-05-09T04:40:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:40.052+0000 7fe31a9dc700 1 -- 172.21.15.46:0/3717408333 shutdown_connections 2024-05-09T04:40:40.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:40.052+0000 7fe31a9dc700 1 --2- 172.21.15.46:0/3717408333 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe2fc03e510 0x7fe2fc040a20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:40.052+0000 7fe31a9dc700 1 --2- 172.21.15.46:0/3717408333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe314102ad0 0x7fe31406c1a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:40.052+0000 7fe31a9dc700 1 -- 172.21.15.46:0/3717408333 >> 172.21.15.46:0/3717408333 conn(0x7fe3140fe110 msgr2=0x7fe3140fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:40.053+0000 7fe31a9dc700 1 -- 172.21.15.46:0/3717408333 shutdown_connections 2024-05-09T04:40:40.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:40.053+0000 7fe31a9dc700 1 -- 172.21.15.46:0/3717408333 wait complete. 2024-05-09T04:40:40.678 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-09T04:40:40.678 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-09T04:40:40.945 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:40:41.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:41 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3717408333' entity='client.admin' 2024-05-09T04:40:42.375 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:42 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:42.376 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:42 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:42.376 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:42 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi046", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:40:42.376 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:42 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:42.376 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:42 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:40:42.376 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:42 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:42.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.870+0000 7f59f68aa700 1 -- 172.21.15.46:0/2187810277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0102ae0 msgr2=0x7f59f0102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.870+0000 7f59f68aa700 1 --2- 172.21.15.46:0/2187810277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0102ae0 0x7f59f0102ef0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009910 tx=0x7f59e0009c20 comp rx=0 tx=0).stop 2024-05-09T04:40:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.870+0000 7f59f68aa700 1 -- 172.21.15.46:0/2187810277 shutdown_connections 2024-05-09T04:40:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.870+0000 7f59f68aa700 1 --2- 172.21.15.46:0/2187810277 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0102ae0 0x7f59f0102ef0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.870+0000 7f59f68aa700 1 -- 172.21.15.46:0/2187810277 >> 172.21.15.46:0/2187810277 conn(0x7f59f00fe140 msgr2=0x7f59f01005d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.870+0000 7f59f68aa700 1 -- 172.21.15.46:0/2187810277 shutdown_connections 2024-05-09T04:40:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.870+0000 7f59f68aa700 1 -- 172.21.15.46:0/2187810277 wait complete. 2024-05-09T04:40:42.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.871+0000 7f59f68aa700 1 Processor -- start 2024-05-09T04:40:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.871+0000 7f59f68aa700 1 -- start start 2024-05-09T04:40:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.871+0000 7f59f68aa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0102ae0 0x7f59f0199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.871+0000 7f59f68aa700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f59f0108aa0 con 0x7f59f0102ae0 2024-05-09T04:40:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.871+0000 7f59effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0102ae0 0x7f59f0199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.871+0000 7f59effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0102ae0 0x7f59f0199710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46254/0 (socket says 172.21.15.46:46254) 2024-05-09T04:40:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.871+0000 7f59effff700 1 -- 172.21.15.46:0/1499237099 learned_addr learned my addr 172.21.15.46:0/1499237099 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.871+0000 7f59effff700 1 -- 172.21.15.46:0/1499237099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f0199c50 con 0x7f59f0102ae0 2024-05-09T04:40:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.872+0000 7f59effff700 1 --2- 172.21.15.46:0/1499237099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0102ae0 0x7f59f0199710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f59e00057d0 tx=0x7f59e0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.872+0000 7f59edffb700 1 -- 172.21.15.46:0/1499237099 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59e0027070 con 0x7f59f0102ae0 2024-05-09T04:40:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.872+0000 7f59f68aa700 1 -- 172.21.15.46:0/1499237099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f0199ee0 con 0x7f59f0102ae0 2024-05-09T04:40:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.872+0000 7f59edffb700 1 -- 172.21.15.46:0/1499237099 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e000be10 con 0x7f59f0102ae0 2024-05-09T04:40:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.872+0000 7f59edffb700 1 -- 172.21.15.46:0/1499237099 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59e0022dd0 con 0x7f59f0102ae0 2024-05-09T04:40:42.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.872+0000 7f59f68aa700 1 -- 172.21.15.46:0/1499237099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f019caf0 con 0x7f59f0102ae0 2024-05-09T04:40:42.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.873+0000 7f59f68aa700 1 -- 172.21.15.46:0/1499237099 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59f0109220 con 0x7f59f0102ae0 2024-05-09T04:40:42.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.873+0000 7f59edffb700 1 -- 172.21.15.46:0/1499237099 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f59e002a070 con 0x7f59f0102ae0 2024-05-09T04:40:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.873+0000 7f59edffb700 1 --2- 172.21.15.46:0/1499237099 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f59dc03e4c0 0x7f59dc0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.873+0000 7f59edffb700 1 -- 172.21.15.46:0/1499237099 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f59e001e030 con 0x7f59f0102ae0 2024-05-09T04:40:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.873+0000 7f59e7fff700 1 --2- 172.21.15.46:0/1499237099 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f59dc03e4c0 0x7f59dc0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:42.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.874+0000 7f59e7fff700 1 --2- 172.21.15.46:0/1499237099 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f59dc03e4c0 0x7f59dc0409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f59d8006fd0 tx=0x7f59d8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:42.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:42.879+0000 7f59edffb700 1 -- 172.21.15.46:0/1499237099 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f59e0013070 con 0x7f59f0102ae0 2024-05-09T04:40:43.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:43.095+0000 7f59f68aa700 1 -- 172.21.15.46:0/1499237099 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f59f002e700 con 0x7f59dc03e4c0 2024-05-09T04:40:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:43.099+0000 7f59edffb700 1 -- 172.21.15.46:0/1499237099 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f59f002e700 con 0x7f59dc03e4c0 2024-05-09T04:40:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:43.102+0000 7f59f68aa700 1 -- 172.21.15.46:0/1499237099 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f59dc03e4c0 msgr2=0x7f59dc0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:43.102+0000 7f59f68aa700 1 --2- 172.21.15.46:0/1499237099 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f59dc03e4c0 0x7f59dc0409d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f59d8006fd0 tx=0x7f59d8006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:43.102+0000 7f59f68aa700 1 -- 172.21.15.46:0/1499237099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0102ae0 msgr2=0x7f59f0199710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:43.102+0000 7f59f68aa700 1 --2- 172.21.15.46:0/1499237099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0102ae0 0x7f59f0199710 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f59e00057d0 tx=0x7f59e0005800 comp rx=0 tx=0).stop 2024-05-09T04:40:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:43.102+0000 7f59f68aa700 1 -- 172.21.15.46:0/1499237099 shutdown_connections 2024-05-09T04:40:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:43.102+0000 7f59f68aa700 1 --2- 172.21.15.46:0/1499237099 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f59dc03e4c0 0x7f59dc0409d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:43.102+0000 7f59f68aa700 1 --2- 172.21.15.46:0/1499237099 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59f0102ae0 0x7f59f0199710 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:43.102+0000 7f59f68aa700 1 -- 172.21.15.46:0/1499237099 >> 172.21.15.46:0/1499237099 conn(0x7f59f00fe140 msgr2=0x7f59f006c180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:43.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:43.102+0000 7f59f68aa700 1 -- 172.21.15.46:0/1499237099 shutdown_connections 2024-05-09T04:40:43.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:43.102+0000 7f59f68aa700 1 -- 172.21.15.46:0/1499237099 wait complete. 2024-05-09T04:40:43.832 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi072 2024-05-09T04:40:43.833 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-09T04:40:43.833 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/etc/ceph/ceph.conf 2024-05-09T04:40:43.851 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-09T04:40:43.851 DEBUG:teuthology.orchestra.run.smithi072:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:40:43.909 INFO:tasks.cephadm:Adding host smithi072 to orchestrator... 2024-05-09T04:40:43.909 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch host add smithi072 2024-05-09T04:40:44.160 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:40:44.387 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:44 smithi046 ceph-mon[30136]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:44.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:44 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:44.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:44 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:40:44.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:44 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:40:44.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:44 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:40:44.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:44 smithi046 ceph-mon[30136]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-09T04:40:44.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:44 smithi046 ceph-mon[30136]: Updating smithi046:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:40:44.388 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:44 smithi046 ceph-mon[30136]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:40:45.424 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:45 smithi046 ceph-mon[30136]: Updating smithi046:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:40:45.424 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:45 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:45.424 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:45 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:45.425 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:45 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:45.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.629+0000 7f6825438700 1 -- 172.21.15.46:0/3816029834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820108b90 msgr2=0x7f6820108fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:45.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.629+0000 7f6825438700 1 --2- 172.21.15.46:0/3816029834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820108b90 0x7f6820108fa0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6810009910 tx=0x7f6810009c20 comp rx=0 tx=0).stop 2024-05-09T04:40:45.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.630+0000 7f6825438700 1 -- 172.21.15.46:0/3816029834 shutdown_connections 2024-05-09T04:40:45.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.630+0000 7f6825438700 1 --2- 172.21.15.46:0/3816029834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820108b90 0x7f6820108fa0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:45.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.630+0000 7f6825438700 1 -- 172.21.15.46:0/3816029834 >> 172.21.15.46:0/3816029834 conn(0x7f6820078d60 msgr2=0x7f68200791a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:45.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.630+0000 7f6825438700 1 -- 172.21.15.46:0/3816029834 shutdown_connections 2024-05-09T04:40:45.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.630+0000 7f6825438700 1 -- 172.21.15.46:0/3816029834 wait complete. 2024-05-09T04:40:45.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.630+0000 7f6825438700 1 Processor -- start 2024-05-09T04:40:45.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.630+0000 7f6825438700 1 -- start start 2024-05-09T04:40:45.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.631+0000 7f6825438700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820108b90 0x7f682019f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:45.631 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.631+0000 7f6825438700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f682010e230 con 0x7f6820108b90 2024-05-09T04:40:45.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.631+0000 7f681effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820108b90 0x7f682019f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:45.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.631+0000 7f681effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820108b90 0x7f682019f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46796/0 (socket says 172.21.15.46:46796) 2024-05-09T04:40:45.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.631+0000 7f681effd700 1 -- 172.21.15.46:0/387806741 learned_addr learned my addr 172.21.15.46:0/387806741 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:45.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.631+0000 7f681effd700 1 -- 172.21.15.46:0/387806741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f682019fb30 con 0x7f6820108b90 2024-05-09T04:40:45.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.631+0000 7f681effd700 1 --2- 172.21.15.46:0/387806741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820108b90 0x7f682019f5f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f68100056d0 tx=0x7f68100057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:45.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.631+0000 7f6807fff700 1 -- 172.21.15.46:0/387806741 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6810027070 con 0x7f6820108b90 2024-05-09T04:40:45.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.631+0000 7f6807fff700 1 -- 172.21.15.46:0/387806741 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f681000be10 con 0x7f6820108b90 2024-05-09T04:40:45.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.631+0000 7f6825438700 1 -- 172.21.15.46:0/387806741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f682019fdc0 con 0x7f6820108b90 2024-05-09T04:40:45.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.631+0000 7f6807fff700 1 -- 172.21.15.46:0/387806741 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6810022d20 con 0x7f6820108b90 2024-05-09T04:40:45.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.632+0000 7f6825438700 1 -- 172.21.15.46:0/387806741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6820199830 con 0x7f6820108b90 2024-05-09T04:40:45.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.632+0000 7f6825438700 1 -- 172.21.15.46:0/387806741 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f68201053b0 con 0x7f6820108b90 2024-05-09T04:40:45.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.633+0000 7f6807fff700 1 -- 172.21.15.46:0/387806741 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f681002a070 con 0x7f6820108b90 2024-05-09T04:40:45.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.633+0000 7f6807fff700 1 --2- 172.21.15.46:0/387806741 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f680803e510 0x7f6808040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:45.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.633+0000 7f6807fff700 1 -- 172.21.15.46:0/387806741 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f681001e030 con 0x7f6820108b90 2024-05-09T04:40:45.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.633+0000 7f681e7fc700 1 --2- 172.21.15.46:0/387806741 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f680803e510 0x7f6808040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:45.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.633+0000 7f681e7fc700 1 --2- 172.21.15.46:0/387806741 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f680803e510 0x7f6808040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6814006fd0 tx=0x7f6814006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.638+0000 7f6807fff700 1 -- 172.21.15.46:0/387806741 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6810013070 con 0x7f6820108b90 2024-05-09T04:40:45.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:45.859+0000 7f6825438700 1 -- 172.21.15.46:0/387806741 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi072", "target": ["mon-mgr", ""]}) -- 0x7f682019ad40 con 0x7f680803e510 2024-05-09T04:40:47.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:46 smithi046 ceph-mon[30136]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi072", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:47.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:46 smithi046 ceph-mon[30136]: Deploying cephadm binary to smithi072 2024-05-09T04:40:48.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:47 smithi046 ceph-mon[30136]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:40:50.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:50.084+0000 7f6807fff700 1 -- 172.21.15.46:0/387806741 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f682019ad40 con 0x7f680803e510 2024-05-09T04:40:50.085 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi072' with addr '172.21.15.72' 2024-05-09T04:40:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:50.088+0000 7f6825438700 1 -- 172.21.15.46:0/387806741 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f680803e510 msgr2=0x7f6808040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:50.088+0000 7f6825438700 1 --2- 172.21.15.46:0/387806741 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f680803e510 0x7f6808040a20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6814006fd0 tx=0x7f6814006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:50.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:50.088+0000 7f6825438700 1 -- 172.21.15.46:0/387806741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820108b90 msgr2=0x7f682019f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:50.088+0000 7f6825438700 1 --2- 172.21.15.46:0/387806741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820108b90 0x7f682019f5f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f68100056d0 tx=0x7f68100057b0 comp rx=0 tx=0).stop 2024-05-09T04:40:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:50.088+0000 7f6825438700 1 -- 172.21.15.46:0/387806741 shutdown_connections 2024-05-09T04:40:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:50.088+0000 7f6825438700 1 --2- 172.21.15.46:0/387806741 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f680803e510 0x7f6808040a20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:50.088+0000 7f6825438700 1 --2- 172.21.15.46:0/387806741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6820108b90 0x7f682019f5f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:50.088+0000 7f6825438700 1 -- 172.21.15.46:0/387806741 >> 172.21.15.46:0/387806741 conn(0x7f6820078d60 msgr2=0x7f68200fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:50.088+0000 7f6825438700 1 -- 172.21.15.46:0/387806741 shutdown_connections 2024-05-09T04:40:50.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:50.088+0000 7f6825438700 1 -- 172.21.15.46:0/387806741 wait complete. 2024-05-09T04:40:50.404 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:50 smithi046 ceph-mon[30136]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:40:50.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:50.405 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:40:50.757 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T04:40:51.008 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:40:52.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:51 smithi046 ceph-mon[30136]: Added host smithi072 2024-05-09T04:40:52.044 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:51 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:52.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:51 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:52.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.349+0000 7fd68e90b700 1 -- 172.21.15.46:0/3692825365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd688103dd0 msgr2=0x7fd6881041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:52.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.349+0000 7fd68e90b700 1 --2- 172.21.15.46:0/3692825365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd688103dd0 0x7fd6881041e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fd678009910 tx=0x7fd678009c20 comp rx=0 tx=0).stop 2024-05-09T04:40:52.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.349+0000 7fd68e90b700 1 -- 172.21.15.46:0/3692825365 shutdown_connections 2024-05-09T04:40:52.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.349+0000 7fd68e90b700 1 --2- 172.21.15.46:0/3692825365 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd688103dd0 0x7fd6881041e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:52.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.349+0000 7fd68e90b700 1 -- 172.21.15.46:0/3692825365 >> 172.21.15.46:0/3692825365 conn(0x7fd6880ff660 msgr2=0x7fd688101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:52.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.349+0000 7fd68e90b700 1 -- 172.21.15.46:0/3692825365 shutdown_connections 2024-05-09T04:40:52.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.349+0000 7fd68e90b700 1 -- 172.21.15.46:0/3692825365 wait complete. 2024-05-09T04:40:52.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.349+0000 7fd68e90b700 1 Processor -- start 2024-05-09T04:40:52.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.349+0000 7fd68e90b700 1 -- start start 2024-05-09T04:40:52.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.350+0000 7fd68e90b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd688103dd0 0x7fd68819f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:52.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.350+0000 7fd68e90b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd68810e0c0 con 0x7fd688103dd0 2024-05-09T04:40:52.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.350+0000 7fd687fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd688103dd0 0x7fd68819f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:52.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.350+0000 7fd687fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd688103dd0 0x7fd68819f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46810/0 (socket says 172.21.15.46:46810) 2024-05-09T04:40:52.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.350+0000 7fd687fff700 1 -- 172.21.15.46:0/2522784240 learned_addr learned my addr 172.21.15.46:0/2522784240 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:52.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.350+0000 7fd687fff700 1 -- 172.21.15.46:0/2522784240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd68819fb30 con 0x7fd688103dd0 2024-05-09T04:40:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.350+0000 7fd687fff700 1 --2- 172.21.15.46:0/2522784240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd688103dd0 0x7fd68819f5f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd678006010 tx=0x7fd678005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.350+0000 7fd685ffb700 1 -- 172.21.15.46:0/2522784240 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd678026070 con 0x7fd688103dd0 2024-05-09T04:40:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.350+0000 7fd685ffb700 1 -- 172.21.15.46:0/2522784240 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd67800bba0 con 0x7fd688103dd0 2024-05-09T04:40:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.350+0000 7fd685ffb700 1 -- 172.21.15.46:0/2522784240 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd678003910 con 0x7fd688103dd0 2024-05-09T04:40:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.350+0000 7fd68e90b700 1 -- 172.21.15.46:0/2522784240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd68819fe20 con 0x7fd688103dd0 2024-05-09T04:40:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.351+0000 7fd68e90b700 1 -- 172.21.15.46:0/2522784240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd688199950 con 0x7fd688103dd0 2024-05-09T04:40:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.351+0000 7fd68e90b700 1 -- 172.21.15.46:0/2522784240 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd688105280 con 0x7fd688103dd0 2024-05-09T04:40:52.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.352+0000 7fd685ffb700 1 -- 172.21.15.46:0/2522784240 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd678029070 con 0x7fd688103dd0 2024-05-09T04:40:52.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.352+0000 7fd685ffb700 1 --2- 172.21.15.46:0/2522784240 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd67403e510 0x7fd674040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:52.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.352+0000 7fd685ffb700 1 -- 172.21.15.46:0/2522784240 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd67801e070 con 0x7fd688103dd0 2024-05-09T04:40:52.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.352+0000 7fd67ffff700 1 --2- 172.21.15.46:0/2522784240 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd67403e510 0x7fd674040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:52.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.352+0000 7fd67ffff700 1 --2- 172.21.15.46:0/2522784240 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd67403e510 0x7fd674040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd670006fd0 tx=0x7fd670006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:52.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.357+0000 7fd685ffb700 1 -- 172.21.15.46:0/2522784240 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd678013070 con 0x7fd688103dd0 2024-05-09T04:40:52.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.576+0000 7fd68e90b700 1 -- 172.21.15.46:0/2522784240 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd68819ac90 con 0x7fd67403e510 2024-05-09T04:40:52.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.578+0000 7fd685ffb700 1 -- 172.21.15.46:0/2522784240 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7fd68819ac90 con 0x7fd67403e510 2024-05-09T04:40:52.577 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:40:52.577 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.72", "hostname": "smithi072", "labels": [], "status": ""}] 2024-05-09T04:40:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.580+0000 7fd68e90b700 1 -- 172.21.15.46:0/2522784240 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd67403e510 msgr2=0x7fd674040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.580+0000 7fd68e90b700 1 --2- 172.21.15.46:0/2522784240 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd67403e510 0x7fd674040a20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd670006fd0 tx=0x7fd670006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.580+0000 7fd68e90b700 1 -- 172.21.15.46:0/2522784240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd688103dd0 msgr2=0x7fd68819f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.580+0000 7fd68e90b700 1 --2- 172.21.15.46:0/2522784240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd688103dd0 0x7fd68819f5f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fd678006010 tx=0x7fd678005780 comp rx=0 tx=0).stop 2024-05-09T04:40:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.580+0000 7fd68e90b700 1 -- 172.21.15.46:0/2522784240 shutdown_connections 2024-05-09T04:40:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.580+0000 7fd68e90b700 1 --2- 172.21.15.46:0/2522784240 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd67403e510 0x7fd674040a20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.580+0000 7fd68e90b700 1 --2- 172.21.15.46:0/2522784240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd688103dd0 0x7fd68819f5f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.580+0000 7fd68e90b700 1 -- 172.21.15.46:0/2522784240 >> 172.21.15.46:0/2522784240 conn(0x7fd6880ff660 msgr2=0x7fd688109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:52.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.580+0000 7fd68e90b700 1 -- 172.21.15.46:0/2522784240 shutdown_connections 2024-05-09T04:40:52.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:52.580+0000 7fd68e90b700 1 -- 172.21.15.46:0/2522784240 wait complete. 2024-05-09T04:40:52.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:52 smithi046 ceph-mon[30136]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:40:53.165 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi110 2024-05-09T04:40:53.165 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-09T04:40:53.166 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/etc/ceph/ceph.conf 2024-05-09T04:40:53.184 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-09T04:40:53.184 DEBUG:teuthology.orchestra.run.smithi110:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:40:53.241 INFO:tasks.cephadm:Adding host smithi110 to orchestrator... 2024-05-09T04:40:53.242 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch host add smithi110 2024-05-09T04:40:53.493 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:40:53.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:53 smithi046 ceph-mon[30136]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:40:53.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:53 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:54.735 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:54 smithi046 ceph-mon[30136]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:40:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.948+0000 7fcc733d7700 1 -- 172.21.15.46:0/2960024934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc6c103dd0 msgr2=0x7fcc6c1041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.948+0000 7fcc733d7700 1 --2- 172.21.15.46:0/2960024934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc6c103dd0 0x7fcc6c1041e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fcc60009910 tx=0x7fcc60009c20 comp rx=0 tx=0).stop 2024-05-09T04:40:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.948+0000 7fcc733d7700 1 -- 172.21.15.46:0/2960024934 shutdown_connections 2024-05-09T04:40:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.948+0000 7fcc733d7700 1 --2- 172.21.15.46:0/2960024934 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc6c103dd0 0x7fcc6c1041e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.948+0000 7fcc733d7700 1 -- 172.21.15.46:0/2960024934 >> 172.21.15.46:0/2960024934 conn(0x7fcc6c0ff660 msgr2=0x7fcc6c101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.948+0000 7fcc733d7700 1 -- 172.21.15.46:0/2960024934 shutdown_connections 2024-05-09T04:40:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.948+0000 7fcc733d7700 1 -- 172.21.15.46:0/2960024934 wait complete. 2024-05-09T04:40:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.949+0000 7fcc733d7700 1 Processor -- start 2024-05-09T04:40:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.949+0000 7fcc733d7700 1 -- start start 2024-05-09T04:40:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.949+0000 7fcc733d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc6c103dd0 0x7fcc6c075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.949+0000 7fcc733d7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcc6c10e0c0 con 0x7fcc6c103dd0 2024-05-09T04:40:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.949+0000 7fcc71173700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc6c103dd0 0x7fcc6c075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.949+0000 7fcc71173700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc6c103dd0 0x7fcc6c075ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58832/0 (socket says 172.21.15.46:58832) 2024-05-09T04:40:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.949+0000 7fcc71173700 1 -- 172.21.15.46:0/3063965323 learned_addr learned my addr 172.21.15.46:0/3063965323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:40:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.949+0000 7fcc71173700 1 -- 172.21.15.46:0/3063965323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6c077910 con 0x7fcc6c103dd0 2024-05-09T04:40:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.949+0000 7fcc71173700 1 --2- 172.21.15.46:0/3063965323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc6c103dd0 0x7fcc6c075ac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc60006010 tx=0x7fcc60005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.950+0000 7fcc5e7fc700 1 -- 172.21.15.46:0/3063965323 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc60026070 con 0x7fcc6c103dd0 2024-05-09T04:40:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.950+0000 7fcc5e7fc700 1 -- 172.21.15.46:0/3063965323 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc6000bba0 con 0x7fcc6c103dd0 2024-05-09T04:40:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.950+0000 7fcc733d7700 1 -- 172.21.15.46:0/3063965323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6c076000 con 0x7fcc6c103dd0 2024-05-09T04:40:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.950+0000 7fcc5e7fc700 1 -- 172.21.15.46:0/3063965323 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc60003910 con 0x7fcc6c103dd0 2024-05-09T04:40:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.950+0000 7fcc733d7700 1 -- 172.21.15.46:0/3063965323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc6c0764c0 con 0x7fcc6c103dd0 2024-05-09T04:40:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.951+0000 7fcc733d7700 1 -- 172.21.15.46:0/3063965323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc6c105280 con 0x7fcc6c103dd0 2024-05-09T04:40:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.951+0000 7fcc5e7fc700 1 -- 172.21.15.46:0/3063965323 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fcc60029070 con 0x7fcc6c103dd0 2024-05-09T04:40:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.951+0000 7fcc5e7fc700 1 --2- 172.21.15.46:0/3063965323 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fcc5803e570 0x7fcc58040a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:40:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.951+0000 7fcc5e7fc700 1 -- 172.21.15.46:0/3063965323 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcc6001e070 con 0x7fcc6c103dd0 2024-05-09T04:40:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.951+0000 7fcc70972700 1 --2- 172.21.15.46:0/3063965323 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fcc5803e570 0x7fcc58040a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:40:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.952+0000 7fcc70972700 1 --2- 172.21.15.46:0/3063965323 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fcc5803e570 0x7fcc58040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcc68006fd0 tx=0x7fcc68006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:40:54.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:54.956+0000 7fcc5e7fc700 1 -- 172.21.15.46:0/3063965323 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc60013070 con 0x7fcc6c103dd0 2024-05-09T04:40:55.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:55.176+0000 7fcc733d7700 1 -- 172.21.15.46:0/3063965323 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi110", "target": ["mon-mgr", ""]}) -- 0x7fcc6c077060 con 0x7fcc5803e570 2024-05-09T04:40:56.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:56 smithi046 ceph-mon[30136]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi110", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:40:56.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:56 smithi046 ceph-mon[30136]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:40:56.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:56.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:56.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:56.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:40:56.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi072", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:40:56.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:40:56.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:40:56.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:56 smithi046 ceph-mon[30136]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-09T04:40:58.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:57 smithi046 ceph-mon[30136]: Deploying cephadm binary to smithi110 2024-05-09T04:40:58.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:57 smithi046 ceph-mon[30136]: Updating smithi072:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:40:58.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:57 smithi046 ceph-mon[30136]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:40:59.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:40:58 smithi046 ceph-mon[30136]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:40:59.509 INFO:teuthology.orchestra.run.smithi046.stdout:Added host 'smithi110' with addr '172.21.15.110' 2024-05-09T04:40:59.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:59.509+0000 7fcc5e7fc700 1 -- 172.21.15.46:0/3063965323 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fcc6c077060 con 0x7fcc5803e570 2024-05-09T04:40:59.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:59.512+0000 7fcc733d7700 1 -- 172.21.15.46:0/3063965323 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fcc5803e570 msgr2=0x7fcc58040a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:59.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:59.512+0000 7fcc733d7700 1 --2- 172.21.15.46:0/3063965323 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fcc5803e570 0x7fcc58040a80 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fcc68006fd0 tx=0x7fcc68006e40 comp rx=0 tx=0).stop 2024-05-09T04:40:59.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:59.512+0000 7fcc733d7700 1 -- 172.21.15.46:0/3063965323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc6c103dd0 msgr2=0x7fcc6c075ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:40:59.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:59.512+0000 7fcc733d7700 1 --2- 172.21.15.46:0/3063965323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc6c103dd0 0x7fcc6c075ac0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fcc60006010 tx=0x7fcc60005780 comp rx=0 tx=0).stop 2024-05-09T04:40:59.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:59.512+0000 7fcc733d7700 1 -- 172.21.15.46:0/3063965323 shutdown_connections 2024-05-09T04:40:59.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:59.512+0000 7fcc733d7700 1 --2- 172.21.15.46:0/3063965323 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fcc5803e570 0x7fcc58040a80 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:59.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:59.512+0000 7fcc733d7700 1 --2- 172.21.15.46:0/3063965323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc6c103dd0 0x7fcc6c075ac0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:40:59.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:59.512+0000 7fcc733d7700 1 -- 172.21.15.46:0/3063965323 >> 172.21.15.46:0/3063965323 conn(0x7fcc6c0ff660 msgr2=0x7fcc6c109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:40:59.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:59.512+0000 7fcc733d7700 1 -- 172.21.15.46:0/3063965323 shutdown_connections 2024-05-09T04:40:59.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:40:59.512+0000 7fcc733d7700 1 -- 172.21.15.46:0/3063965323 wait complete. 2024-05-09T04:41:00.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:00 smithi046 ceph-mon[30136]: Updating smithi072:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:41:00.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:00 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:00.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:00 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:00.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:00 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:00.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:00 smithi046 ceph-mon[30136]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:00.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:00 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:00.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:00 smithi046 ceph-mon[30136]: Added host smithi110 2024-05-09T04:41:00.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:00 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:00.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:00 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:00.371 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-09T04:41:00.623 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:41:01.660 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:01 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:02.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.141+0000 7f2e6c274700 1 -- 172.21.15.46:0/1820948442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e64108a90 msgr2=0x7f2e64108ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:02.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.141+0000 7f2e6c274700 1 --2- 172.21.15.46:0/1820948442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e64108a90 0x7f2e64108ea0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000b0a0 tx=0x7f2e6000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:41:02.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.142+0000 7f2e6c274700 1 -- 172.21.15.46:0/1820948442 shutdown_connections 2024-05-09T04:41:02.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.142+0000 7f2e6c274700 1 --2- 172.21.15.46:0/1820948442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e64108a90 0x7f2e64108ea0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:02.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.142+0000 7f2e6c274700 1 -- 172.21.15.46:0/1820948442 >> 172.21.15.46:0/1820948442 conn(0x7f2e640785b0 msgr2=0x7f2e64078a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:02.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.142+0000 7f2e6c274700 1 -- 172.21.15.46:0/1820948442 shutdown_connections 2024-05-09T04:41:02.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.142+0000 7f2e6c274700 1 -- 172.21.15.46:0/1820948442 wait complete. 2024-05-09T04:41:02.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.142+0000 7f2e6c274700 1 Processor -- start 2024-05-09T04:41:02.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.142+0000 7f2e6c274700 1 -- start start 2024-05-09T04:41:02.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.143+0000 7f2e6c274700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e64108a90 0x7f2e640ff880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:02.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.143+0000 7f2e6c274700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2e6410cfe0 con 0x7f2e64108a90 2024-05-09T04:41:02.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.143+0000 7f2e6a010700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e64108a90 0x7f2e640ff880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:02.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.143+0000 7f2e6a010700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e64108a90 0x7f2e640ff880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58850/0 (socket says 172.21.15.46:58850) 2024-05-09T04:41:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.143+0000 7f2e6a010700 1 -- 172.21.15.46:0/3617053618 learned_addr learned my addr 172.21.15.46:0/3617053618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:41:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.143+0000 7f2e6a010700 1 -- 172.21.15.46:0/3617053618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e641016d0 con 0x7f2e64108a90 2024-05-09T04:41:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.143+0000 7f2e6a010700 1 --2- 172.21.15.46:0/3617053618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e64108a90 0x7f2e640ff880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000d010 tx=0x7f2e60003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.143+0000 7f2e5b7fe700 1 -- 172.21.15.46:0/3617053618 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e60021020 con 0x7f2e64108a90 2024-05-09T04:41:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.143+0000 7f2e5b7fe700 1 -- 172.21.15.46:0/3617053618 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e60007a40 con 0x7f2e64108a90 2024-05-09T04:41:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.143+0000 7f2e5b7fe700 1 -- 172.21.15.46:0/3617053618 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2e60017040 con 0x7f2e64108a90 2024-05-09T04:41:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.143+0000 7f2e6c274700 1 -- 172.21.15.46:0/3617053618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e640ffdc0 con 0x7f2e64108a90 2024-05-09T04:41:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.143+0000 7f2e6c274700 1 -- 172.21.15.46:0/3617053618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e64100220 con 0x7f2e64108a90 2024-05-09T04:41:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.144+0000 7f2e6c274700 1 -- 172.21.15.46:0/3617053618 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e641041a0 con 0x7f2e64108a90 2024-05-09T04:41:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.144+0000 7f2e5b7fe700 1 -- 172.21.15.46:0/3617053618 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f2e60024020 con 0x7f2e64108a90 2024-05-09T04:41:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.145+0000 7f2e5b7fe700 1 --2- 172.21.15.46:0/3617053618 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2e5003e560 0x7f2e50040a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.145+0000 7f2e5b7fe700 1 -- 172.21.15.46:0/3617053618 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2e6000f050 con 0x7f2e64108a90 2024-05-09T04:41:02.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.145+0000 7f2e6980f700 1 --2- 172.21.15.46:0/3617053618 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2e5003e560 0x7f2e50040a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.145+0000 7f2e6980f700 1 --2- 172.21.15.46:0/3617053618 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2e5003e560 0x7f2e50040a70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2e54006fd0 tx=0x7f2e54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:02.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.150+0000 7f2e5b7fe700 1 -- 172.21.15.46:0/3617053618 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2e6002b020 con 0x7f2e64108a90 2024-05-09T04:41:02.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.381+0000 7f2e6c274700 1 -- 172.21.15.46:0/3617053618 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e64100db0 con 0x7f2e5003e560 2024-05-09T04:41:02.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.382+0000 7f2e5b7fe700 1 -- 172.21.15.46:0/3617053618 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f2e64100db0 con 0x7f2e5003e560 2024-05-09T04:41:02.381 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:41:02.382 INFO:teuthology.orchestra.run.smithi046.stdout:[{"addr": "172.21.15.46", "hostname": "smithi046", "labels": [], "status": ""}, {"addr": "172.21.15.72", "hostname": "smithi072", "labels": [], "status": ""}, {"addr": "172.21.15.110", "hostname": "smithi110", "labels": [], "status": ""}] 2024-05-09T04:41:02.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.384+0000 7f2e6c274700 1 -- 172.21.15.46:0/3617053618 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2e5003e560 msgr2=0x7f2e50040a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:02.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.384+0000 7f2e6c274700 1 --2- 172.21.15.46:0/3617053618 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2e5003e560 0x7f2e50040a70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f2e54006fd0 tx=0x7f2e54006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:02.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.384+0000 7f2e6c274700 1 -- 172.21.15.46:0/3617053618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e64108a90 msgr2=0x7f2e640ff880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:02.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.384+0000 7f2e6c274700 1 --2- 172.21.15.46:0/3617053618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e64108a90 0x7f2e640ff880 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f2e6000d010 tx=0x7f2e60003ce0 comp rx=0 tx=0).stop 2024-05-09T04:41:02.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.384+0000 7f2e6c274700 1 -- 172.21.15.46:0/3617053618 shutdown_connections 2024-05-09T04:41:02.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.384+0000 7f2e6c274700 1 --2- 172.21.15.46:0/3617053618 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2e5003e560 0x7f2e50040a70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:02.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.384+0000 7f2e6c274700 1 --2- 172.21.15.46:0/3617053618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2e64108a90 0x7f2e640ff880 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:02.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.384+0000 7f2e6c274700 1 -- 172.21.15.46:0/3617053618 >> 172.21.15.46:0/3617053618 conn(0x7f2e640785b0 msgr2=0x7f2e6406adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:02.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.384+0000 7f2e6c274700 1 -- 172.21.15.46:0/3617053618 shutdown_connections 2024-05-09T04:41:02.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:02.384+0000 7f2e6c274700 1 -- 172.21.15.46:0/3617053618 wait complete. 2024-05-09T04:41:02.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:02 smithi046 ceph-mon[30136]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:02.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:02.913 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-09T04:41:02.913 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-09T04:41:03.159 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:41:03.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:03 smithi046 ceph-mon[30136]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:41:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.792+0000 7f1d5a661700 1 -- 172.21.15.46:0/1884298496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d54108bb0 msgr2=0x7f1d54108fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.792+0000 7f1d5a661700 1 --2- 172.21.15.46:0/1884298496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d54108bb0 0x7f1d54108fc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1d44009910 tx=0x7f1d44009c20 comp rx=0 tx=0).stop 2024-05-09T04:41:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.792+0000 7f1d5a661700 1 -- 172.21.15.46:0/1884298496 shutdown_connections 2024-05-09T04:41:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.792+0000 7f1d5a661700 1 --2- 172.21.15.46:0/1884298496 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d54108bb0 0x7f1d54108fc0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.792+0000 7f1d5a661700 1 -- 172.21.15.46:0/1884298496 >> 172.21.15.46:0/1884298496 conn(0x7f1d54077d30 msgr2=0x7f1d54078170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:04.792 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.793+0000 7f1d5a661700 1 -- 172.21.15.46:0/1884298496 shutdown_connections 2024-05-09T04:41:04.793 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.793+0000 7f1d5a661700 1 -- 172.21.15.46:0/1884298496 wait complete. 2024-05-09T04:41:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.793+0000 7f1d5a661700 1 Processor -- start 2024-05-09T04:41:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.793+0000 7f1d5a661700 1 -- start start 2024-05-09T04:41:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.794+0000 7f1d5a661700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d54108bb0 0x7f1d5419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.794+0000 7f1d5a661700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1d5410e230 con 0x7f1d54108bb0 2024-05-09T04:41:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.794+0000 7f1d53fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d54108bb0 0x7f1d5419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.794+0000 7f1d53fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d54108bb0 0x7f1d5419f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36712/0 (socket says 172.21.15.46:36712) 2024-05-09T04:41:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.794+0000 7f1d53fff700 1 -- 172.21.15.46:0/1964691053 learned_addr learned my addr 172.21.15.46:0/1964691053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:41:04.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.794+0000 7f1d53fff700 1 -- 172.21.15.46:0/1964691053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d5419fb30 con 0x7f1d54108bb0 2024-05-09T04:41:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.794+0000 7f1d53fff700 1 --2- 172.21.15.46:0/1964691053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d54108bb0 0x7f1d5419f5f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400b5c0 tx=0x7f1d44005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.794+0000 7f1d51ffb700 1 -- 172.21.15.46:0/1964691053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d44026070 con 0x7f1d54108bb0 2024-05-09T04:41:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.794+0000 7f1d51ffb700 1 -- 172.21.15.46:0/1964691053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d4400b750 con 0x7f1d54108bb0 2024-05-09T04:41:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.794+0000 7f1d51ffb700 1 -- 172.21.15.46:0/1964691053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d440037d0 con 0x7f1d54108bb0 2024-05-09T04:41:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.794+0000 7f1d5a661700 1 -- 172.21.15.46:0/1964691053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d5419fdc0 con 0x7f1d54108bb0 2024-05-09T04:41:04.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.794+0000 7f1d5a661700 1 -- 172.21.15.46:0/1964691053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d54199830 con 0x7f1d54108bb0 2024-05-09T04:41:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.795+0000 7f1d51ffb700 1 -- 172.21.15.46:0/1964691053 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f1d44029070 con 0x7f1d54108bb0 2024-05-09T04:41:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.796+0000 7f1d51ffb700 1 --2- 172.21.15.46:0/1964691053 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1d4003e510 0x7f1d40040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.796+0000 7f1d5a661700 1 -- 172.21.15.46:0/1964691053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d541052c0 con 0x7f1d54108bb0 2024-05-09T04:41:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.796+0000 7f1d51ffb700 1 -- 172.21.15.46:0/1964691053 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1d4401e070 con 0x7f1d54108bb0 2024-05-09T04:41:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.796+0000 7f1d4bfff700 1 --2- 172.21.15.46:0/1964691053 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1d4003e510 0x7f1d40040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:04.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.796+0000 7f1d4bfff700 1 --2- 172.21.15.46:0/1964691053 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1d4003e510 0x7f1d40040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c006fd0 tx=0x7f1d3c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:04.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:04.801+0000 7f1d51ffb700 1 -- 172.21.15.46:0/1964691053 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1d44013070 con 0x7f1d54108bb0 2024-05-09T04:41:05.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:05.021+0000 7f1d5a661700 1 -- 172.21.15.46:0/1964691053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f1d5419ad40 con 0x7f1d54108bb0 2024-05-09T04:41:05.020 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:04 smithi046 ceph-mon[30136]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:05.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:05.585+0000 7f1d51ffb700 1 -- 172.21.15.46:0/1964691053 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7f1d4400e3e0 con 0x7f1d54108bb0 2024-05-09T04:41:05.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:05.589+0000 7f1d5a661700 1 -- 172.21.15.46:0/1964691053 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1d4003e510 msgr2=0x7f1d40040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:05.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:05.589+0000 7f1d5a661700 1 --2- 172.21.15.46:0/1964691053 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1d4003e510 0x7f1d40040a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c006fd0 tx=0x7f1d3c006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:05.589+0000 7f1d5a661700 1 -- 172.21.15.46:0/1964691053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d54108bb0 msgr2=0x7f1d5419f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:05.589+0000 7f1d5a661700 1 --2- 172.21.15.46:0/1964691053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d54108bb0 0x7f1d5419f5f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1d4400b5c0 tx=0x7f1d44005780 comp rx=0 tx=0).stop 2024-05-09T04:41:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:05.589+0000 7f1d5a661700 1 -- 172.21.15.46:0/1964691053 shutdown_connections 2024-05-09T04:41:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:05.589+0000 7f1d5a661700 1 --2- 172.21.15.46:0/1964691053 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1d4003e510 0x7f1d40040a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:05.589+0000 7f1d5a661700 1 --2- 172.21.15.46:0/1964691053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1d54108bb0 0x7f1d5419f5f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:05.589+0000 7f1d5a661700 1 -- 172.21.15.46:0/1964691053 >> 172.21.15.46:0/1964691053 conn(0x7f1d54077d30 msgr2=0x7f1d54100010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:05.589+0000 7f1d5a661700 1 -- 172.21.15.46:0/1964691053 shutdown_connections 2024-05-09T04:41:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:05.589+0000 7f1d5a661700 1 -- 172.21.15.46:0/1964691053 wait complete. 2024-05-09T04:41:05.589 INFO:teuthology.orchestra.run.smithi046.stderr:adjusted tunables profile to default 2024-05-09T04:41:05.895 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:05 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1964691053' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T04:41:05.895 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:05.896 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:05.896 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:05.896 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:05.896 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:41:05.896 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:05.896 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:06.209 INFO:tasks.cephadm:Adding mon.a on smithi046 2024-05-09T04:41:06.210 INFO:tasks.cephadm:Adding mon.b on smithi072 2024-05-09T04:41:06.210 INFO:tasks.cephadm:Adding mon.c on smithi110 2024-05-09T04:41:06.210 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi046:172.21.15.46=a;smithi072:172.21.15.72=b;smithi110:172.21.15.110=c' 2024-05-09T04:41:06.437 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:07.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:06 smithi046 ceph-mon[30136]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:07.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:06 smithi046 ceph-mon[30136]: Updating smithi110:/etc/ceph/ceph.conf 2024-05-09T04:41:07.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:06 smithi046 ceph-mon[30136]: Updating smithi110:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:07.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:06 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1964691053' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T04:41:07.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:06 smithi046 ceph-mon[30136]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:41:07.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:06 smithi046 ceph-mon[30136]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:41:07.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:06 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:07.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:06 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:07.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:06 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:07.845 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.845+0000 7f5d589f2700 1 -- 172.21.15.110:0/3845158762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d54103dd0 msgr2=0x7f5d541041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:07.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.845+0000 7f5d589f2700 1 --2- 172.21.15.110:0/3845158762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d54103dd0 0x7f5d541041e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5d44009910 tx=0x7f5d44009c20 comp rx=0 tx=0).stop 2024-05-09T04:41:07.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.845+0000 7f5d589f2700 1 -- 172.21.15.110:0/3845158762 shutdown_connections 2024-05-09T04:41:07.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.845+0000 7f5d589f2700 1 --2- 172.21.15.110:0/3845158762 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d54103dd0 0x7f5d541041e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:07.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.845+0000 7f5d589f2700 1 -- 172.21.15.110:0/3845158762 >> 172.21.15.110:0/3845158762 conn(0x7f5d540ff680 msgr2=0x7f5d54101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:07.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.845+0000 7f5d589f2700 1 -- 172.21.15.110:0/3845158762 shutdown_connections 2024-05-09T04:41:07.846 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.845+0000 7f5d589f2700 1 -- 172.21.15.110:0/3845158762 wait complete. 2024-05-09T04:41:07.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.846+0000 7f5d589f2700 1 Processor -- start 2024-05-09T04:41:07.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.846+0000 7f5d589f2700 1 -- start start 2024-05-09T04:41:07.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.846+0000 7f5d589f2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d54103dd0 0x7f5d5419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:07.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.846+0000 7f5d589f2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d5410e0c0 con 0x7f5d54103dd0 2024-05-09T04:41:07.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.846+0000 7f5d5259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d54103dd0 0x7f5d5419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:07.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.846+0000 7f5d5259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d54103dd0 0x7f5d5419f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.110:53780/0 (socket says 172.21.15.110:53780) 2024-05-09T04:41:07.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.846+0000 7f5d5259c700 1 -- 172.21.15.110:0/1956073574 learned_addr learned my addr 172.21.15.110:0/1956073574 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-09T04:41:07.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.847+0000 7f5d5259c700 1 -- 172.21.15.110:0/1956073574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d5419faa0 con 0x7f5d54103dd0 2024-05-09T04:41:07.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.847+0000 7f5d5259c700 1 --2- 172.21.15.110:0/1956073574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d54103dd0 0x7f5d5419f560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5d440056d0 tx=0x7f5d440057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:07.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.847+0000 7f5d437fe700 1 -- 172.21.15.110:0/1956073574 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d44026070 con 0x7f5d54103dd0 2024-05-09T04:41:07.848 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.847+0000 7f5d437fe700 1 -- 172.21.15.110:0/1956073574 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d4400bba0 con 0x7f5d54103dd0 2024-05-09T04:41:07.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.847+0000 7f5d589f2700 1 -- 172.21.15.110:0/1956073574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d5419fd90 con 0x7f5d54103dd0 2024-05-09T04:41:07.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.847+0000 7f5d437fe700 1 -- 172.21.15.110:0/1956073574 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5d44003920 con 0x7f5d54103dd0 2024-05-09T04:41:07.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.847+0000 7f5d589f2700 1 -- 172.21.15.110:0/1956073574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d541997a0 con 0x7f5d54103dd0 2024-05-09T04:41:07.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.848+0000 7f5d589f2700 1 -- 172.21.15.110:0/1956073574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d54105280 con 0x7f5d54103dd0 2024-05-09T04:41:07.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.848+0000 7f5d437fe700 1 -- 172.21.15.110:0/1956073574 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f5d44029070 con 0x7f5d54103dd0 2024-05-09T04:41:07.849 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.848+0000 7f5d437fe700 1 --2- 172.21.15.110:0/1956073574 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5d3c03e510 0x7f5d3c040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:07.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.848+0000 7f5d437fe700 1 -- 172.21.15.110:0/1956073574 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5d4401e070 con 0x7f5d54103dd0 2024-05-09T04:41:07.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.849+0000 7f5d51d9b700 1 --2- 172.21.15.110:0/1956073574 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5d3c03e510 0x7f5d3c040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:07.850 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.849+0000 7f5d51d9b700 1 --2- 172.21.15.110:0/1956073574 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5d3c03e510 0x7f5d3c040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5d48006fd0 tx=0x7f5d48006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:07.854 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:07.854+0000 7f5d437fe700 1 -- 172.21.15.110:0/1956073574 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5d44013070 con 0x7f5d54103dd0 2024-05-09T04:41:08.068 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:08.068+0000 7f5d589f2700 1 -- 172.21.15.110:0/1956073574 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi072:172.21.15.72=b;smithi110:172.21.15.110=c", "target": ["mon-mgr", ""]}) -- 0x7f5d5419ad70 con 0x7f5d3c03e510 2024-05-09T04:41:08.075 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:08.076+0000 7f5d437fe700 1 -- 172.21.15.110:0/1956073574 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5d5419ad70 con 0x7f5d3c03e510 2024-05-09T04:41:08.076 INFO:teuthology.orchestra.run.smithi110.stdout:Scheduled mon update... 2024-05-09T04:41:08.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:08.078+0000 7f5d589f2700 1 -- 172.21.15.110:0/1956073574 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5d3c03e510 msgr2=0x7f5d3c040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:08.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:08.078+0000 7f5d589f2700 1 --2- 172.21.15.110:0/1956073574 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5d3c03e510 0x7f5d3c040a20 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5d48006fd0 tx=0x7f5d48006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:08.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:08.078+0000 7f5d589f2700 1 -- 172.21.15.110:0/1956073574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d54103dd0 msgr2=0x7f5d5419f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:08.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:08.078+0000 7f5d589f2700 1 --2- 172.21.15.110:0/1956073574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d54103dd0 0x7f5d5419f560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5d440056d0 tx=0x7f5d440057b0 comp rx=0 tx=0).stop 2024-05-09T04:41:08.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:08.078+0000 7f5d589f2700 1 -- 172.21.15.110:0/1956073574 shutdown_connections 2024-05-09T04:41:08.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:08.078+0000 7f5d589f2700 1 --2- 172.21.15.110:0/1956073574 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5d3c03e510 0x7f5d3c040a20 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:08.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:08.078+0000 7f5d589f2700 1 --2- 172.21.15.110:0/1956073574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d54103dd0 0x7f5d5419f560 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:08.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:08.078+0000 7f5d589f2700 1 -- 172.21.15.110:0/1956073574 >> 172.21.15.110:0/1956073574 conn(0x7f5d540ff680 msgr2=0x7f5d54101aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:08.078 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:08.078+0000 7f5d589f2700 1 -- 172.21.15.110:0/1956073574 shutdown_connections 2024-05-09T04:41:08.079 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:08.078+0000 7f5d589f2700 1 -- 172.21.15.110:0/1956073574 wait complete. 2024-05-09T04:41:08.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:07 smithi046 ceph-mon[30136]: Updating smithi110:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:41:08.662 DEBUG:teuthology.orchestra.run.smithi072:mon.b> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mon.b.service 2024-05-09T04:41:08.665 DEBUG:teuthology.orchestra.run.smithi110:mon.c> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mon.c.service 2024-05-09T04:41:08.668 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T04:41:08.668 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T04:41:09.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:08 smithi046 ceph-mon[30136]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:09.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:09.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:09.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:09.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:09.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:09.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:09.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:09.758 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:10.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:09 smithi046 ceph-mon[30136]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi072:172.21.15.72=b;smithi110:172.21.15.110=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:41:10.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:09 smithi046 ceph-mon[30136]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi072:172.21.15.72=b;smithi110:172.21.15.110=c;count:3 2024-05-09T04:41:10.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:09 smithi046 ceph-mon[30136]: Deploying daemon mon.c on smithi110 2024-05-09T04:41:11.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:10 smithi046 ceph-mon[30136]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:11.993 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:11 smithi110 systemd[1]: Starting Ceph mon.c for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:41:12.525 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 podman[34863]: 2024-05-09 04:41:12.2997726 +0000 UTC m=+0.024553719 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:41:12.525 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 podman[34863]: 2024-05-09 04:41:12.497514443 +0000 UTC m=+0.222295525 container create f5239ba2c3bb771e0f04a1e1b500fac11908b55343b85b904b0afbf21250f737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-c, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-09T04:41:12.698 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.524+0000 7f3f94524700 1 -- 172.21.15.110:0/3880634005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f8c103dd0 msgr2=0x7f3f8c1041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:12.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.524+0000 7f3f94524700 1 --2- 172.21.15.110:0/3880634005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f8c103dd0 0x7f3f8c1041e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3f88009910 tx=0x7f3f88009c20 comp rx=0 tx=0).stop 2024-05-09T04:41:12.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.524+0000 7f3f94524700 1 -- 172.21.15.110:0/3880634005 shutdown_connections 2024-05-09T04:41:12.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.524+0000 7f3f94524700 1 --2- 172.21.15.110:0/3880634005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f8c103dd0 0x7f3f8c1041e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:12.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.524+0000 7f3f94524700 1 -- 172.21.15.110:0/3880634005 >> 172.21.15.110:0/3880634005 conn(0x7f3f8c0ff660 msgr2=0x7f3f8c101ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:12.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.524+0000 7f3f94524700 1 -- 172.21.15.110:0/3880634005 shutdown_connections 2024-05-09T04:41:12.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.524+0000 7f3f94524700 1 -- 172.21.15.110:0/3880634005 wait complete. 2024-05-09T04:41:12.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.525+0000 7f3f94524700 1 Processor -- start 2024-05-09T04:41:12.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.525+0000 7f3f94524700 1 -- start start 2024-05-09T04:41:12.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.525+0000 7f3f94524700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f8c103dd0 0x7f3f8c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:12.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.525+0000 7f3f94524700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3f8c10e0c0 con 0x7f3f8c103dd0 2024-05-09T04:41:12.699 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.526+0000 7f3f922c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f8c103dd0 0x7f3f8c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:12.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.526+0000 7f3f922c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f8c103dd0 0x7f3f8c19f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.110:42084/0 (socket says 172.21.15.110:42084) 2024-05-09T04:41:12.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.526+0000 7f3f922c0700 1 -- 172.21.15.110:0/2914926971 learned_addr learned my addr 172.21.15.110:0/2914926971 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-09T04:41:12.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.526+0000 7f3f922c0700 1 -- 172.21.15.110:0/2914926971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f8c19fb40 con 0x7f3f8c103dd0 2024-05-09T04:41:12.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.526+0000 7f3f922c0700 1 --2- 172.21.15.110:0/2914926971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f8c103dd0 0x7f3f8c19f600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3f880056d0 tx=0x7f3f880057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:12.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.527+0000 7f3f837fe700 1 -- 172.21.15.110:0/2914926971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f88026070 con 0x7f3f8c103dd0 2024-05-09T04:41:12.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.527+0000 7f3f837fe700 1 -- 172.21.15.110:0/2914926971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3f8800bba0 con 0x7f3f8c103dd0 2024-05-09T04:41:12.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.527+0000 7f3f837fe700 1 -- 172.21.15.110:0/2914926971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3f88003920 con 0x7f3f8c103dd0 2024-05-09T04:41:12.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.527+0000 7f3f94524700 1 -- 172.21.15.110:0/2914926971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f8c19fe30 con 0x7f3f8c103dd0 2024-05-09T04:41:12.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.527+0000 7f3f94524700 1 -- 172.21.15.110:0/2914926971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f8c199960 con 0x7f3f8c103dd0 2024-05-09T04:41:12.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.527+0000 7f3f94524700 1 -- 172.21.15.110:0/2914926971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f8c105280 con 0x7f3f8c103dd0 2024-05-09T04:41:12.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.528+0000 7f3f837fe700 1 -- 172.21.15.110:0/2914926971 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3f88029070 con 0x7f3f8c103dd0 2024-05-09T04:41:12.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.528+0000 7f3f837fe700 1 --2- 172.21.15.110:0/2914926971 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3f7803e510 0x7f3f78040a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:12.700 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.528+0000 7f3f837fe700 1 -- 172.21.15.110:0/2914926971 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f3f8801e070 con 0x7f3f8c103dd0 2024-05-09T04:41:12.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.528+0000 7f3f91abf700 1 --2- 172.21.15.110:0/2914926971 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3f7803e510 0x7f3f78040a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:12.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.529+0000 7f3f91abf700 1 --2- 172.21.15.110:0/2914926971 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3f7803e510 0x7f3f78040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006fd0 tx=0x7f3f7c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:12.701 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.534+0000 7f3f837fe700 1 -- 172.21.15.110:0/2914926971 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3f88013070 con 0x7f3f8c103dd0 2024-05-09T04:41:12.827 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.827+0000 7f3f94524700 1 -- 172.21.15.110:0/2914926971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f3f8c1a0060 con 0x7f3f8c103dd0 2024-05-09T04:41:12.828 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.828+0000 7f3f837fe700 1 -- 172.21.15.110:0/2914926971 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f3f88030300 con 0x7f3f8c103dd0 2024-05-09T04:41:12.828 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:41:12.828 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":1,"fsid":"ecf9ae2c-0dbd-11ef-bc97-c7b262605968","modified":"2024-05-09T04:39:22.726361Z","created":"2024-05-09T04:39:22.726361Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-09T04:41:12.830 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.830+0000 7f3f94524700 1 -- 172.21.15.110:0/2914926971 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3f7803e510 msgr2=0x7f3f78040a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:12.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.830+0000 7f3f94524700 1 --2- 172.21.15.110:0/2914926971 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3f7803e510 0x7f3f78040a20 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3f7c006fd0 tx=0x7f3f7c006e40 comp rx=0 tx=0).stop 2024-05-09T04:41:12.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.831+0000 7f3f94524700 1 -- 172.21.15.110:0/2914926971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f8c103dd0 msgr2=0x7f3f8c19f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:12.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.831+0000 7f3f94524700 1 --2- 172.21.15.110:0/2914926971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f8c103dd0 0x7f3f8c19f600 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3f880056d0 tx=0x7f3f880057b0 comp rx=0 tx=0).stop 2024-05-09T04:41:12.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.831+0000 7f3f94524700 1 -- 172.21.15.110:0/2914926971 shutdown_connections 2024-05-09T04:41:12.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.831+0000 7f3f94524700 1 --2- 172.21.15.110:0/2914926971 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3f7803e510 0x7f3f78040a20 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:12.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.831+0000 7f3f94524700 1 --2- 172.21.15.110:0/2914926971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3f8c103dd0 0x7f3f8c19f600 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:12.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.831+0000 7f3f94524700 1 -- 172.21.15.110:0/2914926971 >> 172.21.15.110:0/2914926971 conn(0x7f3f8c0ff660 msgr2=0x7f3f8c109900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:12.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.831+0000 7f3f94524700 1 -- 172.21.15.110:0/2914926971 shutdown_connections 2024-05-09T04:41:12.831 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:12.831+0000 7f3f94524700 1 -- 172.21.15.110:0/2914926971 wait complete. 2024-05-09T04:41:12.832 INFO:teuthology.orchestra.run.smithi110.stderr:dumped monmap epoch 1 2024-05-09T04:41:13.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 podman[34863]: 2024-05-09 04:41:12.800522474 +0000 UTC m=+0.525303569 container init f5239ba2c3bb771e0f04a1e1b500fac11908b55343b85b904b0afbf21250f737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-c, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T04:41:13.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 podman[34863]: 2024-05-09 04:41:12.803930248 +0000 UTC m=+0.528711333 container start f5239ba2c3bb771e0f04a1e1b500fac11908b55343b85b904b0afbf21250f737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-c, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T04:41:13.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: set uid:gid to 167:167 (ceph:ceph) 2024-05-09T04:41:13.060 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: ceph version 19.0.0-3594-gc9da4049 (c9da4049f638c0f84bdd9b4e33c42e86046ac554) squid (dev), process ceph-mon, pid 2 2024-05-09T04:41:13.060 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: pidfile_write: ignore empty --pid-file 2024-05-09T04:41:13.060 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: load: jerasure load: lrc 2024-05-09T04:41:13.060 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: RocksDB version: 7.9.2 2024-05-09T04:41:13.060 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Git sha 0 2024-05-09T04:41:13.060 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Compile date 2024-05-08 19:33:14 2024-05-09T04:41:13.060 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: DB SUMMARY 2024-05-09T04:41:13.060 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: DB Session ID: ENVYM0L1S9VBUVSWTIGY 2024-05-09T04:41:13.060 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: CURRENT file: CURRENT 2024-05-09T04:41:13.060 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: IDENTITY file: IDENTITY 2024-05-09T04:41:13.060 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-05-09T04:41:13.060 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-05-09T04:41:13.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-05-09T04:41:13.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.error_if_exists: 0 2024-05-09T04:41:13.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.create_if_missing: 0 2024-05-09T04:41:13.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.paranoid_checks: 1 2024-05-09T04:41:13.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-09T04:41:13.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-09T04:41:13.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-09T04:41:13.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.env: 0x5613f54d0500 2024-05-09T04:41:13.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.fs: PosixFileSystem 2024-05-09T04:41:13.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.info_log: 0x5613f7476020 2024-05-09T04:41:13.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_file_opening_threads: 16 2024-05-09T04:41:13.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.statistics: (nil) 2024-05-09T04:41:13.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.use_fsync: 0 2024-05-09T04:41:13.061 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_log_file_size: 0 2024-05-09T04:41:13.062 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-09T04:41:13.062 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-09T04:41:13.062 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.keep_log_file_num: 1000 2024-05-09T04:41:13.062 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.recycle_log_file_num: 0 2024-05-09T04:41:13.062 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.allow_fallocate: 1 2024-05-09T04:41:13.062 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.allow_mmap_reads: 0 2024-05-09T04:41:13.062 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.allow_mmap_writes: 0 2024-05-09T04:41:13.062 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.use_direct_reads: 0 2024-05-09T04:41:13.062 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-09T04:41:13.062 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.create_missing_column_families: 0 2024-05-09T04:41:13.062 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.db_log_dir: 2024-05-09T04:41:13.062 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.wal_dir: 2024-05-09T04:41:13.063 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-09T04:41:13.063 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-09T04:41:13.063 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-09T04:41:13.063 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-09T04:41:13.063 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-09T04:41:13.063 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-09T04:41:13.063 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.advise_random_on_open: 1 2024-05-09T04:41:13.063 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.db_write_buffer_size: 0 2024-05-09T04:41:13.063 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.write_buffer_manager: 0x5613f73408c0 2024-05-09T04:41:13.063 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-09T04:41:13.063 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-09T04:41:13.063 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-09T04:41:13.063 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.rate_limiter: (nil) 2024-05-09T04:41:13.063 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-09T04:41:13.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.wal_recovery_mode: 2 2024-05-09T04:41:13.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.enable_thread_tracking: 0 2024-05-09T04:41:13.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.enable_pipelined_write: 0 2024-05-09T04:41:13.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.unordered_write: 0 2024-05-09T04:41:13.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-09T04:41:13.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-09T04:41:13.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-09T04:41:13.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-09T04:41:13.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.row_cache: None 2024-05-09T04:41:13.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.wal_filter: None 2024-05-09T04:41:13.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-09T04:41:13.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.allow_ingest_behind: 0 2024-05-09T04:41:13.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.two_write_queues: 0 2024-05-09T04:41:13.064 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.manual_wal_flush: 0 2024-05-09T04:41:13.065 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.wal_compression: 0 2024-05-09T04:41:13.065 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.atomic_flush: 0 2024-05-09T04:41:13.065 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-09T04:41:13.065 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-09T04:41:13.065 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-09T04:41:13.065 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.log_readahead_size: 0 2024-05-09T04:41:13.065 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-09T04:41:13.065 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.best_efforts_recovery: 0 2024-05-09T04:41:13.065 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-09T04:41:13.065 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-09T04:41:13.065 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.allow_data_in_errors: 0 2024-05-09T04:41:13.065 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.db_host_id: __hostname__ 2024-05-09T04:41:13.065 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-09T04:41:13.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_background_jobs: 2 2024-05-09T04:41:13.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_background_compactions: -1 2024-05-09T04:41:13.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_subcompactions: 1 2024-05-09T04:41:13.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-09T04:41:13.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-09T04:41:13.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-09T04:41:13.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_total_wal_size: 0 2024-05-09T04:41:13.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-09T04:41:13.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-09T04:41:13.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-09T04:41:13.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-09T04:41:13.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_open_files: -1 2024-05-09T04:41:13.066 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.bytes_per_sync: 0 2024-05-09T04:41:13.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-09T04:41:13.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-09T04:41:13.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compaction_readahead_size: 0 2024-05-09T04:41:13.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_background_flushes: -1 2024-05-09T04:41:13.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Compression algorithms supported: 2024-05-09T04:41:13.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-09T04:41:13.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: kZSTD supported: 0 2024-05-09T04:41:13.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: kXpressCompression supported: 0 2024-05-09T04:41:13.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: kLZ4HCCompression supported: 1 2024-05-09T04:41:13.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: kZlibCompression supported: 1 2024-05-09T04:41:13.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: kSnappyCompression supported: 1 2024-05-09T04:41:13.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: kLZ4Compression supported: 1 2024-05-09T04:41:13.067 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: kBZip2Compression supported: 0 2024-05-09T04:41:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-09T04:41:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-09T04:41:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-05-09T04:41:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-09T04:41:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-09T04:41:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.merge_operator: 2024-05-09T04:41:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compaction_filter: None 2024-05-09T04:41:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compaction_filter_factory: None 2024-05-09T04:41:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.sst_partitioner_factory: None 2024-05-09T04:41:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-09T04:41:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-09T04:41:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5613f74761e0) 2024-05-09T04:41:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout: cache_index_and_filter_blocks: 1 2024-05-09T04:41:13.068 INFO:journalctl@ceph.mon.c.smithi110.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-09T04:41:13.069 INFO:journalctl@ceph.mon.c.smithi110.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-09T04:41:13.069 INFO:journalctl@ceph.mon.c.smithi110.stdout: pin_top_level_index_and_filter: 1 2024-05-09T04:41:13.069 INFO:journalctl@ceph.mon.c.smithi110.stdout: index_type: 0 2024-05-09T04:41:13.069 INFO:journalctl@ceph.mon.c.smithi110.stdout: data_block_index_type: 0 2024-05-09T04:41:13.069 INFO:journalctl@ceph.mon.c.smithi110.stdout: index_shortening: 1 2024-05-09T04:41:13.069 INFO:journalctl@ceph.mon.c.smithi110.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-09T04:41:13.069 INFO:journalctl@ceph.mon.c.smithi110.stdout: checksum: 4 2024-05-09T04:41:13.069 INFO:journalctl@ceph.mon.c.smithi110.stdout: no_block_cache: 0 2024-05-09T04:41:13.069 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache: 0x5613f73731f0 2024-05-09T04:41:13.069 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache_name: BinnedLRUCache 2024-05-09T04:41:13.069 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache_options: 2024-05-09T04:41:13.069 INFO:journalctl@ceph.mon.c.smithi110.stdout: capacity : 536870912 2024-05-09T04:41:13.069 INFO:journalctl@ceph.mon.c.smithi110.stdout: num_shard_bits : 4 2024-05-09T04:41:13.069 INFO:journalctl@ceph.mon.c.smithi110.stdout: strict_capacity_limit : 0 2024-05-09T04:41:13.070 INFO:journalctl@ceph.mon.c.smithi110.stdout: high_pri_pool_ratio: 0.000 2024-05-09T04:41:13.070 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_cache_compressed: (nil) 2024-05-09T04:41:13.070 INFO:journalctl@ceph.mon.c.smithi110.stdout: persistent_cache: (nil) 2024-05-09T04:41:13.070 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_size: 4096 2024-05-09T04:41:13.070 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_size_deviation: 10 2024-05-09T04:41:13.070 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_restart_interval: 16 2024-05-09T04:41:13.070 INFO:journalctl@ceph.mon.c.smithi110.stdout: index_block_restart_interval: 1 2024-05-09T04:41:13.070 INFO:journalctl@ceph.mon.c.smithi110.stdout: metadata_block_size: 4096 2024-05-09T04:41:13.070 INFO:journalctl@ceph.mon.c.smithi110.stdout: partition_filters: 0 2024-05-09T04:41:13.070 INFO:journalctl@ceph.mon.c.smithi110.stdout: use_delta_encoding: 1 2024-05-09T04:41:13.070 INFO:journalctl@ceph.mon.c.smithi110.stdout: filter_policy: bloomfilter 2024-05-09T04:41:13.070 INFO:journalctl@ceph.mon.c.smithi110.stdout: whole_key_filtering: 1 2024-05-09T04:41:13.071 INFO:journalctl@ceph.mon.c.smithi110.stdout: verify_compression: 0 2024-05-09T04:41:13.071 INFO:journalctl@ceph.mon.c.smithi110.stdout: read_amp_bytes_per_bit: 0 2024-05-09T04:41:13.071 INFO:journalctl@ceph.mon.c.smithi110.stdout: format_version: 5 2024-05-09T04:41:13.071 INFO:journalctl@ceph.mon.c.smithi110.stdout: enable_index_compression: 1 2024-05-09T04:41:13.071 INFO:journalctl@ceph.mon.c.smithi110.stdout: block_align: 0 2024-05-09T04:41:13.071 INFO:journalctl@ceph.mon.c.smithi110.stdout: max_auto_readahead_size: 262144 2024-05-09T04:41:13.071 INFO:journalctl@ceph.mon.c.smithi110.stdout: prepopulate_block_cache: 0 2024-05-09T04:41:13.071 INFO:journalctl@ceph.mon.c.smithi110.stdout: initial_auto_readahead_size: 8192 2024-05-09T04:41:13.071 INFO:journalctl@ceph.mon.c.smithi110.stdout: num_file_reads_for_auto_readahead: 2 2024-05-09T04:41:13.071 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.write_buffer_size: 33554432 2024-05-09T04:41:13.071 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_write_buffer_number: 2 2024-05-09T04:41:13.071 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compression: NoCompression 2024-05-09T04:41:13.071 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.bottommost_compression: Disabled 2024-05-09T04:41:13.071 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.prefix_extractor: nullptr 2024-05-09T04:41:13.072 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-09T04:41:13.072 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.num_levels: 7 2024-05-09T04:41:13.072 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-09T04:41:13.072 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-09T04:41:13.072 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-09T04:41:13.072 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-09T04:41:13.072 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-09T04:41:13.072 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-09T04:41:13.072 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-09T04:41:13.072 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-09T04:41:13.072 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-09T04:41:13.072 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-09T04:41:13.072 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-09T04:41:13.072 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-09T04:41:13.072 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-09T04:41:13.073 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compression_opts.level: 32767 2024-05-09T04:41:13.073 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compression_opts.strategy: 0 2024-05-09T04:41:13.073 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-09T04:41:13.073 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-09T04:41:13.073 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-09T04:41:13.073 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-09T04:41:13.073 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compression_opts.enabled: false 2024-05-09T04:41:13.073 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-09T04:41:13.073 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-09T04:41:13.073 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-09T04:41:13.073 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-09T04:41:13.073 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.target_file_size_base: 67108864 2024-05-09T04:41:13.073 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-09T04:41:13.073 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-09T04:41:13.074 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-09T04:41:13.074 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-09T04:41:13.074 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-09T04:41:13.074 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-09T04:41:13.074 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-09T04:41:13.074 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-09T04:41:13.074 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-09T04:41:13.074 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-09T04:41:13.074 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-09T04:41:13.074 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-09T04:41:13.074 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-09T04:41:13.074 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-09T04:41:13.074 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.arena_block_size: 1048576 2024-05-09T04:41:13.075 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-09T04:41:13.075 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-09T04:41:13.075 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.disable_auto_compactions: 0 2024-05-09T04:41:13.075 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-09T04:41:13.075 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-09T04:41:13.075 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-09T04:41:13.075 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-09T04:41:13.075 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-09T04:41:13.075 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-09T04:41:13.075 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-09T04:41:13.075 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-09T04:41:13.075 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-09T04:41:13.075 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-09T04:41:13.075 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-09T04:41:13.076 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.inplace_update_support: 0 2024-05-09T04:41:13.076 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-09T04:41:13.076 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-09T04:41:13.076 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-09T04:41:13.076 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-09T04:41:13.076 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.bloom_locality: 0 2024-05-09T04:41:13.076 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.max_successive_merges: 0 2024-05-09T04:41:13.076 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-09T04:41:13.076 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.paranoid_file_checks: 0 2024-05-09T04:41:13.076 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.force_consistency_checks: 1 2024-05-09T04:41:13.076 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.report_bg_io_stats: 0 2024-05-09T04:41:13.076 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.ttl: 2592000 2024-05-09T04:41:13.076 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-09T04:41:13.076 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-09T04:41:13.076 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-09T04:41:13.077 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.enable_blob_files: false 2024-05-09T04:41:13.077 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.min_blob_size: 0 2024-05-09T04:41:13.077 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.blob_file_size: 268435456 2024-05-09T04:41:13.077 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-09T04:41:13.077 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-09T04:41:13.077 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-09T04:41:13.077 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-09T04:41:13.077 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-09T04:41:13.077 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.blob_file_starting_level: 0 2024-05-09T04:41:13.077 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-09T04:41:13.077 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-05-09T04:41:13.077 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-05-09T04:41:13.077 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 2eaa38ba-f793-4c5c-9fc9-26fe99240529 2024-05-09T04:41:13.077 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715229672845332, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-05-09T04:41:13.078 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-05-09T04:41:13.078 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715229672845755, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715229672, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "2eaa38ba-f793-4c5c-9fc9-26fe99240529", "db_session_id": "ENVYM0L1S9VBUVSWTIGY", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-05-09T04:41:13.078 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715229672845835, "job": 1, "event": "recovery_finished"} 2024-05-09T04:41:13.078 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-05-09T04:41:13.078 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-09T04:41:13.078 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5613f7357c00 2024-05-09T04:41:13.078 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: DB pointer 0x5613fc14a000 2024-05-09T04:41:13.078 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-09T04:41:13.078 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-09T04:41:13.078 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** DB Stats ** 2024-05-09T04:41:13.078 INFO:journalctl@ceph.mon.c.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T04:41:13.078 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-09T04:41:13.078 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T04:41:13.079 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T04:41:13.079 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-09T04:41:13.079 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-09T04:41:13.079 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-09T04:41:13.079 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-05-09T04:41:13.079 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** Compaction Stats [default] ** 2024-05-09T04:41:13.079 INFO:journalctl@ceph.mon.c.smithi110.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T04:41:13.079 INFO:journalctl@ceph.mon.c.smithi110.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-09T04:41:13.079 INFO:journalctl@ceph.mon.c.smithi110.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T04:41:13.079 INFO:journalctl@ceph.mon.c.smithi110.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T04:41:13.079 INFO:journalctl@ceph.mon.c.smithi110.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T04:41:13.079 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-05-09T04:41:13.079 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** Compaction Stats [default] ** 2024-05-09T04:41:13.079 INFO:journalctl@ceph.mon.c.smithi110.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-09T04:41:13.080 INFO:journalctl@ceph.mon.c.smithi110.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-09T04:41:13.080 INFO:journalctl@ceph.mon.c.smithi110.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.9 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-05-09T04:41:13.080 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-05-09T04:41:13.080 INFO:journalctl@ceph.mon.c.smithi110.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-09T04:41:13.080 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-05-09T04:41:13.080 INFO:journalctl@ceph.mon.c.smithi110.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-09T04:41:13.080 INFO:journalctl@ceph.mon.c.smithi110.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-09T04:41:13.080 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-09T04:41:13.080 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-09T04:41:13.080 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-09T04:41:13.080 INFO:journalctl@ceph.mon.c.smithi110.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-09T04:41:13.080 INFO:journalctl@ceph.mon.c.smithi110.stdout: Cumulative compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T04:41:13.080 INFO:journalctl@ceph.mon.c.smithi110.stdout: Interval compaction: 0.00 GB write, 0.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-09T04:41:13.081 INFO:journalctl@ceph.mon.c.smithi110.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-09T04:41:13.081 INFO:journalctl@ceph.mon.c.smithi110.stdout: Block cache BinnedLRUCache@0x5613f73731f0#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2024-05-09T04:41:13.081 INFO:journalctl@ceph.mon.c.smithi110.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-05-09T04:41:13.081 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-05-09T04:41:13.081 INFO:journalctl@ceph.mon.c.smithi110.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-09T04:41:13.081 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-05-09T04:41:13.081 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: using public_addr v2:172.21.15.110:0/0 -> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] 2024-05-09T04:41:13.081 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: starting mon.c rank -1 at public addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] at bind addrs [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:13.081 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: mon.c@-1(???) e0 preinit fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:13.081 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: mon.c@-1(synchronizing).mds e1 new map 2024-05-09T04:41:13.081 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: mon.c@-1(synchronizing).mds e1 print_map 2024-05-09T04:41:13.081 INFO:journalctl@ceph.mon.c.smithi110.stdout: e1 2024-05-09T04:41:13.081 INFO:journalctl@ceph.mon.c.smithi110.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T04:41:13.081 INFO:journalctl@ceph.mon.c.smithi110.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T04:41:13.082 INFO:journalctl@ceph.mon.c.smithi110.stdout: legacy client fscid: -1 2024-05-09T04:41:13.082 INFO:journalctl@ceph.mon.c.smithi110.stdout: 2024-05-09T04:41:13.082 INFO:journalctl@ceph.mon.c.smithi110.stdout: No filesystems configured 2024-05-09T04:41:13.082 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-05-09T04:41:13.082 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-05-09T04:41:13.082 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-05-09T04:41:13.082 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-05-09T04:41:13.082 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-05-09T04:41:13.082 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-05-09T04:41:13.082 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-09T04:41:13.082 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T04:41:13.082 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T04:41:13.082 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-05-09T04:41:13.083 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:13.083 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi110", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:41:13.083 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:13.083 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.083 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.083 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.083 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.083 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi072", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:41:13.083 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:13.083 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:13.083 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-09T04:41:13.083 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: Deploying cephadm binary to smithi110 2024-05-09T04:41:13.083 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: Updating smithi072:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:13.083 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:41:13.084 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:13.084 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: Updating smithi072:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:41:13.084 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.084 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.084 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.084 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:13.084 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.084 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: Added host smithi110 2024-05-09T04:41:13.084 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:13.084 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.084 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.084 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:13.084 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.084 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:41:13.085 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:13.085 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1964691053' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-09T04:41:13.085 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.085 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.085 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.085 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.085 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi110", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:41:13.085 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:13.086 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:12 smithi046 ceph-mon[30136]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:13.086 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:13.086 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:13.086 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: Updating smithi110:/etc/ceph/ceph.conf 2024-05-09T04:41:13.086 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: Updating smithi110:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:13.086 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1964691053' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-09T04:41:13.086 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:41:13.086 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:41:13.086 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.086 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.087 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.087 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: Updating smithi110:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:41:13.087 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:13.087 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.087 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:13.087 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:13.087 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:13.087 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:13.087 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:13.087 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:13.087 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi046:172.21.15.46=a;smithi072:172.21.15.72=b;smithi110:172.21.15.110=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:41:13.087 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: Saving service mon spec with placement smithi046:172.21.15.46=a;smithi072:172.21.15.72=b;smithi110:172.21.15.110=c;count:3 2024-05-09T04:41:13.087 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: Deploying daemon mon.c on smithi110 2024-05-09T04:41:13.087 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:13.087 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:13.088 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 ceph-mon[34904]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-05-09T04:41:13.088 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 bash[34863]: f5239ba2c3bb771e0f04a1e1b500fac11908b55343b85b904b0afbf21250f737 2024-05-09T04:41:13.088 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:12 smithi110 systemd[1]: Started Ceph mon.c for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:41:14.447 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T04:41:14.447 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T04:41:14.699 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:41:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:41:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: mon.a calling monitor election 2024-05-09T04:41:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:18.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:18.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: mon.c calling monitor election 2024-05-09T04:41:18.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:18.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:18.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:18.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:18.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:18.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T04:41:18.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: monmap epoch 2 2024-05-09T04:41:18.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:18.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: last_changed 2024-05-09T04:41:12.871696+0000 2024-05-09T04:41:18.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: created 2024-05-09T04:39:22.726361+0000 2024-05-09T04:41:18.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: min_mon_release 19 (squid) 2024-05-09T04:41:18.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: election_strategy: 1 2024-05-09T04:41:18.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T04:41:18.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-09T04:41:18.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: fsmap 2024-05-09T04:41:18.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:41:18.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: mgrmap e13: x(active, since 50s) 2024-05-09T04:41:18.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: overall HEALTH_OK 2024-05-09T04:41:18.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:18.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:18.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:18.310 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:18.310 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:18.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:41:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: mon.a calling monitor election 2024-05-09T04:41:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: mon.c calling monitor election 2024-05-09T04:41:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T04:41:18.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: monmap epoch 2 2024-05-09T04:41:18.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:18.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: last_changed 2024-05-09T04:41:12.871696+0000 2024-05-09T04:41:18.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: created 2024-05-09T04:39:22.726361+0000 2024-05-09T04:41:18.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: min_mon_release 19 (squid) 2024-05-09T04:41:18.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: election_strategy: 1 2024-05-09T04:41:18.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T04:41:18.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-09T04:41:18.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: fsmap 2024-05-09T04:41:18.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:41:18.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: mgrmap e13: x(active, since 50s) 2024-05-09T04:41:18.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: overall HEALTH_OK 2024-05-09T04:41:18.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:18.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:18.332 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:18.332 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:18.332 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:19.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.016+0000 7f4d328b1700 1 -- 172.21.15.110:0/3718811235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d140035c0 msgr2=0x7f4d14005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:19.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.016+0000 7f4d328b1700 1 --2- 172.21.15.110:0/3718811235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d140035c0 0x7f4d14005aa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c103b70 tx=0x7f4d2000feb0 comp rx=0 tx=0).stop 2024-05-09T04:41:19.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.016+0000 7f4d328b1700 1 -- 172.21.15.110:0/3718811235 shutdown_connections 2024-05-09T04:41:19.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.016+0000 7f4d328b1700 1 --2- 172.21.15.110:0/3718811235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d140035c0 0x7f4d14005aa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:19.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.016+0000 7f4d328b1700 1 -- 172.21.15.110:0/3718811235 >> 172.21.15.110:0/3718811235 conn(0x7f4d2c0ff7a0 msgr2=0x7f4d2c101bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:19.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.016+0000 7f4d328b1700 1 -- 172.21.15.110:0/3718811235 shutdown_connections 2024-05-09T04:41:19.016 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.016+0000 7f4d328b1700 1 -- 172.21.15.110:0/3718811235 wait complete. 2024-05-09T04:41:19.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.017+0000 7f4d328b1700 1 Processor -- start 2024-05-09T04:41:19.017 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.017+0000 7f4d328b1700 1 -- start start 2024-05-09T04:41:19.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.018+0000 7f4d328b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d140035c0 0x7f4d2c077660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:19.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.018+0000 7f4d328b1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d2c075ac0 0x7f4d2c075ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:19.019 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.018+0000 7f4d328b1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c10bbf0 con 0x7f4d140035c0 2024-05-09T04:41:19.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.018+0000 7f4d328b1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4d2c10ba70 con 0x7f4d2c075ac0 2024-05-09T04:41:19.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.018+0000 7f4d2bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d140035c0 0x7f4d2c077660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:19.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.018+0000 7f4d2bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d140035c0 0x7f4d2c077660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.110:40780/0 (socket says 172.21.15.110:40780) 2024-05-09T04:41:19.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.018+0000 7f4d2bfff700 1 -- 172.21.15.110:0/2395404565 learned_addr learned my addr 172.21.15.110:0/2395404565 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-09T04:41:19.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.018+0000 7f4d2b7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d2c075ac0 0x7f4d2c075ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:19.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.018+0000 7f4d2b7fe700 1 -- 172.21.15.110:0/2395404565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d2c075ac0 msgr2=0x7f4d2c075ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-09T04:41:19.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.018+0000 7f4d2b7fe700 1 -- 172.21.15.110:0/2395404565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d2c075ac0 msgr2=0x7f4d2c075ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T04:41:19.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.018+0000 7f4d2b7fe700 1 --2- 172.21.15.110:0/2395404565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d2c075ac0 0x7f4d2c075ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:41:19.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.019+0000 7f4d2b7fe700 1 --2- 172.21.15.110:0/2395404565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d2c075ac0 0x7f4d2c075ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:41:19.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.019+0000 7f4d2bfff700 1 -- 172.21.15.110:0/2395404565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d2c075ac0 msgr2=0x7f4d2c075ed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:41:19.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.019+0000 7f4d2bfff700 1 --2- 172.21.15.110:0/2395404565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d2c075ac0 0x7f4d2c075ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:19.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.019+0000 7f4d2bfff700 1 -- 172.21.15.110:0/2395404565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d2c0764c0 con 0x7f4d140035c0 2024-05-09T04:41:19.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.019+0000 7f4d2bfff700 1 --2- 172.21.15.110:0/2395404565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d140035c0 0x7f4d2c077660 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000f8d0 tx=0x7f4d200050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:19.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.019+0000 7f4d297fa700 1 -- 172.21.15.110:0/2395404565 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4d20004aa0 con 0x7f4d140035c0 2024-05-09T04:41:19.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.019+0000 7f4d297fa700 1 -- 172.21.15.110:0/2395404565 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4d20004030 con 0x7f4d140035c0 2024-05-09T04:41:19.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.019+0000 7f4d328b1700 1 -- 172.21.15.110:0/2395404565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d2c076750 con 0x7f4d140035c0 2024-05-09T04:41:19.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.019+0000 7f4d297fa700 1 -- 172.21.15.110:0/2395404565 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f4d20011c00 con 0x7f4d140035c0 2024-05-09T04:41:19.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.019+0000 7f4d328b1700 1 -- 172.21.15.110:0/2395404565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d2c1aa610 con 0x7f4d140035c0 2024-05-09T04:41:19.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.020+0000 7f4d297fa700 1 -- 172.21.15.110:0/2395404565 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f4d20023070 con 0x7f4d140035c0 2024-05-09T04:41:19.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.021+0000 7f4d328b1700 1 -- 172.21.15.110:0/2395404565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d2c105380 con 0x7f4d140035c0 2024-05-09T04:41:19.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.021+0000 7f4d297fa700 1 --2- 172.21.15.110:0/2395404565 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4d1803e300 0x7f4d18040810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:19.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.021+0000 7f4d297fa700 1 -- 172.21.15.110:0/2395404565 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f4d20020050 con 0x7f4d140035c0 2024-05-09T04:41:19.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.021+0000 7f4d2b7fe700 1 --2- 172.21.15.110:0/2395404565 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4d1803e300 0x7f4d18040810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:19.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.022+0000 7f4d2b7fe700 1 --2- 172.21.15.110:0/2395404565 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4d1803e300 0x7f4d18040810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c19db90 tx=0x7f4d1c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:19.026 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.027+0000 7f4d297fa700 1 -- 172.21.15.110:0/2395404565 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4d20028020 con 0x7f4d140035c0 2024-05-09T04:41:19.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:18 smithi110 ceph-mon[34904]: Deploying daemon mon.b on smithi072 2024-05-09T04:41:19.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:19.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.323+0000 7f4d328b1700 1 -- 172.21.15.110:0/2395404565 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f4d2c077090 con 0x7f4d140035c0 2024-05-09T04:41:19.323 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.324+0000 7f4d297fa700 1 -- 172.21.15.110:0/2395404565 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f4d2002a300 con 0x7f4d140035c0 2024-05-09T04:41:19.324 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:41:19.324 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":2,"fsid":"ecf9ae2c-0dbd-11ef-bc97-c7b262605968","modified":"2024-05-09T04:41:12.871696Z","created":"2024-05-09T04:39:22.726361Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T04:41:19.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.326+0000 7f4d328b1700 1 -- 172.21.15.110:0/2395404565 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4d1803e300 msgr2=0x7f4d18040810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:19.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.326+0000 7f4d328b1700 1 --2- 172.21.15.110:0/2395404565 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4d1803e300 0x7f4d18040810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4d2c19db90 tx=0x7f4d1c008040 comp rx=0 tx=0).stop 2024-05-09T04:41:19.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.326+0000 7f4d328b1700 1 -- 172.21.15.110:0/2395404565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d140035c0 msgr2=0x7f4d2c077660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:19.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.326+0000 7f4d328b1700 1 --2- 172.21.15.110:0/2395404565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d140035c0 0x7f4d2c077660 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4d2000f8d0 tx=0x7f4d200050d0 comp rx=0 tx=0).stop 2024-05-09T04:41:19.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.326+0000 7f4d328b1700 1 -- 172.21.15.110:0/2395404565 shutdown_connections 2024-05-09T04:41:19.326 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.326+0000 7f4d328b1700 1 --2- 172.21.15.110:0/2395404565 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4d1803e300 0x7f4d18040810 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.326+0000 7f4d328b1700 1 --2- 172.21.15.110:0/2395404565 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4d140035c0 0x7f4d2c077660 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.326+0000 7f4d328b1700 1 --2- 172.21.15.110:0/2395404565 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4d2c075ac0 0x7f4d2c075ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.326+0000 7f4d328b1700 1 -- 172.21.15.110:0/2395404565 >> 172.21.15.110:0/2395404565 conn(0x7f4d2c0ff7a0 msgr2=0x7f4d2c1074c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.326+0000 7f4d328b1700 1 -- 172.21.15.110:0/2395404565 shutdown_connections 2024-05-09T04:41:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:19.326+0000 7f4d328b1700 1 -- 172.21.15.110:0/2395404565 wait complete. 2024-05-09T04:41:19.327 INFO:teuthology.orchestra.run.smithi110.stderr:dumped monmap epoch 2 2024-05-09T04:41:19.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:18 smithi046 ceph-mon[30136]: Deploying daemon mon.b on smithi072 2024-05-09T04:41:19.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:20.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:19 smithi110 ceph-mon[34904]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:20.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:19 smithi110 ceph-mon[34904]: from='client.? 172.21.15.110:0/2395404565' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T04:41:20.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:19 smithi046 ceph-mon[30136]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:20.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:19 smithi046 ceph-mon[30136]: from='client.? 172.21.15.110:0/2395404565' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T04:41:20.330 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:41:19 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:41:19.871+0000 7efedbd68700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-09T04:41:20.902 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-09T04:41:20.905 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-09T04:41:21.149 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:41:22.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:21 smithi072 systemd[1]: Started Ceph mon.b for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:41:27.046 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.046+0000 7f0e3e526700 1 -- 172.21.15.110:0/929132763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200035c0 msgr2=0x7f0e20005aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:27.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.046+0000 7f0e3e526700 1 --2- 172.21.15.110:0/929132763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200035c0 0x7f0e20005aa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0e381021f0 tx=0x7f0e28010e70 comp rx=0 tx=0).stop 2024-05-09T04:41:27.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.046+0000 7f0e3e526700 1 -- 172.21.15.110:0/929132763 shutdown_connections 2024-05-09T04:41:27.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.046+0000 7f0e3e526700 1 --2- 172.21.15.110:0/929132763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200035c0 0x7f0e20005aa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:27.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.046+0000 7f0e3e526700 1 -- 172.21.15.110:0/929132763 >> 172.21.15.110:0/929132763 conn(0x7f0e38077e10 msgr2=0x7f0e38078270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:27.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.047+0000 7f0e3e526700 1 -- 172.21.15.110:0/929132763 shutdown_connections 2024-05-09T04:41:27.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.047+0000 7f0e3e526700 1 -- 172.21.15.110:0/929132763 wait complete. 2024-05-09T04:41:27.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.047+0000 7f0e3e526700 1 Processor -- start 2024-05-09T04:41:27.047 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.048+0000 7f0e3e526700 1 -- start start 2024-05-09T04:41:27.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.048+0000 7f0e3e526700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200035c0 0x7f0e380ffc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:27.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.048+0000 7f0e3e526700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e38100160 0x7f0e38100570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:27.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.048+0000 7f0e3e526700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381a5e80 0x7f0e381a82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:27.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.048+0000 7f0e3e526700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0e381099b0 con 0x7f0e200035c0 2024-05-09T04:41:27.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.048+0000 7f0e3e526700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0e38109830 con 0x7f0e38100160 2024-05-09T04:41:27.050 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.048+0000 7f0e3e526700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0e38109b30 con 0x7f0e381a5e80 2024-05-09T04:41:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.048+0000 7f0e3cac3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381a5e80 0x7f0e381a82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.048+0000 7f0e37fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200035c0 0x7f0e380ffc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.048+0000 7f0e3cac3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381a5e80 0x7f0e381a82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:52860/0 (socket says 172.21.15.110:52860) 2024-05-09T04:41:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.048+0000 7f0e3cac3700 1 -- 172.21.15.110:0/4183574201 learned_addr learned my addr 172.21.15.110:0/4183574201 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-09T04:41:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e377fe700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e38100160 0x7f0e38100570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e3cac3700 1 -- 172.21.15.110:0/4183574201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381a5e80 msgr2=0x7f0e381a82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-05-09T04:41:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e3cac3700 1 -- 172.21.15.110:0/4183574201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381a5e80 msgr2=0x7f0e381a82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T04:41:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e3cac3700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381a5e80 0x7f0e381a82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:41:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e3cac3700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381a5e80 0x7f0e381a82a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:41:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e377fe700 1 -- 172.21.15.110:0/4183574201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e38100160 msgr2=0x7f0e38100570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-09T04:41:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e377fe700 1 -- 172.21.15.110:0/4183574201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e38100160 msgr2=0x7f0e38100570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-09T04:41:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e377fe700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e38100160 0x7f0e38100570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-09T04:41:27.051 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e37fff700 1 -- 172.21.15.110:0/4183574201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381a5e80 msgr2=0x7f0e381a82a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:41:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e37fff700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381a5e80 0x7f0e381a82a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e377fe700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e38100160 0x7f0e38100570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:41:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e37fff700 1 -- 172.21.15.110:0/4183574201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e38100160 msgr2=0x7f0e38100570 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:41:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e37fff700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e38100160 0x7f0e38100570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e37fff700 1 -- 172.21.15.110:0/4183574201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e381a8920 con 0x7f0e200035c0 2024-05-09T04:41:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e37fff700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200035c0 0x7f0e380ffc20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0e28011010 tx=0x7f0e28003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e357fa700 1 -- 172.21.15.110:0/4183574201 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2802a020 con 0x7f0e200035c0 2024-05-09T04:41:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e357fa700 1 -- 172.21.15.110:0/4183574201 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e280056a0 con 0x7f0e200035c0 2024-05-09T04:41:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e3e526700 1 -- 172.21.15.110:0/4183574201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e381a8bb0 con 0x7f0e200035c0 2024-05-09T04:41:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.049+0000 7f0e357fa700 1 -- 172.21.15.110:0/4183574201 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e28022040 con 0x7f0e200035c0 2024-05-09T04:41:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.050+0000 7f0e3e526700 1 -- 172.21.15.110:0/4183574201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e381a9140 con 0x7f0e200035c0 2024-05-09T04:41:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.051+0000 7f0e357fa700 1 -- 172.21.15.110:0/4183574201 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0e2802d020 con 0x7f0e200035c0 2024-05-09T04:41:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.051+0000 7f0e3e526700 1 -- 172.21.15.110:0/4183574201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e38064e20 con 0x7f0e200035c0 2024-05-09T04:41:27.052 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.051+0000 7f0e357fa700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f0e2403e9a0 0x7f0e24040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:27.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.051+0000 7f0e357fa700 1 -- 172.21.15.110:0/4183574201 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f0e28054ec0 con 0x7f0e200035c0 2024-05-09T04:41:27.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.051+0000 7f0e377fe700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f0e2403e9a0 0x7f0e24040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:27.053 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.052+0000 7f0e377fe700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f0e2403e9a0 0x7f0e24040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0e38104180 tx=0x7f0e2c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:27.056 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.057+0000 7f0e357fa700 1 -- 172.21.15.110:0/4183574201 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e28034020 con 0x7f0e200035c0 2024-05-09T04:41:27.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:41:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: mon.a calling monitor election 2024-05-09T04:41:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: mon.c calling monitor election 2024-05-09T04:41:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:27.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:27.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T04:41:27.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: monmap epoch 3 2024-05-09T04:41:27.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:27.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: last_changed 2024-05-09T04:41:21.798409+0000 2024-05-09T04:41:27.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: created 2024-05-09T04:39:22.726361+0000 2024-05-09T04:41:27.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: min_mon_release 19 (squid) 2024-05-09T04:41:27.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: election_strategy: 1 2024-05-09T04:41:27.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T04:41:27.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-09T04:41:27.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: 2: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.b 2024-05-09T04:41:27.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: fsmap 2024-05-09T04:41:27.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:41:27.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: mgrmap e13: x(active, since 59s) 2024-05-09T04:41:27.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: overall HEALTH_OK 2024-05-09T04:41:27.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:27.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:27.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:27.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:27.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:41:27.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: mon.a calling monitor election 2024-05-09T04:41:27.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: mon.c calling monitor election 2024-05-09T04:41:27.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:27.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:27.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:27.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:27.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:27.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:27.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:27.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:27.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:27.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T04:41:27.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: monmap epoch 3 2024-05-09T04:41:27.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:27.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: last_changed 2024-05-09T04:41:21.798409+0000 2024-05-09T04:41:27.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: created 2024-05-09T04:39:22.726361+0000 2024-05-09T04:41:27.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: min_mon_release 19 (squid) 2024-05-09T04:41:27.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: election_strategy: 1 2024-05-09T04:41:27.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T04:41:27.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-09T04:41:27.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: 2: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.b 2024-05-09T04:41:27.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: fsmap 2024-05-09T04:41:27.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:41:27.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: mgrmap e13: x(active, since 59s) 2024-05-09T04:41:27.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: overall HEALTH_OK 2024-05-09T04:41:27.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:27.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:27.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:27.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:27.356 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.357+0000 7f0e3e526700 1 -- 172.21.15.110:0/4183574201 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0e38101150 con 0x7f0e200035c0 2024-05-09T04:41:27.357 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.358+0000 7f0e357fa700 1 -- 172.21.15.110:0/4183574201 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f0e28025e60 con 0x7f0e200035c0 2024-05-09T04:41:27.357 INFO:teuthology.orchestra.run.smithi110.stdout: 2024-05-09T04:41:27.357 INFO:teuthology.orchestra.run.smithi110.stdout:{"epoch":3,"fsid":"ecf9ae2c-0dbd-11ef-bc97-c7b262605968","modified":"2024-05-09T04:41:21.798409Z","created":"2024-05-09T04:39:22.726361Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:3300","nonce":0},{"type":"v1","addr":"172.21.15.46:6789","nonce":0}]},"addr":"172.21.15.46:6789/0","public_addr":"172.21.15.46:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:3300","nonce":0},{"type":"v1","addr":"172.21.15.110:6789","nonce":0}]},"addr":"172.21.15.110:6789/0","public_addr":"172.21.15.110:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:3300","nonce":0},{"type":"v1","addr":"172.21.15.72:6789","nonce":0}]},"addr":"172.21.15.72:6789/0","public_addr":"172.21.15.72:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-09T04:41:27.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.360+0000 7f0e3e526700 1 -- 172.21.15.110:0/4183574201 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f0e2403e9a0 msgr2=0x7f0e24040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:27.359 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.360+0000 7f0e3e526700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f0e2403e9a0 0x7f0e24040eb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0e38104180 tx=0x7f0e2c008040 comp rx=0 tx=0).stop 2024-05-09T04:41:27.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.360+0000 7f0e3e526700 1 -- 172.21.15.110:0/4183574201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200035c0 msgr2=0x7f0e380ffc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:27.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.360+0000 7f0e3e526700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200035c0 0x7f0e380ffc20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0e28011010 tx=0x7f0e28003d00 comp rx=0 tx=0).stop 2024-05-09T04:41:27.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.360+0000 7f0e3e526700 1 -- 172.21.15.110:0/4183574201 shutdown_connections 2024-05-09T04:41:27.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.360+0000 7f0e3e526700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f0e2403e9a0 0x7f0e24040eb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:27.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.360+0000 7f0e3e526700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0e381a5e80 0x7f0e381a82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:27.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.360+0000 7f0e3e526700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0e200035c0 0x7f0e380ffc20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:27.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.360+0000 7f0e3e526700 1 --2- 172.21.15.110:0/4183574201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0e38100160 0x7f0e38100570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:27.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.360+0000 7f0e3e526700 1 -- 172.21.15.110:0/4183574201 >> 172.21.15.110:0/4183574201 conn(0x7f0e38077e10 msgr2=0x7f0e38105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:27.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.361+0000 7f0e3e526700 1 -- 172.21.15.110:0/4183574201 shutdown_connections 2024-05-09T04:41:27.360 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:27.361+0000 7f0e3e526700 1 -- 172.21.15.110:0/4183574201 wait complete. 2024-05-09T04:41:27.361 INFO:teuthology.orchestra.run.smithi110.stderr:dumped monmap epoch 3 2024-05-09T04:41:27.919 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-09T04:41:27.920 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-09T04:41:28.176 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:41:28.298 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:28.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:27 smithi046 ceph-mon[30136]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:28.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:28.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:27 smithi046 ceph-mon[30136]: from='client.? 172.21.15.110:0/4183574201' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T04:41:28.299 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:28.305 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:28.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:27 smithi110 ceph-mon[34904]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:28.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:28.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:27 smithi110 ceph-mon[34904]: from='client.? 172.21.15.110:0/4183574201' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T04:41:28.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:29.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:41:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: mon.a calling monitor election 2024-05-09T04:41:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: mon.c calling monitor election 2024-05-09T04:41:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:41:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:29.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:29.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:29.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-09T04:41:29.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: monmap epoch 3 2024-05-09T04:41:29.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:29.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: last_changed 2024-05-09T04:41:21.798409+0000 2024-05-09T04:41:29.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: created 2024-05-09T04:39:22.726361+0000 2024-05-09T04:41:29.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: min_mon_release 19 (squid) 2024-05-09T04:41:29.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: election_strategy: 1 2024-05-09T04:41:29.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T04:41:29.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-09T04:41:29.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: 2: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.b 2024-05-09T04:41:29.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: fsmap 2024-05-09T04:41:29.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:41:29.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: mgrmap e13: x(active, since 59s) 2024-05-09T04:41:29.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: overall HEALTH_OK 2024-05-09T04:41:29.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:29.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:29.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:29.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:29.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:29.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:29.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:29.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='client.? 172.21.15.110:0/4183574201' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-09T04:41:29.133 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.745+0000 7f2f48c3c700 1 -- 172.21.15.46:0/709678084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f44104d20 msgr2=0x7f2f44105130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.745+0000 7f2f48c3c700 1 --2- 172.21.15.46:0/709678084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f44104d20 0x7f2f44105130 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009910 tx=0x7f2f34009c20 comp rx=0 tx=0).stop 2024-05-09T04:41:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.745+0000 7f2f48c3c700 1 -- 172.21.15.46:0/709678084 shutdown_connections 2024-05-09T04:41:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.745+0000 7f2f48c3c700 1 --2- 172.21.15.46:0/709678084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f44104d20 0x7f2f44105130 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.745+0000 7f2f48c3c700 1 -- 172.21.15.46:0/709678084 >> 172.21.15.46:0/709678084 conn(0x7f2f440fe5b0 msgr2=0x7f2f44100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.745+0000 7f2f48c3c700 1 -- 172.21.15.46:0/709678084 shutdown_connections 2024-05-09T04:41:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.745+0000 7f2f48c3c700 1 -- 172.21.15.46:0/709678084 wait complete. 2024-05-09T04:41:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.746+0000 7f2f48c3c700 1 Processor -- start 2024-05-09T04:41:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.746+0000 7f2f48c3c700 1 -- start start 2024-05-09T04:41:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.746+0000 7f2f48c3c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f44104d20 0x7f2f44102a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.746+0000 7f2f48c3c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f44102f50 0x7f2f44103360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.746+0000 7f2f48c3c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f441a5e20 0x7f2f441a8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.746+0000 7f2f48c3c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4410e110 con 0x7f2f441a5e20 2024-05-09T04:41:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.746+0000 7f2f48c3c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4410df90 con 0x7f2f44104d20 2024-05-09T04:41:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.746+0000 7f2f48c3c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4410e290 con 0x7f2f44102f50 2024-05-09T04:41:29.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.746+0000 7f2f42d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f441a5e20 0x7f2f441a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.746+0000 7f2f42d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f441a5e20 0x7f2f441a8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39484/0 (socket says 172.21.15.46:39484) 2024-05-09T04:41:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.746+0000 7f2f42d9d700 1 -- 172.21.15.46:0/146110390 learned_addr learned my addr 172.21.15.46:0/146110390 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:41:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.746+0000 7f2f41d9b700 1 --2- 172.21.15.46:0/146110390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f44102f50 0x7f2f44103360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.747+0000 7f2f4259c700 1 --2- 172.21.15.46:0/146110390 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f44104d20 0x7f2f44102a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.747+0000 7f2f42d9d700 1 -- 172.21.15.46:0/146110390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f44102f50 msgr2=0x7f2f44103360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.747+0000 7f2f42d9d700 1 --2- 172.21.15.46:0/146110390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f44102f50 0x7f2f44103360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.747+0000 7f2f42d9d700 1 -- 172.21.15.46:0/146110390 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f44104d20 msgr2=0x7f2f44102a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.747+0000 7f2f42d9d700 1 --2- 172.21.15.46:0/146110390 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f44104d20 0x7f2f44102a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.747+0000 7f2f42d9d700 1 -- 172.21.15.46:0/146110390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f441a88c0 con 0x7f2f441a5e20 2024-05-09T04:41:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.747+0000 7f2f42d9d700 1 --2- 172.21.15.46:0/146110390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f441a5e20 0x7f2f441a8240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00ba50 tx=0x7f2f3c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.747+0000 7f2f337fe700 1 -- 172.21.15.46:0/146110390 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c019070 con 0x7f2f441a5e20 2024-05-09T04:41:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.747+0000 7f2f337fe700 1 -- 172.21.15.46:0/146110390 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f3c00cde0 con 0x7f2f441a5e20 2024-05-09T04:41:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.747+0000 7f2f48c3c700 1 -- 172.21.15.46:0/146110390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f441a8bb0 con 0x7f2f441a5e20 2024-05-09T04:41:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.747+0000 7f2f337fe700 1 -- 172.21.15.46:0/146110390 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c007350 con 0x7f2f441a5e20 2024-05-09T04:41:29.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.747+0000 7f2f48c3c700 1 -- 172.21.15.46:0/146110390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f441a90f0 con 0x7f2f441a5e20 2024-05-09T04:41:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.748+0000 7f2f337fe700 1 -- 172.21.15.46:0/146110390 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f2f3c01c070 con 0x7f2f441a5e20 2024-05-09T04:41:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.748+0000 7f2f337fe700 1 --2- 172.21.15.46:0/146110390 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2f2c03e9a0 0x7f2f2c040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.749+0000 7f2f48c3c700 1 -- 172.21.15.46:0/146110390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f44064e20 con 0x7f2f441a5e20 2024-05-09T04:41:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.749+0000 7f2f337fe700 1 -- 172.21.15.46:0/146110390 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2f3c0145d0 con 0x7f2f441a5e20 2024-05-09T04:41:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.749+0000 7f2f4259c700 1 --2- 172.21.15.46:0/146110390 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2f2c03e9a0 0x7f2f2c040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.749+0000 7f2f4259c700 1 --2- 172.21.15.46:0/146110390 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2f2c03e9a0 0x7f2f2c040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009750 tx=0x7f2f3401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.754+0000 7f2f337fe700 1 -- 172.21.15.46:0/146110390 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f3c00e0a0 con 0x7f2f441a5e20 2024-05-09T04:41:29.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.992+0000 7f2f48c3c700 1 -- 172.21.15.46:0/146110390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f2f441a9440 con 0x7f2f441a5e20 2024-05-09T04:41:29.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.992+0000 7f2f337fe700 1 -- 172.21.15.46:0/146110390 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f2f3c01e440 con 0x7f2f441a5e20 2024-05-09T04:41:29.993 INFO:teuthology.orchestra.run.smithi046.stdout:# minimal ceph.conf for ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:29.993 INFO:teuthology.orchestra.run.smithi046.stdout:[global] 2024-05-09T04:41:29.993 INFO:teuthology.orchestra.run.smithi046.stdout: fsid = ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:29.993 INFO:teuthology.orchestra.run.smithi046.stdout: mon_host = [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] 2024-05-09T04:41:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.995+0000 7f2f48c3c700 1 -- 172.21.15.46:0/146110390 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2f2c03e9a0 msgr2=0x7f2f2c040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.995+0000 7f2f48c3c700 1 --2- 172.21.15.46:0/146110390 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2f2c03e9a0 0x7f2f2c040eb0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009750 tx=0x7f2f3401a040 comp rx=0 tx=0).stop 2024-05-09T04:41:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.995+0000 7f2f48c3c700 1 -- 172.21.15.46:0/146110390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f441a5e20 msgr2=0x7f2f441a8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.995+0000 7f2f48c3c700 1 --2- 172.21.15.46:0/146110390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f441a5e20 0x7f2f441a8240 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c00ba50 tx=0x7f2f3c00bd60 comp rx=0 tx=0).stop 2024-05-09T04:41:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.995+0000 7f2f48c3c700 1 -- 172.21.15.46:0/146110390 shutdown_connections 2024-05-09T04:41:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.995+0000 7f2f48c3c700 1 --2- 172.21.15.46:0/146110390 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2f2c03e9a0 0x7f2f2c040eb0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.995+0000 7f2f48c3c700 1 --2- 172.21.15.46:0/146110390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f441a5e20 0x7f2f441a8240 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.995+0000 7f2f48c3c700 1 --2- 172.21.15.46:0/146110390 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f44104d20 0x7f2f44102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.995+0000 7f2f48c3c700 1 --2- 172.21.15.46:0/146110390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f44102f50 0x7f2f44103360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.995+0000 7f2f48c3c700 1 -- 172.21.15.46:0/146110390 >> 172.21.15.46:0/146110390 conn(0x7f2f440fe5b0 msgr2=0x7f2f4410a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:29.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.995+0000 7f2f48c3c700 1 -- 172.21.15.46:0/146110390 shutdown_connections 2024-05-09T04:41:29.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:29.995+0000 7f2f48c3c700 1 -- 172.21.15.46:0/146110390 wait complete. 2024-05-09T04:41:30.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: mon.b calling monitor election 2024-05-09T04:41:30.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-09T04:41:30.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-09T04:41:30.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: Updating smithi110:/etc/ceph/ceph.conf 2024-05-09T04:41:30.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: mon.b calling monitor election 2024-05-09T04:41:30.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: mon.c calling monitor election 2024-05-09T04:41:30.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: mon.a calling monitor election 2024-05-09T04:41:30.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T04:41:30.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: monmap epoch 3 2024-05-09T04:41:30.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:30.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: last_changed 2024-05-09T04:41:21.798409+0000 2024-05-09T04:41:30.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: created 2024-05-09T04:39:22.726361+0000 2024-05-09T04:41:30.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: min_mon_release 19 (squid) 2024-05-09T04:41:30.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: election_strategy: 1 2024-05-09T04:41:30.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T04:41:30.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-09T04:41:30.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: 2: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.b 2024-05-09T04:41:30.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: fsmap 2024-05-09T04:41:30.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:41:30.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: mgrmap e13: x(active, since 61s) 2024-05-09T04:41:30.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: overall HEALTH_OK 2024-05-09T04:41:30.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: Updating smithi046:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:30.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: Updating smithi072:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:30.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: Updating smithi110:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:30.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:30.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.035 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T04:41:30.036 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:30.036 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:41:30.036 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:30.036 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:29 smithi046 ceph-mon[30136]: Reconfiguring daemon mon.a on smithi046 2024-05-09T04:41:30.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: mon.b calling monitor election 2024-05-09T04:41:30.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-09T04:41:30.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-09T04:41:30.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: Updating smithi110:/etc/ceph/ceph.conf 2024-05-09T04:41:30.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: mon.b calling monitor election 2024-05-09T04:41:30.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: mon.c calling monitor election 2024-05-09T04:41:30.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: mon.a calling monitor election 2024-05-09T04:41:30.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T04:41:30.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: monmap epoch 3 2024-05-09T04:41:30.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:30.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: last_changed 2024-05-09T04:41:21.798409+0000 2024-05-09T04:41:30.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: created 2024-05-09T04:39:22.726361+0000 2024-05-09T04:41:30.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: min_mon_release 19 (squid) 2024-05-09T04:41:30.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: election_strategy: 1 2024-05-09T04:41:30.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T04:41:30.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-09T04:41:30.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: 2: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.b 2024-05-09T04:41:30.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: fsmap 2024-05-09T04:41:30.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:41:30.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: mgrmap e13: x(active, since 61s) 2024-05-09T04:41:30.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: overall HEALTH_OK 2024-05-09T04:41:30.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: Updating smithi046:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:30.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: Updating smithi072:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:30.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: Updating smithi110:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:30.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:30.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T04:41:30.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:30.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:41:30.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:30.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:29 smithi110 ceph-mon[34904]: Reconfiguring daemon mon.a on smithi046 2024-05-09T04:41:30.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: mon.b calling monitor election 2024-05-09T04:41:30.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-09T04:41:30.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-09T04:41:30.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: Updating smithi110:/etc/ceph/ceph.conf 2024-05-09T04:41:30.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: mon.b calling monitor election 2024-05-09T04:41:30.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: mon.c calling monitor election 2024-05-09T04:41:30.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: mon.a calling monitor election 2024-05-09T04:41:30.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-09T04:41:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: monmap epoch 3 2024-05-09T04:41:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:41:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: last_changed 2024-05-09T04:41:21.798409+0000 2024-05-09T04:41:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: created 2024-05-09T04:39:22.726361+0000 2024-05-09T04:41:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: min_mon_release 19 (squid) 2024-05-09T04:41:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: election_strategy: 1 2024-05-09T04:41:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: 0: [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] mon.a 2024-05-09T04:41:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: 1: [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] mon.c 2024-05-09T04:41:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: 2: [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] mon.b 2024-05-09T04:41:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: fsmap 2024-05-09T04:41:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: osdmap e4: 0 total, 0 up, 0 in 2024-05-09T04:41:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: mgrmap e13: x(active, since 61s) 2024-05-09T04:41:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: overall HEALTH_OK 2024-05-09T04:41:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: Updating smithi046:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: Updating smithi072:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: Updating smithi110:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: Reconfiguring mon.a (unknown last config time)... 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:41:30.133 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:30.133 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:29 smithi072 ceph-mon[34088]: Reconfiguring daemon mon.a on smithi046 2024-05-09T04:41:31.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:30 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:31.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:30 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/146110390' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:31.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:30 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:31.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:30 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/146110390' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:31.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:30 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:41:31.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:30 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/146110390' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:31.434 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-09T04:41:31.434 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:41:31.434 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T04:41:31.462 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:41:31.462 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:41:31.528 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-09T04:41:31.528 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T04:41:31.555 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-09T04:41:31.555 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:41:31.619 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-09T04:41:31.619 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.conf 2024-05-09T04:41:31.650 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-09T04:41:31.650 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:41:31.718 INFO:tasks.cephadm:Adding mgr.x on smithi046 2024-05-09T04:41:31.718 INFO:tasks.cephadm:Adding mgr.y on smithi072 2024-05-09T04:41:31.718 INFO:tasks.cephadm:Adding mgr.z on smithi110 2024-05-09T04:41:31.718 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi046=x;smithi072=y;smithi110=z' 2024-05-09T04:41:31.993 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:41:32.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:32 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:32.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:32 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:32.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:32 smithi072 ceph-mon[34088]: Reconfiguring mon.b (monmap changed)... 2024-05-09T04:41:32.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:32 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:32 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:41:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:32 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:32 smithi072 ceph-mon[34088]: Reconfiguring daemon mon.b on smithi072 2024-05-09T04:41:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:32 smithi072 ceph-mon[34088]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:32.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:32 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:32.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:32 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:32.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:32 smithi110 ceph-mon[34904]: Reconfiguring mon.b (monmap changed)... 2024-05-09T04:41:32.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:32 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:32.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:32 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:41:32.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:32 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:32.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:32 smithi110 ceph-mon[34904]: Reconfiguring daemon mon.b on smithi072 2024-05-09T04:41:32.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:32 smithi110 ceph-mon[34904]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:32.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:32 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:32.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:32 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:32.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:32 smithi046 ceph-mon[30136]: Reconfiguring mon.b (monmap changed)... 2024-05-09T04:41:32.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:32 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:32.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:32 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:41:32.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:32 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:32.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:32 smithi046 ceph-mon[30136]: Reconfiguring daemon mon.b on smithi072 2024-05-09T04:41:32.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:32 smithi046 ceph-mon[30136]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:33.409 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.410+0000 7fade7691700 1 -- 172.21.15.110:0/901324091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fade0103d10 msgr2=0x7fade0104120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:33.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.410+0000 7fade7691700 1 --2- 172.21.15.110:0/901324091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fade0103d10 0x7fade0104120 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7faddc009910 tx=0x7faddc009c20 comp rx=0 tx=0).stop 2024-05-09T04:41:33.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.410+0000 7fade7691700 1 -- 172.21.15.110:0/901324091 shutdown_connections 2024-05-09T04:41:33.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.410+0000 7fade7691700 1 --2- 172.21.15.110:0/901324091 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fade0103d10 0x7fade0104120 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:33.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.410+0000 7fade7691700 1 -- 172.21.15.110:0/901324091 >> 172.21.15.110:0/901324091 conn(0x7fade00ff350 msgr2=0x7fade0101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:33.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.410+0000 7fade7691700 1 -- 172.21.15.110:0/901324091 shutdown_connections 2024-05-09T04:41:33.410 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.410+0000 7fade7691700 1 -- 172.21.15.110:0/901324091 wait complete. 2024-05-09T04:41:33.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.411+0000 7fade7691700 1 Processor -- start 2024-05-09T04:41:33.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.411+0000 7fade7691700 1 -- start start 2024-05-09T04:41:33.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.411+0000 7fade7691700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fade0103d10 0x7fade01999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:33.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.411+0000 7fade7691700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade0199f10 0x7fade019eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:33.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.411+0000 7fade7691700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fade019f3e0 0x7fade01a1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:33.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.411+0000 7fade7691700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fade0109cd0 con 0x7fade0103d10 2024-05-09T04:41:33.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.411+0000 7fade7691700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fade0109b50 con 0x7fade019f3e0 2024-05-09T04:41:33.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.411+0000 7fade7691700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fade0109e50 con 0x7fade0199f10 2024-05-09T04:41:33.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fade4c2c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade0199f10 0x7fade019eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:33.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fade4c2c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade0199f10 0x7fade019eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:51948/0 (socket says 172.21.15.110:51948) 2024-05-09T04:41:33.412 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fade4c2c700 1 -- 172.21.15.110:0/3443975239 learned_addr learned my addr 172.21.15.110:0/3443975239 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-09T04:41:33.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fade542d700 1 --2- 172.21.15.110:0/3443975239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fade0103d10 0x7fade01999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:33.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fade5c2e700 1 --2- 172.21.15.110:0/3443975239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fade019f3e0 0x7fade01a1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:33.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fade4c2c700 1 -- 172.21.15.110:0/3443975239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fade019f3e0 msgr2=0x7fade01a1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:33.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fade4c2c700 1 --2- 172.21.15.110:0/3443975239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fade019f3e0 0x7fade01a1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:33.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fade4c2c700 1 -- 172.21.15.110:0/3443975239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fade0103d10 msgr2=0x7fade01999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:33.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fade4c2c700 1 --2- 172.21.15.110:0/3443975239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fade0103d10 0x7fade01999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:33.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fade4c2c700 1 -- 172.21.15.110:0/3443975239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fade01a1d70 con 0x7fade0199f10 2024-05-09T04:41:33.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fade4c2c700 1 --2- 172.21.15.110:0/3443975239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade0199f10 0x7fade019eea0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fadd000b780 tx=0x7fadd000ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:33.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fadd67fc700 1 -- 172.21.15.110:0/3443975239 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd0016070 con 0x7fade0199f10 2024-05-09T04:41:33.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fadd67fc700 1 -- 172.21.15.110:0/3443975239 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadd00036a0 con 0x7fade0199f10 2024-05-09T04:41:33.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fade7691700 1 -- 172.21.15.110:0/3443975239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fade01a2060 con 0x7fade0199f10 2024-05-09T04:41:33.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fadd67fc700 1 -- 172.21.15.110:0/3443975239 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadd00119c0 con 0x7fade0199f10 2024-05-09T04:41:33.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.412+0000 7fade7691700 1 -- 172.21.15.110:0/3443975239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fade01a2620 con 0x7fade0199f10 2024-05-09T04:41:33.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.413+0000 7fade7691700 1 -- 172.21.15.110:0/3443975239 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fade0064e20 con 0x7fade0199f10 2024-05-09T04:41:33.413 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.413+0000 7fadd67fc700 1 -- 172.21.15.110:0/3443975239 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fadd0019070 con 0x7fade0199f10 2024-05-09T04:41:33.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.413+0000 7fadd67fc700 1 --2- 172.21.15.110:0/3443975239 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fadcc03e9a0 0x7fadcc040eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:33.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.414+0000 7fadd67fc700 1 -- 172.21.15.110:0/3443975239 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fadd0051300 con 0x7fade0199f10 2024-05-09T04:41:33.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.414+0000 7fade542d700 1 --2- 172.21.15.110:0/3443975239 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fadcc03e9a0 0x7fadcc040eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:33.414 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.414+0000 7fade542d700 1 --2- 172.21.15.110:0/3443975239 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fadcc03e9a0 0x7fadcc040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faddc006010 tx=0x7faddc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:33.418 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.419+0000 7fadd67fc700 1 -- 172.21.15.110:0/3443975239 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fadd001d030 con 0x7fade0199f10 2024-05-09T04:41:33.638 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.639+0000 7fade7691700 1 -- 172.21.15.110:0/3443975239 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi072=y;smithi110=z", "target": ["mon-mgr", ""]}) -- 0x7fade01011c0 con 0x7fadcc03e9a0 2024-05-09T04:41:33.645 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.646+0000 7fadd67fc700 1 -- 172.21.15.110:0/3443975239 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fade01011c0 con 0x7fadcc03e9a0 2024-05-09T04:41:33.645 INFO:teuthology.orchestra.run.smithi110.stdout:Scheduled mgr update... 2024-05-09T04:41:33.647 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.648+0000 7fade7691700 1 -- 172.21.15.110:0/3443975239 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fadcc03e9a0 msgr2=0x7fadcc040eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.648+0000 7fade7691700 1 --2- 172.21.15.110:0/3443975239 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fadcc03e9a0 0x7fadcc040eb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faddc006010 tx=0x7faddc01a040 comp rx=0 tx=0).stop 2024-05-09T04:41:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.648+0000 7fade7691700 1 -- 172.21.15.110:0/3443975239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade0199f10 msgr2=0x7fade019eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.648+0000 7fade7691700 1 --2- 172.21.15.110:0/3443975239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade0199f10 0x7fade019eea0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fadd000b780 tx=0x7fadd000ba90 comp rx=0 tx=0).stop 2024-05-09T04:41:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.649+0000 7fade7691700 1 -- 172.21.15.110:0/3443975239 shutdown_connections 2024-05-09T04:41:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.649+0000 7fade7691700 1 --2- 172.21.15.110:0/3443975239 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fadcc03e9a0 0x7fadcc040eb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.649+0000 7fade7691700 1 --2- 172.21.15.110:0/3443975239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fade019f3e0 0x7fade01a1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.649+0000 7fade7691700 1 --2- 172.21.15.110:0/3443975239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fade0103d10 0x7fade01999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.649+0000 7fade7691700 1 --2- 172.21.15.110:0/3443975239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fade0199f10 0x7fade019eea0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.649+0000 7fade7691700 1 -- 172.21.15.110:0/3443975239 >> 172.21.15.110:0/3443975239 conn(0x7fade00ff350 msgr2=0x7fade0100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.649+0000 7fade7691700 1 -- 172.21.15.110:0/3443975239 shutdown_connections 2024-05-09T04:41:33.648 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:41:33.649+0000 7fade7691700 1 -- 172.21.15.110:0/3443975239 wait complete. 2024-05-09T04:41:33.988 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:33 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:33.988 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:33 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:33.988 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:33 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:33.988 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:33 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:41:33.988 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:33 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:33.988 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:33 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:34.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:33 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:34.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:33 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:34.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:33 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:34.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:33 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:41:34.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:33 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:34.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:33 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:34.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:33 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:34.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:33 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:34.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:33 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-09T04:41:34.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:33 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-09T04:41:34.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:33 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:34.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:33 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:34.603 DEBUG:teuthology.orchestra.run.smithi072:mgr.y> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.y.service 2024-05-09T04:41:34.605 DEBUG:teuthology.orchestra.run.smithi110:mgr.z> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.z.service 2024-05-09T04:41:34.608 INFO:tasks.cephadm:Deploying OSDs... 2024-05-09T04:41:34.608 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:41:34.608 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T04:41:34.628 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T04:41:34.628 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_1 2024-05-09T04:41:34.688 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T04:41:34.688 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:41:34.688 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-09T04:41:34.689 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:41:34.689 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:41:34.689 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-09 04:40:40.015410903 +0000 2024-05-09T04:41:34.689 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-09 04:35:12.409844297 +0000 2024-05-09T04:41:34.689 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-09 04:35:12.409844297 +0000 2024-05-09T04:41:34.689 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-09 04:35:12.409844297 +0000 2024-05-09T04:41:34.689 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T04:41:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-09T04:41:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-09T04:41:34.758 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000116949 s, 4.4 MB/s 2024-05-09T04:41:34.760 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T04:41:34.819 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_2 2024-05-09T04:41:34.878 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T04:41:34.878 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:41:34.878 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-05-09T04:41:34.878 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:41:34.878 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:41:34.878 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-09 04:40:40.053410508 +0000 2024-05-09T04:41:34.878 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-09 04:35:12.747840850 +0000 2024-05-09T04:41:34.878 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-09 04:35:12.747840850 +0000 2024-05-09T04:41:34.878 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-09 04:35:12.747840850 +0000 2024-05-09T04:41:34.879 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T04:41:34.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:34 smithi072 ceph-mon[34088]: Reconfiguring mon.c (monmap changed)... 2024-05-09T04:41:34.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:34 smithi072 ceph-mon[34088]: Reconfiguring daemon mon.c on smithi110 2024-05-09T04:41:34.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:34 smithi072 ceph-mon[34088]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:34.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:34 smithi072 ceph-mon[34088]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi072=y;smithi110=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:41:34.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:34 smithi072 ceph-mon[34088]: Saving service mgr spec with placement smithi046=x;smithi072=y;smithi110=z;count:3 2024-05-09T04:41:34.945 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-09T04:41:34.945 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-09T04:41:34.945 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000105908 s, 4.8 MB/s 2024-05-09T04:41:34.946 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T04:41:35.004 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_3 2024-05-09T04:41:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:34 smithi110 ceph-mon[34904]: Reconfiguring mon.c (monmap changed)... 2024-05-09T04:41:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:34 smithi110 ceph-mon[34904]: Reconfiguring daemon mon.c on smithi110 2024-05-09T04:41:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:34 smithi110 ceph-mon[34904]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:34 smithi110 ceph-mon[34904]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi072=y;smithi110=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:41:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:34 smithi110 ceph-mon[34904]: Saving service mgr spec with placement smithi046=x;smithi072=y;smithi110=z;count:3 2024-05-09T04:41:35.061 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T04:41:35.061 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:41:35.062 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-09T04:41:35.062 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:41:35.062 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:41:35.062 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-09 04:40:40.096410061 +0000 2024-05-09T04:41:35.062 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-09 04:35:13.085837404 +0000 2024-05-09T04:41:35.062 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-09 04:35:13.085837404 +0000 2024-05-09T04:41:35.062 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-09 04:35:13.085837404 +0000 2024-05-09T04:41:35.062 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T04:41:35.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:34 smithi046 ceph-mon[30136]: Reconfiguring mon.c (monmap changed)... 2024-05-09T04:41:35.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:34 smithi046 ceph-mon[30136]: Reconfiguring daemon mon.c on smithi110 2024-05-09T04:41:35.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:34 smithi046 ceph-mon[30136]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:35.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:34 smithi046 ceph-mon[30136]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi046=x;smithi072=y;smithi110=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:41:35.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:34 smithi046 ceph-mon[30136]: Saving service mgr spec with placement smithi046=x;smithi072=y;smithi110=z;count:3 2024-05-09T04:41:35.104 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-09T04:41:35.104 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-09T04:41:35.104 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 0.000116137 s, 4.4 MB/s 2024-05-09T04:41:35.106 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T04:41:35.163 DEBUG:teuthology.orchestra.run.smithi046:> stat /dev/vg_nvme/lv_4 2024-05-09T04:41:35.220 INFO:teuthology.orchestra.run.smithi046.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T04:41:35.220 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:41:35.220 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-09T04:41:35.220 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:41:35.220 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:41:35.220 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-09 04:40:40.130409708 +0000 2024-05-09T04:41:35.220 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-09 04:35:13.429833895 +0000 2024-05-09T04:41:35.220 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-09 04:35:13.429833895 +0000 2024-05-09T04:41:35.220 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-09 04:35:13.429833895 +0000 2024-05-09T04:41:35.221 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T04:41:35.286 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records in 2024-05-09T04:41:35.286 INFO:teuthology.orchestra.run.smithi046.stderr:1+0 records out 2024-05-09T04:41:35.286 INFO:teuthology.orchestra.run.smithi046.stderr:512 bytes copied, 9.0178e-05 s, 5.7 MB/s 2024-05-09T04:41:35.287 DEBUG:teuthology.orchestra.run.smithi046:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T04:41:35.345 DEBUG:teuthology.orchestra.run.smithi072:> set -ex 2024-05-09T04:41:35.345 DEBUG:teuthology.orchestra.run.smithi072:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T04:41:35.364 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T04:41:35.364 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_1 2024-05-09T04:41:35.422 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T04:41:35.422 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:41:35.422 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 661 Links: 1 2024-05-09T04:41:35.422 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:41:35.423 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:41:35.423 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-05-09 04:41:21.234468187 +0000 2024-05-09T04:41:35.423 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-05-09 04:35:25.222809325 +0000 2024-05-09T04:41:35.423 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-05-09 04:35:25.222809325 +0000 2024-05-09T04:41:35.423 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-05-09 04:35:25.222809325 +0000 2024-05-09T04:41:35.423 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T04:41:35.488 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-05-09T04:41:35.488 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-05-09T04:41:35.488 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000279655 s, 1.8 MB/s 2024-05-09T04:41:35.489 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T04:41:35.547 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_2 2024-05-09T04:41:35.604 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T04:41:35.604 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:41:35.604 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-05-09T04:41:35.604 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:41:35.604 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:41:35.604 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-05-09 04:41:21.235468165 +0000 2024-05-09T04:41:35.604 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-05-09 04:35:25.553802060 +0000 2024-05-09T04:41:35.604 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-05-09 04:35:25.553802060 +0000 2024-05-09T04:41:35.604 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-05-09 04:35:25.553802060 +0000 2024-05-09T04:41:35.604 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T04:41:35.668 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-05-09T04:41:35.668 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-05-09T04:41:35.668 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000193483 s, 2.6 MB/s 2024-05-09T04:41:35.669 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T04:41:35.725 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_3 2024-05-09T04:41:35.780 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T04:41:35.781 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:41:35.781 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 700 Links: 1 2024-05-09T04:41:35.781 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:41:35.781 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:41:35.781 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-05-09 04:41:21.235468165 +0000 2024-05-09T04:41:35.781 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-05-09 04:35:25.900794443 +0000 2024-05-09T04:41:35.781 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-05-09 04:35:25.900794443 +0000 2024-05-09T04:41:35.781 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-05-09 04:35:25.900794443 +0000 2024-05-09T04:41:35.781 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T04:41:35.843 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-05-09T04:41:35.843 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-05-09T04:41:35.843 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.000199519 s, 2.6 MB/s 2024-05-09T04:41:35.845 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T04:41:35.901 DEBUG:teuthology.orchestra.run.smithi072:> stat /dev/vg_nvme/lv_4 2024-05-09T04:41:35.956 INFO:teuthology.orchestra.run.smithi072.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T04:41:35.956 INFO:teuthology.orchestra.run.smithi072.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:41:35.956 INFO:teuthology.orchestra.run.smithi072.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-05-09T04:41:35.956 INFO:teuthology.orchestra.run.smithi072.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:41:35.956 INFO:teuthology.orchestra.run.smithi072.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:41:35.956 INFO:teuthology.orchestra.run.smithi072.stdout:Access: 2024-05-09 04:41:21.235468165 +0000 2024-05-09T04:41:35.957 INFO:teuthology.orchestra.run.smithi072.stdout:Modify: 2024-05-09 04:35:26.240786981 +0000 2024-05-09T04:41:35.957 INFO:teuthology.orchestra.run.smithi072.stdout:Change: 2024-05-09 04:35:26.240786981 +0000 2024-05-09T04:41:35.957 INFO:teuthology.orchestra.run.smithi072.stdout: Birth: 2024-05-09 04:35:26.240786981 +0000 2024-05-09T04:41:35.957 DEBUG:teuthology.orchestra.run.smithi072:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T04:41:36.018 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records in 2024-05-09T04:41:36.019 INFO:teuthology.orchestra.run.smithi072.stderr:1+0 records out 2024-05-09T04:41:36.019 INFO:teuthology.orchestra.run.smithi072.stderr:512 bytes copied, 0.00018545 s, 2.8 MB/s 2024-05-09T04:41:36.020 DEBUG:teuthology.orchestra.run.smithi072:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T04:41:36.077 DEBUG:teuthology.orchestra.run.smithi110:> set -ex 2024-05-09T04:41:36.077 DEBUG:teuthology.orchestra.run.smithi110:> dd if=/scratch_devs of=/dev/stdout 2024-05-09T04:41:36.093 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-09T04:41:36.093 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_1 2024-05-09T04:41:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-09T04:41:36.148 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:41:36.148 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-05-09T04:41:36.149 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:41:36.149 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:41:36.149 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-05-09 04:41:11.766422621 +0000 2024-05-09T04:41:36.149 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-05-09 04:35:16.974449442 +0000 2024-05-09T04:41:36.149 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-05-09 04:35:16.974449442 +0000 2024-05-09T04:41:36.149 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-05-09 04:35:16.974449442 +0000 2024-05-09T04:41:36.149 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-09T04:41:36.211 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-05-09T04:41:36.211 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-05-09T04:41:36.211 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000113656 s, 4.5 MB/s 2024-05-09T04:41:36.212 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-09T04:41:36.268 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_2 2024-05-09T04:41:36.323 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-09T04:41:36.323 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:41:36.323 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-05-09T04:41:36.323 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:41:36.323 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:41:36.323 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-05-09 04:41:11.767422604 +0000 2024-05-09T04:41:36.324 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-05-09 04:35:17.297443810 +0000 2024-05-09T04:41:36.324 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-05-09 04:35:17.297443810 +0000 2024-05-09T04:41:36.324 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-05-09 04:35:17.297443810 +0000 2024-05-09T04:41:36.324 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-09T04:41:36.386 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-05-09T04:41:36.386 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-05-09T04:41:36.386 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 9.4306e-05 s, 5.4 MB/s 2024-05-09T04:41:36.388 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-09T04:41:36.444 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_3 2024-05-09T04:41:36.499 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-09T04:41:36.499 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:41:36.499 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-09T04:41:36.499 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:41:36.499 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:41:36.499 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-05-09 04:41:11.767422604 +0000 2024-05-09T04:41:36.500 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-05-09 04:35:17.629438021 +0000 2024-05-09T04:41:36.500 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-05-09 04:35:17.629438021 +0000 2024-05-09T04:41:36.500 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-05-09 04:35:17.629438021 +0000 2024-05-09T04:41:36.500 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-09T04:41:36.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:36 smithi110 ceph-mon[34904]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:36.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:36.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:36.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:36.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:36.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:36.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:36.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:41:36.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T04:41:36.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:41:36.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:36.562 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-05-09T04:41:36.562 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-05-09T04:41:36.562 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 0.000104981 s, 4.9 MB/s 2024-05-09T04:41:36.563 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-09T04:41:36.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:36 smithi046 ceph-mon[30136]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:36.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:36.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:36.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:36.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:36.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:36.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:36.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:41:36.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T04:41:36.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:41:36.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:36.619 DEBUG:teuthology.orchestra.run.smithi110:> stat /dev/vg_nvme/lv_4 2024-05-09T04:41:36.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:36 smithi072 ceph-mon[34088]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:36.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:36.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:36.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:36.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:36.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:36.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:36.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:41:36.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T04:41:36.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:41:36.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:36.673 INFO:teuthology.orchestra.run.smithi110.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-09T04:41:36.673 INFO:teuthology.orchestra.run.smithi110.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-09T04:41:36.673 INFO:teuthology.orchestra.run.smithi110.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-05-09T04:41:36.673 INFO:teuthology.orchestra.run.smithi110.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:41:36.673 INFO:teuthology.orchestra.run.smithi110.stdout:Context: system_u:object_r:device_t:s0 2024-05-09T04:41:36.673 INFO:teuthology.orchestra.run.smithi110.stdout:Access: 2024-05-09 04:41:11.768422587 +0000 2024-05-09T04:41:36.673 INFO:teuthology.orchestra.run.smithi110.stdout:Modify: 2024-05-09 04:35:17.968432109 +0000 2024-05-09T04:41:36.674 INFO:teuthology.orchestra.run.smithi110.stdout:Change: 2024-05-09 04:35:17.968432109 +0000 2024-05-09T04:41:36.674 INFO:teuthology.orchestra.run.smithi110.stdout: Birth: 2024-05-09 04:35:17.968432109 +0000 2024-05-09T04:41:36.674 DEBUG:teuthology.orchestra.run.smithi110:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-09T04:41:36.736 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records in 2024-05-09T04:41:36.737 INFO:teuthology.orchestra.run.smithi110.stderr:1+0 records out 2024-05-09T04:41:36.737 INFO:teuthology.orchestra.run.smithi110.stderr:512 bytes copied, 8.345e-05 s, 6.1 MB/s 2024-05-09T04:41:36.738 DEBUG:teuthology.orchestra.run.smithi110:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-09T04:41:36.793 INFO:tasks.cephadm:Deploying osd.0 on smithi046 with /dev/vg_nvme/lv_4... 2024-05-09T04:41:36.794 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T04:41:37.050 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:41:37.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:37 smithi046 ceph-mon[30136]: Deploying daemon mgr.z on smithi110 2024-05-09T04:41:37.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:37 smithi110 ceph-mon[34904]: Deploying daemon mgr.z on smithi110 2024-05-09T04:41:37.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:37 smithi072 ceph-mon[34088]: Deploying daemon mgr.z on smithi110 2024-05-09T04:41:38.028 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:37 smithi110 systemd[1]: Starting Ceph mgr.z for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:41:38.296 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:38 smithi110 podman[36767]: 2024-05-09 04:41:37.932322993 +0000 UTC m=+0.017847216 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:41:38.296 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:38 smithi110 podman[36767]: 2024-05-09 04:41:38.031258572 +0000 UTC m=+0.116782831 container create e9f541a521cfda53d9d64308914d0e55dcccff919a5523fcba58072b63ac962d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T04:41:38.296 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:38 smithi110 podman[36767]: 2024-05-09 04:41:38.233596256 +0000 UTC m=+0.319120470 container init e9f541a521cfda53d9d64308914d0e55dcccff919a5523fcba58072b63ac962d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-09T04:41:38.296 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:38 smithi110 podman[36767]: 2024-05-09 04:41:38.23696427 +0000 UTC m=+0.322488491 container start e9f541a521cfda53d9d64308914d0e55dcccff919a5523fcba58072b63ac962d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T04:41:38.296 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:38 smithi110 ceph-mgr[36788]: -- 172.21.15.110:0/11754590 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a751d225a0 con 0x55a756aac480 2024-05-09T04:41:38.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:38 smithi110 ceph-mon[34904]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:38.556 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:38 smithi110 bash[36767]: e9f541a521cfda53d9d64308914d0e55dcccff919a5523fcba58072b63ac962d 2024-05-09T04:41:38.556 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:38 smithi110 systemd[1]: Started Ceph mgr.z for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:41:38.557 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:38.407+0000 7f0eb57dd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T04:41:38.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:38 smithi046 ceph-mon[30136]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:38.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:38 smithi072 ceph-mon[34088]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T04:41:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T04:41:38.821 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-05-09T04:41:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-05-09T04:41:38.821 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00836034 s, 1.3 GB/s 2024-05-09T04:41:38.821 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-05-09T04:41:39.022 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:38.602+0000 7f0eb57dd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:41:39.022 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:38.707+0000 7f0eb57dd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T04:41:39.022 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:39 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:39.022+0000 7f0eb57dd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T04:41:39.306 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:39 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:39.108+0000 7f0eb57dd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T04:41:39.432 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:39 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:39.432 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:39 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:39.432 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:39 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:39.432 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:39 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:41:39.432 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:39 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T04:41:39.432 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:39 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:41:39.432 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:39 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:39.432 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:39 smithi046 ceph-mon[30136]: Deploying daemon mgr.y on smithi072 2024-05-09T04:41:39.433 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_4 2024-05-09T04:41:39.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:39 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:39.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:39 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:39.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:39 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:39.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:39 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:41:39.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:39 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T04:41:39.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:39 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:41:39.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:39 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:39.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:39 smithi072 ceph-mon[34088]: Deploying daemon mgr.y on smithi072 2024-05-09T04:41:39.679 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:41:39.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:39 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:39.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:39 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:39.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:39 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:39.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:39 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:41:39.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:39 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-09T04:41:39.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:39 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:41:39.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:39 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:39.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:39 smithi110 ceph-mon[34904]: Deploying daemon mgr.y on smithi072 2024-05-09T04:41:40.428 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:40 smithi072 systemd[1]: Starting Ceph mgr.y for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:41:40.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:40 smithi110 ceph-mon[34904]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:40.556 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:40 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:40.293+0000 7f0eb57dd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T04:41:40.556 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:40 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:40.460+0000 7f0eb57dd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T04:41:40.727 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:40 smithi072 ceph-mon[34088]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:40.727 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:40 smithi072 podman[35454]: 2024-05-09 04:41:40.439558908 +0000 UTC m=+0.018493178 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:41:40.727 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:40 smithi072 podman[35454]: 2024-05-09 04:41:40.541323119 +0000 UTC m=+0.120257374 container create feb402c9a46772cd2eb68eff3aa711638840d0a71042bbee1cbd70e4d1913eb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T04:41:40.727 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:40 smithi072 podman[35454]: 2024-05-09 04:41:40.728420885 +0000 UTC m=+0.307355141 container init feb402c9a46772cd2eb68eff3aa711638840d0a71042bbee1cbd70e4d1913eb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-09T04:41:40.763 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:40 smithi046 ceph-mon[30136]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:40.914 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:40 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:40.627+0000 7f0eb57dd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T04:41:41.083 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:40 smithi072 podman[35454]: 2024-05-09 04:41:40.730981995 +0000 UTC m=+0.309916252 container start feb402c9a46772cd2eb68eff3aa711638840d0a71042bbee1cbd70e4d1913eb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True) 2024-05-09T04:41:41.084 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:40 smithi072 ceph-mgr[35473]: -- 172.21.15.72:0/268340443 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c704f305a0 con 0x55c709cba480 2024-05-09T04:41:41.084 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:40 smithi072 bash[35454]: feb402c9a46772cd2eb68eff3aa711638840d0a71042bbee1cbd70e4d1913eb8 2024-05-09T04:41:41.084 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:40 smithi072 systemd[1]: Started Ceph mgr.y for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:41:41.084 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:40 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:40.894+0000 7f87af53e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T04:41:41.084 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:41 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:41.084+0000 7f87af53e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:41:41.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.140+0000 7f8878dfa700 1 -- 172.21.15.46:0/2943766865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88741026a0 msgr2=0x7f8874102ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.140+0000 7f8878dfa700 1 --2- 172.21.15.46:0/2943766865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88741026a0 0x7f8874102ab0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8864009990 tx=0x7f8864009ca0 comp rx=0 tx=0).stop 2024-05-09T04:41:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.140+0000 7f8878dfa700 1 -- 172.21.15.46:0/2943766865 shutdown_connections 2024-05-09T04:41:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.140+0000 7f8878dfa700 1 --2- 172.21.15.46:0/2943766865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f887410a180 0x7f887410c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.140+0000 7f8878dfa700 1 --2- 172.21.15.46:0/2943766865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f88741026a0 0x7f8874102ab0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.140+0000 7f8878dfa700 1 --2- 172.21.15.46:0/2943766865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8874107820 0x7f8874109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.140+0000 7f8878dfa700 1 -- 172.21.15.46:0/2943766865 >> 172.21.15.46:0/2943766865 conn(0x7f88740fe020 msgr2=0x7f88741004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:41:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.140+0000 7f8878dfa700 1 -- 172.21.15.46:0/2943766865 shutdown_connections 2024-05-09T04:41:41.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.140+0000 7f8878dfa700 1 -- 172.21.15.46:0/2943766865 wait complete. 2024-05-09T04:41:41.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.141+0000 7f8878dfa700 1 Processor -- start 2024-05-09T04:41:41.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.141+0000 7f8878dfa700 1 -- start start 2024-05-09T04:41:41.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.141+0000 7f8878dfa700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88741026a0 0x7f887419a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:41.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.141+0000 7f8878dfa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8874107820 0x7f887419ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:41.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.141+0000 7f8878dfa700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f887410a180 0x7f88741a1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:41.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.141+0000 7f8878dfa700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f887410fdc0 con 0x7f8874107820 2024-05-09T04:41:41.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.141+0000 7f8878dfa700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f887410fc40 con 0x7f887410a180 2024-05-09T04:41:41.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.141+0000 7f8878dfa700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f887410ff40 con 0x7f88741026a0 2024-05-09T04:41:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f8871d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8874107820 0x7f887419ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f8871d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8874107820 0x7f887419ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48608/0 (socket says 172.21.15.46:48608) 2024-05-09T04:41:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f8871d9b700 1 -- 172.21.15.46:0/2493890059 learned_addr learned my addr 172.21.15.46:0/2493890059 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:41:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f887259c700 1 --2- 172.21.15.46:0/2493890059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88741026a0 0x7f887419a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f8872d9d700 1 --2- 172.21.15.46:0/2493890059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f887410a180 0x7f88741a1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f8871d9b700 1 -- 172.21.15.46:0/2493890059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88741026a0 msgr2=0x7f887419a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f8871d9b700 1 --2- 172.21.15.46:0/2493890059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88741026a0 0x7f887419a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f8871d9b700 1 -- 172.21.15.46:0/2493890059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f887410a180 msgr2=0x7f88741a1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:41:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f8871d9b700 1 --2- 172.21.15.46:0/2493890059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f887410a180 0x7f88741a1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:41:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f8871d9b700 1 -- 172.21.15.46:0/2493890059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88741a2470 con 0x7f8874107820 2024-05-09T04:41:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f8871d9b700 1 --2- 172.21.15.46:0/2493890059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8874107820 0x7f887419ae50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f886800e9c0 tx=0x7f886800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f88637fe700 1 -- 172.21.15.46:0/2493890059 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f886800cbd0 con 0x7f8874107820 2024-05-09T04:41:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f88637fe700 1 -- 172.21.15.46:0/2493890059 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f886800ee60 con 0x7f8874107820 2024-05-09T04:41:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f88637fe700 1 -- 172.21.15.46:0/2493890059 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88680055c0 con 0x7f8874107820 2024-05-09T04:41:41.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f8878dfa700 1 -- 172.21.15.46:0/2493890059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88741a2760 con 0x7f8874107820 2024-05-09T04:41:41.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.142+0000 7f8878dfa700 1 -- 172.21.15.46:0/2493890059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88741a2ca0 con 0x7f8874107820 2024-05-09T04:41:41.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.143+0000 7f88637fe700 1 -- 172.21.15.46:0/2493890059 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f8868021440 con 0x7f8874107820 2024-05-09T04:41:41.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.143+0000 7f8878dfa700 1 -- 172.21.15.46:0/2493890059 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88740512c0 con 0x7f8874107820 2024-05-09T04:41:41.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.144+0000 7f88637fe700 1 --2- 172.21.15.46:0/2493890059 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f885c03e750 0x7f885c040c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:41:41.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.144+0000 7f88637fe700 1 -- 172.21.15.46:0/2493890059 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8868052b80 con 0x7f8874107820 2024-05-09T04:41:41.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.144+0000 7f887259c700 1 --2- 172.21.15.46:0/2493890059 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f885c03e750 0x7f885c040c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:41:41.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.144+0000 7f887259c700 1 --2- 172.21.15.46:0/2493890059 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f885c03e750 0x7f885c040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8864006010 tx=0x7f886400b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:41:41.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.149+0000 7f88637fe700 1 -- 172.21.15.46:0/2493890059 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8868016590 con 0x7f8874107820 2024-05-09T04:41:41.216 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:40 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:40.914+0000 7f0eb57dd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T04:41:41.217 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:41 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:41.006+0000 7f0eb57dd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T04:41:41.217 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:41 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:41.216+0000 7f0eb57dd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T04:41:41.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:41.372+0000 7f8878dfa700 1 -- 172.21.15.46:0/2493890059 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8874063c00 con 0x7f885c03e750 2024-05-09T04:41:41.379 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:41 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:41.192+0000 7f87af53e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T04:41:41.556 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:41 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:41.414+0000 7f0eb57dd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:41:41.817 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:41 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:41.526+0000 7f87af53e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T04:41:41.818 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:41 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:41.622+0000 7f87af53e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T04:41:42.043 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:42 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:42.043+0000 7f0eb57dd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T04:41:42.044 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:41 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:42.044 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:41 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:42.044 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:41 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:42.044 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:41 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:42.044 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:41 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:42.044 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:41 smithi110 ceph-mon[34904]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:42.044 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:41 smithi110 ceph-mon[34904]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:41:42.044 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:41 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:41:42.044 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:41 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:41:42.044 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:41 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:42.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:41 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:42.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:41 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:42.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:41 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:42.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:41 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:42.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:41 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:42.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:41 smithi072 ceph-mon[34088]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:42.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:41 smithi072 ceph-mon[34088]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:41:42.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:41 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:41:42.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:41 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:41:42.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:41 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:42.198 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:41 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:42.198 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:41 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:42.198 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:41 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:42.198 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:41 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:42.198 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:41 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:42.199 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:41 smithi046 ceph-mon[30136]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:42.199 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:41 smithi046 ceph-mon[30136]: from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:41:42.199 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:41 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:41:42.199 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:41 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:41:42.199 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:41 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:42.306 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:42 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:42.275+0000 7f0eb57dd200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T04:41:43.056 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:42 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:42.695+0000 7f0eb57dd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T04:41:43.056 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:42 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:42.792+0000 7f0eb57dd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T04:41:43.129 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:42 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:42.853+0000 7f87af53e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T04:41:43.129 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:43 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:43.022+0000 7f87af53e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T04:41:43.478 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:43 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:43.191+0000 7f87af53e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T04:41:43.478 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.479 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.479 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.479 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.479 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.479 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:43.479 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:43.479 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.479 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:43 smithi072 ceph-mon[34088]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T04:41:43.479 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:41:43.479 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:41:43.479 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:43.479 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:43 smithi072 ceph-mon[34088]: Reconfiguring daemon mgr.x on smithi046 2024-05-09T04:41:43.510 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:43 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:43.157+0000 7f0eb57dd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T04:41:43.510 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:43 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:43.510+0000 7f0eb57dd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T04:41:43.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:43.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:43.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:43 smithi110 ceph-mon[34904]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T04:41:43.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:41:43.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:41:43.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:43.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:43 smithi110 ceph-mon[34904]: Reconfiguring daemon mgr.x on smithi046 2024-05-09T04:41:43.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:43.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:43.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:43.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:43 smithi046 ceph-mon[30136]: Reconfiguring mgr.x (unknown last config time)... 2024-05-09T04:41:43.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-09T04:41:43.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-09T04:41:43.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:43.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:43 smithi046 ceph-mon[30136]: Reconfiguring daemon mgr.x on smithi046 2024-05-09T04:41:43.771 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:43 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:43.478+0000 7f87af53e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T04:41:43.772 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:43 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:43.570+0000 7f87af53e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T04:41:43.772 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:43 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:43.770+0000 7f87af53e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T04:41:43.806 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:43 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:43.602+0000 7f0eb57dd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T04:41:44.129 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:43 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:43.962+0000 7f87af53e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:41:44.306 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:43 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:43.845+0000 7f0eb57dd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T04:41:44.306 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:43 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:43.976+0000 7f0eb57dd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T04:41:44.306 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:44 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:44.070+0000 7f0eb57dd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T04:41:44.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:44 smithi046 ceph-mon[30136]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:44.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:44 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1778744318' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d79cd3e7-6f82-4d60-96d9-26d8146c962f"}]: dispatch 2024-05-09T04:41:44.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:44 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1778744318' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d79cd3e7-6f82-4d60-96d9-26d8146c962f"}]': finished 2024-05-09T04:41:44.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:44 smithi046 ceph-mon[30136]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T04:41:44.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:44 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:41:44.629 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:44 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:44.564+0000 7f87af53e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T04:41:44.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:44 smithi072 ceph-mon[34088]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:44.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:44 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1778744318' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d79cd3e7-6f82-4d60-96d9-26d8146c962f"}]: dispatch 2024-05-09T04:41:44.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:44 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1778744318' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d79cd3e7-6f82-4d60-96d9-26d8146c962f"}]': finished 2024-05-09T04:41:44.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:44 smithi072 ceph-mon[34088]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T04:41:44.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:44 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:41:44.806 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:44 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:44.394+0000 7f0eb57dd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T04:41:44.806 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:44 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:44.486+0000 7f0eb57dd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T04:41:44.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:44 smithi110 ceph-mon[34904]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:44.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:44 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1778744318' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d79cd3e7-6f82-4d60-96d9-26d8146c962f"}]: dispatch 2024-05-09T04:41:44.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:44 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1778744318' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d79cd3e7-6f82-4d60-96d9-26d8146c962f"}]': finished 2024-05-09T04:41:44.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:44 smithi110 ceph-mon[34904]: osdmap e5: 1 total, 0 up, 1 in 2024-05-09T04:41:44.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:44 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:41:45.129 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:44 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:44.782+0000 7f87af53e200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T04:41:45.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:45.324+0000 7f88637fe700 1 -- 172.21.15.46:0/2493890059 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f8868014230 con 0x7f8874107820 2024-05-09T04:41:45.556 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:45 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:45.084+0000 7f0eb57dd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T04:41:45.557 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:41:45 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:41:45.298+0000 7f0eb57dd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T04:41:45.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:45 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/986205951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T04:41:45.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:45 smithi110 ceph-mon[34904]: Standby manager daemon z started 2024-05-09T04:41:45.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:45 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.110:0/4265318933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T04:41:45.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:45 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.110:0/4265318933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:41:45.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:45 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.110:0/4265318933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T04:41:45.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:45 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.110:0/4265318933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:41:45.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:45 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/986205951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T04:41:45.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:45 smithi046 ceph-mon[30136]: Standby manager daemon z started 2024-05-09T04:41:45.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:45 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.110:0/4265318933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T04:41:45.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:45 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.110:0/4265318933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:41:45.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:45 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.110:0/4265318933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T04:41:45.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:45 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.110:0/4265318933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:41:45.629 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:45 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:45.202+0000 7f87af53e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T04:41:45.629 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:45 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:45.298+0000 7f87af53e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T04:41:45.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:45 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/986205951' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-0/activate.monmap"}]: dispatch 2024-05-09T04:41:45.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:45 smithi072 ceph-mon[34088]: Standby manager daemon z started 2024-05-09T04:41:45.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:45 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.110:0/4265318933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T04:41:45.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:45 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.110:0/4265318933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:41:45.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:45 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.110:0/4265318933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T04:41:45.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:45 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.110:0/4265318933' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:41:46.018 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:45 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:45.665+0000 7f87af53e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T04:41:46.322 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:46 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:46.018+0000 7f87af53e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T04:41:46.322 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:46 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:46.111+0000 7f87af53e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T04:41:46.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:46 smithi046 ceph-mon[30136]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:46.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:46 smithi046 ceph-mon[30136]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-09T04:41:46.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:46 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T04:41:46.580 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:46 smithi072 ceph-mon[34088]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:46.580 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:46 smithi072 ceph-mon[34088]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-09T04:41:46.580 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:46 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T04:41:46.580 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:46 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:46.354+0000 7f87af53e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T04:41:46.580 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:46 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:46.486+0000 7f87af53e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T04:41:46.580 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:46 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:46.580+0000 7f87af53e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T04:41:46.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:46 smithi110 ceph-mon[34904]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:46.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:46 smithi110 ceph-mon[34904]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-09T04:41:46.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:46 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T04:41:47.379 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:46 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:46.902+0000 7f87af53e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T04:41:47.379 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:46 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:46.993+0000 7f87af53e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T04:41:47.879 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:47 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:47.588+0000 7f87af53e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T04:41:47.879 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:41:47 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:41:47.803+0000 7f87af53e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T04:41:48.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:41:48.333+0000 7f88637fe700 1 -- 172.21.15.46:0/2493890059 <== mon.0 v2:172.21.15.46:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f8868066a60 con 0x7f8874107820 2024-05-09T04:41:48.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:48 smithi072 ceph-mon[34088]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:48.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:48 smithi072 ceph-mon[34088]: Standby manager daemon y started 2024-05-09T04:41:48.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:48 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.72:0/815533352' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T04:41:48.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:48 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.72:0/815533352' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:41:48.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:48 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.72:0/815533352' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T04:41:48.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:48 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.72:0/815533352' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:41:48.676 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:48 smithi046 ceph-mon[30136]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:48.676 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:48 smithi046 ceph-mon[30136]: Standby manager daemon y started 2024-05-09T04:41:48.676 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:48 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.72:0/815533352' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T04:41:48.676 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:48 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.72:0/815533352' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:41:48.676 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:48 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.72:0/815533352' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T04:41:48.676 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:48 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.72:0/815533352' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:41:48.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:48 smithi110 ceph-mon[34904]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:48 smithi110 ceph-mon[34904]: Standby manager daemon y started 2024-05-09T04:41:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:48 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.72:0/815533352' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T04:41:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:48 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.72:0/815533352' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:41:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:48 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.72:0/815533352' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T04:41:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:48 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.72:0/815533352' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:41:49.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:49 smithi046 ceph-mon[30136]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-09T04:41:49.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:49 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T04:41:49.626 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:49 smithi110 ceph-mon[34904]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-09T04:41:49.654 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:49 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T04:41:49.655 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:49 smithi072 ceph-mon[34088]: mgrmap e15: x(active, since 81s), standbys: z, y 2024-05-09T04:41:49.655 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:49 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T04:41:50.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:50 smithi046 ceph-mon[30136]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:50.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:50.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:50.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:50.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:50.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:50.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:50.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:50 smithi072 ceph-mon[34088]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:50.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:50.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:50.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:50.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:50.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:50.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:50 smithi110 ceph-mon[34904]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:50.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:50.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:50.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:50.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:50.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:41:50.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:52.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:52 smithi046 ceph-mon[30136]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:52.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:52 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T04:41:52.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:52 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:52.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:52 smithi072 ceph-mon[34088]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:52.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:52 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T04:41:52.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:52 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:52.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:52 smithi110 ceph-mon[34904]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:52.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:52 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-09T04:41:52.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:52 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:41:53.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:53 smithi046 ceph-mon[30136]: Deploying daemon osd.0 on smithi046 2024-05-09T04:41:53.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:53 smithi072 ceph-mon[34088]: Deploying daemon osd.0 on smithi046 2024-05-09T04:41:53.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:53 smithi110 ceph-mon[34904]: Deploying daemon osd.0 on smithi046 2024-05-09T04:41:54.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:54 smithi072 ceph-mon[34088]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:54.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:54 smithi110 ceph-mon[34904]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:54.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:54 smithi046 ceph-mon[30136]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:56.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:56 smithi072 ceph-mon[34088]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:56.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:56 smithi110 ceph-mon[34904]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:56.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:56 smithi046 ceph-mon[30136]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:58.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:58 smithi046 ceph-mon[30136]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:58.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:58 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:58.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:58 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:58.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:41:58 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:59.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:58 smithi110 ceph-mon[34904]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:59.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:58 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:59.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:58 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:59.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:41:58 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:41:59.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:58 smithi072 ceph-mon[34088]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:41:59.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:58 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:59.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:58 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:41:59.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:41:58 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:00.986 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:00 smithi046 ceph-mon[30136]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:01.039 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 0 on host 'smithi046' 2024-05-09T04:42:01.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:01.039+0000 7f88637fe700 1 -- 172.21.15.46:0/2493890059 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8874063c00 con 0x7f885c03e750 2024-05-09T04:42:01.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:01.042+0000 7f8878dfa700 1 -- 172.21.15.46:0/2493890059 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f885c03e750 msgr2=0x7f885c040c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:01.042+0000 7f8878dfa700 1 --2- 172.21.15.46:0/2493890059 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f885c03e750 0x7f885c040c60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8864006010 tx=0x7f886400b720 comp rx=0 tx=0).stop 2024-05-09T04:42:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:01.042+0000 7f8878dfa700 1 -- 172.21.15.46:0/2493890059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8874107820 msgr2=0x7f887419ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:01.042+0000 7f8878dfa700 1 --2- 172.21.15.46:0/2493890059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8874107820 0x7f887419ae50 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f886800e9c0 tx=0x7f886800ecd0 comp rx=0 tx=0).stop 2024-05-09T04:42:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:01.042+0000 7f8878dfa700 1 -- 172.21.15.46:0/2493890059 shutdown_connections 2024-05-09T04:42:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:01.042+0000 7f8878dfa700 1 --2- 172.21.15.46:0/2493890059 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f885c03e750 0x7f885c040c60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:01.042+0000 7f8878dfa700 1 --2- 172.21.15.46:0/2493890059 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f887410a180 0x7f88741a1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:01.042+0000 7f8878dfa700 1 --2- 172.21.15.46:0/2493890059 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f88741026a0 0x7f887419a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:01.042+0000 7f8878dfa700 1 --2- 172.21.15.46:0/2493890059 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8874107820 0x7f887419ae50 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:01.042+0000 7f8878dfa700 1 -- 172.21.15.46:0/2493890059 >> 172.21.15.46:0/2493890059 conn(0x7f88740fe020 msgr2=0x7f8874108760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:42:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:01.042+0000 7f8878dfa700 1 -- 172.21.15.46:0/2493890059 shutdown_connections 2024-05-09T04:42:01.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:01.042+0000 7f8878dfa700 1 -- 172.21.15.46:0/2493890059 wait complete. 2024-05-09T04:42:01.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:00 smithi110 ceph-mon[34904]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:01.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:00 smithi072 ceph-mon[34088]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:01.566 DEBUG:teuthology.orchestra.run.smithi046:osd.0> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.0.service 2024-05-09T04:42:01.569 INFO:tasks.cephadm:Deploying osd.1 on smithi072 with /dev/vg_nvme/lv_4... 2024-05-09T04:42:01.569 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T04:42:01.816 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:42:01.829 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 04:42:01 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-0[45186]: 2024-05-09T04:42:01.641+0000 7f762eed17c0 -1 osd.0 0 log_to_monitors true 2024-05-09T04:42:02.244 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:02.244 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:02.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:02 smithi072 ceph-mon[34088]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:02.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:02 smithi072 ceph-mon[34088]: from='osd.0 [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T04:42:02.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:02.245 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:02.290 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:02.290 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:02.290 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:02 smithi046 ceph-mon[30136]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:02.290 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:02 smithi046 ceph-mon[30136]: from='osd.0 [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T04:42:02.290 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:02.290 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:02.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:02.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:02.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:02 smithi110 ceph-mon[34904]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:02.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:02 smithi110 ceph-mon[34904]: from='osd.0 [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-09T04:42:02.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:02.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:03.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:03 smithi072 ceph-mon[34088]: from='osd.0 [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T04:42:03.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:03 smithi072 ceph-mon[34088]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T04:42:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:03 smithi072 ceph-mon[34088]: from='osd.0 [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T04:42:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:03 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:03.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:03 smithi110 ceph-mon[34904]: from='osd.0 [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T04:42:03.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:03 smithi110 ceph-mon[34904]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T04:42:03.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:03 smithi110 ceph-mon[34904]: from='osd.0 [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T04:42:03.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:03 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:03.327 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 04:42:03 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-0[45186]: 2024-05-09T04:42:03.044+0000 7f7624535700 -1 osd.0 0 waiting for initial osdmap 2024-05-09T04:42:03.327 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 04:42:03 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-0[45186]: 2024-05-09T04:42:03.051+0000 7f76202ea700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:42:03.328 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:03 smithi046 ceph-mon[30136]: from='osd.0 [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-09T04:42:03.328 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:03 smithi046 ceph-mon[30136]: osdmap e6: 1 total, 0 up, 1 in 2024-05-09T04:42:03.328 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:03 smithi046 ceph-mon[30136]: from='osd.0 [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T04:42:03.328 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:03 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:03.681 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T04:42:03.681 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T04:42:03.681 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-05-09T04:42:03.681 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-05-09T04:42:03.681 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0133353 s, 786 MB/s 2024-05-09T04:42:03.681 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-05-09T04:42:04.266 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:04 smithi072 ceph-mon[34088]: from='osd.0 [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T04:42:04.266 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:04 smithi072 ceph-mon[34088]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T04:42:04.266 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:04 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:04.266 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:04 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:04.266 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:04 smithi072 ceph-mon[34088]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:04.294 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_4 2024-05-09T04:42:04.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:04 smithi110 ceph-mon[34904]: from='osd.0 [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T04:42:04.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:04 smithi110 ceph-mon[34904]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T04:42:04.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:04 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:04.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:04 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:04.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:04 smithi110 ceph-mon[34904]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:04.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:04 smithi046 ceph-mon[30136]: from='osd.0 [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T04:42:04.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:04 smithi046 ceph-mon[30136]: osdmap e7: 1 total, 0 up, 1 in 2024-05-09T04:42:04.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:04 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:04.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:04 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:04.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:04 smithi046 ceph-mon[30136]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-09T04:42:04.536 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:42:05.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:05 smithi072 ceph-mon[34088]: purged_snaps scrub starts 2024-05-09T04:42:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:05 smithi072 ceph-mon[34088]: purged_snaps scrub ok 2024-05-09T04:42:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:05 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:05 smithi072 ceph-mon[34088]: osd.0 [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508] boot 2024-05-09T04:42:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:05 smithi072 ceph-mon[34088]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T04:42:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:05 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:05 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:05 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:05.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:05 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:42:05.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:05 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:05.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:05 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:05.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:05 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:05.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:05 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:05.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:05 smithi110 ceph-mon[34904]: purged_snaps scrub starts 2024-05-09T04:42:05.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:05 smithi110 ceph-mon[34904]: purged_snaps scrub ok 2024-05-09T04:42:05.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:05 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:05.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:05 smithi110 ceph-mon[34904]: osd.0 [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508] boot 2024-05-09T04:42:05.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:05 smithi110 ceph-mon[34904]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T04:42:05.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:05 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:05.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:05 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:05.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:05 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:05.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:05 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:42:05.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:05 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:05.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:05 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:05.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:05 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:05.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:05 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:05.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:05 smithi046 ceph-mon[30136]: purged_snaps scrub starts 2024-05-09T04:42:05.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:05 smithi046 ceph-mon[30136]: purged_snaps scrub ok 2024-05-09T04:42:05.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:05.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:05 smithi046 ceph-mon[30136]: osd.0 [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508] boot 2024-05-09T04:42:05.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:05 smithi046 ceph-mon[30136]: osdmap e8: 1 total, 1 up, 1 in 2024-05-09T04:42:05.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:42:05.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:05.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:05.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:42:05.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:05.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:05.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:05.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:06.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.050+0000 7fa0868d4700 1 -- 172.21.15.72:0/635921302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa080102350 msgr2=0x7fa080102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:06.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.050+0000 7fa0868d4700 1 --2- 172.21.15.72:0/635921302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa080102350 0x7fa080102760 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa074009990 tx=0x7fa074009ca0 comp rx=0 tx=0).stop 2024-05-09T04:42:06.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.050+0000 7fa0868d4700 1 -- 172.21.15.72:0/635921302 shutdown_connections 2024-05-09T04:42:06.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.050+0000 7fa0868d4700 1 --2- 172.21.15.72:0/635921302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa080102ca0 0x7fa08010c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:06.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.050+0000 7fa0868d4700 1 --2- 172.21.15.72:0/635921302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa080105900 0x7fa080105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:06.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.050+0000 7fa0868d4700 1 --2- 172.21.15.72:0/635921302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa080102350 0x7fa080102760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:06.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.050+0000 7fa0868d4700 1 -- 172.21.15.72:0/635921302 >> 172.21.15.72:0/635921302 conn(0x7fa0800fe020 msgr2=0x7fa0801004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:42:06.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.050+0000 7fa0868d4700 1 -- 172.21.15.72:0/635921302 shutdown_connections 2024-05-09T04:42:06.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.050+0000 7fa0868d4700 1 -- 172.21.15.72:0/635921302 wait complete. 2024-05-09T04:42:06.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.050+0000 7fa0868d4700 1 Processor -- start 2024-05-09T04:42:06.052 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.050+0000 7fa0868d4700 1 -- start start 2024-05-09T04:42:06.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa0868d4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa080102350 0x7fa080104ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:06.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa0868d4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa080102ca0 0x7fa080103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:06.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa0868d4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa080105900 0x7fa080103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:06.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa0868d4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa08010fdc0 con 0x7fa080105900 2024-05-09T04:42:06.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa0868d4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa08010fc40 con 0x7fa080102ca0 2024-05-09T04:42:06.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa0868d4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa08010ff40 con 0x7fa080102350 2024-05-09T04:42:06.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa07ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa080102350 0x7fa080104ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:06.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa07f7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa080102ca0 0x7fa080103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:06.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa07ffff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa080102350 0x7fa080104ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.72:37300/0 (socket says 172.21.15.72:37300) 2024-05-09T04:42:06.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa07ffff700 1 -- 172.21.15.72:0/1524542704 learned_addr learned my addr 172.21.15.72:0/1524542704 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-05-09T04:42:06.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa084e71700 1 --2- 172.21.15.72:0/1524542704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa080105900 0x7fa080103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:06.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa07ffff700 1 -- 172.21.15.72:0/1524542704 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa080102ca0 msgr2=0x7fa080103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:06.053 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa07ffff700 1 --2- 172.21.15.72:0/1524542704 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa080102ca0 0x7fa080103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:06.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa07ffff700 1 -- 172.21.15.72:0/1524542704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa080105900 msgr2=0x7fa080103ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:06.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa07ffff700 1 --2- 172.21.15.72:0/1524542704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa080105900 0x7fa080103ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:06.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa07ffff700 1 -- 172.21.15.72:0/1524542704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa080104190 con 0x7fa080102350 2024-05-09T04:42:06.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa084e71700 1 --2- 172.21.15.72:0/1524542704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa080105900 0x7fa080103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:42:06.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.051+0000 7fa07f7fe700 1 --2- 172.21.15.72:0/1524542704 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa080102ca0 0x7fa080103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:42:06.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.052+0000 7fa07ffff700 1 --2- 172.21.15.72:0/1524542704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa080102350 0x7fa080104ed0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa07000e940 tx=0x7fa07000ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:06.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.052+0000 7fa07d7fa700 1 -- 172.21.15.72:0/1524542704 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa07000cbd0 con 0x7fa080102350 2024-05-09T04:42:06.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.052+0000 7fa07d7fa700 1 -- 172.21.15.72:0/1524542704 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa07000ee90 con 0x7fa080102350 2024-05-09T04:42:06.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.052+0000 7fa0868d4700 1 -- 172.21.15.72:0/1524542704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0801a49d0 con 0x7fa080102350 2024-05-09T04:42:06.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.052+0000 7fa07d7fa700 1 -- 172.21.15.72:0/1524542704 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa070003dc0 con 0x7fa080102350 2024-05-09T04:42:06.054 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.052+0000 7fa0868d4700 1 -- 172.21.15.72:0/1524542704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0801a4e80 con 0x7fa080102350 2024-05-09T04:42:06.055 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.053+0000 7fa0868d4700 1 -- 172.21.15.72:0/1524542704 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0800512c0 con 0x7fa080102350 2024-05-09T04:42:06.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.054+0000 7fa07d7fa700 1 -- 172.21.15.72:0/1524542704 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fa0700468d0 con 0x7fa080102350 2024-05-09T04:42:06.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.055+0000 7fa07d7fa700 1 --2- 172.21.15.72:0/1524542704 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa0680b2870 0x7fa0680b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:06.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.055+0000 7fa07d7fa700 1 -- 172.21.15.72:0/1524542704 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fa0700e09a0 con 0x7fa080102350 2024-05-09T04:42:06.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.055+0000 7fa07f7fe700 1 --2- 172.21.15.72:0/1524542704 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa0680b2870 0x7fa0680b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:06.056 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.056+0000 7fa07f7fe700 1 --2- 172.21.15.72:0/1524542704 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa0680b2870 0x7fa0680b4d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa080104880 tx=0x7fa07400b720 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:06.058 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.059+0000 7fa07d7fa700 1 -- 172.21.15.72:0/1524542704 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa0700aa090 con 0x7fa080102350 2024-05-09T04:42:06.275 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:06.276+0000 7fa0868d4700 1 -- 172.21.15.72:0/1524542704 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fa080063c00 con 0x7fa0680b2870 2024-05-09T04:42:06.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:06 smithi046 ceph-mon[30136]: Detected new or changed devices on smithi046 2024-05-09T04:42:06.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:06 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi046 to 17093M 2024-05-09T04:42:06.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:06 smithi046 ceph-mon[30136]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T04:42:06.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:06 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:06.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:06 smithi046 ceph-mon[30136]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:06.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:06 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:06.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:06 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:06.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:06 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:06.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:06 smithi072 ceph-mon[34088]: Detected new or changed devices on smithi046 2024-05-09T04:42:06.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:06 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi046 to 17093M 2024-05-09T04:42:06.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:06 smithi072 ceph-mon[34088]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T04:42:06.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:06 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:06.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:06 smithi072 ceph-mon[34088]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:06.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:06 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:06.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:06 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:06.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:06 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:06.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:06 smithi110 ceph-mon[34904]: Detected new or changed devices on smithi046 2024-05-09T04:42:06.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:06 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi046 to 17093M 2024-05-09T04:42:06.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:06 smithi110 ceph-mon[34904]: osdmap e9: 1 total, 1 up, 1 in 2024-05-09T04:42:06.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:06 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:06.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:06 smithi110 ceph-mon[34904]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:06.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:06 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:06.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:06 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:06.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:06 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:07.186 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:07 smithi072 ceph-mon[34088]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:07.190 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:07 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:42:07.190 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:07 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:42:07.190 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:07 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:07.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:07 smithi046 ceph-mon[30136]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:07.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:07 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:42:07.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:07 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:42:07.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:07 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:07.458 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:07 smithi110 ceph-mon[34904]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:07.458 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:07 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:42:07.458 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:07 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:42:07.458 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:07 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:08.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:08 smithi046 ceph-mon[30136]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:08.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:08 smithi072 ceph-mon[34088]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:08.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:08 smithi110 ceph-mon[34904]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:09.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:09 smithi046 ceph-mon[30136]: from='client.? 172.21.15.72:0/153251047' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "698eba80-1ebd-4076-b6f9-953d2e7daf76"}]: dispatch 2024-05-09T04:42:09.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:09 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "698eba80-1ebd-4076-b6f9-953d2e7daf76"}]: dispatch 2024-05-09T04:42:09.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:09 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "698eba80-1ebd-4076-b6f9-953d2e7daf76"}]': finished 2024-05-09T04:42:09.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:09 smithi046 ceph-mon[30136]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T04:42:09.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:09 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:09.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:09 smithi072 ceph-mon[34088]: from='client.? 172.21.15.72:0/153251047' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "698eba80-1ebd-4076-b6f9-953d2e7daf76"}]: dispatch 2024-05-09T04:42:09.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:09 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "698eba80-1ebd-4076-b6f9-953d2e7daf76"}]: dispatch 2024-05-09T04:42:09.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:09 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "698eba80-1ebd-4076-b6f9-953d2e7daf76"}]': finished 2024-05-09T04:42:09.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:09 smithi072 ceph-mon[34088]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T04:42:09.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:09 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:09.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:09 smithi110 ceph-mon[34904]: from='client.? 172.21.15.72:0/153251047' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "698eba80-1ebd-4076-b6f9-953d2e7daf76"}]: dispatch 2024-05-09T04:42:09.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:09 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "698eba80-1ebd-4076-b6f9-953d2e7daf76"}]: dispatch 2024-05-09T04:42:09.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:09 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "698eba80-1ebd-4076-b6f9-953d2e7daf76"}]': finished 2024-05-09T04:42:09.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:09 smithi110 ceph-mon[34904]: osdmap e10: 2 total, 1 up, 2 in 2024-05-09T04:42:09.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:09 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:10.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:10 smithi046 ceph-mon[30136]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:10.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:10 smithi046 ceph-mon[30136]: from='client.? 172.21.15.72:0/2648943687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T04:42:10.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:10 smithi072 ceph-mon[34088]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:10.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:10 smithi072 ceph-mon[34088]: from='client.? 172.21.15.72:0/2648943687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T04:42:10.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:10 smithi110 ceph-mon[34904]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:10.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:10 smithi110 ceph-mon[34904]: from='client.? 172.21.15.72:0/2648943687' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-1/activate.monmap"}]: dispatch 2024-05-09T04:42:12.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:12 smithi110 ceph-mon[34904]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:12.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:12 smithi046 ceph-mon[30136]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:12.605 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:12 smithi072 ceph-mon[34088]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:14.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:14 smithi072 ceph-mon[34088]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:14.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:14 smithi110 ceph-mon[34904]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:14.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:14 smithi046 ceph-mon[30136]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:16.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:16 smithi110 ceph-mon[34904]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:16.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:16 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T04:42:16.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:16 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:16.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:16 smithi046 ceph-mon[30136]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:16.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:16 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T04:42:16.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:16 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:16.610 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:16 smithi072 ceph-mon[34088]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:16.610 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:16 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-09T04:42:16.610 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:16 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:17.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:17 smithi110 ceph-mon[34904]: Deploying daemon osd.1 on smithi072 2024-05-09T04:42:17.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:17 smithi046 ceph-mon[30136]: Deploying daemon osd.1 on smithi072 2024-05-09T04:42:17.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:17 smithi072 ceph-mon[34088]: Deploying daemon osd.1 on smithi072 2024-05-09T04:42:18.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:18 smithi110 ceph-mon[34904]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:18.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:18 smithi046 ceph-mon[30136]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:18.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:18 smithi072 ceph-mon[34088]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:20.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:20 smithi110 ceph-mon[34904]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:20.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:20 smithi046 ceph-mon[30136]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:20.600 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:20 smithi072 ceph-mon[34088]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:22.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:22 smithi072 ceph-mon[34088]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:22.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:22 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:22.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:22 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:22.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:22 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:22.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:22 smithi110 ceph-mon[34904]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:22.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:22 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:22.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:22 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:22.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:22 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:22.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:22 smithi046 ceph-mon[30136]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:22.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:22 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:22.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:22 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:22.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:22 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:24.778 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:24 smithi072 ceph-mon[34088]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:24.778 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:24 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:24.778 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:24 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:24.778 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:24 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:24.779 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:24 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:24.779 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:24 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:25.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:24 smithi110 ceph-mon[34904]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:25.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:24 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:25.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:24 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:25.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:24 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:25.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:24 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:25.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:24 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:25.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:24 smithi046 ceph-mon[30136]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:25.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:24 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:25.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:24 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:25.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:24 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:25.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:24 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:25.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:24 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:25.145 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:25.144+0000 7fa07d7fa700 1 -- 172.21.15.72:0/1524542704 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa080063c00 con 0x7fa0680b2870 2024-05-09T04:42:25.145 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 1 on host 'smithi072' 2024-05-09T04:42:25.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:25.147+0000 7fa0868d4700 1 -- 172.21.15.72:0/1524542704 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa0680b2870 msgr2=0x7fa0680b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:25.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:25.147+0000 7fa0868d4700 1 --2- 172.21.15.72:0/1524542704 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa0680b2870 0x7fa0680b4d80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa080104880 tx=0x7fa07400b720 comp rx=0 tx=0).stop 2024-05-09T04:42:25.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:25.147+0000 7fa0868d4700 1 -- 172.21.15.72:0/1524542704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa080102350 msgr2=0x7fa080104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:25.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:25.147+0000 7fa0868d4700 1 --2- 172.21.15.72:0/1524542704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa080102350 0x7fa080104ed0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa07000e940 tx=0x7fa07000ec50 comp rx=0 tx=0).stop 2024-05-09T04:42:25.147 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:25.147+0000 7fa0868d4700 1 -- 172.21.15.72:0/1524542704 shutdown_connections 2024-05-09T04:42:25.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:25.147+0000 7fa0868d4700 1 --2- 172.21.15.72:0/1524542704 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa0680b2870 0x7fa0680b4d80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:25.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:25.147+0000 7fa0868d4700 1 --2- 172.21.15.72:0/1524542704 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa080105900 0x7fa080103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:25.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:25.147+0000 7fa0868d4700 1 --2- 172.21.15.72:0/1524542704 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa080102350 0x7fa080104ed0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:25.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:25.147+0000 7fa0868d4700 1 --2- 172.21.15.72:0/1524542704 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa080102ca0 0x7fa080103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:25.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:25.147+0000 7fa0868d4700 1 -- 172.21.15.72:0/1524542704 >> 172.21.15.72:0/1524542704 conn(0x7fa0800fe020 msgr2=0x7fa0801088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:42:25.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:25.147+0000 7fa0868d4700 1 -- 172.21.15.72:0/1524542704 shutdown_connections 2024-05-09T04:42:25.148 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:42:25.147+0000 7fa0868d4700 1 -- 172.21.15.72:0/1524542704 wait complete. 2024-05-09T04:42:25.780 DEBUG:teuthology.orchestra.run.smithi072:osd.1> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.1.service 2024-05-09T04:42:25.783 INFO:tasks.cephadm:Deploying osd.2 on smithi110 with /dev/vg_nvme/lv_4... 2024-05-09T04:42:25.783 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-09T04:42:26.030 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:42:26.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:26 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:26.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:26 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:26.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:26 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:26.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:26 smithi072 ceph-mon[34088]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:26.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:26 smithi072 ceph-mon[34088]: from='osd.1 [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T04:42:26.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:26 smithi072 ceph-mon[34088]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T04:42:26.411 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:26.411 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:26.411 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:26.411 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:26 smithi110 ceph-mon[34904]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:26.411 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:26 smithi110 ceph-mon[34904]: from='osd.1 [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T04:42:26.411 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:26 smithi110 ceph-mon[34904]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T04:42:26.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:26.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:26.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:26.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:26 smithi046 ceph-mon[30136]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:26.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:26 smithi046 ceph-mon[30136]: from='osd.1 [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T04:42:26.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:26 smithi046 ceph-mon[30136]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-09T04:42:27.203 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:27 smithi072 ceph-mon[34088]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T04:42:27.204 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:27 smithi072 ceph-mon[34088]: from='osd.1 [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:42:27.204 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:27 smithi072 ceph-mon[34088]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T04:42:27.204 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:27.204 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:27 smithi072 ceph-mon[34088]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:42:27.367 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:27 smithi110 ceph-mon[34904]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T04:42:27.367 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:27 smithi110 ceph-mon[34904]: from='osd.1 [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:42:27.367 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:27 smithi110 ceph-mon[34904]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T04:42:27.367 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:27.367 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:27 smithi110 ceph-mon[34904]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:42:27.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:27 smithi046 ceph-mon[30136]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-09T04:42:27.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:27 smithi046 ceph-mon[30136]: from='osd.1 [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:42:27.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:27 smithi046 ceph-mon[30136]: osdmap e11: 2 total, 1 up, 2 in 2024-05-09T04:42:27.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:27.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:27 smithi046 ceph-mon[30136]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:42:27.689 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-09T04:42:27.689 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-09T04:42:27.689 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-05-09T04:42:27.689 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-05-09T04:42:27.689 INFO:teuthology.orchestra.run.smithi110.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00858835 s, 1.2 GB/s 2024-05-09T04:42:27.690 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 2024-05-09T04:42:27.690 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-05-09T04:42:27.879 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 04:42:27 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-1[38706]: 2024-05-09T04:42:27.594+0000 7f26fb6be700 -1 osd.1 0 waiting for initial osdmap 2024-05-09T04:42:27.880 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 04:42:27 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-1[38706]: 2024-05-09T04:42:27.598+0000 7f26f346b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:42:28.271 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_4 2024-05-09T04:42:28.375 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:28 smithi072 ceph-mon[34088]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-09T04:42:28.376 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:28 smithi072 ceph-mon[34088]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T04:42:28.376 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:28.376 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:28.376 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:28 smithi072 ceph-mon[34088]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:28.376 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:28 smithi072 ceph-mon[34088]: from='osd.1 ' entity='osd.1' 2024-05-09T04:42:28.510 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:42:28.549 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:28 smithi110 ceph-mon[34904]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-09T04:42:28.549 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:28 smithi110 ceph-mon[34904]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T04:42:28.549 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:28.549 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:28.549 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:28 smithi110 ceph-mon[34904]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:28.549 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:28 smithi110 ceph-mon[34904]: from='osd.1 ' entity='osd.1' 2024-05-09T04:42:28.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:28 smithi046 ceph-mon[30136]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-09T04:42:28.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:28 smithi046 ceph-mon[30136]: osdmap e12: 2 total, 1 up, 2 in 2024-05-09T04:42:28.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:28.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:28.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:28 smithi046 ceph-mon[30136]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-09T04:42:28.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:28 smithi046 ceph-mon[30136]: from='osd.1 ' entity='osd.1' 2024-05-09T04:42:29.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:29 smithi072 ceph-mon[34088]: purged_snaps scrub starts 2024-05-09T04:42:29.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:29 smithi072 ceph-mon[34088]: purged_snaps scrub ok 2024-05-09T04:42:29.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:29.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:29 smithi072 ceph-mon[34088]: osd.1 [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775] boot 2024-05-09T04:42:29.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:29 smithi072 ceph-mon[34088]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T04:42:29.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:29.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:29.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:29.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:42:29.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:29.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:29.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:29.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:29.491 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:29 smithi110 ceph-mon[34904]: purged_snaps scrub starts 2024-05-09T04:42:29.492 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:29 smithi110 ceph-mon[34904]: purged_snaps scrub ok 2024-05-09T04:42:29.492 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:29.492 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:29 smithi110 ceph-mon[34904]: osd.1 [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775] boot 2024-05-09T04:42:29.492 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:29 smithi110 ceph-mon[34904]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T04:42:29.492 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:29.492 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:29.492 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:29.492 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:42:29.492 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:29.492 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:29.492 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:29.492 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:29.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:29 smithi046 ceph-mon[30136]: purged_snaps scrub starts 2024-05-09T04:42:29.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:29 smithi046 ceph-mon[30136]: purged_snaps scrub ok 2024-05-09T04:42:29.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:29.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:29 smithi046 ceph-mon[30136]: osd.1 [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775] boot 2024-05-09T04:42:29.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:29 smithi046 ceph-mon[30136]: osdmap e13: 2 total, 2 up, 2 in 2024-05-09T04:42:29.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:42:29.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:29.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:29.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:42:29.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:29.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:29.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:29.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:30.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.019+0000 7f5ac59d9700 1 -- 172.21.15.110:0/723739993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac01059b0 msgr2=0x7f5ac0105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:30.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.019+0000 7f5ac59d9700 1 --2- 172.21.15.110:0/723739993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac01059b0 0x7f5ac0105dc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000b0a0 tx=0x7f5ab000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:42:30.020 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.019+0000 7f5ac59d9700 1 -- 172.21.15.110:0/723739993 shutdown_connections 2024-05-09T04:42:30.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.019+0000 7f5ac59d9700 1 --2- 172.21.15.110:0/723739993 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac01049e0 0x7f5ac010c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:30.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.019+0000 7f5ac59d9700 1 --2- 172.21.15.110:0/723739993 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac01059b0 0x7f5ac0105dc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:30.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.019+0000 7f5ac59d9700 1 --2- 172.21.15.110:0/723739993 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac0103f60 0x7f5ac0104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:30.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.019+0000 7f5ac59d9700 1 -- 172.21.15.110:0/723739993 >> 172.21.15.110:0/723739993 conn(0x7f5ac00ff9d0 msgr2=0x7f5ac0101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:42:30.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.019+0000 7f5ac59d9700 1 -- 172.21.15.110:0/723739993 shutdown_connections 2024-05-09T04:42:30.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.019+0000 7f5ac59d9700 1 -- 172.21.15.110:0/723739993 wait complete. 2024-05-09T04:42:30.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5ac59d9700 1 Processor -- start 2024-05-09T04:42:30.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5ac59d9700 1 -- start start 2024-05-09T04:42:30.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5ac59d9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac0103f60 0x7f5ac019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:30.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5ac59d9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac01049e0 0x7f5ac019af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:30.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5ac59d9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac01059b0 0x7f5ac01a1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:30.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5ac59d9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac010fed0 con 0x7f5ac0103f60 2024-05-09T04:42:30.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5ac59d9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac010fd50 con 0x7f5ac01059b0 2024-05-09T04:42:30.021 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5ac59d9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac0110050 con 0x7f5ac01049e0 2024-05-09T04:42:30.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5ab67fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac01049e0 0x7f5ac019af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:30.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5ab67fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac01049e0 0x7f5ac019af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:39020/0 (socket says 172.21.15.110:39020) 2024-05-09T04:42:30.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5ab67fc700 1 -- 172.21.15.110:0/2612567121 learned_addr learned my addr 172.21.15.110:0/2612567121 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-09T04:42:30.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5abeffd700 1 --2- 172.21.15.110:0/2612567121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac0103f60 0x7f5ac019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:30.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5abf7fe700 1 --2- 172.21.15.110:0/2612567121 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac01059b0 0x7f5ac01a1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:30.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5ab67fc700 1 -- 172.21.15.110:0/2612567121 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac01059b0 msgr2=0x7f5ac01a1ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:30.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5ab67fc700 1 --2- 172.21.15.110:0/2612567121 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac01059b0 0x7f5ac01a1ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:30.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5ab67fc700 1 -- 172.21.15.110:0/2612567121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac0103f60 msgr2=0x7f5ac019a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:30.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.020+0000 7f5ab67fc700 1 --2- 172.21.15.110:0/2612567121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac0103f60 0x7f5ac019a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:30.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.021+0000 7f5ab67fc700 1 -- 172.21.15.110:0/2612567121 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac01a2540 con 0x7f5ac01049e0 2024-05-09T04:42:30.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.021+0000 7f5abeffd700 1 --2- 172.21.15.110:0/2612567121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac0103f60 0x7f5ac019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:42:30.022 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.021+0000 7f5ab67fc700 1 --2- 172.21.15.110:0/2612567121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac01049e0 0x7f5ac019af20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800d950 tx=0x7f5aa800dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:30.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.021+0000 7f5ac49d7700 1 -- 172.21.15.110:0/2612567121 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa8016070 con 0x7f5ac01049e0 2024-05-09T04:42:30.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.021+0000 7f5ac59d9700 1 -- 172.21.15.110:0/2612567121 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac01a2830 con 0x7f5ac01049e0 2024-05-09T04:42:30.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.021+0000 7f5ac49d7700 1 -- 172.21.15.110:0/2612567121 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5aa8010460 con 0x7f5ac01049e0 2024-05-09T04:42:30.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.021+0000 7f5ac49d7700 1 -- 172.21.15.110:0/2612567121 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5aa800f620 con 0x7f5ac01049e0 2024-05-09T04:42:30.023 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.021+0000 7f5ac59d9700 1 -- 172.21.15.110:0/2612567121 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac01a2d70 con 0x7f5ac01049e0 2024-05-09T04:42:30.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.022+0000 7f5ac59d9700 1 -- 172.21.15.110:0/2612567121 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac00512c0 con 0x7f5ac01049e0 2024-05-09T04:42:30.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.024+0000 7f5ac49d7700 1 -- 172.21.15.110:0/2612567121 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f5aa8019070 con 0x7f5ac01049e0 2024-05-09T04:42:30.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.025+0000 7f5ac49d7700 1 --2- 172.21.15.110:0/2612567121 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5aac0b27f0 0x7f5aac0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:30.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.025+0000 7f5ac49d7700 1 -- 172.21.15.110:0/2612567121 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f5aa80e05f0 con 0x7f5ac01049e0 2024-05-09T04:42:30.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.025+0000 7f5abeffd700 1 --2- 172.21.15.110:0/2612567121 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5aac0b27f0 0x7f5aac0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:30.024 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.025+0000 7f5abeffd700 1 --2- 172.21.15.110:0/2612567121 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5aac0b27f0 0x7f5aac0b4d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009fd0 tx=0x7f5ab0009c10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:30.027 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.029+0000 7f5ac49d7700 1 -- 172.21.15.110:0/2612567121 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5aa80a9b60 con 0x7f5ac01049e0 2024-05-09T04:42:30.239 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:30.241+0000 7f5ac59d9700 1 -- 172.21.15.110:0/2612567121 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f5ac002e960 con 0x7f5aac0b27f0 2024-05-09T04:42:30.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:30 smithi110 ceph-mon[34904]: Detected new or changed devices on smithi072 2024-05-09T04:42:30.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:30 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi072 to 17088M 2024-05-09T04:42:30.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:30 smithi110 ceph-mon[34904]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:30.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:30 smithi046 ceph-mon[30136]: Detected new or changed devices on smithi072 2024-05-09T04:42:30.665 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:30 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi072 to 17088M 2024-05-09T04:42:30.666 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:30 smithi046 ceph-mon[30136]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:30.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:30 smithi072 ceph-mon[34088]: Detected new or changed devices on smithi072 2024-05-09T04:42:30.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:30 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi072 to 17088M 2024-05-09T04:42:30.666 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:30 smithi072 ceph-mon[34088]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:31.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:31 smithi072 ceph-mon[34088]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T04:42:31.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:31 smithi072 ceph-mon[34088]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:31.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:31 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:42:31.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:31 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:42:31.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:31 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:31.535 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:31 smithi110 ceph-mon[34904]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T04:42:31.535 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:31 smithi110 ceph-mon[34904]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:31.535 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:31 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:42:31.535 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:31 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:42:31.535 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:31 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:31.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:31 smithi046 ceph-mon[30136]: osdmap e14: 2 total, 2 up, 2 in 2024-05-09T04:42:31.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:31 smithi046 ceph-mon[30136]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:31.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:31 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:42:31.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:31 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:42:31.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:31 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:32.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:32 smithi110 ceph-mon[34904]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:32.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:32 smithi046 ceph-mon[30136]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:32.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:32 smithi072 ceph-mon[34088]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:33.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:33 smithi110 ceph-mon[34904]: from='client.? 172.21.15.110:0/2763174751' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c549b131-cfa7-4724-926f-83e91a752d44"}]: dispatch 2024-05-09T04:42:33.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:33 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c549b131-cfa7-4724-926f-83e91a752d44"}]: dispatch 2024-05-09T04:42:33.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:33 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c549b131-cfa7-4724-926f-83e91a752d44"}]': finished 2024-05-09T04:42:33.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:33 smithi110 ceph-mon[34904]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T04:42:33.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:33 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:42:33.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:33 smithi046 ceph-mon[30136]: from='client.? 172.21.15.110:0/2763174751' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c549b131-cfa7-4724-926f-83e91a752d44"}]: dispatch 2024-05-09T04:42:33.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:33 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c549b131-cfa7-4724-926f-83e91a752d44"}]: dispatch 2024-05-09T04:42:33.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:33 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c549b131-cfa7-4724-926f-83e91a752d44"}]': finished 2024-05-09T04:42:33.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:33 smithi046 ceph-mon[30136]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T04:42:33.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:33 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:42:33.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:33 smithi072 ceph-mon[34088]: from='client.? 172.21.15.110:0/2763174751' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c549b131-cfa7-4724-926f-83e91a752d44"}]: dispatch 2024-05-09T04:42:33.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:33 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c549b131-cfa7-4724-926f-83e91a752d44"}]: dispatch 2024-05-09T04:42:33.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:33 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c549b131-cfa7-4724-926f-83e91a752d44"}]': finished 2024-05-09T04:42:33.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:33 smithi072 ceph-mon[34088]: osdmap e15: 3 total, 2 up, 3 in 2024-05-09T04:42:33.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:33 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:42:34.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:34 smithi110 ceph-mon[34904]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:34.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:34 smithi110 ceph-mon[34904]: from='client.? 172.21.15.110:0/1265947686' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T04:42:34.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:34 smithi046 ceph-mon[30136]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:34.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:34 smithi046 ceph-mon[30136]: from='client.? 172.21.15.110:0/1265947686' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T04:42:34.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:34 smithi072 ceph-mon[34088]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:34.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:34 smithi072 ceph-mon[34088]: from='client.? 172.21.15.110:0/1265947686' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-2/activate.monmap"}]: dispatch 2024-05-09T04:42:36.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:36 smithi110 ceph-mon[34904]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:36.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:36 smithi046 ceph-mon[30136]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:36.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:36 smithi072 ceph-mon[34088]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:38.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:38 smithi110 ceph-mon[34904]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:38.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:38 smithi046 ceph-mon[30136]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:38.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:38 smithi072 ceph-mon[34088]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:40.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:40 smithi110 ceph-mon[34904]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:40.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T04:42:40.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:40.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:40 smithi046 ceph-mon[30136]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:40.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T04:42:40.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:40.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:40 smithi072 ceph-mon[34088]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:40.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-09T04:42:40.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:41.461 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:41 smithi110 ceph-mon[34904]: Deploying daemon osd.2 on smithi110 2024-05-09T04:42:41.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:41 smithi046 ceph-mon[30136]: Deploying daemon osd.2 on smithi110 2024-05-09T04:42:41.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:41 smithi072 ceph-mon[34088]: Deploying daemon osd.2 on smithi110 2024-05-09T04:42:42.409 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:42 smithi110 ceph-mon[34904]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:42.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:42 smithi046 ceph-mon[30136]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:42.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:42 smithi072 ceph-mon[34088]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:44.501 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:44 smithi110 ceph-mon[34904]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:44.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:44 smithi046 ceph-mon[30136]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:44.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:44 smithi072 ceph-mon[34088]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:46.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:46 smithi072 ceph-mon[34088]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:46.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:46 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:46.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:46 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:46.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:46 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:46.650 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:46 smithi110 ceph-mon[34904]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:46.650 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:46 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:46.651 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:46 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:46.651 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:46 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:46.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:46 smithi046 ceph-mon[30136]: pgmap v74: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:46.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:46 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:46.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:46 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:46.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:46 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:48.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:48 smithi072 ceph-mon[34088]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:48.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:48.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:48.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:48.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:48.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:48.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:48 smithi046 ceph-mon[30136]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:48.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:48.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:48.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:48.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:48.632 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:48.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:48 smithi110 ceph-mon[34904]: pgmap v75: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:49.162 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:49.162+0000 7f5ac49d7700 1 -- 172.21.15.110:0/2612567121 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5ac002e960 con 0x7f5aac0b27f0 2024-05-09T04:42:49.162 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 2 on host 'smithi110' 2024-05-09T04:42:49.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:49.165+0000 7f5ac59d9700 1 -- 172.21.15.110:0/2612567121 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5aac0b27f0 msgr2=0x7f5aac0b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:49.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:49.165+0000 7f5ac59d9700 1 --2- 172.21.15.110:0/2612567121 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5aac0b27f0 0x7f5aac0b4d00 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009fd0 tx=0x7f5ab0009c10 comp rx=0 tx=0).stop 2024-05-09T04:42:49.165 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:49.165+0000 7f5ac59d9700 1 -- 172.21.15.110:0/2612567121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac01049e0 msgr2=0x7f5ac019af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:49.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:49.165+0000 7f5ac59d9700 1 --2- 172.21.15.110:0/2612567121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac01049e0 0x7f5ac019af20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5aa800d950 tx=0x7f5aa800dd10 comp rx=0 tx=0).stop 2024-05-09T04:42:49.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:49.165+0000 7f5ac59d9700 1 -- 172.21.15.110:0/2612567121 shutdown_connections 2024-05-09T04:42:49.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:49.165+0000 7f5ac59d9700 1 --2- 172.21.15.110:0/2612567121 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5aac0b27f0 0x7f5aac0b4d00 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:49.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:49.165+0000 7f5ac59d9700 1 --2- 172.21.15.110:0/2612567121 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ac01059b0 0x7f5ac01a1ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:49.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:49.165+0000 7f5ac59d9700 1 --2- 172.21.15.110:0/2612567121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ac0103f60 0x7f5ac019a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:49.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:49.165+0000 7f5ac59d9700 1 --2- 172.21.15.110:0/2612567121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ac01049e0 0x7f5ac019af20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:49.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:49.165+0000 7f5ac59d9700 1 -- 172.21.15.110:0/2612567121 >> 172.21.15.110:0/2612567121 conn(0x7f5ac00ff9d0 msgr2=0x7f5ac01086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:42:49.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:49.165+0000 7f5ac59d9700 1 -- 172.21.15.110:0/2612567121 shutdown_connections 2024-05-09T04:42:49.166 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:42:49.165+0000 7f5ac59d9700 1 -- 172.21.15.110:0/2612567121 wait complete. 2024-05-09T04:42:49.803 DEBUG:teuthology.orchestra.run.smithi110:osd.2> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.2.service 2024-05-09T04:42:49.806 INFO:tasks.cephadm:Deploying osd.3 on smithi046 with /dev/vg_nvme/lv_3... 2024-05-09T04:42:49.806 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T04:42:50.052 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:42:50.478 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:50.479 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:50.479 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:50.479 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:50 smithi046 ceph-mon[30136]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:50.479 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:50 smithi046 ceph-mon[30136]: from='osd.2 [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T04:42:50.479 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:50 smithi046 ceph-mon[30136]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T04:42:50.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:50.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:50.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:50.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:50 smithi110 ceph-mon[34904]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:50.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:50 smithi110 ceph-mon[34904]: from='osd.2 [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T04:42:50.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:50 smithi110 ceph-mon[34904]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T04:42:50.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:42:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:50 smithi072 ceph-mon[34088]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:50 smithi072 ceph-mon[34088]: from='osd.2 [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T04:42:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:50 smithi072 ceph-mon[34088]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-09T04:42:51.205 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:51 smithi110 ceph-mon[34904]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T04:42:51.205 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:51 smithi110 ceph-mon[34904]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T04:42:51.205 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:51 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:42:51.205 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:51 smithi110 ceph-mon[34904]: from='osd.2 [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:42:51.205 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:51 smithi110 ceph-mon[34904]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:42:51.206 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 04:42:51 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-2[40098]: 2024-05-09T04:42:51.167+0000 7f9d75af0700 -1 osd.2 0 waiting for initial osdmap 2024-05-09T04:42:51.206 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 04:42:51 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-2[40098]: 2024-05-09T04:42:51.172+0000 7f9d718a5700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:42:51.486 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:51 smithi046 ceph-mon[30136]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T04:42:51.486 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:51 smithi046 ceph-mon[30136]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T04:42:51.486 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:51 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:42:51.486 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:51 smithi046 ceph-mon[30136]: from='osd.2 [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:42:51.486 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:51 smithi046 ceph-mon[30136]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:42:51.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:51 smithi072 ceph-mon[34088]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-09T04:42:51.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:51 smithi072 ceph-mon[34088]: osdmap e16: 3 total, 2 up, 3 in 2024-05-09T04:42:51.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:51 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:42:51.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:51 smithi072 ceph-mon[34088]: from='osd.2 [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:42:51.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:51 smithi072 ceph-mon[34088]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:42:52.086 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T04:42:52.086 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T04:42:52.086 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-05-09T04:42:52.086 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-05-09T04:42:52.086 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00851772 s, 1.2 GB/s 2024-05-09T04:42:52.086 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-05-09T04:42:52.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:52 smithi110 ceph-mon[34904]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-09T04:42:52.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:52 smithi110 ceph-mon[34904]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T04:42:52.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:52 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:42:52.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:52 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:42:52.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:52 smithi110 ceph-mon[34904]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:52.413 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:52 smithi046 ceph-mon[30136]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-09T04:42:52.413 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:52 smithi046 ceph-mon[30136]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T04:42:52.414 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:52 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:42:52.414 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:52 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:42:52.414 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:52 smithi046 ceph-mon[30136]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:52.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:52 smithi072 ceph-mon[34088]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-09T04:42:52.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:52 smithi072 ceph-mon[34088]: osdmap e17: 3 total, 2 up, 3 in 2024-05-09T04:42:52.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:52 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:42:52.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:52 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:42:52.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:52 smithi072 ceph-mon[34088]: pgmap v79: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-05-09T04:42:52.767 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_3 2024-05-09T04:42:53.036 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:42:53.467 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:53 smithi046 ceph-mon[30136]: purged_snaps scrub starts 2024-05-09T04:42:53.467 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:53 smithi046 ceph-mon[30136]: purged_snaps scrub ok 2024-05-09T04:42:53.467 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:53 smithi046 ceph-mon[30136]: osd.2 [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227] boot 2024-05-09T04:42:53.467 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:53 smithi046 ceph-mon[30136]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T04:42:53.467 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:53 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:42:53.467 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:53 smithi046 ceph-mon[30136]: Detected new or changed devices on smithi110 2024-05-09T04:42:53.467 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:53 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:53.467 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:53 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:53.468 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:53 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:42:53.468 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:53 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi110 to 17088M 2024-05-09T04:42:53.468 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:53 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:53.468 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:53 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:53.468 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:53 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:53.468 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:53 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:53.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:53 smithi110 ceph-mon[34904]: purged_snaps scrub starts 2024-05-09T04:42:53.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:53 smithi110 ceph-mon[34904]: purged_snaps scrub ok 2024-05-09T04:42:53.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:53 smithi110 ceph-mon[34904]: osd.2 [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227] boot 2024-05-09T04:42:53.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:53 smithi110 ceph-mon[34904]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T04:42:53.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:53 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:42:53.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:53 smithi110 ceph-mon[34904]: Detected new or changed devices on smithi110 2024-05-09T04:42:53.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:53 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:53.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:53 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:53.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:53 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:42:53.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:53 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi110 to 17088M 2024-05-09T04:42:53.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:53 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:53.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:53 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:53.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:53 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:53.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:53 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:53.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:53 smithi072 ceph-mon[34088]: purged_snaps scrub starts 2024-05-09T04:42:53.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:53 smithi072 ceph-mon[34088]: purged_snaps scrub ok 2024-05-09T04:42:53.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:53 smithi072 ceph-mon[34088]: osd.2 [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227] boot 2024-05-09T04:42:53.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:53 smithi072 ceph-mon[34088]: osdmap e18: 3 total, 3 up, 3 in 2024-05-09T04:42:53.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:53 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:42:53.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:53 smithi072 ceph-mon[34088]: Detected new or changed devices on smithi110 2024-05-09T04:42:53.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:53 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:53.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:53 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:53.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:53 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:42:53.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:53 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi110 to 17088M 2024-05-09T04:42:53.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:53 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:53.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:53 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:53.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:53 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:42:53.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:53 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:54.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:54 smithi046 ceph-mon[30136]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T04:42:54.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:54 smithi046 ceph-mon[30136]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:42:54.390 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:54 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T04:42:54.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:54 smithi110 ceph-mon[34904]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T04:42:54.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:54 smithi110 ceph-mon[34904]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:42:54.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:54 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T04:42:54.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:54 smithi072 ceph-mon[34088]: osdmap e19: 3 total, 3 up, 3 in 2024-05-09T04:42:54.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:54 smithi072 ceph-mon[34088]: pgmap v82: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:42:54.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:54 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-09T04:42:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.648+0000 7fa901ad9700 1 -- 172.21.15.46:0/452319073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8fc1074b0 msgr2=0x7fa8fc109950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.648+0000 7fa901ad9700 1 --2- 172.21.15.46:0/452319073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8fc1074b0 0x7fa8fc109950 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0009990 tx=0x7fa8f0009ca0 comp rx=0 tx=0).stop 2024-05-09T04:42:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.648+0000 7fa901ad9700 1 -- 172.21.15.46:0/452319073 shutdown_connections 2024-05-09T04:42:54.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.648+0000 7fa901ad9700 1 --2- 172.21.15.46:0/452319073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8fc109fe0 0x7fa8fc10c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.648+0000 7fa901ad9700 1 --2- 172.21.15.46:0/452319073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8fc06ccc0 0x7fa8fc106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.649+0000 7fa901ad9700 1 --2- 172.21.15.46:0/452319073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8fc1074b0 0x7fa8fc109950 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.649+0000 7fa901ad9700 1 -- 172.21.15.46:0/452319073 >> 172.21.15.46:0/452319073 conn(0x7fa8fc0fe870 msgr2=0x7fa8fc100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:42:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.649+0000 7fa901ad9700 1 -- 172.21.15.46:0/452319073 shutdown_connections 2024-05-09T04:42:54.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.649+0000 7fa901ad9700 1 -- 172.21.15.46:0/452319073 wait complete. 2024-05-09T04:42:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.649+0000 7fa901ad9700 1 Processor -- start 2024-05-09T04:42:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.649+0000 7fa901ad9700 1 -- start start 2024-05-09T04:42:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.650+0000 7fa901ad9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8fc06ccc0 0x7fa8fc104730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.650+0000 7fa901ad9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8fc1074b0 0x7fa8fc102de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.650+0000 7fa901ad9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8fc109fe0 0x7fa8fc103320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.650+0000 7fa901ad9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc10fd90 con 0x7fa8fc06ccc0 2024-05-09T04:42:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.650+0000 7fa901ad9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc10fc10 con 0x7fa8fc1074b0 2024-05-09T04:42:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.650+0000 7fa901ad9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa8fc10ff10 con 0x7fa8fc109fe0 2024-05-09T04:42:54.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.650+0000 7fa8fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8fc06ccc0 0x7fa8fc104730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:54.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.650+0000 7fa8fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8fc06ccc0 0x7fa8fc104730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50190/0 (socket says 172.21.15.46:50190) 2024-05-09T04:42:54.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.650+0000 7fa8fb7fe700 1 -- 172.21.15.46:0/898241012 learned_addr learned my addr 172.21.15.46:0/898241012 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:42:54.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.650+0000 7fa8fbfff700 1 --2- 172.21.15.46:0/898241012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8fc109fe0 0x7fa8fc103320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:54.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.650+0000 7fa8fb7fe700 1 -- 172.21.15.46:0/898241012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8fc109fe0 msgr2=0x7fa8fc103320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:42:54.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.650+0000 7fa8fb7fe700 1 --2- 172.21.15.46:0/898241012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8fc109fe0 0x7fa8fc103320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:54.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.650+0000 7fa8fb7fe700 1 -- 172.21.15.46:0/898241012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8fc1074b0 msgr2=0x7fa8fc102de0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T04:42:54.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.650+0000 7fa8fb7fe700 1 --2- 172.21.15.46:0/898241012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8fc1074b0 0x7fa8fc102de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:42:54.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.650+0000 7fa8fb7fe700 1 -- 172.21.15.46:0/898241012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8fc1039f0 con 0x7fa8fc06ccc0 2024-05-09T04:42:54.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.651+0000 7fa8fb7fe700 1 --2- 172.21.15.46:0/898241012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8fc06ccc0 0x7fa8fc104730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec007f10 tx=0x7fa8ec00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:54.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.651+0000 7fa8f8ff9700 1 -- 172.21.15.46:0/898241012 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00ddd0 con 0x7fa8fc06ccc0 2024-05-09T04:42:54.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.651+0000 7fa901ad9700 1 -- 172.21.15.46:0/898241012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8fc1a49b0 con 0x7fa8fc06ccc0 2024-05-09T04:42:54.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.651+0000 7fa8f8ff9700 1 -- 172.21.15.46:0/898241012 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8ec018070 con 0x7fa8fc06ccc0 2024-05-09T04:42:54.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.651+0000 7fa8f8ff9700 1 -- 172.21.15.46:0/898241012 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec012640 con 0x7fa8fc06ccc0 2024-05-09T04:42:54.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.651+0000 7fa901ad9700 1 -- 172.21.15.46:0/898241012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8fc1a4e00 con 0x7fa8fc06ccc0 2024-05-09T04:42:54.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.652+0000 7fa901ad9700 1 -- 172.21.15.46:0/898241012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8fc0512c0 con 0x7fa8fc06ccc0 2024-05-09T04:42:54.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.655+0000 7fa8f8ff9700 1 -- 172.21.15.46:0/898241012 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fa8ec0139c0 con 0x7fa8fc06ccc0 2024-05-09T04:42:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.655+0000 7fa8f8ff9700 1 --2- 172.21.15.46:0/898241012 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa8e40b27f0 0x7fa8e40b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:42:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.656+0000 7fa8faffd700 1 --2- 172.21.15.46:0/898241012 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa8e40b27f0 0x7fa8e40b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:42:54.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.656+0000 7fa8f8ff9700 1 -- 172.21.15.46:0/898241012 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 3226+0+0 (secure 0 0 0) 0x7fa8ec0e0df0 con 0x7fa8fc06ccc0 2024-05-09T04:42:54.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.656+0000 7fa8faffd700 1 --2- 172.21.15.46:0/898241012 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa8e40b27f0 0x7fa8e40b4d00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000b5c0 tx=0x7fa8f0003860 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:42:54.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.659+0000 7fa8f8ff9700 1 -- 172.21.15.46:0/898241012 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa8ec010070 con 0x7fa8fc06ccc0 2024-05-09T04:42:54.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:54.882+0000 7fa901ad9700 1 -- 172.21.15.46:0/898241012 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fa8fc063cc0 con 0x7fa8e40b27f0 2024-05-09T04:42:55.455 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:55 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T04:42:55.456 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:55 smithi046 ceph-mon[30136]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T04:42:55.456 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:55 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T04:42:55.456 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:55 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:42:55.456 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:55 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:42:55.456 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:55 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:55.456 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49640]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T04:42:55.456 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49640]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:42:55.456 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49640]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:42:55.457 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49640]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:42:55.457 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49643]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T04:42:55.457 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49643]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:42:55.457 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49643]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:42:55.457 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49643]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:42:55.495 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:55 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T04:42:55.495 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:55 smithi110 ceph-mon[34904]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T04:42:55.495 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:55 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T04:42:55.495 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:55 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:42:55.495 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:55 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:42:55.495 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:55 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:55.496 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43843]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T04:42:55.496 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43843]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:42:55.496 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43843]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:42:55.496 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43843]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:42:55.496 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43846]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T04:42:55.496 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43846]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:42:55.496 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43846]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:42:55.496 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43846]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:42:55.531 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:55 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-09T04:42:55.532 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:55 smithi072 ceph-mon[34088]: osdmap e20: 3 total, 3 up, 3 in 2024-05-09T04:42:55.532 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:55 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T04:42:55.532 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:55 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:42:55.532 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:55 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:42:55.532 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:55 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:42:55.532 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42399]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T04:42:55.532 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42399]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:42:55.532 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42399]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:42:55.532 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42399]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:42:55.533 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42402]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T04:42:55.533 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42402]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:42:55.533 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42402]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:42:55.533 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42402]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:42:55.760 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49702]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T04:42:55.760 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49702]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:42:55.760 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49702]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:42:55.760 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49702]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:42:55.760 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49705]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T04:42:55.760 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49705]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:42:55.760 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49705]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:42:55.761 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:55 smithi046 sudo[49705]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:42:55.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43849]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T04:42:55.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43849]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:42:55.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43849]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:42:55.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43849]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:42:55.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43852]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T04:42:55.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43852]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:42:55.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43852]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:42:55.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:55 smithi110 sudo[43852]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:42:55.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42405]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-09T04:42:55.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42405]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:42:55.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42405]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:42:55.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42405]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:42:55.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42408]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-09T04:42:55.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42408]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-09T04:42:55.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42408]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-09T04:42:55.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:55 smithi072 sudo[42408]: pam_unix(sudo:session): session closed for user root 2024-05-09T04:42:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:42:56.209+0000 7fa8f8ff9700 1 -- 172.21.15.46:0/898241012 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa8ec0a5b60 con 0x7fa8fc06ccc0 2024-05-09T04:42:56.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:56.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T04:42:56.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T04:42:56.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:42:56.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:56.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:56.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:56.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:42:56.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:42:56.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:42:56.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:56.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:56.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:42:56.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:42:56.537 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:56.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:56.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:42:56.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:42:56.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:42:56.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:56.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:56.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:56 smithi046 ceph-mon[30136]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:42:56.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:56.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T04:42:56.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T04:42:56.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:42:56.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:56.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:56.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:56.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:42:56.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:42:56.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:42:56.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:56.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:56.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:42:56.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:42:56.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:56.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:56.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:42:56.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:42:56.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:42:56.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:56.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:56.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:56 smithi110 ceph-mon[34904]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:42:56.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='client.14301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:42:56.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-09T04:42:56.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: osdmap e21: 3 total, 3 up, 3 in 2024-05-09T04:42:56.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: pgmap v85: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:42:56.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:56.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:56.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:42:56.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:42:56.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:42:56.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:42:56.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:56.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:56.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:42:56.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:42:56.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:56.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:56.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:42:56.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-09T04:42:56.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:42:56.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:42:56.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:42:56.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:56 smithi072 ceph-mon[34088]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-09T04:42:57.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:57 smithi110 ceph-mon[34904]: osdmap e22: 3 total, 3 up, 3 in 2024-05-09T04:42:57.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:57 smithi110 ceph-mon[34904]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T04:42:57.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:57 smithi046 ceph-mon[30136]: osdmap e22: 3 total, 3 up, 3 in 2024-05-09T04:42:57.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:57 smithi046 ceph-mon[30136]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T04:42:57.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:57 smithi072 ceph-mon[34088]: osdmap e22: 3 total, 3 up, 3 in 2024-05-09T04:42:57.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:57 smithi072 ceph-mon[34088]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-09T04:42:58.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:58 smithi046 ceph-mon[30136]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:42:58.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:58 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/360388877' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "549bb4f7-54d9-4cbc-ad40-7f3e717964f3"}]: dispatch 2024-05-09T04:42:58.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:58 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/360388877' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "549bb4f7-54d9-4cbc-ad40-7f3e717964f3"}]': finished 2024-05-09T04:42:58.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:58 smithi046 ceph-mon[30136]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T04:42:58.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:58 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:42:58.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:58 smithi110 ceph-mon[34904]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:42:58.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:58 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/360388877' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "549bb4f7-54d9-4cbc-ad40-7f3e717964f3"}]: dispatch 2024-05-09T04:42:58.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:58 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/360388877' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "549bb4f7-54d9-4cbc-ad40-7f3e717964f3"}]': finished 2024-05-09T04:42:58.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:58 smithi110 ceph-mon[34904]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T04:42:58.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:58 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:42:58.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:58 smithi072 ceph-mon[34088]: pgmap v87: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:42:58.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:58 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/360388877' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "549bb4f7-54d9-4cbc-ad40-7f3e717964f3"}]: dispatch 2024-05-09T04:42:58.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:58 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/360388877' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "549bb4f7-54d9-4cbc-ad40-7f3e717964f3"}]': finished 2024-05-09T04:42:58.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:58 smithi072 ceph-mon[34088]: osdmap e23: 4 total, 3 up, 4 in 2024-05-09T04:42:58.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:58 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:42:59.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:42:59 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1878230604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T04:42:59.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:42:59 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1878230604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T04:42:59.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:42:59 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1878230604' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-3/activate.monmap"}]: dispatch 2024-05-09T04:43:00.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:00 smithi110 ceph-mon[34904]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:00.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:00 smithi046 ceph-mon[30136]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:00.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:00 smithi072 ceph-mon[34088]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:02.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:02 smithi110 ceph-mon[34904]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:02.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:02 smithi046 ceph-mon[30136]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:02.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:02 smithi072 ceph-mon[34088]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:04.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:04 smithi110 ceph-mon[34904]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:04.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:04 smithi046 ceph-mon[30136]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:04.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:04 smithi072 ceph-mon[34088]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:05.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:05 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T04:43:05.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:05 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:05.568 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T04:43:05.569 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:05.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:05 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-09T04:43:05.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:05 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:06.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:06 smithi110 ceph-mon[34904]: Deploying daemon osd.3 on smithi046 2024-05-09T04:43:06.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:06 smithi110 ceph-mon[34904]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:06.572 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:06 smithi046 ceph-mon[30136]: Deploying daemon osd.3 on smithi046 2024-05-09T04:43:06.573 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:06 smithi046 ceph-mon[30136]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:06.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:06 smithi072 ceph-mon[34088]: Deploying daemon osd.3 on smithi046 2024-05-09T04:43:06.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:06 smithi072 ceph-mon[34088]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:08.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:08 smithi110 ceph-mon[34904]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:08.565 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:08 smithi046 ceph-mon[30136]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:08.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:08 smithi072 ceph-mon[34088]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:10.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:10 smithi110 ceph-mon[34904]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:10.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:10 smithi046 ceph-mon[30136]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:10.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:10 smithi072 ceph-mon[34088]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:11.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:11 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:11.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:11 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:11.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:11 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:11.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:11 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:11.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:11 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:11.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:11 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:11.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:11 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:11.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:11 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:11.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:11 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:12.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:12 smithi046 ceph-mon[30136]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:12.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:12 smithi110 ceph-mon[34904]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:12.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:12 smithi072 ceph-mon[34088]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:13.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:43:13.606+0000 7fa8f8ff9700 1 -- 172.21.15.46:0/898241012 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa8fc063cc0 con 0x7fa8e40b27f0 2024-05-09T04:43:13.607 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 3 on host 'smithi046' 2024-05-09T04:43:13.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:43:13.609+0000 7fa901ad9700 1 -- 172.21.15.46:0/898241012 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa8e40b27f0 msgr2=0x7fa8e40b4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:43:13.609+0000 7fa901ad9700 1 --2- 172.21.15.46:0/898241012 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa8e40b27f0 0x7fa8e40b4d00 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa8f000b5c0 tx=0x7fa8f0003860 comp rx=0 tx=0).stop 2024-05-09T04:43:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:43:13.609+0000 7fa901ad9700 1 -- 172.21.15.46:0/898241012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8fc06ccc0 msgr2=0x7fa8fc104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:43:13.609+0000 7fa901ad9700 1 --2- 172.21.15.46:0/898241012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8fc06ccc0 0x7fa8fc104730 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec007f10 tx=0x7fa8ec00d3b0 comp rx=0 tx=0).stop 2024-05-09T04:43:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:43:13.609+0000 7fa901ad9700 1 -- 172.21.15.46:0/898241012 shutdown_connections 2024-05-09T04:43:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:43:13.609+0000 7fa901ad9700 1 --2- 172.21.15.46:0/898241012 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa8e40b27f0 0x7fa8e40b4d00 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:43:13.609+0000 7fa901ad9700 1 --2- 172.21.15.46:0/898241012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa8fc109fe0 0x7fa8fc103320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:43:13.609+0000 7fa901ad9700 1 --2- 172.21.15.46:0/898241012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8fc06ccc0 0x7fa8fc104730 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:43:13.609+0000 7fa901ad9700 1 --2- 172.21.15.46:0/898241012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8fc1074b0 0x7fa8fc102de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:43:13.609+0000 7fa901ad9700 1 -- 172.21.15.46:0/898241012 >> 172.21.15.46:0/898241012 conn(0x7fa8fc0fe870 msgr2=0x7fa8fc1056b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:43:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:43:13.609+0000 7fa901ad9700 1 -- 172.21.15.46:0/898241012 shutdown_connections 2024-05-09T04:43:13.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:43:13.609+0000 7fa901ad9700 1 -- 172.21.15.46:0/898241012 wait complete. 2024-05-09T04:43:14.575 DEBUG:teuthology.orchestra.run.smithi046:osd.3> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.3.service 2024-05-09T04:43:14.577 INFO:tasks.cephadm:Deploying osd.4 on smithi072 with /dev/vg_nvme/lv_3... 2024-05-09T04:43:14.577 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T04:43:14.821 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:43:14.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:14 smithi046 ceph-mon[30136]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:14.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:14 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:14.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:14 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:14.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:14 smithi046 ceph-mon[30136]: from='osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T04:43:14.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:14 smithi072 ceph-mon[34088]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:14.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:14 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:14.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:14 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:14.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:14 smithi072 ceph-mon[34088]: from='osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T04:43:15.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:14 smithi110 ceph-mon[34904]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:15.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:14 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:15.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:14 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:15.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:14 smithi110 ceph-mon[34904]: from='osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-09T04:43:15.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:15 smithi046 ceph-mon[30136]: from='osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T04:43:15.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:15 smithi046 ceph-mon[30136]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T04:43:15.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:15 smithi046 ceph-mon[30136]: from='osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T04:43:15.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:15 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:15.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:15 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:15.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:15 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:15.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:15 smithi072 ceph-mon[34088]: from='osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T04:43:15.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:15 smithi072 ceph-mon[34088]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T04:43:15.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:15 smithi072 ceph-mon[34088]: from='osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T04:43:15.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:15 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:15.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:15 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:15.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:15 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:16.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:15 smithi110 ceph-mon[34904]: from='osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-09T04:43:16.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:15 smithi110 ceph-mon[34904]: osdmap e24: 4 total, 3 up, 4 in 2024-05-09T04:43:16.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:15 smithi110 ceph-mon[34904]: from='osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T04:43:16.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:15 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:16.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:15 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:16.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:15 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:16.375 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 04:43:16 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3[51847]: 2024-05-09T04:43:16.093+0000 7fe114311700 -1 osd.3 0 waiting for initial osdmap 2024-05-09T04:43:16.375 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 04:43:16 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3[51847]: 2024-05-09T04:43:16.101+0000 7fe10c0be700 -1 osd.3 26 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:43:16.662 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:16 smithi046 ceph-mon[30136]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:16.662 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:16 smithi046 ceph-mon[30136]: from='osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T04:43:16.662 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:16 smithi046 ceph-mon[30136]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T04:43:16.663 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:16 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:16.663 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:16 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:16.663 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:16 smithi046 ceph-mon[30136]: osdmap e26: 4 total, 3 up, 4 in 2024-05-09T04:43:16.663 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:16 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:16.663 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:16 smithi046 ceph-mon[30136]: from='osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653]' entity='osd.3' 2024-05-09T04:43:16.748 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T04:43:16.748 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T04:43:16.748 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-05-09T04:43:16.748 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-05-09T04:43:16.749 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113033 s, 928 MB/s 2024-05-09T04:43:16.749 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-05-09T04:43:16.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:16 smithi072 ceph-mon[34088]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:16.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:16 smithi072 ceph-mon[34088]: from='osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T04:43:16.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:16 smithi072 ceph-mon[34088]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T04:43:16.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:16 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:16.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:16 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:16.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:16 smithi072 ceph-mon[34088]: osdmap e26: 4 total, 3 up, 4 in 2024-05-09T04:43:16.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:16 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:16.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:16 smithi072 ceph-mon[34088]: from='osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653]' entity='osd.3' 2024-05-09T04:43:17.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:16 smithi110 ceph-mon[34904]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:16 smithi110 ceph-mon[34904]: from='osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T04:43:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:16 smithi110 ceph-mon[34904]: osdmap e25: 4 total, 3 up, 4 in 2024-05-09T04:43:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:16 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:16 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:16 smithi110 ceph-mon[34904]: osdmap e26: 4 total, 3 up, 4 in 2024-05-09T04:43:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:16 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:16 smithi110 ceph-mon[34904]: from='osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653]' entity='osd.3' 2024-05-09T04:43:17.464 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_3 2024-05-09T04:43:17.702 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:43:17.752 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:17 smithi072 ceph-mon[34088]: purged_snaps scrub starts 2024-05-09T04:43:17.752 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:17 smithi072 ceph-mon[34088]: purged_snaps scrub ok 2024-05-09T04:43:17.752 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:17 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:17.752 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:17 smithi072 ceph-mon[34088]: osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653] boot 2024-05-09T04:43:17.752 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:17 smithi072 ceph-mon[34088]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T04:43:17.752 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:17 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:17.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:17 smithi046 ceph-mon[30136]: purged_snaps scrub starts 2024-05-09T04:43:17.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:17 smithi046 ceph-mon[30136]: purged_snaps scrub ok 2024-05-09T04:43:17.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:17.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:17 smithi046 ceph-mon[30136]: osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653] boot 2024-05-09T04:43:17.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:17 smithi046 ceph-mon[30136]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T04:43:17.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:18.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:17 smithi110 ceph-mon[34904]: purged_snaps scrub starts 2024-05-09T04:43:18.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:17 smithi110 ceph-mon[34904]: purged_snaps scrub ok 2024-05-09T04:43:18.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:18.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:17 smithi110 ceph-mon[34904]: osd.3 [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653] boot 2024-05-09T04:43:18.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:17 smithi110 ceph-mon[34904]: osdmap e27: 4 total, 4 up, 4 in 2024-05-09T04:43:18.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:43:18.853 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:18 smithi072 ceph-mon[34088]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:18.854 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:18 smithi072 ceph-mon[34088]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T04:43:18.854 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:18.854 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:18.854 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:18.854 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:18.854 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:18.854 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:18.854 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:18.854 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:18.854 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:18.854 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:18.855 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:18.855 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:19.055 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:18 smithi110 ceph-mon[34904]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:19.055 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:18 smithi110 ceph-mon[34904]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T04:43:19.055 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:19.055 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:19.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:19.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:19.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:19.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:19.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:19.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:19.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:19.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:19.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:19.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:19.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:18 smithi046 ceph-mon[30136]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-09T04:43:19.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:18 smithi046 ceph-mon[30136]: osdmap e28: 4 total, 4 up, 4 in 2024-05-09T04:43:19.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:19.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:19.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:19.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:19.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.354+0000 7f83227c2700 1 -- 172.21.15.72:0/1550207642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f831c104380 msgr2=0x7f831c104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:19.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.354+0000 7f83227c2700 1 --2- 172.21.15.72:0/1550207642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f831c104380 0x7f831c104790 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8310009990 tx=0x7f8310009ca0 comp rx=0 tx=0).stop 2024-05-09T04:43:19.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.354+0000 7f83227c2700 1 -- 172.21.15.72:0/1550207642 shutdown_connections 2024-05-09T04:43:19.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.354+0000 7f83227c2700 1 --2- 172.21.15.72:0/1550207642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f831c104d20 0x7f831c10b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:19.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.354+0000 7f83227c2700 1 --2- 172.21.15.72:0/1550207642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c103250 0x7f831c103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:19.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.354+0000 7f83227c2700 1 --2- 172.21.15.72:0/1550207642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f831c104380 0x7f831c104790 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:19.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.354+0000 7f83227c2700 1 -- 172.21.15.72:0/1550207642 >> 172.21.15.72:0/1550207642 conn(0x7f831c0fe890 msgr2=0x7f831c100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:43:19.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.354+0000 7f83227c2700 1 -- 172.21.15.72:0/1550207642 shutdown_connections 2024-05-09T04:43:19.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.354+0000 7f83227c2700 1 -- 172.21.15.72:0/1550207642 wait complete. 2024-05-09T04:43:19.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.355+0000 7f83227c2700 1 Processor -- start 2024-05-09T04:43:19.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.355+0000 7f83227c2700 1 -- start start 2024-05-09T04:43:19.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.355+0000 7f83227c2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f831c103250 0x7f831c19a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:19.368 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.355+0000 7f83227c2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c104380 0x7f831c19ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:19.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.355+0000 7f83227c2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f831c104d20 0x7f831c1a1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:19.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.355+0000 7f83227c2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f831c10ecb0 con 0x7f831c103250 2024-05-09T04:43:19.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.355+0000 7f83227c2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f831c10eb30 con 0x7f831c104d20 2024-05-09T04:43:19.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.355+0000 7f83227c2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f831c10ee30 con 0x7f831c104380 2024-05-09T04:43:19.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.355+0000 7f8320d5f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f831c104d20 0x7f831c1a1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:43:19.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.355+0000 7f8320d5f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f831c104d20 0x7f831c1a1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:43562/0 (socket says 172.21.15.72:43562) 2024-05-09T04:43:19.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.355+0000 7f8320d5f700 1 -- 172.21.15.72:0/3249949084 learned_addr learned my addr 172.21.15.72:0/3249949084 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-05-09T04:43:19.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.355+0000 7f831b7fe700 1 --2- 172.21.15.72:0/3249949084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c104380 0x7f831c19ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:43:19.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.355+0000 7f831bfff700 1 --2- 172.21.15.72:0/3249949084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f831c103250 0x7f831c19a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:43:19.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.356+0000 7f8320d5f700 1 -- 172.21.15.72:0/3249949084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c104380 msgr2=0x7f831c19ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:19.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.356+0000 7f8320d5f700 1 --2- 172.21.15.72:0/3249949084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c104380 0x7f831c19ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:19.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.356+0000 7f8320d5f700 1 -- 172.21.15.72:0/3249949084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f831c103250 msgr2=0x7f831c19a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:19.369 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.356+0000 7f8320d5f700 1 --2- 172.21.15.72:0/3249949084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f831c103250 0x7f831c19a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:19.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.356+0000 7f8320d5f700 1 -- 172.21.15.72:0/3249949084 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f831c1a2480 con 0x7f831c104d20 2024-05-09T04:43:19.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.356+0000 7f8320d5f700 1 --2- 172.21.15.72:0/3249949084 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f831c104d20 0x7f831c1a1e00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8314007ee0 tx=0x7f831400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:43:19.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.356+0000 7f83197fa700 1 -- 172.21.15.72:0/3249949084 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831400ddd0 con 0x7f831c104d20 2024-05-09T04:43:19.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.356+0000 7f83227c2700 1 -- 172.21.15.72:0/3249949084 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f831c1a2770 con 0x7f831c104d20 2024-05-09T04:43:19.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.356+0000 7f83197fa700 1 -- 172.21.15.72:0/3249949084 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8314004020 con 0x7f831c104d20 2024-05-09T04:43:19.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.356+0000 7f83197fa700 1 -- 172.21.15.72:0/3249949084 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8314015660 con 0x7f831c104d20 2024-05-09T04:43:19.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.356+0000 7f83227c2700 1 -- 172.21.15.72:0/3249949084 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f831c1a2cb0 con 0x7f831c104d20 2024-05-09T04:43:19.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.357+0000 7f83227c2700 1 -- 172.21.15.72:0/3249949084 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f831c0512c0 con 0x7f831c104d20 2024-05-09T04:43:19.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.358+0000 7f83197fa700 1 -- 172.21.15.72:0/3249949084 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f83140041d0 con 0x7f831c104d20 2024-05-09T04:43:19.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.359+0000 7f83197fa700 1 --2- 172.21.15.72:0/3249949084 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f83040b2880 0x7f83040b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:19.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.359+0000 7f83197fa700 1 -- 172.21.15.72:0/3249949084 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(29..29 src has 1..29) ==== 3669+0+0 (secure 0 0 0) 0x7f83140e0a00 con 0x7f831c104d20 2024-05-09T04:43:19.370 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.359+0000 7f831bfff700 1 --2- 172.21.15.72:0/3249949084 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f83040b2880 0x7f83040b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:43:19.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.360+0000 7f831bfff700 1 --2- 172.21.15.72:0/3249949084 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f83040b2880 0x7f83040b4d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f831c104280 tx=0x7f830c00b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:43:19.371 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.364+0000 7f83197fa700 1 -- 172.21.15.72:0/3249949084 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8314010070 con 0x7f831c104d20 2024-05-09T04:43:19.572 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:19.572+0000 7f83227c2700 1 -- 172.21.15.72:0/3249949084 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f831c1008e0 con 0x7f83040b2880 2024-05-09T04:43:19.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:19 smithi072 ceph-mon[34088]: Detected new or changed devices on smithi046 2024-05-09T04:43:19.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:19 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi046 to 8546M 2024-05-09T04:43:19.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:19 smithi072 ceph-mon[34088]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T04:43:19.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:19 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:43:19.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:19 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:43:19.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:19 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:19.937 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:19 smithi046 ceph-mon[30136]: Detected new or changed devices on smithi046 2024-05-09T04:43:19.937 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:19 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi046 to 8546M 2024-05-09T04:43:19.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:19 smithi046 ceph-mon[30136]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T04:43:19.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:19 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:43:19.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:19 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:43:19.938 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:19 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:20.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:19 smithi110 ceph-mon[34904]: Detected new or changed devices on smithi046 2024-05-09T04:43:20.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:19 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi046 to 8546M 2024-05-09T04:43:20.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:19 smithi110 ceph-mon[34904]: osdmap e29: 4 total, 4 up, 4 in 2024-05-09T04:43:20.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:19 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:43:20.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:19 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:43:20.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:19 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:20.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:20 smithi072 ceph-mon[34088]: pgmap v105: 1 pgs: 1 activating+degraded; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T04:43:20.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:20 smithi072 ceph-mon[34088]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:43:20.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:20 smithi072 ceph-mon[34088]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-09T04:43:20.916 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:20 smithi046 ceph-mon[30136]: pgmap v105: 1 pgs: 1 activating+degraded; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T04:43:20.917 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:20 smithi046 ceph-mon[30136]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:43:20.917 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:20 smithi046 ceph-mon[30136]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-09T04:43:21.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:20 smithi110 ceph-mon[34904]: pgmap v105: 1 pgs: 1 activating+degraded; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T04:43:21.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:20 smithi110 ceph-mon[34904]: from='client.24212 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:43:21.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:20 smithi110 ceph-mon[34904]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-09T04:43:22.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:22 smithi072 ceph-mon[34088]: pgmap v106: 1 pgs: 1 activating+degraded; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T04:43:22.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:22 smithi072 ceph-mon[34088]: from='client.? 172.21.15.72:0/3751396260' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "444cd36b-6034-46cd-9a6b-29dce4ba6405"}]: dispatch 2024-05-09T04:43:22.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:22 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "444cd36b-6034-46cd-9a6b-29dce4ba6405"}]: dispatch 2024-05-09T04:43:22.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:22 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "444cd36b-6034-46cd-9a6b-29dce4ba6405"}]': finished 2024-05-09T04:43:22.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:22 smithi072 ceph-mon[34088]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T04:43:22.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:22 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:43:23.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:22 smithi110 ceph-mon[34904]: pgmap v106: 1 pgs: 1 activating+degraded; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T04:43:23.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:22 smithi110 ceph-mon[34904]: from='client.? 172.21.15.72:0/3751396260' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "444cd36b-6034-46cd-9a6b-29dce4ba6405"}]: dispatch 2024-05-09T04:43:23.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:22 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "444cd36b-6034-46cd-9a6b-29dce4ba6405"}]: dispatch 2024-05-09T04:43:23.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:22 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "444cd36b-6034-46cd-9a6b-29dce4ba6405"}]': finished 2024-05-09T04:43:23.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:22 smithi110 ceph-mon[34904]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T04:43:23.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:22 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:43:23.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:22 smithi046 ceph-mon[30136]: pgmap v106: 1 pgs: 1 activating+degraded; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T04:43:23.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:22 smithi046 ceph-mon[30136]: from='client.? 172.21.15.72:0/3751396260' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "444cd36b-6034-46cd-9a6b-29dce4ba6405"}]: dispatch 2024-05-09T04:43:23.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:22 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "444cd36b-6034-46cd-9a6b-29dce4ba6405"}]: dispatch 2024-05-09T04:43:23.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:22 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "444cd36b-6034-46cd-9a6b-29dce4ba6405"}]': finished 2024-05-09T04:43:23.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:22 smithi046 ceph-mon[30136]: osdmap e30: 5 total, 4 up, 5 in 2024-05-09T04:43:23.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:22 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:43:23.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:23 smithi072 ceph-mon[34088]: from='client.? 172.21.15.72:0/2910290758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T04:43:24.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:23 smithi110 ceph-mon[34904]: from='client.? 172.21.15.72:0/2910290758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T04:43:24.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:23 smithi046 ceph-mon[30136]: from='client.? 172.21.15.72:0/2910290758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-4/activate.monmap"}]: dispatch 2024-05-09T04:43:24.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:24 smithi072 ceph-mon[34088]: pgmap v108: 1 pgs: 1 activating+degraded; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T04:43:25.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:24 smithi110 ceph-mon[34904]: pgmap v108: 1 pgs: 1 activating+degraded; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T04:43:25.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:24 smithi046 ceph-mon[30136]: pgmap v108: 1 pgs: 1 activating+degraded; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 2/6 objects degraded (33.333%) 2024-05-09T04:43:25.970 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:25 smithi072 ceph-mon[34088]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-09T04:43:25.971 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:25 smithi072 ceph-mon[34088]: Cluster is now healthy 2024-05-09T04:43:26.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:25 smithi110 ceph-mon[34904]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-09T04:43:26.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:25 smithi110 ceph-mon[34904]: Cluster is now healthy 2024-05-09T04:43:26.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:25 smithi046 ceph-mon[30136]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-09T04:43:26.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:25 smithi046 ceph-mon[30136]: Cluster is now healthy 2024-05-09T04:43:26.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:26 smithi072 ceph-mon[34088]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-09T04:43:27.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:26 smithi110 ceph-mon[34904]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-09T04:43:27.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:26 smithi046 ceph-mon[30136]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-09T04:43:28.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:28 smithi072 ceph-mon[34088]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-09T04:43:29.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:28 smithi110 ceph-mon[34904]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-09T04:43:29.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:28 smithi046 ceph-mon[30136]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-09T04:43:30.992 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:30 smithi046 ceph-mon[30136]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:43:30.992 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:30 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T04:43:30.992 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:30 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:31.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:30 smithi110 ceph-mon[34904]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:43:31.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:30 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T04:43:31.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:30 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:31.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:30 smithi072 ceph-mon[34088]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:43:31.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:30 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-09T04:43:31.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:30 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:32.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:31 smithi110 ceph-mon[34904]: Deploying daemon osd.4 on smithi072 2024-05-09T04:43:32.066 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:31 smithi072 ceph-mon[34088]: Deploying daemon osd.4 on smithi072 2024-05-09T04:43:32.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:31 smithi046 ceph-mon[30136]: Deploying daemon osd.4 on smithi072 2024-05-09T04:43:33.042 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:32 smithi072 ceph-mon[34088]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:43:33.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:32 smithi110 ceph-mon[34904]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:43:33.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:32 smithi046 ceph-mon[30136]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:43:34.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:34 smithi072 ceph-mon[34088]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-09T04:43:35.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:34 smithi110 ceph-mon[34904]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-09T04:43:35.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:34 smithi046 ceph-mon[30136]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-05-09T04:43:36.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:36 smithi072 ceph-mon[34088]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T04:43:36.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:36.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:36.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:36.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:36 smithi110 ceph-mon[34904]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T04:43:36.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:36.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:36.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:36.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:36 smithi046 ceph-mon[30136]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T04:43:36.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:36.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:36.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:38.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:38 smithi110 ceph-mon[34904]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:43:38.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:38 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:38.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:38 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:38.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:38 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:38.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:38 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:38.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:38 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:38.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:38 smithi046 ceph-mon[30136]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:43:38.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:38 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:38.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:38 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:38.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:38 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:38.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:38 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:38.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:38 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:38.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:38 smithi072 ceph-mon[34088]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:43:38.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:38 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:38.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:38 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:38.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:38 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:38.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:38 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:38.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:38 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:39.394 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:39.394+0000 7f83197fa700 1 -- 172.21.15.72:0/3249949084 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f831c1008e0 con 0x7f83040b2880 2024-05-09T04:43:39.394 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 4 on host 'smithi072' 2024-05-09T04:43:39.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:39.396+0000 7f83227c2700 1 -- 172.21.15.72:0/3249949084 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f83040b2880 msgr2=0x7f83040b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:39.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:39.396+0000 7f83227c2700 1 --2- 172.21.15.72:0/3249949084 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f83040b2880 0x7f83040b4d90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f831c104280 tx=0x7f830c00b5b0 comp rx=0 tx=0).stop 2024-05-09T04:43:39.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:39.397+0000 7f83227c2700 1 -- 172.21.15.72:0/3249949084 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f831c104d20 msgr2=0x7f831c1a1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:39.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:39.397+0000 7f83227c2700 1 --2- 172.21.15.72:0/3249949084 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f831c104d20 0x7f831c1a1e00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f8314007ee0 tx=0x7f831400d3b0 comp rx=0 tx=0).stop 2024-05-09T04:43:39.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:39.397+0000 7f83227c2700 1 -- 172.21.15.72:0/3249949084 shutdown_connections 2024-05-09T04:43:39.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:39.397+0000 7f83227c2700 1 --2- 172.21.15.72:0/3249949084 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f83040b2880 0x7f83040b4d90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:39.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:39.397+0000 7f83227c2700 1 --2- 172.21.15.72:0/3249949084 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f831c104d20 0x7f831c1a1e00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:39.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:39.397+0000 7f83227c2700 1 --2- 172.21.15.72:0/3249949084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f831c103250 0x7f831c19a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:39.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:39.397+0000 7f83227c2700 1 --2- 172.21.15.72:0/3249949084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f831c104380 0x7f831c19ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:39.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:39.397+0000 7f83227c2700 1 -- 172.21.15.72:0/3249949084 >> 172.21.15.72:0/3249949084 conn(0x7f831c0fe890 msgr2=0x7f831c1001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:43:39.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:39.397+0000 7f83227c2700 1 -- 172.21.15.72:0/3249949084 shutdown_connections 2024-05-09T04:43:39.397 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:43:39.397+0000 7f83227c2700 1 -- 172.21.15.72:0/3249949084 wait complete. 2024-05-09T04:43:40.068 DEBUG:teuthology.orchestra.run.smithi072:osd.4> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.4.service 2024-05-09T04:43:40.070 INFO:tasks.cephadm:Deploying osd.5 on smithi110 with /dev/vg_nvme/lv_3... 2024-05-09T04:43:40.070 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-09T04:43:40.314 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:43:40.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:40 smithi072 ceph-mon[34088]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:43:40.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:40 smithi072 ceph-mon[34088]: from='osd.4 [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T04:43:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:40 smithi072 ceph-mon[34088]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T04:43:40.636 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:40 smithi110 ceph-mon[34904]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:43:40.636 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:40.636 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:40.636 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:40.636 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:40 smithi110 ceph-mon[34904]: from='osd.4 [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T04:43:40.636 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:40 smithi110 ceph-mon[34904]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T04:43:40.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:40 smithi046 ceph-mon[30136]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:43:40.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:40.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:40.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:43:40.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:40 smithi046 ceph-mon[30136]: from='osd.4 [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T04:43:40.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:40 smithi046 ceph-mon[30136]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-09T04:43:41.540 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:41 smithi072 ceph-mon[34088]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T04:43:41.540 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:41 smithi072 ceph-mon[34088]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T04:43:41.540 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:41 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:43:41.540 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:41 smithi072 ceph-mon[34088]: from='osd.4 [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:43:41.540 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:41 smithi072 ceph-mon[34088]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:43:41.540 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 04:43:41 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4[45207]: 2024-05-09T04:43:41.378+0000 7f9bea80e700 -1 osd.4 0 waiting for initial osdmap 2024-05-09T04:43:41.541 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 04:43:41 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4[45207]: 2024-05-09T04:43:41.384+0000 7f9be65c3700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:43:41.631 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:41 smithi110 ceph-mon[34904]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T04:43:41.632 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:41 smithi110 ceph-mon[34904]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T04:43:41.632 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:41 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:43:41.632 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:41 smithi110 ceph-mon[34904]: from='osd.4 [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:43:41.632 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:41 smithi110 ceph-mon[34904]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:43:41.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:41 smithi046 ceph-mon[30136]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-09T04:43:41.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:41 smithi046 ceph-mon[30136]: osdmap e31: 5 total, 4 up, 5 in 2024-05-09T04:43:41.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:41 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:43:41.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:41 smithi046 ceph-mon[30136]: from='osd.4 [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:43:41.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:41 smithi046 ceph-mon[30136]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:43:42.288 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-09T04:43:42.288 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-09T04:43:42.288 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-05-09T04:43:42.288 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-05-09T04:43:42.288 INFO:teuthology.orchestra.run.smithi110.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00809352 s, 1.3 GB/s 2024-05-09T04:43:42.289 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 2024-05-09T04:43:42.289 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-05-09T04:43:42.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:42 smithi110 ceph-mon[34904]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:43:42.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:42 smithi110 ceph-mon[34904]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-09T04:43:42.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:42 smithi110 ceph-mon[34904]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T04:43:42.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:42 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:43:42.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:42 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:43:42.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:42 smithi072 ceph-mon[34088]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:43:42.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:42 smithi072 ceph-mon[34088]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-09T04:43:42.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:42 smithi072 ceph-mon[34088]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T04:43:42.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:42 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:43:42.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:42 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:43:42.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:42 smithi046 ceph-mon[30136]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 119 MiB used, 357 GiB / 358 GiB avail 2024-05-09T04:43:42.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:42 smithi046 ceph-mon[30136]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-09T04:43:42.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:42 smithi046 ceph-mon[30136]: osdmap e32: 5 total, 4 up, 5 in 2024-05-09T04:43:42.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:42 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:43:42.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:42 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:43:42.966 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_3 2024-05-09T04:43:43.218 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:43:43.592 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:43 smithi110 ceph-mon[34904]: purged_snaps scrub starts 2024-05-09T04:43:43.592 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:43 smithi110 ceph-mon[34904]: purged_snaps scrub ok 2024-05-09T04:43:43.592 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:43 smithi110 ceph-mon[34904]: osd.4 [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693] boot 2024-05-09T04:43:43.592 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:43 smithi110 ceph-mon[34904]: osdmap e33: 5 total, 5 up, 5 in 2024-05-09T04:43:43.592 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:43:43.592 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:43.593 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:43.593 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:43.593 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:43.593 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:43.593 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:43.593 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:43.593 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:43.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:43 smithi046 ceph-mon[30136]: purged_snaps scrub starts 2024-05-09T04:43:43.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:43 smithi046 ceph-mon[30136]: purged_snaps scrub ok 2024-05-09T04:43:43.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:43 smithi046 ceph-mon[30136]: osd.4 [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693] boot 2024-05-09T04:43:43.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:43 smithi046 ceph-mon[30136]: osdmap e33: 5 total, 5 up, 5 in 2024-05-09T04:43:43.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:43:43.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:43.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:43.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:43.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:43.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:43.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:43.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:43.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:43.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:43 smithi072 ceph-mon[34088]: purged_snaps scrub starts 2024-05-09T04:43:43.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:43 smithi072 ceph-mon[34088]: purged_snaps scrub ok 2024-05-09T04:43:43.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:43 smithi072 ceph-mon[34088]: osd.4 [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693] boot 2024-05-09T04:43:43.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:43 smithi072 ceph-mon[34088]: osdmap e33: 5 total, 5 up, 5 in 2024-05-09T04:43:43.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:43:43.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:43.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:43.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:43.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:43:43.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:43.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:43.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:43:43.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:43:44.657 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:44 smithi110 ceph-mon[34904]: Detected new or changed devices on smithi072 2024-05-09T04:43:44.657 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:44 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi072 to 8544M 2024-05-09T04:43:44.657 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:44 smithi110 ceph-mon[34904]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:44.657 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:44 smithi110 ceph-mon[34904]: osdmap e34: 5 total, 5 up, 5 in 2024-05-09T04:43:44.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:44 smithi046 ceph-mon[30136]: Detected new or changed devices on smithi072 2024-05-09T04:43:44.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:44 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi072 to 8544M 2024-05-09T04:43:44.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:44 smithi046 ceph-mon[30136]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:44.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:44 smithi046 ceph-mon[30136]: osdmap e34: 5 total, 5 up, 5 in 2024-05-09T04:43:44.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:44 smithi072 ceph-mon[34088]: Detected new or changed devices on smithi072 2024-05-09T04:43:44.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:44 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi072 to 8544M 2024-05-09T04:43:44.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:44 smithi072 ceph-mon[34088]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:44.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:44 smithi072 ceph-mon[34088]: osdmap e34: 5 total, 5 up, 5 in 2024-05-09T04:43:44.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.904+0000 7fbd4d050700 1 -- 172.21.15.110:0/2966810554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd48102350 msgr2=0x7fbd48102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:44.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.904+0000 7fbd4d050700 1 --2- 172.21.15.110:0/2966810554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd48102350 0x7fbd48102760 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c009990 tx=0x7fbd3c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:43:44.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.904+0000 7fbd4d050700 1 -- 172.21.15.110:0/2966810554 shutdown_connections 2024-05-09T04:43:44.904 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.904+0000 7fbd4d050700 1 --2- 172.21.15.110:0/2966810554 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd48102ca0 0x7fbd4810c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:44.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.904+0000 7fbd4d050700 1 --2- 172.21.15.110:0/2966810554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd48105900 0x7fbd48105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:44.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.904+0000 7fbd4d050700 1 --2- 172.21.15.110:0/2966810554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd48102350 0x7fbd48102760 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:44.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.904+0000 7fbd4d050700 1 -- 172.21.15.110:0/2966810554 >> 172.21.15.110:0/2966810554 conn(0x7fbd480fe000 msgr2=0x7fbd481004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:43:44.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.904+0000 7fbd4d050700 1 -- 172.21.15.110:0/2966810554 shutdown_connections 2024-05-09T04:43:44.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.904+0000 7fbd4d050700 1 -- 172.21.15.110:0/2966810554 wait complete. 2024-05-09T04:43:44.905 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.904+0000 7fbd4d050700 1 Processor -- start 2024-05-09T04:43:44.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4d050700 1 -- start start 2024-05-09T04:43:44.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4d050700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd48102350 0x7fbd4819a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:44.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4d050700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd48102ca0 0x7fbd4819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:44.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4d050700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd48105900 0x7fbd481a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:44.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4d050700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4810fe40 con 0x7fbd48102350 2024-05-09T04:43:44.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4d050700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4810fcc0 con 0x7fbd48105900 2024-05-09T04:43:44.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4d050700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbd4810ffc0 con 0x7fbd48102ca0 2024-05-09T04:43:44.907 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd48102ca0 0x7fbd4819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:43:44.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd48102ca0 0x7fbd4819ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:34966/0 (socket says 172.21.15.110:34966) 2024-05-09T04:43:44.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4659c700 1 -- 172.21.15.110:0/2079452471 learned_addr learned my addr 172.21.15.110:0/2079452471 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-09T04:43:44.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd46d9d700 1 --2- 172.21.15.110:0/2079452471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd48102350 0x7fbd4819a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:43:44.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4759e700 1 --2- 172.21.15.110:0/2079452471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd48105900 0x7fbd481a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:43:44.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4659c700 1 -- 172.21.15.110:0/2079452471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd48105900 msgr2=0x7fbd481a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:44.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4659c700 1 --2- 172.21.15.110:0/2079452471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd48105900 0x7fbd481a1e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:44.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4659c700 1 -- 172.21.15.110:0/2079452471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd48102350 msgr2=0x7fbd4819a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:43:44.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4659c700 1 --2- 172.21.15.110:0/2079452471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd48102350 0x7fbd4819a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:43:44.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4659c700 1 -- 172.21.15.110:0/2079452471 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd481a24b0 con 0x7fbd48102ca0 2024-05-09T04:43:44.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4759e700 1 --2- 172.21.15.110:0/2079452471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd48105900 0x7fbd481a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:43:44.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.905+0000 7fbd4659c700 1 --2- 172.21.15.110:0/2079452471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd48102ca0 0x7fbd4819ae90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c006010 tx=0x7fbd3c0047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:43:44.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.906+0000 7fbd2ffff700 1 -- 172.21.15.110:0/2079452471 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c027070 con 0x7fbd48102ca0 2024-05-09T04:43:44.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.906+0000 7fbd2ffff700 1 -- 172.21.15.110:0/2079452471 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd3c0040c0 con 0x7fbd48102ca0 2024-05-09T04:43:44.908 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.906+0000 7fbd4d050700 1 -- 172.21.15.110:0/2079452471 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd481a2740 con 0x7fbd48102ca0 2024-05-09T04:43:44.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.906+0000 7fbd2ffff700 1 -- 172.21.15.110:0/2079452471 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c022d90 con 0x7fbd48102ca0 2024-05-09T04:43:44.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.906+0000 7fbd4d050700 1 -- 172.21.15.110:0/2079452471 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd481a2c10 con 0x7fbd48102ca0 2024-05-09T04:43:44.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.907+0000 7fbd4d050700 1 -- 172.21.15.110:0/2079452471 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd480512c0 con 0x7fbd48102ca0 2024-05-09T04:43:44.909 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.908+0000 7fbd2ffff700 1 -- 172.21.15.110:0/2079452471 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbd3c02a070 con 0x7fbd48102ca0 2024-05-09T04:43:44.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.909+0000 7fbd2ffff700 1 --2- 172.21.15.110:0/2079452471 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbd300b2880 0x7fbd300b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:43:44.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.909+0000 7fbd2ffff700 1 -- 172.21.15.110:0/2079452471 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7fbd3c0e2360 con 0x7fbd48102ca0 2024-05-09T04:43:44.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.909+0000 7fbd46d9d700 1 --2- 172.21.15.110:0/2079452471 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbd300b2880 0x7fbd300b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:43:44.910 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.910+0000 7fbd46d9d700 1 --2- 172.21.15.110:0/2079452471 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbd300b2880 0x7fbd300b4d90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbd4806c040 tx=0x7fbd3800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:43:44.913 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:44.913+0000 7fbd2ffff700 1 -- 172.21.15.110:0/2079452471 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbd3c013070 con 0x7fbd48102ca0 2024-05-09T04:43:45.125 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:43:45.125+0000 7fbd4d050700 1 -- 172.21.15.110:0/2079452471 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbd48063c00 con 0x7fbd300b2880 2024-05-09T04:43:45.738 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:45 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:43:45.738 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:45 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:43:45.738 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:45 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:45.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:45 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:43:45.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:45 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:43:45.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:45 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:45.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:45 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:43:45.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:45 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:43:45.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:45 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:46.755 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:46 smithi110 ceph-mon[34904]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:43:46.755 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:46 smithi110 ceph-mon[34904]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:46.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:46 smithi046 ceph-mon[30136]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:43:46.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:46 smithi046 ceph-mon[30136]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:46.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:46 smithi072 ceph-mon[34088]: from='client.24241 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:43:46.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:46 smithi072 ceph-mon[34088]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:48.682 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:48 smithi072 ceph-mon[34088]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:48.682 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:48 smithi072 ceph-mon[34088]: from='client.? 172.21.15.110:0/3724371532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c2bdd2f-43cf-4208-9ae1-f912807c84e0"}]: dispatch 2024-05-09T04:43:48.682 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:48 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c2bdd2f-43cf-4208-9ae1-f912807c84e0"}]: dispatch 2024-05-09T04:43:48.682 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:48 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c2bdd2f-43cf-4208-9ae1-f912807c84e0"}]': finished 2024-05-09T04:43:48.682 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:48 smithi072 ceph-mon[34088]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T04:43:48.682 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:43:48.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:48 smithi110 ceph-mon[34904]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:48.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:48 smithi110 ceph-mon[34904]: from='client.? 172.21.15.110:0/3724371532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c2bdd2f-43cf-4208-9ae1-f912807c84e0"}]: dispatch 2024-05-09T04:43:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:48 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c2bdd2f-43cf-4208-9ae1-f912807c84e0"}]: dispatch 2024-05-09T04:43:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:48 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c2bdd2f-43cf-4208-9ae1-f912807c84e0"}]': finished 2024-05-09T04:43:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:48 smithi110 ceph-mon[34904]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T04:43:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:43:48.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:48 smithi046 ceph-mon[30136]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:48.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:48 smithi046 ceph-mon[30136]: from='client.? 172.21.15.110:0/3724371532' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c2bdd2f-43cf-4208-9ae1-f912807c84e0"}]: dispatch 2024-05-09T04:43:48.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:48 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2c2bdd2f-43cf-4208-9ae1-f912807c84e0"}]: dispatch 2024-05-09T04:43:48.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:48 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2c2bdd2f-43cf-4208-9ae1-f912807c84e0"}]': finished 2024-05-09T04:43:48.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:48 smithi046 ceph-mon[30136]: osdmap e35: 6 total, 5 up, 6 in 2024-05-09T04:43:48.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:43:49.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:49 smithi110 ceph-mon[34904]: from='client.? 172.21.15.110:0/2339617396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T04:43:49.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:49 smithi046 ceph-mon[30136]: from='client.? 172.21.15.110:0/2339617396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T04:43:49.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:49 smithi072 ceph-mon[34088]: from='client.? 172.21.15.110:0/2339617396' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-5/activate.monmap"}]: dispatch 2024-05-09T04:43:50.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:50 smithi110 ceph-mon[34904]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:50.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:50 smithi046 ceph-mon[30136]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:50.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:50 smithi072 ceph-mon[34088]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:52.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:52 smithi110 ceph-mon[34904]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:52.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:52 smithi046 ceph-mon[30136]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:52.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:52 smithi072 ceph-mon[34088]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:54.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:54 smithi110 ceph-mon[34904]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:54.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:54 smithi046 ceph-mon[30136]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:54.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:54 smithi072 ceph-mon[34088]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:55.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:55 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T04:43:55.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:55 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:55.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:55 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T04:43:55.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:55 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:55.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:55 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-09T04:43:55.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:55 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:43:56.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:56 smithi110 ceph-mon[34904]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:56.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:56 smithi110 ceph-mon[34904]: Deploying daemon osd.5 on smithi110 2024-05-09T04:43:56.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:56 smithi046 ceph-mon[30136]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:56.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:56 smithi046 ceph-mon[30136]: Deploying daemon osd.5 on smithi110 2024-05-09T04:43:56.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:56 smithi072 ceph-mon[34088]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:56.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:56 smithi072 ceph-mon[34088]: Deploying daemon osd.5 on smithi110 2024-05-09T04:43:58.456 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:43:58 smithi110 ceph-mon[34904]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:58.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:43:58 smithi046 ceph-mon[30136]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:43:58.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:43:58 smithi072 ceph-mon[34088]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:00.523 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:00 smithi110 ceph-mon[34904]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:00.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:00 smithi046 ceph-mon[30136]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:00.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:00 smithi072 ceph-mon[34088]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:02.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:02.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:02.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:02.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:02 smithi110 ceph-mon[34904]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:02.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:02.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:02.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:02.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:02 smithi072 ceph-mon[34088]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:02.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:02.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:02.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:02.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:02 smithi046 ceph-mon[30136]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:04.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:04 smithi110 ceph-mon[34904]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:04.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:04 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:04.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:04 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:04.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:04 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:04.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:04 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:04.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:04 smithi046 ceph-mon[30136]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:04.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:04 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:04.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:04 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:04.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:04 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:04.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:04 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:04.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:04 smithi072 ceph-mon[34088]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:04.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:04 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:04.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:04 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:04.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:04 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:04.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:04 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:04.750 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:44:04.748+0000 7fbd2ffff700 1 -- 172.21.15.110:0/2079452471 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbd48063c00 con 0x7fbd300b2880 2024-05-09T04:44:04.750 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 5 on host 'smithi110' 2024-05-09T04:44:04.751 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:44:04.751+0000 7fbd4d050700 1 -- 172.21.15.110:0/2079452471 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbd300b2880 msgr2=0x7fbd300b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:04.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:44:04.751+0000 7fbd4d050700 1 --2- 172.21.15.110:0/2079452471 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbd300b2880 0x7fbd300b4d90 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fbd4806c040 tx=0x7fbd3800b410 comp rx=0 tx=0).stop 2024-05-09T04:44:04.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:44:04.751+0000 7fbd4d050700 1 -- 172.21.15.110:0/2079452471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd48102ca0 msgr2=0x7fbd4819ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:04.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:44:04.751+0000 7fbd4d050700 1 --2- 172.21.15.110:0/2079452471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd48102ca0 0x7fbd4819ae90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c006010 tx=0x7fbd3c0047b0 comp rx=0 tx=0).stop 2024-05-09T04:44:04.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:44:04.751+0000 7fbd4d050700 1 -- 172.21.15.110:0/2079452471 shutdown_connections 2024-05-09T04:44:04.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:44:04.751+0000 7fbd4d050700 1 --2- 172.21.15.110:0/2079452471 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbd300b2880 0x7fbd300b4d90 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:04.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:44:04.751+0000 7fbd4d050700 1 --2- 172.21.15.110:0/2079452471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbd48105900 0x7fbd481a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:04.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:44:04.751+0000 7fbd4d050700 1 --2- 172.21.15.110:0/2079452471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbd48102350 0x7fbd4819a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:04.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:44:04.751+0000 7fbd4d050700 1 --2- 172.21.15.110:0/2079452471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbd48102ca0 0x7fbd4819ae90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:04.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:44:04.751+0000 7fbd4d050700 1 -- 172.21.15.110:0/2079452471 >> 172.21.15.110:0/2079452471 conn(0x7fbd480fe000 msgr2=0x7fbd481088a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:44:04.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:44:04.752+0000 7fbd4d050700 1 -- 172.21.15.110:0/2079452471 shutdown_connections 2024-05-09T04:44:04.752 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:44:04.752+0000 7fbd4d050700 1 -- 172.21.15.110:0/2079452471 wait complete. 2024-05-09T04:44:05.424 DEBUG:teuthology.orchestra.run.smithi110:osd.5> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.5.service 2024-05-09T04:44:05.426 INFO:tasks.cephadm:Deploying osd.6 on smithi046 with /dev/vg_nvme/lv_2... 2024-05-09T04:44:05.426 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T04:44:05.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:05 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:05.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:05 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:05.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:05 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:05.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:05 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:05.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:05 smithi110 ceph-mon[34904]: from='osd.5 [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T04:44:05.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:05 smithi110 ceph-mon[34904]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T04:44:05.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:05.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:05.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:05.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:05 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:05.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:05 smithi046 ceph-mon[30136]: from='osd.5 [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T04:44:05.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:05 smithi046 ceph-mon[30136]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T04:44:05.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:05 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:05.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:05 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:05.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:05 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:05.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:05 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:05.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:05 smithi072 ceph-mon[34088]: from='osd.5 [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T04:44:05.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:05 smithi072 ceph-mon[34088]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-09T04:44:05.686 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:44:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:06 smithi046 ceph-mon[30136]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:06.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:06 smithi046 ceph-mon[30136]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T04:44:06.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:06 smithi046 ceph-mon[30136]: from='osd.5 [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:44:06.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:06 smithi046 ceph-mon[30136]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T04:44:06.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:06 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:44:06.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:06 smithi046 ceph-mon[30136]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:44:06.995 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:06 smithi110 ceph-mon[34904]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:06.995 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:06 smithi110 ceph-mon[34904]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T04:44:06.995 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:06 smithi110 ceph-mon[34904]: from='osd.5 [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:44:06.995 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:06 smithi110 ceph-mon[34904]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T04:44:06.995 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:06 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:44:06.995 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:06 smithi110 ceph-mon[34904]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:44:07.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:06 smithi072 ceph-mon[34088]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:07.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:06 smithi072 ceph-mon[34088]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-09T04:44:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:06 smithi072 ceph-mon[34088]: from='osd.5 [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:44:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:06 smithi072 ceph-mon[34088]: osdmap e36: 6 total, 5 up, 6 in 2024-05-09T04:44:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:06 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:44:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:06 smithi072 ceph-mon[34088]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:44:07.306 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 04:44:07 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T04:44:07.206+0000 7fe8f1d20700 -1 osd.5 0 waiting for initial osdmap 2024-05-09T04:44:07.307 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 04:44:07 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T04:44:07.214+0000 7fe8e9acd700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:44:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T04:44:07.649 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T04:44:07.650 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-05-09T04:44:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-05-09T04:44:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00836114 s, 1.3 GB/s 2024-05-09T04:44:07.650 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 2024-05-09T04:44:07.650 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-05-09T04:44:08.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:07 smithi046 ceph-mon[30136]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-09T04:44:08.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:07 smithi046 ceph-mon[30136]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T04:44:08.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:07 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:44:08.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:07 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:44:08.043 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:07 smithi046 ceph-mon[30136]: from='osd.5 ' entity='osd.5' 2024-05-09T04:44:08.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:07 smithi110 ceph-mon[34904]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-09T04:44:08.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:07 smithi110 ceph-mon[34904]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T04:44:08.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:07 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:44:08.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:07 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:44:08.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:07 smithi110 ceph-mon[34904]: from='osd.5 ' entity='osd.5' 2024-05-09T04:44:08.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:07 smithi072 ceph-mon[34088]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-09T04:44:08.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:07 smithi072 ceph-mon[34088]: osdmap e37: 6 total, 5 up, 6 in 2024-05-09T04:44:08.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:07 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:44:08.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:07 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:44:08.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:07 smithi072 ceph-mon[34088]: from='osd.5 ' entity='osd.5' 2024-05-09T04:44:08.328 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_2 2024-05-09T04:44:08.593 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:44:08.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:08 smithi046 ceph-mon[30136]: purged_snaps scrub starts 2024-05-09T04:44:08.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:08 smithi046 ceph-mon[30136]: purged_snaps scrub ok 2024-05-09T04:44:08.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:08 smithi046 ceph-mon[30136]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:08.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:08 smithi046 ceph-mon[30136]: osd.5 [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502] boot 2024-05-09T04:44:08.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:08 smithi046 ceph-mon[30136]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T04:44:08.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:44:08.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:08.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:08.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:08.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:08.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:08.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:08.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:08.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:08 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:09.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:08 smithi110 ceph-mon[34904]: purged_snaps scrub starts 2024-05-09T04:44:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:08 smithi110 ceph-mon[34904]: purged_snaps scrub ok 2024-05-09T04:44:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:08 smithi110 ceph-mon[34904]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:08 smithi110 ceph-mon[34904]: osd.5 [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502] boot 2024-05-09T04:44:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:08 smithi110 ceph-mon[34904]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T04:44:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:08 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:44:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:08 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:08 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:08 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:08 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:09.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:08 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:09.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:08 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:09.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:08 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:09.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:08 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:08 smithi072 ceph-mon[34088]: purged_snaps scrub starts 2024-05-09T04:44:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:08 smithi072 ceph-mon[34088]: purged_snaps scrub ok 2024-05-09T04:44:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:08 smithi072 ceph-mon[34088]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-09T04:44:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:08 smithi072 ceph-mon[34088]: osd.5 [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502] boot 2024-05-09T04:44:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:08 smithi072 ceph-mon[34088]: osdmap e38: 6 total, 6 up, 6 in 2024-05-09T04:44:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:08 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:44:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:08 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:08 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:08 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:09.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:08 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:09.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:08 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:09.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:08 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:09.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:08 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:09.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:08 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:09.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:09 smithi046 ceph-mon[30136]: Detected new or changed devices on smithi110 2024-05-09T04:44:09.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:09 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi110 to 8544M 2024-05-09T04:44:09.905 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:09 smithi046 ceph-mon[30136]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T04:44:10.000 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:09 smithi110 ceph-mon[34904]: Detected new or changed devices on smithi110 2024-05-09T04:44:10.001 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:09 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi110 to 8544M 2024-05-09T04:44:10.001 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:09 smithi110 ceph-mon[34904]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T04:44:10.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:09 smithi072 ceph-mon[34088]: Detected new or changed devices on smithi110 2024-05-09T04:44:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:09 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi110 to 8544M 2024-05-09T04:44:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:09 smithi072 ceph-mon[34088]: osdmap e39: 6 total, 6 up, 6 in 2024-05-09T04:44:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.131+0000 7f823fa0e700 1 -- 172.21.15.46:0/4236497847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8238104e40 msgr2=0x7f8238107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.131+0000 7f823fa0e700 1 --2- 172.21.15.46:0/4236497847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8238104e40 0x7f8238107260 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f822800b0a0 tx=0x7f822800b3b0 comp rx=0 tx=0).stop 2024-05-09T04:44:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.131+0000 7f823fa0e700 1 -- 172.21.15.46:0/4236497847 shutdown_connections 2024-05-09T04:44:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.131+0000 7f823fa0e700 1 --2- 172.21.15.46:0/4236497847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f823810a100 0x7f823810c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.131+0000 7f823fa0e700 1 --2- 172.21.15.46:0/4236497847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8238104e40 0x7f8238107260 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:10.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.131+0000 7f823fa0e700 1 --2- 172.21.15.46:0/4236497847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82381077a0 0x7f8238109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.131+0000 7f823fa0e700 1 -- 172.21.15.46:0/4236497847 >> 172.21.15.46:0/4236497847 conn(0x7f82380fe880 msgr2=0x7f8238100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:44:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.131+0000 7f823fa0e700 1 -- 172.21.15.46:0/4236497847 shutdown_connections 2024-05-09T04:44:10.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.131+0000 7f823fa0e700 1 -- 172.21.15.46:0/4236497847 wait complete. 2024-05-09T04:44:10.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.132+0000 7f823fa0e700 1 Processor -- start 2024-05-09T04:44:10.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.132+0000 7f823fa0e700 1 -- start start 2024-05-09T04:44:10.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823fa0e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8238104e40 0x7f823819a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:10.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823fa0e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82381077a0 0x7f823819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:10.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823fa0e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823810a100 0x7f82381a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:10.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823fa0e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f823810fd70 con 0x7f8238104e40 2024-05-09T04:44:10.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823fa0e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f823810fbf0 con 0x7f82381077a0 2024-05-09T04:44:10.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823fa0e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f823810fef0 con 0x7f823810a100 2024-05-09T04:44:10.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823d7aa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8238104e40 0x7f823819a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:44:10.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823d7aa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8238104e40 0x7f823819a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57554/0 (socket says 172.21.15.46:57554) 2024-05-09T04:44:10.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823d7aa700 1 -- 172.21.15.46:0/1323187095 learned_addr learned my addr 172.21.15.46:0/1323187095 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:44:10.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823dfab700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823810a100 0x7f82381a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:44:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823cfa9700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82381077a0 0x7f823819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:44:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823d7aa700 1 -- 172.21.15.46:0/1323187095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823810a100 msgr2=0x7f82381a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823d7aa700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823810a100 0x7f82381a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823d7aa700 1 -- 172.21.15.46:0/1323187095 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82381077a0 msgr2=0x7f823819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823d7aa700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82381077a0 0x7f823819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823d7aa700 1 -- 172.21.15.46:0/1323187095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82381a2440 con 0x7f8238104e40 2024-05-09T04:44:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823dfab700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823810a100 0x7f82381a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:44:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823cfa9700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82381077a0 0x7f823819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:44:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823d7aa700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8238104e40 0x7f823819a8e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f82280095c0 tx=0x7f8228004ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:44:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f82367fc700 1 -- 172.21.15.46:0/1323187095 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8228021070 con 0x7f8238104e40 2024-05-09T04:44:10.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f82367fc700 1 -- 172.21.15.46:0/1323187095 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f822801d900 con 0x7f8238104e40 2024-05-09T04:44:10.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f823fa0e700 1 -- 172.21.15.46:0/1323187095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82381a26d0 con 0x7f8238104e40 2024-05-09T04:44:10.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.133+0000 7f82367fc700 1 -- 172.21.15.46:0/1323187095 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82280052a0 con 0x7f8238104e40 2024-05-09T04:44:10.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.134+0000 7f823fa0e700 1 -- 172.21.15.46:0/1323187095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82381a2c60 con 0x7f8238104e40 2024-05-09T04:44:10.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.135+0000 7f823fa0e700 1 -- 172.21.15.46:0/1323187095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82380512c0 con 0x7f8238104e40 2024-05-09T04:44:10.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.136+0000 7f82367fc700 1 -- 172.21.15.46:0/1323187095 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8228024070 con 0x7f8238104e40 2024-05-09T04:44:10.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.137+0000 7f82367fc700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f82200b2900 0x7f82200b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:10.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.137+0000 7f823cfa9700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f82200b2900 0x7f82200b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:44:10.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.137+0000 7f82367fc700 1 -- 172.21.15.46:0/1323187095 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4533+0+0 (secure 0 0 0) 0x7f822800f020 con 0x7f8238104e40 2024-05-09T04:44:10.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.138+0000 7f823cfa9700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f82200b2900 0x7f82200b4e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f822c005950 tx=0x7f822c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:44:10.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.142+0000 7f82367fc700 1 -- 172.21.15.46:0/1323187095 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f822802b020 con 0x7f8238104e40 2024-05-09T04:44:10.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:10.366+0000 7f823fa0e700 1 -- 172.21.15.46:0/1323187095 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f8238063c00 con 0x7f82200b2900 2024-05-09T04:44:11.004 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:10 smithi046 ceph-mon[30136]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:11.004 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:10 smithi046 ceph-mon[30136]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T04:44:11.005 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:10 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:44:11.005 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:10 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:44:11.005 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:10 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:10 smithi110 ceph-mon[34904]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:10 smithi110 ceph-mon[34904]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T04:44:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:10 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:44:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:10 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:44:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:10 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:11.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:10 smithi072 ceph-mon[34088]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:10 smithi072 ceph-mon[34088]: osdmap e40: 6 total, 6 up, 6 in 2024-05-09T04:44:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:10 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:44:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:10 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:44:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:10 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:11.944 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:11 smithi046 ceph-mon[30136]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:44:12.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:11 smithi110 ceph-mon[34904]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:44:12.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:11 smithi072 ceph-mon[34088]: from='client.14376 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:44:13.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:12 smithi110 ceph-mon[34904]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:13.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:12 smithi046 ceph-mon[30136]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:13.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:12 smithi072 ceph-mon[34088]: pgmap v142: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:13.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:13 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2408547551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beb0b87f-c3a4-40dd-b224-1843e9301ca9"}]: dispatch 2024-05-09T04:44:13.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:13 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2408547551' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beb0b87f-c3a4-40dd-b224-1843e9301ca9"}]': finished 2024-05-09T04:44:13.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:13 smithi046 ceph-mon[30136]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T04:44:13.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:13 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:14.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:13 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2408547551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beb0b87f-c3a4-40dd-b224-1843e9301ca9"}]: dispatch 2024-05-09T04:44:14.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:13 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2408547551' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beb0b87f-c3a4-40dd-b224-1843e9301ca9"}]': finished 2024-05-09T04:44:14.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:13 smithi110 ceph-mon[34904]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T04:44:14.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:13 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:14.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:13 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2408547551' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "beb0b87f-c3a4-40dd-b224-1843e9301ca9"}]: dispatch 2024-05-09T04:44:14.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:13 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2408547551' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "beb0b87f-c3a4-40dd-b224-1843e9301ca9"}]': finished 2024-05-09T04:44:14.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:13 smithi072 ceph-mon[34088]: osdmap e41: 7 total, 6 up, 7 in 2024-05-09T04:44:14.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:13 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:15.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:14 smithi110 ceph-mon[34904]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:15.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:14 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3562784588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T04:44:15.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:14 smithi046 ceph-mon[30136]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:15.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:14 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3562784588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T04:44:15.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:14 smithi072 ceph-mon[34088]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:15.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:14 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3562784588' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-6/activate.monmap"}]: dispatch 2024-05-09T04:44:17.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:16 smithi110 ceph-mon[34904]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-09T04:44:17.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:16 smithi046 ceph-mon[30136]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-09T04:44:17.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:16 smithi072 ceph-mon[34088]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-05-09T04:44:19.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:18 smithi110 ceph-mon[34904]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T04:44:19.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:18 smithi046 ceph-mon[30136]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T04:44:19.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:18 smithi072 ceph-mon[34088]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-09T04:44:20.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:20 smithi046 ceph-mon[30136]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-09T04:44:21.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:20 smithi110 ceph-mon[34904]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-09T04:44:21.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:20 smithi072 ceph-mon[34088]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-09T04:44:22.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:21 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T04:44:22.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:21 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:22.078 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:21 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T04:44:22.078 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:21 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:22.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:21 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-09T04:44:22.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:21 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:22.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:22 smithi046 ceph-mon[30136]: Deploying daemon osd.6 on smithi046 2024-05-09T04:44:22.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:22 smithi046 ceph-mon[30136]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:44:23.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:22 smithi110 ceph-mon[34904]: Deploying daemon osd.6 on smithi046 2024-05-09T04:44:23.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:22 smithi110 ceph-mon[34904]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:44:23.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:22 smithi072 ceph-mon[34088]: Deploying daemon osd.6 on smithi046 2024-05-09T04:44:23.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:22 smithi072 ceph-mon[34088]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:44:24.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:24 smithi046 ceph-mon[30136]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-09T04:44:25.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:24 smithi110 ceph-mon[34904]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-09T04:44:25.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:24 smithi072 ceph-mon[34088]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-09T04:44:26.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:25 smithi046 ceph-mon[30136]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T04:44:26.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:25 smithi110 ceph-mon[34904]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T04:44:26.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:25 smithi072 ceph-mon[34088]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-09T04:44:28.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:28.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:28.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:28.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:28 smithi110 ceph-mon[34904]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:28.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:28.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:28.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:28.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:28 smithi046 ceph-mon[30136]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:28.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:28.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:28.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:28.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:28 smithi072 ceph-mon[34088]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:30.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:30 smithi110 ceph-mon[34904]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:30.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:30 smithi046 ceph-mon[30136]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:30.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:30 smithi072 ceph-mon[34088]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:30.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:30.726+0000 7f82367fc700 1 -- 172.21.15.46:0/1323187095 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8238063c00 con 0x7f82200b2900 2024-05-09T04:44:30.727 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 6 on host 'smithi046' 2024-05-09T04:44:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:30.729+0000 7f823fa0e700 1 -- 172.21.15.46:0/1323187095 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f82200b2900 msgr2=0x7f82200b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:30.729+0000 7f823fa0e700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f82200b2900 0x7f82200b4e10 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f822c005950 tx=0x7f822c0058e0 comp rx=0 tx=0).stop 2024-05-09T04:44:30.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:30.729+0000 7f823fa0e700 1 -- 172.21.15.46:0/1323187095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8238104e40 msgr2=0x7f823819a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:30.729+0000 7f823fa0e700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8238104e40 0x7f823819a8e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f82280095c0 tx=0x7f8228004ef0 comp rx=0 tx=0).stop 2024-05-09T04:44:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:30.729+0000 7f823fa0e700 1 -- 172.21.15.46:0/1323187095 shutdown_connections 2024-05-09T04:44:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:30.729+0000 7f823fa0e700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f82200b2900 0x7f82200b4e10 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:30.729+0000 7f823fa0e700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f823810a100 0x7f82381a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:30.729+0000 7f823fa0e700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8238104e40 0x7f823819a8e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:30.729+0000 7f823fa0e700 1 --2- 172.21.15.46:0/1323187095 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82381077a0 0x7f823819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:30.729+0000 7f823fa0e700 1 -- 172.21.15.46:0/1323187095 >> 172.21.15.46:0/1323187095 conn(0x7f82380fe880 msgr2=0x7f8238105a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:44:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:30.730+0000 7f823fa0e700 1 -- 172.21.15.46:0/1323187095 shutdown_connections 2024-05-09T04:44:30.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:44:30.730+0000 7f823fa0e700 1 -- 172.21.15.46:0/1323187095 wait complete. 2024-05-09T04:44:31.542 DEBUG:teuthology.orchestra.run.smithi046:osd.6> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.6.service 2024-05-09T04:44:31.545 INFO:tasks.cephadm:Deploying osd.7 on smithi072 with /dev/vg_nvme/lv_2... 2024-05-09T04:44:31.545 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T04:44:31.791 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:44:31.826 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:31 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:31.826 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:31 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:31.826 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:31 smithi046 ceph-mon[30136]: from='osd.6 [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T04:44:31.826 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:31 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:31.826 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:31 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:31.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:31 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:31.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:31 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:31.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:31 smithi072 ceph-mon[34088]: from='osd.6 [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T04:44:31.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:31 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:31.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:31 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:32.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:31 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:31 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:31 smithi110 ceph-mon[34904]: from='osd.6 [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-09T04:44:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:31 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:31 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:32.994 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:32 smithi072 ceph-mon[34088]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:32.994 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:32 smithi072 ceph-mon[34088]: from='osd.6 [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T04:44:32.994 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:32 smithi072 ceph-mon[34088]: osdmap e42: 7 total, 6 up, 7 in 2024-05-09T04:44:32.994 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:32 smithi072 ceph-mon[34088]: from='osd.6 [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T04:44:32.994 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:32 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:33.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:32 smithi110 ceph-mon[34904]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:33.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:32 smithi110 ceph-mon[34904]: from='osd.6 [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T04:44:33.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:32 smithi110 ceph-mon[34904]: osdmap e42: 7 total, 6 up, 7 in 2024-05-09T04:44:33.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:32 smithi110 ceph-mon[34904]: from='osd.6 [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T04:44:33.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:32 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:33.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:32 smithi046 ceph-mon[30136]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:33.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:32 smithi046 ceph-mon[30136]: from='osd.6 [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-09T04:44:33.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:32 smithi046 ceph-mon[30136]: osdmap e42: 7 total, 6 up, 7 in 2024-05-09T04:44:33.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:32 smithi046 ceph-mon[30136]: from='osd.6 [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T04:44:33.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:32 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:33.065 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 04:44:32 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T04:44:32.710+0000 7f4e277f2700 -1 osd.6 0 waiting for initial osdmap 2024-05-09T04:44:33.065 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 04:44:32 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T04:44:32.720+0000 7f4e235a7700 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:44:33.848 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T04:44:33.848 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T04:44:33.848 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-05-09T04:44:33.848 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-05-09T04:44:33.848 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129744 s, 808 MB/s 2024-05-09T04:44:33.848 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-05-09T04:44:33.970 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:33 smithi046 ceph-mon[30136]: from='osd.6 [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T04:44:33.970 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:33 smithi046 ceph-mon[30136]: osdmap e43: 7 total, 6 up, 7 in 2024-05-09T04:44:33.970 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:33 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:33.970 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:33 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:34.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:33 smithi110 ceph-mon[34904]: from='osd.6 [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T04:44:34.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:33 smithi110 ceph-mon[34904]: osdmap e43: 7 total, 6 up, 7 in 2024-05-09T04:44:34.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:33 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:34.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:33 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:34.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:33 smithi072 ceph-mon[34088]: from='osd.6 [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T04:44:34.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:33 smithi072 ceph-mon[34088]: osdmap e43: 7 total, 6 up, 7 in 2024-05-09T04:44:34.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:33 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:34.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:33 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:34.490 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_2 2024-05-09T04:44:34.714 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:34 smithi072 ceph-mon[34088]: purged_snaps scrub starts 2024-05-09T04:44:34.714 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:34 smithi072 ceph-mon[34088]: purged_snaps scrub ok 2024-05-09T04:44:34.714 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:34 smithi072 ceph-mon[34088]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:34.714 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:34 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:34.714 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:34 smithi072 ceph-mon[34088]: osd.6 [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065] boot 2024-05-09T04:44:34.714 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:34 smithi072 ceph-mon[34088]: osdmap e44: 7 total, 7 up, 7 in 2024-05-09T04:44:34.714 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:34 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:34.732 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:44:35.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:34 smithi110 ceph-mon[34904]: purged_snaps scrub starts 2024-05-09T04:44:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:34 smithi110 ceph-mon[34904]: purged_snaps scrub ok 2024-05-09T04:44:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:34 smithi110 ceph-mon[34904]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:34 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:34 smithi110 ceph-mon[34904]: osd.6 [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065] boot 2024-05-09T04:44:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:34 smithi110 ceph-mon[34904]: osdmap e44: 7 total, 7 up, 7 in 2024-05-09T04:44:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:34 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:35.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:34 smithi046 ceph-mon[30136]: purged_snaps scrub starts 2024-05-09T04:44:35.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:34 smithi046 ceph-mon[30136]: purged_snaps scrub ok 2024-05-09T04:44:35.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:34 smithi046 ceph-mon[30136]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 181 MiB used, 536 GiB / 536 GiB avail 2024-05-09T04:44:35.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:34 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:35.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:34 smithi046 ceph-mon[30136]: osd.6 [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065] boot 2024-05-09T04:44:35.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:34 smithi046 ceph-mon[30136]: osdmap e44: 7 total, 7 up, 7 in 2024-05-09T04:44:35.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:34 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:44:35.877 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:35 smithi072 ceph-mon[34088]: osdmap e45: 7 total, 7 up, 7 in 2024-05-09T04:44:35.877 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:35 smithi072 ceph-mon[34088]: Detected new or changed devices on smithi046 2024-05-09T04:44:35.877 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:35 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:35.877 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:35 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:35.877 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:35 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:35.877 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:35 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:35.877 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:35 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:35.877 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:35 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:35.877 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:35 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:35.878 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:35 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:35.878 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:35 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:35.878 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:35 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:35.878 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:35 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:35.878 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:35 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:35.878 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:35 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:36.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:35 smithi110 ceph-mon[34904]: osdmap e45: 7 total, 7 up, 7 in 2024-05-09T04:44:36.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:35 smithi110 ceph-mon[34904]: Detected new or changed devices on smithi046 2024-05-09T04:44:36.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:35 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:36.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:35 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:36.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:35 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:36.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:35 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:36.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:35 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:36.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:35 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:36.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:35 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:36.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:35 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:36.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:35 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:36.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:35 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:36.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:35 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:36.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:35 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:36.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:35 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:36.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:35 smithi046 ceph-mon[30136]: osdmap e45: 7 total, 7 up, 7 in 2024-05-09T04:44:36.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:35 smithi046 ceph-mon[30136]: Detected new or changed devices on smithi046 2024-05-09T04:44:36.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:35 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:36.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:35 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:36.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:35 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:36.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:35 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:36.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:35 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:44:36.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:35 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:36.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:35 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:36.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:35 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:36.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:35 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:36.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:35 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:36.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:35 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:36.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:35 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:36.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:35 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:36.426 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.422+0000 7fb4976cf700 1 -- 172.21.15.72:0/236762649 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4901059b0 msgr2=0x7fb490105dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:36.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.422+0000 7fb4976cf700 1 --2- 172.21.15.72:0/236762649 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4901059b0 0x7fb490105dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb48c009990 tx=0x7fb48c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:44:36.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.422+0000 7fb4976cf700 1 -- 172.21.15.72:0/236762649 shutdown_connections 2024-05-09T04:44:36.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.422+0000 7fb4976cf700 1 --2- 172.21.15.72:0/236762649 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4901049e0 0x7fb49010c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:36.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.422+0000 7fb4976cf700 1 --2- 172.21.15.72:0/236762649 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4901059b0 0x7fb490105dc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:36.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.422+0000 7fb4976cf700 1 --2- 172.21.15.72:0/236762649 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb490103f60 0x7fb490104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:36.427 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.422+0000 7fb4976cf700 1 -- 172.21.15.72:0/236762649 >> 172.21.15.72:0/236762649 conn(0x7fb4900ff9d0 msgr2=0x7fb490101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:44:36.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.422+0000 7fb4976cf700 1 -- 172.21.15.72:0/236762649 shutdown_connections 2024-05-09T04:44:36.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.422+0000 7fb4976cf700 1 -- 172.21.15.72:0/236762649 wait complete. 2024-05-09T04:44:36.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.423+0000 7fb4976cf700 1 Processor -- start 2024-05-09T04:44:36.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.423+0000 7fb4976cf700 1 -- start start 2024-05-09T04:44:36.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.423+0000 7fb4976cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490103f60 0x7fb49019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:36.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.423+0000 7fb4976cf700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4901049e0 0x7fb49019afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:36.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.423+0000 7fb4976cf700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4901059b0 0x7fb4901a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:36.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.423+0000 7fb4976cf700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb49010ff20 con 0x7fb490103f60 2024-05-09T04:44:36.428 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.423+0000 7fb4976cf700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb49010fda0 con 0x7fb4901049e0 2024-05-09T04:44:36.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.423+0000 7fb4976cf700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb4901100a0 con 0x7fb4901059b0 2024-05-09T04:44:36.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.423+0000 7fb494c6a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4901049e0 0x7fb49019afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:44:36.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb49546b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490103f60 0x7fb49019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:44:36.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb494c6a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4901049e0 0x7fb49019afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:41924/0 (socket says 172.21.15.72:41924) 2024-05-09T04:44:36.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb494c6a700 1 -- 172.21.15.72:0/269168478 learned_addr learned my addr 172.21.15.72:0/269168478 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-05-09T04:44:36.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb49546b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490103f60 0x7fb49019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.72:60762/0 (socket says 172.21.15.72:60762) 2024-05-09T04:44:36.429 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb495c6c700 1 --2- 172.21.15.72:0/269168478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4901059b0 0x7fb4901a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:44:36.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb494c6a700 1 -- 172.21.15.72:0/269168478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4901059b0 msgr2=0x7fb4901a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:36.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb494c6a700 1 --2- 172.21.15.72:0/269168478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4901059b0 0x7fb4901a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:36.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb494c6a700 1 -- 172.21.15.72:0/269168478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490103f60 msgr2=0x7fb49019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:36.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb494c6a700 1 --2- 172.21.15.72:0/269168478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490103f60 0x7fb49019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:36.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb494c6a700 1 -- 172.21.15.72:0/269168478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4901a25c0 con 0x7fb4901049e0 2024-05-09T04:44:36.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb49546b700 1 --2- 172.21.15.72:0/269168478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490103f60 0x7fb49019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:44:36.430 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb494c6a700 1 --2- 172.21.15.72:0/269168478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4901049e0 0x7fb49019afa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb48000d950 tx=0x7fb48000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:44:36.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb4867fc700 1 -- 172.21.15.72:0/269168478 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb480016070 con 0x7fb4901049e0 2024-05-09T04:44:36.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb4976cf700 1 -- 172.21.15.72:0/269168478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4901a28b0 con 0x7fb4901049e0 2024-05-09T04:44:36.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb4867fc700 1 -- 172.21.15.72:0/269168478 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb48000fd40 con 0x7fb4901049e0 2024-05-09T04:44:36.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb4867fc700 1 -- 172.21.15.72:0/269168478 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb48000e3b0 con 0x7fb4901049e0 2024-05-09T04:44:36.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.424+0000 7fb4976cf700 1 -- 172.21.15.72:0/269168478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4901a2df0 con 0x7fb4901049e0 2024-05-09T04:44:36.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.425+0000 7fb4976cf700 1 -- 172.21.15.72:0/269168478 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4900512c0 con 0x7fb4901049e0 2024-05-09T04:44:36.431 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.427+0000 7fb4867fc700 1 -- 172.21.15.72:0/269168478 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb480019070 con 0x7fb4901049e0 2024-05-09T04:44:36.432 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.427+0000 7fb4867fc700 1 --2- 172.21.15.72:0/269168478 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb47c0b2880 0x7fb47c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:44:36.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.428+0000 7fb49546b700 1 --2- 172.21.15.72:0/269168478 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb47c0b2880 0x7fb47c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:44:36.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.428+0000 7fb4867fc700 1 -- 172.21.15.72:0/269168478 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 4965+0+0 (secure 0 0 0) 0x7fb4800e0ff0 con 0x7fb4901049e0 2024-05-09T04:44:36.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.428+0000 7fb49546b700 1 --2- 172.21.15.72:0/269168478 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb47c0b2880 0x7fb47c0b4d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0038a0 tx=0x7fb48c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:44:36.433 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.433+0000 7fb4867fc700 1 -- 172.21.15.72:0/269168478 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb4800a9a20 con 0x7fb4901049e0 2024-05-09T04:44:36.649 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:36.647+0000 7fb4976cf700 1 -- 172.21.15.72:0/269168478 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fb49002ea20 con 0x7fb47c0b2880 2024-05-09T04:44:37.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:36 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi046 to 5697M 2024-05-09T04:44:37.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:36 smithi110 ceph-mon[34904]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:37.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:44:37.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:44:37.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:37.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:36 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi046 to 5697M 2024-05-09T04:44:37.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:36 smithi046 ceph-mon[30136]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:37.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:44:37.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:44:37.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:37.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:36 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi046 to 5697M 2024-05-09T04:44:37.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:36 smithi072 ceph-mon[34088]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:37.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:44:37.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:44:37.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:37.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:37 smithi072 ceph-mon[34088]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:44:38.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:37 smithi110 ceph-mon[34904]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:44:38.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:37 smithi046 ceph-mon[30136]: from='client.24281 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:44:38.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:38 smithi072 ceph-mon[34088]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:39.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:38 smithi110 ceph-mon[34904]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:39.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:38 smithi046 ceph-mon[30136]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:40.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:39 smithi110 ceph-mon[34904]: from='client.? 172.21.15.72:0/3087227767' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a634675-fad7-4a43-840c-8dec2631dec9"}]: dispatch 2024-05-09T04:44:40.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:39 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a634675-fad7-4a43-840c-8dec2631dec9"}]: dispatch 2024-05-09T04:44:40.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:39 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a634675-fad7-4a43-840c-8dec2631dec9"}]': finished 2024-05-09T04:44:40.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:39 smithi110 ceph-mon[34904]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T04:44:40.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:39 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:44:40.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:39 smithi046 ceph-mon[30136]: from='client.? 172.21.15.72:0/3087227767' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a634675-fad7-4a43-840c-8dec2631dec9"}]: dispatch 2024-05-09T04:44:40.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:39 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a634675-fad7-4a43-840c-8dec2631dec9"}]: dispatch 2024-05-09T04:44:40.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:39 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a634675-fad7-4a43-840c-8dec2631dec9"}]': finished 2024-05-09T04:44:40.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:39 smithi046 ceph-mon[30136]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T04:44:40.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:39 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:44:40.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:39 smithi072 ceph-mon[34088]: from='client.? 172.21.15.72:0/3087227767' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a634675-fad7-4a43-840c-8dec2631dec9"}]: dispatch 2024-05-09T04:44:40.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:39 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4a634675-fad7-4a43-840c-8dec2631dec9"}]: dispatch 2024-05-09T04:44:40.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:39 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4a634675-fad7-4a43-840c-8dec2631dec9"}]': finished 2024-05-09T04:44:40.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:39 smithi072 ceph-mon[34088]: osdmap e46: 8 total, 7 up, 8 in 2024-05-09T04:44:40.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:39 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:44:41.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:40 smithi110 ceph-mon[34904]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:41.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:40 smithi110 ceph-mon[34904]: from='client.? 172.21.15.72:0/552744534' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T04:44:41.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:40 smithi046 ceph-mon[30136]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:41.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:40 smithi046 ceph-mon[30136]: from='client.? 172.21.15.72:0/552744534' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T04:44:41.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:40 smithi072 ceph-mon[34088]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:41.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:40 smithi072 ceph-mon[34088]: from='client.? 172.21.15.72:0/552744534' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-7/activate.monmap"}]: dispatch 2024-05-09T04:44:43.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:42 smithi110 ceph-mon[34904]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:43.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:42 smithi046 ceph-mon[30136]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:43.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:42 smithi072 ceph-mon[34088]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:44.849 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:44 smithi072 ceph-mon[34088]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:45.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:44 smithi110 ceph-mon[34904]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:45.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:44 smithi046 ceph-mon[30136]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:46.976 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:46 smithi072 ceph-mon[34088]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:47.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:46 smithi110 ceph-mon[34904]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:47.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:46 smithi046 ceph-mon[30136]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:47.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:47 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T04:44:47.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:47 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:48.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:47 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T04:44:48.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:47 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:48.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:47 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-09T04:44:48.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:47 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:48.892 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:48 smithi072 ceph-mon[34088]: Deploying daemon osd.7 on smithi072 2024-05-09T04:44:48.892 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:48 smithi072 ceph-mon[34088]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:49.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:48 smithi110 ceph-mon[34904]: Deploying daemon osd.7 on smithi072 2024-05-09T04:44:49.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:48 smithi110 ceph-mon[34904]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:49.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:48 smithi046 ceph-mon[30136]: Deploying daemon osd.7 on smithi072 2024-05-09T04:44:49.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:48 smithi046 ceph-mon[30136]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:51.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:50 smithi110 ceph-mon[34904]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:51.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:50 smithi046 ceph-mon[30136]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:51.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:50 smithi072 ceph-mon[34088]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:53.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:52 smithi110 ceph-mon[34904]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:53.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:52 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:53.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:52 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:53.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:52 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:53.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:52 smithi046 ceph-mon[30136]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:53.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:52 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:53.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:52 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:53.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:52 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:53.114 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:52 smithi072 ceph-mon[34088]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:53.114 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:52 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:53.114 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:52 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:53.114 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:52 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:55.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:54 smithi110 ceph-mon[34904]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:55.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:54 smithi046 ceph-mon[30136]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:55.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:54 smithi072 ceph-mon[34088]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:56.769 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 7 on host 'smithi072' 2024-05-09T04:44:56.770 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:56.769+0000 7fb4867fc700 1 -- 172.21.15.72:0/269168478 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb49002ea20 con 0x7fb47c0b2880 2024-05-09T04:44:56.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:56.772+0000 7fb4976cf700 1 -- 172.21.15.72:0/269168478 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb47c0b2880 msgr2=0x7fb47c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:56.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:56.772+0000 7fb4976cf700 1 --2- 172.21.15.72:0/269168478 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb47c0b2880 0x7fb47c0b4d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb48c0038a0 tx=0x7fb48c01a040 comp rx=0 tx=0).stop 2024-05-09T04:44:56.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:56.772+0000 7fb4976cf700 1 -- 172.21.15.72:0/269168478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4901049e0 msgr2=0x7fb49019afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:44:56.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:56.772+0000 7fb4976cf700 1 --2- 172.21.15.72:0/269168478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4901049e0 0x7fb49019afa0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb48000d950 tx=0x7fb48000dc60 comp rx=0 tx=0).stop 2024-05-09T04:44:56.772 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:56.772+0000 7fb4976cf700 1 -- 172.21.15.72:0/269168478 shutdown_connections 2024-05-09T04:44:56.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:56.772+0000 7fb4976cf700 1 --2- 172.21.15.72:0/269168478 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb47c0b2880 0x7fb47c0b4d90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:56.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:56.772+0000 7fb4976cf700 1 --2- 172.21.15.72:0/269168478 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4901059b0 0x7fb4901a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:56.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:56.772+0000 7fb4976cf700 1 --2- 172.21.15.72:0/269168478 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb490103f60 0x7fb49019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:56.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:56.772+0000 7fb4976cf700 1 --2- 172.21.15.72:0/269168478 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4901049e0 0x7fb49019afa0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:44:56.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:56.772+0000 7fb4976cf700 1 -- 172.21.15.72:0/269168478 >> 172.21.15.72:0/269168478 conn(0x7fb4900ff9d0 msgr2=0x7fb4901086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:44:56.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:56.772+0000 7fb4976cf700 1 -- 172.21.15.72:0/269168478 shutdown_connections 2024-05-09T04:44:56.773 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:44:56.772+0000 7fb4976cf700 1 -- 172.21.15.72:0/269168478 wait complete. 2024-05-09T04:44:57.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:56 smithi110 ceph-mon[34904]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:56 smithi110 ceph-mon[34904]: from='osd.7 [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T04:44:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:56 smithi110 ceph-mon[34904]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T04:44:57.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:56 smithi046 ceph-mon[30136]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:57.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:57.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:57.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:57.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:57.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:57.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:56 smithi046 ceph-mon[30136]: from='osd.7 [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T04:44:57.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:56 smithi046 ceph-mon[30136]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T04:44:57.113 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:56 smithi072 ceph-mon[34088]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:57.113 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:57.113 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:57.113 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:44:57.113 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:44:57.113 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:57.113 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:56 smithi072 ceph-mon[34088]: from='osd.7 [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T04:44:57.113 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:56 smithi072 ceph-mon[34088]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-09T04:44:57.460 DEBUG:teuthology.orchestra.run.smithi072:osd.7> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.7.service 2024-05-09T04:44:57.463 INFO:tasks.cephadm:Deploying osd.8 on smithi110 with /dev/vg_nvme/lv_2... 2024-05-09T04:44:57.463 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-09T04:44:57.710 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:44:57.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:57 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:57.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:57 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:57.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:57 smithi110 ceph-mon[34904]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T04:44:57.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:57 smithi110 ceph-mon[34904]: osdmap e47: 8 total, 7 up, 8 in 2024-05-09T04:44:57.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:57 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:44:57.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:57 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:57.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:57 smithi110 ceph-mon[34904]: from='osd.7 [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:44:57.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:57 smithi110 ceph-mon[34904]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:44:57.978 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:57 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:57.978 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:57 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:57.978 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:57 smithi072 ceph-mon[34088]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T04:44:57.978 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:57 smithi072 ceph-mon[34088]: osdmap e47: 8 total, 7 up, 8 in 2024-05-09T04:44:57.978 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:57 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:44:57.978 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:57 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:57.979 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:57 smithi072 ceph-mon[34088]: from='osd.7 [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:44:57.979 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:57 smithi072 ceph-mon[34088]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:44:57.979 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 04:44:57 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T04:44:57.771+0000 7f931bb82700 -1 osd.7 0 waiting for initial osdmap 2024-05-09T04:44:57.979 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 04:44:57 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T04:44:57.781+0000 7f9317937700 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:44:58.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:57 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:58.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:57 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:44:58.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:57 smithi046 ceph-mon[30136]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-09T04:44:58.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:57 smithi046 ceph-mon[30136]: osdmap e47: 8 total, 7 up, 8 in 2024-05-09T04:44:58.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:57 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:44:58.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:57 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:44:58.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:57 smithi046 ceph-mon[30136]: from='osd.7 [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:44:58.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:57 smithi046 ceph-mon[30136]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:44:58.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:58 smithi110 ceph-mon[34904]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:58.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:58 smithi110 ceph-mon[34904]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-09T04:44:58.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:58 smithi110 ceph-mon[34904]: osdmap e48: 8 total, 7 up, 8 in 2024-05-09T04:44:58.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:58 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:44:58.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:58 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:44:58.825 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:58 smithi072 ceph-mon[34088]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:58.825 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:58 smithi072 ceph-mon[34088]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-09T04:44:58.825 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:58 smithi072 ceph-mon[34088]: osdmap e48: 8 total, 7 up, 8 in 2024-05-09T04:44:58.825 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:58 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:44:58.825 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:58 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:44:59.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:58 smithi046 ceph-mon[30136]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:44:59.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:58 smithi046 ceph-mon[30136]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-09T04:44:59.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:58 smithi046 ceph-mon[30136]: osdmap e48: 8 total, 7 up, 8 in 2024-05-09T04:44:59.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:58 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:44:59.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:58 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:44:59.889 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-09T04:44:59.889 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-09T04:44:59.889 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-05-09T04:44:59.889 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-05-09T04:44:59.889 INFO:teuthology.orchestra.run.smithi110.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0082762 s, 1.3 GB/s 2024-05-09T04:44:59.889 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-05-09T04:45:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:59 smithi110 ceph-mon[34904]: purged_snaps scrub starts 2024-05-09T04:45:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:59 smithi110 ceph-mon[34904]: purged_snaps scrub ok 2024-05-09T04:45:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:59 smithi110 ceph-mon[34904]: osd.7 [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672] boot 2024-05-09T04:45:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:59 smithi110 ceph-mon[34904]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T04:45:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:59 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:45:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:44:59 smithi110 ceph-mon[34904]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:45:00.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:59 smithi046 ceph-mon[30136]: purged_snaps scrub starts 2024-05-09T04:45:00.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:59 smithi046 ceph-mon[30136]: purged_snaps scrub ok 2024-05-09T04:45:00.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:59 smithi046 ceph-mon[30136]: osd.7 [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672] boot 2024-05-09T04:45:00.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:59 smithi046 ceph-mon[30136]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T04:45:00.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:59 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:45:00.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:44:59 smithi046 ceph-mon[30136]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:45:00.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:59 smithi072 ceph-mon[34088]: purged_snaps scrub starts 2024-05-09T04:45:00.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:59 smithi072 ceph-mon[34088]: purged_snaps scrub ok 2024-05-09T04:45:00.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:59 smithi072 ceph-mon[34088]: osd.7 [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672] boot 2024-05-09T04:45:00.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:59 smithi072 ceph-mon[34088]: osdmap e49: 8 total, 8 up, 8 in 2024-05-09T04:45:00.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:59 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:45:00.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:44:59 smithi072 ceph-mon[34088]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 208 MiB used, 626 GiB / 626 GiB avail 2024-05-09T04:45:00.628 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_2 2024-05-09T04:45:00.879 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:45:00.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:00 smithi072 ceph-mon[34088]: osdmap e50: 8 total, 8 up, 8 in 2024-05-09T04:45:00.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:00 smithi072 ceph-mon[34088]: Detected new or changed devices on smithi072 2024-05-09T04:45:00.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:00 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:00.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:00 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:00.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:00 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:00.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:00 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:00.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:00 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:00.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:00 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi072 to 5696M 2024-05-09T04:45:00.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:00 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:00.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:00 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:00.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:00 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:00.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:00 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:00.919 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:00 smithi110 ceph-mon[34904]: osdmap e50: 8 total, 8 up, 8 in 2024-05-09T04:45:00.919 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:00 smithi110 ceph-mon[34904]: Detected new or changed devices on smithi072 2024-05-09T04:45:00.919 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:00 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:00.919 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:00 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:00.920 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:00 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:00.920 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:00 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:00.920 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:00 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:00.920 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:00 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi072 to 5696M 2024-05-09T04:45:00.920 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:00 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:00.920 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:00 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:00.920 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:00 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:00.920 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:00 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:01.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:00 smithi046 ceph-mon[30136]: osdmap e50: 8 total, 8 up, 8 in 2024-05-09T04:45:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:00 smithi046 ceph-mon[30136]: Detected new or changed devices on smithi072 2024-05-09T04:45:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:00 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:00 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:00 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:00 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:00 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:00 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi072 to 5696M 2024-05-09T04:45:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:00 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:00 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:00 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:00 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:02.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:01 smithi110 ceph-mon[34904]: osdmap e51: 8 total, 8 up, 8 in 2024-05-09T04:45:02.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:01 smithi110 ceph-mon[34904]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:02.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:01 smithi046 ceph-mon[30136]: osdmap e51: 8 total, 8 up, 8 in 2024-05-09T04:45:02.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:01 smithi046 ceph-mon[30136]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:02.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:01 smithi072 ceph-mon[34088]: osdmap e51: 8 total, 8 up, 8 in 2024-05-09T04:45:02.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:01 smithi072 ceph-mon[34088]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:02.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.636+0000 7f3b312ef700 1 -- 172.21.15.110:0/1255903903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b2c10a110 msgr2=0x7f3b2c10c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:02.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.636+0000 7f3b312ef700 1 --2- 172.21.15.110:0/1255903903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b2c10a110 0x7f3b2c10c530 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400b0a0 tx=0x7f3b2400b3b0 comp rx=0 tx=0).stop 2024-05-09T04:45:02.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.636+0000 7f3b312ef700 1 -- 172.21.15.110:0/1255903903 shutdown_connections 2024-05-09T04:45:02.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.636+0000 7f3b312ef700 1 --2- 172.21.15.110:0/1255903903 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b2c10a110 0x7f3b2c10c530 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:02.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.636+0000 7f3b312ef700 1 --2- 172.21.15.110:0/1255903903 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b2c1026c0 0x7f3b2c102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:02.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.637+0000 7f3b312ef700 1 --2- 172.21.15.110:0/1255903903 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c1077b0 0x7f3b2c109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:02.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.637+0000 7f3b312ef700 1 -- 172.21.15.110:0/1255903903 >> 172.21.15.110:0/1255903903 conn(0x7f3b2c0fe020 msgr2=0x7f3b2c1004d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:45:02.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.637+0000 7f3b312ef700 1 -- 172.21.15.110:0/1255903903 shutdown_connections 2024-05-09T04:45:02.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.637+0000 7f3b312ef700 1 -- 172.21.15.110:0/1255903903 wait complete. 2024-05-09T04:45:02.639 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.637+0000 7f3b312ef700 1 Processor -- start 2024-05-09T04:45:02.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.637+0000 7f3b312ef700 1 -- start start 2024-05-09T04:45:02.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b312ef700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b2c1026c0 0x7f3b2c19a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:02.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b312ef700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c1077b0 0x7f3b2c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:02.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b312ef700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b2c10a110 0x7f3b2c1a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:02.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b312ef700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c10fd90 con 0x7f3b2c1077b0 2024-05-09T04:45:02.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b312ef700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c10fc10 con 0x7f3b2c10a110 2024-05-09T04:45:02.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b312ef700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3b2c10ff10 con 0x7f3b2c1026c0 2024-05-09T04:45:02.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b2affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b2c1026c0 0x7f3b2c19a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:02.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b2affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b2c1026c0 0x7f3b2c19a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:58024/0 (socket says 172.21.15.110:58024) 2024-05-09T04:45:02.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b2affd700 1 -- 172.21.15.110:0/11808577 learned_addr learned my addr 172.21.15.110:0/11808577 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-09T04:45:02.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b2a7fc700 1 --2- 172.21.15.110:0/11808577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c1077b0 0x7f3b2c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:02.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b2b7fe700 1 --2- 172.21.15.110:0/11808577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b2c10a110 0x7f3b2c1a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:02.640 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b2affd700 1 -- 172.21.15.110:0/11808577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b2c10a110 msgr2=0x7f3b2c1a1cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:02.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b2affd700 1 --2- 172.21.15.110:0/11808577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b2c10a110 0x7f3b2c1a1cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:02.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b2affd700 1 -- 172.21.15.110:0/11808577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c1077b0 msgr2=0x7f3b2c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:02.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b2affd700 1 --2- 172.21.15.110:0/11808577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c1077b0 0x7f3b2c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:02.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b2affd700 1 -- 172.21.15.110:0/11808577 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b2c1a22e0 con 0x7f3b2c1026c0 2024-05-09T04:45:02.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b2b7fe700 1 --2- 172.21.15.110:0/11808577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b2c10a110 0x7f3b2c1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:45:02.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b2affd700 1 --2- 172.21.15.110:0/11808577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b2c1026c0 0x7f3b2c19a880 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00d980 tx=0x7f3b1c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:45:02.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b23fff700 1 -- 172.21.15.110:0/11808577 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c016070 con 0x7f3b2c1026c0 2024-05-09T04:45:02.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b312ef700 1 -- 172.21.15.110:0/11808577 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b2c1a25d0 con 0x7f3b2c1026c0 2024-05-09T04:45:02.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b23fff700 1 -- 172.21.15.110:0/11808577 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b1c004830 con 0x7f3b2c1026c0 2024-05-09T04:45:02.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b23fff700 1 -- 172.21.15.110:0/11808577 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c0054c0 con 0x7f3b2c1026c0 2024-05-09T04:45:02.641 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.638+0000 7f3b312ef700 1 -- 172.21.15.110:0/11808577 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b2c1a2b10 con 0x7f3b2c1026c0 2024-05-09T04:45:02.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.640+0000 7f3b312ef700 1 -- 172.21.15.110:0/11808577 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b2c0512c0 con 0x7f3b2c1026c0 2024-05-09T04:45:02.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.642+0000 7f3b23fff700 1 -- 172.21.15.110:0/11808577 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3b1c004220 con 0x7f3b2c1026c0 2024-05-09T04:45:02.642 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.642+0000 7f3b23fff700 1 --2- 172.21.15.110:0/11808577 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b180b2880 0x7f3b180b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:02.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.643+0000 7f3b23fff700 1 -- 172.21.15.110:0/11808577 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5397+0+0 (secure 0 0 0) 0x7f3b1c0e1aa0 con 0x7f3b2c1026c0 2024-05-09T04:45:02.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.643+0000 7f3b2a7fc700 1 --2- 172.21.15.110:0/11808577 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b180b2880 0x7f3b180b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:02.643 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.643+0000 7f3b2a7fc700 1 --2- 172.21.15.110:0/11808577 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b180b2880 0x7f3b180b4d90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c19b930 tx=0x7f3b1400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:45:02.646 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.646+0000 7f3b23fff700 1 -- 172.21.15.110:0/11808577 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3b1c01f020 con 0x7f3b2c1026c0 2024-05-09T04:45:02.859 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:02.859+0000 7f3b312ef700 1 -- 172.21.15.110:0/11808577 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f3b2c063c00 con 0x7f3b180b2880 2024-05-09T04:45:03.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:45:03.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:45:03.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:03.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:45:03.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:45:03.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:03.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:45:03.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:45:03.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:04.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:03 smithi110 ceph-mon[34904]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:45:04.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:03 smithi110 ceph-mon[34904]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:04.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:03 smithi046 ceph-mon[30136]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:45:04.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:03 smithi046 ceph-mon[30136]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:04.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:03 smithi072 ceph-mon[34088]: from='client.24301 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:45:04.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:03 smithi072 ceph-mon[34088]: pgmap v179: 1 pgs: 1 unknown; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:06.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:06 smithi046 ceph-mon[30136]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 178 KiB/s, 0 objects/s recovering 2024-05-09T04:45:06.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:06 smithi046 ceph-mon[30136]: from='client.? 172.21.15.110:0/2556596889' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01b5f638-90f0-4cf4-8042-20d20a725669"}]: dispatch 2024-05-09T04:45:06.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:06 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01b5f638-90f0-4cf4-8042-20d20a725669"}]: dispatch 2024-05-09T04:45:06.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:06 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01b5f638-90f0-4cf4-8042-20d20a725669"}]': finished 2024-05-09T04:45:06.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:06 smithi046 ceph-mon[30136]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T04:45:06.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:06 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:06.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:06 smithi072 ceph-mon[34088]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 178 KiB/s, 0 objects/s recovering 2024-05-09T04:45:06.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:06 smithi072 ceph-mon[34088]: from='client.? 172.21.15.110:0/2556596889' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01b5f638-90f0-4cf4-8042-20d20a725669"}]: dispatch 2024-05-09T04:45:06.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:06 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01b5f638-90f0-4cf4-8042-20d20a725669"}]: dispatch 2024-05-09T04:45:06.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:06 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01b5f638-90f0-4cf4-8042-20d20a725669"}]': finished 2024-05-09T04:45:06.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:06 smithi072 ceph-mon[34088]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T04:45:06.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:06 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:06.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:06 smithi110 ceph-mon[34904]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 178 KiB/s, 0 objects/s recovering 2024-05-09T04:45:06.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:06 smithi110 ceph-mon[34904]: from='client.? 172.21.15.110:0/2556596889' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01b5f638-90f0-4cf4-8042-20d20a725669"}]: dispatch 2024-05-09T04:45:06.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:06 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01b5f638-90f0-4cf4-8042-20d20a725669"}]: dispatch 2024-05-09T04:45:06.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:06 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01b5f638-90f0-4cf4-8042-20d20a725669"}]': finished 2024-05-09T04:45:06.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:06 smithi110 ceph-mon[34904]: osdmap e52: 9 total, 8 up, 9 in 2024-05-09T04:45:06.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:06 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:07.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:07 smithi046 ceph-mon[30136]: from='client.? 172.21.15.110:0/3554696066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T04:45:07.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:07 smithi072 ceph-mon[34088]: from='client.? 172.21.15.110:0/3554696066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T04:45:07.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:07 smithi110 ceph-mon[34904]: from='client.? 172.21.15.110:0/3554696066' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-8/activate.monmap"}]: dispatch 2024-05-09T04:45:08.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:08 smithi046 ceph-mon[30136]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 154 KiB/s, 0 objects/s recovering 2024-05-09T04:45:08.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:08 smithi072 ceph-mon[34088]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 154 KiB/s, 0 objects/s recovering 2024-05-09T04:45:08.708 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:08 smithi110 ceph-mon[34904]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 154 KiB/s, 0 objects/s recovering 2024-05-09T04:45:10.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:10 smithi046 ceph-mon[30136]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-05-09T04:45:10.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:10 smithi072 ceph-mon[34088]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-05-09T04:45:10.658 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:10 smithi110 ceph-mon[34904]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-05-09T04:45:12.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:12 smithi046 ceph-mon[30136]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T04:45:12.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:12 smithi072 ceph-mon[34088]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T04:45:12.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:12 smithi110 ceph-mon[34904]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T04:45:13.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:13 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T04:45:13.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:13 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:13.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:13 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T04:45:13.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:13 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:13.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:13 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-09T04:45:13.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:13 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:14.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:14 smithi072 ceph-mon[34088]: Deploying daemon osd.8 on smithi110 2024-05-09T04:45:14.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:14 smithi072 ceph-mon[34088]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T04:45:14.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:14 smithi110 ceph-mon[34904]: Deploying daemon osd.8 on smithi110 2024-05-09T04:45:14.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:14 smithi110 ceph-mon[34904]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T04:45:14.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:14 smithi046 ceph-mon[30136]: Deploying daemon osd.8 on smithi110 2024-05-09T04:45:14.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:14 smithi046 ceph-mon[30136]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-09T04:45:16.605 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:16 smithi110 ceph-mon[34904]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:16.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:16 smithi072 ceph-mon[34088]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:16.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:16 smithi046 ceph-mon[30136]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:18.514 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:18 smithi110 ceph-mon[34904]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:18.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:18 smithi072 ceph-mon[34088]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:18.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:18 smithi046 ceph-mon[30136]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:20.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:20 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:20.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:20 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:20.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:20 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:20.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:20 smithi046 ceph-mon[30136]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:20.534 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:20 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:20.534 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:20 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:20.534 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:20 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:20.534 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:20 smithi110 ceph-mon[34904]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:20.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:20 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:20.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:20 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:20.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:20 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:20.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:20 smithi072 ceph-mon[34088]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:22.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:22 smithi046 ceph-mon[30136]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:22.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:22 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:22.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:22 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:22.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:22 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:22.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:22 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:22.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:22 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:22.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:22 smithi072 ceph-mon[34088]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:22.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:22 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:22.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:22 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:22.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:22 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:22.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:22 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:22.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:22 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:22.793 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:22 smithi110 ceph-mon[34904]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:22.793 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:22 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:22.793 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:22 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:22.793 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:22 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:22.793 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:22 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:22.793 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:22 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:23.183 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:23.183+0000 7f3b23fff700 1 -- 172.21.15.110:0/11808577 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f3b2c063c00 con 0x7f3b180b2880 2024-05-09T04:45:23.183 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 8 on host 'smithi110' 2024-05-09T04:45:23.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:23.186+0000 7f3b312ef700 1 -- 172.21.15.110:0/11808577 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b180b2880 msgr2=0x7f3b180b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:23.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:23.186+0000 7f3b312ef700 1 --2- 172.21.15.110:0/11808577 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b180b2880 0x7f3b180b4d90 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c19b930 tx=0x7f3b1400a380 comp rx=0 tx=0).stop 2024-05-09T04:45:23.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:23.186+0000 7f3b312ef700 1 -- 172.21.15.110:0/11808577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b2c1026c0 msgr2=0x7f3b2c19a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:23.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:23.186+0000 7f3b312ef700 1 --2- 172.21.15.110:0/11808577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b2c1026c0 0x7f3b2c19a880 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00d980 tx=0x7f3b1c00dc90 comp rx=0 tx=0).stop 2024-05-09T04:45:23.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:23.186+0000 7f3b312ef700 1 -- 172.21.15.110:0/11808577 shutdown_connections 2024-05-09T04:45:23.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:23.186+0000 7f3b312ef700 1 --2- 172.21.15.110:0/11808577 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b180b2880 0x7f3b180b4d90 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:23.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:23.186+0000 7f3b312ef700 1 --2- 172.21.15.110:0/11808577 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b2c10a110 0x7f3b2c1a1cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:23.185 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:23.186+0000 7f3b312ef700 1 --2- 172.21.15.110:0/11808577 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b2c1026c0 0x7f3b2c19a880 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:23.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:23.186+0000 7f3b312ef700 1 --2- 172.21.15.110:0/11808577 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b2c1077b0 0x7f3b2c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:23.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:23.186+0000 7f3b312ef700 1 -- 172.21.15.110:0/11808577 >> 172.21.15.110:0/11808577 conn(0x7f3b2c0fe020 msgr2=0x7f3b2c1086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:45:23.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:23.186+0000 7f3b312ef700 1 -- 172.21.15.110:0/11808577 shutdown_connections 2024-05-09T04:45:23.186 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:45:23.186+0000 7f3b312ef700 1 -- 172.21.15.110:0/11808577 wait complete. 2024-05-09T04:45:23.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:23 smithi110 ceph-mon[34904]: from='osd.8 [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T04:45:23.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:23 smithi110 ceph-mon[34904]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T04:45:23.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:23 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:23.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:23 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:23.473 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:23 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:23.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:23 smithi046 ceph-mon[30136]: from='osd.8 [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T04:45:23.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:23 smithi046 ceph-mon[30136]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T04:45:23.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:23 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:23.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:23 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:23.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:23 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:23.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:23 smithi072 ceph-mon[34088]: from='osd.8 [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T04:45:23.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:23 smithi072 ceph-mon[34088]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-09T04:45:23.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:23 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:23.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:23 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:23.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:23 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:23.829 DEBUG:teuthology.orchestra.run.smithi110:osd.8> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.8.service 2024-05-09T04:45:23.831 INFO:tasks.cephadm:Deploying osd.9 on smithi046 with /dev/vg_nvme/lv_1... 2024-05-09T04:45:23.831 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T04:45:24.093 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:45:24.535 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:24 smithi046 ceph-mon[30136]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:24.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:24 smithi046 ceph-mon[30136]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T04:45:24.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:24 smithi046 ceph-mon[30136]: osdmap e53: 9 total, 8 up, 9 in 2024-05-09T04:45:24.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:24 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:24.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:24 smithi046 ceph-mon[30136]: from='osd.8 [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:45:24.536 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:24 smithi046 ceph-mon[30136]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:45:24.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:24 smithi110 ceph-mon[34904]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:24.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:24 smithi110 ceph-mon[34904]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T04:45:24.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:24 smithi110 ceph-mon[34904]: osdmap e53: 9 total, 8 up, 9 in 2024-05-09T04:45:24.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:24 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:24.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:24 smithi110 ceph-mon[34904]: from='osd.8 [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:45:24.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:24 smithi110 ceph-mon[34904]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:45:24.557 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 04:45:24 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T04:45:24.331+0000 7f9d21afc700 -1 osd.8 0 waiting for initial osdmap 2024-05-09T04:45:24.557 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 04:45:24 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T04:45:24.341+0000 7f9d1d8b1700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:45:24.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:24 smithi072 ceph-mon[34088]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:24.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:24 smithi072 ceph-mon[34088]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-09T04:45:24.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:24 smithi072 ceph-mon[34088]: osdmap e53: 9 total, 8 up, 9 in 2024-05-09T04:45:24.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:24 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:24.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:24 smithi072 ceph-mon[34088]: from='osd.8 [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:45:24.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:24 smithi072 ceph-mon[34088]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:45:25.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:25 smithi110 ceph-mon[34904]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-09T04:45:25.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:25 smithi110 ceph-mon[34904]: osdmap e54: 9 total, 8 up, 9 in 2024-05-09T04:45:25.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:25 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:25.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:25 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:25.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:25 smithi046 ceph-mon[30136]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-09T04:45:25.558 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:25 smithi046 ceph-mon[30136]: osdmap e54: 9 total, 8 up, 9 in 2024-05-09T04:45:25.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:25 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:25.559 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:25 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:25.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:25 smithi072 ceph-mon[34088]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-09T04:45:25.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:25 smithi072 ceph-mon[34088]: osdmap e54: 9 total, 8 up, 9 in 2024-05-09T04:45:25.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:25 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:25.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:25 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:26.323 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T04:45:26.323 INFO:teuthology.orchestra.run.smithi046.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T04:45:26.323 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10+0 records in 2024-05-09T04:45:26.323 INFO:teuthology.orchestra.run.smithi046.stderr:10+0 records out 2024-05-09T04:45:26.323 INFO:teuthology.orchestra.run.smithi046.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.00859963 s, 1.2 GB/s 2024-05-09T04:45:26.323 INFO:teuthology.orchestra.run.smithi046.stderr:--> Zapping successful for: 2024-05-09T04:45:26.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:26 smithi046 ceph-mon[30136]: purged_snaps scrub starts 2024-05-09T04:45:26.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:26 smithi046 ceph-mon[30136]: purged_snaps scrub ok 2024-05-09T04:45:26.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:26 smithi046 ceph-mon[30136]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:26.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:26.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:26 smithi046 ceph-mon[30136]: osd.8 [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203] boot 2024-05-09T04:45:26.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:26 smithi046 ceph-mon[30136]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T04:45:26.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:26.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:26 smithi046 ceph-mon[30136]: osdmap e56: 9 total, 9 up, 9 in 2024-05-09T04:45:26.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:26 smithi072 ceph-mon[34088]: purged_snaps scrub starts 2024-05-09T04:45:26.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:26 smithi072 ceph-mon[34088]: purged_snaps scrub ok 2024-05-09T04:45:26.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:26 smithi072 ceph-mon[34088]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:26.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:26 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:26.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:26 smithi072 ceph-mon[34088]: osd.8 [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203] boot 2024-05-09T04:45:26.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:26 smithi072 ceph-mon[34088]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T04:45:26.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:26 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:26.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:26 smithi072 ceph-mon[34088]: osdmap e56: 9 total, 9 up, 9 in 2024-05-09T04:45:26.756 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:26 smithi110 ceph-mon[34904]: purged_snaps scrub starts 2024-05-09T04:45:26.757 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:26 smithi110 ceph-mon[34904]: purged_snaps scrub ok 2024-05-09T04:45:26.757 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:26 smithi110 ceph-mon[34904]: pgmap v193: 1 pgs: 1 peering; 577 KiB data, 235 MiB used, 715 GiB / 715 GiB avail 2024-05-09T04:45:26.757 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:26.757 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:26 smithi110 ceph-mon[34904]: osd.8 [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203] boot 2024-05-09T04:45:26.757 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:26 smithi110 ceph-mon[34904]: osdmap e55: 9 total, 9 up, 9 in 2024-05-09T04:45:26.757 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:45:26.757 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:26 smithi110 ceph-mon[34904]: osdmap e56: 9 total, 9 up, 9 in 2024-05-09T04:45:27.035 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi046:vg_nvme/lv_1 2024-05-09T04:45:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:45:28.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:27 smithi046 ceph-mon[30136]: Detected new or changed devices on smithi110 2024-05-09T04:45:28.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:28.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:28.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:28.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:28.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:28.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:27 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi110 to 5696M 2024-05-09T04:45:28.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:28.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:28.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:28.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:28.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:27 smithi046 ceph-mon[30136]: osdmap e57: 9 total, 9 up, 9 in 2024-05-09T04:45:28.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:27 smithi046 ceph-mon[30136]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:28.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:27 smithi072 ceph-mon[34088]: Detected new or changed devices on smithi110 2024-05-09T04:45:28.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:28.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:28.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:28.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:28.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:28.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:27 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi110 to 5696M 2024-05-09T04:45:28.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:28.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:28.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:28.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:28.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:27 smithi072 ceph-mon[34088]: osdmap e57: 9 total, 9 up, 9 in 2024-05-09T04:45:28.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:27 smithi072 ceph-mon[34088]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:28.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:27 smithi110 ceph-mon[34904]: Detected new or changed devices on smithi110 2024-05-09T04:45:28.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:28.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:28.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:28.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:28.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:28.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:27 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi110 to 5696M 2024-05-09T04:45:28.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:28.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:28.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:28.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:28.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:27 smithi110 ceph-mon[34904]: osdmap e57: 9 total, 9 up, 9 in 2024-05-09T04:45:28.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:27 smithi110 ceph-mon[34904]: pgmap v197: 1 pgs: 1 peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:29.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.067+0000 7fbde85e1700 1 -- 172.21.15.46:0/1703513416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde01049e0 msgr2=0x7fbde010c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:29.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.067+0000 7fbde85e1700 1 --2- 172.21.15.46:0/1703513416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde01049e0 0x7fbde010c6f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009990 tx=0x7fbdd8009ca0 comp rx=0 tx=0).stop 2024-05-09T04:45:29.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.067+0000 7fbde85e1700 1 -- 172.21.15.46:0/1703513416 shutdown_connections 2024-05-09T04:45:29.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.067+0000 7fbde85e1700 1 --2- 172.21.15.46:0/1703513416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde01049e0 0x7fbde010c6f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:29.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.067+0000 7fbde85e1700 1 --2- 172.21.15.46:0/1703513416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde01059b0 0x7fbde0105dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.067+0000 7fbde85e1700 1 --2- 172.21.15.46:0/1703513416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbde0103f60 0x7fbde0104370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.067+0000 7fbde85e1700 1 -- 172.21.15.46:0/1703513416 >> 172.21.15.46:0/1703513416 conn(0x7fbde00ff9d0 msgr2=0x7fbde0101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:45:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.067+0000 7fbde85e1700 1 -- 172.21.15.46:0/1703513416 shutdown_connections 2024-05-09T04:45:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.067+0000 7fbde85e1700 1 -- 172.21.15.46:0/1703513416 wait complete. 2024-05-09T04:45:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.068+0000 7fbde85e1700 1 Processor -- start 2024-05-09T04:45:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.068+0000 7fbde85e1700 1 -- start start 2024-05-09T04:45:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.068+0000 7fbde85e1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0103f60 0x7fbde0078570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.068+0000 7fbde85e1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbde01049e0 0x7fbde0076c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.068+0000 7fbde85e1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde01059b0 0x7fbde0077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde85e1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbde010ffc0 con 0x7fbde0103f60 2024-05-09T04:45:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde85e1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbde010fe40 con 0x7fbde01049e0 2024-05-09T04:45:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde85e1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbde0110140 con 0x7fbde01059b0 2024-05-09T04:45:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde637d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0103f60 0x7fbde0078570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde637d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0103f60 0x7fbde0078570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56916/0 (socket says 172.21.15.46:56916) 2024-05-09T04:45:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde637d700 1 -- 172.21.15.46:0/2878237911 learned_addr learned my addr 172.21.15.46:0/2878237911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:45:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde5b7c700 1 --2- 172.21.15.46:0/2878237911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbde01049e0 0x7fbde0076c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde6b7e700 1 --2- 172.21.15.46:0/2878237911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde01059b0 0x7fbde0077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde637d700 1 -- 172.21.15.46:0/2878237911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde01059b0 msgr2=0x7fbde0077160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde637d700 1 --2- 172.21.15.46:0/2878237911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde01059b0 0x7fbde0077160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde637d700 1 -- 172.21.15.46:0/2878237911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbde01049e0 msgr2=0x7fbde0076c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde637d700 1 --2- 172.21.15.46:0/2878237911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbde01049e0 0x7fbde0076c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde637d700 1 -- 172.21.15.46:0/2878237911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde00778c0 con 0x7fbde0103f60 2024-05-09T04:45:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde6b7e700 1 --2- 172.21.15.46:0/2878237911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde01059b0 0x7fbde0077160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:45:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde637d700 1 --2- 172.21.15.46:0/2878237911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0103f60 0x7fbde0078570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00b990 tx=0x7fbddc00bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:45:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbdd77fe700 1 -- 172.21.15.46:0/2878237911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc019070 con 0x7fbde0103f60 2024-05-09T04:45:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.069+0000 7fbde85e1700 1 -- 172.21.15.46:0/2878237911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde01a4b20 con 0x7fbde0103f60 2024-05-09T04:45:29.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.070+0000 7fbdd77fe700 1 -- 172.21.15.46:0/2878237911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbddc00cde0 con 0x7fbde0103f60 2024-05-09T04:45:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.070+0000 7fbdd77fe700 1 -- 172.21.15.46:0/2878237911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbddc012340 con 0x7fbde0103f60 2024-05-09T04:45:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.070+0000 7fbde85e1700 1 -- 172.21.15.46:0/2878237911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde01a5050 con 0x7fbde0103f60 2024-05-09T04:45:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.071+0000 7fbde85e1700 1 -- 172.21.15.46:0/2878237911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbde00512c0 con 0x7fbde0103f60 2024-05-09T04:45:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.072+0000 7fbdd77fe700 1 -- 172.21.15.46:0/2878237911 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbddc01c070 con 0x7fbde0103f60 2024-05-09T04:45:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.073+0000 7fbdd77fe700 1 --2- 172.21.15.46:0/2878237911 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbdcc0b2900 0x7fbdcc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.073+0000 7fbde5b7c700 1 --2- 172.21.15.46:0/2878237911 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbdcc0b2900 0x7fbdcc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.073+0000 7fbdd77fe700 1 -- 172.21.15.46:0/2878237911 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5829+0+0 (secure 0 0 0) 0x7fbddc0e13e0 con 0x7fbde0103f60 2024-05-09T04:45:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.074+0000 7fbde5b7c700 1 --2- 172.21.15.46:0/2878237911 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbdcc0b2900 0x7fbdcc0b4e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0005950 tx=0x7fbdd000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:45:29.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.077+0000 7fbdd77fe700 1 -- 172.21.15.46:0/2878237911 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbddc023070 con 0x7fbde0103f60 2024-05-09T04:45:29.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:29.304+0000 7fbde85e1700 1 -- 172.21.15.46:0/2878237911 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fbde002e990 con 0x7fbdcc0b2900 2024-05-09T04:45:29.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:45:29.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:45:29.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:29.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:45:29.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:45:29.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:29.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:45:29.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:45:29.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:30.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:30 smithi046 ceph-mon[30136]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:30.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:30 smithi046 ceph-mon[30136]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:45:30.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:30 smithi072 ceph-mon[34088]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:30.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:30 smithi072 ceph-mon[34088]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:45:30.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:30 smithi110 ceph-mon[34904]: pgmap v198: 1 pgs: 1 peering; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:30.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:30 smithi110 ceph-mon[34904]: from='client.14439 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi046:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:45:32.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:32 smithi072 ceph-mon[34088]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T04:45:32.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:32 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/565744203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e423de41-50ec-48f4-a4ed-216de33c4fb5"}]: dispatch 2024-05-09T04:45:32.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:32 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/565744203' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e423de41-50ec-48f4-a4ed-216de33c4fb5"}]': finished 2024-05-09T04:45:32.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:32 smithi072 ceph-mon[34088]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T04:45:32.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:32 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:45:32.700 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:32 smithi046 ceph-mon[30136]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T04:45:32.700 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:32 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/565744203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e423de41-50ec-48f4-a4ed-216de33c4fb5"}]: dispatch 2024-05-09T04:45:32.700 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:32 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/565744203' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e423de41-50ec-48f4-a4ed-216de33c4fb5"}]': finished 2024-05-09T04:45:32.701 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:32 smithi046 ceph-mon[30136]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T04:45:32.701 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:32 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:45:32.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:32 smithi110 ceph-mon[34904]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-09T04:45:32.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:32 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/565744203' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e423de41-50ec-48f4-a4ed-216de33c4fb5"}]: dispatch 2024-05-09T04:45:32.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:32 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/565744203' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e423de41-50ec-48f4-a4ed-216de33c4fb5"}]': finished 2024-05-09T04:45:32.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:32 smithi110 ceph-mon[34904]: osdmap e58: 10 total, 9 up, 10 in 2024-05-09T04:45:32.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:32 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:45:33.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:33 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1039311221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T04:45:33.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:33 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1039311221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T04:45:33.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:33 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1039311221' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-9/activate.monmap"}]: dispatch 2024-05-09T04:45:34.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:34 smithi072 ceph-mon[34088]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-09T04:45:34.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:34 smithi110 ceph-mon[34904]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-09T04:45:34.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:34 smithi046 ceph-mon[30136]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 78 KiB/s, 0 objects/s recovering 2024-05-09T04:45:36.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:36 smithi072 ceph-mon[34088]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-09T04:45:36.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:36 smithi110 ceph-mon[34904]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-09T04:45:36.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:36 smithi046 ceph-mon[30136]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-05-09T04:45:38.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:38 smithi072 ceph-mon[34088]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:45:38.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:38 smithi046 ceph-mon[30136]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:45:38.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:38 smithi110 ceph-mon[34904]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:45:40.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:40 smithi072 ceph-mon[34088]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:45:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T04:45:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:40.748 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:40 smithi046 ceph-mon[30136]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:45:40.748 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T04:45:40.748 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:40.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:40 smithi110 ceph-mon[34904]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:45:40.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-09T04:45:40.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:41.785 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:41 smithi046 ceph-mon[30136]: Deploying daemon osd.9 on smithi046 2024-05-09T04:45:41.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:41 smithi110 ceph-mon[34904]: Deploying daemon osd.9 on smithi046 2024-05-09T04:45:41.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:41 smithi072 ceph-mon[34088]: Deploying daemon osd.9 on smithi046 2024-05-09T04:45:42.628 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:42 smithi046 ceph-mon[30136]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:42.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:42 smithi110 ceph-mon[34904]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:42.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:42 smithi072 ceph-mon[34088]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:44.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:44 smithi046 ceph-mon[30136]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:44.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:44 smithi110 ceph-mon[34904]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:44.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:44 smithi072 ceph-mon[34088]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:46.705 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:46 smithi046 ceph-mon[30136]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:46.705 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:46 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:46.706 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:46 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:46.706 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:46 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:46.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:46 smithi110 ceph-mon[34904]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:46.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:46 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:46.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:46 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:46.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:46 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:46.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:46 smithi072 ceph-mon[34088]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:46.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:46 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:46.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:46 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:46.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:46 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:48.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:48 smithi046 ceph-mon[30136]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:48.682 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:48 smithi072 ceph-mon[34088]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:48.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:48 smithi110 ceph-mon[34904]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:50.244 INFO:teuthology.orchestra.run.smithi046.stdout:Created osd(s) 9 on host 'smithi046' 2024-05-09T04:45:50.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:50.243+0000 7fbdd77fe700 1 -- 172.21.15.46:0/2878237911 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbde002e990 con 0x7fbdcc0b2900 2024-05-09T04:45:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:50.246+0000 7fbde85e1700 1 -- 172.21.15.46:0/2878237911 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbdcc0b2900 msgr2=0x7fbdcc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:50.246+0000 7fbde85e1700 1 --2- 172.21.15.46:0/2878237911 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbdcc0b2900 0x7fbdcc0b4e10 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fbdd0005950 tx=0x7fbdd000a300 comp rx=0 tx=0).stop 2024-05-09T04:45:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:50.246+0000 7fbde85e1700 1 -- 172.21.15.46:0/2878237911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0103f60 msgr2=0x7fbde0078570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:50.246+0000 7fbde85e1700 1 --2- 172.21.15.46:0/2878237911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0103f60 0x7fbde0078570 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00b990 tx=0x7fbddc00bca0 comp rx=0 tx=0).stop 2024-05-09T04:45:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:50.246+0000 7fbde85e1700 1 -- 172.21.15.46:0/2878237911 shutdown_connections 2024-05-09T04:45:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:50.246+0000 7fbde85e1700 1 --2- 172.21.15.46:0/2878237911 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbdcc0b2900 0x7fbdcc0b4e10 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:50.246+0000 7fbde85e1700 1 --2- 172.21.15.46:0/2878237911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbde01059b0 0x7fbde0077160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:50.246+0000 7fbde85e1700 1 --2- 172.21.15.46:0/2878237911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbde0103f60 0x7fbde0078570 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:50.246+0000 7fbde85e1700 1 --2- 172.21.15.46:0/2878237911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbde01049e0 0x7fbde0076c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:50.246+0000 7fbde85e1700 1 -- 172.21.15.46:0/2878237911 >> 172.21.15.46:0/2878237911 conn(0x7fbde00ff9d0 msgr2=0x7fbde01086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:45:50.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:50.246+0000 7fbde85e1700 1 -- 172.21.15.46:0/2878237911 shutdown_connections 2024-05-09T04:45:50.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:45:50.246+0000 7fbde85e1700 1 -- 172.21.15.46:0/2878237911 wait complete. 2024-05-09T04:45:50.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:50 smithi046 ceph-mon[30136]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:50.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:50.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:50.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:50.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:50.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:50.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:50 smithi046 ceph-mon[30136]: from='osd.9 [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T04:45:50.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:50.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:50.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:50 smithi072 ceph-mon[34088]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:50 smithi072 ceph-mon[34088]: from='osd.9 [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T04:45:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:50.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:50 smithi110 ceph-mon[34904]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:50.874 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:50.874 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:50.874 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:50.874 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:50.874 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:50.874 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:50 smithi110 ceph-mon[34904]: from='osd.9 [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-09T04:45:50.875 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:50.875 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:50.875 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:45:51.020 DEBUG:teuthology.orchestra.run.smithi046:osd.9> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.9.service 2024-05-09T04:45:51.023 INFO:tasks.cephadm:Deploying osd.10 on smithi072 with /dev/vg_nvme/lv_1... 2024-05-09T04:45:51.023 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T04:45:51.271 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:45:51.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:51 smithi046 ceph-mon[30136]: from='osd.9 [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T04:45:51.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:51 smithi046 ceph-mon[30136]: osdmap e59: 10 total, 9 up, 10 in 2024-05-09T04:45:51.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:51 smithi046 ceph-mon[30136]: from='osd.9 [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T04:45:51.505 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:51 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:45:51.506 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 04:45:51 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T04:45:51.408+0000 7fa11abc8700 -1 osd.9 0 waiting for initial osdmap 2024-05-09T04:45:51.506 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 04:45:51 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T04:45:51.424+0000 7fa11697d700 -1 osd.9 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:45:51.678 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:51 smithi072 ceph-mon[34088]: from='osd.9 [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T04:45:51.678 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:51 smithi072 ceph-mon[34088]: osdmap e59: 10 total, 9 up, 10 in 2024-05-09T04:45:51.678 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:51 smithi072 ceph-mon[34088]: from='osd.9 [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T04:45:51.678 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:51 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:45:51.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:51 smithi110 ceph-mon[34904]: from='osd.9 [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-09T04:45:51.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:51 smithi110 ceph-mon[34904]: osdmap e59: 10 total, 9 up, 10 in 2024-05-09T04:45:51.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:51 smithi110 ceph-mon[34904]: from='osd.9 [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]: dispatch 2024-05-09T04:45:51.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:51 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:45:52.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:52 smithi046 ceph-mon[30136]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:52.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:52 smithi046 ceph-mon[30136]: from='osd.9 [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T04:45:52.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:52 smithi046 ceph-mon[30136]: osdmap e60: 10 total, 9 up, 10 in 2024-05-09T04:45:52.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:52 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:45:52.643 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:52 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:45:52.690 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:52 smithi072 ceph-mon[34088]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:52.691 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:52 smithi072 ceph-mon[34088]: from='osd.9 [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T04:45:52.691 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:52 smithi072 ceph-mon[34088]: osdmap e60: 10 total, 9 up, 10 in 2024-05-09T04:45:52.691 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:52 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:45:52.691 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:52 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:45:52.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:52 smithi110 ceph-mon[34904]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:52.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:52 smithi110 ceph-mon[34904]: from='osd.9 [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi046", "root=default"]}]': finished 2024-05-09T04:45:52.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:52 smithi110 ceph-mon[34904]: osdmap e60: 10 total, 9 up, 10 in 2024-05-09T04:45:52.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:52 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:45:52.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:52 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:45:53.383 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T04:45:53.384 INFO:teuthology.orchestra.run.smithi072.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T04:45:53.384 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10+0 records in 2024-05-09T04:45:53.384 INFO:teuthology.orchestra.run.smithi072.stderr:10+0 records out 2024-05-09T04:45:53.384 INFO:teuthology.orchestra.run.smithi072.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113374 s, 925 MB/s 2024-05-09T04:45:53.384 INFO:teuthology.orchestra.run.smithi072.stderr:--> Zapping successful for: 2024-05-09T04:45:53.677 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:53 smithi072 ceph-mon[34088]: purged_snaps scrub starts 2024-05-09T04:45:53.678 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:53 smithi072 ceph-mon[34088]: purged_snaps scrub ok 2024-05-09T04:45:53.678 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:53 smithi072 ceph-mon[34088]: osd.9 [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822] boot 2024-05-09T04:45:53.678 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:53 smithi072 ceph-mon[34088]: osdmap e61: 10 total, 10 up, 10 in 2024-05-09T04:45:53.678 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:53 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:45:53.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:53 smithi110 ceph-mon[34904]: purged_snaps scrub starts 2024-05-09T04:45:53.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:53 smithi110 ceph-mon[34904]: purged_snaps scrub ok 2024-05-09T04:45:53.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:53 smithi110 ceph-mon[34904]: osd.9 [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822] boot 2024-05-09T04:45:53.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:53 smithi110 ceph-mon[34904]: osdmap e61: 10 total, 10 up, 10 in 2024-05-09T04:45:53.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:53 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:45:53.826 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:53 smithi046 ceph-mon[30136]: purged_snaps scrub starts 2024-05-09T04:45:53.826 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:53 smithi046 ceph-mon[30136]: purged_snaps scrub ok 2024-05-09T04:45:53.826 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:53 smithi046 ceph-mon[30136]: osd.9 [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822] boot 2024-05-09T04:45:53.826 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:53 smithi046 ceph-mon[30136]: osdmap e61: 10 total, 10 up, 10 in 2024-05-09T04:45:53.827 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:53 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:45:54.154 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi072:vg_nvme/lv_1 2024-05-09T04:45:54.404 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T04:45:54.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:54 smithi046 ceph-mon[30136]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:54.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:54 smithi046 ceph-mon[30136]: osdmap e62: 10 total, 10 up, 10 in 2024-05-09T04:45:54.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:54 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:54.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:54 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:54.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:54 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:54.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:54 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:54.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:54 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:54.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:54 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:54.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:54 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:54.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:54 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:54.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:54 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:54.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:54 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:54.674 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:54 smithi072 ceph-mon[34088]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:54.675 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:54 smithi072 ceph-mon[34088]: osdmap e62: 10 total, 10 up, 10 in 2024-05-09T04:45:54.675 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:54 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:54.675 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:54 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:54.675 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:54 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:54.675 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:54 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:54.675 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:54 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:54.675 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:54 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:54.675 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:54 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:54.675 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:54 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:54.675 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:54 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:54.675 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:54 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:54.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:54 smithi110 ceph-mon[34904]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-09T04:45:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:54 smithi110 ceph-mon[34904]: osdmap e62: 10 total, 10 up, 10 in 2024-05-09T04:45:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:54 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:54 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:54 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:54 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:54 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:54 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:45:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:54 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:54 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:54 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:45:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:54 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:45:55.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:55 smithi072 ceph-mon[34088]: Detected new or changed devices on smithi046 2024-05-09T04:45:55.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:55 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi046 to 4273M 2024-05-09T04:45:55.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:55 smithi110 ceph-mon[34904]: Detected new or changed devices on smithi046 2024-05-09T04:45:55.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:55 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi046 to 4273M 2024-05-09T04:45:55.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:55 smithi046 ceph-mon[30136]: Detected new or changed devices on smithi046 2024-05-09T04:45:55.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:55 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi046 to 4273M 2024-05-09T04:45:56.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.306+0000 7f799cfb9700 1 -- 172.21.15.72:0/381420734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f799810a0f0 msgr2=0x7f799810c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:56.306 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.306+0000 7f799cfb9700 1 --2- 172.21.15.72:0/381420734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f799810a0f0 0x7f799810c510 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f799000b0a0 tx=0x7f799000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:45:56.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.306+0000 7f799cfb9700 1 -- 172.21.15.72:0/381420734 shutdown_connections 2024-05-09T04:45:56.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.306+0000 7f799cfb9700 1 --2- 172.21.15.72:0/381420734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f799810a0f0 0x7f799810c510 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:56.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.306+0000 7f799cfb9700 1 --2- 172.21.15.72:0/381420734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7998104e30 0x7f7998107250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:56.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.306+0000 7f799cfb9700 1 --2- 172.21.15.72:0/381420734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998107790 0x7f7998109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:56.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.306+0000 7f799cfb9700 1 -- 172.21.15.72:0/381420734 >> 172.21.15.72:0/381420734 conn(0x7f79980fe890 msgr2=0x7f7998100d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:45:56.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.306+0000 7f799cfb9700 1 -- 172.21.15.72:0/381420734 shutdown_connections 2024-05-09T04:45:56.307 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.306+0000 7f799cfb9700 1 -- 172.21.15.72:0/381420734 wait complete. 2024-05-09T04:45:56.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.307+0000 7f799cfb9700 1 Processor -- start 2024-05-09T04:45:56.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.307+0000 7f799cfb9700 1 -- start start 2024-05-09T04:45:56.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.307+0000 7f799cfb9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998104e30 0x7f799819ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:56.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.307+0000 7f799cfb9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7998107790 0x7f799819cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:56.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.307+0000 7f799cfb9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f799810a0f0 0x7f79981a3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:56.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.307+0000 7f799cfb9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f799810fd90 con 0x7f7998104e30 2024-05-09T04:45:56.309 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.307+0000 7f799cfb9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f799810fc10 con 0x7f7998107790 2024-05-09T04:45:56.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.307+0000 7f799cfb9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f799810ff10 con 0x7f799810a0f0 2024-05-09T04:45:56.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.307+0000 7f798dd9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7998107790 0x7f799819cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:56.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f798dd9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7998107790 0x7f799819cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.72:45972/0 (socket says 172.21.15.72:45972) 2024-05-09T04:45:56.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f798dd9b700 1 -- 172.21.15.72:0/174087187 learned_addr learned my addr 172.21.15.72:0/174087187 (peer_addr_for_me v2:172.21.15.72:0/0) 2024-05-09T04:45:56.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f799659c700 1 --2- 172.21.15.72:0/174087187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998104e30 0x7f799819ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:56.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f7996d9d700 1 --2- 172.21.15.72:0/174087187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f799810a0f0 0x7f79981a3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:56.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f798dd9b700 1 -- 172.21.15.72:0/174087187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f799810a0f0 msgr2=0x7f79981a3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:56.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f798dd9b700 1 --2- 172.21.15.72:0/174087187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f799810a0f0 0x7f79981a3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:56.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f798dd9b700 1 -- 172.21.15.72:0/174087187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998104e30 msgr2=0x7f799819ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:45:56.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f798dd9b700 1 --2- 172.21.15.72:0/174087187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998104e30 0x7f799819ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:45:56.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f798dd9b700 1 -- 172.21.15.72:0/174087187 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79981a4520 con 0x7f7998107790 2024-05-09T04:45:56.310 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f799659c700 1 --2- 172.21.15.72:0/174087187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998104e30 0x7f799819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:45:56.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f798dd9b700 1 --2- 172.21.15.72:0/174087187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7998107790 0x7f799819cf90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f798000c920 tx=0x7f798000cce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:45:56.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f798ffff700 1 -- 172.21.15.72:0/174087187 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980007b00 con 0x7f7998107790 2024-05-09T04:45:56.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f799cfb9700 1 -- 172.21.15.72:0/174087187 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79981a4810 con 0x7f7998107790 2024-05-09T04:45:56.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f798ffff700 1 -- 172.21.15.72:0/174087187 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7980004ce0 con 0x7f7998107790 2024-05-09T04:45:56.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f798ffff700 1 -- 172.21.15.72:0/174087187 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980005650 con 0x7f7998107790 2024-05-09T04:45:56.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.308+0000 7f799cfb9700 1 -- 172.21.15.72:0/174087187 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79981a4da0 con 0x7f7998107790 2024-05-09T04:45:56.311 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.309+0000 7f799cfb9700 1 -- 172.21.15.72:0/174087187 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79980512c0 con 0x7f7998107790 2024-05-09T04:45:56.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.311+0000 7f798ffff700 1 -- 172.21.15.72:0/174087187 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7980007ca0 con 0x7f7998107790 2024-05-09T04:45:56.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.311+0000 7f798ffff700 1 --2- 172.21.15.72:0/174087187 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f79840b2880 0x7f79840b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:45:56.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.312+0000 7f798ffff700 1 -- 172.21.15.72:0/174087187 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6261+0+0 (secure 0 0 0) 0x7f79800e1c80 con 0x7f7998107790 2024-05-09T04:45:56.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.312+0000 7f799659c700 1 --2- 172.21.15.72:0/174087187 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f79840b2880 0x7f79840b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:45:56.312 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.312+0000 7f799659c700 1 --2- 172.21.15.72:0/174087187 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f79840b2880 0x7f79840b4d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7988005fd0 tx=0x7f7988005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:45:56.315 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.315+0000 7f798ffff700 1 -- 172.21.15.72:0/174087187 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7980014030 con 0x7f7998107790 2024-05-09T04:45:56.526 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:45:56.526+0000 7f799cfb9700 1 -- 172.21.15.72:0/174087187 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7998063c00 con 0x7f79840b2880 2024-05-09T04:45:56.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:56 smithi072 ceph-mon[34088]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:45:56.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:56 smithi110 ceph-mon[34904]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:45:56.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:56 smithi046 ceph-mon[30136]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:45:57.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:57 smithi110 ceph-mon[34904]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:45:57.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:57 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:45:57.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:57 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:45:57.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:57 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:57.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:57 smithi046 ceph-mon[30136]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:45:57.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:57 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:45:57.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:57 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:45:57.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:57 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:57.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:57 smithi072 ceph-mon[34088]: from='client.24350 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi072:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:45:57.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:57 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:45:57.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:57 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:45:57.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:57 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:45:58.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:58 smithi110 ceph-mon[34904]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:45:58.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:58 smithi046 ceph-mon[30136]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:45:58.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:58 smithi072 ceph-mon[34088]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:45:59.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:59 smithi110 ceph-mon[34904]: from='client.? 172.21.15.72:0/409761729' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f594d004-9f80-4508-b127-fb693d355f40"}]: dispatch 2024-05-09T04:45:59.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:59 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f594d004-9f80-4508-b127-fb693d355f40"}]: dispatch 2024-05-09T04:45:59.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:59 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f594d004-9f80-4508-b127-fb693d355f40"}]': finished 2024-05-09T04:45:59.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:59 smithi110 ceph-mon[34904]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T04:45:59.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:45:59 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:45:59.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:59 smithi046 ceph-mon[30136]: from='client.? 172.21.15.72:0/409761729' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f594d004-9f80-4508-b127-fb693d355f40"}]: dispatch 2024-05-09T04:45:59.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:59 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f594d004-9f80-4508-b127-fb693d355f40"}]: dispatch 2024-05-09T04:45:59.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:59 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f594d004-9f80-4508-b127-fb693d355f40"}]': finished 2024-05-09T04:45:59.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:59 smithi046 ceph-mon[30136]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T04:45:59.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:45:59 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:45:59.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:59 smithi072 ceph-mon[34088]: from='client.? 172.21.15.72:0/409761729' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f594d004-9f80-4508-b127-fb693d355f40"}]: dispatch 2024-05-09T04:45:59.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:59 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f594d004-9f80-4508-b127-fb693d355f40"}]: dispatch 2024-05-09T04:45:59.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:59 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f594d004-9f80-4508-b127-fb693d355f40"}]': finished 2024-05-09T04:45:59.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:59 smithi072 ceph-mon[34088]: osdmap e63: 11 total, 10 up, 11 in 2024-05-09T04:45:59.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:45:59 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:46:00.735 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:00 smithi046 ceph-mon[30136]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:00.735 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:00 smithi046 ceph-mon[30136]: from='client.? 172.21.15.72:0/2731777350' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T04:46:00.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:00 smithi110 ceph-mon[34904]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:00.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:00 smithi110 ceph-mon[34904]: from='client.? 172.21.15.72:0/2731777350' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T04:46:00.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:00 smithi072 ceph-mon[34088]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:00.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:00 smithi072 ceph-mon[34088]: from='client.? 172.21.15.72:0/2731777350' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-10/activate.monmap"}]: dispatch 2024-05-09T04:46:02.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:02 smithi110 ceph-mon[34904]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:02.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:02 smithi046 ceph-mon[30136]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:02.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:02 smithi072 ceph-mon[34088]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:04.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:04 smithi072 ceph-mon[34088]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:04.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:04 smithi110 ceph-mon[34904]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:04.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:04 smithi046 ceph-mon[30136]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:06.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:06 smithi110 ceph-mon[34904]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:06.822 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:06 smithi072 ceph-mon[34088]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:06.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:06 smithi046 ceph-mon[30136]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:07.721 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:07 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T04:46:07.721 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:07 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:07.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:07 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T04:46:07.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:07 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:07.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:07 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-09T04:46:07.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:07 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:08.708 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:08 smithi110 ceph-mon[34904]: Deploying daemon osd.10 on smithi072 2024-05-09T04:46:08.708 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:08 smithi110 ceph-mon[34904]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:08.791 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:08 smithi072 ceph-mon[34088]: Deploying daemon osd.10 on smithi072 2024-05-09T04:46:08.791 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:08 smithi072 ceph-mon[34088]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:08.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:08 smithi046 ceph-mon[30136]: Deploying daemon osd.10 on smithi072 2024-05-09T04:46:08.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:08 smithi046 ceph-mon[30136]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:10.751 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:10 smithi072 ceph-mon[34088]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:10.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:10 smithi110 ceph-mon[34904]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:10.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:10 smithi046 ceph-mon[30136]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:12.721 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:12 smithi072 ceph-mon[34088]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:12.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:12 smithi110 ceph-mon[34904]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:12.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:12 smithi046 ceph-mon[30136]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:14.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:13 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:14.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:13 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:14.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:13 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:14.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:13 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:14.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:13 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:14.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:13 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:14.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:13 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:14.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:13 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:14.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:13 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:14.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:14 smithi072 ceph-mon[34088]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:15.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:14 smithi110 ceph-mon[34904]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:15.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:14 smithi046 ceph-mon[30136]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:16.858 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:16 smithi072 ceph-mon[34088]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:16.858 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:16 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:16.858 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:16 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:16.858 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:16 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:16.859 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:16 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:16.859 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:16 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:16.859 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:16 smithi072 ceph-mon[34088]: from='osd.10 [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T04:46:16.859 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:16 smithi072 ceph-mon[34088]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T04:46:16.898 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:46:16.898+0000 7f798ffff700 1 -- 172.21.15.72:0/174087187 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7998063c00 con 0x7f79840b2880 2024-05-09T04:46:16.899 INFO:teuthology.orchestra.run.smithi072.stdout:Created osd(s) 10 on host 'smithi072' 2024-05-09T04:46:16.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:46:16.901+0000 7f799cfb9700 1 -- 172.21.15.72:0/174087187 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f79840b2880 msgr2=0x7f79840b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:16.901 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:46:16.901+0000 7f799cfb9700 1 --2- 172.21.15.72:0/174087187 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f79840b2880 0x7f79840b4d90 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7988005fd0 tx=0x7f7988005e30 comp rx=0 tx=0).stop 2024-05-09T04:46:16.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:46:16.901+0000 7f799cfb9700 1 -- 172.21.15.72:0/174087187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7998107790 msgr2=0x7f799819cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:16.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:46:16.901+0000 7f799cfb9700 1 --2- 172.21.15.72:0/174087187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7998107790 0x7f799819cf90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f798000c920 tx=0x7f798000cce0 comp rx=0 tx=0).stop 2024-05-09T04:46:16.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:46:16.901+0000 7f799cfb9700 1 -- 172.21.15.72:0/174087187 shutdown_connections 2024-05-09T04:46:16.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:46:16.901+0000 7f799cfb9700 1 --2- 172.21.15.72:0/174087187 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f79840b2880 0x7f79840b4d90 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:16.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:46:16.901+0000 7f799cfb9700 1 --2- 172.21.15.72:0/174087187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f799810a0f0 0x7f79981a3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:16.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:46:16.901+0000 7f799cfb9700 1 --2- 172.21.15.72:0/174087187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7998104e30 0x7f799819ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:16.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:46:16.901+0000 7f799cfb9700 1 --2- 172.21.15.72:0/174087187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7998107790 0x7f799819cf90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:16.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:46:16.901+0000 7f799cfb9700 1 -- 172.21.15.72:0/174087187 >> 172.21.15.72:0/174087187 conn(0x7f79980fe890 msgr2=0x7f79981046c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:16.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:46:16.901+0000 7f799cfb9700 1 -- 172.21.15.72:0/174087187 shutdown_connections 2024-05-09T04:46:16.902 INFO:teuthology.orchestra.run.smithi072.stderr:2024-05-09T04:46:16.901+0000 7f799cfb9700 1 -- 172.21.15.72:0/174087187 wait complete. 2024-05-09T04:46:17.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:16 smithi110 ceph-mon[34904]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:17.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:16 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:16 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:16 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:16 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:16 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:16 smithi110 ceph-mon[34904]: from='osd.10 [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T04:46:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:16 smithi110 ceph-mon[34904]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T04:46:17.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:16 smithi046 ceph-mon[30136]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:17.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:16 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:17.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:16 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:17.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:16 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:17.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:16 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:17.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:16 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:17.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:16 smithi046 ceph-mon[30136]: from='osd.10 [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T04:46:17.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:16 smithi046 ceph-mon[30136]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-09T04:46:17.545 DEBUG:teuthology.orchestra.run.smithi072:osd.10> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.10.service 2024-05-09T04:46:17.547 INFO:tasks.cephadm:Deploying osd.11 on smithi110 with /dev/vg_nvme/lv_1... 2024-05-09T04:46:17.547 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-09T04:46:17.797 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:46:18.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 04:46:18 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T04:46:18.035+0000 7f189c990700 -1 osd.10 0 waiting for initial osdmap 2024-05-09T04:46:18.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 04:46:18 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T04:46:18.049+0000 7f1898745700 -1 osd.10 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:46:18.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:17 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:18.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:17 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:18.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:17 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:18.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:17 smithi072 ceph-mon[34088]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T04:46:18.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:17 smithi072 ceph-mon[34088]: from='osd.10 [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:46:18.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:17 smithi072 ceph-mon[34088]: osdmap e64: 11 total, 10 up, 11 in 2024-05-09T04:46:18.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:17 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:46:18.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:17 smithi072 ceph-mon[34088]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:46:18.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:17 smithi072 ceph-mon[34088]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:18.176 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:18.176 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:18.176 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:18.176 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:17 smithi110 ceph-mon[34904]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T04:46:18.176 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:17 smithi110 ceph-mon[34904]: from='osd.10 [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:46:18.176 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:17 smithi110 ceph-mon[34904]: osdmap e64: 11 total, 10 up, 11 in 2024-05-09T04:46:18.176 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:17 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:46:18.176 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:17 smithi110 ceph-mon[34904]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:46:18.177 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:17 smithi110 ceph-mon[34904]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:17 smithi046 ceph-mon[30136]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-09T04:46:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:17 smithi046 ceph-mon[30136]: from='osd.10 [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:46:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:17 smithi046 ceph-mon[30136]: osdmap e64: 11 total, 10 up, 11 in 2024-05-09T04:46:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:17 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:46:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:17 smithi046 ceph-mon[30136]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]: dispatch 2024-05-09T04:46:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:17 smithi046 ceph-mon[30136]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-09T04:46:19.208 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:19 smithi110 ceph-mon[34904]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-09T04:46:19.208 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:19 smithi110 ceph-mon[34904]: osdmap e65: 11 total, 10 up, 11 in 2024-05-09T04:46:19.209 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:19 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:46:19.209 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:19 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:46:19.282 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:19 smithi072 ceph-mon[34088]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-09T04:46:19.282 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:19 smithi072 ceph-mon[34088]: osdmap e65: 11 total, 10 up, 11 in 2024-05-09T04:46:19.282 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:19 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:46:19.282 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:19 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:46:19.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:19 smithi046 ceph-mon[30136]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi072", "root=default"]}]': finished 2024-05-09T04:46:19.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:19 smithi046 ceph-mon[30136]: osdmap e65: 11 total, 10 up, 11 in 2024-05-09T04:46:19.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:19 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:46:19.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:19 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:46:19.919 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-09T04:46:19.919 INFO:teuthology.orchestra.run.smithi110.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-09T04:46:19.919 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 10+0 records in 2024-05-09T04:46:19.919 INFO:teuthology.orchestra.run.smithi110.stderr:10+0 records out 2024-05-09T04:46:19.919 INFO:teuthology.orchestra.run.smithi110.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.00836752 s, 1.3 GB/s 2024-05-09T04:46:19.919 INFO:teuthology.orchestra.run.smithi110.stderr: stderr: 2024-05-09T04:46:19.919 INFO:teuthology.orchestra.run.smithi110.stderr:--> Zapping successful for: 2024-05-09T04:46:20.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:20 smithi110 ceph-mon[34904]: purged_snaps scrub starts 2024-05-09T04:46:20.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:20 smithi110 ceph-mon[34904]: purged_snaps scrub ok 2024-05-09T04:46:20.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:20 smithi110 ceph-mon[34904]: osd.10 [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898] boot 2024-05-09T04:46:20.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:20 smithi110 ceph-mon[34904]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T04:46:20.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:20 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:46:20.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:20 smithi110 ceph-mon[34904]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:20.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:20 smithi110 ceph-mon[34904]: osdmap e67: 11 total, 11 up, 11 in 2024-05-09T04:46:20.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:20 smithi046 ceph-mon[30136]: purged_snaps scrub starts 2024-05-09T04:46:20.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:20 smithi046 ceph-mon[30136]: purged_snaps scrub ok 2024-05-09T04:46:20.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:20 smithi046 ceph-mon[30136]: osd.10 [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898] boot 2024-05-09T04:46:20.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:20 smithi046 ceph-mon[30136]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T04:46:20.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:20 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:46:20.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:20 smithi046 ceph-mon[30136]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:20.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:20 smithi046 ceph-mon[30136]: osdmap e67: 11 total, 11 up, 11 in 2024-05-09T04:46:20.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:20 smithi072 ceph-mon[34088]: purged_snaps scrub starts 2024-05-09T04:46:20.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:20 smithi072 ceph-mon[34088]: purged_snaps scrub ok 2024-05-09T04:46:20.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:20 smithi072 ceph-mon[34088]: osd.10 [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898] boot 2024-05-09T04:46:20.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:20 smithi072 ceph-mon[34088]: osdmap e66: 11 total, 11 up, 11 in 2024-05-09T04:46:20.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:20 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:46:20.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:20 smithi072 ceph-mon[34088]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:20.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:20 smithi072 ceph-mon[34088]: osdmap e67: 11 total, 11 up, 11 in 2024-05-09T04:46:20.767 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi110:vg_nvme/lv_1 2024-05-09T04:46:21.013 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:46:22.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:21 smithi110 ceph-mon[34904]: Detected new or changed devices on smithi072 2024-05-09T04:46:22.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:21 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:22.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:21 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:22.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:21 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:21 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:21 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:21 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:21 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi072 to 4272M 2024-05-09T04:46:22.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:21 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:22.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:21 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:22.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:21 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:22.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:21 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:22.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:21 smithi110 ceph-mon[34904]: osdmap e68: 11 total, 11 up, 11 in 2024-05-09T04:46:22.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:21 smithi110 ceph-mon[34904]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:22.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:21 smithi046 ceph-mon[30136]: Detected new or changed devices on smithi072 2024-05-09T04:46:22.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:21 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:22.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:21 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:22.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:21 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:21 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:21 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:21 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:21 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi072 to 4272M 2024-05-09T04:46:22.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:21 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:22.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:21 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:22.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:21 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:22.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:21 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:22.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:21 smithi046 ceph-mon[30136]: osdmap e68: 11 total, 11 up, 11 in 2024-05-09T04:46:22.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:21 smithi046 ceph-mon[30136]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:22.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:21 smithi072 ceph-mon[34088]: Detected new or changed devices on smithi072 2024-05-09T04:46:22.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:21 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:22.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:21 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:22.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:21 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:21 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:21 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:21 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:22.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:21 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi072 to 4272M 2024-05-09T04:46:22.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:21 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:22.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:21 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:22.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:21 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:22.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:21 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:22.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:21 smithi072 ceph-mon[34088]: osdmap e68: 11 total, 11 up, 11 in 2024-05-09T04:46:22.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:21 smithi072 ceph-mon[34088]: pgmap v235: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:22.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.875+0000 7f7a582eb700 1 -- 172.21.15.110:0/2194426822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a501054a0 msgr2=0x7f7a501058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:22.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.875+0000 7f7a582eb700 1 --2- 172.21.15.110:0/2194426822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a501054a0 0x7f7a501058b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009990 tx=0x7f7a4c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:46:22.895 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.875+0000 7f7a582eb700 1 -- 172.21.15.110:0/2194426822 shutdown_connections 2024-05-09T04:46:22.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.875+0000 7f7a582eb700 1 --2- 172.21.15.110:0/2194426822 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a50105e40 0x7f7a5010c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:22.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.875+0000 7f7a582eb700 1 --2- 172.21.15.110:0/2194426822 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a50104370 0x7f7a50104780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:22.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.875+0000 7f7a582eb700 1 --2- 172.21.15.110:0/2194426822 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a501054a0 0x7f7a501058b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:22.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.875+0000 7f7a582eb700 1 -- 172.21.15.110:0/2194426822 >> 172.21.15.110:0/2194426822 conn(0x7f7a500ff9d0 msgr2=0x7f7a50101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:22.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.875+0000 7f7a582eb700 1 -- 172.21.15.110:0/2194426822 shutdown_connections 2024-05-09T04:46:22.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.875+0000 7f7a582eb700 1 -- 172.21.15.110:0/2194426822 wait complete. 2024-05-09T04:46:22.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.875+0000 7f7a582eb700 1 Processor -- start 2024-05-09T04:46:22.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.875+0000 7f7a582eb700 1 -- start start 2024-05-09T04:46:22.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a582eb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a50104370 0x7f7a5019a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:22.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a582eb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a501054a0 0x7f7a5019ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:22.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a582eb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a50105e40 0x7f7a501a1de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:22.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a582eb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5010fdf0 con 0x7f7a50104370 2024-05-09T04:46:22.896 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a582eb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5010fc70 con 0x7f7a501054a0 2024-05-09T04:46:22.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a582eb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7a5010ff70 con 0x7f7a50105e40 2024-05-09T04:46:22.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a56888700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a50105e40 0x7f7a501a1de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:22.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a56888700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a50105e40 0x7f7a501a1de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:57700/0 (socket says 172.21.15.110:57700) 2024-05-09T04:46:22.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a56888700 1 -- 172.21.15.110:0/2919551674 learned_addr learned my addr 172.21.15.110:0/2919551674 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-09T04:46:22.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a56087700 1 --2- 172.21.15.110:0/2919551674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a50104370 0x7f7a5019a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:22.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a55886700 1 --2- 172.21.15.110:0/2919551674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a501054a0 0x7f7a5019ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:22.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a56888700 1 -- 172.21.15.110:0/2919551674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a501054a0 msgr2=0x7f7a5019ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:22.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a56888700 1 --2- 172.21.15.110:0/2919551674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a501054a0 0x7f7a5019ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:22.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a56888700 1 -- 172.21.15.110:0/2919551674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a50104370 msgr2=0x7f7a5019a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:22.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a56888700 1 --2- 172.21.15.110:0/2919551674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a50104370 0x7f7a5019a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:22.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a56888700 1 -- 172.21.15.110:0/2919551674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a501a2460 con 0x7f7a50105e40 2024-05-09T04:46:22.897 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a56888700 1 --2- 172.21.15.110:0/2919551674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a50105e40 0x7f7a501a1de0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800ada0 tx=0x7f7a4800c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:22.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a477fe700 1 -- 172.21.15.110:0/2919551674 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48017020 con 0x7f7a50105e40 2024-05-09T04:46:22.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.876+0000 7f7a477fe700 1 -- 172.21.15.110:0/2919551674 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a4800cd80 con 0x7f7a50105e40 2024-05-09T04:46:22.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.877+0000 7f7a477fe700 1 -- 172.21.15.110:0/2919551674 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a48005260 con 0x7f7a50105e40 2024-05-09T04:46:22.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.877+0000 7f7a582eb700 1 -- 172.21.15.110:0/2919551674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a501a2750 con 0x7f7a50105e40 2024-05-09T04:46:22.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.877+0000 7f7a582eb700 1 -- 172.21.15.110:0/2919551674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a501a2c90 con 0x7f7a50105e40 2024-05-09T04:46:22.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.877+0000 7f7a582eb700 1 -- 172.21.15.110:0/2919551674 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a500512c0 con 0x7f7a50105e40 2024-05-09T04:46:22.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.879+0000 7f7a477fe700 1 -- 172.21.15.110:0/2919551674 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7a4800a090 con 0x7f7a50105e40 2024-05-09T04:46:22.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.880+0000 7f7a477fe700 1 --2- 172.21.15.110:0/2919551674 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7a3c0b2900 0x7f7a3c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:22.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.880+0000 7f7a477fe700 1 -- 172.21.15.110:0/2919551674 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6694+0+0 (secure 0 0 0) 0x7f7a480e17c0 con 0x7f7a50105e40 2024-05-09T04:46:22.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.880+0000 7f7a56087700 1 --2- 172.21.15.110:0/2919551674 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7a3c0b2900 0x7f7a3c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:22.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.881+0000 7f7a56087700 1 --2- 172.21.15.110:0/2919551674 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7a3c0b2900 0x7f7a3c0b4e10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7a501053a0 tx=0x7f7a40007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:22.898 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:22.885+0000 7f7a477fe700 1 -- 172.21.15.110:0/2919551674 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7a48021020 con 0x7f7a50105e40 2024-05-09T04:46:23.097 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:23.097+0000 7f7a582eb700 1 -- 172.21.15.110:0/2919551674 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7a50063c00 con 0x7f7a3c0b2900 2024-05-09T04:46:23.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:23 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:46:23.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:23 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:46:23.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:23 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:23.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:23 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:46:23.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:23 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:46:23.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:23 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:23.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:23 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-09T04:46:23.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:23 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-09T04:46:23.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:23 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:24.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:24 smithi110 ceph-mon[34904]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:46:24.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:24 smithi110 ceph-mon[34904]: pgmap v236: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:24.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:24 smithi046 ceph-mon[30136]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:46:24.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:24 smithi046 ceph-mon[30136]: pgmap v236: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:24.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:24 smithi072 ceph-mon[34088]: from='client.24352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi110:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:46:24.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:24 smithi072 ceph-mon[34088]: pgmap v236: 1 pgs: 1 remapped+peering; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:26.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:26 smithi110 ceph-mon[34904]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-09T04:46:26.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:26 smithi110 ceph-mon[34904]: from='client.? 172.21.15.110:0/1145421351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f771d29f-89e3-4f96-9c29-00e424230c1a"}]: dispatch 2024-05-09T04:46:26.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:26 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f771d29f-89e3-4f96-9c29-00e424230c1a"}]: dispatch 2024-05-09T04:46:26.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:26 smithi110 ceph-mon[34904]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f771d29f-89e3-4f96-9c29-00e424230c1a"}]': finished 2024-05-09T04:46:26.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:26 smithi110 ceph-mon[34904]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T04:46:26.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:26.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:26 smithi046 ceph-mon[30136]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-09T04:46:26.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:26 smithi046 ceph-mon[30136]: from='client.? 172.21.15.110:0/1145421351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f771d29f-89e3-4f96-9c29-00e424230c1a"}]: dispatch 2024-05-09T04:46:26.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:26 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f771d29f-89e3-4f96-9c29-00e424230c1a"}]: dispatch 2024-05-09T04:46:26.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:26 smithi046 ceph-mon[30136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f771d29f-89e3-4f96-9c29-00e424230c1a"}]': finished 2024-05-09T04:46:26.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:26 smithi046 ceph-mon[30136]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T04:46:26.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:26.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:26 smithi072 ceph-mon[34088]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 92 KiB/s, 0 objects/s recovering 2024-05-09T04:46:26.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:26 smithi072 ceph-mon[34088]: from='client.? 172.21.15.110:0/1145421351' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f771d29f-89e3-4f96-9c29-00e424230c1a"}]: dispatch 2024-05-09T04:46:26.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:26 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f771d29f-89e3-4f96-9c29-00e424230c1a"}]: dispatch 2024-05-09T04:46:26.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:26 smithi072 ceph-mon[34088]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f771d29f-89e3-4f96-9c29-00e424230c1a"}]': finished 2024-05-09T04:46:26.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:26 smithi072 ceph-mon[34088]: osdmap e69: 12 total, 11 up, 12 in 2024-05-09T04:46:26.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:26 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:27.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:27 smithi072 ceph-mon[34088]: from='client.? 172.21.15.110:0/2134008750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T04:46:27.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:27 smithi110 ceph-mon[34904]: from='client.? 172.21.15.110:0/2134008750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T04:46:27.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:27 smithi046 ceph-mon[30136]: from='client.? 172.21.15.110:0/2134008750' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap", "output-file": "/var/lib/ceph/osd/ceph-11/activate.monmap"}]: dispatch 2024-05-09T04:46:28.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:28 smithi072 ceph-mon[34088]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-09T04:46:28.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:28 smithi110 ceph-mon[34904]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-09T04:46:28.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:28 smithi046 ceph-mon[30136]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-05-09T04:46:30.591 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:30 smithi046 ceph-mon[30136]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-09T04:46:30.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:30 smithi072 ceph-mon[34088]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-09T04:46:30.680 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:30 smithi110 ceph-mon[34904]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-05-09T04:46:32.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:32 smithi072 ceph-mon[34088]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:46:32.651 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:32 smithi110 ceph-mon[34904]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:46:32.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:32 smithi046 ceph-mon[30136]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:46:34.585 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:34 smithi110 ceph-mon[34904]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:46:34.585 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:34 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T04:46:34.585 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:34 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:34.585 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:34 smithi110 ceph-mon[34904]: Deploying daemon osd.11 on smithi110 2024-05-09T04:46:34.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:34 smithi072 ceph-mon[34088]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:46:34.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:34 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T04:46:34.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:34 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:34.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:34 smithi072 ceph-mon[34088]: Deploying daemon osd.11 on smithi110 2024-05-09T04:46:34.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:34 smithi046 ceph-mon[30136]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-09T04:46:34.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:34 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-09T04:46:34.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:34 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:34.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:34 smithi046 ceph-mon[30136]: Deploying daemon osd.11 on smithi110 2024-05-09T04:46:36.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:36 smithi072 ceph-mon[34088]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:36.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:36 smithi110 ceph-mon[34904]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:36.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:36 smithi046 ceph-mon[30136]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:38.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:38 smithi072 ceph-mon[34088]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:38.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:38 smithi110 ceph-mon[34904]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:38.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:38 smithi046 ceph-mon[30136]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:40 smithi110 ceph-mon[34904]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:41.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:40 smithi046 ceph-mon[30136]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:41.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:41.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:41.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:40 smithi072 ceph-mon[34088]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:42.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:42 smithi110 ceph-mon[34904]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:43.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:42 smithi046 ceph-mon[30136]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:43.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:42 smithi072 ceph-mon[34088]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:43.870 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:43.870+0000 7f7a477fe700 1 -- 172.21.15.110:0/2919551674 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7a50063c00 con 0x7f7a3c0b2900 2024-05-09T04:46:43.871 INFO:teuthology.orchestra.run.smithi110.stdout:Created osd(s) 11 on host 'smithi110' 2024-05-09T04:46:43.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:43.873+0000 7f7a582eb700 1 -- 172.21.15.110:0/2919551674 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7a3c0b2900 msgr2=0x7f7a3c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:43.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:43.873+0000 7f7a582eb700 1 --2- 172.21.15.110:0/2919551674 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7a3c0b2900 0x7f7a3c0b4e10 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7a501053a0 tx=0x7f7a40007400 comp rx=0 tx=0).stop 2024-05-09T04:46:43.873 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:43.873+0000 7f7a582eb700 1 -- 172.21.15.110:0/2919551674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a50105e40 msgr2=0x7f7a501a1de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:43.873+0000 7f7a582eb700 1 --2- 172.21.15.110:0/2919551674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a50105e40 0x7f7a501a1de0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800ada0 tx=0x7f7a4800c330 comp rx=0 tx=0).stop 2024-05-09T04:46:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:43.873+0000 7f7a582eb700 1 -- 172.21.15.110:0/2919551674 shutdown_connections 2024-05-09T04:46:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:43.873+0000 7f7a582eb700 1 --2- 172.21.15.110:0/2919551674 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7a3c0b2900 0x7f7a3c0b4e10 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:43.873+0000 7f7a582eb700 1 --2- 172.21.15.110:0/2919551674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a50105e40 0x7f7a501a1de0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:43.873+0000 7f7a582eb700 1 --2- 172.21.15.110:0/2919551674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a50104370 0x7f7a5019a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:43.873+0000 7f7a582eb700 1 --2- 172.21.15.110:0/2919551674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a501054a0 0x7f7a5019ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:43.873+0000 7f7a582eb700 1 -- 172.21.15.110:0/2919551674 >> 172.21.15.110:0/2919551674 conn(0x7f7a500ff9d0 msgr2=0x7f7a501086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:43.873+0000 7f7a582eb700 1 -- 172.21.15.110:0/2919551674 shutdown_connections 2024-05-09T04:46:43.874 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:43.873+0000 7f7a582eb700 1 -- 172.21.15.110:0/2919551674 wait complete. 2024-05-09T04:46:44.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:44.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:44.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:44.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:44.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:44.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:43 smithi110 ceph-mon[34904]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:44.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:43 smithi110 ceph-mon[34904]: from='osd.11 [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T04:46:44.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:43 smithi110 ceph-mon[34904]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T04:46:44.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:44.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:44.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:43 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:44.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:44.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:44.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:44.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:44.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:44.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:43 smithi046 ceph-mon[30136]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:44.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:43 smithi046 ceph-mon[30136]: from='osd.11 [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T04:46:44.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:43 smithi046 ceph-mon[30136]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T04:46:44.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:44.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:44.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:43 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:44.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:44.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:44.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:44.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:44.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:44.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:43 smithi072 ceph-mon[34088]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:44.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:43 smithi072 ceph-mon[34088]: from='osd.11 [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T04:46:44.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:43 smithi072 ceph-mon[34088]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-09T04:46:44.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:44.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:44.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:43 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:44.584 DEBUG:teuthology.orchestra.run.smithi110:osd.11> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.11.service 2024-05-09T04:46:44.587 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-09T04:46:44.587 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-09T04:46:44.851 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:46:45.105 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:44 smithi110 ceph-mon[34904]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T04:46:45.105 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:44 smithi110 ceph-mon[34904]: osdmap e70: 12 total, 11 up, 12 in 2024-05-09T04:46:45.105 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:44 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:45.105 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:44 smithi110 ceph-mon[34904]: from='osd.11 [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:46:45.105 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:44 smithi110 ceph-mon[34904]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:46:45.306 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:44 smithi046 ceph-mon[30136]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T04:46:45.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:44 smithi046 ceph-mon[30136]: osdmap e70: 12 total, 11 up, 12 in 2024-05-09T04:46:45.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:44 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:45.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:44 smithi046 ceph-mon[30136]: from='osd.11 [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:46:45.307 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:44 smithi046 ceph-mon[30136]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:46:45.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:44 smithi072 ceph-mon[34088]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-09T04:46:45.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:44 smithi072 ceph-mon[34088]: osdmap e70: 12 total, 11 up, 12 in 2024-05-09T04:46:45.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:44 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:45.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:44 smithi072 ceph-mon[34088]: from='osd.11 [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:46:45.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:44 smithi072 ceph-mon[34088]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]: dispatch 2024-05-09T04:46:45.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 04:46:45 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T04:46:45.415+0000 7f28fe98c700 -1 osd.11 0 waiting for initial osdmap 2024-05-09T04:46:45.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 04:46:45 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T04:46:45.429+0000 7f28f6739700 -1 osd.11 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-09T04:46:46.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:45 smithi046 ceph-mon[30136]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-09T04:46:46.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:45 smithi046 ceph-mon[30136]: osdmap e71: 12 total, 11 up, 12 in 2024-05-09T04:46:46.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:45 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:46.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:45 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:46.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:45 smithi046 ceph-mon[30136]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:46.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:45 smithi046 ceph-mon[30136]: from='osd.11 ' entity='osd.11' 2024-05-09T04:46:46.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:45 smithi110 ceph-mon[34904]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-09T04:46:46.160 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:45 smithi110 ceph-mon[34904]: osdmap e71: 12 total, 11 up, 12 in 2024-05-09T04:46:46.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:45 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:46.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:45 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:46.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:45 smithi110 ceph-mon[34904]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:46.161 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:45 smithi110 ceph-mon[34904]: from='osd.11 ' entity='osd.11' 2024-05-09T04:46:46.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:45 smithi072 ceph-mon[34088]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi110", "root=default"]}]': finished 2024-05-09T04:46:46.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:45 smithi072 ceph-mon[34088]: osdmap e71: 12 total, 11 up, 12 in 2024-05-09T04:46:46.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:45 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:46.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:45 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:46.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:45 smithi072 ceph-mon[34088]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-09T04:46:46.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:45 smithi072 ceph-mon[34088]: from='osd.11 ' entity='osd.11' 2024-05-09T04:46:46.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.632+0000 7f8b269b9700 1 -- 172.21.15.46:0/2759915990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b20105490 msgr2=0x7f8b201058a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:46.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.632+0000 7f8b269b9700 1 --2- 172.21.15.46:0/2759915990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b20105490 0x7f8b201058a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8b14009990 tx=0x7f8b14009ca0 comp rx=0 tx=0).stop 2024-05-09T04:46:46.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.632+0000 7f8b269b9700 1 -- 172.21.15.46:0/2759915990 shutdown_connections 2024-05-09T04:46:46.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.632+0000 7f8b269b9700 1 --2- 172.21.15.46:0/2759915990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b20105e30 0x7f8b2010c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:46.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.632+0000 7f8b269b9700 1 --2- 172.21.15.46:0/2759915990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b20104360 0x7f8b20104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:46.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.632+0000 7f8b269b9700 1 --2- 172.21.15.46:0/2759915990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b20105490 0x7f8b201058a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:46.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.632+0000 7f8b269b9700 1 -- 172.21.15.46:0/2759915990 >> 172.21.15.46:0/2759915990 conn(0x7f8b200ff9c0 msgr2=0x7f8b20101e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:46.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.632+0000 7f8b269b9700 1 -- 172.21.15.46:0/2759915990 shutdown_connections 2024-05-09T04:46:46.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.632+0000 7f8b269b9700 1 -- 172.21.15.46:0/2759915990 wait complete. 2024-05-09T04:46:46.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.633+0000 7f8b269b9700 1 Processor -- start 2024-05-09T04:46:46.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.633+0000 7f8b269b9700 1 -- start start 2024-05-09T04:46:46.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b269b9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b20104360 0x7f8b2019cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:46.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b269b9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b20105490 0x7f8b2019d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:46.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b269b9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b20105e30 0x7f8b201a3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:46.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b269b9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2010fdc0 con 0x7f8b20104360 2024-05-09T04:46:46.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b269b9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2010fc40 con 0x7f8b20105490 2024-05-09T04:46:46.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b269b9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8b2010ff40 con 0x7f8b20105e30 2024-05-09T04:46:46.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b1ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b20104360 0x7f8b2019cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:46.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b1ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b20104360 0x7f8b2019cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55750/0 (socket says 172.21.15.46:55750) 2024-05-09T04:46:46.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b1ffff700 1 -- 172.21.15.46:0/594207862 learned_addr learned my addr 172.21.15.46:0/594207862 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:46:46.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b24f56700 1 --2- 172.21.15.46:0/594207862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b20105e30 0x7f8b201a3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:46.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b1f7fe700 1 --2- 172.21.15.46:0/594207862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b20105490 0x7f8b2019d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:46.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b1ffff700 1 -- 172.21.15.46:0/594207862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b20105e30 msgr2=0x7f8b201a3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b1ffff700 1 --2- 172.21.15.46:0/594207862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b20105e30 0x7f8b201a3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b1ffff700 1 -- 172.21.15.46:0/594207862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b20105490 msgr2=0x7f8b2019d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b1ffff700 1 --2- 172.21.15.46:0/594207862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b20105490 0x7f8b2019d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b1ffff700 1 -- 172.21.15.46:0/594207862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b201a4670 con 0x7f8b20104360 2024-05-09T04:46:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b1f7fe700 1 --2- 172.21.15.46:0/594207862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b20105490 0x7f8b2019d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:46:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b1ffff700 1 --2- 172.21.15.46:0/594207862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b20104360 0x7f8b2019cb10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000e940 tx=0x7f8b1000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.634+0000 7f8b1d7fa700 1 -- 172.21.15.46:0/594207862 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b1000cbd0 con 0x7f8b20104360 2024-05-09T04:46:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.635+0000 7f8b269b9700 1 -- 172.21.15.46:0/594207862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b201a4960 con 0x7f8b20104360 2024-05-09T04:46:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.635+0000 7f8b1d7fa700 1 -- 172.21.15.46:0/594207862 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b1000ee80 con 0x7f8b20104360 2024-05-09T04:46:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.635+0000 7f8b1d7fa700 1 -- 172.21.15.46:0/594207862 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b10003dc0 con 0x7f8b20104360 2024-05-09T04:46:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.635+0000 7f8b269b9700 1 -- 172.21.15.46:0/594207862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b201a4ef0 con 0x7f8b20104360 2024-05-09T04:46:46.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.636+0000 7f8b269b9700 1 -- 172.21.15.46:0/594207862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b200512c0 con 0x7f8b20104360 2024-05-09T04:46:46.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.638+0000 7f8b1d7fa700 1 -- 172.21.15.46:0/594207862 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8b10003f60 con 0x7f8b20104360 2024-05-09T04:46:46.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.639+0000 7f8b1d7fa700 1 --2- 172.21.15.46:0/594207862 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f8b080b2880 0x7f8b080b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.639+0000 7f8b1f7fe700 1 --2- 172.21.15.46:0/594207862 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f8b080b2880 0x7f8b080b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.639+0000 7f8b1d7fa700 1 -- 172.21.15.46:0/594207862 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f8b100e2110 con 0x7f8b20104360 2024-05-09T04:46:46.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.639+0000 7f8b1f7fe700 1 --2- 172.21.15.46:0/594207862 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f8b080b2880 0x7f8b080b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400b2e0 tx=0x7f8b1400b680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:46.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.643+0000 7f8b1d7fa700 1 -- 172.21.15.46:0/594207862 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8b100aa2c0 con 0x7f8b20104360 2024-05-09T04:46:46.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.856+0000 7f8b269b9700 1 -- 172.21.15.46:0/594207862 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f8b201a5180 con 0x7f8b20104360 2024-05-09T04:46:46.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.856+0000 7f8b1d7fa700 1 -- 172.21.15.46:0/594207862 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7f8b100a9bc0 con 0x7f8b20104360 2024-05-09T04:46:46.858 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:46:46.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.859+0000 7f8b269b9700 1 -- 172.21.15.46:0/594207862 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f8b080b2880 msgr2=0x7f8b080b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:46.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.859+0000 7f8b269b9700 1 --2- 172.21.15.46:0/594207862 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f8b080b2880 0x7f8b080b4d90 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f8b1400b2e0 tx=0x7f8b1400b680 comp rx=0 tx=0).stop 2024-05-09T04:46:46.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.859+0000 7f8b269b9700 1 -- 172.21.15.46:0/594207862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b20104360 msgr2=0x7f8b2019cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:46.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.859+0000 7f8b269b9700 1 --2- 172.21.15.46:0/594207862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b20104360 0x7f8b2019cb10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8b1000e940 tx=0x7f8b1000ed00 comp rx=0 tx=0).stop 2024-05-09T04:46:46.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.859+0000 7f8b269b9700 1 -- 172.21.15.46:0/594207862 shutdown_connections 2024-05-09T04:46:46.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.859+0000 7f8b269b9700 1 --2- 172.21.15.46:0/594207862 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f8b080b2880 0x7f8b080b4d90 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:46.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.859+0000 7f8b269b9700 1 --2- 172.21.15.46:0/594207862 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8b20105e30 0x7f8b201a3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:46.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.859+0000 7f8b269b9700 1 --2- 172.21.15.46:0/594207862 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8b20104360 0x7f8b2019cb10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:46.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.859+0000 7f8b269b9700 1 --2- 172.21.15.46:0/594207862 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8b20105490 0x7f8b2019d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:46.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.859+0000 7f8b269b9700 1 -- 172.21.15.46:0/594207862 >> 172.21.15.46:0/594207862 conn(0x7f8b200ff9c0 msgr2=0x7f8b200759f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:46.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.859+0000 7f8b269b9700 1 -- 172.21.15.46:0/594207862 shutdown_connections 2024-05-09T04:46:46.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:46.859+0000 7f8b269b9700 1 -- 172.21.15.46:0/594207862 wait complete. 2024-05-09T04:46:47.248 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:46 smithi046 ceph-mon[30136]: purged_snaps scrub starts 2024-05-09T04:46:47.249 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:46 smithi046 ceph-mon[30136]: purged_snaps scrub ok 2024-05-09T04:46:47.249 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:46 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:47.249 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:46 smithi046 ceph-mon[30136]: osd.11 [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632] boot 2024-05-09T04:46:47.249 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:46 smithi046 ceph-mon[30136]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T04:46:47.249 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:46 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:47.249 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:46 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/594207862' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T04:46:47.294 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:46 smithi110 ceph-mon[34904]: purged_snaps scrub starts 2024-05-09T04:46:47.294 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:46 smithi110 ceph-mon[34904]: purged_snaps scrub ok 2024-05-09T04:46:47.294 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:46 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:47.294 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:46 smithi110 ceph-mon[34904]: osd.11 [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632] boot 2024-05-09T04:46:47.295 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:46 smithi110 ceph-mon[34904]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T04:46:47.295 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:46 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:47.295 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:46 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/594207862' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T04:46:47.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:46 smithi072 ceph-mon[34088]: purged_snaps scrub starts 2024-05-09T04:46:47.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:46 smithi072 ceph-mon[34088]: purged_snaps scrub ok 2024-05-09T04:46:47.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:46 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:47.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:46 smithi072 ceph-mon[34088]: osd.11 [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632] boot 2024-05-09T04:46:47.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:46 smithi072 ceph-mon[34088]: osdmap e72: 12 total, 12 up, 12 in 2024-05-09T04:46:47.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:46 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:46:47.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:46 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/594207862' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-09T04:46:47.648 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1715230006,"num_in_osds":12,"osd_in_since":1715229985,"num_remapped_pgs":0} 2024-05-09T04:46:47.649 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:46:47.914 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:46:48.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:48 smithi046 ceph-mon[30136]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:48.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:48 smithi046 ceph-mon[30136]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T04:46:48.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:48 smithi046 ceph-mon[30136]: Detected new or changed devices on smithi110 2024-05-09T04:46:48.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:48.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:48.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:48 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi110 to 4272M 2024-05-09T04:46:48.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:48.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:48.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:48.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:48 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:48 smithi110 ceph-mon[34904]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:48 smithi110 ceph-mon[34904]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T04:46:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:48 smithi110 ceph-mon[34904]: Detected new or changed devices on smithi110 2024-05-09T04:46:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:48 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi110 to 4272M 2024-05-09T04:46:48.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:48.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:48.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:48.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:48 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:48.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:48 smithi072 ceph-mon[34088]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:48.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:48 smithi072 ceph-mon[34088]: osdmap e73: 12 total, 12 up, 12 in 2024-05-09T04:46:48.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:48 smithi072 ceph-mon[34088]: Detected new or changed devices on smithi110 2024-05-09T04:46:48.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:48.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:48.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:46:48.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:48 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi110 to 4272M 2024-05-09T04:46:48.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:48.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:48.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:48.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:48 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:49.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.649+0000 7fe3fdafc700 1 -- 172.21.15.46:0/1070481684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8109860 msgr2=0x7fe3f81111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:49.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.649+0000 7fe3fdafc700 1 --2- 172.21.15.46:0/1070481684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8109860 0x7fe3f81111c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe3f00099c0 tx=0x7fe3f0009cd0 comp rx=0 tx=0).stop 2024-05-09T04:46:49.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.649+0000 7fe3fdafc700 1 -- 172.21.15.46:0/1070481684 shutdown_connections 2024-05-09T04:46:49.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.649+0000 7fe3fdafc700 1 --2- 172.21.15.46:0/1070481684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8109860 0x7fe3f81111c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:49.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.649+0000 7fe3fdafc700 1 --2- 172.21.15.46:0/1070481684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3f81045e0 0x7fe3f81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:49.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.649+0000 7fe3fdafc700 1 --2- 172.21.15.46:0/1070481684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f8108f10 0x7fe3f8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:49.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.649+0000 7fe3fdafc700 1 -- 172.21.15.46:0/1070481684 >> 172.21.15.46:0/1070481684 conn(0x7fe3f80ffed0 msgr2=0x7fe3f8102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:49.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.649+0000 7fe3fdafc700 1 -- 172.21.15.46:0/1070481684 shutdown_connections 2024-05-09T04:46:49.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.649+0000 7fe3fdafc700 1 -- 172.21.15.46:0/1070481684 wait complete. 2024-05-09T04:46:49.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.650+0000 7fe3fdafc700 1 Processor -- start 2024-05-09T04:46:49.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.650+0000 7fe3fdafc700 1 -- start start 2024-05-09T04:46:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.650+0000 7fe3fdafc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f81045e0 0x7fe3f819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.650+0000 7fe3fdafc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3f8108f10 0x7fe3f819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.650+0000 7fe3fdafc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8109860 0x7fe3f819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3fdafc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f8114490 con 0x7fe3f8109860 2024-05-09T04:46:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3fdafc700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f8114310 con 0x7fe3f81045e0 2024-05-09T04:46:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3fdafc700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe3f8114610 con 0x7fe3f8108f10 2024-05-09T04:46:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8109860 0x7fe3f819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8109860 0x7fe3f819f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55772/0 (socket says 172.21.15.46:55772) 2024-05-09T04:46:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3f7fff700 1 -- 172.21.15.46:0/925658241 learned_addr learned my addr 172.21.15.46:0/925658241 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:46:49.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3f77fe700 1 --2- 172.21.15.46:0/925658241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f81045e0 0x7fe3f819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:49.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3f6ffd700 1 --2- 172.21.15.46:0/925658241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3f8108f10 0x7fe3f819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:49.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3f7fff700 1 -- 172.21.15.46:0/925658241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3f8108f10 msgr2=0x7fe3f819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:49.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3f7fff700 1 --2- 172.21.15.46:0/925658241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3f8108f10 0x7fe3f819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:49.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3f7fff700 1 -- 172.21.15.46:0/925658241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f81045e0 msgr2=0x7fe3f819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:49.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3f7fff700 1 --2- 172.21.15.46:0/925658241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f81045e0 0x7fe3f819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:49.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3f7fff700 1 -- 172.21.15.46:0/925658241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3f819fbc0 con 0x7fe3f8109860 2024-05-09T04:46:49.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3f7fff700 1 --2- 172.21.15.46:0/925658241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8109860 0x7fe3f819f5d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0005910 tx=0x7fe3f00055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:49.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3f4ff9700 1 -- 172.21.15.46:0/925658241 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0026070 con 0x7fe3f8109860 2024-05-09T04:46:49.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3fdafc700 1 -- 172.21.15.46:0/925658241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3f819fe50 con 0x7fe3f8109860 2024-05-09T04:46:49.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3f4ff9700 1 -- 172.21.15.46:0/925658241 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3f000f460 con 0x7fe3f8109860 2024-05-09T04:46:49.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3f4ff9700 1 -- 172.21.15.46:0/925658241 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0009e60 con 0x7fe3f8109860 2024-05-09T04:46:49.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.651+0000 7fe3fdafc700 1 -- 172.21.15.46:0/925658241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3f81a0380 con 0x7fe3f8109860 2024-05-09T04:46:49.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.653+0000 7fe3fdafc700 1 -- 172.21.15.46:0/925658241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3f80512c0 con 0x7fe3f8109860 2024-05-09T04:46:49.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.654+0000 7fe3f4ff9700 1 -- 172.21.15.46:0/925658241 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe3f0029070 con 0x7fe3f8109860 2024-05-09T04:46:49.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.655+0000 7fe3f4ff9700 1 --2- 172.21.15.46:0/925658241 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe3e40b2880 0x7fe3e40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:49.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.655+0000 7fe3f77fe700 1 --2- 172.21.15.46:0/925658241 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe3e40b2880 0x7fe3e40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:49.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.655+0000 7fe3f4ff9700 1 -- 172.21.15.46:0/925658241 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fe3f00e25e0 con 0x7fe3f8109860 2024-05-09T04:46:49.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.655+0000 7fe3f77fe700 1 --2- 172.21.15.46:0/925658241 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe3e40b2880 0x7fe3e40b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800f440 tx=0x7fe3e80079e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:49.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.660+0000 7fe3f4ff9700 1 -- 172.21.15.46:0/925658241 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe3f0013070 con 0x7fe3f8109860 2024-05-09T04:46:49.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.873+0000 7fe3fdafc700 1 -- 172.21.15.46:0/925658241 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe3f802e5f0 con 0x7fe3f8109860 2024-05-09T04:46:49.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.875+0000 7fe3f4ff9700 1 -- 172.21.15.46:0/925658241 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+18347 (secure 0 0 0) 0x7fe3f00aa790 con 0x7fe3f8109860 2024-05-09T04:46:49.875 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:46:49.875 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":73,"fsid":"ecf9ae2c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:25.633876+0000","modified":"2024-05-09T04:46:47.420214+0000","last_up_change":"2024-05-09T04:46:46.418597+0000","last_in_change":"2024-05-09T04:46:25.903582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:42:53.301400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d79cd3e7-6f82-4d60-96d9-26d8146c962f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6803","nonce":1576436508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6805","nonce":1576436508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6809","nonce":1576436508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6807","nonce":1576436508}]},"public_addr":"172.21.15.46:6803/1576436508","cluster_addr":"172.21.15.46:6805/1576436508","heartbeat_back_addr":"172.21.15.46:6809/1576436508","heartbeat_front_addr":"172.21.15.46:6807/1576436508","state":["exists","up"]},{"osd":1,"uuid":"698eba80-1ebd-4076-b6f9-953d2e7daf76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6801","nonce":379039775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6803","nonce":379039775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6807","nonce":379039775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6805","nonce":379039775}]},"public_addr":"172.21.15.72:6801/379039775","cluster_addr":"172.21.15.72:6803/379039775","heartbeat_back_addr":"172.21.15.72:6807/379039775","heartbeat_front_addr":"172.21.15.72:6805/379039775","state":["exists","up"]},{"osd":2,"uuid":"c549b131-cfa7-4724-926f-83e91a752d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6801","nonce":293447227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6803","nonce":293447227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6807","nonce":293447227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6805","nonce":293447227}]},"public_addr":"172.21.15.110:6801/293447227","cluster_addr":"172.21.15.110:6803/293447227","heartbeat_back_addr":"172.21.15.110:6807/293447227","heartbeat_front_addr":"172.21.15.110:6805/293447227","state":["exists","up"]},{"osd":3,"uuid":"549bb4f7-54d9-4cbc-ad40-7f3e717964f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6811","nonce":1115835653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6813","nonce":1115835653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6817","nonce":1115835653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6815","nonce":1115835653}]},"public_addr":"172.21.15.46:6811/1115835653","cluster_addr":"172.21.15.46:6813/1115835653","heartbeat_back_addr":"172.21.15.46:6817/1115835653","heartbeat_front_addr":"172.21.15.46:6815/1115835653","state":["exists","up"]},{"osd":4,"uuid":"444cd36b-6034-46cd-9a6b-29dce4ba6405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6809","nonce":59186693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6811","nonce":59186693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6815","nonce":59186693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6813","nonce":59186693}]},"public_addr":"172.21.15.72:6809/59186693","cluster_addr":"172.21.15.72:6811/59186693","heartbeat_back_addr":"172.21.15.72:6815/59186693","heartbeat_front_addr":"172.21.15.72:6813/59186693","state":["exists","up"]},{"osd":5,"uuid":"2c2bdd2f-43cf-4208-9ae1-f912807c84e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6809","nonce":1712162502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6811","nonce":1712162502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6815","nonce":1712162502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6813","nonce":1712162502}]},"public_addr":"172.21.15.110:6809/1712162502","cluster_addr":"172.21.15.110:6811/1712162502","heartbeat_back_addr":"172.21.15.110:6815/1712162502","heartbeat_front_addr":"172.21.15.110:6813/1712162502","state":["exists","up"]},{"osd":6,"uuid":"beb0b87f-c3a4-40dd-b224-1843e9301ca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6819","nonce":2429650065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6821","nonce":2429650065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6825","nonce":2429650065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6823","nonce":2429650065}]},"public_addr":"172.21.15.46:6819/2429650065","cluster_addr":"172.21.15.46:6821/2429650065","heartbeat_back_addr":"172.21.15.46:6825/2429650065","heartbeat_front_addr":"172.21.15.46:6823/2429650065","state":["exists","up"]},{"osd":7,"uuid":"4a634675-fad7-4a43-840c-8dec2631dec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6817","nonce":337796672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6819","nonce":337796672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6823","nonce":337796672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6821","nonce":337796672}]},"public_addr":"172.21.15.72:6817/337796672","cluster_addr":"172.21.15.72:6819/337796672","heartbeat_back_addr":"172.21.15.72:6823/337796672","heartbeat_front_addr":"172.21.15.72:6821/337796672","state":["exists","up"]},{"osd":8,"uuid":"01b5f638-90f0-4cf4-8042-20d20a725669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6817","nonce":189513203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6819","nonce":189513203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6823","nonce":189513203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6821","nonce":189513203}]},"public_addr":"172.21.15.110:6817/189513203","cluster_addr":"172.21.15.110:6819/189513203","heartbeat_back_addr":"172.21.15.110:6823/189513203","heartbeat_front_addr":"172.21.15.110:6821/189513203","state":["exists","up"]},{"osd":9,"uuid":"e423de41-50ec-48f4-a4ed-216de33c4fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6827","nonce":1762262822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6829","nonce":1762262822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6833","nonce":1762262822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6831","nonce":1762262822}]},"public_addr":"172.21.15.46:6827/1762262822","cluster_addr":"172.21.15.46:6829/1762262822","heartbeat_back_addr":"172.21.15.46:6833/1762262822","heartbeat_front_addr":"172.21.15.46:6831/1762262822","state":["exists","up"]},{"osd":10,"uuid":"f594d004-9f80-4508-b127-fb693d355f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6825","nonce":38453898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6827","nonce":38453898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6831","nonce":38453898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6829","nonce":38453898}]},"public_addr":"172.21.15.72:6825/38453898","cluster_addr":"172.21.15.72:6827/38453898","heartbeat_back_addr":"172.21.15.72:6831/38453898","heartbeat_front_addr":"172.21.15.72:6829/38453898","state":["exists","up"]},{"osd":11,"uuid":"f771d29f-89e3-4f96-9c29-00e424230c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6825","nonce":4153529632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6827","nonce":4153529632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6831","nonce":4153529632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6829","nonce":4153529632}]},"public_addr":"172.21.15.110:6825/4153529632","cluster_addr":"172.21.15.110:6827/4153529632","heartbeat_back_addr":"172.21.15.110:6831/4153529632","heartbeat_front_addr":"172.21.15.110:6829/4153529632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:02.615801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:26.453477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:50.334627+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:15.357320+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:40.403549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:06.030093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:31.904091+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:57.616372+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:24.045975+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:51.160898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:17.597845+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:44.636629+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1596672353":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1846482551":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/1069571878":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6800/2738158981":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/3625745146":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6801/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/564539954":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1797199863":"2024-05-10T04:40:27.201119+0000","172.21.15.46:6801/2738158981":"2024-05-10T04:39:59.515628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:46:49.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.877+0000 7fe3fdafc700 1 -- 172.21.15.46:0/925658241 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe3e40b2880 msgr2=0x7fe3e40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:49.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.877+0000 7fe3fdafc700 1 --2- 172.21.15.46:0/925658241 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe3e40b2880 0x7fe3e40b4d90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe3e800f440 tx=0x7fe3e80079e0 comp rx=0 tx=0).stop 2024-05-09T04:46:49.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.877+0000 7fe3fdafc700 1 -- 172.21.15.46:0/925658241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8109860 msgr2=0x7fe3f819f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:49.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.877+0000 7fe3fdafc700 1 --2- 172.21.15.46:0/925658241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8109860 0x7fe3f819f5d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe3f0005910 tx=0x7fe3f00055f0 comp rx=0 tx=0).stop 2024-05-09T04:46:49.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.877+0000 7fe3fdafc700 1 -- 172.21.15.46:0/925658241 shutdown_connections 2024-05-09T04:46:49.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.877+0000 7fe3fdafc700 1 --2- 172.21.15.46:0/925658241 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe3e40b2880 0x7fe3e40b4d90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:49.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.877+0000 7fe3fdafc700 1 --2- 172.21.15.46:0/925658241 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe3f8109860 0x7fe3f819f5d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:49.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.877+0000 7fe3fdafc700 1 --2- 172.21.15.46:0/925658241 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3f81045e0 0x7fe3f819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:49.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.877+0000 7fe3fdafc700 1 --2- 172.21.15.46:0/925658241 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3f8108f10 0x7fe3f819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:49.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.877+0000 7fe3fdafc700 1 -- 172.21.15.46:0/925658241 >> 172.21.15.46:0/925658241 conn(0x7fe3f80ffed0 msgr2=0x7fe3f8108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:49.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.877+0000 7fe3fdafc700 1 -- 172.21.15.46:0/925658241 shutdown_connections 2024-05-09T04:46:49.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:49.877+0000 7fe3fdafc700 1 -- 172.21.15.46:0/925658241 wait complete. 2024-05-09T04:46:50.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:50 smithi110 ceph-mon[34904]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:50.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:50 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/925658241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:46:50.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:50 smithi046 ceph-mon[30136]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:50.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:50 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/925658241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:46:50.617 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T04:42:53.301400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T04:46:50.618 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-09T04:46:50.865 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:46:50.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:50 smithi072 ceph-mon[34088]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:50.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:50 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/925658241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:46:52.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:52 smithi046 ceph-mon[30136]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:52.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.606+0000 7fe5a7594700 1 -- 172.21.15.46:0/4180360905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a0103a60 msgr2=0x7fe5a0110780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.606+0000 7fe5a7594700 1 --2- 172.21.15.46:0/4180360905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a0103a60 0x7fe5a0110780 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe590009990 tx=0x7fe590009ca0 comp rx=0 tx=0).stop 2024-05-09T04:46:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.606+0000 7fe5a7594700 1 -- 172.21.15.46:0/4180360905 shutdown_connections 2024-05-09T04:46:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.606+0000 7fe5a7594700 1 --2- 172.21.15.46:0/4180360905 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5a0110cc0 0x7fe5a01130e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.606+0000 7fe5a7594700 1 --2- 172.21.15.46:0/4180360905 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5a0103110 0x7fe5a0103520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.606+0000 7fe5a7594700 1 --2- 172.21.15.46:0/4180360905 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a0103a60 0x7fe5a0110780 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.606+0000 7fe5a7594700 1 -- 172.21.15.46:0/4180360905 >> 172.21.15.46:0/4180360905 conn(0x7fe5a0078e00 msgr2=0x7fe5a0079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.606+0000 7fe5a7594700 1 -- 172.21.15.46:0/4180360905 shutdown_connections 2024-05-09T04:46:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.606+0000 7fe5a7594700 1 -- 172.21.15.46:0/4180360905 wait complete. 2024-05-09T04:46:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.607+0000 7fe5a7594700 1 Processor -- start 2024-05-09T04:46:52.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.607+0000 7fe5a7594700 1 -- start start 2024-05-09T04:46:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.607+0000 7fe5a7594700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a0103110 0x7fe5a019d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.607+0000 7fe5a7594700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5a0103a60 0x7fe5a019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.607+0000 7fe5a7594700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5a0110cc0 0x7fe5a01a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.607+0000 7fe5a7594700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a0116690 con 0x7fe5a0103110 2024-05-09T04:46:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.607+0000 7fe5a7594700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a0116510 con 0x7fe5a0103a60 2024-05-09T04:46:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.607+0000 7fe5a7594700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe5a0116810 con 0x7fe5a0110cc0 2024-05-09T04:46:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.607+0000 7fe5a5330700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a0103110 0x7fe5a019d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:52.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.608+0000 7fe5a5330700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a0103110 0x7fe5a019d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55780/0 (socket says 172.21.15.46:55780) 2024-05-09T04:46:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.608+0000 7fe5a5330700 1 -- 172.21.15.46:0/2533957385 learned_addr learned my addr 172.21.15.46:0/2533957385 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:46:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.608+0000 7fe5a5b31700 1 --2- 172.21.15.46:0/2533957385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5a0110cc0 0x7fe5a01a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.608+0000 7fe5a4b2f700 1 --2- 172.21.15.46:0/2533957385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5a0103a60 0x7fe5a019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.608+0000 7fe5a5330700 1 -- 172.21.15.46:0/2533957385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5a0110cc0 msgr2=0x7fe5a01a18d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.608+0000 7fe5a5330700 1 --2- 172.21.15.46:0/2533957385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5a0110cc0 0x7fe5a01a18d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.608+0000 7fe5a5330700 1 -- 172.21.15.46:0/2533957385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5a0103a60 msgr2=0x7fe5a019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.608+0000 7fe5a5330700 1 --2- 172.21.15.46:0/2533957385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5a0103a60 0x7fe5a019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.608+0000 7fe5a5330700 1 -- 172.21.15.46:0/2533957385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5a01a1ec0 con 0x7fe5a0103110 2024-05-09T04:46:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.608+0000 7fe5a5330700 1 --2- 172.21.15.46:0/2533957385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a0103110 0x7fe5a019d0a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00c9b0 tx=0x7fe59c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.608+0000 7fe5967fc700 1 -- 172.21.15.46:0/2533957385 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c007b00 con 0x7fe5a0103110 2024-05-09T04:46:52.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.608+0000 7fe5967fc700 1 -- 172.21.15.46:0/2533957385 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe59c007ca0 con 0x7fe5a0103110 2024-05-09T04:46:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.608+0000 7fe5967fc700 1 -- 172.21.15.46:0/2533957385 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe59c018660 con 0x7fe5a0103110 2024-05-09T04:46:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.608+0000 7fe5a7594700 1 -- 172.21.15.46:0/2533957385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5a01a21b0 con 0x7fe5a0103110 2024-05-09T04:46:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.608+0000 7fe5a7594700 1 -- 172.21.15.46:0/2533957385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5a01a9b80 con 0x7fe5a0103110 2024-05-09T04:46:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.609+0000 7fe5a7594700 1 -- 172.21.15.46:0/2533957385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5a00512c0 con 0x7fe5a0103110 2024-05-09T04:46:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.611+0000 7fe5967fc700 1 -- 172.21.15.46:0/2533957385 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe59c00f450 con 0x7fe5a0103110 2024-05-09T04:46:52.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.612+0000 7fe5967fc700 1 --2- 172.21.15.46:0/2533957385 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe58c0b2900 0x7fe58c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:52.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.612+0000 7fe5a4b2f700 1 --2- 172.21.15.46:0/2533957385 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe58c0b2900 0x7fe58c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:52.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.612+0000 7fe5967fc700 1 -- 172.21.15.46:0/2533957385 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fe59c0e1f90 con 0x7fe5a0103110 2024-05-09T04:46:52.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.612+0000 7fe5a4b2f700 1 --2- 172.21.15.46:0/2533957385 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe58c0b2900 0x7fe58c0b4e10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe590006010 tx=0x7fe59001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:52.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.615+0000 7fe5967fc700 1 -- 172.21.15.46:0/2533957385 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe59c014030 con 0x7fe5a0103110 2024-05-09T04:46:52.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:52 smithi110 ceph-mon[34904]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:52.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.826+0000 7fe5a7594700 1 -- 172.21.15.46:0/2533957385 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe5a019e400 con 0x7fe5a0103110 2024-05-09T04:46:52.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.826+0000 7fe5967fc700 1 -- 172.21.15.46:0/2533957385 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v73) ==== 93+0+10 (secure 0 0 0) 0x7fe59c0aa260 con 0x7fe5a0103110 2024-05-09T04:46:52.826 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-09T04:46:52.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.828+0000 7fe5a7594700 1 -- 172.21.15.46:0/2533957385 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe58c0b2900 msgr2=0x7fe58c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:52.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.828+0000 7fe5a7594700 1 --2- 172.21.15.46:0/2533957385 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe58c0b2900 0x7fe58c0b4e10 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe590006010 tx=0x7fe59001a040 comp rx=0 tx=0).stop 2024-05-09T04:46:52.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.828+0000 7fe5a7594700 1 -- 172.21.15.46:0/2533957385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a0103110 msgr2=0x7fe5a019d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:52.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.828+0000 7fe5a7594700 1 --2- 172.21.15.46:0/2533957385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a0103110 0x7fe5a019d0a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fe59c00c9b0 tx=0x7fe59c00ccc0 comp rx=0 tx=0).stop 2024-05-09T04:46:52.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.829+0000 7fe5a7594700 1 -- 172.21.15.46:0/2533957385 shutdown_connections 2024-05-09T04:46:52.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.829+0000 7fe5a7594700 1 --2- 172.21.15.46:0/2533957385 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe58c0b2900 0x7fe58c0b4e10 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:52.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.829+0000 7fe5a7594700 1 --2- 172.21.15.46:0/2533957385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5a0110cc0 0x7fe5a01a18d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:52.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.829+0000 7fe5a7594700 1 --2- 172.21.15.46:0/2533957385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5a0103110 0x7fe5a019d0a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:52.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.829+0000 7fe5a7594700 1 --2- 172.21.15.46:0/2533957385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5a0103a60 0x7fe5a019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:52.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.829+0000 7fe5a7594700 1 -- 172.21.15.46:0/2533957385 >> 172.21.15.46:0/2533957385 conn(0x7fe5a0078e00 msgr2=0x7fe5a01121e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:52.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.829+0000 7fe5a7594700 1 -- 172.21.15.46:0/2533957385 shutdown_connections 2024-05-09T04:46:52.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:52.829+0000 7fe5a7594700 1 -- 172.21.15.46:0/2533957385 wait complete. 2024-05-09T04:46:52.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:52 smithi072 ceph-mon[34088]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:53.565 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:53 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2533957385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T04:46:53.598 INFO:tasks.cephadm:Adding mds.a on smithi046 2024-05-09T04:46:53.598 INFO:tasks.cephadm:Adding mds.d on smithi046 2024-05-09T04:46:53.598 INFO:tasks.cephadm:Adding mds.g on smithi046 2024-05-09T04:46:53.598 INFO:tasks.cephadm:Adding mds.j on smithi046 2024-05-09T04:46:53.598 INFO:tasks.cephadm:Adding mds.b on smithi072 2024-05-09T04:46:53.598 INFO:tasks.cephadm:Adding mds.e on smithi072 2024-05-09T04:46:53.598 INFO:tasks.cephadm:Adding mds.h on smithi072 2024-05-09T04:46:53.598 INFO:tasks.cephadm:Adding mds.k on smithi072 2024-05-09T04:46:53.598 INFO:tasks.cephadm:Adding mds.c on smithi110 2024-05-09T04:46:53.598 INFO:tasks.cephadm:Adding mds.f on smithi110 2024-05-09T04:46:53.598 INFO:tasks.cephadm:Adding mds.i on smithi110 2024-05-09T04:46:53.598 INFO:tasks.cephadm:Adding mds.l on smithi110 2024-05-09T04:46:53.599 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l' 2024-05-09T04:46:53.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:53 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2533957385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T04:46:53.849 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T04:46:53.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:53 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2533957385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T04:46:54.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:54 smithi110 ceph-mon[34904]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:54.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:54 smithi046 ceph-mon[30136]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:54.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:54 smithi072 ceph-mon[34088]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:55.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.575+0000 7fe110f98700 1 -- 172.21.15.110:0/3264482486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe10c104370 msgr2=0x7fe10c104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:55.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.575+0000 7fe110f98700 1 --2- 172.21.15.110:0/3264482486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe10c104370 0x7fe10c104780 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc009990 tx=0x7fe0fc009ca0 comp rx=0 tx=0).stop 2024-05-09T04:46:55.578 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.575+0000 7fe110f98700 1 -- 172.21.15.110:0/3264482486 shutdown_connections 2024-05-09T04:46:55.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.575+0000 7fe110f98700 1 --2- 172.21.15.110:0/3264482486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe10c105e40 0x7fe10c10c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:55.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.575+0000 7fe110f98700 1 --2- 172.21.15.110:0/3264482486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe10c104370 0x7fe10c104780 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:55.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.575+0000 7fe110f98700 1 --2- 172.21.15.110:0/3264482486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c1054a0 0x7fe10c1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:55.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.575+0000 7fe110f98700 1 -- 172.21.15.110:0/3264482486 >> 172.21.15.110:0/3264482486 conn(0x7fe10c0ff9d0 msgr2=0x7fe10c101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:55.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.575+0000 7fe110f98700 1 -- 172.21.15.110:0/3264482486 shutdown_connections 2024-05-09T04:46:55.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.575+0000 7fe110f98700 1 -- 172.21.15.110:0/3264482486 wait complete. 2024-05-09T04:46:55.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe110f98700 1 Processor -- start 2024-05-09T04:46:55.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe110f98700 1 -- start start 2024-05-09T04:46:55.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe110f98700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c104370 0x7fe10c19a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:55.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe110f98700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe10c1054a0 0x7fe10c19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:55.579 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe110f98700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe10c105e40 0x7fe10c1a1db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:55.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe110f98700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10fdb0 con 0x7fe10c1054a0 2024-05-09T04:46:55.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe110f98700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10fc30 con 0x7fe10c104370 2024-05-09T04:46:55.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe110f98700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe10c10ff30 con 0x7fe10c105e40 2024-05-09T04:46:55.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe10ad9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe10c105e40 0x7fe10c1a1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:55.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe10ad9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe10c105e40 0x7fe10c1a1db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.110:48396/0 (socket says 172.21.15.110:48396) 2024-05-09T04:46:55.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe10ad9d700 1 -- 172.21.15.110:0/4117283912 learned_addr learned my addr 172.21.15.110:0/4117283912 (peer_addr_for_me v2:172.21.15.110:0/0) 2024-05-09T04:46:55.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe109d9b700 1 --2- 172.21.15.110:0/4117283912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe10c1054a0 0x7fe10c19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:55.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe10a59c700 1 --2- 172.21.15.110:0/4117283912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c104370 0x7fe10c19a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:55.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe10ad9d700 1 -- 172.21.15.110:0/4117283912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c104370 msgr2=0x7fe10c19a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:55.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe10ad9d700 1 --2- 172.21.15.110:0/4117283912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c104370 0x7fe10c19a8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:55.580 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe10ad9d700 1 -- 172.21.15.110:0/4117283912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe10c1054a0 msgr2=0x7fe10c19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:55.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe10ad9d700 1 --2- 172.21.15.110:0/4117283912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe10c1054a0 0x7fe10c19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:55.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe10ad9d700 1 -- 172.21.15.110:0/4117283912 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe10c1a23a0 con 0x7fe10c105e40 2024-05-09T04:46:55.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.576+0000 7fe10a59c700 1 --2- 172.21.15.110:0/4117283912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c104370 0x7fe10c19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:46:55.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.577+0000 7fe10ad9d700 1 --2- 172.21.15.110:0/4117283912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe10c105e40 0x7fe10c1a1db0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe10400ef30 tx=0x7fe10400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:55.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.577+0000 7fe0fb7fe700 1 -- 172.21.15.110:0/4117283912 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1040092e0 con 0x7fe10c105e40 2024-05-09T04:46:55.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.577+0000 7fe0fb7fe700 1 -- 172.21.15.110:0/4117283912 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe104004bb0 con 0x7fe10c105e40 2024-05-09T04:46:55.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.577+0000 7fe110f98700 1 -- 172.21.15.110:0/4117283912 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe10c1a2630 con 0x7fe10c105e40 2024-05-09T04:46:55.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.577+0000 7fe0fb7fe700 1 -- 172.21.15.110:0/4117283912 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe104004280 con 0x7fe10c105e40 2024-05-09T04:46:55.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.577+0000 7fe110f98700 1 -- 172.21.15.110:0/4117283912 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe10c1a2b20 con 0x7fe10c105e40 2024-05-09T04:46:55.581 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.578+0000 7fe110f98700 1 -- 172.21.15.110:0/4117283912 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe10c0512c0 con 0x7fe10c105e40 2024-05-09T04:46:55.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.579+0000 7fe0fb7fe700 1 -- 172.21.15.110:0/4117283912 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe10401b030 con 0x7fe10c105e40 2024-05-09T04:46:55.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.580+0000 7fe0fb7fe700 1 --2- 172.21.15.110:0/4117283912 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe0f40b2880 0x7fe0f40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:55.582 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.580+0000 7fe0fb7fe700 1 -- 172.21.15.110:0/4117283912 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fe1040e22c0 con 0x7fe10c105e40 2024-05-09T04:46:55.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.580+0000 7fe10a59c700 1 --2- 172.21.15.110:0/4117283912 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe0f40b2880 0x7fe0f40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:55.583 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.581+0000 7fe10a59c700 1 --2- 172.21.15.110:0/4117283912 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe0f40b2880 0x7fe0f40b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00b760 tx=0x7fe0fc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:55.584 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.584+0000 7fe0fb7fe700 1 -- 172.21.15.110:0/4117283912 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe104022020 con 0x7fe10c105e40 2024-05-09T04:46:55.794 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.794+0000 7fe110f98700 1 -- 172.21.15.110:0/4117283912 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l", "target": ["mon-mgr", ""]}) -- 0x7fe10c063dc0 con 0x7fe0f40b2880 2024-05-09T04:46:55.803 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.803+0000 7fe0fb7fe700 1 -- 172.21.15.110:0/4117283912 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7fe10c063dc0 con 0x7fe0f40b2880 2024-05-09T04:46:55.803 INFO:teuthology.orchestra.run.smithi110.stdout:Scheduled mds.all update... 2024-05-09T04:46:55.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.805+0000 7fe110f98700 1 -- 172.21.15.110:0/4117283912 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe0f40b2880 msgr2=0x7fe0f40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:55.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.805+0000 7fe110f98700 1 --2- 172.21.15.110:0/4117283912 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe0f40b2880 0x7fe0f40b4d90 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc00b760 tx=0x7fe0fc005fb0 comp rx=0 tx=0).stop 2024-05-09T04:46:55.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.805+0000 7fe110f98700 1 -- 172.21.15.110:0/4117283912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe10c105e40 msgr2=0x7fe10c1a1db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:55.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.805+0000 7fe110f98700 1 --2- 172.21.15.110:0/4117283912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe10c105e40 0x7fe10c1a1db0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe10400ef30 tx=0x7fe10400c3a0 comp rx=0 tx=0).stop 2024-05-09T04:46:55.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.805+0000 7fe110f98700 1 -- 172.21.15.110:0/4117283912 shutdown_connections 2024-05-09T04:46:55.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.805+0000 7fe110f98700 1 --2- 172.21.15.110:0/4117283912 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe0f40b2880 0x7fe0f40b4d90 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:55.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.805+0000 7fe110f98700 1 --2- 172.21.15.110:0/4117283912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe10c105e40 0x7fe10c1a1db0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:55.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.805+0000 7fe110f98700 1 --2- 172.21.15.110:0/4117283912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe10c104370 0x7fe10c19a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:55.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.805+0000 7fe110f98700 1 --2- 172.21.15.110:0/4117283912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe10c1054a0 0x7fe10c19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:55.806 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.805+0000 7fe110f98700 1 -- 172.21.15.110:0/4117283912 >> 172.21.15.110:0/4117283912 conn(0x7fe10c0ff9d0 msgr2=0x7fe10c1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:55.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.805+0000 7fe110f98700 1 -- 172.21.15.110:0/4117283912 shutdown_connections 2024-05-09T04:46:55.807 INFO:teuthology.orchestra.run.smithi110.stderr:2024-05-09T04:46:55.805+0000 7fe110f98700 1 -- 172.21.15.110:0/4117283912 wait complete. 2024-05-09T04:46:56.508 DEBUG:teuthology.orchestra.run.smithi046:mds.a> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.a.service 2024-05-09T04:46:56.511 DEBUG:teuthology.orchestra.run.smithi046:mds.d> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.d.service 2024-05-09T04:46:56.553 DEBUG:teuthology.orchestra.run.smithi046:mds.g> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.g.service 2024-05-09T04:46:56.596 DEBUG:teuthology.orchestra.run.smithi046:mds.j> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.j.service 2024-05-09T04:46:56.639 DEBUG:teuthology.orchestra.run.smithi072:mds.b> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.b.service 2024-05-09T04:46:56.642 DEBUG:teuthology.orchestra.run.smithi072:mds.e> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.e.service 2024-05-09T04:46:56.685 DEBUG:teuthology.orchestra.run.smithi072:mds.h> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.h.service 2024-05-09T04:46:56.702 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:56 smithi072 ceph-mon[34088]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:56.702 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:56.702 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:56.703 DEBUG:teuthology.orchestra.run.smithi072:mds.k> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.k.service 2024-05-09T04:46:56.745 DEBUG:teuthology.orchestra.run.smithi110:mds.c> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.c.service 2024-05-09T04:46:56.748 DEBUG:teuthology.orchestra.run.smithi110:mds.f> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.f.service 2024-05-09T04:46:56.767 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:56 smithi110 ceph-mon[34904]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:56.767 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:56.767 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:56.768 DEBUG:teuthology.orchestra.run.smithi110:mds.i> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.i.service 2024-05-09T04:46:56.811 DEBUG:teuthology.orchestra.run.smithi110:mds.l> sudo journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.l.service 2024-05-09T04:46:56.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:56 smithi046 ceph-mon[30136]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:56.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:56.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:46:56.854 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-09T04:46:56.854 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:46:57.103 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:46:57.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:57 smithi046 ceph-mon[30136]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:46:57.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:57 smithi046 ceph-mon[30136]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l;count:12 2024-05-09T04:46:57.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:57 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:57.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:57 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:57.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:57 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:57.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:57 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:46:57.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:57 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:46:57.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:57 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:57.528 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:57 smithi046 ceph-mon[30136]: Deploying daemon mds.l on smithi110 2024-05-09T04:46:57.643 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:57 smithi110 ceph-mon[34904]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:46:57.644 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:57 smithi110 ceph-mon[34904]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l;count:12 2024-05-09T04:46:57.644 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:57 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:57.644 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:57 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:57.644 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:57 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:57.645 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:57 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:46:57.645 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:57 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:46:57.645 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:57 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:57.645 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:57 smithi110 ceph-mon[34904]: Deploying daemon mds.l on smithi110 2024-05-09T04:46:57.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:57 smithi072 ceph-mon[34088]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:46:57.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:57 smithi072 ceph-mon[34088]: Saving service mds.all spec with placement smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l;count:12 2024-05-09T04:46:57.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:57 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:57.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:57 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:46:57.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:57 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:57.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:57 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:46:57.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:57 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:46:57.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:57 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:46:57.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:57 smithi072 ceph-mon[34088]: Deploying daemon mds.l on smithi110 2024-05-09T04:46:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.846+0000 7fb8015d6700 1 -- 172.21.15.46:0/438871448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc1045d0 msgr2=0x7fb7fc1049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.846+0000 7fb8015d6700 1 --2- 172.21.15.46:0/438871448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc1045d0 0x7fb7fc1049e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec00b0d0 tx=0x7fb7ec00b3e0 comp rx=0 tx=0).stop 2024-05-09T04:46:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.846+0000 7fb8015d6700 1 -- 172.21.15.46:0/438871448 shutdown_connections 2024-05-09T04:46:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.846+0000 7fb8015d6700 1 --2- 172.21.15.46:0/438871448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7fc109850 0x7fb7fc1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.846+0000 7fb8015d6700 1 --2- 172.21.15.46:0/438871448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc1045d0 0x7fb7fc1049e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.846+0000 7fb8015d6700 1 --2- 172.21.15.46:0/438871448 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7fc108f00 0x7fb7fc109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.846+0000 7fb8015d6700 1 -- 172.21.15.46:0/438871448 >> 172.21.15.46:0/438871448 conn(0x7fb7fc0ffea0 msgr2=0x7fb7fc1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.846+0000 7fb8015d6700 1 -- 172.21.15.46:0/438871448 shutdown_connections 2024-05-09T04:46:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.846+0000 7fb8015d6700 1 -- 172.21.15.46:0/438871448 wait complete. 2024-05-09T04:46:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.846+0000 7fb8015d6700 1 Processor -- start 2024-05-09T04:46:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.847+0000 7fb8015d6700 1 -- start start 2024-05-09T04:46:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.847+0000 7fb8015d6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7fc1045d0 0x7fb7fc19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.847+0000 7fb8015d6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7fc108f00 0x7fb7fc19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.847+0000 7fb8015d6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc109850 0x7fb7fc19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.847+0000 7fb8015d6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc114480 con 0x7fb7fc109850 2024-05-09T04:46:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.847+0000 7fb8015d6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc114300 con 0x7fb7fc1045d0 2024-05-09T04:46:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.847+0000 7fb8015d6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb7fc114600 con 0x7fb7fc108f00 2024-05-09T04:46:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.847+0000 7fb7fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc109850 0x7fb7fc19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.847+0000 7fb7fa7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7fc108f00 0x7fb7fc19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.847+0000 7fb7fa7fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7fc108f00 0x7fb7fc19b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:49206/0 (socket says 172.21.15.46:49206) 2024-05-09T04:46:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.847+0000 7fb7fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc109850 0x7fb7fc19f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34506/0 (socket says 172.21.15.46:34506) 2024-05-09T04:46:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.847+0000 7fb7fa7fc700 1 -- 172.21.15.46:0/622268315 learned_addr learned my addr 172.21.15.46:0/622268315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:46:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.847+0000 7fb7faffd700 1 --2- 172.21.15.46:0/622268315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7fc1045d0 0x7fb7fc19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.848+0000 7fb7fb7fe700 1 -- 172.21.15.46:0/622268315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7fc108f00 msgr2=0x7fb7fc19b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.848+0000 7fb7fb7fe700 1 --2- 172.21.15.46:0/622268315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7fc108f00 0x7fb7fc19b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.848+0000 7fb7fb7fe700 1 -- 172.21.15.46:0/622268315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7fc1045d0 msgr2=0x7fb7fc19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.848+0000 7fb7fb7fe700 1 --2- 172.21.15.46:0/622268315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7fc1045d0 0x7fb7fc19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.848+0000 7fb7fb7fe700 1 -- 172.21.15.46:0/622268315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7fc19fbf0 con 0x7fb7fc109850 2024-05-09T04:46:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.848+0000 7fb7fb7fe700 1 --2- 172.21.15.46:0/622268315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc109850 0x7fb7fc19f600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400c970 tx=0x7fb7f400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.848+0000 7fb7f3fff700 1 -- 172.21.15.46:0/622268315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f400e880 con 0x7fb7fc109850 2024-05-09T04:46:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.848+0000 7fb8015d6700 1 -- 172.21.15.46:0/622268315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7fc19fee0 con 0x7fb7fc109850 2024-05-09T04:46:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.848+0000 7fb8015d6700 1 -- 172.21.15.46:0/622268315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7fc1a0420 con 0x7fb7fc109850 2024-05-09T04:46:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.848+0000 7fb7f3fff700 1 -- 172.21.15.46:0/622268315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7f400f450 con 0x7fb7fc109850 2024-05-09T04:46:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.848+0000 7fb7f3fff700 1 -- 172.21.15.46:0/622268315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7f4018660 con 0x7fb7fc109850 2024-05-09T04:46:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.849+0000 7fb8015d6700 1 -- 172.21.15.46:0/622268315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7fc0512c0 con 0x7fb7fc109850 2024-05-09T04:46:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.851+0000 7fb7f3fff700 1 -- 172.21.15.46:0/622268315 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb7f400ea20 con 0x7fb7fc109850 2024-05-09T04:46:58.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.851+0000 7fb7f3fff700 1 --2- 172.21.15.46:0/622268315 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb7e80b2900 0x7fb7e80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:46:58.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.852+0000 7fb7faffd700 1 --2- 172.21.15.46:0/622268315 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb7e80b2900 0x7fb7e80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:46:58.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.852+0000 7fb7f3fff700 1 -- 172.21.15.46:0/622268315 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fb7f40e1b20 con 0x7fb7fc109850 2024-05-09T04:46:58.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.852+0000 7fb7faffd700 1 --2- 172.21.15.46:0/622268315 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb7e80b2900 0x7fb7e80b4e10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009900 tx=0x7fb7ec00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:46:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:58.855+0000 7fb7f3fff700 1 -- 172.21.15.46:0/622268315 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7f4014030 con 0x7fb7fc109850 2024-05-09T04:46:58.991 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:58 smithi110 ceph-mon[34904]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:58.991 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 09 04:46:58 smithi110 systemd[1]: Starting Ceph mds.l for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:46:59.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:58 smithi046 ceph-mon[30136]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:59.116 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:59.116+0000 7fb8015d6700 1 -- 172.21.15.46:0/622268315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7fc19c130 con 0x7fb7fc109850 2024-05-09T04:46:59.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:59.117+0000 7fb7f3fff700 1 -- 172.21.15.46:0/622268315 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fb7f40a9df0 con 0x7fb7fc109850 2024-05-09T04:46:59.117 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:46:59.117 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-05-09T04:46:59.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:59.119+0000 7fb8015d6700 1 -- 172.21.15.46:0/622268315 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb7e80b2900 msgr2=0x7fb7e80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:59.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:59.119+0000 7fb8015d6700 1 --2- 172.21.15.46:0/622268315 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb7e80b2900 0x7fb7e80b4e10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb7ec009900 tx=0x7fb7ec00b040 comp rx=0 tx=0).stop 2024-05-09T04:46:59.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:59.119+0000 7fb8015d6700 1 -- 172.21.15.46:0/622268315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc109850 msgr2=0x7fb7fc19f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:46:59.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:59.119+0000 7fb8015d6700 1 --2- 172.21.15.46:0/622268315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc109850 0x7fb7fc19f600 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fb7f400c970 tx=0x7fb7f400cc80 comp rx=0 tx=0).stop 2024-05-09T04:46:59.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:59.119+0000 7fb8015d6700 1 -- 172.21.15.46:0/622268315 shutdown_connections 2024-05-09T04:46:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:59.119+0000 7fb8015d6700 1 --2- 172.21.15.46:0/622268315 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb7e80b2900 0x7fb7e80b4e10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:59.119+0000 7fb8015d6700 1 --2- 172.21.15.46:0/622268315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb7fc109850 0x7fb7fc19f600 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:59.119+0000 7fb8015d6700 1 --2- 172.21.15.46:0/622268315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb7fc1045d0 0x7fb7fc19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:59.119+0000 7fb8015d6700 1 --2- 172.21.15.46:0/622268315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb7fc108f00 0x7fb7fc19b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:46:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:59.119+0000 7fb8015d6700 1 -- 172.21.15.46:0/622268315 >> 172.21.15.46:0/622268315 conn(0x7fb7fc0ffea0 msgr2=0x7fb7fc10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:46:59.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:59.120+0000 7fb8015d6700 1 -- 172.21.15.46:0/622268315 shutdown_connections 2024-05-09T04:46:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:46:59.120+0000 7fb8015d6700 1 -- 172.21.15.46:0/622268315 wait complete. 2024-05-09T04:46:59.122 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 1 2024-05-09T04:46:59.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:58 smithi072 ceph-mon[34088]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:46:59.253 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 09 04:46:58 smithi110 podman[64577]: 2024-05-09 04:46:58.894695233 +0000 UTC m=+0.018151901 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:46:59.253 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 09 04:46:59 smithi110 podman[64577]: 2024-05-09 04:46:59.02645874 +0000 UTC m=+0.149915407 container create 7f7cc7082be409806ee37dd6e4406ff6a44d4504c724de12fd5a17ac6754b672 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-l, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-09T04:46:59.253 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 09 04:46:59 smithi110 podman[64577]: 2024-05-09 04:46:59.253945258 +0000 UTC m=+0.377401925 container init 7f7cc7082be409806ee37dd6e4406ff6a44d4504c724de12fd5a17ac6754b672 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-l, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2) 2024-05-09T04:46:59.557 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 09 04:46:59 smithi110 podman[64577]: 2024-05-09 04:46:59.256567396 +0000 UTC m=+0.380024064 container start 7f7cc7082be409806ee37dd6e4406ff6a44d4504c724de12fd5a17ac6754b672 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T04:46:59.557 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 09 04:46:59 smithi110 ceph-mds[64595]: -- 172.21.15.110:0/4147898419 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c4ec44fa40 con 0x55c4ec497180 2024-05-09T04:46:59.557 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 09 04:46:59 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-l[64591]: starting mds.l at 2024-05-09T04:46:59.557 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 09 04:46:59 smithi110 bash[64577]: 7f7cc7082be409806ee37dd6e4406ff6a44d4504c724de12fd5a17ac6754b672 2024-05-09T04:46:59.557 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 09 04:46:59 smithi110 systemd[1]: Started Ceph mds.l for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:46:59.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:59 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/622268315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:46:59.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:59 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.110:6832/2118887975,v1:172.21.15.110:6833/2118887975] up:boot 2024-05-09T04:46:59.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:59 smithi046 ceph-mon[30136]: fsmap 1 up:standby 2024-05-09T04:46:59.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:59 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T04:46:59.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:59 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:59.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:59 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:59.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:59 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:46:59.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:59 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:46:59.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:59 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:46:59.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:46:59 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:59 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/622268315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:59 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.110:6832/2118887975,v1:172.21.15.110:6833/2118887975] up:boot 2024-05-09T04:47:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:59 smithi110 ceph-mon[34904]: fsmap 1 up:standby 2024-05-09T04:47:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:59 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T04:47:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:59 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:59 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:59 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:59 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:59 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:00.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:46:59 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:00.078 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:59 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/622268315' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:00.078 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:59 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.110:6832/2118887975,v1:172.21.15.110:6833/2118887975] up:boot 2024-05-09T04:47:00.079 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:59 smithi072 ceph-mon[34088]: fsmap 1 up:standby 2024-05-09T04:47:00.079 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:59 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T04:47:00.079 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:59 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:00.079 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:59 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:00.079 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:59 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:00.079 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:59 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:00.079 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:59 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:00.079 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:46:59 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:01.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:00 smithi110 ceph-mon[34904]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:01.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:00 smithi110 ceph-mon[34904]: Deploying daemon mds.k on smithi072 2024-05-09T04:47:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:00 smithi046 ceph-mon[30136]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:00 smithi046 ceph-mon[30136]: Deploying daemon mds.k on smithi072 2024-05-09T04:47:01.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:00 smithi072 ceph-mon[34088]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:01.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:00 smithi072 ceph-mon[34088]: Deploying daemon mds.k on smithi072 2024-05-09T04:47:01.779 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:47:02.041 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:47:02.130 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 09 04:47:02 smithi072 podman[62791]: 2024-05-09 04:47:02.04619701 +0000 UTC m=+0.121283551 container create 83a0c56c6f0a5346eaeb8d34939fc3d7e15cb91a7c4abdb3e773c4184f3b83d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-k, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T04:47:02.630 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 09 04:47:02 smithi072 podman[62791]: 2024-05-09 04:47:02.284699093 +0000 UTC m=+0.359785613 container init 83a0c56c6f0a5346eaeb8d34939fc3d7e15cb91a7c4abdb3e773c4184f3b83d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-k, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2) 2024-05-09T04:47:02.630 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 09 04:47:02 smithi072 podman[62791]: 2024-05-09 04:47:02.28727946 +0000 UTC m=+0.362365956 container start 83a0c56c6f0a5346eaeb8d34939fc3d7e15cb91a7c4abdb3e773c4184f3b83d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-k, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2) 2024-05-09T04:47:02.630 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 09 04:47:02 smithi072 ceph-mds[62810]: -- 172.21.15.72:0/232635593 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ee167afa40 con 0x55ee167f7600 2024-05-09T04:47:02.630 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 09 04:47:02 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-k[62806]: starting mds.k at 2024-05-09T04:47:02.630 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 09 04:47:02 smithi072 bash[62791]: 83a0c56c6f0a5346eaeb8d34939fc3d7e15cb91a7c4abdb3e773c4184f3b83d1 2024-05-09T04:47:02.630 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 09 04:47:02 smithi072 systemd[1]: Started Ceph mds.k for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:47:02.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:02 smithi046 ceph-mon[30136]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:02.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:02.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:02.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:02.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:02.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:02.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:02 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:03.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:02 smithi110 ceph-mon[34904]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:03.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:03.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:03.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:03.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:03.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:03.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:02 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:02 smithi072 ceph-mon[34088]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:03.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:03.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:03.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:02 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.879+0000 7f73ca42b700 1 -- 172.21.15.46:0/875865950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c41034c0 msgr2=0x7f73c41038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.879+0000 7f73ca42b700 1 --2- 172.21.15.46:0/875865950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c41034c0 0x7f73c41038d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f73b40099c0 tx=0x7f73b4009cd0 comp rx=0 tx=0).stop 2024-05-09T04:47:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.879+0000 7f73ca42b700 1 -- 172.21.15.46:0/875865950 shutdown_connections 2024-05-09T04:47:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.879+0000 7f73ca42b700 1 --2- 172.21.15.46:0/875865950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c4108740 0x7f73c41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.879+0000 7f73ca42b700 1 --2- 172.21.15.46:0/875865950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c41034c0 0x7f73c41038d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.879+0000 7f73ca42b700 1 --2- 172.21.15.46:0/875865950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73c4107df0 0x7f73c4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.879+0000 7f73ca42b700 1 -- 172.21.15.46:0/875865950 >> 172.21.15.46:0/875865950 conn(0x7f73c40fedc0 msgr2=0x7f73c41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.879+0000 7f73ca42b700 1 -- 172.21.15.46:0/875865950 shutdown_connections 2024-05-09T04:47:03.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.879+0000 7f73ca42b700 1 -- 172.21.15.46:0/875865950 wait complete. 2024-05-09T04:47:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.880+0000 7f73ca42b700 1 Processor -- start 2024-05-09T04:47:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.880+0000 7f73ca42b700 1 -- start start 2024-05-09T04:47:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.880+0000 7f73ca42b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73c41034c0 0x7f73c419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.880+0000 7f73ca42b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c4107df0 0x7f73c419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:03.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.880+0000 7f73ca42b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4108740 0x7f73c419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.880+0000 7f73ca42b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f73c4113370 con 0x7f73c4108740 2024-05-09T04:47:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.880+0000 7f73ca42b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f73c41131f0 con 0x7f73c4107df0 2024-05-09T04:47:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.880+0000 7f73ca42b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f73c41134f0 con 0x7f73c41034c0 2024-05-09T04:47:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.880+0000 7f73c89c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4108740 0x7f73c419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.880+0000 7f73c89c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4108740 0x7f73c419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52026/0 (socket says 172.21.15.46:52026) 2024-05-09T04:47:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.880+0000 7f73c89c8700 1 -- 172.21.15.46:0/3899351266 learned_addr learned my addr 172.21.15.46:0/3899351266 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:47:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.880+0000 7f73c37fe700 1 --2- 172.21.15.46:0/3899351266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c4107df0 0x7f73c419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.880+0000 7f73c3fff700 1 --2- 172.21.15.46:0/3899351266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73c41034c0 0x7f73c419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.881+0000 7f73c89c8700 1 -- 172.21.15.46:0/3899351266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73c41034c0 msgr2=0x7f73c419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.881+0000 7f73c89c8700 1 --2- 172.21.15.46:0/3899351266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73c41034c0 0x7f73c419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.881+0000 7f73c89c8700 1 -- 172.21.15.46:0/3899351266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c4107df0 msgr2=0x7f73c419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.881+0000 7f73c89c8700 1 --2- 172.21.15.46:0/3899351266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c4107df0 0x7f73c419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.881+0000 7f73c89c8700 1 -- 172.21.15.46:0/3899351266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73c419fbf0 con 0x7f73c4108740 2024-05-09T04:47:03.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.881+0000 7f73c89c8700 1 --2- 172.21.15.46:0/3899351266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4108740 0x7f73c419f600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00ef30 tx=0x7f73bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.881+0000 7f73c17fa700 1 -- 172.21.15.46:0/3899351266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc0092e0 con 0x7f73c4108740 2024-05-09T04:47:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.881+0000 7f73c17fa700 1 -- 172.21.15.46:0/3899351266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73bc004bb0 con 0x7f73c4108740 2024-05-09T04:47:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.881+0000 7f73ca42b700 1 -- 172.21.15.46:0/3899351266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73c419fe80 con 0x7f73c4108740 2024-05-09T04:47:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.881+0000 7f73c17fa700 1 -- 172.21.15.46:0/3899351266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73bc004020 con 0x7f73c4108740 2024-05-09T04:47:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.881+0000 7f73ca42b700 1 -- 172.21.15.46:0/3899351266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73c41a03c0 con 0x7f73c4108740 2024-05-09T04:47:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.882+0000 7f73ca42b700 1 -- 172.21.15.46:0/3899351266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73c40512c0 con 0x7f73c4108740 2024-05-09T04:47:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.884+0000 7f73c17fa700 1 -- 172.21.15.46:0/3899351266 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f73bc01b030 con 0x7f73c4108740 2024-05-09T04:47:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.885+0000 7f73c17fa700 1 --2- 172.21.15.46:0/3899351266 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f73ac0b2900 0x7f73ac0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.885+0000 7f73c3fff700 1 --2- 172.21.15.46:0/3899351266 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f73ac0b2900 0x7f73ac0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.885+0000 7f73c17fa700 1 -- 172.21.15.46:0/3899351266 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f73bc0e2160 con 0x7f73c4108740 2024-05-09T04:47:03.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.885+0000 7f73c3fff700 1 --2- 172.21.15.46:0/3899351266 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f73ac0b2900 0x7f73ac0b4e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f73b4003760 tx=0x7f73b401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:03.888+0000 7f73c17fa700 1 -- 172.21.15.46:0/3899351266 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f73bc022020 con 0x7f73c4108740 2024-05-09T04:47:03.906 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:03 smithi072 ceph-mon[34088]: Deploying daemon mds.j on smithi046 2024-05-09T04:47:03.906 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:03 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.72:6832/815890099,v1:172.21.15.72:6833/815890099] up:boot 2024-05-09T04:47:03.906 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:03 smithi072 ceph-mon[34088]: fsmap 2 up:standby 2024-05-09T04:47:03.906 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:03 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T04:47:04.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:03 smithi046 ceph-mon[30136]: Deploying daemon mds.j on smithi046 2024-05-09T04:47:04.033 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:03 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.72:6832/815890099,v1:172.21.15.72:6833/815890099] up:boot 2024-05-09T04:47:04.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:03 smithi046 ceph-mon[30136]: fsmap 2 up:standby 2024-05-09T04:47:04.034 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:03 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T04:47:04.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:03 smithi110 ceph-mon[34904]: Deploying daemon mds.j on smithi046 2024-05-09T04:47:04.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:03 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.72:6832/815890099,v1:172.21.15.72:6833/815890099] up:boot 2024-05-09T04:47:04.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:03 smithi110 ceph-mon[34904]: fsmap 2 up:standby 2024-05-09T04:47:04.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:03 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T04:47:04.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:04.156+0000 7f73ca42b700 1 -- 172.21.15.46:0/3899351266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f73c402e5f0 con 0x7f73c4108740 2024-05-09T04:47:04.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:04.156+0000 7f73c17fa700 1 -- 172.21.15.46:0/3899351266 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2218 (secure 0 0 0) 0x7f73bc010040 con 0x7f73c4108740 2024-05-09T04:47:04.156 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:47:04.156 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T04:47:04.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:04.159+0000 7f73ca42b700 1 -- 172.21.15.46:0/3899351266 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f73ac0b2900 msgr2=0x7f73ac0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:04.159+0000 7f73ca42b700 1 --2- 172.21.15.46:0/3899351266 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f73ac0b2900 0x7f73ac0b4e10 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f73b4003760 tx=0x7f73b401a040 comp rx=0 tx=0).stop 2024-05-09T04:47:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:04.159+0000 7f73ca42b700 1 -- 172.21.15.46:0/3899351266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4108740 msgr2=0x7f73c419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:04.159+0000 7f73ca42b700 1 --2- 172.21.15.46:0/3899351266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4108740 0x7f73c419f600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f73bc00ef30 tx=0x7f73bc00c3a0 comp rx=0 tx=0).stop 2024-05-09T04:47:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:04.159+0000 7f73ca42b700 1 -- 172.21.15.46:0/3899351266 shutdown_connections 2024-05-09T04:47:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:04.159+0000 7f73ca42b700 1 --2- 172.21.15.46:0/3899351266 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f73ac0b2900 0x7f73ac0b4e10 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:04.159+0000 7f73ca42b700 1 --2- 172.21.15.46:0/3899351266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f73c4108740 0x7f73c419f600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:04.159+0000 7f73ca42b700 1 --2- 172.21.15.46:0/3899351266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f73c41034c0 0x7f73c419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:04.159+0000 7f73ca42b700 1 --2- 172.21.15.46:0/3899351266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f73c4107df0 0x7f73c419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:04.159+0000 7f73ca42b700 1 -- 172.21.15.46:0/3899351266 >> 172.21.15.46:0/3899351266 conn(0x7f73c40fedc0 msgr2=0x7f73c4101e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:04.159+0000 7f73ca42b700 1 -- 172.21.15.46:0/3899351266 shutdown_connections 2024-05-09T04:47:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:04.159+0000 7f73ca42b700 1 -- 172.21.15.46:0/3899351266 wait complete. 2024-05-09T04:47:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 3 2024-05-09T04:47:04.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:04 smithi046 ceph-mon[30136]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:04.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:04 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3899351266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:05.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:04 smithi110 ceph-mon[34904]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:05.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:04 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3899351266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:04 smithi072 ceph-mon[34088]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:04 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3899351266' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:06.580 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 04:47:06 smithi046 systemd[1]: Starting Ceph mds.j for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:47:06.868 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:06 smithi046 ceph-mon[30136]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:06.869 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 04:47:06 smithi046 podman[70947]: 2024-05-09 04:47:06.491416682 +0000 UTC m=+0.018496447 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:47:06.869 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 04:47:06 smithi046 podman[70947]: 2024-05-09 04:47:06.605033166 +0000 UTC m=+0.132112916 container create 03a8ee9e792ca3ffec1c2215cef6a618db7e46383d918a32d842592079a8f5b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-j, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-09T04:47:06.869 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 04:47:06 smithi046 podman[70947]: 2024-05-09 04:47:06.833032747 +0000 UTC m=+0.360112496 container init 03a8ee9e792ca3ffec1c2215cef6a618db7e46383d918a32d842592079a8f5b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-j, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-09T04:47:06.869 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 04:47:06 smithi046 podman[70947]: 2024-05-09 04:47:06.836190597 +0000 UTC m=+0.363270357 container start 03a8ee9e792ca3ffec1c2215cef6a618db7e46383d918a32d842592079a8f5b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T04:47:07.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:06 smithi110 ceph-mon[34904]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:06 smithi072 ceph-mon[34088]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:07.330 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 04:47:06 smithi046 ceph-mds[70967]: -- 172.21.15.46:0/86732510 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56138c36ba40 con 0x56138c3b3600 2024-05-09T04:47:07.331 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 04:47:06 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-j[70963]: starting mds.j at 2024-05-09T04:47:07.331 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 04:47:06 smithi046 bash[70947]: 03a8ee9e792ca3ffec1c2215cef6a618db7e46383d918a32d842592079a8f5b9 2024-05-09T04:47:07.331 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 04:47:06 smithi046 systemd[1]: Started Ceph mds.j for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:47:07.409 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:47:08.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:07 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:08.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:07 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:08.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:07 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:08.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:07 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:08.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:07 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:08.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:07 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:08.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:07 smithi110 ceph-mon[34904]: Deploying daemon mds.d on smithi046 2024-05-09T04:47:08.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:07 smithi110 ceph-mon[34904]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:08.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:07 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.46:6834/3023074192,v1:172.21.15.46:6835/3023074192] up:boot 2024-05-09T04:47:08.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:07 smithi110 ceph-mon[34904]: fsmap 3 up:standby 2024-05-09T04:47:08.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:07 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T04:47:08.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:07 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:08.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:07 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:08.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:07 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:08.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:07 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:08.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:07 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:08.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:07 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:08.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:07 smithi046 ceph-mon[30136]: Deploying daemon mds.d on smithi046 2024-05-09T04:47:08.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:07 smithi046 ceph-mon[30136]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:08.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:07 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.46:6834/3023074192,v1:172.21.15.46:6835/3023074192] up:boot 2024-05-09T04:47:08.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:07 smithi046 ceph-mon[30136]: fsmap 3 up:standby 2024-05-09T04:47:08.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:07 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T04:47:08.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:07 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:08.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:07 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:08.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:07 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:08.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:07 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:08.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:07 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:08.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:07 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:08.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:07 smithi072 ceph-mon[34088]: Deploying daemon mds.d on smithi046 2024-05-09T04:47:08.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:07 smithi072 ceph-mon[34088]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:08.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:07 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.46:6834/3023074192,v1:172.21.15.46:6835/3023074192] up:boot 2024-05-09T04:47:08.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:07 smithi072 ceph-mon[34088]: fsmap 3 up:standby 2024-05-09T04:47:08.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:07 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T04:47:08.927 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:47:10.438 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:10 smithi046 ceph-mon[30136]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:10.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:10 smithi072 ceph-mon[34088]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:10.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:10 smithi110 ceph-mon[34904]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.901+0000 7f997afe2700 1 -- 172.21.15.46:0/1381440310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99741045e0 msgr2=0x7f99741049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.901+0000 7f997afe2700 1 --2- 172.21.15.46:0/1381440310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99741045e0 0x7f99741049f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f99680099c0 tx=0x7f9968009cd0 comp rx=0 tx=0).stop 2024-05-09T04:47:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.901+0000 7f997afe2700 1 -- 172.21.15.46:0/1381440310 shutdown_connections 2024-05-09T04:47:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.901+0000 7f997afe2700 1 --2- 172.21.15.46:0/1381440310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9974109860 0x7f99741111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.901+0000 7f997afe2700 1 --2- 172.21.15.46:0/1381440310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f99741045e0 0x7f99741049f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.901+0000 7f997afe2700 1 --2- 172.21.15.46:0/1381440310 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9974108f10 0x7f9974109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.901+0000 7f997afe2700 1 -- 172.21.15.46:0/1381440310 >> 172.21.15.46:0/1381440310 conn(0x7f99740ffed0 msgr2=0x7f9974102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.901+0000 7f997afe2700 1 -- 172.21.15.46:0/1381440310 shutdown_connections 2024-05-09T04:47:10.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.901+0000 7f997afe2700 1 -- 172.21.15.46:0/1381440310 wait complete. 2024-05-09T04:47:10.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.902+0000 7f997afe2700 1 Processor -- start 2024-05-09T04:47:10.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.902+0000 7f997afe2700 1 -- start start 2024-05-09T04:47:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f997afe2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99741045e0 0x7f997419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f997afe2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9974108f10 0x7f997419b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f997afe2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9974109860 0x7f997419f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f997afe2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9974114490 con 0x7f9974108f10 2024-05-09T04:47:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f997afe2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9974114310 con 0x7f9974109860 2024-05-09T04:47:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f997afe2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9974114610 con 0x7f99741045e0 2024-05-09T04:47:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f9973fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9974108f10 0x7f997419b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f9973fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9974108f10 0x7f997419b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52046/0 (socket says 172.21.15.46:52046) 2024-05-09T04:47:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f9973fff700 1 -- 172.21.15.46:0/4192493825 learned_addr learned my addr 172.21.15.46:0/4192493825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:47:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f9978d7e700 1 --2- 172.21.15.46:0/4192493825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99741045e0 0x7f997419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f997957f700 1 --2- 172.21.15.46:0/4192493825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9974109860 0x7f997419f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f9973fff700 1 -- 172.21.15.46:0/4192493825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99741045e0 msgr2=0x7f997419ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:10.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f9973fff700 1 --2- 172.21.15.46:0/4192493825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99741045e0 0x7f997419ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f9973fff700 1 -- 172.21.15.46:0/4192493825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9974109860 msgr2=0x7f997419f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f9973fff700 1 --2- 172.21.15.46:0/4192493825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9974109860 0x7f997419f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f9973fff700 1 -- 172.21.15.46:0/4192493825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f997419fcd0 con 0x7f9974108f10 2024-05-09T04:47:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f997957f700 1 --2- 172.21.15.46:0/4192493825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9974109860 0x7f997419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:47:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.903+0000 7f9973fff700 1 --2- 172.21.15.46:0/4192493825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9974108f10 0x7f997419b3a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f996000e970 tx=0x7f996000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.904+0000 7f9971ffb700 1 -- 172.21.15.46:0/4192493825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f996000cbd0 con 0x7f9974108f10 2024-05-09T04:47:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.904+0000 7f997afe2700 1 -- 172.21.15.46:0/4192493825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f997419ffc0 con 0x7f9974108f10 2024-05-09T04:47:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.904+0000 7f9971ffb700 1 -- 172.21.15.46:0/4192493825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f996000cd70 con 0x7f9974108f10 2024-05-09T04:47:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.904+0000 7f9971ffb700 1 -- 172.21.15.46:0/4192493825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9960005680 con 0x7f9974108f10 2024-05-09T04:47:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.904+0000 7f997afe2700 1 -- 172.21.15.46:0/4192493825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99741a78e0 con 0x7f9974108f10 2024-05-09T04:47:10.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.905+0000 7f997afe2700 1 -- 172.21.15.46:0/4192493825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99740512c0 con 0x7f9974108f10 2024-05-09T04:47:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.907+0000 7f9971ffb700 1 -- 172.21.15.46:0/4192493825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9960004750 con 0x7f9974108f10 2024-05-09T04:47:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.907+0000 7f9971ffb700 1 --2- 172.21.15.46:0/4192493825 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f99640b2880 0x7f99640b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.907+0000 7f9978d7e700 1 --2- 172.21.15.46:0/4192493825 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f99640b2880 0x7f99640b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:10.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.908+0000 7f9971ffb700 1 -- 172.21.15.46:0/4192493825 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f99600e2270 con 0x7f9974108f10 2024-05-09T04:47:10.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.908+0000 7f9978d7e700 1 --2- 172.21.15.46:0/4192493825 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f99640b2880 0x7f99640b4d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9968000c00 tx=0x7f996801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:10.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:10.911+0000 7f9971ffb700 1 -- 172.21.15.46:0/4192493825 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9960054df0 con 0x7f9974108f10 2024-05-09T04:47:11.048 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 04:47:11 smithi046 podman[71365]: 2024-05-09 04:47:11.015571177 +0000 UTC m=+0.813171025 container start eeecbdc5532bfe295b8ece6481dd3af60589bc2b70025f8f29668cdcae0aa8f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-d, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T04:47:11.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:11.178+0000 7f997afe2700 1 -- 172.21.15.46:0/4192493825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f997419c160 con 0x7f9974108f10 2024-05-09T04:47:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:11.180+0000 7f9971ffb700 1 -- 172.21.15.46:0/4192493825 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3043 (secure 0 0 0) 0x7f99600aa390 con 0x7f9974108f10 2024-05-09T04:47:11.180 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:47:11.180 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-09T04:47:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:11.182+0000 7f997afe2700 1 -- 172.21.15.46:0/4192493825 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f99640b2880 msgr2=0x7f99640b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:11.182+0000 7f997afe2700 1 --2- 172.21.15.46:0/4192493825 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f99640b2880 0x7f99640b4d90 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f9968000c00 tx=0x7f996801a040 comp rx=0 tx=0).stop 2024-05-09T04:47:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:11.182+0000 7f997afe2700 1 -- 172.21.15.46:0/4192493825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9974108f10 msgr2=0x7f997419b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:11.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:11.182+0000 7f997afe2700 1 --2- 172.21.15.46:0/4192493825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9974108f10 0x7f997419b3a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f996000e970 tx=0x7f996000ec80 comp rx=0 tx=0).stop 2024-05-09T04:47:11.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:11.182+0000 7f997afe2700 1 -- 172.21.15.46:0/4192493825 shutdown_connections 2024-05-09T04:47:11.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:11.182+0000 7f997afe2700 1 --2- 172.21.15.46:0/4192493825 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f99640b2880 0x7f99640b4d90 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:11.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:11.182+0000 7f997afe2700 1 --2- 172.21.15.46:0/4192493825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9974109860 0x7f997419f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:11.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:11.182+0000 7f997afe2700 1 --2- 172.21.15.46:0/4192493825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f99741045e0 0x7f997419ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:11.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:11.182+0000 7f997afe2700 1 --2- 172.21.15.46:0/4192493825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9974108f10 0x7f997419b3a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:11.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:11.182+0000 7f997afe2700 1 -- 172.21.15.46:0/4192493825 >> 172.21.15.46:0/4192493825 conn(0x7f99740ffed0 msgr2=0x7f9974108700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:11.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:11.183+0000 7f997afe2700 1 -- 172.21.15.46:0/4192493825 shutdown_connections 2024-05-09T04:47:11.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:11.183+0000 7f997afe2700 1 -- 172.21.15.46:0/4192493825 wait complete. 2024-05-09T04:47:11.183 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 4 2024-05-09T04:47:11.329 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 04:47:11 smithi046 ceph-mds[71413]: -- 172.21.15.46:0/2407804452 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563c25907a40 con 0x563c2594f180 2024-05-09T04:47:11.330 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 04:47:11 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-d[71387]: starting mds.d at 2024-05-09T04:47:11.330 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 04:47:11 smithi046 bash[71365]: eeecbdc5532bfe295b8ece6481dd3af60589bc2b70025f8f29668cdcae0aa8f0 2024-05-09T04:47:11.330 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 04:47:11 smithi046 systemd[1]: Started Ceph mds.d for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:47:12.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:12 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:12.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:12 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:12.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:12 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/4192493825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:12.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:12 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:12.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:12 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:12.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:12 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:12.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:12 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:12.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:12 smithi072 ceph-mon[34088]: Deploying daemon mds.e on smithi072 2024-05-09T04:47:12.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:12 smithi072 ceph-mon[34088]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:12.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:12 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.46:6836/3965071509,v1:172.21.15.46:6837/3965071509] up:boot 2024-05-09T04:47:12.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:12 smithi072 ceph-mon[34088]: fsmap 4 up:standby 2024-05-09T04:47:12.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:12 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T04:47:12.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:12.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:12.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:12 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/4192493825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:12.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:12.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:12.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:12.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:12.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:12 smithi110 ceph-mon[34904]: Deploying daemon mds.e on smithi072 2024-05-09T04:47:12.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:12 smithi110 ceph-mon[34904]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:12.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:12 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.46:6836/3965071509,v1:172.21.15.46:6837/3965071509] up:boot 2024-05-09T04:47:12.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:12 smithi110 ceph-mon[34904]: fsmap 4 up:standby 2024-05-09T04:47:12.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:12 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T04:47:12.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:12 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:12.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:12 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:12.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:12 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/4192493825' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:12.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:12 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:12.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:12 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:12.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:12 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:12.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:12 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:12.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:12 smithi046 ceph-mon[30136]: Deploying daemon mds.e on smithi072 2024-05-09T04:47:12.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:12 smithi046 ceph-mon[30136]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:12.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:12 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.46:6836/3965071509,v1:172.21.15.46:6837/3965071509] up:boot 2024-05-09T04:47:12.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:12 smithi046 ceph-mon[30136]: fsmap 4 up:standby 2024-05-09T04:47:12.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:12 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T04:47:13.882 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 04:47:13 smithi072 podman[63110]: 2024-05-09 04:47:13.80036331 +0000 UTC m=+0.142048011 container create 169e3dba0d2916e5797d495978f5001611bb599bbb3e201155c2272ef7300ee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-e, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-09T04:47:13.905 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:47:14.170 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:47:14.326 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 04:47:14 smithi072 podman[63110]: 2024-05-09 04:47:14.055071363 +0000 UTC m=+0.396756073 container init 169e3dba0d2916e5797d495978f5001611bb599bbb3e201155c2272ef7300ee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-e, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-09T04:47:14.327 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 04:47:14 smithi072 podman[63110]: 2024-05-09 04:47:14.05758182 +0000 UTC m=+0.399266515 container start 169e3dba0d2916e5797d495978f5001611bb599bbb3e201155c2272ef7300ee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-09T04:47:14.327 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 04:47:14 smithi072 ceph-mds[63130]: -- 172.21.15.72:0/1484730984 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5642d3203a40 con 0x5642d324b600 2024-05-09T04:47:14.327 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 04:47:14 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-e[63126]: starting mds.e at 2024-05-09T04:47:14.327 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 04:47:14 smithi072 bash[63110]: 169e3dba0d2916e5797d495978f5001611bb599bbb3e201155c2272ef7300ee7 2024-05-09T04:47:14.327 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 04:47:14 smithi072 systemd[1]: Started Ceph mds.e for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:47:14.578 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:14 smithi046 ceph-mon[30136]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:14.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:14 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:14.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:14 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:14.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:14 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:14.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:14 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:14.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:14 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:14.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:14 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:14.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:14 smithi072 ceph-mon[34088]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:14.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:14 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:14.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:14 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:14.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:14 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:14.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:14 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:14.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:14 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:14.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:14 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:14.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:14 smithi110 ceph-mon[34904]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:14.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:14 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:14.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:14 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:14.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:14 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:14.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:14 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:14.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:14 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:14.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:14 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:15.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:15 smithi110 ceph-mon[34904]: Deploying daemon mds.f on smithi110 2024-05-09T04:47:15.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:15 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.72:6834/4256517546,v1:172.21.15.72:6835/4256517546] up:boot 2024-05-09T04:47:15.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:15 smithi110 ceph-mon[34904]: fsmap 5 up:standby 2024-05-09T04:47:15.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:15 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T04:47:15.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:15 smithi072 ceph-mon[34088]: Deploying daemon mds.f on smithi110 2024-05-09T04:47:15.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:15 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.72:6834/4256517546,v1:172.21.15.72:6835/4256517546] up:boot 2024-05-09T04:47:15.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:15 smithi072 ceph-mon[34088]: fsmap 5 up:standby 2024-05-09T04:47:15.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:15 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T04:47:15.684 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:15 smithi046 ceph-mon[30136]: Deploying daemon mds.f on smithi110 2024-05-09T04:47:15.684 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:15 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.72:6834/4256517546,v1:172.21.15.72:6835/4256517546] up:boot 2024-05-09T04:47:15.684 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:15 smithi046 ceph-mon[30136]: fsmap 5 up:standby 2024-05-09T04:47:15.684 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:15 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T04:47:15.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.940+0000 7fe191403700 1 -- 172.21.15.46:0/441800787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c105c50 msgr2=0x7fe18c10e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:15.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.940+0000 7fe191403700 1 --2- 172.21.15.46:0/441800787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c105c50 0x7fe18c10e1a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fe17c009990 tx=0x7fe17c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:47:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.940+0000 7fe191403700 1 -- 172.21.15.46:0/441800787 shutdown_connections 2024-05-09T04:47:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.940+0000 7fe191403700 1 --2- 172.21.15.46:0/441800787 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe18c10e770 0x7fe18c110c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.940+0000 7fe191403700 1 --2- 172.21.15.46:0/441800787 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe18c105300 0x7fe18c105710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.940+0000 7fe191403700 1 --2- 172.21.15.46:0/441800787 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c105c50 0x7fe18c10e1a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.940+0000 7fe191403700 1 -- 172.21.15.46:0/441800787 >> 172.21.15.46:0/441800787 conn(0x7fe18c078c00 msgr2=0x7fe18c079040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.940+0000 7fe191403700 1 -- 172.21.15.46:0/441800787 shutdown_connections 2024-05-09T04:47:15.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.940+0000 7fe191403700 1 -- 172.21.15.46:0/441800787 wait complete. 2024-05-09T04:47:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.941+0000 7fe191403700 1 Processor -- start 2024-05-09T04:47:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.941+0000 7fe191403700 1 -- start start 2024-05-09T04:47:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe191403700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe18c105300 0x7fe18c1a0850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe191403700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c105c50 0x7fe18c1a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe191403700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe18c10e770 0x7fe18c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe191403700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c1142d0 con 0x7fe18c105c50 2024-05-09T04:47:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe191403700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c114150 con 0x7fe18c10e770 2024-05-09T04:47:15.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe191403700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe18c114450 con 0x7fe18c105300 2024-05-09T04:47:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe18a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c105c50 0x7fe18c1a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe18a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c105c50 0x7fe18c1a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40324/0 (socket says 172.21.15.46:40324) 2024-05-09T04:47:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe18a7fc700 1 -- 172.21.15.46:0/3641049300 learned_addr learned my addr 172.21.15.46:0/3641049300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:47:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe18b7fe700 1 --2- 172.21.15.46:0/3641049300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe18c10e770 0x7fe18c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe18affd700 1 --2- 172.21.15.46:0/3641049300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe18c105300 0x7fe18c1a0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe18a7fc700 1 -- 172.21.15.46:0/3641049300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe18c105300 msgr2=0x7fe18c1a0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe18a7fc700 1 --2- 172.21.15.46:0/3641049300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe18c105300 0x7fe18c1a0850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe18a7fc700 1 -- 172.21.15.46:0/3641049300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe18c10e770 msgr2=0x7fe18c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe18a7fc700 1 --2- 172.21.15.46:0/3641049300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe18c10e770 0x7fe18c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe18a7fc700 1 -- 172.21.15.46:0/3641049300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe18c19b120 con 0x7fe18c105c50 2024-05-09T04:47:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.942+0000 7fe18b7fe700 1 --2- 172.21.15.46:0/3641049300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe18c10e770 0x7fe18c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:47:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.943+0000 7fe18a7fc700 1 --2- 172.21.15.46:0/3641049300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c105c50 0x7fe18c1a0d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe17c000c00 tx=0x7fe17c009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.943+0000 7fe173fff700 1 -- 172.21.15.46:0/3641049300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c027070 con 0x7fe18c105c50 2024-05-09T04:47:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.943+0000 7fe191403700 1 -- 172.21.15.46:0/3641049300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe18c19b3b0 con 0x7fe18c105c50 2024-05-09T04:47:15.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.943+0000 7fe173fff700 1 -- 172.21.15.46:0/3641049300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe17c022470 con 0x7fe18c105c50 2024-05-09T04:47:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.943+0000 7fe173fff700 1 -- 172.21.15.46:0/3641049300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe17c017650 con 0x7fe18c105c50 2024-05-09T04:47:15.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.943+0000 7fe191403700 1 -- 172.21.15.46:0/3641049300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe18c19b890 con 0x7fe18c105c50 2024-05-09T04:47:15.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.944+0000 7fe191403700 1 -- 172.21.15.46:0/3641049300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe18c0512c0 con 0x7fe18c105c50 2024-05-09T04:47:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.946+0000 7fe173fff700 1 -- 172.21.15.46:0/3641049300 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fe17c02a070 con 0x7fe18c105c50 2024-05-09T04:47:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.946+0000 7fe173fff700 1 --2- 172.21.15.46:0/3641049300 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe1740b2880 0x7fe1740b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.947+0000 7fe18affd700 1 --2- 172.21.15.46:0/3641049300 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe1740b2880 0x7fe1740b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.947+0000 7fe173fff700 1 -- 172.21.15.46:0/3641049300 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fe17c0e2b10 con 0x7fe18c105c50 2024-05-09T04:47:15.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.947+0000 7fe18affd700 1 --2- 172.21.15.46:0/3641049300 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe1740b2880 0x7fe1740b4d90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe180005950 tx=0x7fe18000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:15.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:15.951+0000 7fe173fff700 1 -- 172.21.15.46:0/3641049300 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe17c013070 con 0x7fe18c105c50 2024-05-09T04:47:16.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:16.213+0000 7fe191403700 1 -- 172.21.15.46:0/3641049300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe18c02e5f0 con 0x7fe18c105c50 2024-05-09T04:47:16.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:16.214+0000 7fe173fff700 1 -- 172.21.15.46:0/3641049300 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7fe17c0aad60 con 0x7fe18c105c50 2024-05-09T04:47:16.214 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:47:16.214 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-09T04:47:16.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:16.216+0000 7fe191403700 1 -- 172.21.15.46:0/3641049300 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe1740b2880 msgr2=0x7fe1740b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:16.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:16.216+0000 7fe191403700 1 --2- 172.21.15.46:0/3641049300 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe1740b2880 0x7fe1740b4d90 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fe180005950 tx=0x7fe18000a300 comp rx=0 tx=0).stop 2024-05-09T04:47:16.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:16.216+0000 7fe191403700 1 -- 172.21.15.46:0/3641049300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c105c50 msgr2=0x7fe18c1a0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:16.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:16.216+0000 7fe191403700 1 --2- 172.21.15.46:0/3641049300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c105c50 0x7fe18c1a0d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe17c000c00 tx=0x7fe17c009580 comp rx=0 tx=0).stop 2024-05-09T04:47:16.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:16.216+0000 7fe191403700 1 -- 172.21.15.46:0/3641049300 shutdown_connections 2024-05-09T04:47:16.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:16.216+0000 7fe191403700 1 --2- 172.21.15.46:0/3641049300 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe1740b2880 0x7fe1740b4d90 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:16.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:16.216+0000 7fe191403700 1 --2- 172.21.15.46:0/3641049300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe18c10e770 0x7fe18c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:16.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:16.216+0000 7fe191403700 1 --2- 172.21.15.46:0/3641049300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe18c105300 0x7fe18c1a0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:16.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:16.216+0000 7fe191403700 1 --2- 172.21.15.46:0/3641049300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe18c105c50 0x7fe18c1a0d90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:16.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:16.216+0000 7fe191403700 1 -- 172.21.15.46:0/3641049300 >> 172.21.15.46:0/3641049300 conn(0x7fe18c078c00 msgr2=0x7fe18c0fff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:16.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:16.217+0000 7fe191403700 1 -- 172.21.15.46:0/3641049300 shutdown_connections 2024-05-09T04:47:16.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:16.217+0000 7fe191403700 1 -- 172.21.15.46:0/3641049300 wait complete. 2024-05-09T04:47:16.218 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 6 2024-05-09T04:47:16.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:16 smithi046 ceph-mon[30136]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:16.542 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:16 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3641049300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:16.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:16 smithi110 ceph-mon[34904]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:16.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:16 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3641049300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:16.557 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 04:47:16 smithi110 systemd[1]: Starting Ceph mds.f for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:47:16.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:16 smithi072 ceph-mon[34088]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:16.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:16 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3641049300' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:16.900 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 04:47:16 smithi110 podman[64893]: 2024-05-09 04:47:16.536560262 +0000 UTC m=+0.018225237 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:47:16.900 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 04:47:16 smithi110 podman[64893]: 2024-05-09 04:47:16.656650327 +0000 UTC m=+0.138315321 container create 6c87c4863332aacc042950c0fcd796fad9c54d1c1bcc878a3cf9c8d816ac7d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-09T04:47:16.900 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 04:47:16 smithi110 podman[64893]: 2024-05-09 04:47:16.900727633 +0000 UTC m=+0.382392585 container init 6c87c4863332aacc042950c0fcd796fad9c54d1c1bcc878a3cf9c8d816ac7d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-f, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T04:47:17.307 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 04:47:16 smithi110 podman[64893]: 2024-05-09 04:47:16.903396349 +0000 UTC m=+0.385061301 container start 6c87c4863332aacc042950c0fcd796fad9c54d1c1bcc878a3cf9c8d816ac7d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-09T04:47:17.307 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 04:47:16 smithi110 ceph-mds[64912]: -- 172.21.15.110:0/2185264641 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56372c02fa40 con 0x56372c077180 2024-05-09T04:47:17.307 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 04:47:16 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-f[64908]: starting mds.f at 2024-05-09T04:47:17.307 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 04:47:17 smithi110 bash[64893]: 6c87c4863332aacc042950c0fcd796fad9c54d1c1bcc878a3cf9c8d816ac7d52 2024-05-09T04:47:17.307 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 04:47:17 smithi110 systemd[1]: Started Ceph mds.f for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:47:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:18 smithi110 ceph-mon[34904]: Deploying daemon mds.c on smithi110 2024-05-09T04:47:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:18 smithi110 ceph-mon[34904]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:18 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.110:6834/2299300184,v1:172.21.15.110:6835/2299300184] up:boot 2024-05-09T04:47:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:18 smithi110 ceph-mon[34904]: fsmap 6 up:standby 2024-05-09T04:47:18.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:18 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T04:47:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:18 smithi046 ceph-mon[30136]: Deploying daemon mds.c on smithi110 2024-05-09T04:47:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:18 smithi046 ceph-mon[30136]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:18 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.110:6834/2299300184,v1:172.21.15.110:6835/2299300184] up:boot 2024-05-09T04:47:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:18 smithi046 ceph-mon[30136]: fsmap 6 up:standby 2024-05-09T04:47:18.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:18 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T04:47:18.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:18.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:18.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:18.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:18.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:18.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:18.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:18 smithi072 ceph-mon[34088]: Deploying daemon mds.c on smithi110 2024-05-09T04:47:18.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:18 smithi072 ceph-mon[34088]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:18.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:18 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.110:6834/2299300184,v1:172.21.15.110:6835/2299300184] up:boot 2024-05-09T04:47:18.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:18 smithi072 ceph-mon[34088]: fsmap 6 up:standby 2024-05-09T04:47:18.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:18 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T04:47:18.879 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:47:19.143 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:47:19.463 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 04:47:19 smithi110 systemd[1]: Starting Ceph mds.c for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:47:19.738 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 04:47:19 smithi110 podman[65203]: 2024-05-09 04:47:19.367152104 +0000 UTC m=+0.017897753 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:47:19.738 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 04:47:19 smithi110 podman[65203]: 2024-05-09 04:47:19.477635924 +0000 UTC m=+0.128381566 container create f31847814bf1ad0722d8b0e6b3f95f313e6ad1f0825926285999d7eeb332b6f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-c, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-09T04:47:19.738 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 04:47:19 smithi110 podman[65203]: 2024-05-09 04:47:19.739114752 +0000 UTC m=+0.389860395 container init f31847814bf1ad0722d8b0e6b3f95f313e6ad1f0825926285999d7eeb332b6f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-c, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-05-09T04:47:20.006 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 04:47:19 smithi110 podman[65203]: 2024-05-09 04:47:19.741670614 +0000 UTC m=+0.392416265 container start f31847814bf1ad0722d8b0e6b3f95f313e6ad1f0825926285999d7eeb332b6f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T04:47:20.006 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 04:47:19 smithi110 ceph-mds[65222]: -- 172.21.15.110:0/1659689117 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564995f27a40 con 0x564995f6f600 2024-05-09T04:47:20.006 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 04:47:19 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-c[65218]: starting mds.c at 2024-05-09T04:47:20.006 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 04:47:19 smithi110 bash[65203]: f31847814bf1ad0722d8b0e6b3f95f313e6ad1f0825926285999d7eeb332b6f5 2024-05-09T04:47:20.006 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 04:47:19 smithi110 systemd[1]: Started Ceph mds.c for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:47:20.560 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:20 smithi046 ceph-mon[30136]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:20.560 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:20 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:20.560 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:20 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:20.560 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:20 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:20.560 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:20 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:20.560 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:20 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:20.560 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:20 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:20.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:20 smithi072 ceph-mon[34088]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:20.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:20 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:20.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:20 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:20.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:20 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:20.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:20 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:20.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:20 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:20.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:20 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:20.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.804+0000 7f7052900700 1 -- 172.21.15.46:0/2389076907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704c1034c0 msgr2=0x7f704c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.804+0000 7f7052900700 1 --2- 172.21.15.46:0/2389076907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704c1034c0 0x7f704c1038d0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f703c0099c0 tx=0x7f703c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:47:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.804+0000 7f7052900700 1 -- 172.21.15.46:0/2389076907 shutdown_connections 2024-05-09T04:47:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.804+0000 7f7052900700 1 --2- 172.21.15.46:0/2389076907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f704c108740 0x7f704c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.804+0000 7f7052900700 1 --2- 172.21.15.46:0/2389076907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704c1034c0 0x7f704c1038d0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.804+0000 7f7052900700 1 --2- 172.21.15.46:0/2389076907 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f704c107df0 0x7f704c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.804+0000 7f7052900700 1 -- 172.21.15.46:0/2389076907 >> 172.21.15.46:0/2389076907 conn(0x7f704c0fedf0 msgr2=0x7f704c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:20.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.804+0000 7f7052900700 1 -- 172.21.15.46:0/2389076907 shutdown_connections 2024-05-09T04:47:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.804+0000 7f7052900700 1 -- 172.21.15.46:0/2389076907 wait complete. 2024-05-09T04:47:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.805+0000 7f7052900700 1 Processor -- start 2024-05-09T04:47:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.805+0000 7f7052900700 1 -- start start 2024-05-09T04:47:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.805+0000 7f7052900700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f704c1034c0 0x7f704c19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.805+0000 7f7052900700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704c107df0 0x7f704c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:20.810 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.805+0000 7f7052900700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f704c108740 0x7f704c19f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.805+0000 7f7052900700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f704c113370 con 0x7f704c107df0 2024-05-09T04:47:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.805+0000 7f7052900700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f704c1131f0 con 0x7f704c1034c0 2024-05-09T04:47:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.805+0000 7f7052900700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f704c1134f0 con 0x7f704c108740 2024-05-09T04:47:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.805+0000 7f7043fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704c107df0 0x7f704c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.805+0000 7f7043fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704c107df0 0x7f704c19b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40346/0 (socket says 172.21.15.46:40346) 2024-05-09T04:47:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.805+0000 7f7043fff700 1 -- 172.21.15.46:0/3331257791 learned_addr learned my addr 172.21.15.46:0/3331257791 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:47:20.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.805+0000 7f704bfff700 1 --2- 172.21.15.46:0/3331257791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f704c1034c0 0x7f704c19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.805+0000 7f7050e9d700 1 --2- 172.21.15.46:0/3331257791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f704c108740 0x7f704c19f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.806+0000 7f7043fff700 1 -- 172.21.15.46:0/3331257791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f704c108740 msgr2=0x7f704c19f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.806+0000 7f7043fff700 1 --2- 172.21.15.46:0/3331257791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f704c108740 0x7f704c19f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.806+0000 7f7043fff700 1 -- 172.21.15.46:0/3331257791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f704c1034c0 msgr2=0x7f704c19ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.806+0000 7f7043fff700 1 --2- 172.21.15.46:0/3331257791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f704c1034c0 0x7f704c19ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.806+0000 7f7043fff700 1 -- 172.21.15.46:0/3331257791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f704c19fcf0 con 0x7f704c107df0 2024-05-09T04:47:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.806+0000 7f7043fff700 1 --2- 172.21.15.46:0/3331257791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704c107df0 0x7f704c19b410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f703400d980 tx=0x7f703400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.806+0000 7f7049ffb700 1 -- 172.21.15.46:0/3331257791 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7034017070 con 0x7f704c107df0 2024-05-09T04:47:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.806+0000 7f7049ffb700 1 -- 172.21.15.46:0/3331257791 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f703400fd40 con 0x7f704c107df0 2024-05-09T04:47:20.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.806+0000 7f7052900700 1 -- 172.21.15.46:0/3331257791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f704c19ffe0 con 0x7f704c107df0 2024-05-09T04:47:20.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.806+0000 7f7049ffb700 1 -- 172.21.15.46:0/3331257791 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f703400e3b0 con 0x7f704c107df0 2024-05-09T04:47:20.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.806+0000 7f7052900700 1 -- 172.21.15.46:0/3331257791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f704c1a7900 con 0x7f704c107df0 2024-05-09T04:47:20.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.807+0000 7f7052900700 1 -- 172.21.15.46:0/3331257791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f704c0512c0 con 0x7f704c107df0 2024-05-09T04:47:20.814 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:20 smithi110 ceph-mon[34904]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:20.814 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:20 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:20.814 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:20 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:20.814 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:20 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:20.814 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:20 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:20.815 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:20 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:20.815 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:20 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.809+0000 7f7049ffb700 1 -- 172.21.15.46:0/3331257791 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7034010610 con 0x7f704c107df0 2024-05-09T04:47:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.810+0000 7f7049ffb700 1 --2- 172.21.15.46:0/3331257791 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f70380b2880 0x7f70380b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.810+0000 7f704bfff700 1 --2- 172.21.15.46:0/3331257791 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f70380b2880 0x7f70380b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.810+0000 7f7049ffb700 1 -- 172.21.15.46:0/3331257791 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f70340e1870 con 0x7f704c107df0 2024-05-09T04:47:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.811+0000 7f704bfff700 1 --2- 172.21.15.46:0/3331257791 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f70380b2880 0x7f70380b4d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f703c000c00 tx=0x7f703c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:20.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:20.814+0000 7f7049ffb700 1 -- 172.21.15.46:0/3331257791 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f70340a9990 con 0x7f704c107df0 2024-05-09T04:47:21.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:21.081+0000 7f7052900700 1 -- 172.21.15.46:0/3331257791 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f704c1a7ad0 con 0x7f704c107df0 2024-05-09T04:47:21.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:21.082+0000 7f7049ffb700 1 -- 172.21.15.46:0/3331257791 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7f70340a9290 con 0x7f704c107df0 2024-05-09T04:47:21.083 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:47:21.083 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24433,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-09T04:47:21.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:21.084+0000 7f7052900700 1 -- 172.21.15.46:0/3331257791 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f70380b2880 msgr2=0x7f70380b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:21.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:21.084+0000 7f7052900700 1 --2- 172.21.15.46:0/3331257791 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f70380b2880 0x7f70380b4d90 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f703c000c00 tx=0x7f703c01a040 comp rx=0 tx=0).stop 2024-05-09T04:47:21.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:21.084+0000 7f7052900700 1 -- 172.21.15.46:0/3331257791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704c107df0 msgr2=0x7f704c19b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:21.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:21.084+0000 7f7052900700 1 --2- 172.21.15.46:0/3331257791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704c107df0 0x7f704c19b410 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f703400d980 tx=0x7f703400dc90 comp rx=0 tx=0).stop 2024-05-09T04:47:21.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:21.084+0000 7f7052900700 1 -- 172.21.15.46:0/3331257791 shutdown_connections 2024-05-09T04:47:21.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:21.084+0000 7f7052900700 1 --2- 172.21.15.46:0/3331257791 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f70380b2880 0x7f70380b4d90 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:21.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:21.084+0000 7f7052900700 1 --2- 172.21.15.46:0/3331257791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f704c108740 0x7f704c19f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:21.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:21.084+0000 7f7052900700 1 --2- 172.21.15.46:0/3331257791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f704c1034c0 0x7f704c19ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:21.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:21.084+0000 7f7052900700 1 --2- 172.21.15.46:0/3331257791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f704c107df0 0x7f704c19b410 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:21.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:21.084+0000 7f7052900700 1 -- 172.21.15.46:0/3331257791 >> 172.21.15.46:0/3331257791 conn(0x7f704c0fedf0 msgr2=0x7f704c1096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:21.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:21.084+0000 7f7052900700 1 -- 172.21.15.46:0/3331257791 shutdown_connections 2024-05-09T04:47:21.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:21.084+0000 7f7052900700 1 -- 172.21.15.46:0/3331257791 wait complete. 2024-05-09T04:47:21.086 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 8 2024-05-09T04:47:21.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:21 smithi072 ceph-mon[34088]: Deploying daemon mds.h on smithi072 2024-05-09T04:47:21.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:21 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.110:6836/2612088705,v1:172.21.15.110:6837/2612088705] up:boot 2024-05-09T04:47:21.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:21 smithi072 ceph-mon[34088]: fsmap 7 up:standby 2024-05-09T04:47:21.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:21 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T04:47:21.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:21 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3331257791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:21.738 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:21 smithi046 ceph-mon[30136]: Deploying daemon mds.h on smithi072 2024-05-09T04:47:21.738 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:21 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.110:6836/2612088705,v1:172.21.15.110:6837/2612088705] up:boot 2024-05-09T04:47:21.738 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:21 smithi046 ceph-mon[30136]: fsmap 7 up:standby 2024-05-09T04:47:21.738 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:21 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T04:47:21.738 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:21 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3331257791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:21.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:21 smithi110 ceph-mon[34904]: Deploying daemon mds.h on smithi072 2024-05-09T04:47:21.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:21 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.110:6836/2612088705,v1:172.21.15.110:6837/2612088705] up:boot 2024-05-09T04:47:21.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:21 smithi110 ceph-mon[34904]: fsmap 7 up:standby 2024-05-09T04:47:21.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:21 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T04:47:21.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:21 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3331257791' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:22.351 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 04:47:22 smithi072 systemd[1]: Starting Ceph mds.h for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:47:22.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:22 smithi072 ceph-mon[34088]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:22.630 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 04:47:22 smithi072 podman[63429]: 2024-05-09 04:47:22.298373527 +0000 UTC m=+0.018000520 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:47:22.630 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 04:47:22 smithi072 podman[63429]: 2024-05-09 04:47:22.416091764 +0000 UTC m=+0.135718745 container create 9e50b6586128f8ca3a64904545d6a5ae8301a395b8864801d4400e224dfe17ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T04:47:22.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:22 smithi110 ceph-mon[34904]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:22.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:22 smithi046 ceph-mon[30136]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:23.130 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 04:47:22 smithi072 podman[63429]: 2024-05-09 04:47:22.689420837 +0000 UTC m=+0.409047818 container init 9e50b6586128f8ca3a64904545d6a5ae8301a395b8864801d4400e224dfe17ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-h, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-09T04:47:23.130 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 04:47:22 smithi072 podman[63429]: 2024-05-09 04:47:22.692237236 +0000 UTC m=+0.411864226 container start 9e50b6586128f8ca3a64904545d6a5ae8301a395b8864801d4400e224dfe17ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-h, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-09T04:47:23.130 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 04:47:22 smithi072 ceph-mds[63447]: -- 172.21.15.72:0/1055968225 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5586cd389a40 con 0x5586cd3d1180 2024-05-09T04:47:23.130 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 04:47:22 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-h[63443]: starting mds.h at 2024-05-09T04:47:23.130 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 04:47:22 smithi072 bash[63429]: 9e50b6586128f8ca3a64904545d6a5ae8301a395b8864801d4400e224dfe17ff 2024-05-09T04:47:23.130 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 04:47:22 smithi072 systemd[1]: Started Ceph mds.h for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:47:23.772 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:47:24.038 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:47:24.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:23 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:24.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:23 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:24.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:23 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:24.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:23 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:24.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:23 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:24.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:23 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:24.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:23 smithi046 ceph-mon[30136]: Deploying daemon mds.i on smithi110 2024-05-09T04:47:24.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:23 smithi046 ceph-mon[30136]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:24.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:23 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] up:boot 2024-05-09T04:47:24.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:23 smithi046 ceph-mon[30136]: fsmap 8 up:standby 2024-05-09T04:47:24.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:23 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T04:47:24.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:23 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:24.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:23 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:24.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:23 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:24.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:23 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:24.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:23 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:24.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:23 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:24.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:23 smithi072 ceph-mon[34088]: Deploying daemon mds.i on smithi110 2024-05-09T04:47:24.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:23 smithi072 ceph-mon[34088]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:24.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:23 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] up:boot 2024-05-09T04:47:24.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:23 smithi072 ceph-mon[34088]: fsmap 8 up:standby 2024-05-09T04:47:24.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:23 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T04:47:24.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:23 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:24.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:23 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:24.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:23 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:24.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:23 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:24.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:23 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:24.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:23 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:24.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:23 smithi110 ceph-mon[34904]: Deploying daemon mds.i on smithi110 2024-05-09T04:47:24.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:23 smithi110 ceph-mon[34904]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:24.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:23 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] up:boot 2024-05-09T04:47:24.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:23 smithi110 ceph-mon[34904]: fsmap 8 up:standby 2024-05-09T04:47:24.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:23 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T04:47:25.307 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 04:47:25 smithi110 systemd[1]: Starting Ceph mds.i for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:47:25.626 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 04:47:25 smithi110 podman[65510]: 2024-05-09 04:47:25.238782525 +0000 UTC m=+0.018158389 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:47:25.627 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 04:47:25 smithi110 podman[65510]: 2024-05-09 04:47:25.358086934 +0000 UTC m=+0.137462832 container create d7005ada0039d91e4762487439d792142af3fdb3b24ae6e8b499003bc860375d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-i, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS) 2024-05-09T04:47:25.627 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 04:47:25 smithi110 podman[65510]: 2024-05-09 04:47:25.627546927 +0000 UTC m=+0.406922775 container init d7005ada0039d91e4762487439d792142af3fdb3b24ae6e8b499003bc860375d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-i, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507) 2024-05-09T04:47:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.726+0000 7fbb57ce0700 1 -- 172.21.15.46:0/2732097394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50108740 msgr2=0x7fbb501100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:25.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.726+0000 7fbb57ce0700 1 --2- 172.21.15.46:0/2732097394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50108740 0x7fbb501100a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbb480099c0 tx=0x7fbb48009cd0 comp rx=0 tx=0).stop 2024-05-09T04:47:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.727+0000 7fbb57ce0700 1 -- 172.21.15.46:0/2732097394 shutdown_connections 2024-05-09T04:47:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.727+0000 7fbb57ce0700 1 --2- 172.21.15.46:0/2732097394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb50108740 0x7fbb501100a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.727+0000 7fbb57ce0700 1 --2- 172.21.15.46:0/2732097394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb501034c0 0x7fbb501038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.727+0000 7fbb57ce0700 1 --2- 172.21.15.46:0/2732097394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb50107df0 0x7fbb50108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.727+0000 7fbb57ce0700 1 -- 172.21.15.46:0/2732097394 >> 172.21.15.46:0/2732097394 conn(0x7fbb500fedf0 msgr2=0x7fbb50101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.727+0000 7fbb57ce0700 1 -- 172.21.15.46:0/2732097394 shutdown_connections 2024-05-09T04:47:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.727+0000 7fbb57ce0700 1 -- 172.21.15.46:0/2732097394 wait complete. 2024-05-09T04:47:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.727+0000 7fbb57ce0700 1 Processor -- start 2024-05-09T04:47:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb57ce0700 1 -- start start 2024-05-09T04:47:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb57ce0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb501034c0 0x7fbb5019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb57ce0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb50107df0 0x7fbb5019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb57ce0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb50108740 0x7fbb5019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:25.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb57ce0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbb50113370 con 0x7fbb501034c0 2024-05-09T04:47:25.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb57ce0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbb501131f0 con 0x7fbb50107df0 2024-05-09T04:47:25.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb57ce0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbb501134f0 con 0x7fbb50108740 2024-05-09T04:47:25.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb55a7c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb501034c0 0x7fbb5019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:25.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb55a7c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb501034c0 0x7fbb5019adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47044/0 (socket says 172.21.15.46:47044) 2024-05-09T04:47:25.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb55a7c700 1 -- 172.21.15.46:0/322917115 learned_addr learned my addr 172.21.15.46:0/322917115 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:47:25.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb5627d700 1 --2- 172.21.15.46:0/322917115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb50108740 0x7fbb5019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:25.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb5527b700 1 --2- 172.21.15.46:0/322917115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb50107df0 0x7fbb5019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:25.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb55a7c700 1 -- 172.21.15.46:0/322917115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb50108740 msgr2=0x7fbb5019f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:25.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb55a7c700 1 --2- 172.21.15.46:0/322917115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb50108740 0x7fbb5019f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:25.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb55a7c700 1 -- 172.21.15.46:0/322917115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb50107df0 msgr2=0x7fbb5019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:25.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb55a7c700 1 --2- 172.21.15.46:0/322917115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb50107df0 0x7fbb5019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:25.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb55a7c700 1 -- 172.21.15.46:0/322917115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb5019fbd0 con 0x7fbb501034c0 2024-05-09T04:47:25.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.728+0000 7fbb55a7c700 1 --2- 172.21.15.46:0/322917115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb501034c0 0x7fbb5019adb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00ef10 tx=0x7fbb4c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.729+0000 7fbb46ffd700 1 -- 172.21.15.46:0/322917115 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c00c9c0 con 0x7fbb501034c0 2024-05-09T04:47:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.729+0000 7fbb46ffd700 1 -- 172.21.15.46:0/322917115 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb4c019070 con 0x7fbb501034c0 2024-05-09T04:47:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.729+0000 7fbb57ce0700 1 -- 172.21.15.46:0/322917115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb5019fe60 con 0x7fbb501034c0 2024-05-09T04:47:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.729+0000 7fbb46ffd700 1 -- 172.21.15.46:0/322917115 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb4c014480 con 0x7fbb501034c0 2024-05-09T04:47:25.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.729+0000 7fbb57ce0700 1 -- 172.21.15.46:0/322917115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb501a03a0 con 0x7fbb501034c0 2024-05-09T04:47:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.730+0000 7fbb57ce0700 1 -- 172.21.15.46:0/322917115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb500512c0 con 0x7fbb501034c0 2024-05-09T04:47:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.731+0000 7fbb46ffd700 1 -- 172.21.15.46:0/322917115 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbb4c012050 con 0x7fbb501034c0 2024-05-09T04:47:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.732+0000 7fbb46ffd700 1 --2- 172.21.15.46:0/322917115 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbb3c0b2880 0x7fbb3c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.732+0000 7fbb5527b700 1 --2- 172.21.15.46:0/322917115 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbb3c0b2880 0x7fbb3c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.732+0000 7fbb46ffd700 1 -- 172.21.15.46:0/322917115 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fbb4c0e2470 con 0x7fbb501034c0 2024-05-09T04:47:25.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.733+0000 7fbb5527b700 1 --2- 172.21.15.46:0/322917115 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbb3c0b2880 0x7fbb3c0b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbb50102810 tx=0x7fbb4000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:25.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:25.736+0000 7fbb46ffd700 1 -- 172.21.15.46:0/322917115 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbb4c010040 con 0x7fbb501034c0 2024-05-09T04:47:26.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:26.010+0000 7fbb57ce0700 1 -- 172.21.15.46:0/322917115 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbb501a79b0 con 0x7fbb501034c0 2024-05-09T04:47:26.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:26.010+0000 7fbb46ffd700 1 -- 172.21.15.46:0/322917115 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7fbb4c0aa590 con 0x7fbb501034c0 2024-05-09T04:47:26.011 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:47:26.011 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24433,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-09T04:47:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:26.012+0000 7fbb57ce0700 1 -- 172.21.15.46:0/322917115 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbb3c0b2880 msgr2=0x7fbb3c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:26.012+0000 7fbb57ce0700 1 --2- 172.21.15.46:0/322917115 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbb3c0b2880 0x7fbb3c0b4d90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fbb50102810 tx=0x7fbb4000a380 comp rx=0 tx=0).stop 2024-05-09T04:47:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:26.012+0000 7fbb57ce0700 1 -- 172.21.15.46:0/322917115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb501034c0 msgr2=0x7fbb5019adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:26.012+0000 7fbb57ce0700 1 --2- 172.21.15.46:0/322917115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb501034c0 0x7fbb5019adb0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbb4c00ef10 tx=0x7fbb4c00c3a0 comp rx=0 tx=0).stop 2024-05-09T04:47:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:26.013+0000 7fbb57ce0700 1 -- 172.21.15.46:0/322917115 shutdown_connections 2024-05-09T04:47:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:26.013+0000 7fbb57ce0700 1 --2- 172.21.15.46:0/322917115 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbb3c0b2880 0x7fbb3c0b4d90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:26.013+0000 7fbb57ce0700 1 --2- 172.21.15.46:0/322917115 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbb50108740 0x7fbb5019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:26.013+0000 7fbb57ce0700 1 --2- 172.21.15.46:0/322917115 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbb501034c0 0x7fbb5019adb0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:26.013+0000 7fbb57ce0700 1 --2- 172.21.15.46:0/322917115 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbb50107df0 0x7fbb5019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:26.013+0000 7fbb57ce0700 1 -- 172.21.15.46:0/322917115 >> 172.21.15.46:0/322917115 conn(0x7fbb500fedf0 msgr2=0x7fbb50107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:26.013+0000 7fbb57ce0700 1 -- 172.21.15.46:0/322917115 shutdown_connections 2024-05-09T04:47:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:26.013+0000 7fbb57ce0700 1 -- 172.21.15.46:0/322917115 wait complete. 2024-05-09T04:47:26.015 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 9 2024-05-09T04:47:26.057 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 04:47:25 smithi110 podman[65510]: 2024-05-09 04:47:25.630245802 +0000 UTC m=+0.409621658 container start d7005ada0039d91e4762487439d792142af3fdb3b24ae6e8b499003bc860375d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-09T04:47:26.057 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 04:47:25 smithi110 ceph-mds[65530]: -- 172.21.15.110:0/1186397019 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562434aa9a40 con 0x562434af1600 2024-05-09T04:47:26.057 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 04:47:25 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-i[65526]: starting mds.i at 2024-05-09T04:47:26.057 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 04:47:25 smithi110 bash[65510]: d7005ada0039d91e4762487439d792142af3fdb3b24ae6e8b499003bc860375d 2024-05-09T04:47:26.057 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 04:47:25 smithi110 systemd[1]: Started Ceph mds.i for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:47:27.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:26 smithi046 ceph-mon[30136]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:27.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:27.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:27.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:27.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:27.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:27.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:27.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:26 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/322917115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:27.025 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:26 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] up:boot 2024-05-09T04:47:27.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:26 smithi046 ceph-mon[30136]: fsmap 9 up:standby 2024-05-09T04:47:27.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:26 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T04:47:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:26 smithi110 ceph-mon[34904]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:26 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/322917115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:26 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] up:boot 2024-05-09T04:47:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:26 smithi110 ceph-mon[34904]: fsmap 9 up:standby 2024-05-09T04:47:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:26 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T04:47:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:26 smithi072 ceph-mon[34088]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:26 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:26 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:26 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:26 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:26 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:26 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:26 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/322917115' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:26 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] up:boot 2024-05-09T04:47:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:26 smithi072 ceph-mon[34088]: fsmap 9 up:standby 2024-05-09T04:47:27.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:26 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T04:47:27.930 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:27 smithi046 ceph-mon[30136]: Deploying daemon mds.g on smithi046 2024-05-09T04:47:28.032 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:27 smithi110 ceph-mon[34904]: Deploying daemon mds.g on smithi046 2024-05-09T04:47:28.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:27 smithi072 ceph-mon[34088]: Deploying daemon mds.g on smithi046 2024-05-09T04:47:28.580 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 04:47:28 smithi046 systemd[1]: Starting Ceph mds.g for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:47:28.744 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:47:28.911 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:28 smithi046 ceph-mon[30136]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:28.912 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 04:47:28 smithi046 podman[72206]: 2024-05-09 04:47:28.534415896 +0000 UTC m=+0.019626784 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:47:28.912 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 04:47:28 smithi046 podman[72206]: 2024-05-09 04:47:28.683696317 +0000 UTC m=+0.168907188 container create 89c1baac619925ee579facfd139178ebbe669ae9827ec0d144a4a833c97a6f15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-g, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-09T04:47:28.912 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 04:47:28 smithi046 podman[72206]: 2024-05-09 04:47:28.911645965 +0000 UTC m=+0.396856838 container init 89c1baac619925ee579facfd139178ebbe669ae9827ec0d144a4a833c97a6f15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-09T04:47:29.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:28 smithi110 ceph-mon[34904]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:29.093 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:47:29.100 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:28 smithi072 ceph-mon[34088]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:29.218 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 04:47:28 smithi046 podman[72206]: 2024-05-09 04:47:28.915511376 +0000 UTC m=+0.400722249 container start 89c1baac619925ee579facfd139178ebbe669ae9827ec0d144a4a833c97a6f15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-g, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T04:47:29.218 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 04:47:28 smithi046 ceph-mds[72243]: -- 172.21.15.46:0/4116786350 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560a9d137a40 con 0x560a9d17f180 2024-05-09T04:47:29.218 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 04:47:28 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-g[72239]: starting mds.g at 2024-05-09T04:47:29.218 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 04:47:29 smithi046 bash[72206]: 89c1baac619925ee579facfd139178ebbe669ae9827ec0d144a4a833c97a6f15 2024-05-09T04:47:29.218 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 04:47:29 smithi046 systemd[1]: Started Ceph mds.g for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:47:30.181 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:30 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:30.181 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:30 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:30.181 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:30 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:30.181 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:30 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:30.181 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:30 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:30.181 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:30 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:30.181 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:30 smithi046 ceph-mon[30136]: Deploying daemon mds.a on smithi046 2024-05-09T04:47:30.181 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:30 smithi046 ceph-mon[30136]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:30.181 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:30 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.46:6838/952386829,v1:172.21.15.46:6839/952386829] up:boot 2024-05-09T04:47:30.181 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:30 smithi046 ceph-mon[30136]: fsmap 10 up:standby 2024-05-09T04:47:30.182 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:30 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T04:47:30.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:30 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:30.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:30 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:30.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:30 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:30.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:30 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:30.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:30 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:30.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:30 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:30.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:30 smithi072 ceph-mon[34088]: Deploying daemon mds.a on smithi046 2024-05-09T04:47:30.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:30 smithi072 ceph-mon[34088]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:30.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:30 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.46:6838/952386829,v1:172.21.15.46:6839/952386829] up:boot 2024-05-09T04:47:30.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:30 smithi072 ceph-mon[34088]: fsmap 10 up:standby 2024-05-09T04:47:30.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:30 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T04:47:30.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:30 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:30.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:30 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:30.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:30 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:30.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:30 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:30.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:30 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:30.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:30 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:30.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:30 smithi110 ceph-mon[34904]: Deploying daemon mds.a on smithi046 2024-05-09T04:47:30.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:30 smithi110 ceph-mon[34904]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:30.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:30 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.46:6838/952386829,v1:172.21.15.46:6839/952386829] up:boot 2024-05-09T04:47:30.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:30 smithi110 ceph-mon[34904]: fsmap 10 up:standby 2024-05-09T04:47:30.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:30 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T04:47:30.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.733+0000 7fc50ff6e700 1 -- 172.21.15.46:0/2629178893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc508107df0 msgr2=0x7fc508108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.733+0000 7fc50ff6e700 1 --2- 172.21.15.46:0/2629178893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc508107df0 0x7fc508108200 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fc4f80099c0 tx=0x7fc4f8009cd0 comp rx=0 tx=0).stop 2024-05-09T04:47:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.733+0000 7fc50ff6e700 1 -- 172.21.15.46:0/2629178893 shutdown_connections 2024-05-09T04:47:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.733+0000 7fc50ff6e700 1 --2- 172.21.15.46:0/2629178893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc508108740 0x7fc5081100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.733+0000 7fc50ff6e700 1 --2- 172.21.15.46:0/2629178893 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5081034c0 0x7fc5081038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.733+0000 7fc50ff6e700 1 --2- 172.21.15.46:0/2629178893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc508107df0 0x7fc508108200 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.733+0000 7fc50ff6e700 1 -- 172.21.15.46:0/2629178893 >> 172.21.15.46:0/2629178893 conn(0x7fc5080fedf0 msgr2=0x7fc508101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.733+0000 7fc50ff6e700 1 -- 172.21.15.46:0/2629178893 shutdown_connections 2024-05-09T04:47:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.733+0000 7fc50ff6e700 1 -- 172.21.15.46:0/2629178893 wait complete. 2024-05-09T04:47:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.734+0000 7fc50ff6e700 1 Processor -- start 2024-05-09T04:47:30.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.734+0000 7fc50ff6e700 1 -- start start 2024-05-09T04:47:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.734+0000 7fc50ff6e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5081034c0 0x7fc50819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.734+0000 7fc50ff6e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc508107df0 0x7fc50819c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.734+0000 7fc50ff6e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc508108740 0x7fc50819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:30.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.734+0000 7fc50ff6e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc508113350 con 0x7fc5081034c0 2024-05-09T04:47:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.734+0000 7fc50ff6e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081131d0 con 0x7fc508107df0 2024-05-09T04:47:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.734+0000 7fc50ff6e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5081134d0 con 0x7fc508108740 2024-05-09T04:47:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc50dd0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5081034c0 0x7fc50819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc50dd0a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5081034c0 0x7fc50819bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47082/0 (socket says 172.21.15.46:47082) 2024-05-09T04:47:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc50dd0a700 1 -- 172.21.15.46:0/2715296659 learned_addr learned my addr 172.21.15.46:0/2715296659 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:47:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc50d509700 1 --2- 172.21.15.46:0/2715296659 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc508107df0 0x7fc50819c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc50e50b700 1 --2- 172.21.15.46:0/2715296659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc508108740 0x7fc50819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc50dd0a700 1 -- 172.21.15.46:0/2715296659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc508108740 msgr2=0x7fc50819c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc50dd0a700 1 --2- 172.21.15.46:0/2715296659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc508108740 0x7fc50819c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc50dd0a700 1 -- 172.21.15.46:0/2715296659 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc508107df0 msgr2=0x7fc50819c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc50dd0a700 1 --2- 172.21.15.46:0/2715296659 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc508107df0 0x7fc50819c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:30.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc50dd0a700 1 -- 172.21.15.46:0/2715296659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5081a7000 con 0x7fc5081034c0 2024-05-09T04:47:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc50d509700 1 --2- 172.21.15.46:0/2715296659 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc508107df0 0x7fc50819c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:47:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc50dd0a700 1 --2- 172.21.15.46:0/2715296659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5081034c0 0x7fc50819bc90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc50400d950 tx=0x7fc50400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc4feffd700 1 -- 172.21.15.46:0/2715296659 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504016070 con 0x7fc5081034c0 2024-05-09T04:47:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc4feffd700 1 -- 172.21.15.46:0/2715296659 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc504010460 con 0x7fc5081034c0 2024-05-09T04:47:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc4feffd700 1 -- 172.21.15.46:0/2715296659 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc50400f620 con 0x7fc5081034c0 2024-05-09T04:47:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc50ff6e700 1 -- 172.21.15.46:0/2715296659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5081a72f0 con 0x7fc5081034c0 2024-05-09T04:47:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.735+0000 7fc50ff6e700 1 -- 172.21.15.46:0/2715296659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5081a7850 con 0x7fc5081034c0 2024-05-09T04:47:30.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.736+0000 7fc50ff6e700 1 -- 172.21.15.46:0/2715296659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5080512c0 con 0x7fc5081034c0 2024-05-09T04:47:30.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.738+0000 7fc4feffd700 1 -- 172.21.15.46:0/2715296659 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc504019070 con 0x7fc5081034c0 2024-05-09T04:47:30.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.739+0000 7fc4feffd700 1 --2- 172.21.15.46:0/2715296659 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc4f40b2900 0x7fc4f40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:30.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.739+0000 7fc50d509700 1 --2- 172.21.15.46:0/2715296659 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc4f40b2900 0x7fc4f40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:30.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.739+0000 7fc4feffd700 1 -- 172.21.15.46:0/2715296659 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fc5040e19c0 con 0x7fc5081034c0 2024-05-09T04:47:30.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.739+0000 7fc50d509700 1 --2- 172.21.15.46:0/2715296659 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc4f40b2900 0x7fc4f40b4e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8000c00 tx=0x7fc4f801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:30.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:30.742+0000 7fc4feffd700 1 -- 172.21.15.46:0/2715296659 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc5040a9ae0 con 0x7fc5081034c0 2024-05-09T04:47:31.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:31.007+0000 7fc50ff6e700 1 -- 172.21.15.46:0/2715296659 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc50802e710 con 0x7fc5081034c0 2024-05-09T04:47:31.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:31.008+0000 7fc4feffd700 1 -- 172.21.15.46:0/2715296659 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8824 (secure 0 0 0) 0x7fc5040a93e0 con 0x7fc5081034c0 2024-05-09T04:47:31.009 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:47:31.009 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24433,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-09T04:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:31.010+0000 7fc50ff6e700 1 -- 172.21.15.46:0/2715296659 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc4f40b2900 msgr2=0x7fc4f40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:31.010+0000 7fc50ff6e700 1 --2- 172.21.15.46:0/2715296659 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc4f40b2900 0x7fc4f40b4e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fc4f8000c00 tx=0x7fc4f801a040 comp rx=0 tx=0).stop 2024-05-09T04:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:31.010+0000 7fc50ff6e700 1 -- 172.21.15.46:0/2715296659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5081034c0 msgr2=0x7fc50819bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:31.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:31.010+0000 7fc50ff6e700 1 --2- 172.21.15.46:0/2715296659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5081034c0 0x7fc50819bc90 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fc50400d950 tx=0x7fc50400dd10 comp rx=0 tx=0).stop 2024-05-09T04:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:31.010+0000 7fc50ff6e700 1 -- 172.21.15.46:0/2715296659 shutdown_connections 2024-05-09T04:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:31.010+0000 7fc50ff6e700 1 --2- 172.21.15.46:0/2715296659 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc4f40b2900 0x7fc4f40b4e10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:31.010+0000 7fc50ff6e700 1 --2- 172.21.15.46:0/2715296659 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc508108740 0x7fc50819c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:31.010+0000 7fc50ff6e700 1 --2- 172.21.15.46:0/2715296659 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5081034c0 0x7fc50819bc90 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:31.010+0000 7fc50ff6e700 1 --2- 172.21.15.46:0/2715296659 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc508107df0 0x7fc50819c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:31.010+0000 7fc50ff6e700 1 -- 172.21.15.46:0/2715296659 >> 172.21.15.46:0/2715296659 conn(0x7fc5080fedf0 msgr2=0x7fc5081096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:31.011+0000 7fc50ff6e700 1 -- 172.21.15.46:0/2715296659 shutdown_connections 2024-05-09T04:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:31.011+0000 7fc50ff6e700 1 -- 172.21.15.46:0/2715296659 wait complete. 2024-05-09T04:47:31.012 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 11 2024-05-09T04:47:31.157 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:31 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2715296659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:31.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:31 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2715296659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:31.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:31 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2715296659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:32.280 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:32 smithi046 ceph-mon[30136]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:32.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:32 smithi072 ceph-mon[34088]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:32.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:32 smithi110 ceph-mon[34904]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:32.830 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 04:47:32 smithi046 podman[72667]: 2024-05-09 04:47:32.642926991 +0000 UTC m=+0.018555040 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:47:33.330 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 04:47:32 smithi046 podman[72667]: 2024-05-09 04:47:32.977840085 +0000 UTC m=+0.353468139 container init 883e51440f9174047758cf9c37997128ef3db67c28c3b8a463fc23e7f0aacf02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-09T04:47:33.330 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 04:47:32 smithi046 podman[72667]: 2024-05-09 04:47:32.980904015 +0000 UTC m=+0.356532064 container start 883e51440f9174047758cf9c37997128ef3db67c28c3b8a463fc23e7f0aacf02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-a, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-09T04:47:33.330 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 04:47:33 smithi046 ceph-mds[72686]: -- 172.21.15.46:0/369552472 <== mon.0 v2:172.21.15.46:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55977337da40 con 0x5597733c5a80 2024-05-09T04:47:33.330 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 04:47:33 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-a[72682]: starting mds.a at 2024-05-09T04:47:33.330 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 04:47:33 smithi046 bash[72667]: 883e51440f9174047758cf9c37997128ef3db67c28c3b8a463fc23e7f0aacf02 2024-05-09T04:47:33.330 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 04:47:33 smithi046 systemd[1]: Started Ceph mds.a for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:47:34.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:34 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.46:6840/204132247,v1:172.21.15.46:6841/204132247] up:boot 2024-05-09T04:47:34.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:34 smithi072 ceph-mon[34088]: fsmap 11 up:standby 2024-05-09T04:47:34.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:34 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T04:47:34.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:34 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:34.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:34 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:34.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:34 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:34.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:34 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:34.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:34 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:34.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:34 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:34.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:34 smithi072 ceph-mon[34088]: Deploying daemon mds.b on smithi072 2024-05-09T04:47:34.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:34 smithi072 ceph-mon[34088]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:34.538 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:47:34.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:34 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.46:6840/204132247,v1:172.21.15.46:6841/204132247] up:boot 2024-05-09T04:47:34.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:34 smithi110 ceph-mon[34904]: fsmap 11 up:standby 2024-05-09T04:47:34.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:34 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T04:47:34.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:34 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:34.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:34 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:34.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:34 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:34.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:34 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:34.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:34 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:34.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:34 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:34.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:34 smithi110 ceph-mon[34904]: Deploying daemon mds.b on smithi072 2024-05-09T04:47:34.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:34 smithi110 ceph-mon[34904]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:34.568 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:34 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.46:6840/204132247,v1:172.21.15.46:6841/204132247] up:boot 2024-05-09T04:47:34.568 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:34 smithi046 ceph-mon[30136]: fsmap 11 up:standby 2024-05-09T04:47:34.568 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:34 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T04:47:34.568 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:34 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:34.568 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:34 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:34.569 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:34 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:34.569 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:34 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-09T04:47:34.569 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:34 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-09T04:47:34.569 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:34 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:34.569 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:34 smithi046 ceph-mon[30136]: Deploying daemon mds.b on smithi072 2024-05-09T04:47:34.569 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:34 smithi046 ceph-mon[30136]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:34.809 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:47:35.630 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 04:47:35 smithi072 systemd[1]: Starting Ceph mds.b for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T04:47:35.881 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 04:47:35 smithi072 podman[63735]: 2024-05-09 04:47:35.630890995 +0000 UTC m=+0.107482110 container create b35740bd40ea853d6842dde24de84482982de9133964f3c9520b48194ed60ee5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-b, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T04:47:35.881 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 04:47:35 smithi072 podman[63735]: 2024-05-09 04:47:35.541675061 +0000 UTC m=+0.018266191 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T04:47:35.881 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 04:47:35 smithi072 podman[63735]: 2024-05-09 04:47:35.879271399 +0000 UTC m=+0.355862511 container init b35740bd40ea853d6842dde24de84482982de9133964f3c9520b48194ed60ee5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T04:47:35.881 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 04:47:35 smithi072 podman[63735]: 2024-05-09 04:47:35.881742604 +0000 UTC m=+0.358333723 container start b35740bd40ea853d6842dde24de84482982de9133964f3c9520b48194ed60ee5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-05-09T04:47:36.350 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 04:47:35 smithi072 ceph-mds[63754]: -- 172.21.15.72:0/6292436 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55bd86afba40 con 0x55bd86b43180 2024-05-09T04:47:36.350 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 04:47:35 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-b[63750]: starting mds.b at 2024-05-09T04:47:36.350 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 04:47:35 smithi072 bash[63735]: b35740bd40ea853d6842dde24de84482982de9133964f3c9520b48194ed60ee5 2024-05-09T04:47:36.350 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 04:47:35 smithi072 systemd[1]: Started Ceph mds.b for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T04:47:36.523 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:36 smithi046 ceph-mon[30136]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:36.523 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:36.524 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:36.524 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:36.524 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:36.524 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:36.524 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:36 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:36.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.548+0000 7f397a350700 1 -- 172.21.15.46:0/151651244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3974108f10 msgr2=0x7f3974109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:36.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.548+0000 7f397a350700 1 --2- 172.21.15.46:0/151651244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3974108f10 0x7f3974109320 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f395c0099c0 tx=0x7f395c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:47:36.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.548+0000 7f397a350700 1 -- 172.21.15.46:0/151651244 shutdown_connections 2024-05-09T04:47:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.548+0000 7f397a350700 1 --2- 172.21.15.46:0/151651244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3974109860 0x7f39741111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.548+0000 7f397a350700 1 --2- 172.21.15.46:0/151651244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39741045e0 0x7f39741049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.548+0000 7f397a350700 1 --2- 172.21.15.46:0/151651244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3974108f10 0x7f3974109320 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.548+0000 7f397a350700 1 -- 172.21.15.46:0/151651244 >> 172.21.15.46:0/151651244 conn(0x7f39740ffef0 msgr2=0x7f3974102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.548+0000 7f397a350700 1 -- 172.21.15.46:0/151651244 shutdown_connections 2024-05-09T04:47:36.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.548+0000 7f397a350700 1 -- 172.21.15.46:0/151651244 wait complete. 2024-05-09T04:47:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.549+0000 7f397a350700 1 Processor -- start 2024-05-09T04:47:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.549+0000 7f397a350700 1 -- start start 2024-05-09T04:47:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f397a350700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39741045e0 0x7f3974076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f397a350700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3974108f10 0x7f3974077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f397a350700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3974109860 0x7f3974077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f397a350700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3974114470 con 0x7f3974109860 2024-05-09T04:47:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f397a350700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f39741142f0 con 0x7f39741045e0 2024-05-09T04:47:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f397a350700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f39741145f0 con 0x7f3974108f10 2024-05-09T04:47:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f3973fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39741045e0 0x7f3974076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f39788ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3974109860 0x7f3974077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f39788ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3974109860 0x7f3974077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58462/0 (socket says 172.21.15.46:58462) 2024-05-09T04:47:36.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f39788ed700 1 -- 172.21.15.46:0/1400508567 learned_addr learned my addr 172.21.15.46:0/1400508567 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:47:36.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f3973fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39741045e0 0x7f3974076bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:48562/0 (socket says 172.21.15.46:48562) 2024-05-09T04:47:36.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f39737fe700 1 --2- 172.21.15.46:0/1400508567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3974108f10 0x7f3974077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:36.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f3973fff700 1 -- 172.21.15.46:0/1400508567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3974108f10 msgr2=0x7f3974077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:36.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f3973fff700 1 --2- 172.21.15.46:0/1400508567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3974108f10 0x7f3974077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:36.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f3973fff700 1 -- 172.21.15.46:0/1400508567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3974109860 msgr2=0x7f3974077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:36.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f3973fff700 1 --2- 172.21.15.46:0/1400508567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3974109860 0x7f3974077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:36.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f3973fff700 1 -- 172.21.15.46:0/1400508567 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39741a91e0 con 0x7f39741045e0 2024-05-09T04:47:36.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f3973fff700 1 --2- 172.21.15.46:0/1400508567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39741045e0 0x7f3974076bd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f396400e9f0 tx=0x7f396400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:36.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.550+0000 7f39717fa700 1 -- 172.21.15.46:0/1400508567 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396400c9c0 con 0x7f39741045e0 2024-05-09T04:47:36.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.551+0000 7f397a350700 1 -- 172.21.15.46:0/1400508567 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39741a94d0 con 0x7f39741045e0 2024-05-09T04:47:36.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.551+0000 7f39717fa700 1 -- 172.21.15.46:0/1400508567 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f396400cb60 con 0x7f39741045e0 2024-05-09T04:47:36.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.551+0000 7f39717fa700 1 -- 172.21.15.46:0/1400508567 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3964018680 con 0x7f39741045e0 2024-05-09T04:47:36.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.551+0000 7f397a350700 1 -- 172.21.15.46:0/1400508567 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39741a99e0 con 0x7f39741045e0 2024-05-09T04:47:36.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.551+0000 7f397a350700 1 -- 172.21.15.46:0/1400508567 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39740512c0 con 0x7f39741045e0 2024-05-09T04:47:36.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.553+0000 7f39717fa700 1 -- 172.21.15.46:0/1400508567 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3964010430 con 0x7f39741045e0 2024-05-09T04:47:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.554+0000 7f39717fa700 1 --2- 172.21.15.46:0/1400508567 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f39600b2880 0x7f39600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.554+0000 7f39737fe700 1 --2- 172.21.15.46:0/1400508567 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f39600b2880 0x7f39600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.554+0000 7f39717fa700 1 -- 172.21.15.46:0/1400508567 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f39640e2240 con 0x7f39741045e0 2024-05-09T04:47:36.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.554+0000 7f39737fe700 1 --2- 172.21.15.46:0/1400508567 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f39600b2880 0x7f39600b4d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f395c000c00 tx=0x7f395c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.557+0000 7f39717fa700 1 -- 172.21.15.46:0/1400508567 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f39640aa3e0 con 0x7f39741045e0 2024-05-09T04:47:36.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:36 smithi072 ceph-mon[34088]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:36.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:36.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:36.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:36.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:36.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:36.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:36 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:36.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:36 smithi110 ceph-mon[34904]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:36.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:36.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:36.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:36.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:36.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:36.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:36 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:36.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.828+0000 7f397a350700 1 -- 172.21.15.46:0/1400508567 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f397402e5f0 con 0x7f39741045e0 2024-05-09T04:47:36.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.829+0000 7f39717fa700 1 -- 172.21.15.46:0/1400508567 <== mon.2 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10473 (secure 0 0 0) 0x7f39640a9ce0 con 0x7f39741045e0 2024-05-09T04:47:36.830 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:47:36.830 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24427,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24433,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24443,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24448,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24473,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24503,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-09T04:47:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.832+0000 7f397a350700 1 -- 172.21.15.46:0/1400508567 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f39600b2880 msgr2=0x7f39600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.832+0000 7f397a350700 1 --2- 172.21.15.46:0/1400508567 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f39600b2880 0x7f39600b4d90 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f395c000c00 tx=0x7f395c01a040 comp rx=0 tx=0).stop 2024-05-09T04:47:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.832+0000 7f397a350700 1 -- 172.21.15.46:0/1400508567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39741045e0 msgr2=0x7f3974076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.832+0000 7f397a350700 1 --2- 172.21.15.46:0/1400508567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39741045e0 0x7f3974076bd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f396400e9f0 tx=0x7f396400ed00 comp rx=0 tx=0).stop 2024-05-09T04:47:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.832+0000 7f397a350700 1 -- 172.21.15.46:0/1400508567 shutdown_connections 2024-05-09T04:47:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.832+0000 7f397a350700 1 --2- 172.21.15.46:0/1400508567 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f39600b2880 0x7f39600b4d90 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.832+0000 7f397a350700 1 --2- 172.21.15.46:0/1400508567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3974109860 0x7f3974077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:36.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.832+0000 7f397a350700 1 --2- 172.21.15.46:0/1400508567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39741045e0 0x7f3974076bd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.832+0000 7f397a350700 1 --2- 172.21.15.46:0/1400508567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3974108f10 0x7f3974077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.832+0000 7f397a350700 1 -- 172.21.15.46:0/1400508567 >> 172.21.15.46:0/1400508567 conn(0x7f39740ffef0 msgr2=0x7f397410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.832+0000 7f397a350700 1 -- 172.21.15.46:0/1400508567 shutdown_connections 2024-05-09T04:47:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:36.832+0000 7f397a350700 1 -- 172.21.15.46:0/1400508567 wait complete. 2024-05-09T04:47:36.834 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 13 2024-05-09T04:47:37.461 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:37 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] up:boot 2024-05-09T04:47:37.462 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:37 smithi046 ceph-mon[30136]: fsmap 12 up:standby 2024-05-09T04:47:37.462 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:37 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T04:47:37.462 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:37 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1400508567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:37.462 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:37 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:37.491 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-09T04:47:37.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:37 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] up:boot 2024-05-09T04:47:37.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:37 smithi072 ceph-mon[34088]: fsmap 12 up:standby 2024-05-09T04:47:37.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:37 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T04:47:37.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:37 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1400508567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:37.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:37 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:37.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:37 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] up:boot 2024-05-09T04:47:37.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:37 smithi110 ceph-mon[34904]: fsmap 12 up:standby 2024-05-09T04:47:37.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:37 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T04:47:37.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:37 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1400508567' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:47:37.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:37 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:38.240 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:47:38.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:38 smithi046 ceph-mon[30136]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:38.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:38 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:38.611 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:38 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:38.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:38 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:38.612 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:38 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:38.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:38 smithi072 ceph-mon[34088]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:38.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:38 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:38.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:38 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:38.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:38 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:38.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:38 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:38.708 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:38 smithi110 ceph-mon[34904]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:38.708 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:38 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:38.708 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:38 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:38.708 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:38 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:38.708 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:38 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.923+0000 7fa6cfc73700 1 -- 172.21.15.46:0/2337432763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c8075d30 msgr2=0x7fa6c8076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.923+0000 7fa6cfc73700 1 --2- 172.21.15.46:0/2337432763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c8075d30 0x7fa6c8076140 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fa6c400b0a0 tx=0x7fa6c400b3b0 comp rx=0 tx=0).stop 2024-05-09T04:47:39.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.923+0000 7fa6cfc73700 1 -- 172.21.15.46:0/2337432763 shutdown_connections 2024-05-09T04:47:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.923+0000 7fa6cfc73700 1 --2- 172.21.15.46:0/2337432763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6c8110ba0 0x7fa6c8113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.923+0000 7fa6cfc73700 1 --2- 172.21.15.46:0/2337432763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c8075d30 0x7fa6c8076140 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.923+0000 7fa6cfc73700 1 --2- 172.21.15.46:0/2337432763 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c8076680 0x7fa6c8110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.923+0000 7fa6cfc73700 1 -- 172.21.15.46:0/2337432763 >> 172.21.15.46:0/2337432763 conn(0x7fa6c80fffe0 msgr2=0x7fa6c8102400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.923+0000 7fa6cfc73700 1 -- 172.21.15.46:0/2337432763 shutdown_connections 2024-05-09T04:47:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.923+0000 7fa6cfc73700 1 -- 172.21.15.46:0/2337432763 wait complete. 2024-05-09T04:47:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.924+0000 7fa6cfc73700 1 Processor -- start 2024-05-09T04:47:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.924+0000 7fa6cfc73700 1 -- start start 2024-05-09T04:47:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.924+0000 7fa6cfc73700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c8075d30 0x7fa6c819d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.924+0000 7fa6cfc73700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6c8076680 0x7fa6c819d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.924+0000 7fa6cfc73700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c8110ba0 0x7fa6c81a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.924+0000 7fa6cfc73700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c8116710 con 0x7fa6c8110ba0 2024-05-09T04:47:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.924+0000 7fa6cfc73700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c8116590 con 0x7fa6c8075d30 2024-05-09T04:47:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.924+0000 7fa6cfc73700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c8116890 con 0x7fa6c8076680 2024-05-09T04:47:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6ce210700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c8110ba0 0x7fa6c81a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6cd20e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6c8076680 0x7fa6c819d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6ce210700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c8110ba0 0x7fa6c81a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58478/0 (socket says 172.21.15.46:58478) 2024-05-09T04:47:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6ce210700 1 -- 172.21.15.46:0/3091922303 learned_addr learned my addr 172.21.15.46:0/3091922303 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:47:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6cda0f700 1 --2- 172.21.15.46:0/3091922303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c8075d30 0x7fa6c819d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6cd20e700 1 -- 172.21.15.46:0/3091922303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c8075d30 msgr2=0x7fa6c819d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:39.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6cd20e700 1 --2- 172.21.15.46:0/3091922303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c8075d30 0x7fa6c819d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6cd20e700 1 -- 172.21.15.46:0/3091922303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c8110ba0 msgr2=0x7fa6c81a1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6cd20e700 1 --2- 172.21.15.46:0/3091922303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c8110ba0 0x7fa6c81a1890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6cd20e700 1 -- 172.21.15.46:0/3091922303 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c81a1f30 con 0x7fa6c8076680 2024-05-09T04:47:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6cd20e700 1 --2- 172.21.15.46:0/3091922303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6c8076680 0x7fa6c819d600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800b450 tx=0x7fa6b800b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6beffd700 1 -- 172.21.15.46:0/3091922303 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8015020 con 0x7fa6c8076680 2024-05-09T04:47:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6beffd700 1 -- 172.21.15.46:0/3091922303 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b8011ce0 con 0x7fa6c8076680 2024-05-09T04:47:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6cfc73700 1 -- 172.21.15.46:0/3091922303 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c81a2220 con 0x7fa6c8076680 2024-05-09T04:47:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6beffd700 1 -- 172.21.15.46:0/3091922303 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b8010be0 con 0x7fa6c8076680 2024-05-09T04:47:39.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.925+0000 7fa6cfc73700 1 -- 172.21.15.46:0/3091922303 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c81a9b40 con 0x7fa6c8076680 2024-05-09T04:47:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.926+0000 7fa6cfc73700 1 -- 172.21.15.46:0/3091922303 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6c80512c0 con 0x7fa6c8076680 2024-05-09T04:47:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.928+0000 7fa6beffd700 1 -- 172.21.15.46:0/3091922303 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa6b8018020 con 0x7fa6c8076680 2024-05-09T04:47:39.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.929+0000 7fa6beffd700 1 --2- 172.21.15.46:0/3091922303 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa6b40b2880 0x7fa6b40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.929+0000 7fa6cda0f700 1 --2- 172.21.15.46:0/3091922303 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa6b40b2880 0x7fa6b40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.929+0000 7fa6beffd700 1 -- 172.21.15.46:0/3091922303 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7fa6b80e1960 con 0x7fa6c8076680 2024-05-09T04:47:39.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.929+0000 7fa6cda0f700 1 --2- 172.21.15.46:0/3091922303 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa6b40b2880 0x7fa6b40b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009900 tx=0x7fa6c400b700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:39.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:39.932+0000 7fa6beffd700 1 -- 172.21.15.46:0/3091922303 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa6b801f020 con 0x7fa6c8076680 2024-05-09T04:47:40.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:40.158+0000 7fa6cfc73700 1 -- 172.21.15.46:0/3091922303 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fa6c819e3c0 con 0x7fa6c8076680 2024-05-09T04:47:40.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:40.158+0000 7fa6beffd700 1 -- 172.21.15.46:0/3091922303 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v73) ==== 84+0+65 (secure 0 0 0) 0x7fa6b80a9ac0 con 0x7fa6c8076680 2024-05-09T04:47:40.159 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:47:40.160 INFO:teuthology.orchestra.run.smithi046.stdout:[ 2024-05-09T04:47:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T04:47:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "poolnum": 1, 2024-05-09T04:47:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: "poolname": ".mgr" 2024-05-09T04:47:40.160 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T04:47:40.160 INFO:teuthology.orchestra.run.smithi046.stdout:] 2024-05-09T04:47:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:40.161+0000 7fa6cfc73700 1 -- 172.21.15.46:0/3091922303 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa6b40b2880 msgr2=0x7fa6b40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:40.161+0000 7fa6cfc73700 1 --2- 172.21.15.46:0/3091922303 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa6b40b2880 0x7fa6b40b4d90 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fa6c4009900 tx=0x7fa6c400b700 comp rx=0 tx=0).stop 2024-05-09T04:47:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:40.161+0000 7fa6cfc73700 1 -- 172.21.15.46:0/3091922303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6c8076680 msgr2=0x7fa6c819d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:40.161+0000 7fa6cfc73700 1 --2- 172.21.15.46:0/3091922303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6c8076680 0x7fa6c819d600 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa6b800b450 tx=0x7fa6b800b760 comp rx=0 tx=0).stop 2024-05-09T04:47:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:40.161+0000 7fa6cfc73700 1 -- 172.21.15.46:0/3091922303 shutdown_connections 2024-05-09T04:47:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:40.161+0000 7fa6cfc73700 1 --2- 172.21.15.46:0/3091922303 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa6b40b2880 0x7fa6b40b4d90 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:40.161+0000 7fa6cfc73700 1 --2- 172.21.15.46:0/3091922303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa6c8110ba0 0x7fa6c81a1890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:40.161+0000 7fa6cfc73700 1 --2- 172.21.15.46:0/3091922303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa6c8075d30 0x7fa6c819d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:40.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:40.161+0000 7fa6cfc73700 1 --2- 172.21.15.46:0/3091922303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa6c8076680 0x7fa6c819d600 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:40.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:40.161+0000 7fa6cfc73700 1 -- 172.21.15.46:0/3091922303 >> 172.21.15.46:0/3091922303 conn(0x7fa6c80fffe0 msgr2=0x7fa6c8101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:40.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:40.161+0000 7fa6cfc73700 1 -- 172.21.15.46:0/3091922303 shutdown_connections 2024-05-09T04:47:40.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:40.161+0000 7fa6cfc73700 1 -- 172.21.15.46:0/3091922303 wait complete. 2024-05-09T04:47:40.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:40 smithi046 ceph-mon[30136]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:40.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:40.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:40.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:40.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:40.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:40.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:40 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3091922303' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T04:47:40.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:40 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:40.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:40 smithi110 ceph-mon[34904]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:40.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:40.556 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:40.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:40.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:40.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:40.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:40 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3091922303' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T04:47:40.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:40 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:40 smithi072 ceph-mon[34088]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:40 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3091922303' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-09T04:47:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:40 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:47:40.871 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-09T04:47:40.871 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-09T04:47:41.690 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:47:42.621 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:42 smithi046 ceph-mon[30136]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:42.621 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:42 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:42.621 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:42 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:42.621 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:42 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:42.621 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:42 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:42.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:42 smithi072 ceph-mon[34088]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:42.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:42 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:42.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:42 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:42.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:42 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:42.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:42 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:42.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:42 smithi110 ceph-mon[34904]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:42.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:42 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:42.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:42 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:42.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:42 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:42.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:42 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:43.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.170+0000 7f74174a4700 1 -- 172.21.15.46:0/2774910763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410075d30 msgr2=0x7f7410076140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:43.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.170+0000 7f74174a4700 1 --2- 172.21.15.46:0/2774910763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410075d30 0x7f7410076140 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f740c00b0a0 tx=0x7f740c00b3b0 comp rx=0 tx=0).stop 2024-05-09T04:47:43.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.170+0000 7f74174a4700 1 -- 172.21.15.46:0/2774910763 shutdown_connections 2024-05-09T04:47:43.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.170+0000 7f74174a4700 1 --2- 172.21.15.46:0/2774910763 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7410110ba0 0x7f7410113000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:43.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.170+0000 7f74174a4700 1 --2- 172.21.15.46:0/2774910763 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410075d30 0x7f7410076140 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:43.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.170+0000 7f74174a4700 1 --2- 172.21.15.46:0/2774910763 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7410076680 0x7f7410110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:43.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.170+0000 7f74174a4700 1 -- 172.21.15.46:0/2774910763 >> 172.21.15.46:0/2774910763 conn(0x7f7410100040 msgr2=0x7f7410102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:43.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.170+0000 7f74174a4700 1 -- 172.21.15.46:0/2774910763 shutdown_connections 2024-05-09T04:47:43.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.171+0000 7f74174a4700 1 -- 172.21.15.46:0/2774910763 wait complete. 2024-05-09T04:47:43.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.172+0000 7f74174a4700 1 Processor -- start 2024-05-09T04:47:43.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.172+0000 7f74174a4700 1 -- start start 2024-05-09T04:47:43.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.172+0000 7f74174a4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7410075d30 0x7f741019d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:43.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.172+0000 7f74174a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7410076680 0x7f741019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:43.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.172+0000 7f74174a4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410110ba0 0x7f74101a1890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:43.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.172+0000 7f74174a4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7410116690 con 0x7f7410110ba0 2024-05-09T04:47:43.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.172+0000 7f74174a4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7410116510 con 0x7f7410075d30 2024-05-09T04:47:43.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.172+0000 7f74174a4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7410116810 con 0x7f7410076680 2024-05-09T04:47:43.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.172+0000 7f7415a41700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410110ba0 0x7f74101a1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:43.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.172+0000 7f7415a41700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410110ba0 0x7f74101a1890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58490/0 (socket says 172.21.15.46:58490) 2024-05-09T04:47:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.172+0000 7f7415a41700 1 -- 172.21.15.46:0/1728725747 learned_addr learned my addr 172.21.15.46:0/1728725747 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:47:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.172+0000 7f7414a3f700 1 --2- 172.21.15.46:0/1728725747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7410076680 0x7f741019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.172+0000 7f7415240700 1 --2- 172.21.15.46:0/1728725747 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7410075d30 0x7f741019d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.172+0000 7f7415a41700 1 -- 172.21.15.46:0/1728725747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7410076680 msgr2=0x7f741019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.172+0000 7f7415a41700 1 --2- 172.21.15.46:0/1728725747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7410076680 0x7f741019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.173+0000 7f7415a41700 1 -- 172.21.15.46:0/1728725747 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7410075d30 msgr2=0x7f741019d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.173+0000 7f7415a41700 1 --2- 172.21.15.46:0/1728725747 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7410075d30 0x7f741019d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.173+0000 7f7415a41700 1 -- 172.21.15.46:0/1728725747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74101a1f30 con 0x7f7410110ba0 2024-05-09T04:47:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.173+0000 7f7415a41700 1 --2- 172.21.15.46:0/1728725747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410110ba0 0x7f74101a1890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f740800c970 tx=0x7f740800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.173+0000 7f74067fc700 1 -- 172.21.15.46:0/1728725747 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7408004a30 con 0x7f7410110ba0 2024-05-09T04:47:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.173+0000 7f74067fc700 1 -- 172.21.15.46:0/1728725747 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74080054d0 con 0x7f7410110ba0 2024-05-09T04:47:43.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.173+0000 7f74174a4700 1 -- 172.21.15.46:0/1728725747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74101a2220 con 0x7f7410110ba0 2024-05-09T04:47:43.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.173+0000 7f74067fc700 1 -- 172.21.15.46:0/1728725747 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f740800ec20 con 0x7f7410110ba0 2024-05-09T04:47:43.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.173+0000 7f74174a4700 1 -- 172.21.15.46:0/1728725747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74101a9b40 con 0x7f7410110ba0 2024-05-09T04:47:43.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.174+0000 7f74174a4700 1 -- 172.21.15.46:0/1728725747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74100512c0 con 0x7f7410110ba0 2024-05-09T04:47:43.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.176+0000 7f74067fc700 1 -- 172.21.15.46:0/1728725747 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7408005000 con 0x7f7410110ba0 2024-05-09T04:47:43.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.176+0000 7f74067fc700 1 --2- 172.21.15.46:0/1728725747 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f73fc0b2900 0x7f73fc0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:43.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.177+0000 7f7415240700 1 --2- 172.21.15.46:0/1728725747 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f73fc0b2900 0x7f73fc0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:43.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.177+0000 7f74067fc700 1 -- 172.21.15.46:0/1728725747 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 7127+0+0 (secure 0 0 0) 0x7f74080e1ad0 con 0x7f7410110ba0 2024-05-09T04:47:43.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.177+0000 7f7415240700 1 --2- 172.21.15.46:0/1728725747 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f73fc0b2900 0x7f73fc0b4e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f740c009900 tx=0x7f740c00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:43.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.180+0000 7f74067fc700 1 -- 172.21.15.46:0/1728725747 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7408014030 con 0x7f7410110ba0 2024-05-09T04:47:43.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:43.408+0000 7f74174a4700 1 -- 172.21.15.46:0/1728725747 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f741002e5f0 con 0x7f7410110ba0 2024-05-09T04:47:44.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:44.366+0000 7f74067fc700 1 -- 172.21.15.46:0/1728725747 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v74) ==== 138+0+0 (secure 0 0 0) 0x7f74080052f0 con 0x7f7410110ba0 2024-05-09T04:47:44.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:44.368+0000 7f74174a4700 1 -- 172.21.15.46:0/1728725747 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f73fc0b2900 msgr2=0x7f73fc0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:44.368+0000 7f74174a4700 1 --2- 172.21.15.46:0/1728725747 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f73fc0b2900 0x7f73fc0b4e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f740c009900 tx=0x7f740c00b7b0 comp rx=0 tx=0).stop 2024-05-09T04:47:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:44.368+0000 7f74174a4700 1 -- 172.21.15.46:0/1728725747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410110ba0 msgr2=0x7f74101a1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:44.368+0000 7f74174a4700 1 --2- 172.21.15.46:0/1728725747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410110ba0 0x7f74101a1890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f740800c970 tx=0x7f740800cc80 comp rx=0 tx=0).stop 2024-05-09T04:47:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:44.369+0000 7f74174a4700 1 -- 172.21.15.46:0/1728725747 shutdown_connections 2024-05-09T04:47:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:44.369+0000 7f74174a4700 1 --2- 172.21.15.46:0/1728725747 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f73fc0b2900 0x7f73fc0b4e10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:44.369+0000 7f74174a4700 1 --2- 172.21.15.46:0/1728725747 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7410110ba0 0x7f74101a1890 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:44.369+0000 7f74174a4700 1 --2- 172.21.15.46:0/1728725747 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7410075d30 0x7f741019d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:44.369+0000 7f74174a4700 1 --2- 172.21.15.46:0/1728725747 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7410076680 0x7f741019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:44.369+0000 7f74174a4700 1 -- 172.21.15.46:0/1728725747 >> 172.21.15.46:0/1728725747 conn(0x7f7410100040 msgr2=0x7f7410100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:44.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:44.369+0000 7f74174a4700 1 -- 172.21.15.46:0/1728725747 shutdown_connections 2024-05-09T04:47:44.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:44.369+0000 7f74174a4700 1 -- 172.21.15.46:0/1728725747 wait complete. 2024-05-09T04:47:44.370 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' created 2024-05-09T04:47:44.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:44 smithi072 ceph-mon[34088]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:44.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:44 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1728725747' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T04:47:44.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:44 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:44.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:44 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:44.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:44 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:44.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:44 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:44.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:44 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:44.695 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:44 smithi046 ceph-mon[30136]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:44.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:44 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1728725747' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T04:47:44.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:44 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:44.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:44 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:44.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:44 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:44.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:44 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:44.696 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:44 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:44.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:44 smithi110 ceph-mon[34904]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:44.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:44 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1728725747' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-09T04:47:44.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:44 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:44.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:44 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:44.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:44 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:47:44.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:44 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:47:44.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:44 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:47:45.055 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-05-09T04:47:45.306 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:47:45.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:45 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1728725747' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T04:47:45.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:45 smithi072 ceph-mon[34088]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T04:47:45.691 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:45 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1728725747' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T04:47:45.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:45 smithi046 ceph-mon[30136]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T04:47:45.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:45 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1728725747' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-09T04:47:45.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:45 smithi110 ceph-mon[34904]: osdmap e74: 12 total, 12 up, 12 in 2024-05-09T04:47:46.471 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:46 smithi046 ceph-mon[30136]: pgmap v283: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:46.472 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:46 smithi046 ceph-mon[30136]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T04:47:46.472 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:46 smithi046 ceph-mon[30136]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T04:47:46.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:46 smithi072 ceph-mon[34088]: pgmap v283: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:46.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:46 smithi072 ceph-mon[34088]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T04:47:46.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:46 smithi072 ceph-mon[34088]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T04:47:46.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:46 smithi110 ceph-mon[34904]: pgmap v283: 33 pgs: 6 creating+peering, 26 unknown, 1 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:46.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:46 smithi110 ceph-mon[34904]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T04:47:46.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:46 smithi110 ceph-mon[34904]: osdmap e75: 12 total, 12 up, 12 in 2024-05-09T04:47:47.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.036+0000 7fdc2b010700 1 -- 172.21.15.46:0/3485954923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc24109310 msgr2=0x7fdc24109720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:47.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.037+0000 7fdc2b010700 1 --2- 172.21.15.46:0/3485954923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc24109310 0x7fdc24109720 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009990 tx=0x7fdc14009ca0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.037+0000 7fdc2b010700 1 -- 172.21.15.46:0/3485954923 shutdown_connections 2024-05-09T04:47:47.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.037+0000 7fdc2b010700 1 --2- 172.21.15.46:0/3485954923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc241030f0 0x7fdc24111180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.037+0000 7fdc2b010700 1 --2- 172.21.15.46:0/3485954923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc24109310 0x7fdc24109720 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.037+0000 7fdc2b010700 1 --2- 172.21.15.46:0/3485954923 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc241027a0 0x7fdc24102bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.037+0000 7fdc2b010700 1 -- 172.21.15.46:0/3485954923 >> 172.21.15.46:0/3485954923 conn(0x7fdc240fe4f0 msgr2=0x7fdc24100940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:47.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.037+0000 7fdc2b010700 1 -- 172.21.15.46:0/3485954923 shutdown_connections 2024-05-09T04:47:47.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.037+0000 7fdc2b010700 1 -- 172.21.15.46:0/3485954923 wait complete. 2024-05-09T04:47:47.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc2b010700 1 Processor -- start 2024-05-09T04:47:47.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc2b010700 1 -- start start 2024-05-09T04:47:47.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc2b010700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc241027a0 0x7fdc24103980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc2b010700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc241030f0 0x7fdc24103ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc2b010700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc24109310 0x7fdc24104400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc2b010700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdc241145a0 con 0x7fdc241027a0 2024-05-09T04:47:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc2b010700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24114420 con 0x7fdc241030f0 2024-05-09T04:47:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc2b010700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdc24114720 con 0x7fdc24109310 2024-05-09T04:47:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc28dac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc241027a0 0x7fdc24103980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc28dac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc241027a0 0x7fdc24103980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42062/0 (socket says 172.21.15.46:42062) 2024-05-09T04:47:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc28dac700 1 -- 172.21.15.46:0/756772564 learned_addr learned my addr 172.21.15.46:0/756772564 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:47:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc295ad700 1 --2- 172.21.15.46:0/756772564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc24109310 0x7fdc24104400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc1bfff700 1 --2- 172.21.15.46:0/756772564 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc241030f0 0x7fdc24103ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc28dac700 1 -- 172.21.15.46:0/756772564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc24109310 msgr2=0x7fdc24104400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:47.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc28dac700 1 --2- 172.21.15.46:0/756772564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc24109310 0x7fdc24104400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc28dac700 1 -- 172.21.15.46:0/756772564 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc241030f0 msgr2=0x7fdc24103ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc28dac700 1 --2- 172.21.15.46:0/756772564 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc241030f0 0x7fdc24103ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc28dac700 1 -- 172.21.15.46:0/756772564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc241a9220 con 0x7fdc241027a0 2024-05-09T04:47:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.038+0000 7fdc1bfff700 1 --2- 172.21.15.46:0/756772564 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc241030f0 0x7fdc24103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:47:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.039+0000 7fdc28dac700 1 --2- 172.21.15.46:0/756772564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc241027a0 0x7fdc24103980 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdc14006010 tx=0x7fdc140047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.039+0000 7fdc19ffb700 1 -- 172.21.15.46:0/756772564 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14027070 con 0x7fdc241027a0 2024-05-09T04:47:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.039+0000 7fdc19ffb700 1 -- 172.21.15.46:0/756772564 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc14003fe0 con 0x7fdc241027a0 2024-05-09T04:47:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.039+0000 7fdc2b010700 1 -- 172.21.15.46:0/756772564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc241a94b0 con 0x7fdc241027a0 2024-05-09T04:47:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.039+0000 7fdc19ffb700 1 -- 172.21.15.46:0/756772564 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc14022b20 con 0x7fdc241027a0 2024-05-09T04:47:47.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.039+0000 7fdc2b010700 1 -- 172.21.15.46:0/756772564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc241a9900 con 0x7fdc241027a0 2024-05-09T04:47:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.041+0000 7fdc2b010700 1 -- 172.21.15.46:0/756772564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc2410c8b0 con 0x7fdc241027a0 2024-05-09T04:47:47.042 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.042+0000 7fdc19ffb700 1 -- 172.21.15.46:0/756772564 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdc1402a070 con 0x7fdc241027a0 2024-05-09T04:47:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.043+0000 7fdc19ffb700 1 --2- 172.21.15.46:0/756772564 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fdc0c0b2880 0x7fdc0c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.043+0000 7fdc1bfff700 1 --2- 172.21.15.46:0/756772564 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fdc0c0b2880 0x7fdc0c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.044+0000 7fdc19ffb700 1 -- 172.21.15.46:0/756772564 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7497+0+0 (secure 0 0 0) 0x7fdc140e3500 con 0x7fdc241027a0 2024-05-09T04:47:47.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.044+0000 7fdc1bfff700 1 --2- 172.21.15.46:0/756772564 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fdc0c0b2880 0x7fdc0c0b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdc20005fd0 tx=0x7fdc20005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:47.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.047+0000 7fdc19ffb700 1 -- 172.21.15.46:0/756772564 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdc14031080 con 0x7fdc241027a0 2024-05-09T04:47:47.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.260+0000 7fdc2b010700 1 -- 172.21.15.46:0/756772564 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fdc24104eb0 con 0x7fdc241027a0 2024-05-09T04:47:47.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.384+0000 7fdc19ffb700 1 -- 172.21.15.46:0/756772564 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) ==== 170+0+0 (secure 0 0 0) 0x7fdc14013070 con 0x7fdc241027a0 2024-05-09T04:47:47.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.387+0000 7fdc2b010700 1 -- 172.21.15.46:0/756772564 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fdc0c0b2880 msgr2=0x7fdc0c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:47.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.387+0000 7fdc2b010700 1 --2- 172.21.15.46:0/756772564 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fdc0c0b2880 0x7fdc0c0b4d90 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fdc20005fd0 tx=0x7fdc20005e30 comp rx=0 tx=0).stop 2024-05-09T04:47:47.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.387+0000 7fdc2b010700 1 -- 172.21.15.46:0/756772564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc241027a0 msgr2=0x7fdc24103980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.387+0000 7fdc2b010700 1 --2- 172.21.15.46:0/756772564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc241027a0 0x7fdc24103980 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fdc14006010 tx=0x7fdc140047b0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.387+0000 7fdc2b010700 1 -- 172.21.15.46:0/756772564 shutdown_connections 2024-05-09T04:47:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.387+0000 7fdc2b010700 1 --2- 172.21.15.46:0/756772564 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fdc0c0b2880 0x7fdc0c0b4d90 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.387+0000 7fdc2b010700 1 --2- 172.21.15.46:0/756772564 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdc24109310 0x7fdc24104400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.387+0000 7fdc2b010700 1 --2- 172.21.15.46:0/756772564 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdc241027a0 0x7fdc24103980 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.387+0000 7fdc2b010700 1 --2- 172.21.15.46:0/756772564 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdc241030f0 0x7fdc24103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.387+0000 7fdc2b010700 1 -- 172.21.15.46:0/756772564 >> 172.21.15.46:0/756772564 conn(0x7fdc240fe4f0 msgr2=0x7fdc241071f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.388+0000 7fdc2b010700 1 -- 172.21.15.46:0/756772564 shutdown_connections 2024-05-09T04:47:47.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:47.388+0000 7fdc2b010700 1 -- 172.21.15.46:0/756772564 wait complete. 2024-05-09T04:47:47.390 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' created 2024-05-09T04:47:47.770 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:47 smithi046 ceph-mon[30136]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T04:47:47.771 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:47 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/756772564' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T04:47:47.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:47 smithi110 ceph-mon[34904]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T04:47:47.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:47 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/756772564' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T04:47:47.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:47 smithi072 ceph-mon[34088]: osdmap e76: 12 total, 12 up, 12 in 2024-05-09T04:47:47.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:47 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/756772564' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-09T04:47:48.111 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-09T04:47:48.366 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:47:48.399 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:48 smithi046 ceph-mon[30136]: pgmap v286: 33 pgs: 6 creating+peering, 20 unknown, 7 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:48.400 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:48 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/756772564' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T04:47:48.400 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:48 smithi046 ceph-mon[30136]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T04:47:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:48 smithi110 ceph-mon[34904]: pgmap v286: 33 pgs: 6 creating+peering, 20 unknown, 7 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:48 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/756772564' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T04:47:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:48 smithi110 ceph-mon[34904]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T04:47:48.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:48 smithi072 ceph-mon[34088]: pgmap v286: 33 pgs: 6 creating+peering, 20 unknown, 7 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:48.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:48 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/756772564' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-09T04:47:48.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:48 smithi072 ceph-mon[34088]: osdmap e77: 12 total, 12 up, 12 in 2024-05-09T04:47:49.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:49 smithi072 ceph-mon[34088]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T04:47:49.657 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:49 smithi046 ceph-mon[30136]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T04:47:49.715 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:49 smithi110 ceph-mon[34904]: osdmap e78: 12 total, 12 up, 12 in 2024-05-09T04:47:50.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.040+0000 7f7df0754700 1 -- 172.21.15.46:0/960573069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de81034c0 msgr2=0x7f7de81038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.040+0000 7f7df0754700 1 --2- 172.21.15.46:0/960573069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de81034c0 0x7f7de81038d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f7de00099c0 tx=0x7f7de0009cd0 comp rx=0 tx=0).stop 2024-05-09T04:47:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.041+0000 7f7df0754700 1 -- 172.21.15.46:0/960573069 shutdown_connections 2024-05-09T04:47:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.041+0000 7f7df0754700 1 --2- 172.21.15.46:0/960573069 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7de8108740 0x7f7de81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.041+0000 7f7df0754700 1 --2- 172.21.15.46:0/960573069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de81034c0 0x7f7de81038d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.041+0000 7f7df0754700 1 --2- 172.21.15.46:0/960573069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7de8107df0 0x7f7de8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.041+0000 7f7df0754700 1 -- 172.21.15.46:0/960573069 >> 172.21.15.46:0/960573069 conn(0x7f7de80fedf0 msgr2=0x7f7de8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.041+0000 7f7df0754700 1 -- 172.21.15.46:0/960573069 shutdown_connections 2024-05-09T04:47:50.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.041+0000 7f7df0754700 1 -- 172.21.15.46:0/960573069 wait complete. 2024-05-09T04:47:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.041+0000 7f7df0754700 1 Processor -- start 2024-05-09T04:47:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.041+0000 7f7df0754700 1 -- start start 2024-05-09T04:47:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7df0754700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7de81034c0 0x7f7de8110a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7df0754700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7de8107df0 0x7f7de8110fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7df0754700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de8108740 0x7f7de8111500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7df0754700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7de81144c0 con 0x7f7de8108740 2024-05-09T04:47:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7df0754700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7de8114340 con 0x7f7de81034c0 2024-05-09T04:47:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7df0754700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7de8114640 con 0x7f7de8107df0 2024-05-09T04:47:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7deecf1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de8108740 0x7f7de8111500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:50.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7deecf1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de8108740 0x7f7de8111500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42072/0 (socket says 172.21.15.46:42072) 2024-05-09T04:47:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7deecf1700 1 -- 172.21.15.46:0/1107221685 learned_addr learned my addr 172.21.15.46:0/1107221685 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:47:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7dee4f0700 1 --2- 172.21.15.46:0/1107221685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7de81034c0 0x7f7de8110a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7dedcef700 1 --2- 172.21.15.46:0/1107221685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7de8107df0 0x7f7de8110fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7deecf1700 1 -- 172.21.15.46:0/1107221685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7de8107df0 msgr2=0x7f7de8110fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7deecf1700 1 --2- 172.21.15.46:0/1107221685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7de8107df0 0x7f7de8110fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7deecf1700 1 -- 172.21.15.46:0/1107221685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7de81034c0 msgr2=0x7f7de8110a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7deecf1700 1 --2- 172.21.15.46:0/1107221685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7de81034c0 0x7f7de8110a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7deecf1700 1 -- 172.21.15.46:0/1107221685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7de81a90e0 con 0x7f7de8108740 2024-05-09T04:47:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.042+0000 7f7deecf1700 1 --2- 172.21.15.46:0/1107221685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de8108740 0x7f7de8111500 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8007ee0 tx=0x7f7dd800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.043+0000 7f7de77fe700 1 -- 172.21.15.46:0/1107221685 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8004bb0 con 0x7f7de8108740 2024-05-09T04:47:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.043+0000 7f7df0754700 1 -- 172.21.15.46:0/1107221685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7de81a93d0 con 0x7f7de8108740 2024-05-09T04:47:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.043+0000 7f7de77fe700 1 -- 172.21.15.46:0/1107221685 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dd8004d50 con 0x7f7de8108740 2024-05-09T04:47:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.043+0000 7f7de77fe700 1 -- 172.21.15.46:0/1107221685 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dd8005920 con 0x7f7de8108740 2024-05-09T04:47:50.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.043+0000 7f7df0754700 1 -- 172.21.15.46:0/1107221685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7de81a9910 con 0x7f7de8108740 2024-05-09T04:47:50.045 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.044+0000 7f7df0754700 1 -- 172.21.15.46:0/1107221685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7de80512c0 con 0x7f7de8108740 2024-05-09T04:47:50.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.047+0000 7f7de77fe700 1 -- 172.21.15.46:0/1107221685 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f7dd801a020 con 0x7f7de8108740 2024-05-09T04:47:50.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.047+0000 7f7de77fe700 1 --2- 172.21.15.46:0/1107221685 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7dd00b2880 0x7f7dd00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:50.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.048+0000 7f7dee4f0700 1 --2- 172.21.15.46:0/1107221685 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7dd00b2880 0x7f7dd00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:50.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.048+0000 7f7de77fe700 1 -- 172.21.15.46:0/1107221685 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7879+0+0 (secure 0 0 0) 0x7f7dd80e1cc0 con 0x7f7de8108740 2024-05-09T04:47:50.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.048+0000 7f7dee4f0700 1 --2- 172.21.15.46:0/1107221685 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7dd00b2880 0x7f7dd00b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7de0006010 tx=0x7f7de0009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:50.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.051+0000 7f7de77fe700 1 -- 172.21.15.46:0/1107221685 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7dd8010070 con 0x7f7de8108740 2024-05-09T04:47:50.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.320+0000 7f7df0754700 1 -- 172.21.15.46:0/1107221685 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7de802e5f0 con 0x7f7de8108740 2024-05-09T04:47:50.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.403+0000 7f7de77fe700 1 -- 172.21.15.46:0/1107221685 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f7dd80a9b90 con 0x7f7de8108740 2024-05-09T04:47:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.406+0000 7f7df0754700 1 -- 172.21.15.46:0/1107221685 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7dd00b2880 msgr2=0x7f7dd00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.406+0000 7f7df0754700 1 --2- 172.21.15.46:0/1107221685 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7dd00b2880 0x7f7dd00b4d90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7de0006010 tx=0x7f7de0009f10 comp rx=0 tx=0).stop 2024-05-09T04:47:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.406+0000 7f7df0754700 1 -- 172.21.15.46:0/1107221685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de8108740 msgr2=0x7f7de8111500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.406+0000 7f7df0754700 1 --2- 172.21.15.46:0/1107221685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de8108740 0x7f7de8111500 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7dd8007ee0 tx=0x7f7dd800d3b0 comp rx=0 tx=0).stop 2024-05-09T04:47:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.406+0000 7f7df0754700 1 -- 172.21.15.46:0/1107221685 shutdown_connections 2024-05-09T04:47:50.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.406+0000 7f7df0754700 1 --2- 172.21.15.46:0/1107221685 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7dd00b2880 0x7f7dd00b4d90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.406+0000 7f7df0754700 1 --2- 172.21.15.46:0/1107221685 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7de8108740 0x7f7de8111500 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.406+0000 7f7df0754700 1 --2- 172.21.15.46:0/1107221685 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7de81034c0 0x7f7de8110a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.406+0000 7f7df0754700 1 --2- 172.21.15.46:0/1107221685 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7de8107df0 0x7f7de8110fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.406+0000 7f7df0754700 1 -- 172.21.15.46:0/1107221685 >> 172.21.15.46:0/1107221685 conn(0x7f7de80fedf0 msgr2=0x7f7de8107100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.406+0000 7f7df0754700 1 -- 172.21.15.46:0/1107221685 shutdown_connections 2024-05-09T04:47:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:50.406+0000 7f7df0754700 1 -- 172.21.15.46:0/1107221685 wait complete. 2024-05-09T04:47:50.407 INFO:teuthology.orchestra.run.smithi046.stderr:new fs with metadata pool 2 and data pool 3 2024-05-09T04:47:50.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:50 smithi110 ceph-mon[34904]: pgmap v289: 97 pgs: 20 creating+peering, 33 unknown, 44 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:50.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:50 smithi110 ceph-mon[34904]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T04:47:50.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:50 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1107221685' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T04:47:50.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:50 smithi110 ceph-mon[34904]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-09T04:47:50.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:50 smithi046 ceph-mon[30136]: pgmap v289: 97 pgs: 20 creating+peering, 33 unknown, 44 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:50.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:50 smithi046 ceph-mon[30136]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T04:47:50.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:50 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1107221685' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T04:47:50.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:50 smithi046 ceph-mon[30136]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-09T04:47:50.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:50 smithi072 ceph-mon[34088]: pgmap v289: 97 pgs: 20 creating+peering, 33 unknown, 44 active+clean; 577 KiB data, 354 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:50.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:50 smithi072 ceph-mon[34088]: osdmap e79: 12 total, 12 up, 12 in 2024-05-09T04:47:50.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:50 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1107221685' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-09T04:47:50.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:50 smithi072 ceph-mon[34088]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-09T04:47:51.211 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:47:51.467 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:47:51.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:51 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1107221685' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T04:47:51.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:51 smithi046 ceph-mon[30136]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T04:47:51.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:51 smithi046 ceph-mon[30136]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-09T04:47:51.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:51 smithi046 ceph-mon[30136]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-09T04:47:51.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:51 smithi046 ceph-mon[30136]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T04:47:51.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:51 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] up:active 2024-05-09T04:47:51.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:51 smithi046 ceph-mon[30136]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:47:51.630 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 04:47:51 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-b[63750]: 2024-05-09T04:47:51.354+0000 7f0ab36a3700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-09T04:47:51.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:51 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1107221685' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T04:47:51.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:51 smithi072 ceph-mon[34088]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T04:47:51.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:51 smithi072 ceph-mon[34088]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-09T04:47:51.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:51 smithi072 ceph-mon[34088]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-09T04:47:51.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:51 smithi072 ceph-mon[34088]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T04:47:51.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:51 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] up:active 2024-05-09T04:47:51.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:51 smithi072 ceph-mon[34088]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:47:51.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:51 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1107221685' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-09T04:47:51.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:51 smithi110 ceph-mon[34904]: osdmap e80: 12 total, 12 up, 12 in 2024-05-09T04:47:51.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:51 smithi110 ceph-mon[34904]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-09T04:47:51.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:51 smithi110 ceph-mon[34904]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-09T04:47:51.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:51 smithi110 ceph-mon[34904]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-09T04:47:51.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:51 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] up:active 2024-05-09T04:47:51.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:51 smithi110 ceph-mon[34904]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:47:52.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:52 smithi046 ceph-mon[30136]: pgmap v292: 97 pgs: 14 creating+peering, 11 unknown, 72 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:52.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:52 smithi046 ceph-mon[30136]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T04:47:52.581 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:52 smithi046 ceph-mon[30136]: Cluster is now healthy 2024-05-09T04:47:52.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:52 smithi110 ceph-mon[34904]: pgmap v292: 97 pgs: 14 creating+peering, 11 unknown, 72 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:52.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:52 smithi110 ceph-mon[34904]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T04:47:52.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:52 smithi110 ceph-mon[34904]: Cluster is now healthy 2024-05-09T04:47:52.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:52 smithi072 ceph-mon[34088]: pgmap v292: 97 pgs: 14 creating+peering, 11 unknown, 72 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:47:52.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:52 smithi072 ceph-mon[34088]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-09T04:47:52.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:52 smithi072 ceph-mon[34088]: Cluster is now healthy 2024-05-09T04:47:53.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.264+0000 7f4cb2836700 1 -- 172.21.15.46:0/2254172179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cac108740 msgr2=0x7f4cac1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:53.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.264+0000 7f4cb2836700 1 --2- 172.21.15.46:0/2254172179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cac108740 0x7f4cac1100a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400b0d0 tx=0x7f4ca400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.264+0000 7f4cb2836700 1 -- 172.21.15.46:0/2254172179 shutdown_connections 2024-05-09T04:47:53.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.264+0000 7f4cb2836700 1 --2- 172.21.15.46:0/2254172179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cac108740 0x7f4cac1100a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.264+0000 7f4cb2836700 1 --2- 172.21.15.46:0/2254172179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cac1034c0 0x7f4cac1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.264+0000 7f4cb2836700 1 --2- 172.21.15.46:0/2254172179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cac107df0 0x7f4cac108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.264+0000 7f4cb2836700 1 -- 172.21.15.46:0/2254172179 >> 172.21.15.46:0/2254172179 conn(0x7f4cac0fedf0 msgr2=0x7f4cac101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:53.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.264+0000 7f4cb2836700 1 -- 172.21.15.46:0/2254172179 shutdown_connections 2024-05-09T04:47:53.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.264+0000 7f4cb2836700 1 -- 172.21.15.46:0/2254172179 wait complete. 2024-05-09T04:47:53.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.265+0000 7f4cb2836700 1 Processor -- start 2024-05-09T04:47:53.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.265+0000 7f4cb2836700 1 -- start start 2024-05-09T04:47:53.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.265+0000 7f4cb2836700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cac1034c0 0x7f4cac19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:53.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.265+0000 7f4cb2836700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cac107df0 0x7f4cac19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:53.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.265+0000 7f4cb2836700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cac108740 0x7f4cac19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:53.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.265+0000 7f4cb2836700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac1133f0 con 0x7f4cac108740 2024-05-09T04:47:53.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.265+0000 7f4cb2836700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac113270 con 0x7f4cac107df0 2024-05-09T04:47:53.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.265+0000 7f4cb2836700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4cac113570 con 0x7f4cac1034c0 2024-05-09T04:47:53.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.265+0000 7f4cb0dd3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cac108740 0x7f4cac19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:53.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.265+0000 7f4cb0dd3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cac108740 0x7f4cac19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42104/0 (socket says 172.21.15.46:42104) 2024-05-09T04:47:53.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.265+0000 7f4cb0dd3700 1 -- 172.21.15.46:0/659573536 learned_addr learned my addr 172.21.15.46:0/659573536 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:47:53.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.265+0000 7f4cab7fe700 1 --2- 172.21.15.46:0/659573536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cac107df0 0x7f4cac19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:53.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.265+0000 7f4cabfff700 1 --2- 172.21.15.46:0/659573536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cac1034c0 0x7f4cac19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:53.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.266+0000 7f4cb0dd3700 1 -- 172.21.15.46:0/659573536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cac1034c0 msgr2=0x7f4cac19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:53.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.266+0000 7f4cb0dd3700 1 --2- 172.21.15.46:0/659573536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cac1034c0 0x7f4cac19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.266+0000 7f4cb0dd3700 1 -- 172.21.15.46:0/659573536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cac107df0 msgr2=0x7f4cac19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:53.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.266+0000 7f4cb0dd3700 1 --2- 172.21.15.46:0/659573536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cac107df0 0x7f4cac19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.266+0000 7f4cb0dd3700 1 -- 172.21.15.46:0/659573536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cac19fce0 con 0x7f4cac108740 2024-05-09T04:47:53.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.266+0000 7f4cabfff700 1 --2- 172.21.15.46:0/659573536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cac1034c0 0x7f4cac19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:47:53.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.266+0000 7f4cb0dd3700 1 --2- 172.21.15.46:0/659573536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cac108740 0x7f4cac19f640 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40095c0 tx=0x7f4ca4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:53.272 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.266+0000 7f4ca97fa700 1 -- 172.21.15.46:0/659573536 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca401f070 con 0x7f4cac108740 2024-05-09T04:47:53.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.266+0000 7f4cb2836700 1 -- 172.21.15.46:0/659573536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cac19ff70 con 0x7f4cac108740 2024-05-09T04:47:53.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.266+0000 7f4ca97fa700 1 -- 172.21.15.46:0/659573536 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ca4003c00 con 0x7f4cac108740 2024-05-09T04:47:53.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.266+0000 7f4ca97fa700 1 -- 172.21.15.46:0/659573536 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca401b6a0 con 0x7f4cac108740 2024-05-09T04:47:53.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.266+0000 7f4cb2836700 1 -- 172.21.15.46:0/659573536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cac1a78f0 con 0x7f4cac108740 2024-05-09T04:47:53.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.267+0000 7f4cb2836700 1 -- 172.21.15.46:0/659573536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cac0512c0 con 0x7f4cac108740 2024-05-09T04:47:53.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.269+0000 7f4ca97fa700 1 -- 172.21.15.46:0/659573536 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4ca4022070 con 0x7f4cac108740 2024-05-09T04:47:53.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.270+0000 7f4ca97fa700 1 --2- 172.21.15.46:0/659573536 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4c940b2880 0x7f4c940b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:53.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.270+0000 7f4cabfff700 1 --2- 172.21.15.46:0/659573536 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4c940b2880 0x7f4c940b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:53.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.270+0000 7f4ca97fa700 1 -- 172.21.15.46:0/659573536 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f4ca400f020 con 0x7f4cac108740 2024-05-09T04:47:53.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.270+0000 7f4cabfff700 1 --2- 172.21.15.46:0/659573536 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4c940b2880 0x7f4c940b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0099d0 tx=0x7f4c9c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:53.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.276+0000 7f4ca97fa700 1 -- 172.21.15.46:0/659573536 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4ca4029020 con 0x7f4cac108740 2024-05-09T04:47:53.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.494+0000 7f4cb2836700 1 -- 172.21.15.46:0/659573536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4cac19c170 con 0x7f4cac108740 2024-05-09T04:47:53.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.496+0000 7f4ca97fa700 1 -- 172.21.15.46:0/659573536 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22110 (secure 0 0 0) 0x7f4ca40ab650 con 0x7f4cac108740 2024-05-09T04:47:53.496 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:47:53.496 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"ecf9ae2c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:25.633876+0000","modified":"2024-05-09T04:47:50.321611+0000","last_up_change":"2024-05-09T04:46:46.418597+0000","last_in_change":"2024-05-09T04:46:25.903582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:42:53.301400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:47:43.413569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:47:47.264915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d79cd3e7-6f82-4d60-96d9-26d8146c962f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6803","nonce":1576436508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6805","nonce":1576436508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6809","nonce":1576436508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6807","nonce":1576436508}]},"public_addr":"172.21.15.46:6803/1576436508","cluster_addr":"172.21.15.46:6805/1576436508","heartbeat_back_addr":"172.21.15.46:6809/1576436508","heartbeat_front_addr":"172.21.15.46:6807/1576436508","state":["exists","up"]},{"osd":1,"uuid":"698eba80-1ebd-4076-b6f9-953d2e7daf76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6801","nonce":379039775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6803","nonce":379039775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6807","nonce":379039775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6805","nonce":379039775}]},"public_addr":"172.21.15.72:6801/379039775","cluster_addr":"172.21.15.72:6803/379039775","heartbeat_back_addr":"172.21.15.72:6807/379039775","heartbeat_front_addr":"172.21.15.72:6805/379039775","state":["exists","up"]},{"osd":2,"uuid":"c549b131-cfa7-4724-926f-83e91a752d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6801","nonce":293447227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6803","nonce":293447227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6807","nonce":293447227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6805","nonce":293447227}]},"public_addr":"172.21.15.110:6801/293447227","cluster_addr":"172.21.15.110:6803/293447227","heartbeat_back_addr":"172.21.15.110:6807/293447227","heartbeat_front_addr":"172.21.15.110:6805/293447227","state":["exists","up"]},{"osd":3,"uuid":"549bb4f7-54d9-4cbc-ad40-7f3e717964f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6811","nonce":1115835653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6813","nonce":1115835653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6817","nonce":1115835653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6815","nonce":1115835653}]},"public_addr":"172.21.15.46:6811/1115835653","cluster_addr":"172.21.15.46:6813/1115835653","heartbeat_back_addr":"172.21.15.46:6817/1115835653","heartbeat_front_addr":"172.21.15.46:6815/1115835653","state":["exists","up"]},{"osd":4,"uuid":"444cd36b-6034-46cd-9a6b-29dce4ba6405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6809","nonce":59186693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6811","nonce":59186693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6815","nonce":59186693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6813","nonce":59186693}]},"public_addr":"172.21.15.72:6809/59186693","cluster_addr":"172.21.15.72:6811/59186693","heartbeat_back_addr":"172.21.15.72:6815/59186693","heartbeat_front_addr":"172.21.15.72:6813/59186693","state":["exists","up"]},{"osd":5,"uuid":"2c2bdd2f-43cf-4208-9ae1-f912807c84e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6809","nonce":1712162502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6811","nonce":1712162502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6815","nonce":1712162502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6813","nonce":1712162502}]},"public_addr":"172.21.15.110:6809/1712162502","cluster_addr":"172.21.15.110:6811/1712162502","heartbeat_back_addr":"172.21.15.110:6815/1712162502","heartbeat_front_addr":"172.21.15.110:6813/1712162502","state":["exists","up"]},{"osd":6,"uuid":"beb0b87f-c3a4-40dd-b224-1843e9301ca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6819","nonce":2429650065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6821","nonce":2429650065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6825","nonce":2429650065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6823","nonce":2429650065}]},"public_addr":"172.21.15.46:6819/2429650065","cluster_addr":"172.21.15.46:6821/2429650065","heartbeat_back_addr":"172.21.15.46:6825/2429650065","heartbeat_front_addr":"172.21.15.46:6823/2429650065","state":["exists","up"]},{"osd":7,"uuid":"4a634675-fad7-4a43-840c-8dec2631dec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6817","nonce":337796672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6819","nonce":337796672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6823","nonce":337796672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6821","nonce":337796672}]},"public_addr":"172.21.15.72:6817/337796672","cluster_addr":"172.21.15.72:6819/337796672","heartbeat_back_addr":"172.21.15.72:6823/337796672","heartbeat_front_addr":"172.21.15.72:6821/337796672","state":["exists","up"]},{"osd":8,"uuid":"01b5f638-90f0-4cf4-8042-20d20a725669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6817","nonce":189513203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6819","nonce":189513203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6823","nonce":189513203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6821","nonce":189513203}]},"public_addr":"172.21.15.110:6817/189513203","cluster_addr":"172.21.15.110:6819/189513203","heartbeat_back_addr":"172.21.15.110:6823/189513203","heartbeat_front_addr":"172.21.15.110:6821/189513203","state":["exists","up"]},{"osd":9,"uuid":"e423de41-50ec-48f4-a4ed-216de33c4fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6827","nonce":1762262822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6829","nonce":1762262822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6833","nonce":1762262822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6831","nonce":1762262822}]},"public_addr":"172.21.15.46:6827/1762262822","cluster_addr":"172.21.15.46:6829/1762262822","heartbeat_back_addr":"172.21.15.46:6833/1762262822","heartbeat_front_addr":"172.21.15.46:6831/1762262822","state":["exists","up"]},{"osd":10,"uuid":"f594d004-9f80-4508-b127-fb693d355f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6825","nonce":38453898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6827","nonce":38453898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6831","nonce":38453898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6829","nonce":38453898}]},"public_addr":"172.21.15.72:6825/38453898","cluster_addr":"172.21.15.72:6827/38453898","heartbeat_back_addr":"172.21.15.72:6831/38453898","heartbeat_front_addr":"172.21.15.72:6829/38453898","state":["exists","up"]},{"osd":11,"uuid":"f771d29f-89e3-4f96-9c29-00e424230c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6825","nonce":4153529632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6827","nonce":4153529632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6831","nonce":4153529632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6829","nonce":4153529632}]},"public_addr":"172.21.15.110:6825/4153529632","cluster_addr":"172.21.15.110:6827/4153529632","heartbeat_back_addr":"172.21.15.110:6831/4153529632","heartbeat_front_addr":"172.21.15.110:6829/4153529632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:02.615801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:26.453477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:50.334627+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:15.357320+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:40.403549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:06.030093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:31.904091+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:57.616372+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:24.045975+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:51.160898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:17.597845+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:44.636629+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1596672353":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1846482551":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/1069571878":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6800/2738158981":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/3625745146":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6801/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/564539954":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1797199863":"2024-05-10T04:40:27.201119+0000","172.21.15.46:6801/2738158981":"2024-05-10T04:39:59.515628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:47:53.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.498+0000 7f4cb2836700 1 -- 172.21.15.46:0/659573536 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4c940b2880 msgr2=0x7f4c940b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:53.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.498+0000 7f4cb2836700 1 --2- 172.21.15.46:0/659573536 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4c940b2880 0x7f4c940b4d90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f4c9c0099d0 tx=0x7f4c9c008040 comp rx=0 tx=0).stop 2024-05-09T04:47:53.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.498+0000 7f4cb2836700 1 -- 172.21.15.46:0/659573536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cac108740 msgr2=0x7f4cac19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:53.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.498+0000 7f4cb2836700 1 --2- 172.21.15.46:0/659573536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cac108740 0x7f4cac19f640 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f4ca40095c0 tx=0x7f4ca4007a20 comp rx=0 tx=0).stop 2024-05-09T04:47:53.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.498+0000 7f4cb2836700 1 -- 172.21.15.46:0/659573536 shutdown_connections 2024-05-09T04:47:53.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.498+0000 7f4cb2836700 1 --2- 172.21.15.46:0/659573536 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4c940b2880 0x7f4c940b4d90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.498+0000 7f4cb2836700 1 --2- 172.21.15.46:0/659573536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4cac108740 0x7f4cac19f640 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.498+0000 7f4cb2836700 1 --2- 172.21.15.46:0/659573536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4cac1034c0 0x7f4cac19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.498+0000 7f4cb2836700 1 --2- 172.21.15.46:0/659573536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4cac107df0 0x7f4cac19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:53.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.498+0000 7f4cb2836700 1 -- 172.21.15.46:0/659573536 >> 172.21.15.46:0/659573536 conn(0x7f4cac0fedf0 msgr2=0x7f4cac075c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:53.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.498+0000 7f4cb2836700 1 -- 172.21.15.46:0/659573536 shutdown_connections 2024-05-09T04:47:53.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:53.498+0000 7f4cb2836700 1 -- 172.21.15.46:0/659573536 wait complete. 2024-05-09T04:47:54.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:47:54.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:54 smithi046 ceph-mon[30136]: pgmap v293: 97 pgs: 97 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T04:47:54.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:54 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/659573536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:47:54.581 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:47:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:54 smithi110 ceph-mon[34904]: pgmap v293: 97 pgs: 97 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T04:47:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:54 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/659573536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:47:54.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:54 smithi072 ceph-mon[34088]: pgmap v293: 97 pgs: 97 active+clean; 577 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T04:47:54.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:54 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/659573536' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:47:56.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.357+0000 7f6a14b9a700 1 -- 172.21.15.46:0/829836497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a101034c0 msgr2=0x7f6a101038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:56.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.357+0000 7f6a14b9a700 1 --2- 172.21.15.46:0/829836497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a101034c0 0x7f6a101038d0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6a0000b0d0 tx=0x7f6a0000b3e0 comp rx=0 tx=0).stop 2024-05-09T04:47:56.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.357+0000 7f6a14b9a700 1 -- 172.21.15.46:0/829836497 shutdown_connections 2024-05-09T04:47:56.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.357+0000 7f6a14b9a700 1 --2- 172.21.15.46:0/829836497 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a10108740 0x7f6a101100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:56.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.357+0000 7f6a14b9a700 1 --2- 172.21.15.46:0/829836497 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a101034c0 0x7f6a101038d0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:56.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.357+0000 7f6a14b9a700 1 --2- 172.21.15.46:0/829836497 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a10107df0 0x7f6a10108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:56.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.357+0000 7f6a14b9a700 1 -- 172.21.15.46:0/829836497 >> 172.21.15.46:0/829836497 conn(0x7f6a100fedc0 msgr2=0x7f6a101011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:56.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.357+0000 7f6a14b9a700 1 -- 172.21.15.46:0/829836497 shutdown_connections 2024-05-09T04:47:56.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.357+0000 7f6a14b9a700 1 -- 172.21.15.46:0/829836497 wait complete. 2024-05-09T04:47:56.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.358+0000 7f6a14b9a700 1 Processor -- start 2024-05-09T04:47:56.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.358+0000 7f6a14b9a700 1 -- start start 2024-05-09T04:47:56.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.358+0000 7f6a14b9a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a101034c0 0x7f6a1019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.358+0000 7f6a14b9a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a10107df0 0x7f6a1019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.358+0000 7f6a14b9a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a10108740 0x7f6a1019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.358+0000 7f6a14b9a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6a101133f0 con 0x7f6a10107df0 2024-05-09T04:47:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.358+0000 7f6a14b9a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10113270 con 0x7f6a101034c0 2024-05-09T04:47:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.358+0000 7f6a14b9a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6a10113570 con 0x7f6a10108740 2024-05-09T04:47:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.358+0000 7f6a0dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a10107df0 0x7f6a1019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.358+0000 7f6a0dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a10107df0 0x7f6a1019b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59648/0 (socket says 172.21.15.46:59648) 2024-05-09T04:47:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.358+0000 7f6a0dd9b700 1 -- 172.21.15.46:0/2520441613 learned_addr learned my addr 172.21.15.46:0/2520441613 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:47:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.358+0000 7f6a0e59c700 1 --2- 172.21.15.46:0/2520441613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a101034c0 0x7f6a1019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.358+0000 7f6a0ed9d700 1 --2- 172.21.15.46:0/2520441613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a10108740 0x7f6a1019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.358+0000 7f6a0dd9b700 1 -- 172.21.15.46:0/2520441613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a10108740 msgr2=0x7f6a1019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.359+0000 7f6a0dd9b700 1 --2- 172.21.15.46:0/2520441613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a10108740 0x7f6a1019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.359+0000 7f6a0dd9b700 1 -- 172.21.15.46:0/2520441613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a101034c0 msgr2=0x7f6a1019ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:56.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.359+0000 7f6a0dd9b700 1 --2- 172.21.15.46:0/2520441613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a101034c0 0x7f6a1019ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:56.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.359+0000 7f6a0dd9b700 1 -- 172.21.15.46:0/2520441613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a1019fc50 con 0x7f6a10107df0 2024-05-09T04:47:56.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.359+0000 7f6a0dd9b700 1 --2- 172.21.15.46:0/2520441613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a10107df0 0x7f6a1019b3d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400c9e0 tx=0x7f6a0400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:56.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.359+0000 7f69ff7fe700 1 -- 172.21.15.46:0/2520441613 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04007aa0 con 0x7f6a10107df0 2024-05-09T04:47:56.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.359+0000 7f6a14b9a700 1 -- 172.21.15.46:0/2520441613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a1019ff40 con 0x7f6a10107df0 2024-05-09T04:47:56.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.359+0000 7f69ff7fe700 1 -- 172.21.15.46:0/2520441613 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a04007c40 con 0x7f6a10107df0 2024-05-09T04:47:56.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.359+0000 7f69ff7fe700 1 -- 172.21.15.46:0/2520441613 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a04018660 con 0x7f6a10107df0 2024-05-09T04:47:56.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.359+0000 7f6a14b9a700 1 -- 172.21.15.46:0/2520441613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a101a7910 con 0x7f6a10107df0 2024-05-09T04:47:56.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.360+0000 7f6a14b9a700 1 -- 172.21.15.46:0/2520441613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a100512c0 con 0x7f6a10107df0 2024-05-09T04:47:56.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.362+0000 7f69ff7fe700 1 -- 172.21.15.46:0/2520441613 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6a04018800 con 0x7f6a10107df0 2024-05-09T04:47:56.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.362+0000 7f69ff7fe700 1 --2- 172.21.15.46:0/2520441613 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f69f80b2880 0x7f69f80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:56.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.363+0000 7f6a0e59c700 1 --2- 172.21.15.46:0/2520441613 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f69f80b2880 0x7f69f80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:56.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.363+0000 7f69ff7fe700 1 -- 172.21.15.46:0/2520441613 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f6a040e20c0 con 0x7f6a10107df0 2024-05-09T04:47:56.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.363+0000 7f6a0e59c700 1 --2- 172.21.15.46:0/2520441613 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f69f80b2880 0x7f69f80b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009900 tx=0x7f6a0000b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:56.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.367+0000 7f69ff7fe700 1 -- 172.21.15.46:0/2520441613 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6a04014030 con 0x7f6a10107df0 2024-05-09T04:47:56.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.587+0000 7f6a14b9a700 1 -- 172.21.15.46:0/2520441613 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6a1019c190 con 0x7f6a10107df0 2024-05-09T04:47:56.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.589+0000 7f69ff7fe700 1 -- 172.21.15.46:0/2520441613 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) ==== 74+0+22110 (secure 0 0 0) 0x7f6a040a9f30 con 0x7f6a10107df0 2024-05-09T04:47:56.589 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:47:56.589 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":80,"fsid":"ecf9ae2c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:25.633876+0000","modified":"2024-05-09T04:47:50.321611+0000","last_up_change":"2024-05-09T04:46:46.418597+0000","last_in_change":"2024-05-09T04:46:25.903582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:42:53.301400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:47:43.413569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:47:47.264915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d79cd3e7-6f82-4d60-96d9-26d8146c962f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6803","nonce":1576436508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6805","nonce":1576436508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6809","nonce":1576436508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6807","nonce":1576436508}]},"public_addr":"172.21.15.46:6803/1576436508","cluster_addr":"172.21.15.46:6805/1576436508","heartbeat_back_addr":"172.21.15.46:6809/1576436508","heartbeat_front_addr":"172.21.15.46:6807/1576436508","state":["exists","up"]},{"osd":1,"uuid":"698eba80-1ebd-4076-b6f9-953d2e7daf76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6801","nonce":379039775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6803","nonce":379039775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6807","nonce":379039775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6805","nonce":379039775}]},"public_addr":"172.21.15.72:6801/379039775","cluster_addr":"172.21.15.72:6803/379039775","heartbeat_back_addr":"172.21.15.72:6807/379039775","heartbeat_front_addr":"172.21.15.72:6805/379039775","state":["exists","up"]},{"osd":2,"uuid":"c549b131-cfa7-4724-926f-83e91a752d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6801","nonce":293447227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6803","nonce":293447227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6807","nonce":293447227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6805","nonce":293447227}]},"public_addr":"172.21.15.110:6801/293447227","cluster_addr":"172.21.15.110:6803/293447227","heartbeat_back_addr":"172.21.15.110:6807/293447227","heartbeat_front_addr":"172.21.15.110:6805/293447227","state":["exists","up"]},{"osd":3,"uuid":"549bb4f7-54d9-4cbc-ad40-7f3e717964f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6811","nonce":1115835653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6813","nonce":1115835653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6817","nonce":1115835653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6815","nonce":1115835653}]},"public_addr":"172.21.15.46:6811/1115835653","cluster_addr":"172.21.15.46:6813/1115835653","heartbeat_back_addr":"172.21.15.46:6817/1115835653","heartbeat_front_addr":"172.21.15.46:6815/1115835653","state":["exists","up"]},{"osd":4,"uuid":"444cd36b-6034-46cd-9a6b-29dce4ba6405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6809","nonce":59186693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6811","nonce":59186693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6815","nonce":59186693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6813","nonce":59186693}]},"public_addr":"172.21.15.72:6809/59186693","cluster_addr":"172.21.15.72:6811/59186693","heartbeat_back_addr":"172.21.15.72:6815/59186693","heartbeat_front_addr":"172.21.15.72:6813/59186693","state":["exists","up"]},{"osd":5,"uuid":"2c2bdd2f-43cf-4208-9ae1-f912807c84e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6809","nonce":1712162502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6811","nonce":1712162502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6815","nonce":1712162502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6813","nonce":1712162502}]},"public_addr":"172.21.15.110:6809/1712162502","cluster_addr":"172.21.15.110:6811/1712162502","heartbeat_back_addr":"172.21.15.110:6815/1712162502","heartbeat_front_addr":"172.21.15.110:6813/1712162502","state":["exists","up"]},{"osd":6,"uuid":"beb0b87f-c3a4-40dd-b224-1843e9301ca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6819","nonce":2429650065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6821","nonce":2429650065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6825","nonce":2429650065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6823","nonce":2429650065}]},"public_addr":"172.21.15.46:6819/2429650065","cluster_addr":"172.21.15.46:6821/2429650065","heartbeat_back_addr":"172.21.15.46:6825/2429650065","heartbeat_front_addr":"172.21.15.46:6823/2429650065","state":["exists","up"]},{"osd":7,"uuid":"4a634675-fad7-4a43-840c-8dec2631dec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6817","nonce":337796672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6819","nonce":337796672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6823","nonce":337796672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6821","nonce":337796672}]},"public_addr":"172.21.15.72:6817/337796672","cluster_addr":"172.21.15.72:6819/337796672","heartbeat_back_addr":"172.21.15.72:6823/337796672","heartbeat_front_addr":"172.21.15.72:6821/337796672","state":["exists","up"]},{"osd":8,"uuid":"01b5f638-90f0-4cf4-8042-20d20a725669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6817","nonce":189513203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6819","nonce":189513203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6823","nonce":189513203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6821","nonce":189513203}]},"public_addr":"172.21.15.110:6817/189513203","cluster_addr":"172.21.15.110:6819/189513203","heartbeat_back_addr":"172.21.15.110:6823/189513203","heartbeat_front_addr":"172.21.15.110:6821/189513203","state":["exists","up"]},{"osd":9,"uuid":"e423de41-50ec-48f4-a4ed-216de33c4fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6827","nonce":1762262822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6829","nonce":1762262822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6833","nonce":1762262822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6831","nonce":1762262822}]},"public_addr":"172.21.15.46:6827/1762262822","cluster_addr":"172.21.15.46:6829/1762262822","heartbeat_back_addr":"172.21.15.46:6833/1762262822","heartbeat_front_addr":"172.21.15.46:6831/1762262822","state":["exists","up"]},{"osd":10,"uuid":"f594d004-9f80-4508-b127-fb693d355f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6825","nonce":38453898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6827","nonce":38453898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6831","nonce":38453898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6829","nonce":38453898}]},"public_addr":"172.21.15.72:6825/38453898","cluster_addr":"172.21.15.72:6827/38453898","heartbeat_back_addr":"172.21.15.72:6831/38453898","heartbeat_front_addr":"172.21.15.72:6829/38453898","state":["exists","up"]},{"osd":11,"uuid":"f771d29f-89e3-4f96-9c29-00e424230c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6825","nonce":4153529632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6827","nonce":4153529632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6831","nonce":4153529632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6829","nonce":4153529632}]},"public_addr":"172.21.15.110:6825/4153529632","cluster_addr":"172.21.15.110:6827/4153529632","heartbeat_back_addr":"172.21.15.110:6831/4153529632","heartbeat_front_addr":"172.21.15.110:6829/4153529632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:02.615801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:26.453477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:50.334627+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:15.357320+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:40.403549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:06.030093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:31.904091+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:57.616372+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:24.045975+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:51.160898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:17.597845+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:44.636629+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1596672353":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1846482551":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/1069571878":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6800/2738158981":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/3625745146":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6801/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/564539954":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1797199863":"2024-05-10T04:40:27.201119+0000","172.21.15.46:6801/2738158981":"2024-05-10T04:39:59.515628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:47:56.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.591+0000 7f6a14b9a700 1 -- 172.21.15.46:0/2520441613 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f69f80b2880 msgr2=0x7f69f80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:56.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.591+0000 7f6a14b9a700 1 --2- 172.21.15.46:0/2520441613 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f69f80b2880 0x7f69f80b4d90 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6a00009900 tx=0x7f6a0000b040 comp rx=0 tx=0).stop 2024-05-09T04:47:56.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.591+0000 7f6a14b9a700 1 -- 172.21.15.46:0/2520441613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a10107df0 msgr2=0x7f6a1019b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:56.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.591+0000 7f6a14b9a700 1 --2- 172.21.15.46:0/2520441613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a10107df0 0x7f6a1019b3d0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6a0400c9e0 tx=0x7f6a0400ccf0 comp rx=0 tx=0).stop 2024-05-09T04:47:56.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.591+0000 7f6a14b9a700 1 -- 172.21.15.46:0/2520441613 shutdown_connections 2024-05-09T04:47:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.591+0000 7f6a14b9a700 1 --2- 172.21.15.46:0/2520441613 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f69f80b2880 0x7f69f80b4d90 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.591+0000 7f6a14b9a700 1 --2- 172.21.15.46:0/2520441613 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6a10108740 0x7f6a1019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.591+0000 7f6a14b9a700 1 --2- 172.21.15.46:0/2520441613 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6a101034c0 0x7f6a1019ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.591+0000 7f6a14b9a700 1 --2- 172.21.15.46:0/2520441613 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6a10107df0 0x7f6a1019b3d0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.591+0000 7f6a14b9a700 1 -- 172.21.15.46:0/2520441613 >> 172.21.15.46:0/2520441613 conn(0x7f6a100fedc0 msgr2=0x7f6a1006b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.591+0000 7f6a14b9a700 1 -- 172.21.15.46:0/2520441613 shutdown_connections 2024-05-09T04:47:56.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:56.591+0000 7f6a14b9a700 1 -- 172.21.15.46:0/2520441613 wait complete. 2024-05-09T04:47:56.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:56 smithi110 ceph-mon[34904]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-09T04:47:56.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:56 smithi046 ceph-mon[30136]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-09T04:47:56.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:56 smithi072 ceph-mon[34088]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2024-05-09T04:47:57.286 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-09T04:47:57.548 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:47:57.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:57 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2520441613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:47:57.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:57 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2520441613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:47:57.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:57 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2520441613' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:47:58.679 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:47:58 smithi072 ceph-mon[34088]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T04:47:58.697 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:47:58 smithi046 ceph-mon[30136]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T04:47:58.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:47:58 smithi110 ceph-mon[34904]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-09T04:47:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.243+0000 7f39aa557700 1 -- 172.21.15.46:0/1321623444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a4107df0 msgr2=0x7f39a4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.243+0000 7f39aa557700 1 --2- 172.21.15.46:0/1321623444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a4107df0 0x7f39a4108200 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f39980099c0 tx=0x7f3998009cd0 comp rx=0 tx=0).stop 2024-05-09T04:47:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.243+0000 7f39aa557700 1 -- 172.21.15.46:0/1321623444 shutdown_connections 2024-05-09T04:47:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.243+0000 7f39aa557700 1 --2- 172.21.15.46:0/1321623444 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39a4108740 0x7f39a41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:59.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.243+0000 7f39aa557700 1 --2- 172.21.15.46:0/1321623444 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39a41034c0 0x7f39a41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:59.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.243+0000 7f39aa557700 1 --2- 172.21.15.46:0/1321623444 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a4107df0 0x7f39a4108200 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:59.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.243+0000 7f39aa557700 1 -- 172.21.15.46:0/1321623444 >> 172.21.15.46:0/1321623444 conn(0x7f39a40fedc0 msgr2=0x7f39a41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:47:59.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.243+0000 7f39aa557700 1 -- 172.21.15.46:0/1321623444 shutdown_connections 2024-05-09T04:47:59.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.243+0000 7f39aa557700 1 -- 172.21.15.46:0/1321623444 wait complete. 2024-05-09T04:47:59.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.244+0000 7f39aa557700 1 Processor -- start 2024-05-09T04:47:59.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.244+0000 7f39aa557700 1 -- start start 2024-05-09T04:47:59.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.244+0000 7f39aa557700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a41034c0 0x7f39a419bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:59.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.244+0000 7f39aa557700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39a4107df0 0x7f39a419c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:59.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.244+0000 7f39aa557700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39a4108740 0x7f39a419c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:59.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.244+0000 7f39aa557700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39a4113350 con 0x7f39a41034c0 2024-05-09T04:47:59.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.244+0000 7f39aa557700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f39a41131d0 con 0x7f39a4107df0 2024-05-09T04:47:59.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.244+0000 7f39aa557700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f39a41134d0 con 0x7f39a4108740 2024-05-09T04:47:59.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.245+0000 7f39a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a41034c0 0x7f39a419bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:59.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.245+0000 7f39a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a41034c0 0x7f39a419bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59668/0 (socket says 172.21.15.46:59668) 2024-05-09T04:47:59.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.245+0000 7f39a3fff700 1 -- 172.21.15.46:0/1386533442 learned_addr learned my addr 172.21.15.46:0/1386533442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:47:59.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.245+0000 7f39a37fe700 1 --2- 172.21.15.46:0/1386533442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39a4107df0 0x7f39a419c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:59.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.245+0000 7f39a8af4700 1 --2- 172.21.15.46:0/1386533442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39a4108740 0x7f39a419c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:59.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.245+0000 7f39a3fff700 1 -- 172.21.15.46:0/1386533442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39a4108740 msgr2=0x7f39a419c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:59.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.245+0000 7f39a3fff700 1 --2- 172.21.15.46:0/1386533442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39a4108740 0x7f39a419c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:59.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.245+0000 7f39a3fff700 1 -- 172.21.15.46:0/1386533442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39a4107df0 msgr2=0x7f39a419c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:47:59.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.245+0000 7f39a3fff700 1 --2- 172.21.15.46:0/1386533442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39a4107df0 0x7f39a419c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:47:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.245+0000 7f39a3fff700 1 -- 172.21.15.46:0/1386533442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a41a6f80 con 0x7f39a41034c0 2024-05-09T04:47:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.245+0000 7f39a37fe700 1 --2- 172.21.15.46:0/1386533442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39a4107df0 0x7f39a419c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:47:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.245+0000 7f39a3fff700 1 --2- 172.21.15.46:0/1386533442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a41034c0 0x7f39a419bc30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f399400e9a0 tx=0x7f399400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.245+0000 7f39a17fa700 1 -- 172.21.15.46:0/1386533442 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399400ca10 con 0x7f39a41034c0 2024-05-09T04:47:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.245+0000 7f39aa557700 1 -- 172.21.15.46:0/1386533442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a41a7270 con 0x7f39a41034c0 2024-05-09T04:47:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.246+0000 7f39a17fa700 1 -- 172.21.15.46:0/1386533442 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f399400cbb0 con 0x7f39a41034c0 2024-05-09T04:47:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.246+0000 7f39a17fa700 1 -- 172.21.15.46:0/1386533442 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3994018680 con 0x7f39a41034c0 2024-05-09T04:47:59.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.246+0000 7f39aa557700 1 -- 172.21.15.46:0/1386533442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a41a77d0 con 0x7f39a41034c0 2024-05-09T04:47:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.247+0000 7f39aa557700 1 -- 172.21.15.46:0/1386533442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a40512c0 con 0x7f39a41034c0 2024-05-09T04:47:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.248+0000 7f39a17fa700 1 -- 172.21.15.46:0/1386533442 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3994010430 con 0x7f39a41034c0 2024-05-09T04:47:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.249+0000 7f39a17fa700 1 --2- 172.21.15.46:0/1386533442 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f398c0b2900 0x7f398c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:47:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.249+0000 7f39a37fe700 1 --2- 172.21.15.46:0/1386533442 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f398c0b2900 0x7f398c0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:47:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.249+0000 7f39a17fa700 1 -- 172.21.15.46:0/1386533442 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f39940e21f0 con 0x7f39a41034c0 2024-05-09T04:47:59.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.249+0000 7f39a37fe700 1 --2- 172.21.15.46:0/1386533442 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f398c0b2900 0x7f398c0b4e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3998000c00 tx=0x7f399801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:47:59.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.254+0000 7f39a17fa700 1 -- 172.21.15.46:0/1386533442 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f39940aa060 con 0x7f39a41034c0 2024-05-09T04:47:59.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:47:59.517+0000 7f39aa557700 1 -- 172.21.15.46:0/1386533442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f39a402e7b0 con 0x7f39a41034c0 2024-05-09T04:48:00.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:00.435+0000 7f39a17fa700 1 -- 172.21.15.46:0/1386533442 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7f39940b20e0 con 0x7f39a41034c0 2024-05-09T04:48:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:00.438+0000 7f39aa557700 1 -- 172.21.15.46:0/1386533442 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f398c0b2900 msgr2=0x7f398c0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:00.438+0000 7f39aa557700 1 --2- 172.21.15.46:0/1386533442 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f398c0b2900 0x7f398c0b4e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3998000c00 tx=0x7f399801a040 comp rx=0 tx=0).stop 2024-05-09T04:48:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:00.438+0000 7f39aa557700 1 -- 172.21.15.46:0/1386533442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a41034c0 msgr2=0x7f39a419bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:00.438+0000 7f39aa557700 1 --2- 172.21.15.46:0/1386533442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a41034c0 0x7f39a419bc30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f399400e9a0 tx=0x7f399400ecb0 comp rx=0 tx=0).stop 2024-05-09T04:48:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:00.438+0000 7f39aa557700 1 -- 172.21.15.46:0/1386533442 shutdown_connections 2024-05-09T04:48:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:00.438+0000 7f39aa557700 1 --2- 172.21.15.46:0/1386533442 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f398c0b2900 0x7f398c0b4e10 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:00.438+0000 7f39aa557700 1 --2- 172.21.15.46:0/1386533442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39a4108740 0x7f39a419c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:00.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:00.438+0000 7f39aa557700 1 --2- 172.21.15.46:0/1386533442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39a41034c0 0x7f39a419bc30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:00.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:00.438+0000 7f39aa557700 1 --2- 172.21.15.46:0/1386533442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f39a4107df0 0x7f39a419c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:00.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:00.438+0000 7f39aa557700 1 -- 172.21.15.46:0/1386533442 >> 172.21.15.46:0/1386533442 conn(0x7f39a40fedc0 msgr2=0x7f39a41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:00.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:00.438+0000 7f39aa557700 1 -- 172.21.15.46:0/1386533442 shutdown_connections 2024-05-09T04:48:00.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:00.438+0000 7f39aa557700 1 -- 172.21.15.46:0/1386533442 wait complete. 2024-05-09T04:48:00.767 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:00 smithi046 ceph-mon[30136]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T04:48:00.767 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:00 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1386533442' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T04:48:00.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:00 smithi110 ceph-mon[34904]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T04:48:00.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:00 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1386533442' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T04:48:00.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:00 smithi072 ceph-mon[34088]: pgmap v296: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 4 op/s 2024-05-09T04:48:00.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:00 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1386533442' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-09T04:48:01.137 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-05-09T04:48:01.137 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-05-09T04:48:01.399 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:01.764 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:01 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1386533442' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T04:48:01.765 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:01 smithi046 ceph-mon[30136]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:48:01.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:01 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1386533442' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T04:48:01.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:01 smithi110 ceph-mon[34904]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:48:01.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:01 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1386533442' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-09T04:48:01.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:01 smithi072 ceph-mon[34088]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:48:02.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:02 smithi046 ceph-mon[30136]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-09T04:48:02.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:02 smithi110 ceph-mon[34904]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-09T04:48:02.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:02 smithi072 ceph-mon[34088]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-05-09T04:48:03.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.176+0000 7f315e114700 1 -- 172.21.15.46:0/4135601607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31581045e0 msgr2=0x7f31581049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:03.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.176+0000 7f315e114700 1 --2- 172.21.15.46:0/4135601607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31581045e0 0x7f31581049f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f31480099c0 tx=0x7f3148009cd0 comp rx=0 tx=0).stop 2024-05-09T04:48:03.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.176+0000 7f315e114700 1 -- 172.21.15.46:0/4135601607 shutdown_connections 2024-05-09T04:48:03.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.176+0000 7f315e114700 1 --2- 172.21.15.46:0/4135601607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3158109860 0x7f31581111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:03.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.176+0000 7f315e114700 1 --2- 172.21.15.46:0/4135601607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f31581045e0 0x7f31581049f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:03.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.176+0000 7f315e114700 1 --2- 172.21.15.46:0/4135601607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3158108f10 0x7f3158109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:03.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.176+0000 7f315e114700 1 -- 172.21.15.46:0/4135601607 >> 172.21.15.46:0/4135601607 conn(0x7f31580ffed0 msgr2=0x7f3158102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:03.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.176+0000 7f315e114700 1 -- 172.21.15.46:0/4135601607 shutdown_connections 2024-05-09T04:48:03.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.176+0000 7f315e114700 1 -- 172.21.15.46:0/4135601607 wait complete. 2024-05-09T04:48:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.177+0000 7f315e114700 1 Processor -- start 2024-05-09T04:48:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.177+0000 7f315e114700 1 -- start start 2024-05-09T04:48:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.177+0000 7f315e114700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31581045e0 0x7f3158076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.177+0000 7f315e114700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3158108f10 0x7f3158077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.177+0000 7f315e114700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3158109860 0x7f3158077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.177+0000 7f315e114700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3158114510 con 0x7f3158109860 2024-05-09T04:48:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.177+0000 7f315e114700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3158114390 con 0x7f31581045e0 2024-05-09T04:48:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.177+0000 7f315e114700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3158114690 con 0x7f3158108f10 2024-05-09T04:48:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.177+0000 7f3157fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3158109860 0x7f3158077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.177+0000 7f3157fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3158109860 0x7f3158077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:59692/0 (socket says 172.21.15.46:59692) 2024-05-09T04:48:03.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.177+0000 7f3157fff700 1 -- 172.21.15.46:0/3661697770 learned_addr learned my addr 172.21.15.46:0/3661697770 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.177+0000 7f3156ffd700 1 --2- 172.21.15.46:0/3661697770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3158108f10 0x7f3158077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.178+0000 7f31577fe700 1 --2- 172.21.15.46:0/3661697770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31581045e0 0x7f3158076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.178+0000 7f3157fff700 1 -- 172.21.15.46:0/3661697770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3158108f10 msgr2=0x7f3158077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.178+0000 7f3157fff700 1 --2- 172.21.15.46:0/3661697770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3158108f10 0x7f3158077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.178+0000 7f3157fff700 1 -- 172.21.15.46:0/3661697770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31581045e0 msgr2=0x7f3158076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.178+0000 7f3157fff700 1 --2- 172.21.15.46:0/3661697770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31581045e0 0x7f3158076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.178+0000 7f3157fff700 1 -- 172.21.15.46:0/3661697770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31581a9240 con 0x7f3158109860 2024-05-09T04:48:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.178+0000 7f3157fff700 1 --2- 172.21.15.46:0/3661697770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3158109860 0x7f3158077650 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f315000b970 tx=0x7f315000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.178+0000 7f3154ff9700 1 -- 172.21.15.46:0/3661697770 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315000d860 con 0x7f3158109860 2024-05-09T04:48:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.178+0000 7f3154ff9700 1 -- 172.21.15.46:0/3661697770 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f315000e430 con 0x7f3158109860 2024-05-09T04:48:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.178+0000 7f315e114700 1 -- 172.21.15.46:0/3661697770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31581a9530 con 0x7f3158109860 2024-05-09T04:48:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.178+0000 7f3154ff9700 1 -- 172.21.15.46:0/3661697770 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3150017660 con 0x7f3158109860 2024-05-09T04:48:03.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.178+0000 7f315e114700 1 -- 172.21.15.46:0/3661697770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31581a9a70 con 0x7f3158109860 2024-05-09T04:48:03.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.180+0000 7f315e114700 1 -- 172.21.15.46:0/3661697770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31580512c0 con 0x7f3158109860 2024-05-09T04:48:03.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.181+0000 7f3154ff9700 1 -- 172.21.15.46:0/3661697770 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f315000da00 con 0x7f3158109860 2024-05-09T04:48:03.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.182+0000 7f3154ff9700 1 --2- 172.21.15.46:0/3661697770 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f31400b2900 0x7f31400b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:03.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.182+0000 7f31577fe700 1 --2- 172.21.15.46:0/3661697770 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f31400b2900 0x7f31400b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:03.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.182+0000 7f3154ff9700 1 -- 172.21.15.46:0/3661697770 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f31500e1c00 con 0x7f3158109860 2024-05-09T04:48:03.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.182+0000 7f31577fe700 1 --2- 172.21.15.46:0/3661697770 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f31400b2900 0x7f31400b4e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3148009990 tx=0x7f3148000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:03.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.187+0000 7f3154ff9700 1 -- 172.21.15.46:0/3661697770 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f31500e8080 con 0x7f3158109860 2024-05-09T04:48:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:03.453+0000 7f315e114700 1 -- 172.21.15.46:0/3661697770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f315802e5f0 con 0x7f3158109860 2024-05-09T04:48:04.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:04.449+0000 7f3154ff9700 1 -- 172.21.15.46:0/3661697770 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v17) ==== 105+0+0 (secure 0 0 0) 0x7f3150013030 con 0x7f3158109860 2024-05-09T04:48:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:04.454+0000 7f315e114700 1 -- 172.21.15.46:0/3661697770 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f31400b2900 msgr2=0x7f31400b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:04.454+0000 7f315e114700 1 --2- 172.21.15.46:0/3661697770 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f31400b2900 0x7f31400b4e10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3148009990 tx=0x7f3148000bc0 comp rx=0 tx=0).stop 2024-05-09T04:48:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:04.454+0000 7f315e114700 1 -- 172.21.15.46:0/3661697770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3158109860 msgr2=0x7f3158077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:04.454+0000 7f315e114700 1 --2- 172.21.15.46:0/3661697770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3158109860 0x7f3158077650 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f315000b970 tx=0x7f315000bc80 comp rx=0 tx=0).stop 2024-05-09T04:48:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:04.454+0000 7f315e114700 1 -- 172.21.15.46:0/3661697770 shutdown_connections 2024-05-09T04:48:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:04.454+0000 7f315e114700 1 --2- 172.21.15.46:0/3661697770 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f31400b2900 0x7f31400b4e10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:04.454+0000 7f315e114700 1 --2- 172.21.15.46:0/3661697770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3158109860 0x7f3158077650 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:04.454+0000 7f315e114700 1 --2- 172.21.15.46:0/3661697770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f31581045e0 0x7f3158076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:04.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:04.454+0000 7f315e114700 1 --2- 172.21.15.46:0/3661697770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3158108f10 0x7f3158077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:04.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:04.454+0000 7f315e114700 1 -- 172.21.15.46:0/3661697770 >> 172.21.15.46:0/3661697770 conn(0x7f31580ffed0 msgr2=0x7f3158108730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:04.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:04.454+0000 7f315e114700 1 -- 172.21.15.46:0/3661697770 shutdown_connections 2024-05-09T04:48:04.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:04.454+0000 7f315e114700 1 -- 172.21.15.46:0/3661697770 wait complete. 2024-05-09T04:48:04.760 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:04 smithi046 ceph-mon[30136]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-09T04:48:04.760 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:04 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3661697770' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-09T04:48:04.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:04 smithi110 ceph-mon[34904]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-09T04:48:04.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:04 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3661697770' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-09T04:48:04.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:04 smithi072 ceph-mon[34088]: pgmap v298: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-09T04:48:04.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:04 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3661697770' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-05-09T04:48:05.214 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-09T04:48:05.478 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:05.744 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:05 smithi046 ceph-mon[30136]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T04:48:05.744 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:05 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3661697770' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-09T04:48:05.744 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:05 smithi046 ceph-mon[30136]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T04:48:05.744 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:05 smithi046 ceph-mon[30136]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:48:05.744 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:05 smithi046 ceph-mon[30136]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-09T04:48:05.744 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:05 smithi046 ceph-mon[30136]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-09T04:48:05.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:05 smithi110 ceph-mon[34904]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T04:48:05.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:05 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3661697770' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-09T04:48:05.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:05 smithi110 ceph-mon[34904]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T04:48:05.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:05 smithi110 ceph-mon[34904]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:48:05.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:05 smithi110 ceph-mon[34904]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-09T04:48:05.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:05 smithi110 ceph-mon[34904]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-09T04:48:05.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:05 smithi072 ceph-mon[34088]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-09T04:48:05.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:05 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3661697770' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-05-09T04:48:05.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:05 smithi072 ceph-mon[34088]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-09T04:48:05.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:05 smithi072 ceph-mon[34088]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-09T04:48:05.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:05 smithi072 ceph-mon[34088]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-09T04:48:05.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:05 smithi072 ceph-mon[34088]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-09T04:48:06.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:06 smithi046 ceph-mon[30136]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-05-09T04:48:06.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:06 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] up:active 2024-05-09T04:48:06.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:06 smithi046 ceph-mon[30136]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T04:48:06.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:06 smithi046 ceph-mon[30136]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T04:48:06.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:06 smithi046 ceph-mon[30136]: Cluster is now healthy 2024-05-09T04:48:06.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:06 smithi046 ceph-mon[30136]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-09T04:48:06.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:06 smithi046 ceph-mon[30136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-09T04:48:06.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:06 smithi046 ceph-mon[30136]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-09T04:48:06.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:06 smithi110 ceph-mon[34904]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-05-09T04:48:06.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:06 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] up:active 2024-05-09T04:48:06.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:06 smithi110 ceph-mon[34904]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T04:48:06.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:06 smithi110 ceph-mon[34904]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T04:48:06.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:06 smithi110 ceph-mon[34904]: Cluster is now healthy 2024-05-09T04:48:06.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:06 smithi110 ceph-mon[34904]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-09T04:48:06.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:06 smithi110 ceph-mon[34904]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-09T04:48:06.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:06 smithi110 ceph-mon[34904]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-09T04:48:06.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:06 smithi072 ceph-mon[34088]: pgmap v299: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-05-09T04:48:06.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:06 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] up:active 2024-05-09T04:48:06.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:06 smithi072 ceph-mon[34088]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-09T04:48:06.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:06 smithi072 ceph-mon[34088]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-09T04:48:06.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:06 smithi072 ceph-mon[34088]: Cluster is now healthy 2024-05-09T04:48:06.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:06 smithi072 ceph-mon[34088]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-09T04:48:06.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:06 smithi072 ceph-mon[34088]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-09T04:48:06.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:06 smithi072 ceph-mon[34088]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-09T04:48:07.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.240+0000 7f65bda66700 1 -- 172.21.15.46:0/801723221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b81054e0 msgr2=0x7f65b81058f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.240+0000 7f65bda66700 1 --2- 172.21.15.46:0/801723221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b81054e0 0x7f65b81058f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f65ac009990 tx=0x7f65ac009ca0 comp rx=0 tx=0).stop 2024-05-09T04:48:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.240+0000 7f65bda66700 1 -- 172.21.15.46:0/801723221 shutdown_connections 2024-05-09T04:48:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.240+0000 7f65bda66700 1 --2- 172.21.15.46:0/801723221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65b810e910 0x7f65b8110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.240+0000 7f65bda66700 1 --2- 172.21.15.46:0/801723221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b81054e0 0x7f65b81058f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.240+0000 7f65bda66700 1 --2- 172.21.15.46:0/801723221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65b8105e30 0x7f65b810e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.240+0000 7f65bda66700 1 -- 172.21.15.46:0/801723221 >> 172.21.15.46:0/801723221 conn(0x7f65b80fedf0 msgr2=0x7f65b8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.240+0000 7f65bda66700 1 -- 172.21.15.46:0/801723221 shutdown_connections 2024-05-09T04:48:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.240+0000 7f65bda66700 1 -- 172.21.15.46:0/801723221 wait complete. 2024-05-09T04:48:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.241+0000 7f65bda66700 1 Processor -- start 2024-05-09T04:48:07.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.241+0000 7f65bda66700 1 -- start start 2024-05-09T04:48:07.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.241+0000 7f65bda66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b81054e0 0x7f65b819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:07.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.241+0000 7f65bda66700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65b8105e30 0x7f65b819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:07.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.241+0000 7f65bda66700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65b810e910 0x7f65b819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:07.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65bda66700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65b81144d0 con 0x7f65b81054e0 2024-05-09T04:48:07.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65bda66700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8114350 con 0x7f65b8105e30 2024-05-09T04:48:07.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65bda66700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f65b8114650 con 0x7f65b810e910 2024-05-09T04:48:07.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65b77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b81054e0 0x7f65b819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:07.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65b77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b81054e0 0x7f65b819aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39426/0 (socket says 172.21.15.46:39426) 2024-05-09T04:48:07.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65b77fe700 1 -- 172.21.15.46:0/3059632891 learned_addr learned my addr 172.21.15.46:0/3059632891 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:07.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65b6ffd700 1 --2- 172.21.15.46:0/3059632891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65b8105e30 0x7f65b819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:07.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65b7fff700 1 --2- 172.21.15.46:0/3059632891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65b810e910 0x7f65b819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:07.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65b77fe700 1 -- 172.21.15.46:0/3059632891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65b810e910 msgr2=0x7f65b819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:07.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65b77fe700 1 --2- 172.21.15.46:0/3059632891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65b810e910 0x7f65b819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:07.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65b77fe700 1 -- 172.21.15.46:0/3059632891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65b8105e30 msgr2=0x7f65b819b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65b77fe700 1 --2- 172.21.15.46:0/3059632891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65b8105e30 0x7f65b819b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65b77fe700 1 -- 172.21.15.46:0/3059632891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65b819fd70 con 0x7f65b81054e0 2024-05-09T04:48:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65b6ffd700 1 --2- 172.21.15.46:0/3059632891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65b8105e30 0x7f65b819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:48:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65b77fe700 1 --2- 172.21.15.46:0/3059632891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b81054e0 0x7f65b819aea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f65ac000c00 tx=0x7f65ac009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.242+0000 7f65b4ff9700 1 -- 172.21.15.46:0/3059632891 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac027070 con 0x7f65b81054e0 2024-05-09T04:48:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.243+0000 7f65b4ff9700 1 -- 172.21.15.46:0/3059632891 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65ac022470 con 0x7f65b81054e0 2024-05-09T04:48:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.243+0000 7f65bda66700 1 -- 172.21.15.46:0/3059632891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65b81a0000 con 0x7f65b81054e0 2024-05-09T04:48:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.243+0000 7f65b4ff9700 1 -- 172.21.15.46:0/3059632891 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ac017650 con 0x7f65b81054e0 2024-05-09T04:48:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.243+0000 7f65bda66700 1 -- 172.21.15.46:0/3059632891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65b81a04e0 con 0x7f65b81054e0 2024-05-09T04:48:07.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.244+0000 7f65bda66700 1 -- 172.21.15.46:0/3059632891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65b80512c0 con 0x7f65b81054e0 2024-05-09T04:48:07.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.245+0000 7f65b4ff9700 1 -- 172.21.15.46:0/3059632891 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f65ac02a070 con 0x7f65b81054e0 2024-05-09T04:48:07.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.246+0000 7f65b4ff9700 1 --2- 172.21.15.46:0/3059632891 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f65a00b2880 0x7f65a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:07.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.246+0000 7f65b6ffd700 1 --2- 172.21.15.46:0/3059632891 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f65a00b2880 0x7f65a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:07.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.246+0000 7f65b4ff9700 1 -- 172.21.15.46:0/3059632891 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f65ac0e2ff0 con 0x7f65b81054e0 2024-05-09T04:48:07.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.247+0000 7f65b6ffd700 1 --2- 172.21.15.46:0/3059632891 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f65a00b2880 0x7f65a00b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f65b8102280 tx=0x7f65a8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:07.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.251+0000 7f65b4ff9700 1 -- 172.21.15.46:0/3059632891 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f65ac013070 con 0x7f65b81054e0 2024-05-09T04:48:07.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:07.521+0000 7f65bda66700 1 -- 172.21.15.46:0/3059632891 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f65b802e5f0 con 0x7f65b81054e0 2024-05-09T04:48:07.521 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:07 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] up:active 2024-05-09T04:48:07.522 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:07 smithi046 ceph-mon[30136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-09T04:48:07.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:07 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] up:active 2024-05-09T04:48:07.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:07 smithi110 ceph-mon[34904]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-09T04:48:07.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:07 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] up:active 2024-05-09T04:48:07.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:07 smithi072 ceph-mon[34088]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-09T04:48:08.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:08.483+0000 7f65b4ff9700 1 -- 172.21.15.46:0/3059632891 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f65ac0aae90 con 0x7f65b81054e0 2024-05-09T04:48:08.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:08.486+0000 7f65bda66700 1 -- 172.21.15.46:0/3059632891 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f65a00b2880 msgr2=0x7f65a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:08.486+0000 7f65bda66700 1 --2- 172.21.15.46:0/3059632891 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f65a00b2880 0x7f65a00b4d90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f65b8102280 tx=0x7f65a8009190 comp rx=0 tx=0).stop 2024-05-09T04:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:08.486+0000 7f65bda66700 1 -- 172.21.15.46:0/3059632891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b81054e0 msgr2=0x7f65b819aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:08.486+0000 7f65bda66700 1 --2- 172.21.15.46:0/3059632891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b81054e0 0x7f65b819aea0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f65ac000c00 tx=0x7f65ac009580 comp rx=0 tx=0).stop 2024-05-09T04:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:08.486+0000 7f65bda66700 1 -- 172.21.15.46:0/3059632891 shutdown_connections 2024-05-09T04:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:08.487+0000 7f65bda66700 1 --2- 172.21.15.46:0/3059632891 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f65a00b2880 0x7f65a00b4d90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:08.487+0000 7f65bda66700 1 --2- 172.21.15.46:0/3059632891 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65b810e910 0x7f65b819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:08.487+0000 7f65bda66700 1 --2- 172.21.15.46:0/3059632891 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65b81054e0 0x7f65b819aea0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:08.487+0000 7f65bda66700 1 --2- 172.21.15.46:0/3059632891 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65b8105e30 0x7f65b819b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:08.487+0000 7f65bda66700 1 -- 172.21.15.46:0/3059632891 >> 172.21.15.46:0/3059632891 conn(0x7f65b80fedf0 msgr2=0x7f65b80ff480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:08.487+0000 7f65bda66700 1 -- 172.21.15.46:0/3059632891 shutdown_connections 2024-05-09T04:48:08.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:08.487+0000 7f65bda66700 1 -- 172.21.15.46:0/3059632891 wait complete. 2024-05-09T04:48:08.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:08 smithi110 ceph-mon[34904]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-05-09T04:48:08.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:08 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3059632891' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T04:48:08.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:08 smithi046 ceph-mon[30136]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-05-09T04:48:08.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:08 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3059632891' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T04:48:08.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:08 smithi072 ceph-mon[34088]: pgmap v300: 97 pgs: 97 active+clean; 579 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-05-09T04:48:08.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:08 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3059632891' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-09T04:48:09.283 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-09T04:48:09.543 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:09.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:09 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3059632891' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T04:48:09.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:09 smithi046 ceph-mon[30136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-09T04:48:09.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:09 smithi046 ceph-mon[30136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T04:48:09.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:09 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3059632891' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T04:48:09.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:09 smithi110 ceph-mon[34904]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-09T04:48:09.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:09 smithi110 ceph-mon[34904]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T04:48:09.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:09 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3059632891' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-09T04:48:09.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:09 smithi072 ceph-mon[34088]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-09T04:48:09.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:09 smithi072 ceph-mon[34088]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T04:48:10.798 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:10 smithi046 ceph-mon[30136]: pgmap v301: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-09T04:48:10.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:10 smithi110 ceph-mon[34904]: pgmap v301: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-09T04:48:10.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:10 smithi072 ceph-mon[34088]: pgmap v301: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 682 B/s rd, 1.9 KiB/s wr, 6 op/s 2024-05-09T04:48:11.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.368+0000 7fb2c7773700 1 -- 172.21.15.46:0/1544311810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c0108f00 msgr2=0x7fb2c0109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:11.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.368+0000 7fb2c7773700 1 --2- 172.21.15.46:0/1544311810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c0108f00 0x7fb2c0109310 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb2b00099c0 tx=0x7fb2b0009cd0 comp rx=0 tx=0).stop 2024-05-09T04:48:11.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.368+0000 7fb2c7773700 1 -- 172.21.15.46:0/1544311810 shutdown_connections 2024-05-09T04:48:11.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.368+0000 7fb2c7773700 1 --2- 172.21.15.46:0/1544311810 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2c0109850 0x7fb2c01111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:11.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.368+0000 7fb2c7773700 1 --2- 172.21.15.46:0/1544311810 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2c01045d0 0x7fb2c01049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:11.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.368+0000 7fb2c7773700 1 --2- 172.21.15.46:0/1544311810 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c0108f00 0x7fb2c0109310 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:11.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.368+0000 7fb2c7773700 1 -- 172.21.15.46:0/1544311810 >> 172.21.15.46:0/1544311810 conn(0x7fb2c00ffea0 msgr2=0x7fb2c01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:11.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.368+0000 7fb2c7773700 1 -- 172.21.15.46:0/1544311810 shutdown_connections 2024-05-09T04:48:11.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.368+0000 7fb2c7773700 1 -- 172.21.15.46:0/1544311810 wait complete. 2024-05-09T04:48:11.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c7773700 1 Processor -- start 2024-05-09T04:48:11.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c7773700 1 -- start start 2024-05-09T04:48:11.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c7773700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2c01045d0 0x7fb2c0076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:11.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c7773700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c0108f00 0x7fb2c0077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:11.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c7773700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2c0109850 0x7fb2c0077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:11.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c7773700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c01144b0 con 0x7fb2c0108f00 2024-05-09T04:48:11.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c7773700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c0114330 con 0x7fb2c01045d0 2024-05-09T04:48:11.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c7773700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb2c0114630 con 0x7fb2c0109850 2024-05-09T04:48:11.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c4d0e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c0108f00 0x7fb2c0077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:11.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c4d0e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c0108f00 0x7fb2c0077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39450/0 (socket says 172.21.15.46:39450) 2024-05-09T04:48:11.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c4d0e700 1 -- 172.21.15.46:0/2022448018 learned_addr learned my addr 172.21.15.46:0/2022448018 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:11.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c550f700 1 --2- 172.21.15.46:0/2022448018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2c01045d0 0x7fb2c0076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:11.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c5d10700 1 --2- 172.21.15.46:0/2022448018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2c0109850 0x7fb2c0077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:11.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c4d0e700 1 -- 172.21.15.46:0/2022448018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2c0109850 msgr2=0x7fb2c0077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:11.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c4d0e700 1 --2- 172.21.15.46:0/2022448018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2c0109850 0x7fb2c0077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:11.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c4d0e700 1 -- 172.21.15.46:0/2022448018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2c01045d0 msgr2=0x7fb2c0076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:11.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c4d0e700 1 --2- 172.21.15.46:0/2022448018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2c01045d0 0x7fb2c0076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:11.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.369+0000 7fb2c4d0e700 1 -- 172.21.15.46:0/2022448018 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2c01a9190 con 0x7fb2c0108f00 2024-05-09T04:48:11.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.370+0000 7fb2c4d0e700 1 --2- 172.21.15.46:0/2022448018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c0108f00 0x7fb2c0077110 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009f50 tx=0x7fb2b00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:11.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.370+0000 7fb2b67fc700 1 -- 172.21.15.46:0/2022448018 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0026070 con 0x7fb2c0108f00 2024-05-09T04:48:11.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.370+0000 7fb2b67fc700 1 -- 172.21.15.46:0/2022448018 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2b0003f20 con 0x7fb2c0108f00 2024-05-09T04:48:11.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.370+0000 7fb2c7773700 1 -- 172.21.15.46:0/2022448018 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2c01a9420 con 0x7fb2c0108f00 2024-05-09T04:48:11.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.370+0000 7fb2b67fc700 1 -- 172.21.15.46:0/2022448018 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b000fd30 con 0x7fb2c0108f00 2024-05-09T04:48:11.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.370+0000 7fb2c7773700 1 -- 172.21.15.46:0/2022448018 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2c01a9870 con 0x7fb2c0108f00 2024-05-09T04:48:11.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.371+0000 7fb2c7773700 1 -- 172.21.15.46:0/2022448018 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2c00512c0 con 0x7fb2c0108f00 2024-05-09T04:48:11.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.374+0000 7fb2b67fc700 1 -- 172.21.15.46:0/2022448018 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb2b0029070 con 0x7fb2c0108f00 2024-05-09T04:48:11.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.375+0000 7fb2b67fc700 1 --2- 172.21.15.46:0/2022448018 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb2ac0b2880 0x7fb2ac0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:11.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.375+0000 7fb2c550f700 1 --2- 172.21.15.46:0/2022448018 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb2ac0b2880 0x7fb2ac0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:11.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.375+0000 7fb2b67fc700 1 -- 172.21.15.46:0/2022448018 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7fb2b00e3110 con 0x7fb2c0108f00 2024-05-09T04:48:11.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.375+0000 7fb2c550f700 1 --2- 172.21.15.46:0/2022448018 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb2ac0b2880 0x7fb2ac0b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc005950 tx=0x7fb2bc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:11.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.378+0000 7fb2b67fc700 1 -- 172.21.15.46:0/2022448018 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2b005edf0 con 0x7fb2c0108f00 2024-05-09T04:48:11.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:11.651+0000 7fb2c7773700 1 -- 172.21.15.46:0/2022448018 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb2c002e7b0 con 0x7fb2c0108f00 2024-05-09T04:48:12.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:12.505+0000 7fb2b67fc700 1 -- 172.21.15.46:0/2022448018 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v24) ==== 115+0+0 (secure 0 0 0) 0x7fb2b0013070 con 0x7fb2c0108f00 2024-05-09T04:48:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:12.509+0000 7fb2c7773700 1 -- 172.21.15.46:0/2022448018 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb2ac0b2880 msgr2=0x7fb2ac0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:12.509+0000 7fb2c7773700 1 --2- 172.21.15.46:0/2022448018 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb2ac0b2880 0x7fb2ac0b4d90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb2bc005950 tx=0x7fb2bc00a300 comp rx=0 tx=0).stop 2024-05-09T04:48:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:12.509+0000 7fb2c7773700 1 -- 172.21.15.46:0/2022448018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c0108f00 msgr2=0x7fb2c0077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:12.509+0000 7fb2c7773700 1 --2- 172.21.15.46:0/2022448018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c0108f00 0x7fb2c0077110 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009f50 tx=0x7fb2b00057c0 comp rx=0 tx=0).stop 2024-05-09T04:48:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:12.509+0000 7fb2c7773700 1 -- 172.21.15.46:0/2022448018 shutdown_connections 2024-05-09T04:48:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:12.509+0000 7fb2c7773700 1 --2- 172.21.15.46:0/2022448018 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb2ac0b2880 0x7fb2ac0b4d90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:12.509+0000 7fb2c7773700 1 --2- 172.21.15.46:0/2022448018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb2c0109850 0x7fb2c0077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:12.509+0000 7fb2c7773700 1 --2- 172.21.15.46:0/2022448018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb2c01045d0 0x7fb2c0076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:12.509+0000 7fb2c7773700 1 --2- 172.21.15.46:0/2022448018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb2c0108f00 0x7fb2c0077110 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:12.509+0000 7fb2c7773700 1 -- 172.21.15.46:0/2022448018 >> 172.21.15.46:0/2022448018 conn(0x7fb2c00ffea0 msgr2=0x7fb2c010a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:12.509+0000 7fb2c7773700 1 -- 172.21.15.46:0/2022448018 shutdown_connections 2024-05-09T04:48:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:12.509+0000 7fb2c7773700 1 -- 172.21.15.46:0/2022448018 wait complete. 2024-05-09T04:48:12.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:12 smithi110 ceph-mon[34904]: pgmap v302: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-09T04:48:12.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:12 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2022448018' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T04:48:12.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:12 smithi046 ceph-mon[30136]: pgmap v302: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-09T04:48:12.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:12 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2022448018' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T04:48:12.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:12 smithi072 ceph-mon[34088]: pgmap v302: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 2.0 KiB/s wr, 7 op/s 2024-05-09T04:48:12.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:12 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2022448018' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-09T04:48:13.285 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-05-09T04:48:13.285 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-09T04:48:13.544 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:13.762 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:13 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2022448018' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T04:48:13.762 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:13 smithi046 ceph-mon[30136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T04:48:13.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:13 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2022448018' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T04:48:13.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:13 smithi110 ceph-mon[34904]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T04:48:13.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:13 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2022448018' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-09T04:48:13.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:13 smithi072 ceph-mon[34088]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T04:48:14.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:14 smithi046 ceph-mon[30136]: pgmap v303: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-09T04:48:14.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:14 smithi110 ceph-mon[34904]: pgmap v303: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-09T04:48:14.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:14 smithi072 ceph-mon[34088]: pgmap v303: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-09T04:48:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.223+0000 7f3d357e8700 1 -- 172.21.15.46:0/364559110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d30108760 msgr2=0x7f3d301100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.223+0000 7f3d357e8700 1 --2- 172.21.15.46:0/364559110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d30108760 0x7f3d301100c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f3d280099c0 tx=0x7f3d28009cd0 comp rx=0 tx=0).stop 2024-05-09T04:48:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.223+0000 7f3d357e8700 1 -- 172.21.15.46:0/364559110 shutdown_connections 2024-05-09T04:48:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.223+0000 7f3d357e8700 1 --2- 172.21.15.46:0/364559110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d30108760 0x7f3d301100c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.223+0000 7f3d357e8700 1 --2- 172.21.15.46:0/364559110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d301034e0 0x7f3d301038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.223+0000 7f3d357e8700 1 --2- 172.21.15.46:0/364559110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d30107e10 0x7f3d30108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.223+0000 7f3d357e8700 1 -- 172.21.15.46:0/364559110 >> 172.21.15.46:0/364559110 conn(0x7f3d300fef50 msgr2=0x7f3d30101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.223+0000 7f3d357e8700 1 -- 172.21.15.46:0/364559110 shutdown_connections 2024-05-09T04:48:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.223+0000 7f3d357e8700 1 -- 172.21.15.46:0/364559110 wait complete. 2024-05-09T04:48:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.224+0000 7f3d357e8700 1 Processor -- start 2024-05-09T04:48:15.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.224+0000 7f3d357e8700 1 -- start start 2024-05-09T04:48:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.224+0000 7f3d357e8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d301034e0 0x7f3d3019f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.224+0000 7f3d357e8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d30107e10 0x7f3d3019fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.224+0000 7f3d357e8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d30108760 0x7f3d30199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.224+0000 7f3d357e8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3d30113390 con 0x7f3d30107e10 2024-05-09T04:48:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.224+0000 7f3d357e8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3d30113210 con 0x7f3d301034e0 2024-05-09T04:48:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.224+0000 7f3d357e8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3d30113510 con 0x7f3d30108760 2024-05-09T04:48:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.224+0000 7f3d2e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d30107e10 0x7f3d3019fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.224+0000 7f3d2e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d30107e10 0x7f3d3019fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52946/0 (socket says 172.21.15.46:52946) 2024-05-09T04:48:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.224+0000 7f3d2e7fc700 1 -- 172.21.15.46:0/524636544 learned_addr learned my addr 172.21.15.46:0/524636544 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.225+0000 7f3d2effd700 1 --2- 172.21.15.46:0/524636544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d301034e0 0x7f3d3019f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.225+0000 7f3d2f7fe700 1 --2- 172.21.15.46:0/524636544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d30108760 0x7f3d30199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.225+0000 7f3d2e7fc700 1 -- 172.21.15.46:0/524636544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d30108760 msgr2=0x7f3d30199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.225+0000 7f3d2e7fc700 1 --2- 172.21.15.46:0/524636544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d30108760 0x7f3d30199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:15.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.225+0000 7f3d2e7fc700 1 -- 172.21.15.46:0/524636544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d301034e0 msgr2=0x7f3d3019f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:15.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.225+0000 7f3d2e7fc700 1 --2- 172.21.15.46:0/524636544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d301034e0 0x7f3d3019f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:15.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.225+0000 7f3d2e7fc700 1 -- 172.21.15.46:0/524636544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3019a160 con 0x7f3d30107e10 2024-05-09T04:48:15.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.225+0000 7f3d2e7fc700 1 --2- 172.21.15.46:0/524636544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d30107e10 0x7f3d3019fdd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3d18007b50 tx=0x7f3d18007e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:15.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.225+0000 7f3d27fff700 1 -- 172.21.15.46:0/524636544 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1800ce20 con 0x7f3d30107e10 2024-05-09T04:48:15.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.225+0000 7f3d27fff700 1 -- 172.21.15.46:0/524636544 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d1800ea30 con 0x7f3d30107e10 2024-05-09T04:48:15.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.225+0000 7f3d357e8700 1 -- 172.21.15.46:0/524636544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3019a450 con 0x7f3d30107e10 2024-05-09T04:48:15.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.225+0000 7f3d27fff700 1 -- 172.21.15.46:0/524636544 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d1800aae0 con 0x7f3d30107e10 2024-05-09T04:48:15.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.225+0000 7f3d357e8700 1 -- 172.21.15.46:0/524636544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d300759f0 con 0x7f3d30107e10 2024-05-09T04:48:15.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.226+0000 7f3d357e8700 1 -- 172.21.15.46:0/524636544 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d30104990 con 0x7f3d30107e10 2024-05-09T04:48:15.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.228+0000 7f3d27fff700 1 -- 172.21.15.46:0/524636544 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f3d1800ac80 con 0x7f3d30107e10 2024-05-09T04:48:15.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.229+0000 7f3d27fff700 1 --2- 172.21.15.46:0/524636544 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3d1c0b2880 0x7f3d1c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:15.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.229+0000 7f3d2effd700 1 --2- 172.21.15.46:0/524636544 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3d1c0b2880 0x7f3d1c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:15.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.229+0000 7f3d27fff700 1 -- 172.21.15.46:0/524636544 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f3d180e3830 con 0x7f3d30107e10 2024-05-09T04:48:15.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.229+0000 7f3d2effd700 1 --2- 172.21.15.46:0/524636544 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3d1c0b2880 0x7f3d1c0b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009550 tx=0x7f3d200091f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:15.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.232+0000 7f3d27fff700 1 -- 172.21.15.46:0/524636544 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3d18023020 con 0x7f3d30107e10 2024-05-09T04:48:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.479+0000 7f3d357e8700 1 -- 172.21.15.46:0/524636544 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f3d30075c50 con 0x7f3d1c0b2880 2024-05-09T04:48:15.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.504+0000 7f3d27fff700 1 -- 172.21.15.46:0/524636544 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3d30075c50 con 0x7f3d1c0b2880 2024-05-09T04:48:15.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.506+0000 7f3d357e8700 1 -- 172.21.15.46:0/524636544 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3d1c0b2880 msgr2=0x7f3d1c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.506+0000 7f3d357e8700 1 --2- 172.21.15.46:0/524636544 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3d1c0b2880 0x7f3d1c0b4d90 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3d20009550 tx=0x7f3d200091f0 comp rx=0 tx=0).stop 2024-05-09T04:48:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.507+0000 7f3d357e8700 1 -- 172.21.15.46:0/524636544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d30107e10 msgr2=0x7f3d3019fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.507+0000 7f3d357e8700 1 --2- 172.21.15.46:0/524636544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d30107e10 0x7f3d3019fdd0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3d18007b50 tx=0x7f3d18007e60 comp rx=0 tx=0).stop 2024-05-09T04:48:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.507+0000 7f3d357e8700 1 -- 172.21.15.46:0/524636544 shutdown_connections 2024-05-09T04:48:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.507+0000 7f3d357e8700 1 --2- 172.21.15.46:0/524636544 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3d1c0b2880 0x7f3d1c0b4d90 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.507+0000 7f3d357e8700 1 --2- 172.21.15.46:0/524636544 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3d30108760 0x7f3d30199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.507+0000 7f3d357e8700 1 --2- 172.21.15.46:0/524636544 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3d301034e0 0x7f3d3019f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.507+0000 7f3d357e8700 1 --2- 172.21.15.46:0/524636544 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3d30107e10 0x7f3d3019fdd0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.507+0000 7f3d357e8700 1 -- 172.21.15.46:0/524636544 >> 172.21.15.46:0/524636544 conn(0x7f3d300fef50 msgr2=0x7f3d30107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.507+0000 7f3d357e8700 1 -- 172.21.15.46:0/524636544 shutdown_connections 2024-05-09T04:48:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:15.507+0000 7f3d357e8700 1 -- 172.21.15.46:0/524636544 wait complete. 2024-05-09T04:48:15.830 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:48:15 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:48:15.484+0000 7efe773ea700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:48:15.830 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:48:15 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:48:15.484+0000 7efe773ea700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:48:15.830 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:48:15 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:48:15.484+0000 7efe773ea700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:48:15.830 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:48:15 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:48:15.484+0000 7efe773ea700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:48:15.830 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:48:15 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:48:15.484+0000 7efe773ea700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:48:16.317 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-09T04:48:16.576 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:16.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:16 smithi046 ceph-mon[30136]: pgmap v304: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-09T04:48:16.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:16 smithi046 ceph-mon[30136]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:16.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:16 smithi110 ceph-mon[34904]: pgmap v304: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-09T04:48:16.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:16 smithi110 ceph-mon[34904]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:16.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:16 smithi072 ceph-mon[34088]: pgmap v304: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-09T04:48:16.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:16 smithi072 ceph-mon[34088]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.429+0000 7f0c57ec6700 1 -- 172.21.15.46:0/3095461986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c50102ae0 msgr2=0x7f0c50102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.429+0000 7f0c57ec6700 1 --2- 172.21.15.46:0/3095461986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c50102ae0 0x7f0c50102ef0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c00b0d0 tx=0x7f0c4c00b3e0 comp rx=0 tx=0).stop 2024-05-09T04:48:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.429+0000 7f0c57ec6700 1 -- 172.21.15.46:0/3095461986 shutdown_connections 2024-05-09T04:48:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.429+0000 7f0c57ec6700 1 --2- 172.21.15.46:0/3095461986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c50109850 0x7f0c501111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.429+0000 7f0c57ec6700 1 --2- 172.21.15.46:0/3095461986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c50102ae0 0x7f0c50102ef0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.429+0000 7f0c57ec6700 1 --2- 172.21.15.46:0/3095461986 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c50108f00 0x7f0c50109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:18.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.429+0000 7f0c57ec6700 1 -- 172.21.15.46:0/3095461986 >> 172.21.15.46:0/3095461986 conn(0x7f0c500fe540 msgr2=0x7f0c50100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:18.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.429+0000 7f0c57ec6700 1 -- 172.21.15.46:0/3095461986 shutdown_connections 2024-05-09T04:48:18.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.429+0000 7f0c57ec6700 1 -- 172.21.15.46:0/3095461986 wait complete. 2024-05-09T04:48:18.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.430+0000 7f0c57ec6700 1 Processor -- start 2024-05-09T04:48:18.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.430+0000 7f0c57ec6700 1 -- start start 2024-05-09T04:48:18.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.430+0000 7f0c57ec6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c50102ae0 0x7f0c50103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:18.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.430+0000 7f0c57ec6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c50108f00 0x7f0c50103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:18.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.430+0000 7f0c57ec6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c50109850 0x7f0c501044b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:18.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.430+0000 7f0c57ec6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0c501144d0 con 0x7f0c50102ae0 2024-05-09T04:48:18.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.430+0000 7f0c57ec6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0c50114350 con 0x7f0c50108f00 2024-05-09T04:48:18.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.430+0000 7f0c57ec6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0c50114650 con 0x7f0c50109850 2024-05-09T04:48:18.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.430+0000 7f0c55c62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c50102ae0 0x7f0c50103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:18.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.430+0000 7f0c55c62700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c50102ae0 0x7f0c50103a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52982/0 (socket says 172.21.15.46:52982) 2024-05-09T04:48:18.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.430+0000 7f0c55c62700 1 -- 172.21.15.46:0/4165962812 learned_addr learned my addr 172.21.15.46:0/4165962812 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:18.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.430+0000 7f0c55461700 1 --2- 172.21.15.46:0/4165962812 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c50108f00 0x7f0c50103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:18.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.430+0000 7f0c56463700 1 --2- 172.21.15.46:0/4165962812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c50109850 0x7f0c501044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:18.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.431+0000 7f0c55c62700 1 -- 172.21.15.46:0/4165962812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c50109850 msgr2=0x7f0c501044b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:18.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.431+0000 7f0c55c62700 1 --2- 172.21.15.46:0/4165962812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c50109850 0x7f0c501044b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:18.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.431+0000 7f0c55c62700 1 -- 172.21.15.46:0/4165962812 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c50108f00 msgr2=0x7f0c50103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:18.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.431+0000 7f0c55c62700 1 --2- 172.21.15.46:0/4165962812 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c50108f00 0x7f0c50103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:18.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.431+0000 7f0c55c62700 1 -- 172.21.15.46:0/4165962812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c501a91e0 con 0x7f0c50102ae0 2024-05-09T04:48:18.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.431+0000 7f0c55c62700 1 --2- 172.21.15.46:0/4165962812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c50102ae0 0x7f0c50103a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c007a20 tx=0x7f0c4c007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:18.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.431+0000 7f0c46ffd700 1 -- 172.21.15.46:0/4165962812 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c01f070 con 0x7f0c50102ae0 2024-05-09T04:48:18.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.431+0000 7f0c57ec6700 1 -- 172.21.15.46:0/4165962812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c501a9470 con 0x7f0c50102ae0 2024-05-09T04:48:18.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.431+0000 7f0c46ffd700 1 -- 172.21.15.46:0/4165962812 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c4c004c50 con 0x7f0c50102ae0 2024-05-09T04:48:18.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.431+0000 7f0c46ffd700 1 -- 172.21.15.46:0/4165962812 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4c01b650 con 0x7f0c50102ae0 2024-05-09T04:48:18.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.431+0000 7f0c57ec6700 1 -- 172.21.15.46:0/4165962812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c501a98c0 con 0x7f0c50102ae0 2024-05-09T04:48:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.432+0000 7f0c57ec6700 1 -- 172.21.15.46:0/4165962812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c500512c0 con 0x7f0c50102ae0 2024-05-09T04:48:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.434+0000 7f0c46ffd700 1 -- 172.21.15.46:0/4165962812 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0c4c022070 con 0x7f0c50102ae0 2024-05-09T04:48:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.434+0000 7f0c46ffd700 1 --2- 172.21.15.46:0/4165962812 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f0c3c0b2910 0x7f0c3c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.435+0000 7f0c55461700 1 --2- 172.21.15.46:0/4165962812 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f0c3c0b2910 0x7f0c3c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:18.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.435+0000 7f0c46ffd700 1 -- 172.21.15.46:0/4165962812 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f0c4c00f020 con 0x7f0c50102ae0 2024-05-09T04:48:18.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.435+0000 7f0c55461700 1 --2- 172.21.15.46:0/4165962812 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f0c3c0b2910 0x7f0c3c0b4e20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0c50101c10 tx=0x7f0c40009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:18.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.438+0000 7f0c46ffd700 1 -- 172.21.15.46:0/4165962812 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0c4c029020 con 0x7f0c50102ae0 2024-05-09T04:48:18.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.687+0000 7f0c57ec6700 1 -- 172.21.15.46:0/4165962812 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f0c5002e660 con 0x7f0c3c0b2910 2024-05-09T04:48:18.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:18 smithi046 ceph-mon[30136]: pgmap v305: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-09T04:48:18.687 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:18 smithi046 ceph-mon[30136]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T04:48:18.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.715+0000 7f0c46ffd700 1 -- 172.21.15.46:0/4165962812 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f0c5002e660 con 0x7f0c3c0b2910 2024-05-09T04:48:18.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.718+0000 7f0c57ec6700 1 -- 172.21.15.46:0/4165962812 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f0c3c0b2910 msgr2=0x7f0c3c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:18.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.718+0000 7f0c57ec6700 1 --2- 172.21.15.46:0/4165962812 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f0c3c0b2910 0x7f0c3c0b4e20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0c50101c10 tx=0x7f0c40009260 comp rx=0 tx=0).stop 2024-05-09T04:48:18.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.718+0000 7f0c57ec6700 1 -- 172.21.15.46:0/4165962812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c50102ae0 msgr2=0x7f0c50103a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:18.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.718+0000 7f0c57ec6700 1 --2- 172.21.15.46:0/4165962812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c50102ae0 0x7f0c50103a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c007a20 tx=0x7f0c4c007a50 comp rx=0 tx=0).stop 2024-05-09T04:48:18.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.718+0000 7f0c57ec6700 1 -- 172.21.15.46:0/4165962812 shutdown_connections 2024-05-09T04:48:18.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.718+0000 7f0c57ec6700 1 --2- 172.21.15.46:0/4165962812 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f0c3c0b2910 0x7f0c3c0b4e20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:18.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.718+0000 7f0c57ec6700 1 --2- 172.21.15.46:0/4165962812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0c50109850 0x7f0c501044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:18.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.718+0000 7f0c57ec6700 1 --2- 172.21.15.46:0/4165962812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0c50102ae0 0x7f0c50103a30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:18.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.718+0000 7f0c57ec6700 1 --2- 172.21.15.46:0/4165962812 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0c50108f00 0x7f0c50103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:18.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.718+0000 7f0c57ec6700 1 -- 172.21.15.46:0/4165962812 >> 172.21.15.46:0/4165962812 conn(0x7f0c500fe540 msgr2=0x7f0c50107170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:18.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.718+0000 7f0c57ec6700 1 -- 172.21.15.46:0/4165962812 shutdown_connections 2024-05-09T04:48:18.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:18.718+0000 7f0c57ec6700 1 -- 172.21.15.46:0/4165962812 wait complete. 2024-05-09T04:48:18.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:18 smithi110 ceph-mon[34904]: pgmap v305: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-09T04:48:18.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:18 smithi110 ceph-mon[34904]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T04:48:18.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:18 smithi072 ceph-mon[34088]: pgmap v305: 97 pgs: 97 active+clean; 582 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-05-09T04:48:18.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:18 smithi072 ceph-mon[34088]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-05-09T04:48:19.437 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-09T04:48:19.691 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:19.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:19 smithi110 ceph-mon[34904]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:19.814 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:19 smithi046 ceph-mon[30136]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:19.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:19 smithi072 ceph-mon[34088]: from='client.14706 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:20.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:20 smithi110 ceph-mon[34904]: pgmap v306: 97 pgs: 97 active+clean; 602 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-05-09T04:48:20.808 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:20 smithi046 ceph-mon[30136]: pgmap v306: 97 pgs: 97 active+clean; 602 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-05-09T04:48:20.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:20 smithi072 ceph-mon[34088]: pgmap v306: 97 pgs: 97 active+clean; 602 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.4 KiB/s wr, 13 op/s 2024-05-09T04:48:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.350+0000 7ff86d066700 1 -- 172.21.15.46:0/1576134589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff868108780 msgr2=0x7ff8681100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.350+0000 7ff86d066700 1 --2- 172.21.15.46:0/1576134589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff868108780 0x7ff8681100e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff86000b0d0 tx=0x7ff86000b3e0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.350+0000 7ff86d066700 1 -- 172.21.15.46:0/1576134589 shutdown_connections 2024-05-09T04:48:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.350+0000 7ff86d066700 1 --2- 172.21.15.46:0/1576134589 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff868108780 0x7ff8681100e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.350+0000 7ff86d066700 1 --2- 172.21.15.46:0/1576134589 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff868103550 0x7ff868103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.350+0000 7ff86d066700 1 --2- 172.21.15.46:0/1576134589 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff868107e30 0x7ff868108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.350+0000 7ff86d066700 1 -- 172.21.15.46:0/1576134589 >> 172.21.15.46:0/1576134589 conn(0x7ff868078e00 msgr2=0x7ff868079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:21.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.350+0000 7ff86d066700 1 -- 172.21.15.46:0/1576134589 shutdown_connections 2024-05-09T04:48:21.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.350+0000 7ff86d066700 1 -- 172.21.15.46:0/1576134589 wait complete. 2024-05-09T04:48:21.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.351+0000 7ff86d066700 1 Processor -- start 2024-05-09T04:48:21.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.351+0000 7ff86d066700 1 -- start start 2024-05-09T04:48:21.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.351+0000 7ff86d066700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff868103550 0x7ff86819bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.351+0000 7ff86d066700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff868107e30 0x7ff86819c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.351+0000 7ff86d066700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff868108780 0x7ff86819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.351+0000 7ff86d066700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff868113420 con 0x7ff868103550 2024-05-09T04:48:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.351+0000 7ff86d066700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681132a0 con 0x7ff868108780 2024-05-09T04:48:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.351+0000 7ff86d066700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff8681135a0 con 0x7ff868107e30 2024-05-09T04:48:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff866d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff868103550 0x7ff86819bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff866d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff868103550 0x7ff86819bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53006/0 (socket says 172.21.15.46:53006) 2024-05-09T04:48:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff866d9d700 1 -- 172.21.15.46:0/881433252 learned_addr learned my addr 172.21.15.46:0/881433252 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff85ffff700 1 --2- 172.21.15.46:0/881433252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff868107e30 0x7ff86819c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff86759e700 1 --2- 172.21.15.46:0/881433252 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff868108780 0x7ff86819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff866d9d700 1 -- 172.21.15.46:0/881433252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff868107e30 msgr2=0x7ff86819c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:21.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff866d9d700 1 --2- 172.21.15.46:0/881433252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff868107e30 0x7ff86819c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff866d9d700 1 -- 172.21.15.46:0/881433252 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff868108780 msgr2=0x7ff86819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff866d9d700 1 --2- 172.21.15.46:0/881433252 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff868108780 0x7ff86819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff866d9d700 1 -- 172.21.15.46:0/881433252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff868199b10 con 0x7ff868103550 2024-05-09T04:48:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff866d9d700 1 --2- 172.21.15.46:0/881433252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff868103550 0x7ff86819bd20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff85800d950 tx=0x7ff85800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff864d99700 1 -- 172.21.15.46:0/881433252 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff858016070 con 0x7ff868103550 2024-05-09T04:48:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff86d066700 1 -- 172.21.15.46:0/881433252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff868199e00 con 0x7ff868103550 2024-05-09T04:48:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff864d99700 1 -- 172.21.15.46:0/881433252 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff85800fd40 con 0x7ff868103550 2024-05-09T04:48:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff864d99700 1 -- 172.21.15.46:0/881433252 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff85800e3b0 con 0x7ff868103550 2024-05-09T04:48:21.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.352+0000 7ff86d066700 1 -- 172.21.15.46:0/881433252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff86819a340 con 0x7ff868103550 2024-05-09T04:48:21.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.353+0000 7ff86d066700 1 -- 172.21.15.46:0/881433252 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8680512c0 con 0x7ff868103550 2024-05-09T04:48:21.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.355+0000 7ff864d99700 1 -- 172.21.15.46:0/881433252 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff858019070 con 0x7ff868103550 2024-05-09T04:48:21.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.356+0000 7ff864d99700 1 --2- 172.21.15.46:0/881433252 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff8540b2910 0x7ff8540b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:21.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.356+0000 7ff85ffff700 1 --2- 172.21.15.46:0/881433252 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff8540b2910 0x7ff8540b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:21.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.356+0000 7ff864d99700 1 -- 172.21.15.46:0/881433252 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7ff8580e1c40 con 0x7ff868103550 2024-05-09T04:48:21.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.356+0000 7ff85ffff700 1 --2- 172.21.15.46:0/881433252 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff8540b2910 0x7ff8540b4e20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff868102d00 tx=0x7ff850009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:21.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.359+0000 7ff864d99700 1 -- 172.21.15.46:0/881433252 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff8580a99b0 con 0x7ff868103550 2024-05-09T04:48:21.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.616+0000 7ff86d066700 1 -- 172.21.15.46:0/881433252 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7ff8681013e0 con 0x7ff8540b2910 2024-05-09T04:48:21.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.622+0000 7ff864d99700 1 -- 172.21.15.46:0/881433252 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff8681013e0 con 0x7ff8540b2910 2024-05-09T04:48:21.622 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/qa/sv_0/8a799555-4379-4e71-aefe-624200d66c8c 2024-05-09T04:48:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.624+0000 7ff86d066700 1 -- 172.21.15.46:0/881433252 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff8540b2910 msgr2=0x7ff8540b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.624+0000 7ff86d066700 1 --2- 172.21.15.46:0/881433252 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff8540b2910 0x7ff8540b4e20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff868102d00 tx=0x7ff850009190 comp rx=0 tx=0).stop 2024-05-09T04:48:21.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.624+0000 7ff86d066700 1 -- 172.21.15.46:0/881433252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff868103550 msgr2=0x7ff86819bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:21.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.624+0000 7ff86d066700 1 --2- 172.21.15.46:0/881433252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff868103550 0x7ff86819bd20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff85800d950 tx=0x7ff85800dc60 comp rx=0 tx=0).stop 2024-05-09T04:48:21.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.624+0000 7ff86d066700 1 -- 172.21.15.46:0/881433252 shutdown_connections 2024-05-09T04:48:21.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.624+0000 7ff86d066700 1 --2- 172.21.15.46:0/881433252 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff8540b2910 0x7ff8540b4e20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.624+0000 7ff86d066700 1 --2- 172.21.15.46:0/881433252 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff868108780 0x7ff86819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.624+0000 7ff86d066700 1 --2- 172.21.15.46:0/881433252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff868103550 0x7ff86819bd20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.624+0000 7ff86d066700 1 --2- 172.21.15.46:0/881433252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff868107e30 0x7ff86819c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:21.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.624+0000 7ff86d066700 1 -- 172.21.15.46:0/881433252 >> 172.21.15.46:0/881433252 conn(0x7ff868078e00 msgr2=0x7ff868100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:21.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.624+0000 7ff86d066700 1 -- 172.21.15.46:0/881433252 shutdown_connections 2024-05-09T04:48:21.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:21.625+0000 7ff86d066700 1 -- 172.21.15.46:0/881433252 wait complete. 2024-05-09T04:48:22.350 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-09T04:48:22.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:22 smithi046 ceph-mon[30136]: pgmap v307: 97 pgs: 97 active+clean; 602 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-05-09T04:48:22.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:22 smithi046 ceph-mon[30136]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:22.610 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:22.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:22 smithi110 ceph-mon[34904]: pgmap v307: 97 pgs: 97 active+clean; 602 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-05-09T04:48:22.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:22 smithi110 ceph-mon[34904]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:22.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:22 smithi072 ceph-mon[34088]: pgmap v307: 97 pgs: 97 active+clean; 602 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 2.2 KiB/s wr, 8 op/s 2024-05-09T04:48:22.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:22 smithi072 ceph-mon[34088]: from='client.14712 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:24.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.366+0000 7f4c0e8fd700 1 -- 172.21.15.46:0/4105499683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c081034c0 msgr2=0x7f4c081038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:24.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.366+0000 7f4c0e8fd700 1 --2- 172.21.15.46:0/4105499683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c081034c0 0x7f4c081038d0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4bf80099c0 tx=0x7f4bf8009cd0 comp rx=0 tx=0).stop 2024-05-09T04:48:24.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.366+0000 7f4c0e8fd700 1 -- 172.21.15.46:0/4105499683 shutdown_connections 2024-05-09T04:48:24.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.366+0000 7f4c0e8fd700 1 --2- 172.21.15.46:0/4105499683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c08108740 0x7f4c081100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:24.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.366+0000 7f4c0e8fd700 1 --2- 172.21.15.46:0/4105499683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c081034c0 0x7f4c081038d0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:24.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.366+0000 7f4c0e8fd700 1 --2- 172.21.15.46:0/4105499683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c08107df0 0x7f4c08108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:24.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.366+0000 7f4c0e8fd700 1 -- 172.21.15.46:0/4105499683 >> 172.21.15.46:0/4105499683 conn(0x7f4c080fedc0 msgr2=0x7f4c081011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:24.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.366+0000 7f4c0e8fd700 1 -- 172.21.15.46:0/4105499683 shutdown_connections 2024-05-09T04:48:24.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.366+0000 7f4c0e8fd700 1 -- 172.21.15.46:0/4105499683 wait complete. 2024-05-09T04:48:24.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.367+0000 7f4c0e8fd700 1 Processor -- start 2024-05-09T04:48:24.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.367+0000 7f4c0e8fd700 1 -- start start 2024-05-09T04:48:24.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.367+0000 7f4c0e8fd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c081034c0 0x7f4c08199d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:24.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.367+0000 7f4c0e8fd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c08107df0 0x7f4c0819a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:24.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.367+0000 7f4c0e8fd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c08108740 0x7f4c0819e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:24.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.367+0000 7f4c0e8fd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c081133e0 con 0x7f4c081034c0 2024-05-09T04:48:24.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.367+0000 7f4c0e8fd700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08113260 con 0x7f4c08108740 2024-05-09T04:48:24.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.367+0000 7f4c0e8fd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4c08113560 con 0x7f4c08107df0 2024-05-09T04:48:24.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.368+0000 7f4c07fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c081034c0 0x7f4c08199d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:24.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.368+0000 7f4c07fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c081034c0 0x7f4c08199d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58432/0 (socket says 172.21.15.46:58432) 2024-05-09T04:48:24.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.368+0000 7f4c07fff700 1 -- 172.21.15.46:0/3077694621 learned_addr learned my addr 172.21.15.46:0/3077694621 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:24.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.368+0000 7f4bfffff700 1 --2- 172.21.15.46:0/3077694621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c08107df0 0x7f4c0819a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:24.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.368+0000 7f4c0ce9a700 1 --2- 172.21.15.46:0/3077694621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c08108740 0x7f4c0819e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:24.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.368+0000 7f4c07fff700 1 -- 172.21.15.46:0/3077694621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c08107df0 msgr2=0x7f4c0819a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:24.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.368+0000 7f4c07fff700 1 --2- 172.21.15.46:0/3077694621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c08107df0 0x7f4c0819a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:24.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.368+0000 7f4c07fff700 1 -- 172.21.15.46:0/3077694621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c08108740 msgr2=0x7f4c0819e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:24.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.368+0000 7f4c07fff700 1 --2- 172.21.15.46:0/3077694621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c08108740 0x7f4c0819e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:24.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.368+0000 7f4c07fff700 1 -- 172.21.15.46:0/3077694621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c0819ec40 con 0x7f4c081034c0 2024-05-09T04:48:24.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.368+0000 7f4c07fff700 1 --2- 172.21.15.46:0/3077694621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c081034c0 0x7f4c08199d70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800b580 tx=0x7f4bf80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:24.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.368+0000 7f4c05ffb700 1 -- 172.21.15.46:0/3077694621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8027070 con 0x7f4c081034c0 2024-05-09T04:48:24.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.368+0000 7f4c0e8fd700 1 -- 172.21.15.46:0/3077694621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c0819eed0 con 0x7f4c081034c0 2024-05-09T04:48:24.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.368+0000 7f4c05ffb700 1 -- 172.21.15.46:0/3077694621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4bf8003f20 con 0x7f4c081034c0 2024-05-09T04:48:24.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.368+0000 7f4c05ffb700 1 -- 172.21.15.46:0/3077694621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4bf8022d70 con 0x7f4c081034c0 2024-05-09T04:48:24.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.369+0000 7f4c0e8fd700 1 -- 172.21.15.46:0/3077694621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c0806b610 con 0x7f4c081034c0 2024-05-09T04:48:24.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.370+0000 7f4c0e8fd700 1 -- 172.21.15.46:0/3077694621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c08104970 con 0x7f4c081034c0 2024-05-09T04:48:24.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.371+0000 7f4c05ffb700 1 -- 172.21.15.46:0/3077694621 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4bf8022500 con 0x7f4c081034c0 2024-05-09T04:48:24.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.372+0000 7f4c05ffb700 1 --2- 172.21.15.46:0/3077694621 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4bf40b2910 0x7f4bf40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:24.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.372+0000 7f4bfffff700 1 --2- 172.21.15.46:0/3077694621 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4bf40b2910 0x7f4bf40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:24.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.372+0000 7f4c05ffb700 1 -- 172.21.15.46:0/3077694621 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f4bf80e2e90 con 0x7f4c081034c0 2024-05-09T04:48:24.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.372+0000 7f4bfffff700 1 --2- 172.21.15.46:0/3077694621 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4bf40b2910 0x7f4bf40b4e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4c081023d0 tx=0x7f4bf0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:24.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.377+0000 7f4c05ffb700 1 -- 172.21.15.46:0/3077694621 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4bf8013070 con 0x7f4c081034c0 2024-05-09T04:48:24.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.623+0000 7f4c0e8fd700 1 -- 172.21.15.46:0/3077694621 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f4c0819b0d0 con 0x7f4bf40b2910 2024-05-09T04:48:24.623 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:24 smithi046 ceph-mon[30136]: pgmap v308: 97 pgs: 97 active+clean; 602 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-09T04:48:24.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.653+0000 7f4c05ffb700 1 -- 172.21.15.46:0/3077694621 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f4c0819b0d0 con 0x7f4bf40b2910 2024-05-09T04:48:24.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.656+0000 7f4c0e8fd700 1 -- 172.21.15.46:0/3077694621 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4bf40b2910 msgr2=0x7f4bf40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:24.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.656+0000 7f4c0e8fd700 1 --2- 172.21.15.46:0/3077694621 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4bf40b2910 0x7f4bf40b4e20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4c081023d0 tx=0x7f4bf0009190 comp rx=0 tx=0).stop 2024-05-09T04:48:24.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.656+0000 7f4c0e8fd700 1 -- 172.21.15.46:0/3077694621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c081034c0 msgr2=0x7f4c08199d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:24.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.656+0000 7f4c0e8fd700 1 --2- 172.21.15.46:0/3077694621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c081034c0 0x7f4c08199d70 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4bf800b580 tx=0x7f4bf80047b0 comp rx=0 tx=0).stop 2024-05-09T04:48:24.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.656+0000 7f4c0e8fd700 1 -- 172.21.15.46:0/3077694621 shutdown_connections 2024-05-09T04:48:24.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.656+0000 7f4c0e8fd700 1 --2- 172.21.15.46:0/3077694621 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4bf40b2910 0x7f4bf40b4e20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:24.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.656+0000 7f4c0e8fd700 1 --2- 172.21.15.46:0/3077694621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c08108740 0x7f4c0819e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.656+0000 7f4c0e8fd700 1 --2- 172.21.15.46:0/3077694621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c081034c0 0x7f4c08199d70 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.656+0000 7f4c0e8fd700 1 --2- 172.21.15.46:0/3077694621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c08107df0 0x7f4c0819a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.656+0000 7f4c0e8fd700 1 -- 172.21.15.46:0/3077694621 >> 172.21.15.46:0/3077694621 conn(0x7f4c080fedc0 msgr2=0x7f4c081096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.656+0000 7f4c0e8fd700 1 -- 172.21.15.46:0/3077694621 shutdown_connections 2024-05-09T04:48:24.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:24.656+0000 7f4c0e8fd700 1 -- 172.21.15.46:0/3077694621 wait complete. 2024-05-09T04:48:24.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:24 smithi110 ceph-mon[34904]: pgmap v308: 97 pgs: 97 active+clean; 602 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-09T04:48:24.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:24 smithi072 ceph-mon[34088]: pgmap v308: 97 pgs: 97 active+clean; 602 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 2.2 KiB/s wr, 7 op/s 2024-05-09T04:48:25.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-09T04:48:25.541 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:25 smithi046 ceph-mon[30136]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:25.571 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:25.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:25 smithi110 ceph-mon[34904]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:25.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:25 smithi072 ceph-mon[34088]: from='client.14718 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:26.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:26 smithi046 ceph-mon[30136]: pgmap v309: 97 pgs: 97 active+clean; 608 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-05-09T04:48:26.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:26 smithi110 ceph-mon[34904]: pgmap v309: 97 pgs: 97 active+clean; 608 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-05-09T04:48:26.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:26 smithi072 ceph-mon[34088]: pgmap v309: 97 pgs: 97 active+clean; 608 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 3.0 KiB/s wr, 9 op/s 2024-05-09T04:48:27.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.278+0000 7f1fc4e39700 1 -- 172.21.15.46:0/1059429210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc0107df0 msgr2=0x7f1fc0108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.278+0000 7f1fc4e39700 1 --2- 172.21.15.46:0/1059429210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc0107df0 0x7f1fc0108200 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1fb40099c0 tx=0x7f1fb4009cd0 comp rx=0 tx=0).stop 2024-05-09T04:48:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.278+0000 7f1fc4e39700 1 -- 172.21.15.46:0/1059429210 shutdown_connections 2024-05-09T04:48:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.278+0000 7f1fc4e39700 1 --2- 172.21.15.46:0/1059429210 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1fc0108740 0x7f1fc01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.278+0000 7f1fc4e39700 1 --2- 172.21.15.46:0/1059429210 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fc01034c0 0x7f1fc01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.278+0000 7f1fc4e39700 1 --2- 172.21.15.46:0/1059429210 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc0107df0 0x7f1fc0108200 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.278+0000 7f1fc4e39700 1 -- 172.21.15.46:0/1059429210 >> 172.21.15.46:0/1059429210 conn(0x7f1fc00fedc0 msgr2=0x7f1fc01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.279+0000 7f1fc4e39700 1 -- 172.21.15.46:0/1059429210 shutdown_connections 2024-05-09T04:48:27.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.279+0000 7f1fc4e39700 1 -- 172.21.15.46:0/1059429210 wait complete. 2024-05-09T04:48:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.279+0000 7f1fc4e39700 1 Processor -- start 2024-05-09T04:48:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.279+0000 7f1fc4e39700 1 -- start start 2024-05-09T04:48:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fc4e39700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1fc01034c0 0x7f1fc019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:27.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fc4e39700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc0107df0 0x7f1fc019c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fc4e39700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fc0108740 0x7f1fc019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fc4e39700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc0113350 con 0x7f1fc0107df0 2024-05-09T04:48:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fc4e39700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc01131d0 con 0x7f1fc01034c0 2024-05-09T04:48:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fc4e39700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1fc01134d0 con 0x7f1fc0108740 2024-05-09T04:48:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fbdd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc0107df0 0x7f1fc019c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fbdd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc0107df0 0x7f1fc019c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58442/0 (socket says 172.21.15.46:58442) 2024-05-09T04:48:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fbdd9b700 1 -- 172.21.15.46:0/18954336 learned_addr learned my addr 172.21.15.46:0/18954336 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fbe59c700 1 --2- 172.21.15.46:0/18954336 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1fc01034c0 0x7f1fc019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fbed9d700 1 --2- 172.21.15.46:0/18954336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fc0108740 0x7f1fc019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fbdd9b700 1 -- 172.21.15.46:0/18954336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fc0108740 msgr2=0x7f1fc019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fbdd9b700 1 --2- 172.21.15.46:0/18954336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fc0108740 0x7f1fc019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fbdd9b700 1 -- 172.21.15.46:0/18954336 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1fc01034c0 msgr2=0x7f1fc019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fbdd9b700 1 --2- 172.21.15.46:0/18954336 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1fc01034c0 0x7f1fc019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:27.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.280+0000 7f1fbdd9b700 1 -- 172.21.15.46:0/18954336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fc01a7000 con 0x7f1fc0107df0 2024-05-09T04:48:27.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.281+0000 7f1fbdd9b700 1 --2- 172.21.15.46:0/18954336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc0107df0 0x7f1fc019c1f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4006010 tx=0x7f1fb40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:27.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.281+0000 7f1faf7fe700 1 -- 172.21.15.46:0/18954336 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4027070 con 0x7f1fc0107df0 2024-05-09T04:48:27.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.281+0000 7f1fc4e39700 1 -- 172.21.15.46:0/18954336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fc01a7290 con 0x7f1fc0107df0 2024-05-09T04:48:27.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.281+0000 7f1faf7fe700 1 -- 172.21.15.46:0/18954336 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fb4003f20 con 0x7f1fc0107df0 2024-05-09T04:48:27.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.281+0000 7f1faf7fe700 1 -- 172.21.15.46:0/18954336 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fb4022cd0 con 0x7f1fc0107df0 2024-05-09T04:48:27.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.281+0000 7f1fc4e39700 1 -- 172.21.15.46:0/18954336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fc01a7740 con 0x7f1fc0107df0 2024-05-09T04:48:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.282+0000 7f1fc4e39700 1 -- 172.21.15.46:0/18954336 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fc00512c0 con 0x7f1fc0107df0 2024-05-09T04:48:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.284+0000 7f1faf7fe700 1 -- 172.21.15.46:0/18954336 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1fb402a070 con 0x7f1fc0107df0 2024-05-09T04:48:27.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.284+0000 7f1faf7fe700 1 --2- 172.21.15.46:0/18954336 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1fa80b2990 0x7f1fa80b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:27.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.284+0000 7f1fbe59c700 1 --2- 172.21.15.46:0/18954336 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1fa80b2990 0x7f1fa80b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:27.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.284+0000 7f1faf7fe700 1 -- 172.21.15.46:0/18954336 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7979+0+0 (secure 0 0 0) 0x7f1fb40e33c0 con 0x7f1fc0107df0 2024-05-09T04:48:27.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.285+0000 7f1fbe59c700 1 --2- 172.21.15.46:0/18954336 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1fa80b2990 0x7f1fa80b4ea0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0005950 tx=0x7f1fb000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:27.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.288+0000 7f1faf7fe700 1 -- 172.21.15.46:0/18954336 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1fb4031080 con 0x7f1fc0107df0 2024-05-09T04:48:27.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.538+0000 7f1fc4e39700 1 -- 172.21.15.46:0/18954336 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f1fc0063c00 con 0x7f1fa80b2990 2024-05-09T04:48:27.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.543+0000 7f1faf7fe700 1 -- 172.21.15.46:0/18954336 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1fc0063c00 con 0x7f1fa80b2990 2024-05-09T04:48:27.543 INFO:teuthology.orchestra.run.smithi046.stdout:/volumes/qa/sv_1/6e0b2d8b-2a4b-496a-a206-fa632876bce3 2024-05-09T04:48:27.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.547+0000 7f1fc4e39700 1 -- 172.21.15.46:0/18954336 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1fa80b2990 msgr2=0x7f1fa80b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:27.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.547+0000 7f1fc4e39700 1 --2- 172.21.15.46:0/18954336 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1fa80b2990 0x7f1fa80b4ea0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1fb0005950 tx=0x7f1fb000b410 comp rx=0 tx=0).stop 2024-05-09T04:48:27.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.547+0000 7f1fc4e39700 1 -- 172.21.15.46:0/18954336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc0107df0 msgr2=0x7f1fc019c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:27.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.547+0000 7f1fc4e39700 1 --2- 172.21.15.46:0/18954336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc0107df0 0x7f1fc019c1f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4006010 tx=0x7f1fb40047b0 comp rx=0 tx=0).stop 2024-05-09T04:48:27.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.547+0000 7f1fc4e39700 1 -- 172.21.15.46:0/18954336 shutdown_connections 2024-05-09T04:48:27.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.547+0000 7f1fc4e39700 1 --2- 172.21.15.46:0/18954336 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1fa80b2990 0x7f1fa80b4ea0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:27.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.547+0000 7f1fc4e39700 1 --2- 172.21.15.46:0/18954336 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1fc0108740 0x7f1fc019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:27.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.547+0000 7f1fc4e39700 1 --2- 172.21.15.46:0/18954336 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1fc01034c0 0x7f1fc019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:27.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.547+0000 7f1fc4e39700 1 --2- 172.21.15.46:0/18954336 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1fc0107df0 0x7f1fc019c1f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:27.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.547+0000 7f1fc4e39700 1 -- 172.21.15.46:0/18954336 >> 172.21.15.46:0/18954336 conn(0x7f1fc00fedc0 msgr2=0x7f1fc01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:27.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.547+0000 7f1fc4e39700 1 -- 172.21.15.46:0/18954336 shutdown_connections 2024-05-09T04:48:27.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:27.547+0000 7f1fc4e39700 1 -- 172.21.15.46:0/18954336 wait complete. 2024-05-09T04:48:27.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-05-09T04:48:27.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-09T04:48:27.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-09T04:48:27.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-05-09T04:48:27.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-05-09T04:48:27.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-09T04:48:27.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-09T04:48:27.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-05-09T04:48:27.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-05-09T04:48:27.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:27 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-09T04:48:27.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-05-09T04:48:27.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-09T04:48:27.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-09T04:48:27.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-05-09T04:48:27.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-05-09T04:48:27.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-09T04:48:27.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-09T04:48:27.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-05-09T04:48:27.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-05-09T04:48:27.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:27 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-09T04:48:27.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]: dispatch 2024-05-09T04:48:27.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]: dispatch 2024-05-09T04:48:27.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-09T04:48:27.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]: dispatch 2024-05-09T04:48:27.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]: dispatch 2024-05-09T04:48:27.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]: dispatch 2024-05-09T04:48:27.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]: dispatch 2024-05-09T04:48:27.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-05-09T04:48:27.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]: dispatch 2024-05-09T04:48:27.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:27 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]: dispatch 2024-05-09T04:48:28.262 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:48:28.524 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:28.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:28 smithi046 ceph-mon[30136]: pgmap v310: 97 pgs: 97 active+clean; 608 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-05-09T04:48:28.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:28 smithi046 ceph-mon[30136]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:28.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-05-09T04:48:28.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-09T04:48:28.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-09T04:48:28.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-05-09T04:48:28.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-05-09T04:48:28.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-09T04:48:28.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-09T04:48:28.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-05-09T04:48:28.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-05-09T04:48:28.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-09T04:48:28.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:28 smithi046 ceph-mon[30136]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T04:48:28.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:28 smithi072 ceph-mon[34088]: pgmap v310: 97 pgs: 97 active+clean; 608 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-05-09T04:48:28.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:28 smithi072 ceph-mon[34088]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:28.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-05-09T04:48:28.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-09T04:48:28.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-09T04:48:28.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-05-09T04:48:28.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-05-09T04:48:28.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-09T04:48:28.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-09T04:48:28.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-05-09T04:48:28.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-05-09T04:48:28.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-09T04:48:28.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:28 smithi072 ceph-mon[34088]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T04:48:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:28 smithi110 ceph-mon[34904]: pgmap v310: 97 pgs: 97 active+clean; 608 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2024-05-09T04:48:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:28 smithi110 ceph-mon[34904]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:48:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-05-09T04:48:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.14", "id": [2, 4]}]': finished 2024-05-09T04:48:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-09T04:48:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [4, 6]}]': finished 2024-05-09T04:48:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.11", "id": [5, 9]}]': finished 2024-05-09T04:48:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-05-09T04:48:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1b", "id": [5, 9]}]': finished 2024-05-09T04:48:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-05-09T04:48:29.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 7]}]': finished 2024-05-09T04:48:29.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.38", "id": [3, 9]}]': finished 2024-05-09T04:48:29.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:28 smithi110 ceph-mon[34904]: osdmap e81: 12 total, 12 up, 12 in 2024-05-09T04:48:29.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:29 smithi046 ceph-mon[30136]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T04:48:29.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:29 smithi072 ceph-mon[34088]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T04:48:30.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:29 smithi110 ceph-mon[34904]: osdmap e82: 12 total, 12 up, 12 in 2024-05-09T04:48:30.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.364+0000 7f83b6ed8700 1 -- 172.21.15.46:0/3792991518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0108f10 msgr2=0x7f83b0109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.364+0000 7f83b6ed8700 1 --2- 172.21.15.46:0/3792991518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0108f10 0x7f83b0109320 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f839c0099c0 tx=0x7f839c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:48:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.364+0000 7f83b6ed8700 1 -- 172.21.15.46:0/3792991518 shutdown_connections 2024-05-09T04:48:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.364+0000 7f83b6ed8700 1 --2- 172.21.15.46:0/3792991518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b0109860 0x7f83b01111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.364+0000 7f83b6ed8700 1 --2- 172.21.15.46:0/3792991518 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83b01045e0 0x7f83b01049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.364+0000 7f83b6ed8700 1 --2- 172.21.15.46:0/3792991518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0108f10 0x7f83b0109320 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.364+0000 7f83b6ed8700 1 -- 172.21.15.46:0/3792991518 >> 172.21.15.46:0/3792991518 conn(0x7f83b00ffed0 msgr2=0x7f83b0102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.365+0000 7f83b6ed8700 1 -- 172.21.15.46:0/3792991518 shutdown_connections 2024-05-09T04:48:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.365+0000 7f83b6ed8700 1 -- 172.21.15.46:0/3792991518 wait complete. 2024-05-09T04:48:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.365+0000 7f83b6ed8700 1 Processor -- start 2024-05-09T04:48:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.365+0000 7f83b6ed8700 1 -- start start 2024-05-09T04:48:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b6ed8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83b01045e0 0x7f83b0076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:30.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b6ed8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b0108f10 0x7f83b0077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b6ed8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0109860 0x7f83b0077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b6ed8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f83b01144c0 con 0x7f83b0109860 2024-05-09T04:48:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b6ed8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f83b0114340 con 0x7f83b01045e0 2024-05-09T04:48:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b6ed8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f83b0114640 con 0x7f83b0108f10 2024-05-09T04:48:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b5475700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0109860 0x7f83b0077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b5475700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0109860 0x7f83b0077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58468/0 (socket says 172.21.15.46:58468) 2024-05-09T04:48:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b5475700 1 -- 172.21.15.46:0/1567146883 learned_addr learned my addr 172.21.15.46:0/1567146883 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83affff700 1 --2- 172.21.15.46:0/1567146883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b0108f10 0x7f83b0077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b4c74700 1 --2- 172.21.15.46:0/1567146883 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83b01045e0 0x7f83b0076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b5475700 1 -- 172.21.15.46:0/1567146883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b0108f10 msgr2=0x7f83b0077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b5475700 1 --2- 172.21.15.46:0/1567146883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b0108f10 0x7f83b0077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b5475700 1 -- 172.21.15.46:0/1567146883 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83b01045e0 msgr2=0x7f83b0076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b5475700 1 --2- 172.21.15.46:0/1567146883 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83b01045e0 0x7f83b0076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:30.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b5475700 1 -- 172.21.15.46:0/1567146883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83b01a9210 con 0x7f83b0109860 2024-05-09T04:48:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b5475700 1 --2- 172.21.15.46:0/1567146883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0109860 0x7f83b0077650 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f83a8007ee0 tx=0x7f83a800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83adffb700 1 -- 172.21.15.46:0/1567146883 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8004bb0 con 0x7f83b0109860 2024-05-09T04:48:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83adffb700 1 -- 172.21.15.46:0/1567146883 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83a8004d50 con 0x7f83b0109860 2024-05-09T04:48:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83b6ed8700 1 -- 172.21.15.46:0/1567146883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83b01a9500 con 0x7f83b0109860 2024-05-09T04:48:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.366+0000 7f83adffb700 1 -- 172.21.15.46:0/1567146883 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83a8005920 con 0x7f83b0109860 2024-05-09T04:48:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.367+0000 7f83b6ed8700 1 -- 172.21.15.46:0/1567146883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83b01a99c0 con 0x7f83b0109860 2024-05-09T04:48:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.368+0000 7f83b6ed8700 1 -- 172.21.15.46:0/1567146883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f83b00512c0 con 0x7f83b0109860 2024-05-09T04:48:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.369+0000 7f83adffb700 1 -- 172.21.15.46:0/1567146883 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f83a801a020 con 0x7f83b0109860 2024-05-09T04:48:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.370+0000 7f83adffb700 1 --2- 172.21.15.46:0/1567146883 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f83a00b2910 0x7f83a00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.370+0000 7f83b4c74700 1 --2- 172.21.15.46:0/1567146883 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f83a00b2910 0x7f83a00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.370+0000 7f83adffb700 1 -- 172.21.15.46:0/1567146883 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f83a80e1a40 con 0x7f83b0109860 2024-05-09T04:48:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.371+0000 7f83b4c74700 1 --2- 172.21.15.46:0/1567146883 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f83a00b2910 0x7f83a00b4e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009c60 tx=0x7f83a4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.375+0000 7f83adffb700 1 -- 172.21.15.46:0/1567146883 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f83a8010070 con 0x7f83b0109860 2024-05-09T04:48:30.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.641+0000 7f83b6ed8700 1 -- 172.21.15.46:0/1567146883 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f83b002e5f0 con 0x7f83b0109860 2024-05-09T04:48:30.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.642+0000 7f83adffb700 1 -- 172.21.15.46:0/1567146883 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11862 (secure 0 0 0) 0x7f83a80a9580 con 0x7f83b0109860 2024-05-09T04:48:30.642 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:48:30.643 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:48:11.653145+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24473,24503,24448]},"id":1}]} 2024-05-09T04:48:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.645+0000 7f83b6ed8700 1 -- 172.21.15.46:0/1567146883 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f83a00b2910 msgr2=0x7f83a00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.645+0000 7f83b6ed8700 1 --2- 172.21.15.46:0/1567146883 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f83a00b2910 0x7f83a00b4e20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f83a4009c60 tx=0x7f83a4009260 comp rx=0 tx=0).stop 2024-05-09T04:48:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.645+0000 7f83b6ed8700 1 -- 172.21.15.46:0/1567146883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0109860 msgr2=0x7f83b0077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.645+0000 7f83b6ed8700 1 --2- 172.21.15.46:0/1567146883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0109860 0x7f83b0077650 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f83a8007ee0 tx=0x7f83a800d3b0 comp rx=0 tx=0).stop 2024-05-09T04:48:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.645+0000 7f83b6ed8700 1 -- 172.21.15.46:0/1567146883 shutdown_connections 2024-05-09T04:48:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.645+0000 7f83b6ed8700 1 --2- 172.21.15.46:0/1567146883 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f83a00b2910 0x7f83a00b4e20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.645+0000 7f83b6ed8700 1 --2- 172.21.15.46:0/1567146883 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f83b0109860 0x7f83b0077650 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.645+0000 7f83b6ed8700 1 --2- 172.21.15.46:0/1567146883 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f83b01045e0 0x7f83b0076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.645+0000 7f83b6ed8700 1 --2- 172.21.15.46:0/1567146883 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f83b0108f10 0x7f83b0077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.645+0000 7f83b6ed8700 1 -- 172.21.15.46:0/1567146883 >> 172.21.15.46:0/1567146883 conn(0x7f83b00ffed0 msgr2=0x7f83b01086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.645+0000 7f83b6ed8700 1 -- 172.21.15.46:0/1567146883 shutdown_connections 2024-05-09T04:48:30.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:30.645+0000 7f83b6ed8700 1 -- 172.21.15.46:0/1567146883 wait complete. 2024-05-09T04:48:30.646 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-05-09T04:48:30.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:30 smithi046 ceph-mon[30136]: pgmap v313: 97 pgs: 4 peering, 93 active+clean; 630 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.5 KiB/s wr, 9 op/s; 6.2 KiB/s, 0 objects/s recovering 2024-05-09T04:48:30.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:30 smithi072 ceph-mon[34088]: pgmap v313: 97 pgs: 4 peering, 93 active+clean; 630 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.5 KiB/s wr, 9 op/s; 6.2 KiB/s, 0 objects/s recovering 2024-05-09T04:48:31.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:30 smithi110 ceph-mon[34904]: pgmap v313: 97 pgs: 4 peering, 93 active+clean; 630 KiB data, 355 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.5 KiB/s wr, 9 op/s; 6.2 KiB/s, 0 objects/s recovering 2024-05-09T04:48:31.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:48:31.606 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:31.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:31 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1567146883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:48:31.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:31 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1567146883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:48:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:31 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1567146883' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:48:32.750 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:32 smithi046 ceph-mon[30136]: pgmap v314: 97 pgs: 4 peering, 93 active+clean; 630 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.5 KiB/s wr, 9 op/s; 6.2 KiB/s, 0 objects/s recovering 2024-05-09T04:48:32.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:32 smithi072 ceph-mon[34088]: pgmap v314: 97 pgs: 4 peering, 93 active+clean; 630 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.5 KiB/s wr, 9 op/s; 6.2 KiB/s, 0 objects/s recovering 2024-05-09T04:48:33.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:32 smithi110 ceph-mon[34904]: pgmap v314: 97 pgs: 4 peering, 93 active+clean; 630 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 4.5 KiB/s wr, 9 op/s; 6.2 KiB/s, 0 objects/s recovering 2024-05-09T04:48:33.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.340+0000 7f7bcd88e700 1 -- 172.21.15.46:0/1739130657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bc8103a80 msgr2=0x7f7bc8110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:33.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.340+0000 7f7bcd88e700 1 --2- 172.21.15.46:0/1739130657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bc8103a80 0x7f7bc8110710 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009990 tx=0x7f7bbc009ca0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.340+0000 7f7bcd88e700 1 -- 172.21.15.46:0/1739130657 shutdown_connections 2024-05-09T04:48:33.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.340+0000 7f7bcd88e700 1 --2- 172.21.15.46:0/1739130657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bc8110c50 0x7f7bc8113070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.340+0000 7f7bcd88e700 1 --2- 172.21.15.46:0/1739130657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bc81030a0 0x7f7bc81034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.340+0000 7f7bcd88e700 1 --2- 172.21.15.46:0/1739130657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bc8103a80 0x7f7bc8110710 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.340+0000 7f7bcd88e700 1 -- 172.21.15.46:0/1739130657 >> 172.21.15.46:0/1739130657 conn(0x7f7bc80fef30 msgr2=0x7f7bc8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.340+0000 7f7bcd88e700 1 -- 172.21.15.46:0/1739130657 shutdown_connections 2024-05-09T04:48:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.340+0000 7f7bcd88e700 1 -- 172.21.15.46:0/1739130657 wait complete. 2024-05-09T04:48:33.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.341+0000 7f7bcd88e700 1 Processor -- start 2024-05-09T04:48:33.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.341+0000 7f7bcd88e700 1 -- start start 2024-05-09T04:48:33.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.341+0000 7f7bcd88e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bc81030a0 0x7f7bc819d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:33.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.341+0000 7f7bcd88e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bc8103a80 0x7f7bc819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:33.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.341+0000 7f7bcd88e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bc8110c50 0x7f7bc81a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:33.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.341+0000 7f7bcd88e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc8116680 con 0x7f7bc8110c50 2024-05-09T04:48:33.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.341+0000 7f7bcd88e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc8116500 con 0x7f7bc81030a0 2024-05-09T04:48:33.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.341+0000 7f7bcd88e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7bc8116800 con 0x7f7bc8103a80 2024-05-09T04:48:33.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.341+0000 7f7bc77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bc8110c50 0x7f7bc81a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.341+0000 7f7bc77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bc8110c50 0x7f7bc81a1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58472/0 (socket says 172.21.15.46:58472) 2024-05-09T04:48:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.341+0000 7f7bc77fe700 1 -- 172.21.15.46:0/1859058557 learned_addr learned my addr 172.21.15.46:0/1859058557 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.341+0000 7f7bc67fc700 1 --2- 172.21.15.46:0/1859058557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bc8103a80 0x7f7bc819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.341+0000 7f7bc6ffd700 1 --2- 172.21.15.46:0/1859058557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bc81030a0 0x7f7bc819d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.342+0000 7f7bc77fe700 1 -- 172.21.15.46:0/1859058557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bc8103a80 msgr2=0x7f7bc819d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.342+0000 7f7bc77fe700 1 --2- 172.21.15.46:0/1859058557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bc8103a80 0x7f7bc819d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.342+0000 7f7bc77fe700 1 -- 172.21.15.46:0/1859058557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bc81030a0 msgr2=0x7f7bc819d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.342+0000 7f7bc77fe700 1 --2- 172.21.15.46:0/1859058557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bc81030a0 0x7f7bc819d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.342+0000 7f7bc77fe700 1 -- 172.21.15.46:0/1859058557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7bc81a1f20 con 0x7f7bc8110c50 2024-05-09T04:48:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.342+0000 7f7bc77fe700 1 --2- 172.21.15.46:0/1859058557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bc8110c50 0x7f7bc81a1880 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0007ee0 tx=0x7f7bc000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.342+0000 7f7bcc88c700 1 -- 172.21.15.46:0/1859058557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0004bb0 con 0x7f7bc8110c50 2024-05-09T04:48:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.342+0000 7f7bcc88c700 1 -- 172.21.15.46:0/1859058557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7bc0004d50 con 0x7f7bc8110c50 2024-05-09T04:48:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.342+0000 7f7bcd88e700 1 -- 172.21.15.46:0/1859058557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7bc81a2210 con 0x7f7bc8110c50 2024-05-09T04:48:33.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.342+0000 7f7bcc88c700 1 -- 172.21.15.46:0/1859058557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7bc0005920 con 0x7f7bc8110c50 2024-05-09T04:48:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.342+0000 7f7bcd88e700 1 -- 172.21.15.46:0/1859058557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7bc81a9b30 con 0x7f7bc8110c50 2024-05-09T04:48:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.343+0000 7f7bcd88e700 1 -- 172.21.15.46:0/1859058557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7bc80512c0 con 0x7f7bc8110c50 2024-05-09T04:48:33.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.346+0000 7f7bcc88c700 1 -- 172.21.15.46:0/1859058557 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7bc001a020 con 0x7f7bc8110c50 2024-05-09T04:48:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.347+0000 7f7bcc88c700 1 --2- 172.21.15.46:0/1859058557 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7bb00b2990 0x7f7bb00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.347+0000 7f7bc6ffd700 1 --2- 172.21.15.46:0/1859058557 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7bb00b2990 0x7f7bb00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:33.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.347+0000 7f7bcc88c700 1 -- 172.21.15.46:0/1859058557 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f7bc00e2060 con 0x7f7bc8110c50 2024-05-09T04:48:33.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.347+0000 7f7bc6ffd700 1 --2- 172.21.15.46:0/1859058557 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7bb00b2990 0x7f7bb00b4ea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8005950 tx=0x7f7bb800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:33.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.350+0000 7f7bcc88c700 1 -- 172.21.15.46:0/1859058557 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7bc0010070 con 0x7f7bc8110c50 2024-05-09T04:48:33.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.567+0000 7f7bcd88e700 1 -- 172.21.15.46:0/1859058557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7bc802e5f0 con 0x7f7bc8110c50 2024-05-09T04:48:33.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.569+0000 7f7bcc88c700 1 -- 172.21.15.46:0/1859058557 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22637 (secure 0 0 0) 0x7f7bc00a9ba0 con 0x7f7bc8110c50 2024-05-09T04:48:33.569 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:48:33.569 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"fsid":"ecf9ae2c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:25.633876+0000","modified":"2024-05-09T04:48:28.548796+0000","last_up_change":"2024-05-09T04:46:46.418597+0000","last_in_change":"2024-05-09T04:46:25.903582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:42:53.301400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:47:43.413569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:47:47.264915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d79cd3e7-6f82-4d60-96d9-26d8146c962f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6803","nonce":1576436508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6805","nonce":1576436508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6809","nonce":1576436508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6807","nonce":1576436508}]},"public_addr":"172.21.15.46:6803/1576436508","cluster_addr":"172.21.15.46:6805/1576436508","heartbeat_back_addr":"172.21.15.46:6809/1576436508","heartbeat_front_addr":"172.21.15.46:6807/1576436508","state":["exists","up"]},{"osd":1,"uuid":"698eba80-1ebd-4076-b6f9-953d2e7daf76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6801","nonce":379039775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6803","nonce":379039775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6807","nonce":379039775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6805","nonce":379039775}]},"public_addr":"172.21.15.72:6801/379039775","cluster_addr":"172.21.15.72:6803/379039775","heartbeat_back_addr":"172.21.15.72:6807/379039775","heartbeat_front_addr":"172.21.15.72:6805/379039775","state":["exists","up"]},{"osd":2,"uuid":"c549b131-cfa7-4724-926f-83e91a752d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6801","nonce":293447227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6803","nonce":293447227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6807","nonce":293447227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6805","nonce":293447227}]},"public_addr":"172.21.15.110:6801/293447227","cluster_addr":"172.21.15.110:6803/293447227","heartbeat_back_addr":"172.21.15.110:6807/293447227","heartbeat_front_addr":"172.21.15.110:6805/293447227","state":["exists","up"]},{"osd":3,"uuid":"549bb4f7-54d9-4cbc-ad40-7f3e717964f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6811","nonce":1115835653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6813","nonce":1115835653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6817","nonce":1115835653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6815","nonce":1115835653}]},"public_addr":"172.21.15.46:6811/1115835653","cluster_addr":"172.21.15.46:6813/1115835653","heartbeat_back_addr":"172.21.15.46:6817/1115835653","heartbeat_front_addr":"172.21.15.46:6815/1115835653","state":["exists","up"]},{"osd":4,"uuid":"444cd36b-6034-46cd-9a6b-29dce4ba6405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6809","nonce":59186693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6811","nonce":59186693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6815","nonce":59186693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6813","nonce":59186693}]},"public_addr":"172.21.15.72:6809/59186693","cluster_addr":"172.21.15.72:6811/59186693","heartbeat_back_addr":"172.21.15.72:6815/59186693","heartbeat_front_addr":"172.21.15.72:6813/59186693","state":["exists","up"]},{"osd":5,"uuid":"2c2bdd2f-43cf-4208-9ae1-f912807c84e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6809","nonce":1712162502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6811","nonce":1712162502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6815","nonce":1712162502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6813","nonce":1712162502}]},"public_addr":"172.21.15.110:6809/1712162502","cluster_addr":"172.21.15.110:6811/1712162502","heartbeat_back_addr":"172.21.15.110:6815/1712162502","heartbeat_front_addr":"172.21.15.110:6813/1712162502","state":["exists","up"]},{"osd":6,"uuid":"beb0b87f-c3a4-40dd-b224-1843e9301ca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6819","nonce":2429650065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6821","nonce":2429650065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6825","nonce":2429650065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6823","nonce":2429650065}]},"public_addr":"172.21.15.46:6819/2429650065","cluster_addr":"172.21.15.46:6821/2429650065","heartbeat_back_addr":"172.21.15.46:6825/2429650065","heartbeat_front_addr":"172.21.15.46:6823/2429650065","state":["exists","up"]},{"osd":7,"uuid":"4a634675-fad7-4a43-840c-8dec2631dec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6817","nonce":337796672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6819","nonce":337796672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6823","nonce":337796672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6821","nonce":337796672}]},"public_addr":"172.21.15.72:6817/337796672","cluster_addr":"172.21.15.72:6819/337796672","heartbeat_back_addr":"172.21.15.72:6823/337796672","heartbeat_front_addr":"172.21.15.72:6821/337796672","state":["exists","up"]},{"osd":8,"uuid":"01b5f638-90f0-4cf4-8042-20d20a725669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6817","nonce":189513203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6819","nonce":189513203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6823","nonce":189513203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6821","nonce":189513203}]},"public_addr":"172.21.15.110:6817/189513203","cluster_addr":"172.21.15.110:6819/189513203","heartbeat_back_addr":"172.21.15.110:6823/189513203","heartbeat_front_addr":"172.21.15.110:6821/189513203","state":["exists","up"]},{"osd":9,"uuid":"e423de41-50ec-48f4-a4ed-216de33c4fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6827","nonce":1762262822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6829","nonce":1762262822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6833","nonce":1762262822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6831","nonce":1762262822}]},"public_addr":"172.21.15.46:6827/1762262822","cluster_addr":"172.21.15.46:6829/1762262822","heartbeat_back_addr":"172.21.15.46:6833/1762262822","heartbeat_front_addr":"172.21.15.46:6831/1762262822","state":["exists","up"]},{"osd":10,"uuid":"f594d004-9f80-4508-b127-fb693d355f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6825","nonce":38453898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6827","nonce":38453898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6831","nonce":38453898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6829","nonce":38453898}]},"public_addr":"172.21.15.72:6825/38453898","cluster_addr":"172.21.15.72:6827/38453898","heartbeat_back_addr":"172.21.15.72:6831/38453898","heartbeat_front_addr":"172.21.15.72:6829/38453898","state":["exists","up"]},{"osd":11,"uuid":"f771d29f-89e3-4f96-9c29-00e424230c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6825","nonce":4153529632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6827","nonce":4153529632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6831","nonce":4153529632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6829","nonce":4153529632}]},"public_addr":"172.21.15.110:6825/4153529632","cluster_addr":"172.21.15.110:6827/4153529632","heartbeat_back_addr":"172.21.15.110:6831/4153529632","heartbeat_front_addr":"172.21.15.110:6829/4153529632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:02.615801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:26.453477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:50.334627+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:15.357320+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:40.403549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:06.030093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:31.904091+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:57.616372+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:24.045975+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:51.160898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:17.597845+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:44.636629+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1596672353":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1846482551":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/1069571878":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6800/2738158981":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/3625745146":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6801/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/564539954":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1797199863":"2024-05-10T04:40:27.201119+0000","172.21.15.46:6801/2738158981":"2024-05-10T04:39:59.515628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:48:33.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.572+0000 7f7bcd88e700 1 -- 172.21.15.46:0/1859058557 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7bb00b2990 msgr2=0x7f7bb00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:33.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.572+0000 7f7bcd88e700 1 --2- 172.21.15.46:0/1859058557 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7bb00b2990 0x7f7bb00b4ea0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7bb8005950 tx=0x7f7bb800b410 comp rx=0 tx=0).stop 2024-05-09T04:48:33.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.572+0000 7f7bcd88e700 1 -- 172.21.15.46:0/1859058557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bc8110c50 msgr2=0x7f7bc81a1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:33.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.572+0000 7f7bcd88e700 1 --2- 172.21.15.46:0/1859058557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bc8110c50 0x7f7bc81a1880 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f7bc0007ee0 tx=0x7f7bc000d3b0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.572+0000 7f7bcd88e700 1 -- 172.21.15.46:0/1859058557 shutdown_connections 2024-05-09T04:48:33.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.572+0000 7f7bcd88e700 1 --2- 172.21.15.46:0/1859058557 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7bb00b2990 0x7f7bb00b4ea0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.572+0000 7f7bcd88e700 1 --2- 172.21.15.46:0/1859058557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7bc8110c50 0x7f7bc81a1880 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.572+0000 7f7bcd88e700 1 --2- 172.21.15.46:0/1859058557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7bc81030a0 0x7f7bc819d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.572+0000 7f7bcd88e700 1 --2- 172.21.15.46:0/1859058557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7bc8103a80 0x7f7bc819d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:33.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.572+0000 7f7bcd88e700 1 -- 172.21.15.46:0/1859058557 >> 172.21.15.46:0/1859058557 conn(0x7f7bc80fef30 msgr2=0x7f7bc8112890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:33.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.572+0000 7f7bcd88e700 1 -- 172.21.15.46:0/1859058557 shutdown_connections 2024-05-09T04:48:33.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:33.572+0000 7f7bcd88e700 1 -- 172.21.15.46:0/1859058557 wait complete. 2024-05-09T04:48:33.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:33 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1859058557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:48:33.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:33 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1859058557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:48:34.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:33 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1859058557' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:48:34.328 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T04:48:34.328 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T04:48:34.593 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:34.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:34 smithi046 ceph-mon[30136]: pgmap v315: 97 pgs: 3 peering, 94 active+clean; 630 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.2 KiB/s wr, 5 op/s; 6.2 KiB/s, 1 keys/s, 1 objects/s recovering 2024-05-09T04:48:34.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:34 smithi072 ceph-mon[34088]: pgmap v315: 97 pgs: 3 peering, 94 active+clean; 630 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.2 KiB/s wr, 5 op/s; 6.2 KiB/s, 1 keys/s, 1 objects/s recovering 2024-05-09T04:48:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:34 smithi110 ceph-mon[34904]: pgmap v315: 97 pgs: 3 peering, 94 active+clean; 630 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 3.2 KiB/s wr, 5 op/s; 6.2 KiB/s, 1 keys/s, 1 objects/s recovering 2024-05-09T04:48:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.336+0000 7ff46cb89700 1 -- 172.21.15.46:0/3801181990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff468107df0 msgr2=0x7ff468108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:36.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.336+0000 7ff46cb89700 1 --2- 172.21.15.46:0/3801181990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff468107df0 0x7ff468108200 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff45c0099c0 tx=0x7ff45c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:48:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.336+0000 7ff46cb89700 1 -- 172.21.15.46:0/3801181990 shutdown_connections 2024-05-09T04:48:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.336+0000 7ff46cb89700 1 --2- 172.21.15.46:0/3801181990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff468108740 0x7ff4681100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.336+0000 7ff46cb89700 1 --2- 172.21.15.46:0/3801181990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff4681034c0 0x7ff4681038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.336+0000 7ff46cb89700 1 --2- 172.21.15.46:0/3801181990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff468107df0 0x7ff468108200 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.336+0000 7ff46cb89700 1 -- 172.21.15.46:0/3801181990 >> 172.21.15.46:0/3801181990 conn(0x7ff4680fedf0 msgr2=0x7ff468101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.336+0000 7ff46cb89700 1 -- 172.21.15.46:0/3801181990 shutdown_connections 2024-05-09T04:48:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.336+0000 7ff46cb89700 1 -- 172.21.15.46:0/3801181990 wait complete. 2024-05-09T04:48:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.337+0000 7ff46cb89700 1 Processor -- start 2024-05-09T04:48:36.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.337+0000 7ff46cb89700 1 -- start start 2024-05-09T04:48:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.337+0000 7ff46cb89700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4681034c0 0x7ff4680ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.337+0000 7ff46cb89700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff468107df0 0x7ff4680ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.337+0000 7ff46cb89700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff468108740 0x7ff4680ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.337+0000 7ff46cb89700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff468113350 con 0x7ff4681034c0 2024-05-09T04:48:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.337+0000 7ff46cb89700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff4681131d0 con 0x7ff468107df0 2024-05-09T04:48:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.337+0000 7ff46cb89700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff4681134d0 con 0x7ff468108740 2024-05-09T04:48:36.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.337+0000 7ff46659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4681034c0 0x7ff4680ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.337+0000 7ff46659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4681034c0 0x7ff4680ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47440/0 (socket says 172.21.15.46:47440) 2024-05-09T04:48:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.337+0000 7ff46659c700 1 -- 172.21.15.46:0/2070709027 learned_addr learned my addr 172.21.15.46:0/2070709027 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.337+0000 7ff466d9d700 1 --2- 172.21.15.46:0/2070709027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff468108740 0x7ff4680ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.337+0000 7ff465d9b700 1 --2- 172.21.15.46:0/2070709027 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff468107df0 0x7ff4680ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.338+0000 7ff46659c700 1 -- 172.21.15.46:0/2070709027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff468108740 msgr2=0x7ff4680ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.338+0000 7ff46659c700 1 --2- 172.21.15.46:0/2070709027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff468108740 0x7ff4680ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.338+0000 7ff46659c700 1 -- 172.21.15.46:0/2070709027 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff468107df0 msgr2=0x7ff4680ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.338+0000 7ff46659c700 1 --2- 172.21.15.46:0/2070709027 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff468107df0 0x7ff4680ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.338+0000 7ff46659c700 1 -- 172.21.15.46:0/2070709027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4681a9270 con 0x7ff4681034c0 2024-05-09T04:48:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.338+0000 7ff465d9b700 1 --2- 172.21.15.46:0/2070709027 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff468107df0 0x7ff4680ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:48:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.338+0000 7ff46659c700 1 --2- 172.21.15.46:0/2070709027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4681034c0 0x7ff4680ff2d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff45800c920 tx=0x7ff45800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.338+0000 7ff4577fe700 1 -- 172.21.15.46:0/2070709027 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458007b00 con 0x7ff4681034c0 2024-05-09T04:48:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.338+0000 7ff46cb89700 1 -- 172.21.15.46:0/2070709027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4681a9560 con 0x7ff4681034c0 2024-05-09T04:48:36.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.338+0000 7ff4577fe700 1 -- 172.21.15.46:0/2070709027 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff458004ce0 con 0x7ff4681034c0 2024-05-09T04:48:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.338+0000 7ff4577fe700 1 -- 172.21.15.46:0/2070709027 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff458005550 con 0x7ff4681034c0 2024-05-09T04:48:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.338+0000 7ff46cb89700 1 -- 172.21.15.46:0/2070709027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4681a9aa0 con 0x7ff4681034c0 2024-05-09T04:48:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.339+0000 7ff46cb89700 1 -- 172.21.15.46:0/2070709027 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4680512c0 con 0x7ff4681034c0 2024-05-09T04:48:36.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.341+0000 7ff4577fe700 1 -- 172.21.15.46:0/2070709027 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff458007ca0 con 0x7ff4681034c0 2024-05-09T04:48:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.341+0000 7ff4577fe700 1 --2- 172.21.15.46:0/2070709027 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff4500b2910 0x7ff4500b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.341+0000 7ff465d9b700 1 --2- 172.21.15.46:0/2070709027 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff4500b2910 0x7ff4500b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.341+0000 7ff4577fe700 1 -- 172.21.15.46:0/2070709027 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7ff4580e22c0 con 0x7ff4681034c0 2024-05-09T04:48:36.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.342+0000 7ff465d9b700 1 --2- 172.21.15.46:0/2070709027 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff4500b2910 0x7ff4500b4e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff45c006010 tx=0x7ff45c005e80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:36.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.346+0000 7ff4577fe700 1 -- 172.21.15.46:0/2070709027 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff458014030 con 0x7ff4681034c0 2024-05-09T04:48:36.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.558+0000 7ff46cb89700 1 -- 172.21.15.46:0/2070709027 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff468063c00 con 0x7ff4500b2910 2024-05-09T04:48:36.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.566+0000 7ff4577fe700 1 -- 172.21.15.46:0/2070709027 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258971 (secure 0 0 0) 0x7ff468063c00 con 0x7ff4500b2910 2024-05-09T04:48:36.570 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:48:36.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.570+0000 7ff46cb89700 1 -- 172.21.15.46:0/2070709027 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff4500b2910 msgr2=0x7ff4500b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.570+0000 7ff46cb89700 1 --2- 172.21.15.46:0/2070709027 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff4500b2910 0x7ff4500b4e20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ff45c006010 tx=0x7ff45c005e80 comp rx=0 tx=0).stop 2024-05-09T04:48:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.570+0000 7ff46cb89700 1 -- 172.21.15.46:0/2070709027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4681034c0 msgr2=0x7ff4680ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.570+0000 7ff46cb89700 1 --2- 172.21.15.46:0/2070709027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4681034c0 0x7ff4680ff2d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ff45800c920 tx=0x7ff45800cc30 comp rx=0 tx=0).stop 2024-05-09T04:48:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.570+0000 7ff46cb89700 1 -- 172.21.15.46:0/2070709027 shutdown_connections 2024-05-09T04:48:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.570+0000 7ff46cb89700 1 --2- 172.21.15.46:0/2070709027 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff4500b2910 0x7ff4500b4e20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.570+0000 7ff46cb89700 1 --2- 172.21.15.46:0/2070709027 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff468108740 0x7ff4680ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.570+0000 7ff46cb89700 1 --2- 172.21.15.46:0/2070709027 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff4681034c0 0x7ff4680ff2d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.570+0000 7ff46cb89700 1 --2- 172.21.15.46:0/2070709027 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff468107df0 0x7ff4680ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.570+0000 7ff46cb89700 1 -- 172.21.15.46:0/2070709027 >> 172.21.15.46:0/2070709027 conn(0x7ff4680fedf0 msgr2=0x7ff468075e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.570+0000 7ff46cb89700 1 -- 172.21.15.46:0/2070709027 shutdown_connections 2024-05-09T04:48:36.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:36.570+0000 7ff46cb89700 1 -- 172.21.15.46:0/2070709027 wait complete. 2024-05-09T04:48:36.574 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T04:48:36.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:36 smithi046 ceph-mon[30136]: pgmap v316: 97 pgs: 97 active+clean; 635 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 6.2 KiB/s, 1 keys/s, 1 objects/s recovering 2024-05-09T04:48:36.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:36 smithi072 ceph-mon[34088]: pgmap v316: 97 pgs: 97 active+clean; 635 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 6.2 KiB/s, 1 keys/s, 1 objects/s recovering 2024-05-09T04:48:37.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:36 smithi110 ceph-mon[34904]: pgmap v316: 97 pgs: 97 active+clean; 635 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 9 op/s; 6.2 KiB/s, 1 keys/s, 1 objects/s recovering 2024-05-09T04:48:37.325 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-05-09T04:48:35.308091+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":650693,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":258,"num_read_kb":214,"num_write":286,"num_write_kb":2258,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3002535,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":231,"ondisk_log_size":231,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":292,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364184,"kb_used_data":10800,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577640,"statfs":{"total":1151940427776,"available":1151567503360,"internally_reserved":0,"allocated":11059200,"data_stored":7129011,"data_compressed":186290,"data_compressed_allocated":245760,"data_compressed_original":2015232,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28007,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":58,"num_read_kb":48,"num_write":19,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":50695,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002405"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.553114+0000","last_change":"2024-05-09T04:47:48.408878+0000","last_active":"2024-05-09T04:48:28.553114+0000","last_peered":"2024-05-09T04:48:28.553114+0000","last_clean":"2024-05-09T04:48:28.553114+0000","last_became_active":"2024-05-09T04:47:48.408479+0000","last_became_peered":"2024-05-09T04:47:48.408479+0000","last_unstale":"2024-05-09T04:48:28.553114+0000","last_undegraded":"2024-05-09T04:48:28.553114+0000","last_fullsized":"2024-05-09T04:48:28.553114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:02.494768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556491+0000","last_change":"2024-05-09T04:47:48.415650+0000","last_active":"2024-05-09T04:48:28.556491+0000","last_peered":"2024-05-09T04:48:28.556491+0000","last_clean":"2024-05-09T04:48:28.556491+0000","last_became_active":"2024-05-09T04:47:48.415322+0000","last_became_peered":"2024-05-09T04:47:48.415322+0000","last_unstale":"2024-05-09T04:48:28.556491+0000","last_undegraded":"2024-05-09T04:48:28.556491+0000","last_fullsized":"2024-05-09T04:48:28.556491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.142513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556243+0000","last_change":"2024-05-09T04:47:48.417706+0000","last_active":"2024-05-09T04:48:28.556243+0000","last_peered":"2024-05-09T04:48:28.556243+0000","last_clean":"2024-05-09T04:48:28.556243+0000","last_became_active":"2024-05-09T04:47:48.412068+0000","last_became_peered":"2024-05-09T04:47:48.412068+0000","last_unstale":"2024-05-09T04:48:28.556243+0000","last_undegraded":"2024-05-09T04:48:28.556243+0000","last_fullsized":"2024-05-09T04:48:28.556243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:44.602567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.553597+0000","last_change":"2024-05-09T04:47:48.415751+0000","last_active":"2024-05-09T04:48:28.553597+0000","last_peered":"2024-05-09T04:48:28.553597+0000","last_clean":"2024-05-09T04:48:28.553597+0000","last_became_active":"2024-05-09T04:47:48.414312+0000","last_became_peered":"2024-05-09T04:47:48.414312+0000","last_unstale":"2024-05-09T04:48:28.553597+0000","last_undegraded":"2024-05-09T04:48:28.553597+0000","last_fullsized":"2024-05-09T04:48:28.553597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:12.202761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556094+0000","last_change":"2024-05-09T04:47:48.415919+0000","last_active":"2024-05-09T04:48:28.556094+0000","last_peered":"2024-05-09T04:48:28.556094+0000","last_clean":"2024-05-09T04:48:28.556094+0000","last_became_active":"2024-05-09T04:47:48.415533+0000","last_became_peered":"2024-05-09T04:47:48.415533+0000","last_unstale":"2024-05-09T04:48:28.556094+0000","last_undegraded":"2024-05-09T04:48:28.556094+0000","last_fullsized":"2024-05-09T04:48:28.556094+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:32.262416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557708+0000","last_change":"2024-05-09T04:47:48.419705+0000","last_active":"2024-05-09T04:48:28.557708+0000","last_peered":"2024-05-09T04:48:28.557708+0000","last_clean":"2024-05-09T04:48:28.557708+0000","last_became_active":"2024-05-09T04:47:48.419339+0000","last_became_peered":"2024-05-09T04:47:48.419339+0000","last_unstale":"2024-05-09T04:48:28.557708+0000","last_undegraded":"2024-05-09T04:48:28.557708+0000","last_fullsized":"2024-05-09T04:48:28.557708+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:50:23.754071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555469+0000","last_change":"2024-05-09T04:47:48.404106+0000","last_active":"2024-05-09T04:48:28.555469+0000","last_peered":"2024-05-09T04:48:28.555469+0000","last_clean":"2024-05-09T04:48:28.555469+0000","last_became_active":"2024-05-09T04:47:48.403724+0000","last_became_peered":"2024-05-09T04:47:48.403724+0000","last_unstale":"2024-05-09T04:48:28.555469+0000","last_undegraded":"2024-05-09T04:48:28.555469+0000","last_fullsized":"2024-05-09T04:48:28.555469+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:41.134752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.562515+0000","last_change":"2024-05-09T04:48:28.562515+0000","last_active":"2024-05-09T04:48:28.562515+0000","last_peered":"2024-05-09T04:48:28.562515+0000","last_clean":"2024-05-09T04:48:28.562515+0000","last_became_active":"2024-05-09T04:48:28.562104+0000","last_became_peered":"2024-05-09T04:48:28.562104+0000","last_unstale":"2024-05-09T04:48:28.562515+0000","last_undegraded":"2024-05-09T04:48:28.562515+0000","last_fullsized":"2024-05-09T04:48:28.562515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:33.342877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.561601+0000","last_change":"2024-05-09T04:47:48.400467+0000","last_active":"2024-05-09T04:48:28.561601+0000","last_peered":"2024-05-09T04:48:28.561601+0000","last_clean":"2024-05-09T04:48:28.561601+0000","last_became_active":"2024-05-09T04:47:48.399960+0000","last_became_peered":"2024-05-09T04:47:48.399960+0000","last_unstale":"2024-05-09T04:48:28.561601+0000","last_undegraded":"2024-05-09T04:48:28.561601+0000","last_fullsized":"2024-05-09T04:48:28.561601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:23:44.473053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558256+0000","last_change":"2024-05-09T04:47:48.418970+0000","last_active":"2024-05-09T04:48:28.558256+0000","last_peered":"2024-05-09T04:48:28.558256+0000","last_clean":"2024-05-09T04:48:28.558256+0000","last_became_active":"2024-05-09T04:47:48.418574+0000","last_became_peered":"2024-05-09T04:47:48.418574+0000","last_unstale":"2024-05-09T04:48:28.558256+0000","last_undegraded":"2024-05-09T04:48:28.558256+0000","last_fullsized":"2024-05-09T04:48:28.558256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:48.075893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556185+0000","last_change":"2024-05-09T04:47:48.409879+0000","last_active":"2024-05-09T04:48:28.556185+0000","last_peered":"2024-05-09T04:48:28.556185+0000","last_clean":"2024-05-09T04:48:28.556185+0000","last_became_active":"2024-05-09T04:47:48.409515+0000","last_became_peered":"2024-05-09T04:47:48.409515+0000","last_unstale":"2024-05-09T04:48:28.556185+0000","last_undegraded":"2024-05-09T04:48:28.556185+0000","last_fullsized":"2024-05-09T04:48:28.556185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:15.546214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.553511+0000","last_change":"2024-05-09T04:47:48.417556+0000","last_active":"2024-05-09T04:48:28.553511+0000","last_peered":"2024-05-09T04:48:28.553511+0000","last_clean":"2024-05-09T04:48:28.553511+0000","last_became_active":"2024-05-09T04:47:48.416787+0000","last_became_peered":"2024-05-09T04:47:48.416787+0000","last_unstale":"2024-05-09T04:48:28.553511+0000","last_undegraded":"2024-05-09T04:48:28.553511+0000","last_fullsized":"2024-05-09T04:48:28.553511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:20.360232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552315+0000","last_change":"2024-05-09T04:47:48.415145+0000","last_active":"2024-05-09T04:48:28.552315+0000","last_peered":"2024-05-09T04:48:28.552315+0000","last_clean":"2024-05-09T04:48:28.552315+0000","last_became_active":"2024-05-09T04:47:48.414830+0000","last_became_peered":"2024-05-09T04:47:48.414830+0000","last_unstale":"2024-05-09T04:48:28.552315+0000","last_undegraded":"2024-05-09T04:48:28.552315+0000","last_fullsized":"2024-05-09T04:48:28.552315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:55.936409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556427+0000","last_change":"2024-05-09T04:47:48.413977+0000","last_active":"2024-05-09T04:48:28.556427+0000","last_peered":"2024-05-09T04:48:28.556427+0000","last_clean":"2024-05-09T04:48:28.556427+0000","last_became_active":"2024-05-09T04:47:48.413604+0000","last_became_peered":"2024-05-09T04:47:48.413604+0000","last_unstale":"2024-05-09T04:48:28.556427+0000","last_undegraded":"2024-05-09T04:48:28.556427+0000","last_fullsized":"2024-05-09T04:48:28.556427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:25.462958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556117+0000","last_change":"2024-05-09T04:47:48.419000+0000","last_active":"2024-05-09T04:48:28.556117+0000","last_peered":"2024-05-09T04:48:28.556117+0000","last_clean":"2024-05-09T04:48:28.556117+0000","last_became_active":"2024-05-09T04:47:48.418457+0000","last_became_peered":"2024-05-09T04:47:48.418457+0000","last_unstale":"2024-05-09T04:48:28.556117+0000","last_undegraded":"2024-05-09T04:48:28.556117+0000","last_fullsized":"2024-05-09T04:48:28.556117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:36.325460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.560792+0000","last_change":"2024-05-09T04:47:48.406754+0000","last_active":"2024-05-09T04:48:28.560792+0000","last_peered":"2024-05-09T04:48:28.560792+0000","last_clean":"2024-05-09T04:48:28.560792+0000","last_became_active":"2024-05-09T04:47:48.406396+0000","last_became_peered":"2024-05-09T04:47:48.406396+0000","last_unstale":"2024-05-09T04:48:28.560792+0000","last_undegraded":"2024-05-09T04:48:28.560792+0000","last_fullsized":"2024-05-09T04:48:28.560792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:46.421061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.553262+0000","last_change":"2024-05-09T04:47:48.409556+0000","last_active":"2024-05-09T04:48:28.553262+0000","last_peered":"2024-05-09T04:48:28.553262+0000","last_clean":"2024-05-09T04:48:28.553262+0000","last_became_active":"2024-05-09T04:47:48.409145+0000","last_became_peered":"2024-05-09T04:47:48.409145+0000","last_unstale":"2024-05-09T04:48:28.553262+0000","last_undegraded":"2024-05-09T04:48:28.553262+0000","last_fullsized":"2024-05-09T04:48:28.553262+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:25.486257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555875+0000","last_change":"2024-05-09T04:47:48.415546+0000","last_active":"2024-05-09T04:48:28.555875+0000","last_peered":"2024-05-09T04:48:28.555875+0000","last_clean":"2024-05-09T04:48:28.555875+0000","last_became_active":"2024-05-09T04:47:48.415221+0000","last_became_peered":"2024-05-09T04:47:48.415221+0000","last_unstale":"2024-05-09T04:48:28.555875+0000","last_undegraded":"2024-05-09T04:48:28.555875+0000","last_fullsized":"2024-05-09T04:48:28.555875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:57:29.112409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558794+0000","last_change":"2024-05-09T04:47:48.413106+0000","last_active":"2024-05-09T04:48:28.558794+0000","last_peered":"2024-05-09T04:48:28.558794+0000","last_clean":"2024-05-09T04:48:28.558794+0000","last_became_active":"2024-05-09T04:47:48.412700+0000","last_became_peered":"2024-05-09T04:47:48.412700+0000","last_unstale":"2024-05-09T04:48:28.558794+0000","last_undegraded":"2024-05-09T04:48:28.558794+0000","last_fullsized":"2024-05-09T04:48:28.558794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:37:31.682008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555642+0000","last_change":"2024-05-09T04:47:45.386931+0000","last_active":"2024-05-09T04:48:28.555642+0000","last_peered":"2024-05-09T04:48:28.555642+0000","last_clean":"2024-05-09T04:48:28.555642+0000","last_became_active":"2024-05-09T04:47:45.386562+0000","last_became_peered":"2024-05-09T04:47:45.386562+0000","last_unstale":"2024-05-09T04:48:28.555642+0000","last_undegraded":"2024-05-09T04:48:28.555642+0000","last_fullsized":"2024-05-09T04:48:28.555642+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:59.335687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.662059+0000","last_change":"2024-05-09T04:47:48.411064+0000","last_active":"2024-05-09T04:48:28.662059+0000","last_peered":"2024-05-09T04:48:28.662059+0000","last_clean":"2024-05-09T04:48:28.662059+0000","last_became_active":"2024-05-09T04:47:48.410689+0000","last_became_peered":"2024-05-09T04:47:48.410689+0000","last_unstale":"2024-05-09T04:48:28.662059+0000","last_undegraded":"2024-05-09T04:48:28.662059+0000","last_fullsized":"2024-05-09T04:48:28.662059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:21.178831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.662343+0000","last_change":"2024-05-09T04:47:45.612837+0000","last_active":"2024-05-09T04:48:28.662343+0000","last_peered":"2024-05-09T04:48:28.662343+0000","last_clean":"2024-05-09T04:48:28.662343+0000","last_became_active":"2024-05-09T04:47:45.612307+0000","last_became_peered":"2024-05-09T04:47:45.612307+0000","last_unstale":"2024-05-09T04:48:28.662343+0000","last_undegraded":"2024-05-09T04:48:28.662343+0000","last_fullsized":"2024-05-09T04:48:28.662343+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:59.686391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556513+0000","last_change":"2024-05-09T04:47:48.412781+0000","last_active":"2024-05-09T04:48:28.556513+0000","last_peered":"2024-05-09T04:48:28.556513+0000","last_clean":"2024-05-09T04:48:28.556513+0000","last_became_active":"2024-05-09T04:47:48.411396+0000","last_became_peered":"2024-05-09T04:47:48.411396+0000","last_unstale":"2024-05-09T04:48:28.556513+0000","last_undegraded":"2024-05-09T04:48:28.556513+0000","last_fullsized":"2024-05-09T04:48:28.556513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:41:22.775432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.568491+0000","last_change":"2024-05-09T04:48:28.568024+0000","last_active":"2024-05-09T04:48:28.568491+0000","last_peered":"2024-05-09T04:48:28.568491+0000","last_clean":"2024-05-09T04:48:28.568491+0000","last_became_active":"2024-05-09T04:48:28.559441+0000","last_became_peered":"2024-05-09T04:48:28.559441+0000","last_unstale":"2024-05-09T04:48:28.568491+0000","last_undegraded":"2024-05-09T04:48:28.568491+0000","last_fullsized":"2024-05-09T04:48:28.568491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:56.257589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.561084+0000","last_change":"2024-05-09T04:47:48.421361+0000","last_active":"2024-05-09T04:48:28.561084+0000","last_peered":"2024-05-09T04:48:28.561084+0000","last_clean":"2024-05-09T04:48:28.561084+0000","last_became_active":"2024-05-09T04:47:48.421018+0000","last_became_peered":"2024-05-09T04:47:48.421018+0000","last_unstale":"2024-05-09T04:48:28.561084+0000","last_undegraded":"2024-05-09T04:48:28.561084+0000","last_fullsized":"2024-05-09T04:48:28.561084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:00.634983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556767+0000","last_change":"2024-05-09T04:47:45.389914+0000","last_active":"2024-05-09T04:48:28.556767+0000","last_peered":"2024-05-09T04:48:28.556767+0000","last_clean":"2024-05-09T04:48:28.556767+0000","last_became_active":"2024-05-09T04:47:45.388361+0000","last_became_peered":"2024-05-09T04:47:45.388361+0000","last_unstale":"2024-05-09T04:48:28.556767+0000","last_undegraded":"2024-05-09T04:48:28.556767+0000","last_fullsized":"2024-05-09T04:48:28.556767+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:08.048767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.564243+0000","last_change":"2024-05-09T04:48:28.564243+0000","last_active":"2024-05-09T04:48:28.564243+0000","last_peered":"2024-05-09T04:48:28.564243+0000","last_clean":"2024-05-09T04:48:28.564243+0000","last_became_active":"2024-05-09T04:48:28.563732+0000","last_became_peered":"2024-05-09T04:48:28.563732+0000","last_unstale":"2024-05-09T04:48:28.564243+0000","last_undegraded":"2024-05-09T04:48:28.564243+0000","last_fullsized":"2024-05-09T04:48:28.564243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:26.982339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":58,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:32.534883+0000","last_change":"2024-05-09T04:47:45.389868+0000","last_active":"2024-05-09T04:48:32.534883+0000","last_peered":"2024-05-09T04:48:32.534883+0000","last_clean":"2024-05-09T04:48:32.534883+0000","last_became_active":"2024-05-09T04:47:45.389508+0000","last_became_peered":"2024-05-09T04:47:45.389508+0000","last_unstale":"2024-05-09T04:48:32.534883+0000","last_undegraded":"2024-05-09T04:48:32.534883+0000","last_fullsized":"2024-05-09T04:48:32.534883+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:45.050187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556021+0000","last_change":"2024-05-09T04:47:48.404457+0000","last_active":"2024-05-09T04:48:28.556021+0000","last_peered":"2024-05-09T04:48:28.556021+0000","last_clean":"2024-05-09T04:48:28.556021+0000","last_became_active":"2024-05-09T04:47:48.403968+0000","last_became_peered":"2024-05-09T04:47:48.403968+0000","last_unstale":"2024-05-09T04:48:28.556021+0000","last_undegraded":"2024-05-09T04:48:28.556021+0000","last_fullsized":"2024-05-09T04:48:28.556021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:23:40.364916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557640+0000","last_change":"2024-05-09T04:47:45.388183+0000","last_active":"2024-05-09T04:48:28.557640+0000","last_peered":"2024-05-09T04:48:28.557640+0000","last_clean":"2024-05-09T04:48:28.557640+0000","last_became_active":"2024-05-09T04:47:45.386603+0000","last_became_peered":"2024-05-09T04:47:45.386603+0000","last_unstale":"2024-05-09T04:48:28.557640+0000","last_undegraded":"2024-05-09T04:48:28.557640+0000","last_fullsized":"2024-05-09T04:48:28.557640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:53.433619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.663120+0000","last_change":"2024-05-09T04:48:28.663120+0000","last_active":"2024-05-09T04:48:28.663120+0000","last_peered":"2024-05-09T04:48:28.663120+0000","last_clean":"2024-05-09T04:48:28.663120+0000","last_became_active":"2024-05-09T04:48:28.662748+0000","last_became_peered":"2024-05-09T04:48:28.662748+0000","last_unstale":"2024-05-09T04:48:28.663120+0000","last_undegraded":"2024-05-09T04:48:28.663120+0000","last_fullsized":"2024-05-09T04:48:28.663120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:57.677876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552103+0000","last_change":"2024-05-09T04:47:45.384192+0000","last_active":"2024-05-09T04:48:28.552103+0000","last_peered":"2024-05-09T04:48:28.552103+0000","last_clean":"2024-05-09T04:48:28.552103+0000","last_became_active":"2024-05-09T04:47:45.383843+0000","last_became_peered":"2024-05-09T04:47:45.383843+0000","last_unstale":"2024-05-09T04:48:28.552103+0000","last_undegraded":"2024-05-09T04:48:28.552103+0000","last_fullsized":"2024-05-09T04:48:28.552103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:28.815207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.661979+0000","last_change":"2024-05-09T04:47:48.409203+0000","last_active":"2024-05-09T04:48:28.661979+0000","last_peered":"2024-05-09T04:48:28.661979+0000","last_clean":"2024-05-09T04:48:28.661979+0000","last_became_active":"2024-05-09T04:47:48.408861+0000","last_became_peered":"2024-05-09T04:47:48.408861+0000","last_unstale":"2024-05-09T04:48:28.661979+0000","last_undegraded":"2024-05-09T04:48:28.661979+0000","last_fullsized":"2024-05-09T04:48:28.661979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:07.160397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557850+0000","last_change":"2024-05-09T04:47:45.392723+0000","last_active":"2024-05-09T04:48:28.557850+0000","last_peered":"2024-05-09T04:48:28.557850+0000","last_clean":"2024-05-09T04:48:28.557850+0000","last_became_active":"2024-05-09T04:47:45.392313+0000","last_became_peered":"2024-05-09T04:47:45.392313+0000","last_unstale":"2024-05-09T04:48:28.557850+0000","last_undegraded":"2024-05-09T04:48:28.557850+0000","last_fullsized":"2024-05-09T04:48:28.557850+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:44.358702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.554865+0000","last_change":"2024-05-09T04:47:48.414000+0000","last_active":"2024-05-09T04:48:28.554865+0000","last_peered":"2024-05-09T04:48:28.554865+0000","last_clean":"2024-05-09T04:48:28.554865+0000","last_became_active":"2024-05-09T04:47:48.413532+0000","last_became_peered":"2024-05-09T04:47:48.413532+0000","last_unstale":"2024-05-09T04:48:28.554865+0000","last_undegraded":"2024-05-09T04:48:28.554865+0000","last_fullsized":"2024-05-09T04:48:28.554865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:22.774814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":54,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:32.534220+0000","last_change":"2024-05-09T04:47:45.388661+0000","last_active":"2024-05-09T04:48:32.534220+0000","last_peered":"2024-05-09T04:48:32.534220+0000","last_clean":"2024-05-09T04:48:32.534220+0000","last_became_active":"2024-05-09T04:47:45.386010+0000","last_became_peered":"2024-05-09T04:47:45.386010+0000","last_unstale":"2024-05-09T04:48:32.534220+0000","last_undegraded":"2024-05-09T04:48:32.534220+0000","last_fullsized":"2024-05-09T04:48:32.534220+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:23.250463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.561394+0000","last_change":"2024-05-09T04:47:48.421706+0000","last_active":"2024-05-09T04:48:28.561394+0000","last_peered":"2024-05-09T04:48:28.561394+0000","last_clean":"2024-05-09T04:48:28.561394+0000","last_became_active":"2024-05-09T04:47:48.420529+0000","last_became_peered":"2024-05-09T04:47:48.420529+0000","last_unstale":"2024-05-09T04:48:28.561394+0000","last_undegraded":"2024-05-09T04:48:28.561394+0000","last_fullsized":"2024-05-09T04:48:28.561394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:10.110366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555946+0000","last_change":"2024-05-09T04:47:45.386625+0000","last_active":"2024-05-09T04:48:28.555946+0000","last_peered":"2024-05-09T04:48:28.555946+0000","last_clean":"2024-05-09T04:48:28.555946+0000","last_became_active":"2024-05-09T04:47:45.385087+0000","last_became_peered":"2024-05-09T04:47:45.385087+0000","last_unstale":"2024-05-09T04:48:28.555946+0000","last_undegraded":"2024-05-09T04:48:28.555946+0000","last_fullsized":"2024-05-09T04:48:28.555946+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:15.546894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.562517+0000","last_change":"2024-05-09T04:48:28.562517+0000","last_active":"2024-05-09T04:48:28.562517+0000","last_peered":"2024-05-09T04:48:28.562517+0000","last_clean":"2024-05-09T04:48:28.562517+0000","last_became_active":"2024-05-09T04:48:28.562047+0000","last_became_peered":"2024-05-09T04:48:28.562047+0000","last_unstale":"2024-05-09T04:48:28.562517+0000","last_undegraded":"2024-05-09T04:48:28.562517+0000","last_fullsized":"2024-05-09T04:48:28.562517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:22.769844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558540+0000","last_change":"2024-05-09T04:47:45.613713+0000","last_active":"2024-05-09T04:48:28.558540+0000","last_peered":"2024-05-09T04:48:28.558540+0000","last_clean":"2024-05-09T04:48:28.558540+0000","last_became_active":"2024-05-09T04:47:45.613313+0000","last_became_peered":"2024-05-09T04:47:45.613313+0000","last_unstale":"2024-05-09T04:48:28.558540+0000","last_undegraded":"2024-05-09T04:48:28.558540+0000","last_fullsized":"2024-05-09T04:48:28.558540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:32:44.825695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.561145+0000","last_change":"2024-05-09T04:47:48.411234+0000","last_active":"2024-05-09T04:48:28.561145+0000","last_peered":"2024-05-09T04:48:28.561145+0000","last_clean":"2024-05-09T04:48:28.561145+0000","last_became_active":"2024-05-09T04:47:48.410881+0000","last_became_peered":"2024-05-09T04:47:48.410881+0000","last_unstale":"2024-05-09T04:48:28.561145+0000","last_undegraded":"2024-05-09T04:48:28.561145+0000","last_fullsized":"2024-05-09T04:48:28.561145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.747871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'28","reported_seq":131,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:33.573560+0000","last_change":"2024-05-09T04:48:28.570426+0000","last_active":"2024-05-09T04:48:33.573560+0000","last_peered":"2024-05-09T04:48:33.573560+0000","last_clean":"2024-05-09T04:48:33.573560+0000","last_became_active":"2024-05-09T04:48:28.561786+0000","last_became_peered":"2024-05-09T04:48:28.561786+0000","last_unstale":"2024-05-09T04:48:33.573560+0000","last_undegraded":"2024-05-09T04:48:33.573560+0000","last_fullsized":"2024-05-09T04:48:33.573560+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:32.367142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56090,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":55,"num_read_kb":73,"num_write":29,"num_write_kb":65,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":50695,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556057+0000","last_change":"2024-05-09T04:47:48.406510+0000","last_active":"2024-05-09T04:48:28.556057+0000","last_peered":"2024-05-09T04:48:28.556057+0000","last_clean":"2024-05-09T04:48:28.556057+0000","last_became_active":"2024-05-09T04:47:48.406166+0000","last_became_peered":"2024-05-09T04:47:48.406166+0000","last_unstale":"2024-05-09T04:48:28.556057+0000","last_undegraded":"2024-05-09T04:48:28.556057+0000","last_fullsized":"2024-05-09T04:48:28.556057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:18.579620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557905+0000","last_change":"2024-05-09T04:47:45.387164+0000","last_active":"2024-05-09T04:48:28.557905+0000","last_peered":"2024-05-09T04:48:28.557905+0000","last_clean":"2024-05-09T04:48:28.557905+0000","last_became_active":"2024-05-09T04:47:45.386755+0000","last_became_peered":"2024-05-09T04:47:45.386755+0000","last_unstale":"2024-05-09T04:48:28.557905+0000","last_undegraded":"2024-05-09T04:48:28.557905+0000","last_fullsized":"2024-05-09T04:48:28.557905+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:07.838782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552103+0000","last_change":"2024-05-09T04:47:48.405436+0000","last_active":"2024-05-09T04:48:28.552103+0000","last_peered":"2024-05-09T04:48:28.552103+0000","last_clean":"2024-05-09T04:48:28.552103+0000","last_became_active":"2024-05-09T04:47:48.405032+0000","last_became_peered":"2024-05-09T04:47:48.405032+0000","last_unstale":"2024-05-09T04:48:28.552103+0000","last_undegraded":"2024-05-09T04:48:28.552103+0000","last_fullsized":"2024-05-09T04:48:28.552103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:27.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558055+0000","last_change":"2024-05-09T04:47:45.391970+0000","last_active":"2024-05-09T04:48:28.558055+0000","last_peered":"2024-05-09T04:48:28.558055+0000","last_clean":"2024-05-09T04:48:28.558055+0000","last_became_active":"2024-05-09T04:47:45.391597+0000","last_became_peered":"2024-05-09T04:47:45.391597+0000","last_unstale":"2024-05-09T04:48:28.558055+0000","last_undegraded":"2024-05-09T04:48:28.558055+0000","last_fullsized":"2024-05-09T04:48:28.558055+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:27.076631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556549+0000","last_change":"2024-05-09T04:47:48.419328+0000","last_active":"2024-05-09T04:48:28.556549+0000","last_peered":"2024-05-09T04:48:28.556549+0000","last_clean":"2024-05-09T04:48:28.556549+0000","last_became_active":"2024-05-09T04:47:48.418972+0000","last_became_peered":"2024-05-09T04:47:48.418972+0000","last_unstale":"2024-05-09T04:48:28.556549+0000","last_undegraded":"2024-05-09T04:48:28.556549+0000","last_fullsized":"2024-05-09T04:48:28.556549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:15.164747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:30.531733+0000","last_change":"2024-05-09T04:47:45.386173+0000","last_active":"2024-05-09T04:48:30.531733+0000","last_peered":"2024-05-09T04:48:30.531733+0000","last_clean":"2024-05-09T04:48:30.531733+0000","last_became_active":"2024-05-09T04:47:45.384296+0000","last_became_peered":"2024-05-09T04:47:45.384296+0000","last_unstale":"2024-05-09T04:48:30.531733+0000","last_undegraded":"2024-05-09T04:48:30.531733+0000","last_fullsized":"2024-05-09T04:48:30.531733+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:03.246187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":28,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555463+0000","last_change":"2024-05-09T04:47:48.415645+0000","last_active":"2024-05-09T04:48:28.555463+0000","last_peered":"2024-05-09T04:48:28.555463+0000","last_clean":"2024-05-09T04:48:28.555463+0000","last_became_active":"2024-05-09T04:47:48.415303+0000","last_became_peered":"2024-05-09T04:47:48.415303+0000","last_unstale":"2024-05-09T04:48:28.555463+0000","last_undegraded":"2024-05-09T04:48:28.555463+0000","last_fullsized":"2024-05-09T04:48:28.555463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:39.703948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555714+0000","last_change":"2024-05-09T04:47:45.613320+0000","last_active":"2024-05-09T04:48:28.555714+0000","last_peered":"2024-05-09T04:48:28.555714+0000","last_clean":"2024-05-09T04:48:28.555714+0000","last_became_active":"2024-05-09T04:47:45.612871+0000","last_became_peered":"2024-05-09T04:47:45.612871+0000","last_unstale":"2024-05-09T04:48:28.555714+0000","last_undegraded":"2024-05-09T04:48:28.555714+0000","last_fullsized":"2024-05-09T04:48:28.555714+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:31.516652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.563376+0000","last_change":"2024-05-09T04:48:28.563376+0000","last_active":"2024-05-09T04:48:28.563376+0000","last_peered":"2024-05-09T04:48:28.563376+0000","last_clean":"2024-05-09T04:48:28.563376+0000","last_became_active":"2024-05-09T04:48:28.562973+0000","last_became_peered":"2024-05-09T04:48:28.562973+0000","last_unstale":"2024-05-09T04:48:28.563376+0000","last_undegraded":"2024-05-09T04:48:28.563376+0000","last_fullsized":"2024-05-09T04:48:28.563376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:20.068517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556678+0000","last_change":"2024-05-09T04:47:45.389785+0000","last_active":"2024-05-09T04:48:28.556678+0000","last_peered":"2024-05-09T04:48:28.556678+0000","last_clean":"2024-05-09T04:48:28.556678+0000","last_became_active":"2024-05-09T04:47:45.389159+0000","last_became_peered":"2024-05-09T04:47:45.389159+0000","last_unstale":"2024-05-09T04:48:28.556678+0000","last_undegraded":"2024-05-09T04:48:28.556678+0000","last_fullsized":"2024-05-09T04:48:28.556678+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:28:51.804123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.563170+0000","last_change":"2024-05-09T04:48:28.563170+0000","last_active":"2024-05-09T04:48:28.563170+0000","last_peered":"2024-05-09T04:48:28.563170+0000","last_clean":"2024-05-09T04:48:28.563170+0000","last_became_active":"2024-05-09T04:48:28.562189+0000","last_became_peered":"2024-05-09T04:48:28.562189+0000","last_unstale":"2024-05-09T04:48:28.563170+0000","last_undegraded":"2024-05-09T04:48:28.563170+0000","last_fullsized":"2024-05-09T04:48:28.563170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:00.643292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555516+0000","last_change":"2024-05-09T04:47:45.613739+0000","last_active":"2024-05-09T04:48:28.555516+0000","last_peered":"2024-05-09T04:48:28.555516+0000","last_clean":"2024-05-09T04:48:28.555516+0000","last_became_active":"2024-05-09T04:47:45.613365+0000","last_became_peered":"2024-05-09T04:47:45.613365+0000","last_unstale":"2024-05-09T04:48:28.555516+0000","last_undegraded":"2024-05-09T04:48:28.555516+0000","last_fullsized":"2024-05-09T04:48:28.555516+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:57.138068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557722+0000","last_change":"2024-05-09T04:47:48.413343+0000","last_active":"2024-05-09T04:48:28.557722+0000","last_peered":"2024-05-09T04:48:28.557722+0000","last_clean":"2024-05-09T04:48:28.557722+0000","last_became_active":"2024-05-09T04:47:48.412941+0000","last_became_peered":"2024-05-09T04:47:48.412941+0000","last_unstale":"2024-05-09T04:48:28.557722+0000","last_undegraded":"2024-05-09T04:48:28.557722+0000","last_fullsized":"2024-05-09T04:48:28.557722+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:28.764553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555616+0000","last_change":"2024-05-09T04:47:45.386487+0000","last_active":"2024-05-09T04:48:28.555616+0000","last_peered":"2024-05-09T04:48:28.555616+0000","last_clean":"2024-05-09T04:48:28.555616+0000","last_became_active":"2024-05-09T04:47:45.386078+0000","last_became_peered":"2024-05-09T04:47:45.386078+0000","last_unstale":"2024-05-09T04:48:28.555616+0000","last_undegraded":"2024-05-09T04:48:28.555616+0000","last_fullsized":"2024-05-09T04:48:28.555616+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:19.998977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557819+0000","last_change":"2024-05-09T04:47:48.418955+0000","last_active":"2024-05-09T04:48:28.557819+0000","last_peered":"2024-05-09T04:48:28.557819+0000","last_clean":"2024-05-09T04:48:28.557819+0000","last_became_active":"2024-05-09T04:47:48.418528+0000","last_became_peered":"2024-05-09T04:47:48.418528+0000","last_unstale":"2024-05-09T04:48:28.557819+0000","last_undegraded":"2024-05-09T04:48:28.557819+0000","last_fullsized":"2024-05-09T04:48:28.557819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:51:02.386342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558153+0000","last_change":"2024-05-09T04:47:48.413304+0000","last_active":"2024-05-09T04:48:28.558153+0000","last_peered":"2024-05-09T04:48:28.558153+0000","last_clean":"2024-05-09T04:48:28.558153+0000","last_became_active":"2024-05-09T04:47:48.412582+0000","last_became_peered":"2024-05-09T04:47:48.412582+0000","last_unstale":"2024-05-09T04:48:28.558153+0000","last_undegraded":"2024-05-09T04:48:28.558153+0000","last_fullsized":"2024-05-09T04:48:28.558153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:40.027082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:33.535610+0000","last_change":"2024-05-09T04:47:45.612742+0000","last_active":"2024-05-09T04:48:33.535610+0000","last_peered":"2024-05-09T04:48:33.535610+0000","last_clean":"2024-05-09T04:48:33.535610+0000","last_became_active":"2024-05-09T04:47:45.612325+0000","last_became_peered":"2024-05-09T04:47:45.612325+0000","last_unstale":"2024-05-09T04:48:33.535610+0000","last_undegraded":"2024-05-09T04:48:33.535610+0000","last_fullsized":"2024-05-09T04:48:33.535610+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:00.573996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.661996+0000","last_change":"2024-05-09T04:47:48.412894+0000","last_active":"2024-05-09T04:48:28.661996+0000","last_peered":"2024-05-09T04:48:28.661996+0000","last_clean":"2024-05-09T04:48:28.661996+0000","last_became_active":"2024-05-09T04:47:48.412265+0000","last_became_peered":"2024-05-09T04:47:48.412265+0000","last_unstale":"2024-05-09T04:48:28.661996+0000","last_undegraded":"2024-05-09T04:48:28.661996+0000","last_fullsized":"2024-05-09T04:48:28.661996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:23.517566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.553331+0000","last_change":"2024-05-09T04:47:45.387715+0000","last_active":"2024-05-09T04:48:28.553331+0000","last_peered":"2024-05-09T04:48:28.553331+0000","last_clean":"2024-05-09T04:48:28.553331+0000","last_became_active":"2024-05-09T04:47:45.387229+0000","last_became_peered":"2024-05-09T04:47:45.387229+0000","last_unstale":"2024-05-09T04:48:28.553331+0000","last_undegraded":"2024-05-09T04:48:28.553331+0000","last_fullsized":"2024-05-09T04:48:28.553331+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:22.540214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":99,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555632+0000","last_change":"2024-05-09T04:46:21.632579+0000","last_active":"2024-05-09T04:48:28.555632+0000","last_peered":"2024-05-09T04:48:28.555632+0000","last_clean":"2024-05-09T04:48:28.555632+0000","last_became_active":"2024-05-09T04:46:21.627483+0000","last_became_peered":"2024-05-09T04:46:21.627483+0000","last_unstale":"2024-05-09T04:48:28.555632+0000","last_undegraded":"2024-05-09T04:48:28.555632+0000","last_fullsized":"2024-05-09T04:48:28.555632+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:42:54.179739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:42:54.179739+0000","last_clean_scrub_stamp":"2024-05-09T04:42:54.179739+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:44.331526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.551975+0000","last_change":"2024-05-09T04:47:48.418570+0000","last_active":"2024-05-09T04:48:28.551975+0000","last_peered":"2024-05-09T04:48:28.551975+0000","last_clean":"2024-05-09T04:48:28.551975+0000","last_became_active":"2024-05-09T04:47:48.418270+0000","last_became_peered":"2024-05-09T04:47:48.418270+0000","last_unstale":"2024-05-09T04:48:28.551975+0000","last_undegraded":"2024-05-09T04:48:28.551975+0000","last_fullsized":"2024-05-09T04:48:28.551975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:28.406846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555742+0000","last_change":"2024-05-09T04:47:45.387161+0000","last_active":"2024-05-09T04:48:28.555742+0000","last_peered":"2024-05-09T04:48:28.555742+0000","last_clean":"2024-05-09T04:48:28.555742+0000","last_became_active":"2024-05-09T04:47:45.386753+0000","last_became_peered":"2024-05-09T04:47:45.386753+0000","last_unstale":"2024-05-09T04:48:28.555742+0000","last_undegraded":"2024-05-09T04:48:28.555742+0000","last_fullsized":"2024-05-09T04:48:28.555742+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:15.402418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556181+0000","last_change":"2024-05-09T04:47:45.613551+0000","last_active":"2024-05-09T04:48:28.556181+0000","last_peered":"2024-05-09T04:48:28.556181+0000","last_clean":"2024-05-09T04:48:28.556181+0000","last_became_active":"2024-05-09T04:47:45.613086+0000","last_became_peered":"2024-05-09T04:47:45.613086+0000","last_unstale":"2024-05-09T04:48:28.556181+0000","last_undegraded":"2024-05-09T04:48:28.556181+0000","last_fullsized":"2024-05-09T04:48:28.556181+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:04.350960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552185+0000","last_change":"2024-05-09T04:47:48.403701+0000","last_active":"2024-05-09T04:48:28.552185+0000","last_peered":"2024-05-09T04:48:28.552185+0000","last_clean":"2024-05-09T04:48:28.552185+0000","last_became_active":"2024-05-09T04:47:48.403337+0000","last_became_peered":"2024-05-09T04:47:48.403337+0000","last_unstale":"2024-05-09T04:48:28.552185+0000","last_undegraded":"2024-05-09T04:48:28.552185+0000","last_fullsized":"2024-05-09T04:48:28.552185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:53.032348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'10","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555797+0000","last_change":"2024-05-09T04:47:45.385935+0000","last_active":"2024-05-09T04:48:28.555797+0000","last_peered":"2024-05-09T04:48:28.555797+0000","last_clean":"2024-05-09T04:48:28.555797+0000","last_became_active":"2024-05-09T04:47:45.385533+0000","last_became_peered":"2024-05-09T04:47:45.385533+0000","last_unstale":"2024-05-09T04:48:28.555797+0000","last_undegraded":"2024-05-09T04:48:28.555797+0000","last_fullsized":"2024-05-09T04:48:28.555797+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:34:38.014201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.662346+0000","last_change":"2024-05-09T04:47:48.409590+0000","last_active":"2024-05-09T04:48:28.662346+0000","last_peered":"2024-05-09T04:48:28.662346+0000","last_clean":"2024-05-09T04:48:28.662346+0000","last_became_active":"2024-05-09T04:47:48.409191+0000","last_became_peered":"2024-05-09T04:47:48.409191+0000","last_unstale":"2024-05-09T04:48:28.662346+0000","last_undegraded":"2024-05-09T04:48:28.662346+0000","last_fullsized":"2024-05-09T04:48:28.662346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:50.440609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.553819+0000","last_change":"2024-05-09T04:47:45.389147+0000","last_active":"2024-05-09T04:48:28.553819+0000","last_peered":"2024-05-09T04:48:28.553819+0000","last_clean":"2024-05-09T04:48:28.553819+0000","last_became_active":"2024-05-09T04:47:45.388797+0000","last_became_peered":"2024-05-09T04:47:45.388797+0000","last_unstale":"2024-05-09T04:48:28.553819+0000","last_undegraded":"2024-05-09T04:48:28.553819+0000","last_fullsized":"2024-05-09T04:48:28.553819+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:19.265278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556603+0000","last_change":"2024-05-09T04:47:48.419100+0000","last_active":"2024-05-09T04:48:28.556603+0000","last_peered":"2024-05-09T04:48:28.556603+0000","last_clean":"2024-05-09T04:48:28.556603+0000","last_became_active":"2024-05-09T04:47:48.418499+0000","last_became_peered":"2024-05-09T04:47:48.418499+0000","last_unstale":"2024-05-09T04:48:28.556603+0000","last_undegraded":"2024-05-09T04:48:28.556603+0000","last_fullsized":"2024-05-09T04:48:28.556603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:23.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.560862+0000","last_change":"2024-05-09T04:47:45.613269+0000","last_active":"2024-05-09T04:48:28.560862+0000","last_peered":"2024-05-09T04:48:28.560862+0000","last_clean":"2024-05-09T04:48:28.560862+0000","last_became_active":"2024-05-09T04:47:45.612895+0000","last_became_peered":"2024-05-09T04:47:45.612895+0000","last_unstale":"2024-05-09T04:48:28.560862+0000","last_undegraded":"2024-05-09T04:48:28.560862+0000","last_fullsized":"2024-05-09T04:48:28.560862+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:52.966974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556360+0000","last_change":"2024-05-09T04:47:48.416658+0000","last_active":"2024-05-09T04:48:28.556360+0000","last_peered":"2024-05-09T04:48:28.556360+0000","last_clean":"2024-05-09T04:48:28.556360+0000","last_became_active":"2024-05-09T04:47:48.416238+0000","last_became_peered":"2024-05-09T04:47:48.416238+0000","last_unstale":"2024-05-09T04:48:28.556360+0000","last_undegraded":"2024-05-09T04:48:28.556360+0000","last_fullsized":"2024-05-09T04:48:28.556360+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:35:50.126723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.661867+0000","last_change":"2024-05-09T04:47:45.384018+0000","last_active":"2024-05-09T04:48:28.661867+0000","last_peered":"2024-05-09T04:48:28.661867+0000","last_clean":"2024-05-09T04:48:28.661867+0000","last_became_active":"2024-05-09T04:47:45.383647+0000","last_became_peered":"2024-05-09T04:47:45.383647+0000","last_unstale":"2024-05-09T04:48:28.661867+0000","last_undegraded":"2024-05-09T04:48:28.661867+0000","last_fullsized":"2024-05-09T04:48:28.661867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:06.431233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552763+0000","last_change":"2024-05-09T04:47:48.409089+0000","last_active":"2024-05-09T04:48:28.552763+0000","last_peered":"2024-05-09T04:48:28.552763+0000","last_clean":"2024-05-09T04:48:28.552763+0000","last_became_active":"2024-05-09T04:47:48.408548+0000","last_became_peered":"2024-05-09T04:47:48.408548+0000","last_unstale":"2024-05-09T04:48:28.552763+0000","last_undegraded":"2024-05-09T04:48:28.552763+0000","last_fullsized":"2024-05-09T04:48:28.552763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:13.633008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557606+0000","last_change":"2024-05-09T04:47:45.387580+0000","last_active":"2024-05-09T04:48:28.557606+0000","last_peered":"2024-05-09T04:48:28.557606+0000","last_clean":"2024-05-09T04:48:28.557606+0000","last_became_active":"2024-05-09T04:47:45.386929+0000","last_became_peered":"2024-05-09T04:47:45.386929+0000","last_unstale":"2024-05-09T04:48:28.557606+0000","last_undegraded":"2024-05-09T04:48:28.557606+0000","last_fullsized":"2024-05-09T04:48:28.557606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:19.661763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.662079+0000","last_change":"2024-05-09T04:47:48.416311+0000","last_active":"2024-05-09T04:48:28.662079+0000","last_peered":"2024-05-09T04:48:28.662079+0000","last_clean":"2024-05-09T04:48:28.662079+0000","last_became_active":"2024-05-09T04:47:48.415969+0000","last_became_peered":"2024-05-09T04:47:48.415969+0000","last_unstale":"2024-05-09T04:48:28.662079+0000","last_undegraded":"2024-05-09T04:48:28.662079+0000","last_fullsized":"2024-05-09T04:48:28.662079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:55.751210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.573916+0000","last_change":"2024-05-09T04:48:28.573916+0000","last_active":"2024-05-09T04:48:28.573916+0000","last_peered":"2024-05-09T04:48:28.573916+0000","last_clean":"2024-05-09T04:48:28.573916+0000","last_became_active":"2024-05-09T04:48:28.562872+0000","last_became_peered":"2024-05-09T04:48:28.562872+0000","last_unstale":"2024-05-09T04:48:28.573916+0000","last_undegraded":"2024-05-09T04:48:28.573916+0000","last_fullsized":"2024-05-09T04:48:28.573916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:02.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555793+0000","last_change":"2024-05-09T04:47:48.408364+0000","last_active":"2024-05-09T04:48:28.555793+0000","last_peered":"2024-05-09T04:48:28.555793+0000","last_clean":"2024-05-09T04:48:28.555793+0000","last_became_active":"2024-05-09T04:47:48.407982+0000","last_became_peered":"2024-05-09T04:47:48.407982+0000","last_unstale":"2024-05-09T04:48:28.555793+0000","last_undegraded":"2024-05-09T04:48:28.555793+0000","last_fullsized":"2024-05-09T04:48:28.555793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:14.103171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558323+0000","last_change":"2024-05-09T04:47:45.614637+0000","last_active":"2024-05-09T04:48:28.558323+0000","last_peered":"2024-05-09T04:48:28.558323+0000","last_clean":"2024-05-09T04:48:28.558323+0000","last_became_active":"2024-05-09T04:47:45.613290+0000","last_became_peered":"2024-05-09T04:47:45.613290+0000","last_unstale":"2024-05-09T04:48:28.558323+0000","last_undegraded":"2024-05-09T04:48:28.558323+0000","last_fullsized":"2024-05-09T04:48:28.558323+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:48.547382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556319+0000","last_change":"2024-05-09T04:47:48.405537+0000","last_active":"2024-05-09T04:48:28.556319+0000","last_peered":"2024-05-09T04:48:28.556319+0000","last_clean":"2024-05-09T04:48:28.556319+0000","last_became_active":"2024-05-09T04:47:48.405148+0000","last_became_peered":"2024-05-09T04:47:48.405148+0000","last_unstale":"2024-05-09T04:48:28.556319+0000","last_undegraded":"2024-05-09T04:48:28.556319+0000","last_fullsized":"2024-05-09T04:48:28.556319+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:46.981401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558388+0000","last_change":"2024-05-09T04:47:45.389151+0000","last_active":"2024-05-09T04:48:28.558388+0000","last_peered":"2024-05-09T04:48:28.558388+0000","last_clean":"2024-05-09T04:48:28.558388+0000","last_became_active":"2024-05-09T04:47:45.388735+0000","last_became_peered":"2024-05-09T04:47:45.388735+0000","last_unstale":"2024-05-09T04:48:28.558388+0000","last_undegraded":"2024-05-09T04:48:28.558388+0000","last_fullsized":"2024-05-09T04:48:28.558388+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:26.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555966+0000","last_change":"2024-05-09T04:47:48.413881+0000","last_active":"2024-05-09T04:48:28.555966+0000","last_peered":"2024-05-09T04:48:28.555966+0000","last_clean":"2024-05-09T04:48:28.555966+0000","last_became_active":"2024-05-09T04:47:48.413559+0000","last_became_peered":"2024-05-09T04:47:48.413559+0000","last_unstale":"2024-05-09T04:48:28.555966+0000","last_undegraded":"2024-05-09T04:48:28.555966+0000","last_fullsized":"2024-05-09T04:48:28.555966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:40:40.305319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552437+0000","last_change":"2024-05-09T04:47:45.383406+0000","last_active":"2024-05-09T04:48:28.552437+0000","last_peered":"2024-05-09T04:48:28.552437+0000","last_clean":"2024-05-09T04:48:28.552437+0000","last_became_active":"2024-05-09T04:47:45.383025+0000","last_became_peered":"2024-05-09T04:47:45.383025+0000","last_unstale":"2024-05-09T04:48:28.552437+0000","last_undegraded":"2024-05-09T04:48:28.552437+0000","last_fullsized":"2024-05-09T04:48:28.552437+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:46.007454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556288+0000","last_change":"2024-05-09T04:47:48.412481+0000","last_active":"2024-05-09T04:48:28.556288+0000","last_peered":"2024-05-09T04:48:28.556288+0000","last_clean":"2024-05-09T04:48:28.556288+0000","last_became_active":"2024-05-09T04:47:48.412034+0000","last_became_peered":"2024-05-09T04:47:48.412034+0000","last_unstale":"2024-05-09T04:48:28.556288+0000","last_undegraded":"2024-05-09T04:48:28.556288+0000","last_fullsized":"2024-05-09T04:48:28.556288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:14.030981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552849+0000","last_change":"2024-05-09T04:47:48.406958+0000","last_active":"2024-05-09T04:48:28.552849+0000","last_peered":"2024-05-09T04:48:28.552849+0000","last_clean":"2024-05-09T04:48:28.552849+0000","last_became_active":"2024-05-09T04:47:48.406575+0000","last_became_peered":"2024-05-09T04:47:48.406575+0000","last_unstale":"2024-05-09T04:48:28.552849+0000","last_undegraded":"2024-05-09T04:48:28.552849+0000","last_fullsized":"2024-05-09T04:48:28.552849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:23.389676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.562376+0000","last_change":"2024-05-09T04:48:28.562376+0000","last_active":"2024-05-09T04:48:28.562376+0000","last_peered":"2024-05-09T04:48:28.562376+0000","last_clean":"2024-05-09T04:48:28.562376+0000","last_became_active":"2024-05-09T04:48:28.561866+0000","last_became_peered":"2024-05-09T04:48:28.561866+0000","last_unstale":"2024-05-09T04:48:28.562376+0000","last_undegraded":"2024-05-09T04:48:28.562376+0000","last_fullsized":"2024-05-09T04:48:28.562376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:51.974146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556736+0000","last_change":"2024-05-09T04:47:48.413908+0000","last_active":"2024-05-09T04:48:28.556736+0000","last_peered":"2024-05-09T04:48:28.556736+0000","last_clean":"2024-05-09T04:48:28.556736+0000","last_became_active":"2024-05-09T04:47:48.413575+0000","last_became_peered":"2024-05-09T04:47:48.413575+0000","last_unstale":"2024-05-09T04:48:28.556736+0000","last_undegraded":"2024-05-09T04:48:28.556736+0000","last_fullsized":"2024-05-09T04:48:28.556736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:40.742510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556432+0000","last_change":"2024-05-09T04:47:48.406607+0000","last_active":"2024-05-09T04:48:28.556432+0000","last_peered":"2024-05-09T04:48:28.556432+0000","last_clean":"2024-05-09T04:48:28.556432+0000","last_became_active":"2024-05-09T04:47:48.406290+0000","last_became_peered":"2024-05-09T04:47:48.406290+0000","last_unstale":"2024-05-09T04:48:28.556432+0000","last_undegraded":"2024-05-09T04:48:28.556432+0000","last_fullsized":"2024-05-09T04:48:28.556432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:22.107953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552307+0000","last_change":"2024-05-09T04:47:48.415153+0000","last_active":"2024-05-09T04:48:28.552307+0000","last_peered":"2024-05-09T04:48:28.552307+0000","last_clean":"2024-05-09T04:48:28.552307+0000","last_became_active":"2024-05-09T04:47:48.414793+0000","last_became_peered":"2024-05-09T04:47:48.414793+0000","last_unstale":"2024-05-09T04:48:28.552307+0000","last_undegraded":"2024-05-09T04:48:28.552307+0000","last_fullsized":"2024-05-09T04:48:28.552307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:57.564091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.553235+0000","last_change":"2024-05-09T04:47:48.408646+0000","last_active":"2024-05-09T04:48:28.553235+0000","last_peered":"2024-05-09T04:48:28.553235+0000","last_clean":"2024-05-09T04:48:28.553235+0000","last_became_active":"2024-05-09T04:47:48.408250+0000","last_became_peered":"2024-05-09T04:47:48.408250+0000","last_unstale":"2024-05-09T04:48:28.553235+0000","last_undegraded":"2024-05-09T04:48:28.553235+0000","last_fullsized":"2024-05-09T04:48:28.553235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:42:50.622937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558944+0000","last_change":"2024-05-09T04:47:48.408686+0000","last_active":"2024-05-09T04:48:28.558944+0000","last_peered":"2024-05-09T04:48:28.558944+0000","last_clean":"2024-05-09T04:48:28.558944+0000","last_became_active":"2024-05-09T04:47:48.408318+0000","last_became_peered":"2024-05-09T04:47:48.408318+0000","last_unstale":"2024-05-09T04:48:28.558944+0000","last_undegraded":"2024-05-09T04:48:28.558944+0000","last_fullsized":"2024-05-09T04:48:28.558944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:51.240485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.662340+0000","last_change":"2024-05-09T04:47:48.421562+0000","last_active":"2024-05-09T04:48:28.662340+0000","last_peered":"2024-05-09T04:48:28.662340+0000","last_clean":"2024-05-09T04:48:28.662340+0000","last_became_active":"2024-05-09T04:47:48.421068+0000","last_became_peered":"2024-05-09T04:47:48.421068+0000","last_unstale":"2024-05-09T04:48:28.662340+0000","last_undegraded":"2024-05-09T04:48:28.662340+0000","last_fullsized":"2024-05-09T04:48:28.662340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:18:41.455187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557871+0000","last_change":"2024-05-09T04:47:48.413942+0000","last_active":"2024-05-09T04:48:28.557871+0000","last_peered":"2024-05-09T04:48:28.557871+0000","last_clean":"2024-05-09T04:48:28.557871+0000","last_became_active":"2024-05-09T04:47:48.413561+0000","last_became_peered":"2024-05-09T04:47:48.413561+0000","last_unstale":"2024-05-09T04:48:28.557871+0000","last_undegraded":"2024-05-09T04:48:28.557871+0000","last_fullsized":"2024-05-09T04:48:28.557871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:33:48.148834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.661780+0000","last_change":"2024-05-09T04:47:48.408752+0000","last_active":"2024-05-09T04:48:28.661780+0000","last_peered":"2024-05-09T04:48:28.661780+0000","last_clean":"2024-05-09T04:48:28.661780+0000","last_became_active":"2024-05-09T04:47:48.408393+0000","last_became_peered":"2024-05-09T04:47:48.408393+0000","last_unstale":"2024-05-09T04:48:28.661780+0000","last_undegraded":"2024-05-09T04:48:28.661780+0000","last_fullsized":"2024-05-09T04:48:28.661780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:06.778002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.561280+0000","last_change":"2024-05-09T04:47:48.409303+0000","last_active":"2024-05-09T04:48:28.561280+0000","last_peered":"2024-05-09T04:48:28.561280+0000","last_clean":"2024-05-09T04:48:28.561280+0000","last_became_active":"2024-05-09T04:47:48.408971+0000","last_became_peered":"2024-05-09T04:47:48.408971+0000","last_unstale":"2024-05-09T04:48:28.561280+0000","last_undegraded":"2024-05-09T04:48:28.561280+0000","last_fullsized":"2024-05-09T04:48:28.561280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:45.830647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552334+0000","last_change":"2024-05-09T04:47:48.404961+0000","last_active":"2024-05-09T04:48:28.552334+0000","last_peered":"2024-05-09T04:48:28.552334+0000","last_clean":"2024-05-09T04:48:28.552334+0000","last_became_active":"2024-05-09T04:47:48.404577+0000","last_became_peered":"2024-05-09T04:47:48.404577+0000","last_unstale":"2024-05-09T04:48:28.552334+0000","last_undegraded":"2024-05-09T04:48:28.552334+0000","last_fullsized":"2024-05-09T04:48:28.552334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:40.528066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552807+0000","last_change":"2024-05-09T04:47:48.424364+0000","last_active":"2024-05-09T04:48:28.552807+0000","last_peered":"2024-05-09T04:48:28.552807+0000","last_clean":"2024-05-09T04:48:28.552807+0000","last_became_active":"2024-05-09T04:47:48.423991+0000","last_became_peered":"2024-05-09T04:47:48.423991+0000","last_unstale":"2024-05-09T04:48:28.552807+0000","last_undegraded":"2024-05-09T04:48:28.552807+0000","last_fullsized":"2024-05-09T04:48:28.552807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:14.040560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249829,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60091,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":177,"num_read_kb":145,"num_write":161,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":50695,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":180273,"data_compressed":8483,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645336,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":880640,"data_stored":431993,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841564,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":1079955,"data_compressed":41932,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005090,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":432229,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201320,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27900,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717252,"statfs":{"total":95995035648,"available":95966466048,"internally_reserved":0,"allocated":897024,"data_stored":432149,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397550,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":1024079,"data_compressed":41932,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561075,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":431618,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574924,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32096,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713056,"statfs":{"total":95995035648,"available":95962169344,"internally_reserved":0,"allocated":942080,"data_stored":487501,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":432889,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432761,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117058,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36224,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708928,"statfs":{"total":95995035648,"available":95957942272,"internally_reserved":0,"allocated":901120,"data_stored":431929,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":917504,"data_stored":488536,"data_compressed":14262,"data_compressed_allocated":20480,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":1023372,"data_compressed":41932,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34532,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34532,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":56090,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":57125,"data_compressed":8483,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:48:37.331 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T04:48:37.590 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:37.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:37 smithi072 ceph-mon[34088]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:48:37.884 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:37 smithi046 ceph-mon[30136]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:48:38.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:37 smithi110 ceph-mon[34904]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:48:38.750 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:38 smithi046 ceph-mon[30136]: pgmap v317: 97 pgs: 97 active+clean; 635 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.5 KiB/s wr, 7 op/s; 5.1 KiB/s, 1 keys/s, 0 objects/s recovering 2024-05-09T04:48:38.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:38 smithi072 ceph-mon[34088]: pgmap v317: 97 pgs: 97 active+clean; 635 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.5 KiB/s wr, 7 op/s; 5.1 KiB/s, 1 keys/s, 0 objects/s recovering 2024-05-09T04:48:38.958 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:38 smithi110 ceph-mon[34904]: pgmap v317: 97 pgs: 97 active+clean; 635 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 3.5 KiB/s wr, 7 op/s; 5.1 KiB/s, 1 keys/s, 0 objects/s recovering 2024-05-09T04:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.432+0000 7f6172f22700 1 -- 172.21.15.46:0/3836521374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f616c1054c0 msgr2=0x7f616c1058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.432+0000 7f6172f22700 1 --2- 172.21.15.46:0/3836521374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f616c1054c0 0x7f616c1058d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f615c00b0a0 tx=0x7f615c00b3b0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.432+0000 7f6172f22700 1 -- 172.21.15.46:0/3836521374 shutdown_connections 2024-05-09T04:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.432+0000 7f6172f22700 1 --2- 172.21.15.46:0/3836521374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f616c10e8f0 0x7f616c110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.432+0000 7f6172f22700 1 --2- 172.21.15.46:0/3836521374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f616c1054c0 0x7f616c1058d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.432+0000 7f6172f22700 1 --2- 172.21.15.46:0/3836521374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f616c105e10 0x7f616c10e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.432+0000 7f6172f22700 1 -- 172.21.15.46:0/3836521374 >> 172.21.15.46:0/3836521374 conn(0x7f616c0fedf0 msgr2=0x7f616c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.432+0000 7f6172f22700 1 -- 172.21.15.46:0/3836521374 shutdown_connections 2024-05-09T04:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.432+0000 7f6172f22700 1 -- 172.21.15.46:0/3836521374 wait complete. 2024-05-09T04:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.433+0000 7f6172f22700 1 Processor -- start 2024-05-09T04:48:39.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.433+0000 7f6172f22700 1 -- start start 2024-05-09T04:48:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.433+0000 7f6172f22700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f616c1054c0 0x7f616c19d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.433+0000 7f6172f22700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f616c105e10 0x7f616c19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.433+0000 7f6172f22700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f616c10e8f0 0x7f616c1a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:39.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.433+0000 7f6172f22700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f616c114440 con 0x7f616c10e8f0 2024-05-09T04:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.433+0000 7f6172f22700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f616c1142c0 con 0x7f616c105e10 2024-05-09T04:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.433+0000 7f6172f22700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f616c1145c0 con 0x7f616c1054c0 2024-05-09T04:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.434+0000 7f61714bf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f616c10e8f0 0x7f616c1a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.434+0000 7f61714bf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f616c10e8f0 0x7f616c1a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47464/0 (socket says 172.21.15.46:47464) 2024-05-09T04:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.434+0000 7f6170cbe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f616c1054c0 0x7f616c19d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.434+0000 7f61714bf700 1 -- 172.21.15.46:0/1856626315 learned_addr learned my addr 172.21.15.46:0/1856626315 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.434+0000 7f616bfff700 1 --2- 172.21.15.46:0/1856626315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f616c105e10 0x7f616c19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.434+0000 7f61714bf700 1 -- 172.21.15.46:0/1856626315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f616c1054c0 msgr2=0x7f616c19d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.434+0000 7f61714bf700 1 --2- 172.21.15.46:0/1856626315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f616c1054c0 0x7f616c19d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.434+0000 7f61714bf700 1 -- 172.21.15.46:0/1856626315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f616c105e10 msgr2=0x7f616c19d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.434+0000 7f61714bf700 1 --2- 172.21.15.46:0/1856626315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f616c105e10 0x7f616c19d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.434+0000 7f61714bf700 1 -- 172.21.15.46:0/1856626315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f616c1a1f00 con 0x7f616c10e8f0 2024-05-09T04:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.434+0000 7f61714bf700 1 --2- 172.21.15.46:0/1856626315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f616c10e8f0 0x7f616c1a1860 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f616400c970 tx=0x7f616400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.434+0000 7f6169ffb700 1 -- 172.21.15.46:0/1856626315 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f616400ce60 con 0x7f616c10e8f0 2024-05-09T04:48:39.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.434+0000 7f6169ffb700 1 -- 172.21.15.46:0/1856626315 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6164004830 con 0x7f616c10e8f0 2024-05-09T04:48:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.434+0000 7f6172f22700 1 -- 172.21.15.46:0/1856626315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f616c1a21f0 con 0x7f616c10e8f0 2024-05-09T04:48:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.434+0000 7f6169ffb700 1 -- 172.21.15.46:0/1856626315 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61640054c0 con 0x7f616c10e8f0 2024-05-09T04:48:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.435+0000 7f6172f22700 1 -- 172.21.15.46:0/1856626315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f616c1a9b10 con 0x7f616c10e8f0 2024-05-09T04:48:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.436+0000 7f6172f22700 1 -- 172.21.15.46:0/1856626315 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f616c0512c0 con 0x7f616c10e8f0 2024-05-09T04:48:39.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.437+0000 7f6169ffb700 1 -- 172.21.15.46:0/1856626315 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f61640049e0 con 0x7f616c10e8f0 2024-05-09T04:48:39.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.438+0000 7f6169ffb700 1 --2- 172.21.15.46:0/1856626315 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f61580b2990 0x7f61580b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:39.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.438+0000 7f6170cbe700 1 --2- 172.21.15.46:0/1856626315 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f61580b2990 0x7f61580b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:39.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.439+0000 7f6169ffb700 1 -- 172.21.15.46:0/1856626315 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f61640e2260 con 0x7f616c10e8f0 2024-05-09T04:48:39.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.439+0000 7f6170cbe700 1 --2- 172.21.15.46:0/1856626315 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f61580b2990 0x7f61580b4ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f615c009900 tx=0x7f615c00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:39.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.442+0000 7f6169ffb700 1 -- 172.21.15.46:0/1856626315 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6164014030 con 0x7f616c10e8f0 2024-05-09T04:48:39.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.654+0000 7f6172f22700 1 -- 172.21.15.46:0/1856626315 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f616c103890 con 0x7f61580b2990 2024-05-09T04:48:39.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.661+0000 7f6169ffb700 1 -- 172.21.15.46:0/1856626315 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258979 (secure 0 0 0) 0x7f616c103890 con 0x7f61580b2990 2024-05-09T04:48:39.665 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:48:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.665+0000 7f6172f22700 1 -- 172.21.15.46:0/1856626315 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f61580b2990 msgr2=0x7f61580b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.665+0000 7f6172f22700 1 --2- 172.21.15.46:0/1856626315 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f61580b2990 0x7f61580b4ea0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f615c009900 tx=0x7f615c00b7b0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.665+0000 7f6172f22700 1 -- 172.21.15.46:0/1856626315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f616c10e8f0 msgr2=0x7f616c1a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.665+0000 7f6172f22700 1 --2- 172.21.15.46:0/1856626315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f616c10e8f0 0x7f616c1a1860 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f616400c970 tx=0x7f616400cc80 comp rx=0 tx=0).stop 2024-05-09T04:48:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.665+0000 7f6172f22700 1 -- 172.21.15.46:0/1856626315 shutdown_connections 2024-05-09T04:48:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.665+0000 7f6172f22700 1 --2- 172.21.15.46:0/1856626315 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f61580b2990 0x7f61580b4ea0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.665+0000 7f6172f22700 1 --2- 172.21.15.46:0/1856626315 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f616c10e8f0 0x7f616c1a1860 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.665+0000 7f6172f22700 1 --2- 172.21.15.46:0/1856626315 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f616c1054c0 0x7f616c19d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.665+0000 7f6172f22700 1 --2- 172.21.15.46:0/1856626315 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f616c105e10 0x7f616c19d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.665+0000 7f6172f22700 1 -- 172.21.15.46:0/1856626315 >> 172.21.15.46:0/1856626315 conn(0x7f616c0fedf0 msgr2=0x7f616c103180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.665+0000 7f6172f22700 1 -- 172.21.15.46:0/1856626315 shutdown_connections 2024-05-09T04:48:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:39.665+0000 7f6172f22700 1 -- 172.21.15.46:0/1856626315 wait complete. 2024-05-09T04:48:39.667 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T04:48:40.380 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":318,"stamp":"2024-05-09T04:48:39.309207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654950,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":284,"num_read_kb":229,"num_write":289,"num_write_kb":2263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":3002535,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364168,"kb_used_data":10812,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577656,"statfs":{"total":1151940427776,"available":1151567519744,"internally_reserved":0,"allocated":11071488,"data_stored":7141782,"data_compressed":186290,"data_compressed_allocated":245760,"data_compressed_original":2015232,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":32264,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":79,"num_read_kb":63,"num_write":15,"num_write_kb":37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":50695,"num_keys_recovered":10,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.758416"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.553114+0000","last_change":"2024-05-09T04:47:48.408878+0000","last_active":"2024-05-09T04:48:28.553114+0000","last_peered":"2024-05-09T04:48:28.553114+0000","last_clean":"2024-05-09T04:48:28.553114+0000","last_became_active":"2024-05-09T04:47:48.408479+0000","last_became_peered":"2024-05-09T04:47:48.408479+0000","last_unstale":"2024-05-09T04:48:28.553114+0000","last_undegraded":"2024-05-09T04:48:28.553114+0000","last_fullsized":"2024-05-09T04:48:28.553114+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:02.494768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556491+0000","last_change":"2024-05-09T04:47:48.415650+0000","last_active":"2024-05-09T04:48:28.556491+0000","last_peered":"2024-05-09T04:48:28.556491+0000","last_clean":"2024-05-09T04:48:28.556491+0000","last_became_active":"2024-05-09T04:47:48.415322+0000","last_became_peered":"2024-05-09T04:47:48.415322+0000","last_unstale":"2024-05-09T04:48:28.556491+0000","last_undegraded":"2024-05-09T04:48:28.556491+0000","last_fullsized":"2024-05-09T04:48:28.556491+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.142513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556243+0000","last_change":"2024-05-09T04:47:48.417706+0000","last_active":"2024-05-09T04:48:28.556243+0000","last_peered":"2024-05-09T04:48:28.556243+0000","last_clean":"2024-05-09T04:48:28.556243+0000","last_became_active":"2024-05-09T04:47:48.412068+0000","last_became_peered":"2024-05-09T04:47:48.412068+0000","last_unstale":"2024-05-09T04:48:28.556243+0000","last_undegraded":"2024-05-09T04:48:28.556243+0000","last_fullsized":"2024-05-09T04:48:28.556243+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:44.602567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.553597+0000","last_change":"2024-05-09T04:47:48.415751+0000","last_active":"2024-05-09T04:48:28.553597+0000","last_peered":"2024-05-09T04:48:28.553597+0000","last_clean":"2024-05-09T04:48:28.553597+0000","last_became_active":"2024-05-09T04:47:48.414312+0000","last_became_peered":"2024-05-09T04:47:48.414312+0000","last_unstale":"2024-05-09T04:48:28.553597+0000","last_undegraded":"2024-05-09T04:48:28.553597+0000","last_fullsized":"2024-05-09T04:48:28.553597+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:12.202761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556094+0000","last_change":"2024-05-09T04:47:48.415919+0000","last_active":"2024-05-09T04:48:28.556094+0000","last_peered":"2024-05-09T04:48:28.556094+0000","last_clean":"2024-05-09T04:48:28.556094+0000","last_became_active":"2024-05-09T04:47:48.415533+0000","last_became_peered":"2024-05-09T04:47:48.415533+0000","last_unstale":"2024-05-09T04:48:28.556094+0000","last_undegraded":"2024-05-09T04:48:28.556094+0000","last_fullsized":"2024-05-09T04:48:28.556094+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:32.262416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557708+0000","last_change":"2024-05-09T04:47:48.419705+0000","last_active":"2024-05-09T04:48:28.557708+0000","last_peered":"2024-05-09T04:48:28.557708+0000","last_clean":"2024-05-09T04:48:28.557708+0000","last_became_active":"2024-05-09T04:47:48.419339+0000","last_became_peered":"2024-05-09T04:47:48.419339+0000","last_unstale":"2024-05-09T04:48:28.557708+0000","last_undegraded":"2024-05-09T04:48:28.557708+0000","last_fullsized":"2024-05-09T04:48:28.557708+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:50:23.754071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555469+0000","last_change":"2024-05-09T04:47:48.404106+0000","last_active":"2024-05-09T04:48:28.555469+0000","last_peered":"2024-05-09T04:48:28.555469+0000","last_clean":"2024-05-09T04:48:28.555469+0000","last_became_active":"2024-05-09T04:47:48.403724+0000","last_became_peered":"2024-05-09T04:47:48.403724+0000","last_unstale":"2024-05-09T04:48:28.555469+0000","last_undegraded":"2024-05-09T04:48:28.555469+0000","last_fullsized":"2024-05-09T04:48:28.555469+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:41.134752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.562515+0000","last_change":"2024-05-09T04:48:28.562515+0000","last_active":"2024-05-09T04:48:28.562515+0000","last_peered":"2024-05-09T04:48:28.562515+0000","last_clean":"2024-05-09T04:48:28.562515+0000","last_became_active":"2024-05-09T04:48:28.562104+0000","last_became_peered":"2024-05-09T04:48:28.562104+0000","last_unstale":"2024-05-09T04:48:28.562515+0000","last_undegraded":"2024-05-09T04:48:28.562515+0000","last_fullsized":"2024-05-09T04:48:28.562515+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:33.342877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.561601+0000","last_change":"2024-05-09T04:47:48.400467+0000","last_active":"2024-05-09T04:48:28.561601+0000","last_peered":"2024-05-09T04:48:28.561601+0000","last_clean":"2024-05-09T04:48:28.561601+0000","last_became_active":"2024-05-09T04:47:48.399960+0000","last_became_peered":"2024-05-09T04:47:48.399960+0000","last_unstale":"2024-05-09T04:48:28.561601+0000","last_undegraded":"2024-05-09T04:48:28.561601+0000","last_fullsized":"2024-05-09T04:48:28.561601+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:23:44.473053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558256+0000","last_change":"2024-05-09T04:47:48.418970+0000","last_active":"2024-05-09T04:48:28.558256+0000","last_peered":"2024-05-09T04:48:28.558256+0000","last_clean":"2024-05-09T04:48:28.558256+0000","last_became_active":"2024-05-09T04:47:48.418574+0000","last_became_peered":"2024-05-09T04:47:48.418574+0000","last_unstale":"2024-05-09T04:48:28.558256+0000","last_undegraded":"2024-05-09T04:48:28.558256+0000","last_fullsized":"2024-05-09T04:48:28.558256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:48.075893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556185+0000","last_change":"2024-05-09T04:47:48.409879+0000","last_active":"2024-05-09T04:48:28.556185+0000","last_peered":"2024-05-09T04:48:28.556185+0000","last_clean":"2024-05-09T04:48:28.556185+0000","last_became_active":"2024-05-09T04:47:48.409515+0000","last_became_peered":"2024-05-09T04:47:48.409515+0000","last_unstale":"2024-05-09T04:48:28.556185+0000","last_undegraded":"2024-05-09T04:48:28.556185+0000","last_fullsized":"2024-05-09T04:48:28.556185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:15.546214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.553511+0000","last_change":"2024-05-09T04:47:48.417556+0000","last_active":"2024-05-09T04:48:28.553511+0000","last_peered":"2024-05-09T04:48:28.553511+0000","last_clean":"2024-05-09T04:48:28.553511+0000","last_became_active":"2024-05-09T04:47:48.416787+0000","last_became_peered":"2024-05-09T04:47:48.416787+0000","last_unstale":"2024-05-09T04:48:28.553511+0000","last_undegraded":"2024-05-09T04:48:28.553511+0000","last_fullsized":"2024-05-09T04:48:28.553511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:20.360232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552315+0000","last_change":"2024-05-09T04:47:48.415145+0000","last_active":"2024-05-09T04:48:28.552315+0000","last_peered":"2024-05-09T04:48:28.552315+0000","last_clean":"2024-05-09T04:48:28.552315+0000","last_became_active":"2024-05-09T04:47:48.414830+0000","last_became_peered":"2024-05-09T04:47:48.414830+0000","last_unstale":"2024-05-09T04:48:28.552315+0000","last_undegraded":"2024-05-09T04:48:28.552315+0000","last_fullsized":"2024-05-09T04:48:28.552315+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:55.936409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556427+0000","last_change":"2024-05-09T04:47:48.413977+0000","last_active":"2024-05-09T04:48:28.556427+0000","last_peered":"2024-05-09T04:48:28.556427+0000","last_clean":"2024-05-09T04:48:28.556427+0000","last_became_active":"2024-05-09T04:47:48.413604+0000","last_became_peered":"2024-05-09T04:47:48.413604+0000","last_unstale":"2024-05-09T04:48:28.556427+0000","last_undegraded":"2024-05-09T04:48:28.556427+0000","last_fullsized":"2024-05-09T04:48:28.556427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:25.462958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556117+0000","last_change":"2024-05-09T04:47:48.419000+0000","last_active":"2024-05-09T04:48:28.556117+0000","last_peered":"2024-05-09T04:48:28.556117+0000","last_clean":"2024-05-09T04:48:28.556117+0000","last_became_active":"2024-05-09T04:47:48.418457+0000","last_became_peered":"2024-05-09T04:47:48.418457+0000","last_unstale":"2024-05-09T04:48:28.556117+0000","last_undegraded":"2024-05-09T04:48:28.556117+0000","last_fullsized":"2024-05-09T04:48:28.556117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:36.325460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.560792+0000","last_change":"2024-05-09T04:47:48.406754+0000","last_active":"2024-05-09T04:48:28.560792+0000","last_peered":"2024-05-09T04:48:28.560792+0000","last_clean":"2024-05-09T04:48:28.560792+0000","last_became_active":"2024-05-09T04:47:48.406396+0000","last_became_peered":"2024-05-09T04:47:48.406396+0000","last_unstale":"2024-05-09T04:48:28.560792+0000","last_undegraded":"2024-05-09T04:48:28.560792+0000","last_fullsized":"2024-05-09T04:48:28.560792+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:46.421061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.553262+0000","last_change":"2024-05-09T04:47:48.409556+0000","last_active":"2024-05-09T04:48:28.553262+0000","last_peered":"2024-05-09T04:48:28.553262+0000","last_clean":"2024-05-09T04:48:28.553262+0000","last_became_active":"2024-05-09T04:47:48.409145+0000","last_became_peered":"2024-05-09T04:47:48.409145+0000","last_unstale":"2024-05-09T04:48:28.553262+0000","last_undegraded":"2024-05-09T04:48:28.553262+0000","last_fullsized":"2024-05-09T04:48:28.553262+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:25.486257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555875+0000","last_change":"2024-05-09T04:47:48.415546+0000","last_active":"2024-05-09T04:48:28.555875+0000","last_peered":"2024-05-09T04:48:28.555875+0000","last_clean":"2024-05-09T04:48:28.555875+0000","last_became_active":"2024-05-09T04:47:48.415221+0000","last_became_peered":"2024-05-09T04:47:48.415221+0000","last_unstale":"2024-05-09T04:48:28.555875+0000","last_undegraded":"2024-05-09T04:48:28.555875+0000","last_fullsized":"2024-05-09T04:48:28.555875+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:57:29.112409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558794+0000","last_change":"2024-05-09T04:47:48.413106+0000","last_active":"2024-05-09T04:48:28.558794+0000","last_peered":"2024-05-09T04:48:28.558794+0000","last_clean":"2024-05-09T04:48:28.558794+0000","last_became_active":"2024-05-09T04:47:48.412700+0000","last_became_peered":"2024-05-09T04:47:48.412700+0000","last_unstale":"2024-05-09T04:48:28.558794+0000","last_undegraded":"2024-05-09T04:48:28.558794+0000","last_fullsized":"2024-05-09T04:48:28.558794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:37:31.682008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":35,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555642+0000","last_change":"2024-05-09T04:47:45.386931+0000","last_active":"2024-05-09T04:48:28.555642+0000","last_peered":"2024-05-09T04:48:28.555642+0000","last_clean":"2024-05-09T04:48:28.555642+0000","last_became_active":"2024-05-09T04:47:45.386562+0000","last_became_peered":"2024-05-09T04:47:45.386562+0000","last_unstale":"2024-05-09T04:48:28.555642+0000","last_undegraded":"2024-05-09T04:48:28.555642+0000","last_fullsized":"2024-05-09T04:48:28.555642+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:59.335687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.662059+0000","last_change":"2024-05-09T04:47:48.411064+0000","last_active":"2024-05-09T04:48:28.662059+0000","last_peered":"2024-05-09T04:48:28.662059+0000","last_clean":"2024-05-09T04:48:28.662059+0000","last_became_active":"2024-05-09T04:47:48.410689+0000","last_became_peered":"2024-05-09T04:47:48.410689+0000","last_unstale":"2024-05-09T04:48:28.662059+0000","last_undegraded":"2024-05-09T04:48:28.662059+0000","last_fullsized":"2024-05-09T04:48:28.662059+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:21.178831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.662343+0000","last_change":"2024-05-09T04:47:45.612837+0000","last_active":"2024-05-09T04:48:28.662343+0000","last_peered":"2024-05-09T04:48:28.662343+0000","last_clean":"2024-05-09T04:48:28.662343+0000","last_became_active":"2024-05-09T04:47:45.612307+0000","last_became_peered":"2024-05-09T04:47:45.612307+0000","last_unstale":"2024-05-09T04:48:28.662343+0000","last_undegraded":"2024-05-09T04:48:28.662343+0000","last_fullsized":"2024-05-09T04:48:28.662343+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:59.686391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556513+0000","last_change":"2024-05-09T04:47:48.412781+0000","last_active":"2024-05-09T04:48:28.556513+0000","last_peered":"2024-05-09T04:48:28.556513+0000","last_clean":"2024-05-09T04:48:28.556513+0000","last_became_active":"2024-05-09T04:47:48.411396+0000","last_became_peered":"2024-05-09T04:47:48.411396+0000","last_unstale":"2024-05-09T04:48:28.556513+0000","last_undegraded":"2024-05-09T04:48:28.556513+0000","last_fullsized":"2024-05-09T04:48:28.556513+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:41:22.775432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":17,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.568491+0000","last_change":"2024-05-09T04:48:28.568024+0000","last_active":"2024-05-09T04:48:28.568491+0000","last_peered":"2024-05-09T04:48:28.568491+0000","last_clean":"2024-05-09T04:48:28.568491+0000","last_became_active":"2024-05-09T04:48:28.559441+0000","last_became_peered":"2024-05-09T04:48:28.559441+0000","last_unstale":"2024-05-09T04:48:28.568491+0000","last_undegraded":"2024-05-09T04:48:28.568491+0000","last_fullsized":"2024-05-09T04:48:28.568491+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:56.257589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.561084+0000","last_change":"2024-05-09T04:47:48.421361+0000","last_active":"2024-05-09T04:48:28.561084+0000","last_peered":"2024-05-09T04:48:28.561084+0000","last_clean":"2024-05-09T04:48:28.561084+0000","last_became_active":"2024-05-09T04:47:48.421018+0000","last_became_peered":"2024-05-09T04:47:48.421018+0000","last_unstale":"2024-05-09T04:48:28.561084+0000","last_undegraded":"2024-05-09T04:48:28.561084+0000","last_fullsized":"2024-05-09T04:48:28.561084+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:00.634983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556767+0000","last_change":"2024-05-09T04:47:45.389914+0000","last_active":"2024-05-09T04:48:28.556767+0000","last_peered":"2024-05-09T04:48:28.556767+0000","last_clean":"2024-05-09T04:48:28.556767+0000","last_became_active":"2024-05-09T04:47:45.388361+0000","last_became_peered":"2024-05-09T04:47:45.388361+0000","last_unstale":"2024-05-09T04:48:28.556767+0000","last_undegraded":"2024-05-09T04:48:28.556767+0000","last_fullsized":"2024-05-09T04:48:28.556767+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:07:08.048767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.564243+0000","last_change":"2024-05-09T04:48:28.564243+0000","last_active":"2024-05-09T04:48:28.564243+0000","last_peered":"2024-05-09T04:48:28.564243+0000","last_clean":"2024-05-09T04:48:28.564243+0000","last_became_active":"2024-05-09T04:48:28.563732+0000","last_became_peered":"2024-05-09T04:48:28.563732+0000","last_unstale":"2024-05-09T04:48:28.564243+0000","last_undegraded":"2024-05-09T04:48:28.564243+0000","last_fullsized":"2024-05-09T04:48:28.564243+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:26.982339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":63,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:37.542635+0000","last_change":"2024-05-09T04:47:45.389868+0000","last_active":"2024-05-09T04:48:37.542635+0000","last_peered":"2024-05-09T04:48:37.542635+0000","last_clean":"2024-05-09T04:48:37.542635+0000","last_became_active":"2024-05-09T04:47:45.389508+0000","last_became_peered":"2024-05-09T04:47:45.389508+0000","last_unstale":"2024-05-09T04:48:37.542635+0000","last_undegraded":"2024-05-09T04:48:37.542635+0000","last_fullsized":"2024-05-09T04:48:37.542635+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:45.050187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556021+0000","last_change":"2024-05-09T04:47:48.404457+0000","last_active":"2024-05-09T04:48:28.556021+0000","last_peered":"2024-05-09T04:48:28.556021+0000","last_clean":"2024-05-09T04:48:28.556021+0000","last_became_active":"2024-05-09T04:47:48.403968+0000","last_became_peered":"2024-05-09T04:47:48.403968+0000","last_unstale":"2024-05-09T04:48:28.556021+0000","last_undegraded":"2024-05-09T04:48:28.556021+0000","last_fullsized":"2024-05-09T04:48:28.556021+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:23:40.364916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557640+0000","last_change":"2024-05-09T04:47:45.388183+0000","last_active":"2024-05-09T04:48:28.557640+0000","last_peered":"2024-05-09T04:48:28.557640+0000","last_clean":"2024-05-09T04:48:28.557640+0000","last_became_active":"2024-05-09T04:47:45.386603+0000","last_became_peered":"2024-05-09T04:47:45.386603+0000","last_unstale":"2024-05-09T04:48:28.557640+0000","last_undegraded":"2024-05-09T04:48:28.557640+0000","last_fullsized":"2024-05-09T04:48:28.557640+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:53.433619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.663120+0000","last_change":"2024-05-09T04:48:28.663120+0000","last_active":"2024-05-09T04:48:28.663120+0000","last_peered":"2024-05-09T04:48:28.663120+0000","last_clean":"2024-05-09T04:48:28.663120+0000","last_became_active":"2024-05-09T04:48:28.662748+0000","last_became_peered":"2024-05-09T04:48:28.662748+0000","last_unstale":"2024-05-09T04:48:28.663120+0000","last_undegraded":"2024-05-09T04:48:28.663120+0000","last_fullsized":"2024-05-09T04:48:28.663120+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:57.677876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552103+0000","last_change":"2024-05-09T04:47:45.384192+0000","last_active":"2024-05-09T04:48:28.552103+0000","last_peered":"2024-05-09T04:48:28.552103+0000","last_clean":"2024-05-09T04:48:28.552103+0000","last_became_active":"2024-05-09T04:47:45.383843+0000","last_became_peered":"2024-05-09T04:47:45.383843+0000","last_unstale":"2024-05-09T04:48:28.552103+0000","last_undegraded":"2024-05-09T04:48:28.552103+0000","last_fullsized":"2024-05-09T04:48:28.552103+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:28.815207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.661979+0000","last_change":"2024-05-09T04:47:48.409203+0000","last_active":"2024-05-09T04:48:28.661979+0000","last_peered":"2024-05-09T04:48:28.661979+0000","last_clean":"2024-05-09T04:48:28.661979+0000","last_became_active":"2024-05-09T04:47:48.408861+0000","last_became_peered":"2024-05-09T04:47:48.408861+0000","last_unstale":"2024-05-09T04:48:28.661979+0000","last_undegraded":"2024-05-09T04:48:28.661979+0000","last_fullsized":"2024-05-09T04:48:28.661979+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:07.160397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557850+0000","last_change":"2024-05-09T04:47:45.392723+0000","last_active":"2024-05-09T04:48:28.557850+0000","last_peered":"2024-05-09T04:48:28.557850+0000","last_clean":"2024-05-09T04:48:28.557850+0000","last_became_active":"2024-05-09T04:47:45.392313+0000","last_became_peered":"2024-05-09T04:47:45.392313+0000","last_unstale":"2024-05-09T04:48:28.557850+0000","last_undegraded":"2024-05-09T04:48:28.557850+0000","last_fullsized":"2024-05-09T04:48:28.557850+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:44.358702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.554865+0000","last_change":"2024-05-09T04:47:48.414000+0000","last_active":"2024-05-09T04:48:28.554865+0000","last_peered":"2024-05-09T04:48:28.554865+0000","last_clean":"2024-05-09T04:48:28.554865+0000","last_became_active":"2024-05-09T04:47:48.413532+0000","last_became_peered":"2024-05-09T04:47:48.413532+0000","last_unstale":"2024-05-09T04:48:28.554865+0000","last_undegraded":"2024-05-09T04:48:28.554865+0000","last_fullsized":"2024-05-09T04:48:28.554865+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:22.774814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":59,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:37.541942+0000","last_change":"2024-05-09T04:47:45.388661+0000","last_active":"2024-05-09T04:48:37.541942+0000","last_peered":"2024-05-09T04:48:37.541942+0000","last_clean":"2024-05-09T04:48:37.541942+0000","last_became_active":"2024-05-09T04:47:45.386010+0000","last_became_peered":"2024-05-09T04:47:45.386010+0000","last_unstale":"2024-05-09T04:48:37.541942+0000","last_undegraded":"2024-05-09T04:48:37.541942+0000","last_fullsized":"2024-05-09T04:48:37.541942+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:23.250463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":31,"num_read_kb":31,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.561394+0000","last_change":"2024-05-09T04:47:48.421706+0000","last_active":"2024-05-09T04:48:28.561394+0000","last_peered":"2024-05-09T04:48:28.561394+0000","last_clean":"2024-05-09T04:48:28.561394+0000","last_became_active":"2024-05-09T04:47:48.420529+0000","last_became_peered":"2024-05-09T04:47:48.420529+0000","last_unstale":"2024-05-09T04:48:28.561394+0000","last_undegraded":"2024-05-09T04:48:28.561394+0000","last_fullsized":"2024-05-09T04:48:28.561394+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:10.110366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555946+0000","last_change":"2024-05-09T04:47:45.386625+0000","last_active":"2024-05-09T04:48:28.555946+0000","last_peered":"2024-05-09T04:48:28.555946+0000","last_clean":"2024-05-09T04:48:28.555946+0000","last_became_active":"2024-05-09T04:47:45.385087+0000","last_became_peered":"2024-05-09T04:47:45.385087+0000","last_unstale":"2024-05-09T04:48:28.555946+0000","last_undegraded":"2024-05-09T04:48:28.555946+0000","last_fullsized":"2024-05-09T04:48:28.555946+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:15.546894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.562517+0000","last_change":"2024-05-09T04:48:28.562517+0000","last_active":"2024-05-09T04:48:28.562517+0000","last_peered":"2024-05-09T04:48:28.562517+0000","last_clean":"2024-05-09T04:48:28.562517+0000","last_became_active":"2024-05-09T04:48:28.562047+0000","last_became_peered":"2024-05-09T04:48:28.562047+0000","last_unstale":"2024-05-09T04:48:28.562517+0000","last_undegraded":"2024-05-09T04:48:28.562517+0000","last_fullsized":"2024-05-09T04:48:28.562517+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:22.769844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558540+0000","last_change":"2024-05-09T04:47:45.613713+0000","last_active":"2024-05-09T04:48:28.558540+0000","last_peered":"2024-05-09T04:48:28.558540+0000","last_clean":"2024-05-09T04:48:28.558540+0000","last_became_active":"2024-05-09T04:47:45.613313+0000","last_became_peered":"2024-05-09T04:47:45.613313+0000","last_unstale":"2024-05-09T04:48:28.558540+0000","last_undegraded":"2024-05-09T04:48:28.558540+0000","last_fullsized":"2024-05-09T04:48:28.558540+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:32:44.825695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.561145+0000","last_change":"2024-05-09T04:47:48.411234+0000","last_active":"2024-05-09T04:48:28.561145+0000","last_peered":"2024-05-09T04:48:28.561145+0000","last_clean":"2024-05-09T04:48:28.561145+0000","last_became_active":"2024-05-09T04:47:48.410881+0000","last_became_peered":"2024-05-09T04:47:48.410881+0000","last_unstale":"2024-05-09T04:48:28.561145+0000","last_undegraded":"2024-05-09T04:48:28.561145+0000","last_fullsized":"2024-05-09T04:48:28.561145+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.747871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'29","reported_seq":143,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:38.581646+0000","last_change":"2024-05-09T04:48:28.570426+0000","last_active":"2024-05-09T04:48:38.581646+0000","last_peered":"2024-05-09T04:48:38.581646+0000","last_clean":"2024-05-09T04:48:38.581646+0000","last_became_active":"2024-05-09T04:48:28.561786+0000","last_became_peered":"2024-05-09T04:48:28.561786+0000","last_unstale":"2024-05-09T04:48:38.581646+0000","last_undegraded":"2024-05-09T04:48:38.581646+0000","last_fullsized":"2024-05-09T04:48:38.581646+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:32.367142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":66,"num_read_kb":83,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":50695,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556057+0000","last_change":"2024-05-09T04:47:48.406510+0000","last_active":"2024-05-09T04:48:28.556057+0000","last_peered":"2024-05-09T04:48:28.556057+0000","last_clean":"2024-05-09T04:48:28.556057+0000","last_became_active":"2024-05-09T04:47:48.406166+0000","last_became_peered":"2024-05-09T04:47:48.406166+0000","last_unstale":"2024-05-09T04:48:28.556057+0000","last_undegraded":"2024-05-09T04:48:28.556057+0000","last_fullsized":"2024-05-09T04:48:28.556057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:18.579620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":32,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557905+0000","last_change":"2024-05-09T04:47:45.387164+0000","last_active":"2024-05-09T04:48:28.557905+0000","last_peered":"2024-05-09T04:48:28.557905+0000","last_clean":"2024-05-09T04:48:28.557905+0000","last_became_active":"2024-05-09T04:47:45.386755+0000","last_became_peered":"2024-05-09T04:47:45.386755+0000","last_unstale":"2024-05-09T04:48:28.557905+0000","last_undegraded":"2024-05-09T04:48:28.557905+0000","last_fullsized":"2024-05-09T04:48:28.557905+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:07.838782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552103+0000","last_change":"2024-05-09T04:47:48.405436+0000","last_active":"2024-05-09T04:48:28.552103+0000","last_peered":"2024-05-09T04:48:28.552103+0000","last_clean":"2024-05-09T04:48:28.552103+0000","last_became_active":"2024-05-09T04:47:48.405032+0000","last_became_peered":"2024-05-09T04:47:48.405032+0000","last_unstale":"2024-05-09T04:48:28.552103+0000","last_undegraded":"2024-05-09T04:48:28.552103+0000","last_fullsized":"2024-05-09T04:48:28.552103+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:27.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558055+0000","last_change":"2024-05-09T04:47:45.391970+0000","last_active":"2024-05-09T04:48:28.558055+0000","last_peered":"2024-05-09T04:48:28.558055+0000","last_clean":"2024-05-09T04:48:28.558055+0000","last_became_active":"2024-05-09T04:47:45.391597+0000","last_became_peered":"2024-05-09T04:47:45.391597+0000","last_unstale":"2024-05-09T04:48:28.558055+0000","last_undegraded":"2024-05-09T04:48:28.558055+0000","last_fullsized":"2024-05-09T04:48:28.558055+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:27.076631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556549+0000","last_change":"2024-05-09T04:47:48.419328+0000","last_active":"2024-05-09T04:48:28.556549+0000","last_peered":"2024-05-09T04:48:28.556549+0000","last_clean":"2024-05-09T04:48:28.556549+0000","last_became_active":"2024-05-09T04:47:48.418972+0000","last_became_peered":"2024-05-09T04:47:48.418972+0000","last_unstale":"2024-05-09T04:48:28.556549+0000","last_undegraded":"2024-05-09T04:48:28.556549+0000","last_fullsized":"2024-05-09T04:48:28.556549+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:15.164747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":62,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:35.539190+0000","last_change":"2024-05-09T04:47:45.386173+0000","last_active":"2024-05-09T04:48:35.539190+0000","last_peered":"2024-05-09T04:48:35.539190+0000","last_clean":"2024-05-09T04:48:35.539190+0000","last_became_active":"2024-05-09T04:47:45.384296+0000","last_became_peered":"2024-05-09T04:47:45.384296+0000","last_unstale":"2024-05-09T04:48:35.539190+0000","last_undegraded":"2024-05-09T04:48:35.539190+0000","last_fullsized":"2024-05-09T04:48:35.539190+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:03.246187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":33,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555463+0000","last_change":"2024-05-09T04:47:48.415645+0000","last_active":"2024-05-09T04:48:28.555463+0000","last_peered":"2024-05-09T04:48:28.555463+0000","last_clean":"2024-05-09T04:48:28.555463+0000","last_became_active":"2024-05-09T04:47:48.415303+0000","last_became_peered":"2024-05-09T04:47:48.415303+0000","last_unstale":"2024-05-09T04:48:28.555463+0000","last_undegraded":"2024-05-09T04:48:28.555463+0000","last_fullsized":"2024-05-09T04:48:28.555463+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:39.703948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":42,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555714+0000","last_change":"2024-05-09T04:47:45.613320+0000","last_active":"2024-05-09T04:48:28.555714+0000","last_peered":"2024-05-09T04:48:28.555714+0000","last_clean":"2024-05-09T04:48:28.555714+0000","last_became_active":"2024-05-09T04:47:45.612871+0000","last_became_peered":"2024-05-09T04:47:45.612871+0000","last_unstale":"2024-05-09T04:48:28.555714+0000","last_undegraded":"2024-05-09T04:48:28.555714+0000","last_fullsized":"2024-05-09T04:48:28.555714+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:31.516652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.563376+0000","last_change":"2024-05-09T04:48:28.563376+0000","last_active":"2024-05-09T04:48:28.563376+0000","last_peered":"2024-05-09T04:48:28.563376+0000","last_clean":"2024-05-09T04:48:28.563376+0000","last_became_active":"2024-05-09T04:48:28.562973+0000","last_became_peered":"2024-05-09T04:48:28.562973+0000","last_unstale":"2024-05-09T04:48:28.563376+0000","last_undegraded":"2024-05-09T04:48:28.563376+0000","last_fullsized":"2024-05-09T04:48:28.563376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:20.068517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556678+0000","last_change":"2024-05-09T04:47:45.389785+0000","last_active":"2024-05-09T04:48:28.556678+0000","last_peered":"2024-05-09T04:48:28.556678+0000","last_clean":"2024-05-09T04:48:28.556678+0000","last_became_active":"2024-05-09T04:47:45.389159+0000","last_became_peered":"2024-05-09T04:47:45.389159+0000","last_unstale":"2024-05-09T04:48:28.556678+0000","last_undegraded":"2024-05-09T04:48:28.556678+0000","last_fullsized":"2024-05-09T04:48:28.556678+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:28:51.804123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.563170+0000","last_change":"2024-05-09T04:48:28.563170+0000","last_active":"2024-05-09T04:48:28.563170+0000","last_peered":"2024-05-09T04:48:28.563170+0000","last_clean":"2024-05-09T04:48:28.563170+0000","last_became_active":"2024-05-09T04:48:28.562189+0000","last_became_peered":"2024-05-09T04:48:28.562189+0000","last_unstale":"2024-05-09T04:48:28.563170+0000","last_undegraded":"2024-05-09T04:48:28.563170+0000","last_fullsized":"2024-05-09T04:48:28.563170+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:00.643292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":25,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555516+0000","last_change":"2024-05-09T04:47:45.613739+0000","last_active":"2024-05-09T04:48:28.555516+0000","last_peered":"2024-05-09T04:48:28.555516+0000","last_clean":"2024-05-09T04:48:28.555516+0000","last_became_active":"2024-05-09T04:47:45.613365+0000","last_became_peered":"2024-05-09T04:47:45.613365+0000","last_unstale":"2024-05-09T04:48:28.555516+0000","last_undegraded":"2024-05-09T04:48:28.555516+0000","last_fullsized":"2024-05-09T04:48:28.555516+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:57.138068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":19,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557722+0000","last_change":"2024-05-09T04:47:48.413343+0000","last_active":"2024-05-09T04:48:28.557722+0000","last_peered":"2024-05-09T04:48:28.557722+0000","last_clean":"2024-05-09T04:48:28.557722+0000","last_became_active":"2024-05-09T04:47:48.412941+0000","last_became_peered":"2024-05-09T04:47:48.412941+0000","last_unstale":"2024-05-09T04:48:28.557722+0000","last_undegraded":"2024-05-09T04:48:28.557722+0000","last_fullsized":"2024-05-09T04:48:28.557722+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:28.764553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555616+0000","last_change":"2024-05-09T04:47:45.386487+0000","last_active":"2024-05-09T04:48:28.555616+0000","last_peered":"2024-05-09T04:48:28.555616+0000","last_clean":"2024-05-09T04:48:28.555616+0000","last_became_active":"2024-05-09T04:47:45.386078+0000","last_became_peered":"2024-05-09T04:47:45.386078+0000","last_unstale":"2024-05-09T04:48:28.555616+0000","last_undegraded":"2024-05-09T04:48:28.555616+0000","last_fullsized":"2024-05-09T04:48:28.555616+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:19.998977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557819+0000","last_change":"2024-05-09T04:47:48.418955+0000","last_active":"2024-05-09T04:48:28.557819+0000","last_peered":"2024-05-09T04:48:28.557819+0000","last_clean":"2024-05-09T04:48:28.557819+0000","last_became_active":"2024-05-09T04:47:48.418528+0000","last_became_peered":"2024-05-09T04:47:48.418528+0000","last_unstale":"2024-05-09T04:48:28.557819+0000","last_undegraded":"2024-05-09T04:48:28.557819+0000","last_fullsized":"2024-05-09T04:48:28.557819+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:51:02.386342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558153+0000","last_change":"2024-05-09T04:47:48.413304+0000","last_active":"2024-05-09T04:48:28.558153+0000","last_peered":"2024-05-09T04:48:28.558153+0000","last_clean":"2024-05-09T04:48:28.558153+0000","last_became_active":"2024-05-09T04:47:48.412582+0000","last_became_peered":"2024-05-09T04:47:48.412582+0000","last_unstale":"2024-05-09T04:48:28.558153+0000","last_undegraded":"2024-05-09T04:48:28.558153+0000","last_fullsized":"2024-05-09T04:48:28.558153+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:40.027082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":57,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:33.535610+0000","last_change":"2024-05-09T04:47:45.612742+0000","last_active":"2024-05-09T04:48:33.535610+0000","last_peered":"2024-05-09T04:48:33.535610+0000","last_clean":"2024-05-09T04:48:33.535610+0000","last_became_active":"2024-05-09T04:47:45.612325+0000","last_became_peered":"2024-05-09T04:47:45.612325+0000","last_unstale":"2024-05-09T04:48:33.535610+0000","last_undegraded":"2024-05-09T04:48:33.535610+0000","last_fullsized":"2024-05-09T04:48:33.535610+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:00.573996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.661996+0000","last_change":"2024-05-09T04:47:48.412894+0000","last_active":"2024-05-09T04:48:28.661996+0000","last_peered":"2024-05-09T04:48:28.661996+0000","last_clean":"2024-05-09T04:48:28.661996+0000","last_became_active":"2024-05-09T04:47:48.412265+0000","last_became_peered":"2024-05-09T04:47:48.412265+0000","last_unstale":"2024-05-09T04:48:28.661996+0000","last_undegraded":"2024-05-09T04:48:28.661996+0000","last_fullsized":"2024-05-09T04:48:28.661996+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:23.517566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":31,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.553331+0000","last_change":"2024-05-09T04:47:45.387715+0000","last_active":"2024-05-09T04:48:28.553331+0000","last_peered":"2024-05-09T04:48:28.553331+0000","last_clean":"2024-05-09T04:48:28.553331+0000","last_became_active":"2024-05-09T04:47:45.387229+0000","last_became_peered":"2024-05-09T04:47:45.387229+0000","last_unstale":"2024-05-09T04:48:28.553331+0000","last_undegraded":"2024-05-09T04:48:28.553331+0000","last_fullsized":"2024-05-09T04:48:28.553331+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:22.540214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":99,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555632+0000","last_change":"2024-05-09T04:46:21.632579+0000","last_active":"2024-05-09T04:48:28.555632+0000","last_peered":"2024-05-09T04:48:28.555632+0000","last_clean":"2024-05-09T04:48:28.555632+0000","last_became_active":"2024-05-09T04:46:21.627483+0000","last_became_peered":"2024-05-09T04:46:21.627483+0000","last_unstale":"2024-05-09T04:48:28.555632+0000","last_undegraded":"2024-05-09T04:48:28.555632+0000","last_fullsized":"2024-05-09T04:48:28.555632+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:42:54.179739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:42:54.179739+0000","last_clean_scrub_stamp":"2024-05-09T04:42:54.179739+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:44.331526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.551975+0000","last_change":"2024-05-09T04:47:48.418570+0000","last_active":"2024-05-09T04:48:28.551975+0000","last_peered":"2024-05-09T04:48:28.551975+0000","last_clean":"2024-05-09T04:48:28.551975+0000","last_became_active":"2024-05-09T04:47:48.418270+0000","last_became_peered":"2024-05-09T04:47:48.418270+0000","last_unstale":"2024-05-09T04:48:28.551975+0000","last_undegraded":"2024-05-09T04:48:28.551975+0000","last_fullsized":"2024-05-09T04:48:28.551975+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:28.406846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555742+0000","last_change":"2024-05-09T04:47:45.387161+0000","last_active":"2024-05-09T04:48:28.555742+0000","last_peered":"2024-05-09T04:48:28.555742+0000","last_clean":"2024-05-09T04:48:28.555742+0000","last_became_active":"2024-05-09T04:47:45.386753+0000","last_became_peered":"2024-05-09T04:47:45.386753+0000","last_unstale":"2024-05-09T04:48:28.555742+0000","last_undegraded":"2024-05-09T04:48:28.555742+0000","last_fullsized":"2024-05-09T04:48:28.555742+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:15.402418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556181+0000","last_change":"2024-05-09T04:47:45.613551+0000","last_active":"2024-05-09T04:48:28.556181+0000","last_peered":"2024-05-09T04:48:28.556181+0000","last_clean":"2024-05-09T04:48:28.556181+0000","last_became_active":"2024-05-09T04:47:45.613086+0000","last_became_peered":"2024-05-09T04:47:45.613086+0000","last_unstale":"2024-05-09T04:48:28.556181+0000","last_undegraded":"2024-05-09T04:48:28.556181+0000","last_fullsized":"2024-05-09T04:48:28.556181+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:04.350960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552185+0000","last_change":"2024-05-09T04:47:48.403701+0000","last_active":"2024-05-09T04:48:28.552185+0000","last_peered":"2024-05-09T04:48:28.552185+0000","last_clean":"2024-05-09T04:48:28.552185+0000","last_became_active":"2024-05-09T04:47:48.403337+0000","last_became_peered":"2024-05-09T04:47:48.403337+0000","last_unstale":"2024-05-09T04:48:28.552185+0000","last_undegraded":"2024-05-09T04:48:28.552185+0000","last_fullsized":"2024-05-09T04:48:28.552185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:53.032348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":36,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:31.417160+0000","last_change":"2024-05-09T04:47:45.385935+0000","last_active":"2024-05-09T04:48:31.417160+0000","last_peered":"2024-05-09T04:48:31.417160+0000","last_clean":"2024-05-09T04:48:31.417160+0000","last_became_active":"2024-05-09T04:47:45.385533+0000","last_became_peered":"2024-05-09T04:47:45.385533+0000","last_unstale":"2024-05-09T04:48:31.417160+0000","last_undegraded":"2024-05-09T04:48:31.417160+0000","last_fullsized":"2024-05-09T04:48:31.417160+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:34:38.014201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.662346+0000","last_change":"2024-05-09T04:47:48.409590+0000","last_active":"2024-05-09T04:48:28.662346+0000","last_peered":"2024-05-09T04:48:28.662346+0000","last_clean":"2024-05-09T04:48:28.662346+0000","last_became_active":"2024-05-09T04:47:48.409191+0000","last_became_peered":"2024-05-09T04:47:48.409191+0000","last_unstale":"2024-05-09T04:48:28.662346+0000","last_undegraded":"2024-05-09T04:48:28.662346+0000","last_fullsized":"2024-05-09T04:48:28.662346+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:50.440609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":30,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.553819+0000","last_change":"2024-05-09T04:47:45.389147+0000","last_active":"2024-05-09T04:48:28.553819+0000","last_peered":"2024-05-09T04:48:28.553819+0000","last_clean":"2024-05-09T04:48:28.553819+0000","last_became_active":"2024-05-09T04:47:45.388797+0000","last_became_peered":"2024-05-09T04:47:45.388797+0000","last_unstale":"2024-05-09T04:48:28.553819+0000","last_undegraded":"2024-05-09T04:48:28.553819+0000","last_fullsized":"2024-05-09T04:48:28.553819+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:19.265278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556603+0000","last_change":"2024-05-09T04:47:48.419100+0000","last_active":"2024-05-09T04:48:28.556603+0000","last_peered":"2024-05-09T04:48:28.556603+0000","last_clean":"2024-05-09T04:48:28.556603+0000","last_became_active":"2024-05-09T04:47:48.418499+0000","last_became_peered":"2024-05-09T04:47:48.418499+0000","last_unstale":"2024-05-09T04:48:28.556603+0000","last_undegraded":"2024-05-09T04:48:28.556603+0000","last_fullsized":"2024-05-09T04:48:28.556603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:23.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.560862+0000","last_change":"2024-05-09T04:47:45.613269+0000","last_active":"2024-05-09T04:48:28.560862+0000","last_peered":"2024-05-09T04:48:28.560862+0000","last_clean":"2024-05-09T04:48:28.560862+0000","last_became_active":"2024-05-09T04:47:45.612895+0000","last_became_peered":"2024-05-09T04:47:45.612895+0000","last_unstale":"2024-05-09T04:48:28.560862+0000","last_undegraded":"2024-05-09T04:48:28.560862+0000","last_fullsized":"2024-05-09T04:48:28.560862+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:52.966974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556360+0000","last_change":"2024-05-09T04:47:48.416658+0000","last_active":"2024-05-09T04:48:28.556360+0000","last_peered":"2024-05-09T04:48:28.556360+0000","last_clean":"2024-05-09T04:48:28.556360+0000","last_became_active":"2024-05-09T04:47:48.416238+0000","last_became_peered":"2024-05-09T04:47:48.416238+0000","last_unstale":"2024-05-09T04:48:28.556360+0000","last_undegraded":"2024-05-09T04:48:28.556360+0000","last_fullsized":"2024-05-09T04:48:28.556360+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:35:50.126723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.661867+0000","last_change":"2024-05-09T04:47:45.384018+0000","last_active":"2024-05-09T04:48:28.661867+0000","last_peered":"2024-05-09T04:48:28.661867+0000","last_clean":"2024-05-09T04:48:28.661867+0000","last_became_active":"2024-05-09T04:47:45.383647+0000","last_became_peered":"2024-05-09T04:47:45.383647+0000","last_unstale":"2024-05-09T04:48:28.661867+0000","last_undegraded":"2024-05-09T04:48:28.661867+0000","last_fullsized":"2024-05-09T04:48:28.661867+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:06.431233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552763+0000","last_change":"2024-05-09T04:47:48.409089+0000","last_active":"2024-05-09T04:48:28.552763+0000","last_peered":"2024-05-09T04:48:28.552763+0000","last_clean":"2024-05-09T04:48:28.552763+0000","last_became_active":"2024-05-09T04:47:48.408548+0000","last_became_peered":"2024-05-09T04:47:48.408548+0000","last_unstale":"2024-05-09T04:48:28.552763+0000","last_undegraded":"2024-05-09T04:48:28.552763+0000","last_fullsized":"2024-05-09T04:48:28.552763+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:13.633008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":26,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557606+0000","last_change":"2024-05-09T04:47:45.387580+0000","last_active":"2024-05-09T04:48:28.557606+0000","last_peered":"2024-05-09T04:48:28.557606+0000","last_clean":"2024-05-09T04:48:28.557606+0000","last_became_active":"2024-05-09T04:47:45.386929+0000","last_became_peered":"2024-05-09T04:47:45.386929+0000","last_unstale":"2024-05-09T04:48:28.557606+0000","last_undegraded":"2024-05-09T04:48:28.557606+0000","last_fullsized":"2024-05-09T04:48:28.557606+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:19.661763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.662079+0000","last_change":"2024-05-09T04:47:48.416311+0000","last_active":"2024-05-09T04:48:28.662079+0000","last_peered":"2024-05-09T04:48:28.662079+0000","last_clean":"2024-05-09T04:48:28.662079+0000","last_became_active":"2024-05-09T04:47:48.415969+0000","last_became_peered":"2024-05-09T04:47:48.415969+0000","last_unstale":"2024-05-09T04:48:28.662079+0000","last_undegraded":"2024-05-09T04:48:28.662079+0000","last_fullsized":"2024-05-09T04:48:28.662079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:55.751210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":43,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.573916+0000","last_change":"2024-05-09T04:48:28.573916+0000","last_active":"2024-05-09T04:48:28.573916+0000","last_peered":"2024-05-09T04:48:28.573916+0000","last_clean":"2024-05-09T04:48:28.573916+0000","last_became_active":"2024-05-09T04:48:28.562872+0000","last_became_peered":"2024-05-09T04:48:28.562872+0000","last_unstale":"2024-05-09T04:48:28.573916+0000","last_undegraded":"2024-05-09T04:48:28.573916+0000","last_fullsized":"2024-05-09T04:48:28.573916+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:02.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555793+0000","last_change":"2024-05-09T04:47:48.408364+0000","last_active":"2024-05-09T04:48:28.555793+0000","last_peered":"2024-05-09T04:48:28.555793+0000","last_clean":"2024-05-09T04:48:28.555793+0000","last_became_active":"2024-05-09T04:47:48.407982+0000","last_became_peered":"2024-05-09T04:47:48.407982+0000","last_unstale":"2024-05-09T04:48:28.555793+0000","last_undegraded":"2024-05-09T04:48:28.555793+0000","last_fullsized":"2024-05-09T04:48:28.555793+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:14.103171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":29,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558323+0000","last_change":"2024-05-09T04:47:45.614637+0000","last_active":"2024-05-09T04:48:28.558323+0000","last_peered":"2024-05-09T04:48:28.558323+0000","last_clean":"2024-05-09T04:48:28.558323+0000","last_became_active":"2024-05-09T04:47:45.613290+0000","last_became_peered":"2024-05-09T04:47:45.613290+0000","last_unstale":"2024-05-09T04:48:28.558323+0000","last_undegraded":"2024-05-09T04:48:28.558323+0000","last_fullsized":"2024-05-09T04:48:28.558323+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:48.547382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556319+0000","last_change":"2024-05-09T04:47:48.405537+0000","last_active":"2024-05-09T04:48:28.556319+0000","last_peered":"2024-05-09T04:48:28.556319+0000","last_clean":"2024-05-09T04:48:28.556319+0000","last_became_active":"2024-05-09T04:47:48.405148+0000","last_became_peered":"2024-05-09T04:47:48.405148+0000","last_unstale":"2024-05-09T04:48:28.556319+0000","last_undegraded":"2024-05-09T04:48:28.556319+0000","last_fullsized":"2024-05-09T04:48:28.556319+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:46.981401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":27,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558388+0000","last_change":"2024-05-09T04:47:45.389151+0000","last_active":"2024-05-09T04:48:28.558388+0000","last_peered":"2024-05-09T04:48:28.558388+0000","last_clean":"2024-05-09T04:48:28.558388+0000","last_became_active":"2024-05-09T04:47:45.388735+0000","last_became_peered":"2024-05-09T04:47:45.388735+0000","last_unstale":"2024-05-09T04:48:28.558388+0000","last_undegraded":"2024-05-09T04:48:28.558388+0000","last_fullsized":"2024-05-09T04:48:28.558388+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:26.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":21,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.555966+0000","last_change":"2024-05-09T04:47:48.413881+0000","last_active":"2024-05-09T04:48:28.555966+0000","last_peered":"2024-05-09T04:48:28.555966+0000","last_clean":"2024-05-09T04:48:28.555966+0000","last_became_active":"2024-05-09T04:47:48.413559+0000","last_became_peered":"2024-05-09T04:47:48.413559+0000","last_unstale":"2024-05-09T04:48:28.555966+0000","last_undegraded":"2024-05-09T04:48:28.555966+0000","last_fullsized":"2024-05-09T04:48:28.555966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:40:40.305319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":28,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552437+0000","last_change":"2024-05-09T04:47:45.383406+0000","last_active":"2024-05-09T04:48:28.552437+0000","last_peered":"2024-05-09T04:48:28.552437+0000","last_clean":"2024-05-09T04:48:28.552437+0000","last_became_active":"2024-05-09T04:47:45.383025+0000","last_became_peered":"2024-05-09T04:47:45.383025+0000","last_unstale":"2024-05-09T04:48:28.552437+0000","last_undegraded":"2024-05-09T04:48:28.552437+0000","last_fullsized":"2024-05-09T04:48:28.552437+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:46.007454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556288+0000","last_change":"2024-05-09T04:47:48.412481+0000","last_active":"2024-05-09T04:48:28.556288+0000","last_peered":"2024-05-09T04:48:28.556288+0000","last_clean":"2024-05-09T04:48:28.556288+0000","last_became_active":"2024-05-09T04:47:48.412034+0000","last_became_peered":"2024-05-09T04:47:48.412034+0000","last_unstale":"2024-05-09T04:48:28.556288+0000","last_undegraded":"2024-05-09T04:48:28.556288+0000","last_fullsized":"2024-05-09T04:48:28.556288+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:14.030981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552849+0000","last_change":"2024-05-09T04:47:48.406958+0000","last_active":"2024-05-09T04:48:28.552849+0000","last_peered":"2024-05-09T04:48:28.552849+0000","last_clean":"2024-05-09T04:48:28.552849+0000","last_became_active":"2024-05-09T04:47:48.406575+0000","last_became_peered":"2024-05-09T04:47:48.406575+0000","last_unstale":"2024-05-09T04:48:28.552849+0000","last_undegraded":"2024-05-09T04:48:28.552849+0000","last_fullsized":"2024-05-09T04:48:28.552849+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:23.389676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.562376+0000","last_change":"2024-05-09T04:48:28.562376+0000","last_active":"2024-05-09T04:48:28.562376+0000","last_peered":"2024-05-09T04:48:28.562376+0000","last_clean":"2024-05-09T04:48:28.562376+0000","last_became_active":"2024-05-09T04:48:28.561866+0000","last_became_peered":"2024-05-09T04:48:28.561866+0000","last_unstale":"2024-05-09T04:48:28.562376+0000","last_undegraded":"2024-05-09T04:48:28.562376+0000","last_fullsized":"2024-05-09T04:48:28.562376+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:51.974146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556736+0000","last_change":"2024-05-09T04:47:48.413908+0000","last_active":"2024-05-09T04:48:28.556736+0000","last_peered":"2024-05-09T04:48:28.556736+0000","last_clean":"2024-05-09T04:48:28.556736+0000","last_became_active":"2024-05-09T04:47:48.413575+0000","last_became_peered":"2024-05-09T04:47:48.413575+0000","last_unstale":"2024-05-09T04:48:28.556736+0000","last_undegraded":"2024-05-09T04:48:28.556736+0000","last_fullsized":"2024-05-09T04:48:28.556736+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:40.742510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.556432+0000","last_change":"2024-05-09T04:47:48.406607+0000","last_active":"2024-05-09T04:48:28.556432+0000","last_peered":"2024-05-09T04:48:28.556432+0000","last_clean":"2024-05-09T04:48:28.556432+0000","last_became_active":"2024-05-09T04:47:48.406290+0000","last_became_peered":"2024-05-09T04:47:48.406290+0000","last_unstale":"2024-05-09T04:48:28.556432+0000","last_undegraded":"2024-05-09T04:48:28.556432+0000","last_fullsized":"2024-05-09T04:48:28.556432+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:22.107953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552307+0000","last_change":"2024-05-09T04:47:48.415153+0000","last_active":"2024-05-09T04:48:28.552307+0000","last_peered":"2024-05-09T04:48:28.552307+0000","last_clean":"2024-05-09T04:48:28.552307+0000","last_became_active":"2024-05-09T04:47:48.414793+0000","last_became_peered":"2024-05-09T04:47:48.414793+0000","last_unstale":"2024-05-09T04:48:28.552307+0000","last_undegraded":"2024-05-09T04:48:28.552307+0000","last_fullsized":"2024-05-09T04:48:28.552307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:57.564091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.553235+0000","last_change":"2024-05-09T04:47:48.408646+0000","last_active":"2024-05-09T04:48:28.553235+0000","last_peered":"2024-05-09T04:48:28.553235+0000","last_clean":"2024-05-09T04:48:28.553235+0000","last_became_active":"2024-05-09T04:47:48.408250+0000","last_became_peered":"2024-05-09T04:47:48.408250+0000","last_unstale":"2024-05-09T04:48:28.553235+0000","last_undegraded":"2024-05-09T04:48:28.553235+0000","last_fullsized":"2024-05-09T04:48:28.553235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:42:50.622937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.558944+0000","last_change":"2024-05-09T04:47:48.408686+0000","last_active":"2024-05-09T04:48:28.558944+0000","last_peered":"2024-05-09T04:48:28.558944+0000","last_clean":"2024-05-09T04:48:28.558944+0000","last_became_active":"2024-05-09T04:47:48.408318+0000","last_became_peered":"2024-05-09T04:47:48.408318+0000","last_unstale":"2024-05-09T04:48:28.558944+0000","last_undegraded":"2024-05-09T04:48:28.558944+0000","last_fullsized":"2024-05-09T04:48:28.558944+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:51.240485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.662340+0000","last_change":"2024-05-09T04:47:48.421562+0000","last_active":"2024-05-09T04:48:28.662340+0000","last_peered":"2024-05-09T04:48:28.662340+0000","last_clean":"2024-05-09T04:48:28.662340+0000","last_became_active":"2024-05-09T04:47:48.421068+0000","last_became_peered":"2024-05-09T04:47:48.421068+0000","last_unstale":"2024-05-09T04:48:28.662340+0000","last_undegraded":"2024-05-09T04:48:28.662340+0000","last_fullsized":"2024-05-09T04:48:28.662340+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:18:41.455187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.557871+0000","last_change":"2024-05-09T04:47:48.413942+0000","last_active":"2024-05-09T04:48:28.557871+0000","last_peered":"2024-05-09T04:48:28.557871+0000","last_clean":"2024-05-09T04:48:28.557871+0000","last_became_active":"2024-05-09T04:47:48.413561+0000","last_became_peered":"2024-05-09T04:47:48.413561+0000","last_unstale":"2024-05-09T04:48:28.557871+0000","last_undegraded":"2024-05-09T04:48:28.557871+0000","last_fullsized":"2024-05-09T04:48:28.557871+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:33:48.148834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.661780+0000","last_change":"2024-05-09T04:47:48.408752+0000","last_active":"2024-05-09T04:48:28.661780+0000","last_peered":"2024-05-09T04:48:28.661780+0000","last_clean":"2024-05-09T04:48:28.661780+0000","last_became_active":"2024-05-09T04:47:48.408393+0000","last_became_peered":"2024-05-09T04:47:48.408393+0000","last_unstale":"2024-05-09T04:48:28.661780+0000","last_undegraded":"2024-05-09T04:48:28.661780+0000","last_fullsized":"2024-05-09T04:48:28.661780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:06.778002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.561280+0000","last_change":"2024-05-09T04:47:48.409303+0000","last_active":"2024-05-09T04:48:28.561280+0000","last_peered":"2024-05-09T04:48:28.561280+0000","last_clean":"2024-05-09T04:48:28.561280+0000","last_became_active":"2024-05-09T04:47:48.408971+0000","last_became_peered":"2024-05-09T04:47:48.408971+0000","last_unstale":"2024-05-09T04:48:28.561280+0000","last_undegraded":"2024-05-09T04:48:28.561280+0000","last_fullsized":"2024-05-09T04:48:28.561280+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:45.830647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552334+0000","last_change":"2024-05-09T04:47:48.404961+0000","last_active":"2024-05-09T04:48:28.552334+0000","last_peered":"2024-05-09T04:48:28.552334+0000","last_clean":"2024-05-09T04:48:28.552334+0000","last_became_active":"2024-05-09T04:47:48.404577+0000","last_became_peered":"2024-05-09T04:47:48.404577+0000","last_unstale":"2024-05-09T04:48:28.552334+0000","last_undegraded":"2024-05-09T04:48:28.552334+0000","last_fullsized":"2024-05-09T04:48:28.552334+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:40.528066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-05-09T04:48:28.552807+0000","last_change":"2024-05-09T04:47:48.424364+0000","last_active":"2024-05-09T04:48:28.552807+0000","last_peered":"2024-05-09T04:48:28.552807+0000","last_clean":"2024-05-09T04:48:28.552807+0000","last_became_active":"2024-05-09T04:47:48.423991+0000","last_became_peered":"2024-05-09T04:47:48.423991+0000","last_unstale":"2024-05-09T04:48:28.552807+0000","last_undegraded":"2024-05-09T04:48:28.552807+0000","last_fullsized":"2024-05-09T04:48:28.552807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:14.040560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249829,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64348,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":203,"num_read_kb":160,"num_write":164,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":9,"num_bytes_recovered":50695,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":193044,"data_compressed":8483,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645337,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":880640,"data_stored":431993,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841565,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":1084212,"data_compressed":41932,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005091,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":432229,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201321,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":897024,"data_stored":432149,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397550,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":1024079,"data_compressed":41932,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561075,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":884736,"data_stored":431618,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32100,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713052,"statfs":{"total":95995035648,"available":95962165248,"internally_reserved":0,"allocated":946176,"data_stored":491758,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738449,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":905216,"data_stored":432889,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":901120,"data_stored":432761,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117059,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36216,"kb_used_data":880,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708936,"statfs":{"total":95995035648,"available":95957950464,"internally_reserved":0,"allocated":901120,"data_stored":431929,"data_compressed":5779,"data_compressed_allocated":8192,"data_compressed_original":16384,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":921600,"data_stored":492793,"data_compressed":14262,"data_compressed_allocated":20480,"data_compressed_original":65536,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757305,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36280,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708872,"statfs":{"total":95995035648,"available":95957884928,"internally_reserved":0,"allocated":958464,"data_stored":1023372,"data_compressed":41932,"data_compressed_allocated":53248,"data_compressed_original":606208,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34532,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34532,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":61382,"data_compressed":8483,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:48:40.387 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T04:48:40.387 INFO:tasks.cephadm:Setting up client nodes... 2024-05-09T04:48:40.388 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-09T04:48:40.639 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:40.761 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:40 smithi046 ceph-mon[30136]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.4 KiB/s wr, 8 op/s; 4.6 KiB/s, 0 keys/s, 0 objects/s recovering 2024-05-09T04:48:40.761 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:40 smithi046 ceph-mon[30136]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:48:40.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:40 smithi072 ceph-mon[34088]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.4 KiB/s wr, 8 op/s; 4.6 KiB/s, 0 keys/s, 0 objects/s recovering 2024-05-09T04:48:40.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:40 smithi072 ceph-mon[34088]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:48:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:40 smithi110 ceph-mon[34904]: pgmap v318: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.4 KiB/s wr, 8 op/s; 4.6 KiB/s, 0 keys/s, 0 objects/s recovering 2024-05-09T04:48:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:40 smithi110 ceph-mon[34904]: from='client.14748 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:48:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.382+0000 7fae945ae700 1 -- 172.21.15.46:0/573463239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8c104d20 msgr2=0x7fae8c10b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.382+0000 7fae945ae700 1 --2- 172.21.15.46:0/573463239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8c104d20 0x7fae8c10b5d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fae8400b0a0 tx=0x7fae8400b3b0 comp rx=0 tx=0).stop 2024-05-09T04:48:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.382+0000 7fae945ae700 1 -- 172.21.15.46:0/573463239 shutdown_connections 2024-05-09T04:48:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.382+0000 7fae945ae700 1 --2- 172.21.15.46:0/573463239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8c104d20 0x7fae8c10b5d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.382+0000 7fae945ae700 1 --2- 172.21.15.46:0/573463239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae8c103250 0x7fae8c103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.382+0000 7fae945ae700 1 --2- 172.21.15.46:0/573463239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae8c104380 0x7fae8c104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.382+0000 7fae945ae700 1 -- 172.21.15.46:0/573463239 >> 172.21.15.46:0/573463239 conn(0x7fae8c0fe890 msgr2=0x7fae8c100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:42.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.382+0000 7fae945ae700 1 -- 172.21.15.46:0/573463239 shutdown_connections 2024-05-09T04:48:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.382+0000 7fae945ae700 1 -- 172.21.15.46:0/573463239 wait complete. 2024-05-09T04:48:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae945ae700 1 Processor -- start 2024-05-09T04:48:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae945ae700 1 -- start start 2024-05-09T04:48:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae945ae700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae8c103250 0x7fae8c199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae945ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8c104380 0x7fae8c199cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae945ae700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae8c104d20 0x7fae8c1a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae945ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c10ecb0 con 0x7fae8c104380 2024-05-09T04:48:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae945ae700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c10eb30 con 0x7fae8c103250 2024-05-09T04:48:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae945ae700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fae8c10ee30 con 0x7fae8c104d20 2024-05-09T04:48:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae91b49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8c104380 0x7fae8c199cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae91b49700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8c104380 0x7fae8c199cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47480/0 (socket says 172.21.15.46:47480) 2024-05-09T04:48:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae91b49700 1 -- 172.21.15.46:0/4209909121 learned_addr learned my addr 172.21.15.46:0/4209909121 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae9234a700 1 --2- 172.21.15.46:0/4209909121 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae8c103250 0x7fae8c199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae92b4b700 1 --2- 172.21.15.46:0/4209909121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae8c104d20 0x7fae8c1a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:42.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae91b49700 1 -- 172.21.15.46:0/4209909121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae8c104d20 msgr2=0x7fae8c1a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae91b49700 1 --2- 172.21.15.46:0/4209909121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae8c104d20 0x7fae8c1a0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae91b49700 1 -- 172.21.15.46:0/4209909121 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae8c103250 msgr2=0x7fae8c199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae91b49700 1 --2- 172.21.15.46:0/4209909121 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae8c103250 0x7fae8c199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae91b49700 1 -- 172.21.15.46:0/4209909121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae8c1a12f0 con 0x7fae8c104380 2024-05-09T04:48:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.383+0000 7fae92b4b700 1 --2- 172.21.15.46:0/4209909121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae8c104d20 0x7fae8c1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:48:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.384+0000 7fae91b49700 1 --2- 172.21.15.46:0/4209909121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8c104380 0x7fae8c199cd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009970 tx=0x7fae7c009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.384+0000 7fae837fe700 1 -- 172.21.15.46:0/4209909121 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c01c070 con 0x7fae8c104380 2024-05-09T04:48:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.384+0000 7fae945ae700 1 -- 172.21.15.46:0/4209909121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae8c075a50 con 0x7fae8c104380 2024-05-09T04:48:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.384+0000 7fae837fe700 1 -- 172.21.15.46:0/4209909121 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fae7c00d260 con 0x7fae8c104380 2024-05-09T04:48:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.384+0000 7fae837fe700 1 -- 172.21.15.46:0/4209909121 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae7c017a60 con 0x7fae8c104380 2024-05-09T04:48:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.384+0000 7fae945ae700 1 -- 172.21.15.46:0/4209909121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae8c075fe0 con 0x7fae8c104380 2024-05-09T04:48:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.385+0000 7fae945ae700 1 -- 172.21.15.46:0/4209909121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae8c0512c0 con 0x7fae8c104380 2024-05-09T04:48:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.386+0000 7fae837fe700 1 -- 172.21.15.46:0/4209909121 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fae7c01f070 con 0x7fae8c104380 2024-05-09T04:48:42.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.387+0000 7fae837fe700 1 --2- 172.21.15.46:0/4209909121 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fae780b2910 0x7fae780b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:42.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.388+0000 7fae9234a700 1 --2- 172.21.15.46:0/4209909121 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fae780b2910 0x7fae780b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:42.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.388+0000 7fae837fe700 1 -- 172.21.15.46:0/4209909121 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fae7c0e3970 con 0x7fae8c104380 2024-05-09T04:48:42.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.388+0000 7fae9234a700 1 --2- 172.21.15.46:0/4209909121 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fae780b2910 0x7fae780b4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fae8c104280 tx=0x7fae88009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:42.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.391+0000 7fae837fe700 1 -- 172.21.15.46:0/4209909121 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fae7c0ab4b0 con 0x7fae8c104380 2024-05-09T04:48:42.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.698+0000 7fae945ae700 1 -- 172.21.15.46:0/4209909121 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7fae8c109ef0 con 0x7fae8c104380 2024-05-09T04:48:42.698 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:42 smithi046 ceph-mon[30136]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T04:48:42.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.701+0000 7fae837fe700 1 -- 172.21.15.46:0/4209909121 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7fae7c0aadb0 con 0x7fae8c104380 2024-05-09T04:48:42.701 INFO:teuthology.orchestra.run.smithi046.stdout:[client.0] 2024-05-09T04:48:42.701 INFO:teuthology.orchestra.run.smithi046.stdout: key = AQCqVTxmlh6yKRAAPZAsG0Aa7Xi759nOqpes5Q== 2024-05-09T04:48:42.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.703+0000 7fae945ae700 1 -- 172.21.15.46:0/4209909121 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fae780b2910 msgr2=0x7fae780b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:42.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.703+0000 7fae945ae700 1 --2- 172.21.15.46:0/4209909121 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fae780b2910 0x7fae780b4e20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fae8c104280 tx=0x7fae88009260 comp rx=0 tx=0).stop 2024-05-09T04:48:42.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.703+0000 7fae945ae700 1 -- 172.21.15.46:0/4209909121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8c104380 msgr2=0x7fae8c199cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:42.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.703+0000 7fae945ae700 1 --2- 172.21.15.46:0/4209909121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8c104380 0x7fae8c199cd0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fae7c009970 tx=0x7fae7c009c80 comp rx=0 tx=0).stop 2024-05-09T04:48:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.703+0000 7fae945ae700 1 -- 172.21.15.46:0/4209909121 shutdown_connections 2024-05-09T04:48:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.703+0000 7fae945ae700 1 --2- 172.21.15.46:0/4209909121 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fae780b2910 0x7fae780b4e20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.703+0000 7fae945ae700 1 --2- 172.21.15.46:0/4209909121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae8c104d20 0x7fae8c1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.703+0000 7fae945ae700 1 --2- 172.21.15.46:0/4209909121 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae8c103250 0x7fae8c199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.703+0000 7fae945ae700 1 --2- 172.21.15.46:0/4209909121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae8c104380 0x7fae8c199cd0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.703+0000 7fae945ae700 1 -- 172.21.15.46:0/4209909121 >> 172.21.15.46:0/4209909121 conn(0x7fae8c0fe890 msgr2=0x7fae8c1075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.703+0000 7fae945ae700 1 -- 172.21.15.46:0/4209909121 shutdown_connections 2024-05-09T04:48:42.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:42.704+0000 7fae945ae700 1 -- 172.21.15.46:0/4209909121 wait complete. 2024-05-09T04:48:42.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:42 smithi072 ceph-mon[34088]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T04:48:43.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:42 smithi110 ceph-mon[34904]: pgmap v319: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.1 KiB/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T04:48:43.380 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:48:43.380 DEBUG:teuthology.orchestra.run.smithi046:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-09T04:48:43.380 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-09T04:48:43.419 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-09T04:48:43.419 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-09T04:48:43.419 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-09T04:48:43.712 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:43.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:43 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/4209909121' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T04:48:43.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:43 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/4209909121' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T04:48:43.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:43 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/4209909121' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T04:48:43.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:43 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/4209909121' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T04:48:44.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:43 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/4209909121' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-09T04:48:44.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:43 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/4209909121' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-09T04:48:45.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:44 smithi110 ceph-mon[34904]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T04:48:45.056 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:44 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:48:45.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:44 smithi046 ceph-mon[30136]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T04:48:45.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:44 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:48:45.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:44 smithi072 ceph-mon[34088]: pgmap v320: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.1 KiB/s wr, 6 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-09T04:48:45.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:44 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:48:45.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.495+0000 7f537b2f9700 1 -- 172.21.15.46:0/3090644893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53741045e0 msgr2=0x7f53741049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.495+0000 7f537b2f9700 1 --2- 172.21.15.46:0/3090644893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53741045e0 0x7f53741049f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f53700099c0 tx=0x7f5370009cd0 comp rx=0 tx=0).stop 2024-05-09T04:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.495+0000 7f537b2f9700 1 -- 172.21.15.46:0/3090644893 shutdown_connections 2024-05-09T04:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.495+0000 7f537b2f9700 1 --2- 172.21.15.46:0/3090644893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5374109860 0x7f53741111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.495+0000 7f537b2f9700 1 --2- 172.21.15.46:0/3090644893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f53741045e0 0x7f53741049f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.495+0000 7f537b2f9700 1 --2- 172.21.15.46:0/3090644893 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5374108f10 0x7f5374109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.495+0000 7f537b2f9700 1 -- 172.21.15.46:0/3090644893 >> 172.21.15.46:0/3090644893 conn(0x7f53740ffe70 msgr2=0x7f53741022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.495+0000 7f537b2f9700 1 -- 172.21.15.46:0/3090644893 shutdown_connections 2024-05-09T04:48:45.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.495+0000 7f537b2f9700 1 -- 172.21.15.46:0/3090644893 wait complete. 2024-05-09T04:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.496+0000 7f537b2f9700 1 Processor -- start 2024-05-09T04:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.496+0000 7f537b2f9700 1 -- start start 2024-05-09T04:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.496+0000 7f537b2f9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53741045e0 0x7f5374075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.496+0000 7f537b2f9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5374108f10 0x7f5374076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.496+0000 7f537b2f9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5374109860 0x7f5374076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.496+0000 7f537b2f9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5374114490 con 0x7f5374109860 2024-05-09T04:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.496+0000 7f537b2f9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5374114310 con 0x7f53741045e0 2024-05-09T04:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.496+0000 7f537b2f9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5374114610 con 0x7f5374108f10 2024-05-09T04:48:45.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f5379896700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5374109860 0x7f5374076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f5379896700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5374109860 0x7f5374076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35404/0 (socket says 172.21.15.46:35404) 2024-05-09T04:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f5378894700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5374108f10 0x7f5374076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f5379896700 1 -- 172.21.15.46:0/3933372198 learned_addr learned my addr 172.21.15.46:0/3933372198 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f5379095700 1 --2- 172.21.15.46:0/3933372198 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53741045e0 0x7f5374075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f5379896700 1 -- 172.21.15.46:0/3933372198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5374108f10 msgr2=0x7f5374076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f5379896700 1 --2- 172.21.15.46:0/3933372198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5374108f10 0x7f5374076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f5379896700 1 -- 172.21.15.46:0/3933372198 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53741045e0 msgr2=0x7f5374075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f5379896700 1 --2- 172.21.15.46:0/3933372198 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53741045e0 0x7f5374075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f5379896700 1 -- 172.21.15.46:0/3933372198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53741a91a0 con 0x7f5374109860 2024-05-09T04:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f5378894700 1 --2- 172.21.15.46:0/3933372198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5374108f10 0x7f5374076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f5379896700 1 --2- 172.21.15.46:0/3933372198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5374109860 0x7f5374076540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f536c00ef30 tx=0x7f536c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f536a7fc700 1 -- 172.21.15.46:0/3933372198 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c0092e0 con 0x7f5374109860 2024-05-09T04:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f536a7fc700 1 -- 172.21.15.46:0/3933372198 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536c004bb0 con 0x7f5374109860 2024-05-09T04:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f537b2f9700 1 -- 172.21.15.46:0/3933372198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53741a9430 con 0x7f5374109860 2024-05-09T04:48:45.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f536a7fc700 1 -- 172.21.15.46:0/3933372198 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536c004020 con 0x7f5374109860 2024-05-09T04:48:45.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.497+0000 7f537b2f9700 1 -- 172.21.15.46:0/3933372198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53741a9920 con 0x7f5374109860 2024-05-09T04:48:45.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.498+0000 7f537b2f9700 1 -- 172.21.15.46:0/3933372198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53740512c0 con 0x7f5374109860 2024-05-09T04:48:45.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.500+0000 7f536a7fc700 1 -- 172.21.15.46:0/3933372198 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f536c01b030 con 0x7f5374109860 2024-05-09T04:48:45.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.501+0000 7f536a7fc700 1 --2- 172.21.15.46:0/3933372198 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f53600b2910 0x7f53600b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:45.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.501+0000 7f5379095700 1 --2- 172.21.15.46:0/3933372198 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f53600b2910 0x7f53600b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:45.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.501+0000 7f536a7fc700 1 -- 172.21.15.46:0/3933372198 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f536c0e2830 con 0x7f5374109860 2024-05-09T04:48:45.501 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.501+0000 7f5379095700 1 --2- 172.21.15.46:0/3933372198 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f53600b2910 0x7f53600b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5370000c00 tx=0x7f537001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:45.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.504+0000 7f536a7fc700 1 -- 172.21.15.46:0/3933372198 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f536c022020 con 0x7f5374109860 2024-05-09T04:48:45.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.784+0000 7f537b2f9700 1 -- 172.21.15.46:0/3933372198 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f537402e7b0 con 0x7f5374109860 2024-05-09T04:48:45.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.789+0000 7f536a7fc700 1 -- 172.21.15.46:0/3933372198 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288646 (secure 0 0 0) 0x7f536c010040 con 0x7f5374109860 2024-05-09T04:48:45.793 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:48:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.794+0000 7f537b2f9700 1 -- 172.21.15.46:0/3933372198 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f53600b2910 msgr2=0x7f53600b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.794+0000 7f537b2f9700 1 --2- 172.21.15.46:0/3933372198 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f53600b2910 0x7f53600b4e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5370000c00 tx=0x7f537001a040 comp rx=0 tx=0).stop 2024-05-09T04:48:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.794+0000 7f537b2f9700 1 -- 172.21.15.46:0/3933372198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5374109860 msgr2=0x7f5374076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.794+0000 7f537b2f9700 1 --2- 172.21.15.46:0/3933372198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5374109860 0x7f5374076540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f536c00ef30 tx=0x7f536c00c3a0 comp rx=0 tx=0).stop 2024-05-09T04:48:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.794+0000 7f537b2f9700 1 -- 172.21.15.46:0/3933372198 shutdown_connections 2024-05-09T04:48:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.794+0000 7f537b2f9700 1 --2- 172.21.15.46:0/3933372198 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f53600b2910 0x7f53600b4e20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.794+0000 7f537b2f9700 1 --2- 172.21.15.46:0/3933372198 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5374109860 0x7f5374076540 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.794+0000 7f537b2f9700 1 --2- 172.21.15.46:0/3933372198 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f53741045e0 0x7f5374075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.794+0000 7f537b2f9700 1 --2- 172.21.15.46:0/3933372198 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5374108f10 0x7f5374076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.794+0000 7f537b2f9700 1 -- 172.21.15.46:0/3933372198 >> 172.21.15.46:0/3933372198 conn(0x7f53740ffe70 msgr2=0x7f5374107190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.794+0000 7f537b2f9700 1 -- 172.21.15.46:0/3933372198 shutdown_connections 2024-05-09T04:48:45.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:45.794+0000 7f537b2f9700 1 -- 172.21.15.46:0/3933372198 wait complete. 2024-05-09T04:48:46.573 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6800","nonce":3691531627},{"type":"v1","addr":"172.21.15.46:6801","nonce":3691531627}]},"active_addr":"172.21.15.46:6801/3691531627","active_change":"2024-05-09T04:40:27.201246+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24116,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.46:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":181034377}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":1963687293}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":2215389074}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":1833069872}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.46:0","nonce":1695100894}]}]} 2024-05-09T04:48:46.579 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-09T04:48:46.579 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-09T04:48:46.579 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:48:46.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:46 smithi046 ceph-mon[30136]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-09T04:48:46.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:46 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3933372198' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T04:48:46.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:46 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:46.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:46 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:46.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:46 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:46.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:46 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:46 smithi110 ceph-mon[34904]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-09T04:48:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:46 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3933372198' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T04:48:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:46 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:46 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:46 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:46 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:46 smithi072 ceph-mon[34088]: pgmap v321: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-05-09T04:48:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:46 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3933372198' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-09T04:48:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:46 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:46 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:46 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:46 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:47.175 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.900+0000 7f86603de700 1 -- 172.21.15.46:0/371906986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86581034c0 msgr2=0x7f86581038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.900+0000 7f86603de700 1 --2- 172.21.15.46:0/371906986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86581034c0 0x7f86581038d0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f86540099c0 tx=0x7f8654009cd0 comp rx=0 tx=0).stop 2024-05-09T04:48:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.900+0000 7f86603de700 1 -- 172.21.15.46:0/371906986 shutdown_connections 2024-05-09T04:48:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.900+0000 7f86603de700 1 --2- 172.21.15.46:0/371906986 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8658108740 0x7f86581100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.900+0000 7f86603de700 1 --2- 172.21.15.46:0/371906986 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f86581034c0 0x7f86581038d0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.900+0000 7f86603de700 1 --2- 172.21.15.46:0/371906986 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8658107df0 0x7f8658108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.900+0000 7f86603de700 1 -- 172.21.15.46:0/371906986 >> 172.21.15.46:0/371906986 conn(0x7f86580fedc0 msgr2=0x7f86581011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:48.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.901+0000 7f86603de700 1 -- 172.21.15.46:0/371906986 shutdown_connections 2024-05-09T04:48:48.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.901+0000 7f86603de700 1 -- 172.21.15.46:0/371906986 wait complete. 2024-05-09T04:48:48.903 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:48 smithi046 ceph-mon[30136]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-09T04:48:48.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.901+0000 7f86603de700 1 Processor -- start 2024-05-09T04:48:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.901+0000 7f86603de700 1 -- start start 2024-05-09T04:48:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f86603de700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86581034c0 0x7f865819bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f86603de700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658107df0 0x7f865819c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f86603de700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8658108740 0x7f865819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f86603de700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8658113460 con 0x7f8658107df0 2024-05-09T04:48:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f86603de700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f86581132e0 con 0x7f8658108740 2024-05-09T04:48:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f86603de700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f86581135e0 con 0x7f86581034c0 2024-05-09T04:48:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f865d979700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658107df0 0x7f865819c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f865d979700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658107df0 0x7f865819c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35426/0 (socket says 172.21.15.46:35426) 2024-05-09T04:48:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f865d979700 1 -- 172.21.15.46:0/2539755493 learned_addr learned my addr 172.21.15.46:0/2539755493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f865e97b700 1 --2- 172.21.15.46:0/2539755493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8658108740 0x7f865819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:48.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f865e17a700 1 --2- 172.21.15.46:0/2539755493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86581034c0 0x7f865819bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f865d979700 1 -- 172.21.15.46:0/2539755493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86581034c0 msgr2=0x7f865819bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f865d979700 1 --2- 172.21.15.46:0/2539755493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86581034c0 0x7f865819bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f865d979700 1 -- 172.21.15.46:0/2539755493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8658108740 msgr2=0x7f865819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f865d979700 1 --2- 172.21.15.46:0/2539755493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8658108740 0x7f865819c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f865d979700 1 -- 172.21.15.46:0/2539755493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86581a7050 con 0x7f8658107df0 2024-05-09T04:48:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f865e97b700 1 --2- 172.21.15.46:0/2539755493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8658108740 0x7f865819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:48:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.902+0000 7f865d979700 1 --2- 172.21.15.46:0/2539755493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658107df0 0x7f865819c240 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f864800b500 tx=0x7f864800b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.903+0000 7f864f7fe700 1 -- 172.21.15.46:0/2539755493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648015020 con 0x7f8658107df0 2024-05-09T04:48:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.903+0000 7f86603de700 1 -- 172.21.15.46:0/2539755493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86581a7340 con 0x7f8658107df0 2024-05-09T04:48:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.903+0000 7f864f7fe700 1 -- 172.21.15.46:0/2539755493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8648011ce0 con 0x7f8658107df0 2024-05-09T04:48:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.903+0000 7f864f7fe700 1 -- 172.21.15.46:0/2539755493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8648010d40 con 0x7f8658107df0 2024-05-09T04:48:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.903+0000 7f86603de700 1 -- 172.21.15.46:0/2539755493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86581a78d0 con 0x7f8658107df0 2024-05-09T04:48:48.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.904+0000 7f86603de700 1 -- 172.21.15.46:0/2539755493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86580512c0 con 0x7f8658107df0 2024-05-09T04:48:48.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.906+0000 7f864f7fe700 1 -- 172.21.15.46:0/2539755493 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8648018020 con 0x7f8658107df0 2024-05-09T04:48:48.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.906+0000 7f864f7fe700 1 --2- 172.21.15.46:0/2539755493 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f86440b2910 0x7f86440b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:48.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.906+0000 7f865e17a700 1 --2- 172.21.15.46:0/2539755493 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f86440b2910 0x7f86440b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:48.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.906+0000 7f864f7fe700 1 -- 172.21.15.46:0/2539755493 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f86480e1fc0 con 0x7f8658107df0 2024-05-09T04:48:48.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.907+0000 7f865e17a700 1 --2- 172.21.15.46:0/2539755493 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f86440b2910 0x7f86440b4e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8654003800 tx=0x7f865401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:48.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:48.910+0000 7f864f7fe700 1 -- 172.21.15.46:0/2539755493 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f864801f020 con 0x7f8658107df0 2024-05-09T04:48:49.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:48 smithi110 ceph-mon[34904]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-09T04:48:49.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:48 smithi072 ceph-mon[34088]: pgmap v322: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-09T04:48:49.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:49.139+0000 7f86603de700 1 -- 172.21.15.46:0/2539755493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f865802e5f0 con 0x7f8658107df0 2024-05-09T04:48:49.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:49.141+0000 7f864f7fe700 1 -- 172.21.15.46:0/2539755493 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22637 (secure 0 0 0) 0x7f86480a9d10 con 0x7f8658107df0 2024-05-09T04:48:49.142 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:48:49.142 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"fsid":"ecf9ae2c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:25.633876+0000","modified":"2024-05-09T04:48:28.548796+0000","last_up_change":"2024-05-09T04:46:46.418597+0000","last_in_change":"2024-05-09T04:46:25.903582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:42:53.301400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:47:43.413569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:47:47.264915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d79cd3e7-6f82-4d60-96d9-26d8146c962f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6803","nonce":1576436508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6805","nonce":1576436508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6809","nonce":1576436508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6807","nonce":1576436508}]},"public_addr":"172.21.15.46:6803/1576436508","cluster_addr":"172.21.15.46:6805/1576436508","heartbeat_back_addr":"172.21.15.46:6809/1576436508","heartbeat_front_addr":"172.21.15.46:6807/1576436508","state":["exists","up"]},{"osd":1,"uuid":"698eba80-1ebd-4076-b6f9-953d2e7daf76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6801","nonce":379039775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6803","nonce":379039775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6807","nonce":379039775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6805","nonce":379039775}]},"public_addr":"172.21.15.72:6801/379039775","cluster_addr":"172.21.15.72:6803/379039775","heartbeat_back_addr":"172.21.15.72:6807/379039775","heartbeat_front_addr":"172.21.15.72:6805/379039775","state":["exists","up"]},{"osd":2,"uuid":"c549b131-cfa7-4724-926f-83e91a752d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6801","nonce":293447227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6803","nonce":293447227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6807","nonce":293447227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6805","nonce":293447227}]},"public_addr":"172.21.15.110:6801/293447227","cluster_addr":"172.21.15.110:6803/293447227","heartbeat_back_addr":"172.21.15.110:6807/293447227","heartbeat_front_addr":"172.21.15.110:6805/293447227","state":["exists","up"]},{"osd":3,"uuid":"549bb4f7-54d9-4cbc-ad40-7f3e717964f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6811","nonce":1115835653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6813","nonce":1115835653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6817","nonce":1115835653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6815","nonce":1115835653}]},"public_addr":"172.21.15.46:6811/1115835653","cluster_addr":"172.21.15.46:6813/1115835653","heartbeat_back_addr":"172.21.15.46:6817/1115835653","heartbeat_front_addr":"172.21.15.46:6815/1115835653","state":["exists","up"]},{"osd":4,"uuid":"444cd36b-6034-46cd-9a6b-29dce4ba6405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6809","nonce":59186693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6811","nonce":59186693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6815","nonce":59186693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6813","nonce":59186693}]},"public_addr":"172.21.15.72:6809/59186693","cluster_addr":"172.21.15.72:6811/59186693","heartbeat_back_addr":"172.21.15.72:6815/59186693","heartbeat_front_addr":"172.21.15.72:6813/59186693","state":["exists","up"]},{"osd":5,"uuid":"2c2bdd2f-43cf-4208-9ae1-f912807c84e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6809","nonce":1712162502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6811","nonce":1712162502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6815","nonce":1712162502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6813","nonce":1712162502}]},"public_addr":"172.21.15.110:6809/1712162502","cluster_addr":"172.21.15.110:6811/1712162502","heartbeat_back_addr":"172.21.15.110:6815/1712162502","heartbeat_front_addr":"172.21.15.110:6813/1712162502","state":["exists","up"]},{"osd":6,"uuid":"beb0b87f-c3a4-40dd-b224-1843e9301ca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6819","nonce":2429650065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6821","nonce":2429650065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6825","nonce":2429650065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6823","nonce":2429650065}]},"public_addr":"172.21.15.46:6819/2429650065","cluster_addr":"172.21.15.46:6821/2429650065","heartbeat_back_addr":"172.21.15.46:6825/2429650065","heartbeat_front_addr":"172.21.15.46:6823/2429650065","state":["exists","up"]},{"osd":7,"uuid":"4a634675-fad7-4a43-840c-8dec2631dec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6817","nonce":337796672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6819","nonce":337796672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6823","nonce":337796672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6821","nonce":337796672}]},"public_addr":"172.21.15.72:6817/337796672","cluster_addr":"172.21.15.72:6819/337796672","heartbeat_back_addr":"172.21.15.72:6823/337796672","heartbeat_front_addr":"172.21.15.72:6821/337796672","state":["exists","up"]},{"osd":8,"uuid":"01b5f638-90f0-4cf4-8042-20d20a725669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6817","nonce":189513203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6819","nonce":189513203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6823","nonce":189513203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6821","nonce":189513203}]},"public_addr":"172.21.15.110:6817/189513203","cluster_addr":"172.21.15.110:6819/189513203","heartbeat_back_addr":"172.21.15.110:6823/189513203","heartbeat_front_addr":"172.21.15.110:6821/189513203","state":["exists","up"]},{"osd":9,"uuid":"e423de41-50ec-48f4-a4ed-216de33c4fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6827","nonce":1762262822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6829","nonce":1762262822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6833","nonce":1762262822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6831","nonce":1762262822}]},"public_addr":"172.21.15.46:6827/1762262822","cluster_addr":"172.21.15.46:6829/1762262822","heartbeat_back_addr":"172.21.15.46:6833/1762262822","heartbeat_front_addr":"172.21.15.46:6831/1762262822","state":["exists","up"]},{"osd":10,"uuid":"f594d004-9f80-4508-b127-fb693d355f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6825","nonce":38453898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6827","nonce":38453898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6831","nonce":38453898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6829","nonce":38453898}]},"public_addr":"172.21.15.72:6825/38453898","cluster_addr":"172.21.15.72:6827/38453898","heartbeat_back_addr":"172.21.15.72:6831/38453898","heartbeat_front_addr":"172.21.15.72:6829/38453898","state":["exists","up"]},{"osd":11,"uuid":"f771d29f-89e3-4f96-9c29-00e424230c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6825","nonce":4153529632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6827","nonce":4153529632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6831","nonce":4153529632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6829","nonce":4153529632}]},"public_addr":"172.21.15.110:6825/4153529632","cluster_addr":"172.21.15.110:6827/4153529632","heartbeat_back_addr":"172.21.15.110:6831/4153529632","heartbeat_front_addr":"172.21.15.110:6829/4153529632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:02.615801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:26.453477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:50.334627+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:15.357320+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:40.403549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:06.030093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:31.904091+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:57.616372+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:24.045975+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:51.160898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:17.597845+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:44.636629+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1596672353":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1846482551":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/1069571878":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6800/2738158981":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/3625745146":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6801/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/564539954":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1797199863":"2024-05-10T04:40:27.201119+0000","172.21.15.46:6801/2738158981":"2024-05-10T04:39:59.515628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:48:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:49.143+0000 7f86603de700 1 -- 172.21.15.46:0/2539755493 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f86440b2910 msgr2=0x7f86440b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:49.143+0000 7f86603de700 1 --2- 172.21.15.46:0/2539755493 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f86440b2910 0x7f86440b4e20 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8654003800 tx=0x7f865401a040 comp rx=0 tx=0).stop 2024-05-09T04:48:49.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:49.143+0000 7f86603de700 1 -- 172.21.15.46:0/2539755493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658107df0 msgr2=0x7f865819c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:49.143+0000 7f86603de700 1 --2- 172.21.15.46:0/2539755493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658107df0 0x7f865819c240 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f864800b500 tx=0x7f864800b8c0 comp rx=0 tx=0).stop 2024-05-09T04:48:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:49.144+0000 7f86603de700 1 -- 172.21.15.46:0/2539755493 shutdown_connections 2024-05-09T04:48:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:49.144+0000 7f86603de700 1 --2- 172.21.15.46:0/2539755493 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f86440b2910 0x7f86440b4e20 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:49.144+0000 7f86603de700 1 --2- 172.21.15.46:0/2539755493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8658108740 0x7f865819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:49.144+0000 7f86603de700 1 --2- 172.21.15.46:0/2539755493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f86581034c0 0x7f865819bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:49.144+0000 7f86603de700 1 --2- 172.21.15.46:0/2539755493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8658107df0 0x7f865819c240 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:49.144+0000 7f86603de700 1 -- 172.21.15.46:0/2539755493 >> 172.21.15.46:0/2539755493 conn(0x7f86580fedc0 msgr2=0x7f86581096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:49.144+0000 7f86603de700 1 -- 172.21.15.46:0/2539755493 shutdown_connections 2024-05-09T04:48:49.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:49.144+0000 7f86603de700 1 -- 172.21.15.46:0/2539755493 wait complete. 2024-05-09T04:48:49.863 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-09T04:48:49.863 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:48:50.134 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:50.257 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:50.257 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:50.257 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:50 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2539755493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:48:50.257 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:50 smithi046 ceph-mon[30136]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T04:48:50.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:50.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:50.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:50 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2539755493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:48:50.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:50 smithi110 ceph-mon[34904]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T04:48:50.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:50.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:50.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:50 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2539755493' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:48:50.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:50 smithi072 ceph-mon[34088]: pgmap v323: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T04:48:51.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:51 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:51.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:51 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:48:51.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:51 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:51.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:51 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:51.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:51 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:48:51.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:51 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:51.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:51 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:48:51.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:51 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:48:51.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:51 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:48:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.910+0000 7f5b48d50700 1 -- 172.21.15.46:0/2974047788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b441045e0 msgr2=0x7f5b441049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.910+0000 7f5b48d50700 1 --2- 172.21.15.46:0/2974047788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b441045e0 0x7f5b441049f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f5b340099c0 tx=0x7f5b34009cd0 comp rx=0 tx=0).stop 2024-05-09T04:48:51.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.910+0000 7f5b48d50700 1 -- 172.21.15.46:0/2974047788 shutdown_connections 2024-05-09T04:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.910+0000 7f5b48d50700 1 --2- 172.21.15.46:0/2974047788 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b44109860 0x7f5b441111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.910+0000 7f5b48d50700 1 --2- 172.21.15.46:0/2974047788 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b441045e0 0x7f5b441049f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.910+0000 7f5b48d50700 1 --2- 172.21.15.46:0/2974047788 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b44108f10 0x7f5b44109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.910+0000 7f5b48d50700 1 -- 172.21.15.46:0/2974047788 >> 172.21.15.46:0/2974047788 conn(0x7f5b440ffef0 msgr2=0x7f5b44102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.910+0000 7f5b48d50700 1 -- 172.21.15.46:0/2974047788 shutdown_connections 2024-05-09T04:48:51.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.910+0000 7f5b48d50700 1 -- 172.21.15.46:0/2974047788 wait complete. 2024-05-09T04:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.911+0000 7f5b48d50700 1 Processor -- start 2024-05-09T04:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.911+0000 7f5b48d50700 1 -- start start 2024-05-09T04:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.911+0000 7f5b48d50700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b441045e0 0x7f5b4419aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b48d50700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b44108f10 0x7f5b4419b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b48d50700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b44109860 0x7f5b4419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b48d50700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5b44114490 con 0x7f5b44108f10 2024-05-09T04:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b48d50700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5b44114310 con 0x7f5b44109860 2024-05-09T04:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b48d50700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5b44114610 con 0x7f5b441045e0 2024-05-09T04:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b41d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b44108f10 0x7f5b4419b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:51.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b41d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b44108f10 0x7f5b4419b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35446/0 (socket says 172.21.15.46:35446) 2024-05-09T04:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b41d9b700 1 -- 172.21.15.46:0/3738953808 learned_addr learned my addr 172.21.15.46:0/3738953808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b4259c700 1 --2- 172.21.15.46:0/3738953808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b441045e0 0x7f5b4419aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b42d9d700 1 --2- 172.21.15.46:0/3738953808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b44109860 0x7f5b4419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b41d9b700 1 -- 172.21.15.46:0/3738953808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b441045e0 msgr2=0x7f5b4419aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b41d9b700 1 --2- 172.21.15.46:0/3738953808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b441045e0 0x7f5b4419aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b41d9b700 1 -- 172.21.15.46:0/3738953808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b44109860 msgr2=0x7f5b4419f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b41d9b700 1 --2- 172.21.15.46:0/3738953808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b44109860 0x7f5b4419f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b41d9b700 1 -- 172.21.15.46:0/3738953808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b4419fd20 con 0x7f5b44108f10 2024-05-09T04:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b41d9b700 1 --2- 172.21.15.46:0/3738953808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b44108f10 0x7f5b4419b440 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800d980 tx=0x7f5b3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.912+0000 7f5b337fe700 1 -- 172.21.15.46:0/3738953808 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b38017070 con 0x7f5b44108f10 2024-05-09T04:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.913+0000 7f5b337fe700 1 -- 172.21.15.46:0/3738953808 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b3800fd40 con 0x7f5b44108f10 2024-05-09T04:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.913+0000 7f5b48d50700 1 -- 172.21.15.46:0/3738953808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b441a0010 con 0x7f5b44108f10 2024-05-09T04:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.913+0000 7f5b337fe700 1 -- 172.21.15.46:0/3738953808 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b3800e3b0 con 0x7f5b44108f10 2024-05-09T04:48:51.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.913+0000 7f5b48d50700 1 -- 172.21.15.46:0/3738953808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b441a7930 con 0x7f5b44108f10 2024-05-09T04:48:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.914+0000 7f5b48d50700 1 -- 172.21.15.46:0/3738953808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b440512c0 con 0x7f5b44108f10 2024-05-09T04:48:51.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.915+0000 7f5b337fe700 1 -- 172.21.15.46:0/3738953808 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5b38010610 con 0x7f5b44108f10 2024-05-09T04:48:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.916+0000 7f5b337fe700 1 --2- 172.21.15.46:0/3738953808 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5b2c0b2990 0x7f5b2c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:48:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.916+0000 7f5b4259c700 1 --2- 172.21.15.46:0/3738953808 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5b2c0b2990 0x7f5b2c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:48:51.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.917+0000 7f5b337fe700 1 -- 172.21.15.46:0/3738953808 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f5b380e1e30 con 0x7f5b44108f10 2024-05-09T04:48:51.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.917+0000 7f5b4259c700 1 --2- 172.21.15.46:0/3738953808 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5b2c0b2990 0x7f5b2c0b4ea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009990 tx=0x7f5b340036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:48:51.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:51.921+0000 7f5b337fe700 1 -- 172.21.15.46:0/3738953808 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b380a9970 con 0x7f5b44108f10 2024-05-09T04:48:52.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:52.138+0000 7f5b48d50700 1 -- 172.21.15.46:0/3738953808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5b4402e5f0 con 0x7f5b44108f10 2024-05-09T04:48:52.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:52.139+0000 7f5b337fe700 1 -- 172.21.15.46:0/3738953808 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) ==== 74+0+22637 (secure 0 0 0) 0x7f5b380a9270 con 0x7f5b44108f10 2024-05-09T04:48:52.140 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:48:52.141 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":82,"fsid":"ecf9ae2c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:25.633876+0000","modified":"2024-05-09T04:48:28.548796+0000","last_up_change":"2024-05-09T04:46:46.418597+0000","last_in_change":"2024-05-09T04:46:25.903582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:42:53.301400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:47:43.413569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:47:47.264915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d79cd3e7-6f82-4d60-96d9-26d8146c962f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6803","nonce":1576436508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6805","nonce":1576436508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6809","nonce":1576436508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6807","nonce":1576436508}]},"public_addr":"172.21.15.46:6803/1576436508","cluster_addr":"172.21.15.46:6805/1576436508","heartbeat_back_addr":"172.21.15.46:6809/1576436508","heartbeat_front_addr":"172.21.15.46:6807/1576436508","state":["exists","up"]},{"osd":1,"uuid":"698eba80-1ebd-4076-b6f9-953d2e7daf76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6801","nonce":379039775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6803","nonce":379039775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6807","nonce":379039775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6805","nonce":379039775}]},"public_addr":"172.21.15.72:6801/379039775","cluster_addr":"172.21.15.72:6803/379039775","heartbeat_back_addr":"172.21.15.72:6807/379039775","heartbeat_front_addr":"172.21.15.72:6805/379039775","state":["exists","up"]},{"osd":2,"uuid":"c549b131-cfa7-4724-926f-83e91a752d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6801","nonce":293447227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6803","nonce":293447227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6807","nonce":293447227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6805","nonce":293447227}]},"public_addr":"172.21.15.110:6801/293447227","cluster_addr":"172.21.15.110:6803/293447227","heartbeat_back_addr":"172.21.15.110:6807/293447227","heartbeat_front_addr":"172.21.15.110:6805/293447227","state":["exists","up"]},{"osd":3,"uuid":"549bb4f7-54d9-4cbc-ad40-7f3e717964f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6811","nonce":1115835653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6813","nonce":1115835653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6817","nonce":1115835653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6815","nonce":1115835653}]},"public_addr":"172.21.15.46:6811/1115835653","cluster_addr":"172.21.15.46:6813/1115835653","heartbeat_back_addr":"172.21.15.46:6817/1115835653","heartbeat_front_addr":"172.21.15.46:6815/1115835653","state":["exists","up"]},{"osd":4,"uuid":"444cd36b-6034-46cd-9a6b-29dce4ba6405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6809","nonce":59186693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6811","nonce":59186693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6815","nonce":59186693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6813","nonce":59186693}]},"public_addr":"172.21.15.72:6809/59186693","cluster_addr":"172.21.15.72:6811/59186693","heartbeat_back_addr":"172.21.15.72:6815/59186693","heartbeat_front_addr":"172.21.15.72:6813/59186693","state":["exists","up"]},{"osd":5,"uuid":"2c2bdd2f-43cf-4208-9ae1-f912807c84e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6809","nonce":1712162502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6811","nonce":1712162502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6815","nonce":1712162502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6813","nonce":1712162502}]},"public_addr":"172.21.15.110:6809/1712162502","cluster_addr":"172.21.15.110:6811/1712162502","heartbeat_back_addr":"172.21.15.110:6815/1712162502","heartbeat_front_addr":"172.21.15.110:6813/1712162502","state":["exists","up"]},{"osd":6,"uuid":"beb0b87f-c3a4-40dd-b224-1843e9301ca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6819","nonce":2429650065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6821","nonce":2429650065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6825","nonce":2429650065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6823","nonce":2429650065}]},"public_addr":"172.21.15.46:6819/2429650065","cluster_addr":"172.21.15.46:6821/2429650065","heartbeat_back_addr":"172.21.15.46:6825/2429650065","heartbeat_front_addr":"172.21.15.46:6823/2429650065","state":["exists","up"]},{"osd":7,"uuid":"4a634675-fad7-4a43-840c-8dec2631dec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6817","nonce":337796672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6819","nonce":337796672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6823","nonce":337796672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6821","nonce":337796672}]},"public_addr":"172.21.15.72:6817/337796672","cluster_addr":"172.21.15.72:6819/337796672","heartbeat_back_addr":"172.21.15.72:6823/337796672","heartbeat_front_addr":"172.21.15.72:6821/337796672","state":["exists","up"]},{"osd":8,"uuid":"01b5f638-90f0-4cf4-8042-20d20a725669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6817","nonce":189513203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6819","nonce":189513203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6823","nonce":189513203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6821","nonce":189513203}]},"public_addr":"172.21.15.110:6817/189513203","cluster_addr":"172.21.15.110:6819/189513203","heartbeat_back_addr":"172.21.15.110:6823/189513203","heartbeat_front_addr":"172.21.15.110:6821/189513203","state":["exists","up"]},{"osd":9,"uuid":"e423de41-50ec-48f4-a4ed-216de33c4fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6827","nonce":1762262822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6829","nonce":1762262822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6833","nonce":1762262822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6831","nonce":1762262822}]},"public_addr":"172.21.15.46:6827/1762262822","cluster_addr":"172.21.15.46:6829/1762262822","heartbeat_back_addr":"172.21.15.46:6833/1762262822","heartbeat_front_addr":"172.21.15.46:6831/1762262822","state":["exists","up"]},{"osd":10,"uuid":"f594d004-9f80-4508-b127-fb693d355f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6825","nonce":38453898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6827","nonce":38453898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6831","nonce":38453898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6829","nonce":38453898}]},"public_addr":"172.21.15.72:6825/38453898","cluster_addr":"172.21.15.72:6827/38453898","heartbeat_back_addr":"172.21.15.72:6831/38453898","heartbeat_front_addr":"172.21.15.72:6829/38453898","state":["exists","up"]},{"osd":11,"uuid":"f771d29f-89e3-4f96-9c29-00e424230c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6825","nonce":4153529632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6827","nonce":4153529632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6831","nonce":4153529632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6829","nonce":4153529632}]},"public_addr":"172.21.15.110:6825/4153529632","cluster_addr":"172.21.15.110:6827/4153529632","heartbeat_back_addr":"172.21.15.110:6831/4153529632","heartbeat_front_addr":"172.21.15.110:6829/4153529632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:02.615801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:26.453477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:50.334627+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:15.357320+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:40.403549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:06.030093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:31.904091+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:57.616372+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:24.045975+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:51.160898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:17.597845+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:44.636629+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1596672353":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1846482551":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/1069571878":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6800/2738158981":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/3625745146":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6801/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/564539954":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1797199863":"2024-05-10T04:40:27.201119+0000","172.21.15.46:6801/2738158981":"2024-05-10T04:39:59.515628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:48:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:52.141+0000 7f5b48d50700 1 -- 172.21.15.46:0/3738953808 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5b2c0b2990 msgr2=0x7f5b2c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:52.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:52.141+0000 7f5b48d50700 1 --2- 172.21.15.46:0/3738953808 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5b2c0b2990 0x7f5b2c0b4ea0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5b34009990 tx=0x7f5b340036e0 comp rx=0 tx=0).stop 2024-05-09T04:48:52.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:52.141+0000 7f5b48d50700 1 -- 172.21.15.46:0/3738953808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b44108f10 msgr2=0x7f5b4419b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:48:52.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:52.141+0000 7f5b48d50700 1 --2- 172.21.15.46:0/3738953808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b44108f10 0x7f5b4419b440 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5b3800d980 tx=0x7f5b3800dc90 comp rx=0 tx=0).stop 2024-05-09T04:48:52.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:52.142+0000 7f5b48d50700 1 -- 172.21.15.46:0/3738953808 shutdown_connections 2024-05-09T04:48:52.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:52.142+0000 7f5b48d50700 1 --2- 172.21.15.46:0/3738953808 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5b2c0b2990 0x7f5b2c0b4ea0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:52.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:52.142+0000 7f5b48d50700 1 --2- 172.21.15.46:0/3738953808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5b44109860 0x7f5b4419f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:52.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:52.142+0000 7f5b48d50700 1 --2- 172.21.15.46:0/3738953808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5b441045e0 0x7f5b4419aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:52.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:52.142+0000 7f5b48d50700 1 --2- 172.21.15.46:0/3738953808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5b44108f10 0x7f5b4419b440 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:48:52.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:52.142+0000 7f5b48d50700 1 -- 172.21.15.46:0/3738953808 >> 172.21.15.46:0/3738953808 conn(0x7f5b440ffef0 msgr2=0x7f5b44102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:48:52.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:52.142+0000 7f5b48d50700 1 -- 172.21.15.46:0/3738953808 shutdown_connections 2024-05-09T04:48:52.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:52.142+0000 7f5b48d50700 1 -- 172.21.15.46:0/3738953808 wait complete. 2024-05-09T04:48:52.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:52 smithi110 ceph-mon[34904]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:48:52.329 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:52 smithi046 ceph-mon[30136]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:48:52.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:52 smithi072 ceph-mon[34088]: pgmap v324: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:48:52.779 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-09T04:48:52.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-09T04:48:52.780 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-09T04:48:52.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-09T04:48:52.781 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-09T04:48:52.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-09T04:48:52.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-09T04:48:52.782 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-09T04:48:52.783 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-09T04:48:52.783 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-09T04:48:52.784 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-09T04:48:52.784 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-09T04:48:53.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:53 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3738953808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:48:53.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:53 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3738953808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:48:53.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:53 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3738953808' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:48:53.598 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:53.604 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:53.604 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:53.610 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:53.611 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:53.614 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:53.614 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:53.618 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:53.625 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:53.625 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:53.626 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:53.629 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:48:54.264 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:54 smithi046 ceph-mon[30136]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:48:54.379 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:54 smithi072 ceph-mon[34088]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:48:54.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:54 smithi110 ceph-mon[34904]: pgmap v325: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:48:56.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:56 smithi046 ceph-mon[30136]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:48:56.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:56 smithi072 ceph-mon[34088]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:48:56.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:56 smithi110 ceph-mon[34904]: pgmap v326: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:48:58.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:48:58 smithi046 ceph-mon[30136]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:48:58.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:48:58 smithi072 ceph-mon[34088]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:48:58.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:48:58 smithi110 ceph-mon[34904]: pgmap v327: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:49:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.675+0000 7f41886a3700 1 -- 172.21.15.46:0/1987490370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41801045e0 msgr2=0x7f41801049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.675+0000 7f41886a3700 1 --2- 172.21.15.46:0/1987490370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41801045e0 0x7f41801049f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f417c0099c0 tx=0x7f417c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.675+0000 7f41886a3700 1 -- 172.21.15.46:0/1987490370 shutdown_connections 2024-05-09T04:49:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.675+0000 7f41886a3700 1 --2- 172.21.15.46:0/1987490370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4180109860 0x7f41801111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.675+0000 7f41886a3700 1 --2- 172.21.15.46:0/1987490370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41801045e0 0x7f41801049f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.675+0000 7f41886a3700 1 --2- 172.21.15.46:0/1987490370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4180108f10 0x7f4180109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.675+0000 7f41886a3700 1 -- 172.21.15.46:0/1987490370 >> 172.21.15.46:0/1987490370 conn(0x7f41800ffe70 msgr2=0x7f41801022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.675+0000 7f41886a3700 1 -- 172.21.15.46:0/1987490370 shutdown_connections 2024-05-09T04:49:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.675+0000 7f41886a3700 1 -- 172.21.15.46:0/1987490370 wait complete. 2024-05-09T04:49:00.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.676+0000 7f41886a3700 1 Processor -- start 2024-05-09T04:49:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.676+0000 7f41886a3700 1 -- start start 2024-05-09T04:49:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.676+0000 7f41886a3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41801045e0 0x7f418019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.676+0000 7f41886a3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4180108f10 0x7f418019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.676+0000 7f41886a3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4180109860 0x7f418019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.676+0000 7f41886a3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4180114490 con 0x7f41801045e0 2024-05-09T04:49:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.676+0000 7f41886a3700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4180114310 con 0x7f4180108f10 2024-05-09T04:49:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.676+0000 7f41886a3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4180114610 con 0x7f4180109860 2024-05-09T04:49:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.676+0000 7f418643f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41801045e0 0x7f418019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.676+0000 7f418643f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41801045e0 0x7f418019ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60958/0 (socket says 172.21.15.46:60958) 2024-05-09T04:49:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.676+0000 7f418643f700 1 -- 172.21.15.46:0/2349644637 learned_addr learned my addr 172.21.15.46:0/2349644637 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.676+0000 7f4186c40700 1 --2- 172.21.15.46:0/2349644637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4180109860 0x7f418019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:00.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.676+0000 7f4185c3e700 1 --2- 172.21.15.46:0/2349644637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4180108f10 0x7f418019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.677+0000 7f418643f700 1 -- 172.21.15.46:0/2349644637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4180109860 msgr2=0x7f418019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.677+0000 7f418643f700 1 --2- 172.21.15.46:0/2349644637 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4180109860 0x7f418019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.677+0000 7f418643f700 1 -- 172.21.15.46:0/2349644637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4180108f10 msgr2=0x7f418019b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.677+0000 7f418643f700 1 --2- 172.21.15.46:0/2349644637 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4180108f10 0x7f418019b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.677+0000 7f418643f700 1 -- 172.21.15.46:0/2349644637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f418019fca0 con 0x7f41801045e0 2024-05-09T04:49:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.677+0000 7f418643f700 1 --2- 172.21.15.46:0/2349644637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41801045e0 0x7f418019ae80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f417c00b580 tx=0x7f417c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.677+0000 7f41777fe700 1 -- 172.21.15.46:0/2349644637 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c027070 con 0x7f41801045e0 2024-05-09T04:49:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.677+0000 7f41777fe700 1 -- 172.21.15.46:0/2349644637 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f417c003f20 con 0x7f41801045e0 2024-05-09T04:49:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.677+0000 7f41886a3700 1 -- 172.21.15.46:0/2349644637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f418019ff30 con 0x7f41801045e0 2024-05-09T04:49:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.677+0000 7f41777fe700 1 -- 172.21.15.46:0/2349644637 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f417c022d10 con 0x7f41801045e0 2024-05-09T04:49:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.677+0000 7f41886a3700 1 -- 172.21.15.46:0/2349644637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41801a0410 con 0x7f41801045e0 2024-05-09T04:49:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.678+0000 7f41886a3700 1 -- 172.21.15.46:0/2349644637 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4180063ed0 con 0x7f41801045e0 2024-05-09T04:49:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.681+0000 7f41777fe700 1 -- 172.21.15.46:0/2349644637 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f417c022470 con 0x7f41801045e0 2024-05-09T04:49:00.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.682+0000 7f41777fe700 1 --2- 172.21.15.46:0/2349644637 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f416c0b2910 0x7f416c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.682+0000 7f4185c3e700 1 --2- 172.21.15.46:0/2349644637 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f416c0b2910 0x7f416c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.682+0000 7f41777fe700 1 -- 172.21.15.46:0/2349644637 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f417c0e35e0 con 0x7f41801045e0 2024-05-09T04:49:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.682+0000 7f41777fe700 1 --2- 172.21.15.46:0/2349644637 >> [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822] conn(0x7f416c0bc5a0 0x7f416c0bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.682+0000 7f41777fe700 1 -- 172.21.15.46:0/2349644637 --> [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f417c01ae50 con 0x7f416c0bc5a0 2024-05-09T04:49:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.682+0000 7f41777fe700 1 -- 172.21.15.46:0/2349644637 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f417c0e3990 con 0x7f41801045e0 2024-05-09T04:49:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.682+0000 7f4186c40700 1 --2- 172.21.15.46:0/2349644637 >> [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822] conn(0x7f416c0bc5a0 0x7f416c0bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.682+0000 7f4185c3e700 1 --2- 172.21.15.46:0/2349644637 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f416c0b2910 0x7f416c0b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4180108780 tx=0x7f417000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.683+0000 7f4186c40700 1 --2- 172.21.15.46:0/2349644637 >> [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822] conn(0x7f416c0bc5a0 0x7f416c0bea90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.683+0000 7f41777fe700 1 -- 172.21.15.46:0/2349644637 <== osd.9 v2:172.21.15.46:6826/1762262822 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f417c01ae50 con 0x7f416c0bc5a0 2024-05-09T04:49:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.701+0000 7f41886a3700 1 -- 172.21.15.46:0/2349644637 --> [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4180064350 con 0x7f416c0bc5a0 2024-05-09T04:49:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.701+0000 7f41777fe700 1 -- 172.21.15.46:0/2349644637 <== osd.9 v2:172.21.15.46:6826/1762262822 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4180064350 con 0x7f416c0bc5a0 2024-05-09T04:49:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.701+0000 7f41886a3700 1 -- 172.21.15.46:0/2349644637 >> [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822] conn(0x7f416c0bc5a0 msgr2=0x7f416c0bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.701+0000 7f41886a3700 1 --2- 172.21.15.46:0/2349644637 >> [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822] conn(0x7f416c0bc5a0 0x7f416c0bea90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:00.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.701+0000 7f41886a3700 1 -- 172.21.15.46:0/2349644637 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f416c0b2910 msgr2=0x7f416c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:00.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.701+0000 7f41886a3700 1 --2- 172.21.15.46:0/2349644637 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f416c0b2910 0x7f416c0b4e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f4180108780 tx=0x7f417000a380 comp rx=0 tx=0).stop 2024-05-09T04:49:00.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.701+0000 7f41886a3700 1 -- 172.21.15.46:0/2349644637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41801045e0 msgr2=0x7f418019ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:00.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.701+0000 7f41886a3700 1 --2- 172.21.15.46:0/2349644637 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f41801045e0 0x7f418019ae80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f417c00b580 tx=0x7f417c0047b0 comp rx=0 tx=0).stop 2024-05-09T04:49:00.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.701+0000 7f4186c40700 1 -- 172.21.15.46:0/2349644637 reap_dead start 2024-05-09T04:49:00.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.702+0000 7f41886a3700 1 -- 172.21.15.46:0/2349644637 shutdown_connections 2024-05-09T04:49:00.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.702+0000 7f41886a3700 1 -- 172.21.15.46:0/2349644637 >> 172.21.15.46:0/2349644637 conn(0x7f41800ffe70 msgr2=0x7f4180107140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:00.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.702+0000 7f41886a3700 1 -- 172.21.15.46:0/2349644637 shutdown_connections 2024-05-09T04:49:00.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:48:59.702+0000 7f41886a3700 1 -- 172.21.15.46:0/2349644637 wait complete. 2024-05-09T04:49:00.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:00 smithi072 ceph-mon[34088]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:49:00.805 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:00 smithi046 ceph-mon[30136]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:49:00.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:00 smithi110 ceph-mon[34904]: pgmap v328: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:49:01.011 INFO:teuthology.orchestra.run.smithi046.stdout:261993005096 2024-05-09T04:49:01.011 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-09T04:49:02.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:02 smithi046 ceph-mon[30136]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:49:02.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:02 smithi072 ceph-mon[34088]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:49:02.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:02 smithi110 ceph-mon[34904]: pgmap v329: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:49:04.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:04 smithi072 ceph-mon[34088]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:49:04.722 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:04 smithi046 ceph-mon[30136]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:49:04.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:04 smithi110 ceph-mon[34904]: pgmap v330: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:49:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.719+0000 7f7a70809700 1 -- 172.21.15.46:0/3213086736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a6c10e8f0 msgr2=0x7f7a6c110dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.719+0000 7f7a70809700 1 --2- 172.21.15.46:0/3213086736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a6c10e8f0 0x7f7a6c110dd0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7a6400b0a0 tx=0x7f7a6400b3b0 comp rx=0 tx=0).stop 2024-05-09T04:49:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.720+0000 7f7a70809700 1 -- 172.21.15.46:0/3213086736 shutdown_connections 2024-05-09T04:49:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.720+0000 7f7a70809700 1 --2- 172.21.15.46:0/3213086736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a6c10e8f0 0x7f7a6c110dd0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.720+0000 7f7a70809700 1 --2- 172.21.15.46:0/3213086736 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a6c105480 0x7f7a6c105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.720+0000 7f7a70809700 1 --2- 172.21.15.46:0/3213086736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a6c105dd0 0x7f7a6c10e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.720+0000 7f7a70809700 1 -- 172.21.15.46:0/3213086736 >> 172.21.15.46:0/3213086736 conn(0x7f7a6c0fef30 msgr2=0x7f7a6c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.720+0000 7f7a70809700 1 -- 172.21.15.46:0/3213086736 shutdown_connections 2024-05-09T04:49:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.720+0000 7f7a70809700 1 -- 172.21.15.46:0/3213086736 wait complete. 2024-05-09T04:49:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.721+0000 7f7a70809700 1 Processor -- start 2024-05-09T04:49:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.721+0000 7f7a70809700 1 -- start start 2024-05-09T04:49:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.721+0000 7f7a70809700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a6c105480 0x7f7a6c19ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.721+0000 7f7a70809700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a6c105dd0 0x7f7a6c19b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.721+0000 7f7a70809700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a6c10e8f0 0x7f7a6c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.721+0000 7f7a70809700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c114450 con 0x7f7a6c105dd0 2024-05-09T04:49:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.721+0000 7f7a70809700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1142d0 con 0x7f7a6c10e8f0 2024-05-09T04:49:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.721+0000 7f7a70809700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7a6c1145d0 con 0x7f7a6c105480 2024-05-09T04:49:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.721+0000 7f7a69d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a6c105dd0 0x7f7a6c19b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.721+0000 7f7a69d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a6c105dd0 0x7f7a6c19b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45034/0 (socket says 172.21.15.46:45034) 2024-05-09T04:49:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.721+0000 7f7a69d9b700 1 -- 172.21.15.46:0/2805005600 learned_addr learned my addr 172.21.15.46:0/2805005600 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.721+0000 7f7a6a59c700 1 --2- 172.21.15.46:0/2805005600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a6c105480 0x7f7a6c19ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.722+0000 7f7a6ad9d700 1 --2- 172.21.15.46:0/2805005600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a6c10e8f0 0x7f7a6c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.722+0000 7f7a69d9b700 1 -- 172.21.15.46:0/2805005600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a6c105480 msgr2=0x7f7a6c19ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.722+0000 7f7a69d9b700 1 --2- 172.21.15.46:0/2805005600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7a6c105480 0x7f7a6c19ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.722+0000 7f7a69d9b700 1 -- 172.21.15.46:0/2805005600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a6c10e8f0 msgr2=0x7f7a6c19f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.722+0000 7f7a69d9b700 1 --2- 172.21.15.46:0/2805005600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7a6c10e8f0 0x7f7a6c19f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.722+0000 7f7a69d9b700 1 -- 172.21.15.46:0/2805005600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a6c19fd10 con 0x7f7a6c105dd0 2024-05-09T04:49:05.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.722+0000 7f7a69d9b700 1 --2- 172.21.15.46:0/2805005600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a6c105dd0 0x7f7a6c19b3e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000e9a0 tx=0x7f7a6000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:05.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.722+0000 7f7a5b7fe700 1 -- 172.21.15.46:0/2805005600 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a6000ca10 con 0x7f7a6c105dd0 2024-05-09T04:49:05.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.722+0000 7f7a5b7fe700 1 -- 172.21.15.46:0/2805005600 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a6000cbb0 con 0x7f7a6c105dd0 2024-05-09T04:49:05.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.723+0000 7f7a5b7fe700 1 -- 172.21.15.46:0/2805005600 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a60018680 con 0x7f7a6c105dd0 2024-05-09T04:49:05.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.723+0000 7f7a70809700 1 -- 172.21.15.46:0/2805005600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a6c1a0000 con 0x7f7a6c105dd0 2024-05-09T04:49:05.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.723+0000 7f7a70809700 1 -- 172.21.15.46:0/2805005600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a6c1a7920 con 0x7f7a6c105dd0 2024-05-09T04:49:05.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.723+0000 7f7a70809700 1 -- 172.21.15.46:0/2805005600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7a6c063e90 con 0x7f7a6c105dd0 2024-05-09T04:49:05.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.725+0000 7f7a5b7fe700 1 -- 172.21.15.46:0/2805005600 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7a60010430 con 0x7f7a6c105dd0 2024-05-09T04:49:05.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.726+0000 7f7a5b7fe700 1 --2- 172.21.15.46:0/2805005600 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7a540b2990 0x7f7a540b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:05.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.726+0000 7f7a6a59c700 1 --2- 172.21.15.46:0/2805005600 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7a540b2990 0x7f7a540b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:05.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.726+0000 7f7a5b7fe700 1 -- 172.21.15.46:0/2805005600 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f7a600e2ff0 con 0x7f7a6c105dd0 2024-05-09T04:49:05.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.727+0000 7f7a5b7fe700 1 --2- 172.21.15.46:0/2805005600 >> [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775] conn(0x7f7a540bc620 0x7f7a540beb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:05.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.727+0000 7f7a5b7fe700 1 -- 172.21.15.46:0/2805005600 --> [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7a600090b0 con 0x7f7a540bc620 2024-05-09T04:49:05.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.727+0000 7f7a5b7fe700 1 -- 172.21.15.46:0/2805005600 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f7a600e3420 con 0x7f7a6c105dd0 2024-05-09T04:49:05.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.727+0000 7f7a6a59c700 1 --2- 172.21.15.46:0/2805005600 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7a540b2990 0x7f7a540b4ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c005950 tx=0x7f7a5c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:05.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.727+0000 7f7a6ad9d700 1 --2- 172.21.15.46:0/2805005600 >> [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775] conn(0x7f7a540bc620 0x7f7a540beb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:05.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.727+0000 7f7a6ad9d700 1 --2- 172.21.15.46:0/2805005600 >> [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775] conn(0x7f7a540bc620 0x7f7a540beb10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:05.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.728+0000 7f7a5b7fe700 1 -- 172.21.15.46:0/2805005600 <== osd.1 v2:172.21.15.72:6800/379039775 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7a600090b0 con 0x7f7a540bc620 2024-05-09T04:49:05.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.745+0000 7f7a70809700 1 -- 172.21.15.46:0/2805005600 --> [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7a6c0642e0 con 0x7f7a540bc620 2024-05-09T04:49:05.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.745+0000 7f7a5b7fe700 1 -- 172.21.15.46:0/2805005600 <== osd.1 v2:172.21.15.72:6800/379039775 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7a6c0642e0 con 0x7f7a540bc620 2024-05-09T04:49:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.746+0000 7f7a70809700 1 -- 172.21.15.46:0/2805005600 >> [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775] conn(0x7f7a540bc620 msgr2=0x7f7a540beb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.746+0000 7f7a70809700 1 --2- 172.21.15.46:0/2805005600 >> [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775] conn(0x7f7a540bc620 0x7f7a540beb10 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.746+0000 7f7a70809700 1 -- 172.21.15.46:0/2805005600 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7a540b2990 msgr2=0x7f7a540b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.746+0000 7f7a70809700 1 --2- 172.21.15.46:0/2805005600 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7a540b2990 0x7f7a540b4ea0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7a5c005950 tx=0x7f7a5c00b410 comp rx=0 tx=0).stop 2024-05-09T04:49:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.746+0000 7f7a70809700 1 -- 172.21.15.46:0/2805005600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a6c105dd0 msgr2=0x7f7a6c19b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.746+0000 7f7a70809700 1 --2- 172.21.15.46:0/2805005600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7a6c105dd0 0x7f7a6c19b3e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000e9a0 tx=0x7f7a6000ecb0 comp rx=0 tx=0).stop 2024-05-09T04:49:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.746+0000 7f7a6ad9d700 1 -- 172.21.15.46:0/2805005600 reap_dead start 2024-05-09T04:49:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.746+0000 7f7a70809700 1 -- 172.21.15.46:0/2805005600 shutdown_connections 2024-05-09T04:49:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.746+0000 7f7a70809700 1 -- 172.21.15.46:0/2805005600 >> 172.21.15.46:0/2805005600 conn(0x7f7a6c0fef30 msgr2=0x7f7a6c0ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.746+0000 7f7a70809700 1 -- 172.21.15.46:0/2805005600 shutdown_connections 2024-05-09T04:49:05.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.746+0000 7f7a70809700 1 -- 172.21.15.46:0/2805005600 wait complete. 2024-05-09T04:49:06.634 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:06 smithi046 ceph-mon[30136]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:49:06.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:06 smithi110 ceph-mon[34904]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:49:06.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.832+0000 7f4c9864e700 1 -- 172.21.15.46:0/1909782254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c901034c0 msgr2=0x7f4c901038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:06.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.832+0000 7f4c9864e700 1 --2- 172.21.15.46:0/1909782254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c901034c0 0x7f4c901038d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0099c0 tx=0x7f4c8c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:06.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.832+0000 7f4c9864e700 1 -- 172.21.15.46:0/1909782254 shutdown_connections 2024-05-09T04:49:06.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.832+0000 7f4c9864e700 1 --2- 172.21.15.46:0/1909782254 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c90108740 0x7f4c901100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:06.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.832+0000 7f4c9864e700 1 --2- 172.21.15.46:0/1909782254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c901034c0 0x7f4c901038d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:06.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.832+0000 7f4c9864e700 1 --2- 172.21.15.46:0/1909782254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c90107df0 0x7f4c90108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:06.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.832+0000 7f4c9864e700 1 -- 172.21.15.46:0/1909782254 >> 172.21.15.46:0/1909782254 conn(0x7f4c900fedf0 msgr2=0x7f4c90101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:06.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.832+0000 7f4c9864e700 1 -- 172.21.15.46:0/1909782254 shutdown_connections 2024-05-09T04:49:06.834 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.832+0000 7f4c9864e700 1 -- 172.21.15.46:0/1909782254 wait complete. 2024-05-09T04:49:06.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.833+0000 7f4c9864e700 1 Processor -- start 2024-05-09T04:49:06.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.833+0000 7f4c9864e700 1 -- start start 2024-05-09T04:49:06.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c9864e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c901034c0 0x7f4c9019d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:06.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c9864e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c90107df0 0x7f4c9019d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:06.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c9864e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c90108740 0x7f4c901a1870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:06.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c9864e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4c90113370 con 0x7f4c901034c0 2024-05-09T04:49:06.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c9864e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4c901131f0 con 0x7f4c90108740 2024-05-09T04:49:06.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c9864e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4c901134f0 con 0x7f4c90107df0 2024-05-09T04:49:06.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c963ea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c901034c0 0x7f4c9019d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:06.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c963ea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c901034c0 0x7f4c9019d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45060/0 (socket says 172.21.15.46:45060) 2024-05-09T04:49:06.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c963ea700 1 -- 172.21.15.46:0/47130721 learned_addr learned my addr 172.21.15.46:0/47130721 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:06.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c96beb700 1 --2- 172.21.15.46:0/47130721 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c90108740 0x7f4c901a1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:06.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c95be9700 1 --2- 172.21.15.46:0/47130721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c90107df0 0x7f4c9019d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:06.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c963ea700 1 -- 172.21.15.46:0/47130721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c90107df0 msgr2=0x7f4c9019d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:06.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c963ea700 1 --2- 172.21.15.46:0/47130721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c90107df0 0x7f4c9019d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:06.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c963ea700 1 -- 172.21.15.46:0/47130721 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c90108740 msgr2=0x7f4c901a1870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:06.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c963ea700 1 --2- 172.21.15.46:0/47130721 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4c90108740 0x7f4c901a1870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:06.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c963ea700 1 -- 172.21.15.46:0/47130721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c901a1f10 con 0x7f4c901034c0 2024-05-09T04:49:06.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c95be9700 1 --2- 172.21.15.46:0/47130721 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4c90107df0 0x7f4c9019d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:49:06.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c963ea700 1 --2- 172.21.15.46:0/47130721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c901034c0 0x7f4c9019d040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0098e0 tx=0x7f4c8c0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:06.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c877fe700 1 -- 172.21.15.46:0/47130721 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c027070 con 0x7f4c901034c0 2024-05-09T04:49:06.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c877fe700 1 -- 172.21.15.46:0/47130721 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c8c003f20 con 0x7f4c901034c0 2024-05-09T04:49:06.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.834+0000 7f4c9864e700 1 -- 172.21.15.46:0/47130721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c901a21a0 con 0x7f4c901034c0 2024-05-09T04:49:06.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.835+0000 7f4c877fe700 1 -- 172.21.15.46:0/47130721 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c022da0 con 0x7f4c901034c0 2024-05-09T04:49:06.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.835+0000 7f4c9864e700 1 -- 172.21.15.46:0/47130721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c901a2680 con 0x7f4c901034c0 2024-05-09T04:49:06.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.836+0000 7f4c9864e700 1 -- 172.21.15.46:0/47130721 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4c901099f0 con 0x7f4c901034c0 2024-05-09T04:49:06.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.837+0000 7f4c877fe700 1 -- 172.21.15.46:0/47130721 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4c8c02a070 con 0x7f4c901034c0 2024-05-09T04:49:06.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.838+0000 7f4c877fe700 1 --2- 172.21.15.46:0/47130721 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4c7c0b2910 0x7f4c7c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:06.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.838+0000 7f4c95be9700 1 --2- 172.21.15.46:0/47130721 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4c7c0b2910 0x7f4c7c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:06.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.838+0000 7f4c877fe700 1 -- 172.21.15.46:0/47130721 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f4c8c0e3b30 con 0x7f4c901034c0 2024-05-09T04:49:06.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.839+0000 7f4c877fe700 1 --2- 172.21.15.46:0/47130721 >> [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203] conn(0x7f4c7c0bc5a0 0x7f4c7c0bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:06.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.839+0000 7f4c877fe700 1 -- 172.21.15.46:0/47130721 --> [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4c8c02ee80 con 0x7f4c7c0bc5a0 2024-05-09T04:49:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.839+0000 7f4c877fe700 1 -- 172.21.15.46:0/47130721 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f4c8c0e3f80 con 0x7f4c901034c0 2024-05-09T04:49:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.839+0000 7f4c95be9700 1 --2- 172.21.15.46:0/47130721 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4c7c0b2910 0x7f4c7c0b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4c900767f0 tx=0x7f4c8000b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.839+0000 7f4c96beb700 1 --2- 172.21.15.46:0/47130721 >> [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203] conn(0x7f4c7c0bc5a0 0x7f4c7c0bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:06.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.839+0000 7f4c96beb700 1 --2- 172.21.15.46:0/47130721 >> [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203] conn(0x7f4c7c0bc5a0 0x7f4c7c0bea90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:06.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.840+0000 7f4c877fe700 1 -- 172.21.15.46:0/47130721 <== osd.8 v2:172.21.15.110:6816/189513203 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4c8c02ee80 con 0x7f4c7c0bc5a0 2024-05-09T04:49:06.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.857+0000 7f4c9864e700 1 -- 172.21.15.46:0/47130721 --> [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4c90104970 con 0x7f4c7c0bc5a0 2024-05-09T04:49:06.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.858+0000 7f4c877fe700 1 -- 172.21.15.46:0/47130721 <== osd.8 v2:172.21.15.110:6816/189513203 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4c90104970 con 0x7f4c7c0bc5a0 2024-05-09T04:49:06.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.858+0000 7f4c9864e700 1 -- 172.21.15.46:0/47130721 >> [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203] conn(0x7f4c7c0bc5a0 msgr2=0x7f4c7c0bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:06.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.858+0000 7f4c9864e700 1 --2- 172.21.15.46:0/47130721 >> [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203] conn(0x7f4c7c0bc5a0 0x7f4c7c0bea90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:06.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.858+0000 7f4c9864e700 1 -- 172.21.15.46:0/47130721 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4c7c0b2910 msgr2=0x7f4c7c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:06.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.858+0000 7f4c9864e700 1 --2- 172.21.15.46:0/47130721 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f4c7c0b2910 0x7f4c7c0b4e20 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4c900767f0 tx=0x7f4c8000b5f0 comp rx=0 tx=0).stop 2024-05-09T04:49:06.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.858+0000 7f4c9864e700 1 -- 172.21.15.46:0/47130721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c901034c0 msgr2=0x7f4c9019d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.858+0000 7f4c9864e700 1 --2- 172.21.15.46:0/47130721 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4c901034c0 0x7f4c9019d040 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4c8c0098e0 tx=0x7f4c8c0057c0 comp rx=0 tx=0).stop 2024-05-09T04:49:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.858+0000 7f4c96beb700 1 -- 172.21.15.46:0/47130721 reap_dead start 2024-05-09T04:49:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.858+0000 7f4c9864e700 1 -- 172.21.15.46:0/47130721 shutdown_connections 2024-05-09T04:49:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.858+0000 7f4c9864e700 1 -- 172.21.15.46:0/47130721 >> 172.21.15.46:0/47130721 conn(0x7f4c900fedf0 msgr2=0x7f4c901076b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.858+0000 7f4c9864e700 1 -- 172.21.15.46:0/47130721 shutdown_connections 2024-05-09T04:49:06.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:06.858+0000 7f4c9864e700 1 -- 172.21.15.46:0/47130721 wait complete. 2024-05-09T04:49:06.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:06 smithi072 ceph-mon[34088]: pgmap v331: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:49:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.034+0000 7fefb952f700 1 -- 172.21.15.46:0/1137468460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb41045d0 msgr2=0x7fefb41049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.034+0000 7fefb952f700 1 --2- 172.21.15.46:0/1137468460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb41045d0 0x7fefb41049e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fefa400b0d0 tx=0x7fefa400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.034+0000 7fefb952f700 1 -- 172.21.15.46:0/1137468460 shutdown_connections 2024-05-09T04:49:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.034+0000 7fefb952f700 1 --2- 172.21.15.46:0/1137468460 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fefb4109850 0x7fefb41111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.034+0000 7fefb952f700 1 --2- 172.21.15.46:0/1137468460 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb41045d0 0x7fefb41049e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.034+0000 7fefb952f700 1 --2- 172.21.15.46:0/1137468460 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefb4108f00 0x7fefb4109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.034+0000 7fefb952f700 1 -- 172.21.15.46:0/1137468460 >> 172.21.15.46:0/1137468460 conn(0x7fefb40ffea0 msgr2=0x7fefb41022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.034+0000 7fefb952f700 1 -- 172.21.15.46:0/1137468460 shutdown_connections 2024-05-09T04:49:07.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.034+0000 7fefb952f700 1 -- 172.21.15.46:0/1137468460 wait complete. 2024-05-09T04:49:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.035+0000 7fefb952f700 1 Processor -- start 2024-05-09T04:49:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.035+0000 7fefb952f700 1 -- start start 2024-05-09T04:49:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb952f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fefb41045d0 0x7fefb419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb952f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb4108f00 0x7fefb419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb952f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefb4109850 0x7fefb419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb952f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4114480 con 0x7fefb4108f00 2024-05-09T04:49:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb952f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4114300 con 0x7fefb41045d0 2024-05-09T04:49:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb952f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fefb4114600 con 0x7fefb4109850 2024-05-09T04:49:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb27fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb4108f00 0x7fefb419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:07.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb27fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb4108f00 0x7fefb419b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45074/0 (socket says 172.21.15.46:45074) 2024-05-09T04:49:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb27fc700 1 -- 172.21.15.46:0/3637359744 learned_addr learned my addr 172.21.15.46:0/3637359744 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb2ffd700 1 --2- 172.21.15.46:0/3637359744 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fefb41045d0 0x7fefb419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb37fe700 1 --2- 172.21.15.46:0/3637359744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefb4109850 0x7fefb419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb27fc700 1 -- 172.21.15.46:0/3637359744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefb4109850 msgr2=0x7fefb419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb27fc700 1 --2- 172.21.15.46:0/3637359744 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fefb4109850 0x7fefb419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb27fc700 1 -- 172.21.15.46:0/3637359744 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fefb41045d0 msgr2=0x7fefb419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb27fc700 1 --2- 172.21.15.46:0/3637359744 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fefb41045d0 0x7fefb419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb27fc700 1 -- 172.21.15.46:0/3637359744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefb419fca0 con 0x7fefb4108f00 2024-05-09T04:49:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.036+0000 7fefb27fc700 1 --2- 172.21.15.46:0/3637359744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb4108f00 0x7fefb419b370 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00c8f0 tx=0x7fef9c00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.037+0000 7fefabfff700 1 -- 172.21.15.46:0/3637359744 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c007880 con 0x7fefb4108f00 2024-05-09T04:49:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.037+0000 7fefb952f700 1 -- 172.21.15.46:0/3637359744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefb419ff90 con 0x7fefb4108f00 2024-05-09T04:49:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.037+0000 7fefabfff700 1 -- 172.21.15.46:0/3637359744 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef9c00f450 con 0x7fefb4108f00 2024-05-09T04:49:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.037+0000 7fefabfff700 1 -- 172.21.15.46:0/3637359744 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c018640 con 0x7fefb4108f00 2024-05-09T04:49:07.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.037+0000 7fefb952f700 1 -- 172.21.15.46:0/3637359744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefb4076b00 con 0x7fefb4108f00 2024-05-09T04:49:07.040 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.037+0000 7fefb952f700 1 -- 172.21.15.46:0/3637359744 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fefb4063f50 con 0x7fefb4108f00 2024-05-09T04:49:07.041 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.040+0000 7fefabfff700 1 -- 172.21.15.46:0/3637359744 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fef9c00f900 con 0x7fefb4108f00 2024-05-09T04:49:07.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.041+0000 7fefabfff700 1 --2- 172.21.15.46:0/3637359744 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fefa00b6d20 0x7fefa00b9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:07.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.041+0000 7fefb2ffd700 1 --2- 172.21.15.46:0/3637359744 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fefa00b6d20 0x7fefa00b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:07.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.041+0000 7fefabfff700 1 -- 172.21.15.46:0/3637359744 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fef9c0e2660 con 0x7fefb4108f00 2024-05-09T04:49:07.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.042+0000 7fefabfff700 1 --2- 172.21.15.46:0/3637359744 >> [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653] conn(0x7fefa00c09b0 0x7fefa00c2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:07.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.042+0000 7fefabfff700 1 -- 172.21.15.46:0/3637359744 --> [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fef9c019ec0 con 0x7fefa00c09b0 2024-05-09T04:49:07.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.042+0000 7fefabfff700 1 -- 172.21.15.46:0/3637359744 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fef9c0e2ab0 con 0x7fefb4108f00 2024-05-09T04:49:07.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.042+0000 7fefb37fe700 1 --2- 172.21.15.46:0/3637359744 >> [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653] conn(0x7fefa00c09b0 0x7fefa00c2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:07.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.042+0000 7fefb2ffd700 1 --2- 172.21.15.46:0/3637359744 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fefa00b6d20 0x7fefa00b9230 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009900 tx=0x7fefa400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:07.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.042+0000 7fefb37fe700 1 --2- 172.21.15.46:0/3637359744 >> [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653] conn(0x7fefa00c09b0 0x7fefa00c2ea0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:07.044 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.043+0000 7fefabfff700 1 -- 172.21.15.46:0/3637359744 <== osd.3 v2:172.21.15.46:6810/1115835653 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fef9c019ec0 con 0x7fefa00c09b0 2024-05-09T04:49:07.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.060+0000 7fefb952f700 1 -- 172.21.15.46:0/3637359744 --> [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fefb4064350 con 0x7fefa00c09b0 2024-05-09T04:49:07.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.060+0000 7fefabfff700 1 -- 172.21.15.46:0/3637359744 <== osd.3 v2:172.21.15.46:6810/1115835653 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fefb4064350 con 0x7fefa00c09b0 2024-05-09T04:49:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.060+0000 7fefb952f700 1 -- 172.21.15.46:0/3637359744 >> [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653] conn(0x7fefa00c09b0 msgr2=0x7fefa00c2ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.060+0000 7fefb952f700 1 --2- 172.21.15.46:0/3637359744 >> [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653] conn(0x7fefa00c09b0 0x7fefa00c2ea0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.060+0000 7fefb952f700 1 -- 172.21.15.46:0/3637359744 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fefa00b6d20 msgr2=0x7fefa00b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.060+0000 7fefb952f700 1 --2- 172.21.15.46:0/3637359744 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fefa00b6d20 0x7fefa00b9230 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fefa4009900 tx=0x7fefa400b040 comp rx=0 tx=0).stop 2024-05-09T04:49:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.060+0000 7fefb952f700 1 -- 172.21.15.46:0/3637359744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb4108f00 msgr2=0x7fefb419b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.060+0000 7fefb952f700 1 --2- 172.21.15.46:0/3637359744 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fefb4108f00 0x7fefb419b370 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00c8f0 tx=0x7fef9c00cc00 comp rx=0 tx=0).stop 2024-05-09T04:49:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.060+0000 7fefb37fe700 1 -- 172.21.15.46:0/3637359744 reap_dead start 2024-05-09T04:49:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.060+0000 7fefb952f700 1 -- 172.21.15.46:0/3637359744 shutdown_connections 2024-05-09T04:49:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.060+0000 7fefb952f700 1 -- 172.21.15.46:0/3637359744 >> 172.21.15.46:0/3637359744 conn(0x7fefb40ffea0 msgr2=0x7fefb410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.061+0000 7fefb952f700 1 -- 172.21.15.46:0/3637359744 shutdown_connections 2024-05-09T04:49:07.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.061+0000 7fefb952f700 1 -- 172.21.15.46:0/3637359744 wait complete. 2024-05-09T04:49:07.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.635+0000 7f3b5003c700 1 -- 172.21.15.46:0/2172324057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b481034c0 msgr2=0x7f3b481038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.635+0000 7f3b5003c700 1 --2- 172.21.15.46:0/2172324057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b481034c0 0x7f3b481038d0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400b0d0 tx=0x7f3b4400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.635+0000 7f3b5003c700 1 -- 172.21.15.46:0/2172324057 shutdown_connections 2024-05-09T04:49:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.635+0000 7f3b5003c700 1 --2- 172.21.15.46:0/2172324057 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b48108740 0x7f3b481100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.635+0000 7f3b5003c700 1 --2- 172.21.15.46:0/2172324057 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b481034c0 0x7f3b481038d0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.635+0000 7f3b5003c700 1 --2- 172.21.15.46:0/2172324057 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b48107df0 0x7f3b48108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.635+0000 7f3b5003c700 1 -- 172.21.15.46:0/2172324057 >> 172.21.15.46:0/2172324057 conn(0x7f3b480fedc0 msgr2=0x7f3b481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.635+0000 7f3b5003c700 1 -- 172.21.15.46:0/2172324057 shutdown_connections 2024-05-09T04:49:07.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.635+0000 7f3b5003c700 1 -- 172.21.15.46:0/2172324057 wait complete. 2024-05-09T04:49:07.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.636+0000 7f3b5003c700 1 Processor -- start 2024-05-09T04:49:07.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.636+0000 7f3b5003c700 1 -- start start 2024-05-09T04:49:07.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.636+0000 7f3b5003c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b481034c0 0x7f3b48113b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:07.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.636+0000 7f3b5003c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b48107df0 0x7f3b481140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:07.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.636+0000 7f3b5003c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b48108740 0x7f3b48114610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:07.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.636+0000 7f3b5003c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b48113370 con 0x7f3b48108740 2024-05-09T04:49:07.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.636+0000 7f3b5003c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481131f0 con 0x7f3b481034c0 2024-05-09T04:49:07.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.636+0000 7f3b5003c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3b481134f0 con 0x7f3b48107df0 2024-05-09T04:49:07.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.636+0000 7f3b4e5d9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b48108740 0x7f3b48114610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:07.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.636+0000 7f3b4e5d9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b48108740 0x7f3b48114610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45024/0 (socket says 172.21.15.46:45024) 2024-05-09T04:49:07.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.636+0000 7f3b4e5d9700 1 -- 172.21.15.46:0/610343314 learned_addr learned my addr 172.21.15.46:0/610343314 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.637+0000 7f3b4d5d7700 1 --2- 172.21.15.46:0/610343314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b48107df0 0x7f3b481140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.637+0000 7f3b4ddd8700 1 --2- 172.21.15.46:0/610343314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b481034c0 0x7f3b48113b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.637+0000 7f3b4e5d9700 1 -- 172.21.15.46:0/610343314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b48107df0 msgr2=0x7f3b481140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.637+0000 7f3b4e5d9700 1 --2- 172.21.15.46:0/610343314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b48107df0 0x7f3b481140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.637+0000 7f3b4e5d9700 1 -- 172.21.15.46:0/610343314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b481034c0 msgr2=0x7f3b48113b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.637+0000 7f3b4e5d9700 1 --2- 172.21.15.46:0/610343314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b481034c0 0x7f3b48113b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.637+0000 7f3b4e5d9700 1 -- 172.21.15.46:0/610343314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b481a9200 con 0x7f3b48108740 2024-05-09T04:49:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.637+0000 7f3b4ddd8700 1 --2- 172.21.15.46:0/610343314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b481034c0 0x7f3b48113b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:49:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.637+0000 7f3b4e5d9700 1 --2- 172.21.15.46:0/610343314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b48108740 0x7f3b48114610 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000c9a0 tx=0x7f3b4000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.637+0000 7f3b3effd700 1 -- 172.21.15.46:0/610343314 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40007ac0 con 0x7f3b48108740 2024-05-09T04:49:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.637+0000 7f3b3effd700 1 -- 172.21.15.46:0/610343314 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b40004ce0 con 0x7f3b48108740 2024-05-09T04:49:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.637+0000 7f3b5003c700 1 -- 172.21.15.46:0/610343314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b481a94f0 con 0x7f3b48108740 2024-05-09T04:49:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.637+0000 7f3b3effd700 1 -- 172.21.15.46:0/610343314 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b40005460 con 0x7f3b48108740 2024-05-09T04:49:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.637+0000 7f3b5003c700 1 -- 172.21.15.46:0/610343314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b481a9a30 con 0x7f3b48108740 2024-05-09T04:49:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.638+0000 7f3b5003c700 1 -- 172.21.15.46:0/610343314 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3b48107870 con 0x7f3b48108740 2024-05-09T04:49:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.640+0000 7f3b3effd700 1 -- 172.21.15.46:0/610343314 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3b40004750 con 0x7f3b48108740 2024-05-09T04:49:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.641+0000 7f3b3effd700 1 --2- 172.21.15.46:0/610343314 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b340b2990 0x7f3b340b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.641+0000 7f3b4ddd8700 1 --2- 172.21.15.46:0/610343314 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b340b2990 0x7f3b340b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.641+0000 7f3b3effd700 1 -- 172.21.15.46:0/610343314 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f3b400e2420 con 0x7f3b48108740 2024-05-09T04:49:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.641+0000 7f3b3effd700 1 --2- 172.21.15.46:0/610343314 >> [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898] conn(0x7f3b340bc620 0x7f3b340beb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.641+0000 7f3b3effd700 1 -- 172.21.15.46:0/610343314 --> [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3b40012ec0 con 0x7f3b340bc620 2024-05-09T04:49:07.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.641+0000 7f3b3effd700 1 -- 172.21.15.46:0/610343314 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f3b400e27d0 con 0x7f3b48108740 2024-05-09T04:49:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.641+0000 7f3b4ddd8700 1 --2- 172.21.15.46:0/610343314 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b340b2990 0x7f3b340b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400d010 tx=0x7f3b4400b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.642+0000 7f3b4d5d7700 1 --2- 172.21.15.46:0/610343314 >> [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898] conn(0x7f3b340bc620 0x7f3b340beb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.642+0000 7f3b4d5d7700 1 --2- 172.21.15.46:0/610343314 >> [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898] conn(0x7f3b340bc620 0x7f3b340beb10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.643+0000 7f3b3effd700 1 -- 172.21.15.46:0/610343314 <== osd.10 v2:172.21.15.72:6824/38453898 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3b40012ec0 con 0x7f3b340bc620 2024-05-09T04:49:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.660+0000 7f3b5003c700 1 -- 172.21.15.46:0/610343314 --> [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3b48104970 con 0x7f3b340bc620 2024-05-09T04:49:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.660+0000 7f3b3effd700 1 -- 172.21.15.46:0/610343314 <== osd.10 v2:172.21.15.72:6824/38453898 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f3b48104970 con 0x7f3b340bc620 2024-05-09T04:49:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.661+0000 7f3b5003c700 1 -- 172.21.15.46:0/610343314 >> [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898] conn(0x7f3b340bc620 msgr2=0x7f3b340beb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.661+0000 7f3b5003c700 1 --2- 172.21.15.46:0/610343314 >> [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898] conn(0x7f3b340bc620 0x7f3b340beb10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.661+0000 7f3b5003c700 1 -- 172.21.15.46:0/610343314 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b340b2990 msgr2=0x7f3b340b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.661+0000 7f3b5003c700 1 --2- 172.21.15.46:0/610343314 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b340b2990 0x7f3b340b4ea0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f3b4400d010 tx=0x7f3b4400b040 comp rx=0 tx=0).stop 2024-05-09T04:49:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.661+0000 7f3b5003c700 1 -- 172.21.15.46:0/610343314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b48108740 msgr2=0x7f3b48114610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.661+0000 7f3b5003c700 1 --2- 172.21.15.46:0/610343314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b48108740 0x7f3b48114610 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000c9a0 tx=0x7f3b4000ccb0 comp rx=0 tx=0).stop 2024-05-09T04:49:07.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.661+0000 7f3b4e5d9700 1 -- 172.21.15.46:0/610343314 reap_dead start 2024-05-09T04:49:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.661+0000 7f3b5003c700 1 -- 172.21.15.46:0/610343314 shutdown_connections 2024-05-09T04:49:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.661+0000 7f3b5003c700 1 -- 172.21.15.46:0/610343314 >> 172.21.15.46:0/610343314 conn(0x7f3b480fedc0 msgr2=0x7f3b481011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.661+0000 7f3b5003c700 1 -- 172.21.15.46:0/610343314 shutdown_connections 2024-05-09T04:49:07.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:05.661+0000 7f3b5003c700 1 -- 172.21.15.46:0/610343314 wait complete. 2024-05-09T04:49:08.563 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:08 smithi046 ceph-mon[30136]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T04:49:08.592 INFO:teuthology.orchestra.run.smithi046.stdout:283467841571 2024-05-09T04:49:08.592 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-09T04:49:08.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:08 smithi110 ceph-mon[34904]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T04:49:08.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:08 smithi072 ceph-mon[34088]: pgmap v332: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T04:49:08.971 INFO:teuthology.orchestra.run.smithi046.stdout:55834574931 2024-05-09T04:49:08.971 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-09T04:49:10.355 INFO:teuthology.orchestra.run.smithi046.stdout:236223201328 2024-05-09T04:49:10.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-09T04:49:10.579 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:10 smithi046 ceph-mon[30136]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-05-09T04:49:10.809 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:10 smithi110 ceph-mon[34904]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-05-09T04:49:10.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:10 smithi072 ceph-mon[34088]: pgmap v333: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 7 op/s 2024-05-09T04:49:10.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.978+0000 7f719b1b7700 1 -- 172.21.15.46:0/3055474991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7194108f20 msgr2=0x7f7194109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:10.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.978+0000 7f719b1b7700 1 --2- 172.21.15.46:0/3055474991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7194108f20 0x7f7194109330 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f71800099c0 tx=0x7f7180009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:10.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.978+0000 7f719b1b7700 1 -- 172.21.15.46:0/3055474991 shutdown_connections 2024-05-09T04:49:10.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.978+0000 7f719b1b7700 1 --2- 172.21.15.46:0/3055474991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7194109870 0x7f71941111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:10.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.978+0000 7f719b1b7700 1 --2- 172.21.15.46:0/3055474991 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71941045f0 0x7f7194104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.978+0000 7f719b1b7700 1 --2- 172.21.15.46:0/3055474991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7194108f20 0x7f7194109330 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.978+0000 7f719b1b7700 1 -- 172.21.15.46:0/3055474991 >> 172.21.15.46:0/3055474991 conn(0x7f71940ffec0 msgr2=0x7f7194102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.978+0000 7f719b1b7700 1 -- 172.21.15.46:0/3055474991 shutdown_connections 2024-05-09T04:49:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.978+0000 7f719b1b7700 1 -- 172.21.15.46:0/3055474991 wait complete. 2024-05-09T04:49:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f719b1b7700 1 Processor -- start 2024-05-09T04:49:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f719b1b7700 1 -- start start 2024-05-09T04:49:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f719b1b7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71941045f0 0x7f719419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f719b1b7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7194108f20 0x7f719419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f719b1b7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7194109870 0x7f719419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f719b1b7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7194114480 con 0x7f7194109870 2024-05-09T04:49:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f719b1b7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7194114300 con 0x7f71941045f0 2024-05-09T04:49:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f719b1b7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7194114600 con 0x7f7194108f20 2024-05-09T04:49:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f7199754700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7194109870 0x7f719419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:10.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f7199754700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7194109870 0x7f719419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45078/0 (socket says 172.21.15.46:45078) 2024-05-09T04:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f7199754700 1 -- 172.21.15.46:0/23622239 learned_addr learned my addr 172.21.15.46:0/23622239 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f7198f53700 1 --2- 172.21.15.46:0/23622239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71941045f0 0x7f719419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f7193fff700 1 --2- 172.21.15.46:0/23622239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7194108f20 0x7f719419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f7199754700 1 -- 172.21.15.46:0/23622239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7194108f20 msgr2=0x7f719419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f7199754700 1 --2- 172.21.15.46:0/23622239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7194108f20 0x7f719419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f7199754700 1 -- 172.21.15.46:0/23622239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71941045f0 msgr2=0x7f719419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f7199754700 1 --2- 172.21.15.46:0/23622239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71941045f0 0x7f719419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f7199754700 1 -- 172.21.15.46:0/23622239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719419fd00 con 0x7f7194109870 2024-05-09T04:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.979+0000 7f7193fff700 1 --2- 172.21.15.46:0/23622239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7194108f20 0x7f719419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.980+0000 7f7199754700 1 --2- 172.21.15.46:0/23622239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7194109870 0x7f719419f660 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f718c00ef30 tx=0x7f718c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.980+0000 7f7191ffb700 1 -- 172.21.15.46:0/23622239 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c0092e0 con 0x7f7194109870 2024-05-09T04:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.980+0000 7f719b1b7700 1 -- 172.21.15.46:0/23622239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719419ff90 con 0x7f7194109870 2024-05-09T04:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.980+0000 7f7191ffb700 1 -- 172.21.15.46:0/23622239 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718c004bb0 con 0x7f7194109870 2024-05-09T04:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.980+0000 7f7191ffb700 1 -- 172.21.15.46:0/23622239 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718c004280 con 0x7f7194109870 2024-05-09T04:49:10.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.980+0000 7f719b1b7700 1 -- 172.21.15.46:0/23622239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71941a0450 con 0x7f7194109870 2024-05-09T04:49:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.981+0000 7f719b1b7700 1 -- 172.21.15.46:0/23622239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7194063f10 con 0x7f7194109870 2024-05-09T04:49:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.983+0000 7f7191ffb700 1 -- 172.21.15.46:0/23622239 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f718c01b030 con 0x7f7194109870 2024-05-09T04:49:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.983+0000 7f7191ffb700 1 --2- 172.21.15.46:0/23622239 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f71840b2910 0x7f71840b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.983+0000 7f7198f53700 1 --2- 172.21.15.46:0/23622239 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f71840b2910 0x7f71840b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.984+0000 7f7191ffb700 1 -- 172.21.15.46:0/23622239 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f718c0e2840 con 0x7f7194109870 2024-05-09T04:49:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.984+0000 7f7191ffb700 1 --2- 172.21.15.46:0/23622239 >> [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065] conn(0x7f71840bc5a0 0x7f71840bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.984+0000 7f7191ffb700 1 -- 172.21.15.46:0/23622239 --> [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f71840bf160 con 0x7f71840bc5a0 2024-05-09T04:49:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.984+0000 7f7191ffb700 1 -- 172.21.15.46:0/23622239 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f718c0e2c90 con 0x7f7194109870 2024-05-09T04:49:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.984+0000 7f7193fff700 1 --2- 172.21.15.46:0/23622239 >> [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065] conn(0x7f71840bc5a0 0x7f71840bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.984+0000 7f7193fff700 1 --2- 172.21.15.46:0/23622239 >> [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065] conn(0x7f71840bc5a0 0x7f71840bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.984+0000 7f7198f53700 1 --2- 172.21.15.46:0/23622239 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f71840b2910 0x7f71840b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7188005950 tx=0x7f718800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:07.985+0000 7f7191ffb700 1 -- 172.21.15.46:0/23622239 <== osd.6 v2:172.21.15.46:6818/2429650065 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f71840bf160 con 0x7f71840bc5a0 2024-05-09T04:49:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:08.001+0000 7f719b1b7700 1 -- 172.21.15.46:0/23622239 --> [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7194064390 con 0x7f71840bc5a0 2024-05-09T04:49:10.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:08.002+0000 7f7191ffb700 1 -- 172.21.15.46:0/23622239 <== osd.6 v2:172.21.15.46:6818/2429650065 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f7194064390 con 0x7f71840bc5a0 2024-05-09T04:49:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:08.002+0000 7f719b1b7700 1 -- 172.21.15.46:0/23622239 >> [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065] conn(0x7f71840bc5a0 msgr2=0x7f71840bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:08.002+0000 7f719b1b7700 1 --2- 172.21.15.46:0/23622239 >> [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065] conn(0x7f71840bc5a0 0x7f71840bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:08.002+0000 7f719b1b7700 1 -- 172.21.15.46:0/23622239 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f71840b2910 msgr2=0x7f71840b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:08.002+0000 7f719b1b7700 1 --2- 172.21.15.46:0/23622239 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f71840b2910 0x7f71840b4e20 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f7188005950 tx=0x7f718800b410 comp rx=0 tx=0).stop 2024-05-09T04:49:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:08.002+0000 7f719b1b7700 1 -- 172.21.15.46:0/23622239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7194109870 msgr2=0x7f719419f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:08.002+0000 7f719b1b7700 1 --2- 172.21.15.46:0/23622239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7194109870 0x7f719419f660 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f718c00ef30 tx=0x7f718c00c3a0 comp rx=0 tx=0).stop 2024-05-09T04:49:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:08.002+0000 7f7199754700 1 -- 172.21.15.46:0/23622239 reap_dead start 2024-05-09T04:49:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:08.002+0000 7f719b1b7700 1 -- 172.21.15.46:0/23622239 shutdown_connections 2024-05-09T04:49:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:08.002+0000 7f719b1b7700 1 -- 172.21.15.46:0/23622239 >> 172.21.15.46:0/23622239 conn(0x7f71940ffec0 msgr2=0x7f719410a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:08.003+0000 7f719b1b7700 1 -- 172.21.15.46:0/23622239 shutdown_connections 2024-05-09T04:49:10.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:08.003+0000 7f719b1b7700 1 -- 172.21.15.46:0/23622239 wait complete. 2024-05-09T04:49:11.614 INFO:teuthology.orchestra.run.smithi046.stdout:188978561082 2024-05-09T04:49:11.615 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-09T04:49:12.671 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:12 smithi046 ceph-mon[30136]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:49:12.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:12 smithi110 ceph-mon[34904]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:49:12.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:12 smithi072 ceph-mon[34088]: pgmap v334: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:49:13.003 INFO:teuthology.orchestra.run.smithi046.stdout:115964117065 2024-05-09T04:49:13.004 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-09T04:49:13.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.207+0000 7f2676ac6700 1 -- 172.21.15.46:0/3173303597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2670107e10 msgr2=0x7f2670108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:13.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.207+0000 7f2676ac6700 1 --2- 172.21.15.46:0/3173303597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2670107e10 0x7f2670108220 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f265c0099c0 tx=0x7f265c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.207+0000 7f2676ac6700 1 -- 172.21.15.46:0/3173303597 shutdown_connections 2024-05-09T04:49:13.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.207+0000 7f2676ac6700 1 --2- 172.21.15.46:0/3173303597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2670108760 0x7f26701100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.207+0000 7f2676ac6700 1 --2- 172.21.15.46:0/3173303597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f26701034e0 0x7f26701038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.207+0000 7f2676ac6700 1 --2- 172.21.15.46:0/3173303597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2670107e10 0x7f2670108220 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.207+0000 7f2676ac6700 1 -- 172.21.15.46:0/3173303597 >> 172.21.15.46:0/3173303597 conn(0x7f26700fef30 msgr2=0x7f2670101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.207+0000 7f2676ac6700 1 -- 172.21.15.46:0/3173303597 shutdown_connections 2024-05-09T04:49:13.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.207+0000 7f2676ac6700 1 -- 172.21.15.46:0/3173303597 wait complete. 2024-05-09T04:49:13.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.208+0000 7f2676ac6700 1 Processor -- start 2024-05-09T04:49:13.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.208+0000 7f2676ac6700 1 -- start start 2024-05-09T04:49:13.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.208+0000 7f2676ac6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26701034e0 0x7f267019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:13.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.208+0000 7f2676ac6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2670107e10 0x7f267019b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:13.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.208+0000 7f2676ac6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2670108760 0x7f267019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.208+0000 7f2676ac6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f26701133c0 con 0x7f26701034e0 2024-05-09T04:49:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.208+0000 7f2676ac6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2670113240 con 0x7f2670107e10 2024-05-09T04:49:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.208+0000 7f2676ac6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2670113540 con 0x7f2670108760 2024-05-09T04:49:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.208+0000 7f2674862700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26701034e0 0x7f267019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.208+0000 7f2674862700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26701034e0 0x7f267019aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45108/0 (socket says 172.21.15.46:45108) 2024-05-09T04:49:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.208+0000 7f2674862700 1 -- 172.21.15.46:0/692847454 learned_addr learned my addr 172.21.15.46:0/692847454 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.209+0000 7f2675063700 1 --2- 172.21.15.46:0/692847454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2670108760 0x7f267019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.209+0000 7f266ffff700 1 --2- 172.21.15.46:0/692847454 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2670107e10 0x7f267019b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.209+0000 7f2674862700 1 -- 172.21.15.46:0/692847454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2670108760 msgr2=0x7f267019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.209+0000 7f2674862700 1 --2- 172.21.15.46:0/692847454 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2670108760 0x7f267019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.209+0000 7f2674862700 1 -- 172.21.15.46:0/692847454 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2670107e10 msgr2=0x7f267019b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:13.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.209+0000 7f2674862700 1 --2- 172.21.15.46:0/692847454 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2670107e10 0x7f267019b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.209+0000 7f2674862700 1 -- 172.21.15.46:0/692847454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f267019fd70 con 0x7f26701034e0 2024-05-09T04:49:13.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.209+0000 7f2674862700 1 --2- 172.21.15.46:0/692847454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26701034e0 0x7f267019aea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f266400d950 tx=0x7f266400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:13.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.209+0000 7f266dffb700 1 -- 172.21.15.46:0/692847454 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2664016070 con 0x7f26701034e0 2024-05-09T04:49:13.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.209+0000 7f2676ac6700 1 -- 172.21.15.46:0/692847454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26701a0060 con 0x7f26701034e0 2024-05-09T04:49:13.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.209+0000 7f266dffb700 1 -- 172.21.15.46:0/692847454 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f266400fd40 con 0x7f26701034e0 2024-05-09T04:49:13.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.210+0000 7f266dffb700 1 -- 172.21.15.46:0/692847454 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f266400e3b0 con 0x7f26701034e0 2024-05-09T04:49:13.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.210+0000 7f2676ac6700 1 -- 172.21.15.46:0/692847454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26700ff340 con 0x7f26701034e0 2024-05-09T04:49:13.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.211+0000 7f2676ac6700 1 -- 172.21.15.46:0/692847454 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2670063f10 con 0x7f26701034e0 2024-05-09T04:49:13.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.213+0000 7f266dffb700 1 -- 172.21.15.46:0/692847454 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2664019070 con 0x7f26701034e0 2024-05-09T04:49:13.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.214+0000 7f266dffb700 1 --2- 172.21.15.46:0/692847454 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f26600b2910 0x7f26600b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:13.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.214+0000 7f266dffb700 1 -- 172.21.15.46:0/692847454 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f26640e1bd0 con 0x7f26701034e0 2024-05-09T04:49:13.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.214+0000 7f266ffff700 1 --2- 172.21.15.46:0/692847454 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f26600b2910 0x7f26600b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:13.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.215+0000 7f266dffb700 1 --2- 172.21.15.46:0/692847454 >> [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227] conn(0x7f26600bc5a0 0x7f26600bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:13.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.215+0000 7f266dffb700 1 -- 172.21.15.46:0/692847454 --> [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f26600bf160 con 0x7f26600bc5a0 2024-05-09T04:49:13.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.215+0000 7f266dffb700 1 -- 172.21.15.46:0/692847454 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f26640e62a0 con 0x7f26701034e0 2024-05-09T04:49:13.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.215+0000 7f266ffff700 1 --2- 172.21.15.46:0/692847454 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f26600b2910 0x7f26600b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f265c006010 tx=0x7f265c009710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:13.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.215+0000 7f2675063700 1 --2- 172.21.15.46:0/692847454 >> [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227] conn(0x7f26600bc5a0 0x7f26600bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:13.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.216+0000 7f2675063700 1 --2- 172.21.15.46:0/692847454 >> [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227] conn(0x7f26600bc5a0 0x7f26600bea90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:13.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.216+0000 7f266dffb700 1 -- 172.21.15.46:0/692847454 <== osd.2 v2:172.21.15.110:6800/293447227 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f26600bf160 con 0x7f26600bc5a0 2024-05-09T04:49:13.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.235+0000 7f2676ac6700 1 -- 172.21.15.46:0/692847454 --> [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2670064390 con 0x7f26600bc5a0 2024-05-09T04:49:13.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.235+0000 7f266dffb700 1 -- 172.21.15.46:0/692847454 <== osd.2 v2:172.21.15.110:6800/293447227 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f2670064390 con 0x7f26600bc5a0 2024-05-09T04:49:13.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.235+0000 7f2676ac6700 1 -- 172.21.15.46:0/692847454 >> [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227] conn(0x7f26600bc5a0 msgr2=0x7f26600bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:13.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.236+0000 7f2676ac6700 1 --2- 172.21.15.46:0/692847454 >> [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227] conn(0x7f26600bc5a0 0x7f26600bea90 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:13.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.236+0000 7f2676ac6700 1 -- 172.21.15.46:0/692847454 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f26600b2910 msgr2=0x7f26600b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:13.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.236+0000 7f2676ac6700 1 --2- 172.21.15.46:0/692847454 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f26600b2910 0x7f26600b4e20 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f265c006010 tx=0x7f265c009710 comp rx=0 tx=0).stop 2024-05-09T04:49:13.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.236+0000 7f2676ac6700 1 -- 172.21.15.46:0/692847454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26701034e0 msgr2=0x7f267019aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:13.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.236+0000 7f2676ac6700 1 --2- 172.21.15.46:0/692847454 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f26701034e0 0x7f267019aea0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f266400d950 tx=0x7f266400dc60 comp rx=0 tx=0).stop 2024-05-09T04:49:13.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.236+0000 7f2675063700 1 -- 172.21.15.46:0/692847454 reap_dead start 2024-05-09T04:49:13.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.236+0000 7f2676ac6700 1 -- 172.21.15.46:0/692847454 shutdown_connections 2024-05-09T04:49:13.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.236+0000 7f2676ac6700 1 -- 172.21.15.46:0/692847454 >> 172.21.15.46:0/692847454 conn(0x7f26700fef30 msgr2=0x7f267006c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:13.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.236+0000 7f2676ac6700 1 -- 172.21.15.46:0/692847454 shutdown_connections 2024-05-09T04:49:13.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:13.236+0000 7f2676ac6700 1 -- 172.21.15.46:0/692847454 wait complete. 2024-05-09T04:49:14.682 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:14 smithi046 ceph-mon[30136]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:49:14.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.574+0000 7fe5c2715700 1 -- 172.21.15.46:0/2955662373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc107e10 msgr2=0x7fe5bc108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:14.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.574+0000 7fe5c2715700 1 --2- 172.21.15.46:0/2955662373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc107e10 0x7fe5bc108220 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40099c0 tx=0x7fe5a4009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:14.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.574+0000 7fe5c2715700 1 -- 172.21.15.46:0/2955662373 shutdown_connections 2024-05-09T04:49:14.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.574+0000 7fe5c2715700 1 --2- 172.21.15.46:0/2955662373 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5bc108760 0x7fe5bc1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:14.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.574+0000 7fe5c2715700 1 --2- 172.21.15.46:0/2955662373 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5bc1034e0 0x7fe5bc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:14.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.574+0000 7fe5c2715700 1 --2- 172.21.15.46:0/2955662373 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc107e10 0x7fe5bc108220 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:14.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.574+0000 7fe5c2715700 1 -- 172.21.15.46:0/2955662373 >> 172.21.15.46:0/2955662373 conn(0x7fe5bc0fef30 msgr2=0x7fe5bc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:14.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.575+0000 7fe5c2715700 1 -- 172.21.15.46:0/2955662373 shutdown_connections 2024-05-09T04:49:14.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.575+0000 7fe5c2715700 1 -- 172.21.15.46:0/2955662373 wait complete. 2024-05-09T04:49:14.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.575+0000 7fe5c2715700 1 Processor -- start 2024-05-09T04:49:14.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.575+0000 7fe5c2715700 1 -- start start 2024-05-09T04:49:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5c2715700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc1034e0 0x7fe5bc19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5c2715700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5bc107e10 0x7fe5bc19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5c2715700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5bc108760 0x7fe5bc19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5c2715700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc113370 con 0x7fe5bc1034e0 2024-05-09T04:49:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5c2715700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1131f0 con 0x7fe5bc108760 2024-05-09T04:49:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5c2715700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe5bc1134f0 con 0x7fe5bc107e10 2024-05-09T04:49:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc1034e0 0x7fe5bc19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5bbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc1034e0 0x7fe5bc19ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54354/0 (socket says 172.21.15.46:54354) 2024-05-09T04:49:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5bbfff700 1 -- 172.21.15.46:0/727444922 learned_addr learned my addr 172.21.15.46:0/727444922 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5b3fff700 1 --2- 172.21.15.46:0/727444922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5bc107e10 0x7fe5bc19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5c0cb2700 1 --2- 172.21.15.46:0/727444922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5bc108760 0x7fe5bc19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5bbfff700 1 -- 172.21.15.46:0/727444922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5bc107e10 msgr2=0x7fe5bc19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5bbfff700 1 --2- 172.21.15.46:0/727444922 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe5bc107e10 0x7fe5bc19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:14.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5bbfff700 1 -- 172.21.15.46:0/727444922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5bc108760 msgr2=0x7fe5bc19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:14.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5bbfff700 1 --2- 172.21.15.46:0/727444922 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe5bc108760 0x7fe5bc19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:14.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.576+0000 7fe5bbfff700 1 -- 172.21.15.46:0/727444922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5bc19fd40 con 0x7fe5bc1034e0 2024-05-09T04:49:14.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.577+0000 7fe5bbfff700 1 --2- 172.21.15.46:0/727444922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc1034e0 0x7fe5bc19ae70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00d950 tx=0x7fe5ac00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:14.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.577+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/727444922 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac016070 con 0x7fe5bc1034e0 2024-05-09T04:49:14.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.577+0000 7fe5c2715700 1 -- 172.21.15.46:0/727444922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5bc1a0030 con 0x7fe5bc1034e0 2024-05-09T04:49:14.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.577+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/727444922 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5ac010460 con 0x7fe5bc1034e0 2024-05-09T04:49:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.577+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/727444922 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac00f620 con 0x7fe5bc1034e0 2024-05-09T04:49:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.577+0000 7fe5c2715700 1 -- 172.21.15.46:0/727444922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5bc1a7950 con 0x7fe5bc1034e0 2024-05-09T04:49:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.578+0000 7fe5c2715700 1 -- 172.21.15.46:0/727444922 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe5bc063f10 con 0x7fe5bc1034e0 2024-05-09T04:49:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.579+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/727444922 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe5ac019070 con 0x7fe5bc1034e0 2024-05-09T04:49:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.580+0000 7fe5b9ffb700 1 --2- 172.21.15.46:0/727444922 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe5a80b2910 0x7fe5a80b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.580+0000 7fe5b3fff700 1 --2- 172.21.15.46:0/727444922 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe5a80b2910 0x7fe5a80b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.580+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/727444922 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fe5ac0e2550 con 0x7fe5bc1034e0 2024-05-09T04:49:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.581+0000 7fe5b9ffb700 1 --2- 172.21.15.46:0/727444922 >> [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502] conn(0x7fe5a80bc5a0 0x7fe5a80bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.581+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/727444922 --> [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe5ac00ae50 con 0x7fe5a80bc5a0 2024-05-09T04:49:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.581+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/727444922 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fe5ac0e2900 con 0x7fe5bc1034e0 2024-05-09T04:49:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.581+0000 7fe5b3fff700 1 --2- 172.21.15.46:0/727444922 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe5a80b2910 0x7fe5a80b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4000c00 tx=0x7fe5a401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.581+0000 7fe5c0cb2700 1 --2- 172.21.15.46:0/727444922 >> [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502] conn(0x7fe5a80bc5a0 0x7fe5a80bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:14.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.581+0000 7fe5c0cb2700 1 --2- 172.21.15.46:0/727444922 >> [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502] conn(0x7fe5a80bc5a0 0x7fe5a80bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.582+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/727444922 <== osd.5 v2:172.21.15.110:6808/1712162502 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe5ac00ae50 con 0x7fe5a80bc5a0 2024-05-09T04:49:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.599+0000 7fe5c2715700 1 -- 172.21.15.46:0/727444922 --> [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe5bc064390 con 0x7fe5a80bc5a0 2024-05-09T04:49:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.600+0000 7fe5b9ffb700 1 -- 172.21.15.46:0/727444922 <== osd.5 v2:172.21.15.110:6808/1712162502 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe5bc064390 con 0x7fe5a80bc5a0 2024-05-09T04:49:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.600+0000 7fe5c2715700 1 -- 172.21.15.46:0/727444922 >> [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502] conn(0x7fe5a80bc5a0 msgr2=0x7fe5a80bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.600+0000 7fe5c2715700 1 --2- 172.21.15.46:0/727444922 >> [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502] conn(0x7fe5a80bc5a0 0x7fe5a80bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.600+0000 7fe5c2715700 1 -- 172.21.15.46:0/727444922 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe5a80b2910 msgr2=0x7fe5a80b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.600+0000 7fe5c2715700 1 --2- 172.21.15.46:0/727444922 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe5a80b2910 0x7fe5a80b4e20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe5a4000c00 tx=0x7fe5a401a040 comp rx=0 tx=0).stop 2024-05-09T04:49:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.600+0000 7fe5c2715700 1 -- 172.21.15.46:0/727444922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc1034e0 msgr2=0x7fe5bc19ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:14.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.600+0000 7fe5c2715700 1 --2- 172.21.15.46:0/727444922 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe5bc1034e0 0x7fe5bc19ae70 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00d950 tx=0x7fe5ac00dd10 comp rx=0 tx=0).stop 2024-05-09T04:49:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.600+0000 7fe5c0cb2700 1 -- 172.21.15.46:0/727444922 reap_dead start 2024-05-09T04:49:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.600+0000 7fe5c2715700 1 -- 172.21.15.46:0/727444922 shutdown_connections 2024-05-09T04:49:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.600+0000 7fe5c2715700 1 -- 172.21.15.46:0/727444922 >> 172.21.15.46:0/727444922 conn(0x7fe5bc0fef30 msgr2=0x7fe5bc06c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.600+0000 7fe5c2715700 1 -- 172.21.15.46:0/727444922 shutdown_connections 2024-05-09T04:49:14.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.600+0000 7fe5c2715700 1 -- 172.21.15.46:0/727444922 wait complete. 2024-05-09T04:49:14.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:14 smithi110 ceph-mon[34904]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:49:14.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:14 smithi072 ceph-mon[34088]: pgmap v335: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:49:15.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.196+0000 7fa52f2a9700 1 -- 172.21.15.46:0/2532003200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa528107df0 msgr2=0x7fa528108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.196+0000 7fa52f2a9700 1 --2- 172.21.15.46:0/2532003200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa528107df0 0x7fa528108200 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa51c0099c0 tx=0x7fa51c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.196+0000 7fa52f2a9700 1 -- 172.21.15.46:0/2532003200 shutdown_connections 2024-05-09T04:49:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.196+0000 7fa52f2a9700 1 --2- 172.21.15.46:0/2532003200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa528108740 0x7fa5281100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.196+0000 7fa52f2a9700 1 --2- 172.21.15.46:0/2532003200 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5281034c0 0x7fa5281038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.196+0000 7fa52f2a9700 1 --2- 172.21.15.46:0/2532003200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa528107df0 0x7fa528108200 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.196+0000 7fa52f2a9700 1 -- 172.21.15.46:0/2532003200 >> 172.21.15.46:0/2532003200 conn(0x7fa5280fedc0 msgr2=0x7fa5281011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.196+0000 7fa52f2a9700 1 -- 172.21.15.46:0/2532003200 shutdown_connections 2024-05-09T04:49:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.196+0000 7fa52f2a9700 1 -- 172.21.15.46:0/2532003200 wait complete. 2024-05-09T04:49:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52f2a9700 1 Processor -- start 2024-05-09T04:49:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52f2a9700 1 -- start start 2024-05-09T04:49:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52f2a9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5281034c0 0x7fa52819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52f2a9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa528107df0 0x7fa52819c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52f2a9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa528108740 0x7fa52819c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52f2a9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa528113350 con 0x7fa528107df0 2024-05-09T04:49:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52f2a9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281131d0 con 0x7fa5281034c0 2024-05-09T04:49:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52f2a9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa5281134d0 con 0x7fa528108740 2024-05-09T04:49:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52c844700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa528107df0 0x7fa52819c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52c844700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa528107df0 0x7fa52819c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54396/0 (socket says 172.21.15.46:54396) 2024-05-09T04:49:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52c844700 1 -- 172.21.15.46:0/908807808 learned_addr learned my addr 172.21.15.46:0/908807808 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52d846700 1 --2- 172.21.15.46:0/908807808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa528108740 0x7fa52819c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52d045700 1 --2- 172.21.15.46:0/908807808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5281034c0 0x7fa52819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52c844700 1 -- 172.21.15.46:0/908807808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa528108740 msgr2=0x7fa52819c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52c844700 1 --2- 172.21.15.46:0/908807808 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa528108740 0x7fa52819c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:15.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52c844700 1 -- 172.21.15.46:0/908807808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5281034c0 msgr2=0x7fa52819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52c844700 1 --2- 172.21.15.46:0/908807808 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa5281034c0 0x7fa52819bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52c844700 1 -- 172.21.15.46:0/908807808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5281a6fe0 con 0x7fa528107df0 2024-05-09T04:49:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.197+0000 7fa52c844700 1 --2- 172.21.15.46:0/908807808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa528107df0 0x7fa52819c1d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa51c006010 tx=0x7fa51c0056a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.198+0000 7fa5267fc700 1 -- 172.21.15.46:0/908807808 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c026070 con 0x7fa528107df0 2024-05-09T04:49:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.198+0000 7fa52f2a9700 1 -- 172.21.15.46:0/908807808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5281a7270 con 0x7fa528107df0 2024-05-09T04:49:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.198+0000 7fa5267fc700 1 -- 172.21.15.46:0/908807808 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa51c00f460 con 0x7fa528107df0 2024-05-09T04:49:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.198+0000 7fa5267fc700 1 -- 172.21.15.46:0/908807808 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa51c009e60 con 0x7fa528107df0 2024-05-09T04:49:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.198+0000 7fa52f2a9700 1 -- 172.21.15.46:0/908807808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5281a7720 con 0x7fa528107df0 2024-05-09T04:49:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.198+0000 7fa52f2a9700 1 -- 172.21.15.46:0/908807808 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa528063e90 con 0x7fa528107df0 2024-05-09T04:49:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.200+0000 7fa5267fc700 1 -- 172.21.15.46:0/908807808 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa51c029070 con 0x7fa528107df0 2024-05-09T04:49:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.201+0000 7fa5267fc700 1 --2- 172.21.15.46:0/908807808 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa5100b2910 0x7fa5100b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.201+0000 7fa52d045700 1 --2- 172.21.15.46:0/908807808 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa5100b2910 0x7fa5100b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:15.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.201+0000 7fa5267fc700 1 -- 172.21.15.46:0/908807808 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fa51c0e3670 con 0x7fa528107df0 2024-05-09T04:49:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.201+0000 7fa5267fc700 1 --2- 172.21.15.46:0/908807808 >> [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672] conn(0x7fa5100bc5a0 0x7fa5100bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.201+0000 7fa5267fc700 1 -- 172.21.15.46:0/908807808 --> [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa51c004ea0 con 0x7fa5100bc5a0 2024-05-09T04:49:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.201+0000 7fa5267fc700 1 -- 172.21.15.46:0/908807808 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fa51c0e3ac0 con 0x7fa528107df0 2024-05-09T04:49:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.202+0000 7fa52d846700 1 --2- 172.21.15.46:0/908807808 >> [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672] conn(0x7fa5100bc5a0 0x7fa5100bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.202+0000 7fa52d045700 1 --2- 172.21.15.46:0/908807808 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa5100b2910 0x7fa5100b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa518009380 tx=0x7fa51800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.202+0000 7fa52d846700 1 --2- 172.21.15.46:0/908807808 >> [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672] conn(0x7fa5100bc5a0 0x7fa5100bea90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.203+0000 7fa5267fc700 1 -- 172.21.15.46:0/908807808 <== osd.7 v2:172.21.15.72:6816/337796672 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa51c004ea0 con 0x7fa5100bc5a0 2024-05-09T04:49:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.219+0000 7fa52f2a9700 1 -- 172.21.15.46:0/908807808 --> [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa5280642c0 con 0x7fa5100bc5a0 2024-05-09T04:49:15.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.220+0000 7fa5267fc700 1 -- 172.21.15.46:0/908807808 <== osd.7 v2:172.21.15.72:6816/337796672 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa5280642c0 con 0x7fa5100bc5a0 2024-05-09T04:49:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.220+0000 7fa52f2a9700 1 -- 172.21.15.46:0/908807808 >> [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672] conn(0x7fa5100bc5a0 msgr2=0x7fa5100bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.220+0000 7fa52f2a9700 1 --2- 172.21.15.46:0/908807808 >> [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672] conn(0x7fa5100bc5a0 0x7fa5100bea90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.220+0000 7fa52f2a9700 1 -- 172.21.15.46:0/908807808 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa5100b2910 msgr2=0x7fa5100b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.220+0000 7fa52f2a9700 1 --2- 172.21.15.46:0/908807808 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fa5100b2910 0x7fa5100b4e20 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa518009380 tx=0x7fa51800b040 comp rx=0 tx=0).stop 2024-05-09T04:49:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.220+0000 7fa52f2a9700 1 -- 172.21.15.46:0/908807808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa528107df0 msgr2=0x7fa52819c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.220+0000 7fa52f2a9700 1 --2- 172.21.15.46:0/908807808 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa528107df0 0x7fa52819c1d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa51c006010 tx=0x7fa51c0056a0 comp rx=0 tx=0).stop 2024-05-09T04:49:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.220+0000 7fa52d846700 1 -- 172.21.15.46:0/908807808 reap_dead start 2024-05-09T04:49:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.220+0000 7fa52f2a9700 1 -- 172.21.15.46:0/908807808 shutdown_connections 2024-05-09T04:49:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.220+0000 7fa52f2a9700 1 -- 172.21.15.46:0/908807808 >> 172.21.15.46:0/908807808 conn(0x7fa5280fedc0 msgr2=0x7fa5281096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.220+0000 7fa52f2a9700 1 -- 172.21.15.46:0/908807808 shutdown_connections 2024-05-09T04:49:15.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:15.221+0000 7fa52f2a9700 1 -- 172.21.15.46:0/908807808 wait complete. 2024-05-09T04:49:15.999 INFO:teuthology.orchestra.run.smithi046.stdout:210453397559 2024-05-09T04:49:16.000 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-09T04:49:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.682+0000 7f1849e32700 1 -- 172.21.15.46:0/3439849437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1844108f00 msgr2=0x7f1844109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.682+0000 7f1849e32700 1 --2- 172.21.15.46:0/3439849437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1844108f00 0x7f1844109310 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f182c0099c0 tx=0x7f182c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.682+0000 7f1849e32700 1 -- 172.21.15.46:0/3439849437 shutdown_connections 2024-05-09T04:49:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.682+0000 7f1849e32700 1 --2- 172.21.15.46:0/3439849437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1844109850 0x7f18441111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.682+0000 7f1849e32700 1 --2- 172.21.15.46:0/3439849437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18441045d0 0x7f18441049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.682+0000 7f1849e32700 1 --2- 172.21.15.46:0/3439849437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1844108f00 0x7f1844109310 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.682+0000 7f1849e32700 1 -- 172.21.15.46:0/3439849437 >> 172.21.15.46:0/3439849437 conn(0x7f18440ffea0 msgr2=0x7f18441022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:16.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.682+0000 7f1849e32700 1 -- 172.21.15.46:0/3439849437 shutdown_connections 2024-05-09T04:49:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.682+0000 7f1849e32700 1 -- 172.21.15.46:0/3439849437 wait complete. 2024-05-09T04:49:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.683+0000 7f1849e32700 1 Processor -- start 2024-05-09T04:49:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.683+0000 7f1849e32700 1 -- start start 2024-05-09T04:49:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.684+0000 7f1849e32700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18441045d0 0x7f184419aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.684+0000 7f1849e32700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1844108f00 0x7f184419b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.684+0000 7f1849e32700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1844109850 0x7f184419f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.684+0000 7f1849e32700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f18441144d0 con 0x7f1844109850 2024-05-09T04:49:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.684+0000 7f1849e32700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1844114350 con 0x7f1844108f00 2024-05-09T04:49:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.684+0000 7f1849e32700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1844114650 con 0x7f18441045d0 2024-05-09T04:49:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.684+0000 7f1843fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1844109850 0x7f184419f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.684+0000 7f1843fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1844109850 0x7f184419f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54378/0 (socket says 172.21.15.46:54378) 2024-05-09T04:49:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.684+0000 7f1843fff700 1 -- 172.21.15.46:0/170778187 learned_addr learned my addr 172.21.15.46:0/170778187 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:16.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.684+0000 7f183affd700 1 --2- 172.21.15.46:0/170778187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1844108f00 0x7f184419b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.684+0000 7f18437fe700 1 --2- 172.21.15.46:0/170778187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18441045d0 0x7f184419aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.684+0000 7f1843fff700 1 -- 172.21.15.46:0/170778187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18441045d0 msgr2=0x7f184419aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.684+0000 7f1843fff700 1 --2- 172.21.15.46:0/170778187 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f18441045d0 0x7f184419aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.685+0000 7f1843fff700 1 -- 172.21.15.46:0/170778187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1844108f00 msgr2=0x7f184419b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.685+0000 7f1843fff700 1 --2- 172.21.15.46:0/170778187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1844108f00 0x7f184419b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.685+0000 7f1843fff700 1 -- 172.21.15.46:0/170778187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f184419fd90 con 0x7f1844109850 2024-05-09T04:49:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.685+0000 7f183affd700 1 --2- 172.21.15.46:0/170778187 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1844108f00 0x7f184419b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:49:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.685+0000 7f1843fff700 1 --2- 172.21.15.46:0/170778187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1844109850 0x7f184419f6f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f183c00b970 tx=0x7f183c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.685+0000 7f18417fa700 1 -- 172.21.15.46:0/170778187 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c019070 con 0x7f1844109850 2024-05-09T04:49:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.685+0000 7f1849e32700 1 -- 172.21.15.46:0/170778187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18441a0080 con 0x7f1844109850 2024-05-09T04:49:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.685+0000 7f18417fa700 1 -- 172.21.15.46:0/170778187 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f183c00cde0 con 0x7f1844109850 2024-05-09T04:49:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.685+0000 7f18417fa700 1 -- 172.21.15.46:0/170778187 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183c007370 con 0x7f1844109850 2024-05-09T04:49:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.685+0000 7f1849e32700 1 -- 172.21.15.46:0/170778187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18441a79a0 con 0x7f1844109850 2024-05-09T04:49:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.686+0000 7f1849e32700 1 -- 172.21.15.46:0/170778187 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1844063ed0 con 0x7f1844109850 2024-05-09T04:49:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.688+0000 7f18417fa700 1 -- 172.21.15.46:0/170778187 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f183c01c070 con 0x7f1844109850 2024-05-09T04:49:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.689+0000 7f18417fa700 1 --2- 172.21.15.46:0/170778187 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f18300b2910 0x7f18300b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.689+0000 7f18437fe700 1 --2- 172.21.15.46:0/170778187 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f18300b2910 0x7f18300b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.689+0000 7f18417fa700 1 -- 172.21.15.46:0/170778187 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f183c0e2510 con 0x7f1844109850 2024-05-09T04:49:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.689+0000 7f18417fa700 1 --2- 172.21.15.46:0/170778187 >> [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693] conn(0x7f18300bc5a0 0x7f18300bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.689+0000 7f18417fa700 1 -- 172.21.15.46:0/170778187 --> [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f18300bf160 con 0x7f18300bc5a0 2024-05-09T04:49:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.689+0000 7f18417fa700 1 -- 172.21.15.46:0/170778187 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f183c0e2960 con 0x7f1844109850 2024-05-09T04:49:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.689+0000 7f18437fe700 1 --2- 172.21.15.46:0/170778187 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f18300b2910 0x7f18300b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1834005950 tx=0x7f18340058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.689+0000 7f183affd700 1 --2- 172.21.15.46:0/170778187 >> [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693] conn(0x7f18300bc5a0 0x7f18300bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.690+0000 7f183affd700 1 --2- 172.21.15.46:0/170778187 >> [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693] conn(0x7f18300bc5a0 0x7f18300bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.691+0000 7f18417fa700 1 -- 172.21.15.46:0/170778187 <== osd.4 v2:172.21.15.72:6808/59186693 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f18300bf160 con 0x7f18300bc5a0 2024-05-09T04:49:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.707+0000 7f1849e32700 1 -- 172.21.15.46:0/170778187 --> [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f18440642d0 con 0x7f18300bc5a0 2024-05-09T04:49:16.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.707+0000 7f18417fa700 1 -- 172.21.15.46:0/170778187 <== osd.4 v2:172.21.15.72:6808/59186693 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f18440642d0 con 0x7f18300bc5a0 2024-05-09T04:49:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.708+0000 7f1849e32700 1 -- 172.21.15.46:0/170778187 >> [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693] conn(0x7f18300bc5a0 msgr2=0x7f18300bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.708+0000 7f1849e32700 1 --2- 172.21.15.46:0/170778187 >> [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693] conn(0x7f18300bc5a0 0x7f18300bea90 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.708+0000 7f1849e32700 1 -- 172.21.15.46:0/170778187 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f18300b2910 msgr2=0x7f18300b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.708+0000 7f1849e32700 1 --2- 172.21.15.46:0/170778187 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f18300b2910 0x7f18300b4e20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1834005950 tx=0x7f18340058e0 comp rx=0 tx=0).stop 2024-05-09T04:49:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.708+0000 7f1849e32700 1 -- 172.21.15.46:0/170778187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1844109850 msgr2=0x7f184419f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.708+0000 7f1849e32700 1 --2- 172.21.15.46:0/170778187 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1844109850 0x7f184419f6f0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f183c00b970 tx=0x7f183c00bc80 comp rx=0 tx=0).stop 2024-05-09T04:49:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.708+0000 7f1843fff700 1 -- 172.21.15.46:0/170778187 reap_dead start 2024-05-09T04:49:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.708+0000 7f1849e32700 1 -- 172.21.15.46:0/170778187 shutdown_connections 2024-05-09T04:49:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.708+0000 7f1849e32700 1 -- 172.21.15.46:0/170778187 >> 172.21.15.46:0/170778187 conn(0x7f18440ffea0 msgr2=0x7f184410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.708+0000 7f1849e32700 1 -- 172.21.15.46:0/170778187 shutdown_connections 2024-05-09T04:49:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:14.708+0000 7f1849e32700 1 -- 172.21.15.46:0/170778187 wait complete. 2024-05-09T04:49:16.779 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:16 smithi046 ceph-mon[30136]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:49:16.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:16 smithi110 ceph-mon[34904]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:49:16.809 INFO:teuthology.orchestra.run.smithi046.stdout:141733920837 2024-05-09T04:49:16.809 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-09T04:49:16.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:16 smithi072 ceph-mon[34088]: pgmap v336: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:49:17.218 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:17.277 INFO:teuthology.orchestra.run.smithi046.stdout:163208757313 2024-05-09T04:49:17.278 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-09T04:49:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.900+0000 7f75234f4700 1 -- 172.21.15.46:0/3331026096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f751c108f00 msgr2=0x7f751c109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.900+0000 7f75234f4700 1 --2- 172.21.15.46:0/3331026096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f751c108f00 0x7f751c109310 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f750c0099c0 tx=0x7f750c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.900+0000 7f75234f4700 1 -- 172.21.15.46:0/3331026096 shutdown_connections 2024-05-09T04:49:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.900+0000 7f75234f4700 1 --2- 172.21.15.46:0/3331026096 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f751c109850 0x7f751c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.900+0000 7f75234f4700 1 --2- 172.21.15.46:0/3331026096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f751c1045d0 0x7f751c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.900+0000 7f75234f4700 1 --2- 172.21.15.46:0/3331026096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f751c108f00 0x7f751c109310 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.900+0000 7f75234f4700 1 -- 172.21.15.46:0/3331026096 >> 172.21.15.46:0/3331026096 conn(0x7f751c0ffea0 msgr2=0x7f751c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:17.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.900+0000 7f75234f4700 1 -- 172.21.15.46:0/3331026096 shutdown_connections 2024-05-09T04:49:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.900+0000 7f75234f4700 1 -- 172.21.15.46:0/3331026096 wait complete. 2024-05-09T04:49:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.901+0000 7f75234f4700 1 Processor -- start 2024-05-09T04:49:17.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.901+0000 7f75234f4700 1 -- start start 2024-05-09T04:49:17.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.901+0000 7f75234f4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f751c1045d0 0x7f751c19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:17.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.901+0000 7f75234f4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f751c108f00 0x7f751c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:17.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f75234f4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f751c109850 0x7f751c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f75234f4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f751c114460 con 0x7f751c109850 2024-05-09T04:49:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f75234f4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f751c1142e0 con 0x7f751c108f00 2024-05-09T04:49:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f75234f4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f751c1145e0 con 0x7f751c1045d0 2024-05-09T04:49:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f7521a91700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f751c109850 0x7f751c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f7521a91700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f751c109850 0x7f751c19f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54412/0 (socket says 172.21.15.46:54412) 2024-05-09T04:49:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f7521a91700 1 -- 172.21.15.46:0/2665730971 learned_addr learned my addr 172.21.15.46:0/2665730971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f7521290700 1 --2- 172.21.15.46:0/2665730971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f751c1045d0 0x7f751c19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f7520a8f700 1 --2- 172.21.15.46:0/2665730971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f751c108f00 0x7f751c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f7521a91700 1 -- 172.21.15.46:0/2665730971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f751c1045d0 msgr2=0x7f751c19ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f7521a91700 1 --2- 172.21.15.46:0/2665730971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f751c1045d0 0x7f751c19ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f7521a91700 1 -- 172.21.15.46:0/2665730971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f751c108f00 msgr2=0x7f751c19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f7521a91700 1 --2- 172.21.15.46:0/2665730971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f751c108f00 0x7f751c19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f7521a91700 1 -- 172.21.15.46:0/2665730971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f751c19fd40 con 0x7f751c109850 2024-05-09T04:49:17.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f7520a8f700 1 --2- 172.21.15.46:0/2665730971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f751c108f00 0x7f751c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:49:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.902+0000 7f7521a91700 1 --2- 172.21.15.46:0/2665730971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f751c109850 0x7f751c19f6a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7514007ee0 tx=0x7f751400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.903+0000 7f75127fc700 1 -- 172.21.15.46:0/2665730971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f751400ddd0 con 0x7f751c109850 2024-05-09T04:49:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.903+0000 7f75127fc700 1 -- 172.21.15.46:0/2665730971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7514004020 con 0x7f751c109850 2024-05-09T04:49:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.903+0000 7f75127fc700 1 -- 172.21.15.46:0/2665730971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7514015660 con 0x7f751c109850 2024-05-09T04:49:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.903+0000 7f75234f4700 1 -- 172.21.15.46:0/2665730971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f751c1a0030 con 0x7f751c109850 2024-05-09T04:49:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.903+0000 7f75234f4700 1 -- 172.21.15.46:0/2665730971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f751c076b00 con 0x7f751c109850 2024-05-09T04:49:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.904+0000 7f75234f4700 1 -- 172.21.15.46:0/2665730971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f751c063f10 con 0x7f751c109850 2024-05-09T04:49:17.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.906+0000 7f75127fc700 1 -- 172.21.15.46:0/2665730971 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f75140041d0 con 0x7f751c109850 2024-05-09T04:49:17.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.907+0000 7f75127fc700 1 --2- 172.21.15.46:0/2665730971 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75080b6da0 0x7f75080b92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.907+0000 7f7521290700 1 --2- 172.21.15.46:0/2665730971 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75080b6da0 0x7f75080b92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.907+0000 7f75127fc700 1 -- 172.21.15.46:0/2665730971 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f75140e2890 con 0x7f751c109850 2024-05-09T04:49:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.907+0000 7f75127fc700 1 --2- 172.21.15.46:0/2665730971 >> [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632] conn(0x7f75080c0a30 0x7f75080c2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.907+0000 7f75127fc700 1 -- 172.21.15.46:0/2665730971 --> [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75080c35f0 con 0x7f75080c0a30 2024-05-09T04:49:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.907+0000 7f75127fc700 1 -- 172.21.15.46:0/2665730971 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7f75140e2ce0 con 0x7f751c109850 2024-05-09T04:49:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.907+0000 7f7521290700 1 --2- 172.21.15.46:0/2665730971 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75080b6da0 0x7f75080b92b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7518009c60 tx=0x7f7518009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:17.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.907+0000 7f7520a8f700 1 --2- 172.21.15.46:0/2665730971 >> [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632] conn(0x7f75080c0a30 0x7f75080c2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:17.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.908+0000 7f7520a8f700 1 --2- 172.21.15.46:0/2665730971 >> [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632] conn(0x7f75080c0a30 0x7f75080c2f20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:17.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.909+0000 7f75127fc700 1 -- 172.21.15.46:0/2665730971 <== osd.11 v2:172.21.15.110:6824/4153529632 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75080c35f0 con 0x7f75080c0a30 2024-05-09T04:49:17.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.927+0000 7f75234f4700 1 -- 172.21.15.46:0/2665730971 --> [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f751c064310 con 0x7f75080c0a30 2024-05-09T04:49:17.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.928+0000 7f75127fc700 1 -- 172.21.15.46:0/2665730971 <== osd.11 v2:172.21.15.110:6824/4153529632 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f751c064310 con 0x7f75080c0a30 2024-05-09T04:49:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.928+0000 7f75234f4700 1 -- 172.21.15.46:0/2665730971 >> [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632] conn(0x7f75080c0a30 msgr2=0x7f75080c2f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.928+0000 7f75234f4700 1 --2- 172.21.15.46:0/2665730971 >> [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632] conn(0x7f75080c0a30 0x7f75080c2f20 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.928+0000 7f75234f4700 1 -- 172.21.15.46:0/2665730971 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75080b6da0 msgr2=0x7f75080b92b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.928+0000 7f75234f4700 1 --2- 172.21.15.46:0/2665730971 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75080b6da0 0x7f75080b92b0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7518009c60 tx=0x7f7518009260 comp rx=0 tx=0).stop 2024-05-09T04:49:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.928+0000 7f75234f4700 1 -- 172.21.15.46:0/2665730971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f751c109850 msgr2=0x7f751c19f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.928+0000 7f75234f4700 1 --2- 172.21.15.46:0/2665730971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f751c109850 0x7f751c19f6a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7514007ee0 tx=0x7f751400d3b0 comp rx=0 tx=0).stop 2024-05-09T04:49:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.928+0000 7f7521a91700 1 -- 172.21.15.46:0/2665730971 reap_dead start 2024-05-09T04:49:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.929+0000 7f75234f4700 1 -- 172.21.15.46:0/2665730971 shutdown_connections 2024-05-09T04:49:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.929+0000 7f75234f4700 1 -- 172.21.15.46:0/2665730971 >> 172.21.15.46:0/2665730971 conn(0x7f751c0ffea0 msgr2=0x7f751c10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.929+0000 7f75234f4700 1 -- 172.21.15.46:0/2665730971 shutdown_connections 2024-05-09T04:49:17.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:17.929+0000 7f75234f4700 1 -- 172.21.15.46:0/2665730971 wait complete. 2024-05-09T04:49:18.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.479+0000 7fea085df700 1 -- 172.21.15.46:0/661162528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea0010e920 msgr2=0x7fea00110e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:18.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.479+0000 7fea085df700 1 --2- 172.21.15.46:0/661162528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea0010e920 0x7fea00110e00 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe9f800b0a0 tx=0x7fe9f800b3b0 comp rx=0 tx=0).stop 2024-05-09T04:49:18.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.479+0000 7fea085df700 1 -- 172.21.15.46:0/661162528 shutdown_connections 2024-05-09T04:49:18.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.479+0000 7fea085df700 1 --2- 172.21.15.46:0/661162528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea0010e920 0x7fea00110e00 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:18.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.479+0000 7fea085df700 1 --2- 172.21.15.46:0/661162528 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea001054f0 0x7fea00105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:18.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.479+0000 7fea085df700 1 --2- 172.21.15.46:0/661162528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea00105e40 0x7fea0010e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:18.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.480+0000 7fea085df700 1 -- 172.21.15.46:0/661162528 >> 172.21.15.46:0/661162528 conn(0x7fea00078e00 msgr2=0x7fea00079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:18.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.480+0000 7fea085df700 1 -- 172.21.15.46:0/661162528 shutdown_connections 2024-05-09T04:49:18.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.480+0000 7fea085df700 1 -- 172.21.15.46:0/661162528 wait complete. 2024-05-09T04:49:18.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.480+0000 7fea085df700 1 Processor -- start 2024-05-09T04:49:18.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea085df700 1 -- start start 2024-05-09T04:49:18.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea085df700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea001054f0 0x7fea0019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:18.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea085df700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea00105e40 0x7fea0019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:18.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea085df700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea0010e920 0x7fea0019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea085df700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fea00114480 con 0x7fea001054f0 2024-05-09T04:49:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea085df700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fea00114300 con 0x7fea0010e920 2024-05-09T04:49:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea085df700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea00114600 con 0x7fea00105e40 2024-05-09T04:49:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea0637b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea001054f0 0x7fea0019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea0637b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea001054f0 0x7fea0019add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54434/0 (socket says 172.21.15.46:54434) 2024-05-09T04:49:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea0637b700 1 -- 172.21.15.46:0/1558872805 learned_addr learned my addr 172.21.15.46:0/1558872805 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea05b7a700 1 --2- 172.21.15.46:0/1558872805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea00105e40 0x7fea0019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea06b7c700 1 --2- 172.21.15.46:0/1558872805 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea0010e920 0x7fea0019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea0637b700 1 -- 172.21.15.46:0/1558872805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea00105e40 msgr2=0x7fea0019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea0637b700 1 --2- 172.21.15.46:0/1558872805 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea00105e40 0x7fea0019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea0637b700 1 -- 172.21.15.46:0/1558872805 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea0010e920 msgr2=0x7fea0019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea0637b700 1 --2- 172.21.15.46:0/1558872805 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea0010e920 0x7fea0019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.481+0000 7fea0637b700 1 -- 172.21.15.46:0/1558872805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea0019fc50 con 0x7fea001054f0 2024-05-09T04:49:18.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.482+0000 7fea0637b700 1 --2- 172.21.15.46:0/1558872805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea001054f0 0x7fea0019add0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00d950 tx=0x7fe9fc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:18.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.482+0000 7fe9f77fe700 1 -- 172.21.15.46:0/1558872805 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc016070 con 0x7fea001054f0 2024-05-09T04:49:18.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.482+0000 7fea085df700 1 -- 172.21.15.46:0/1558872805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea0019ff40 con 0x7fea001054f0 2024-05-09T04:49:18.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.482+0000 7fe9f77fe700 1 -- 172.21.15.46:0/1558872805 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9fc00fd40 con 0x7fea001054f0 2024-05-09T04:49:18.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.482+0000 7fe9f77fe700 1 -- 172.21.15.46:0/1558872805 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc00e3b0 con 0x7fea001054f0 2024-05-09T04:49:18.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.482+0000 7fea085df700 1 -- 172.21.15.46:0/1558872805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea001a7860 con 0x7fea001054f0 2024-05-09T04:49:18.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.483+0000 7fea085df700 1 -- 172.21.15.46:0/1558872805 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fea00063f50 con 0x7fea001054f0 2024-05-09T04:49:18.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.484+0000 7fe9f77fe700 1 -- 172.21.15.46:0/1558872805 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe9fc019070 con 0x7fea001054f0 2024-05-09T04:49:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.485+0000 7fe9f77fe700 1 --2- 172.21.15.46:0/1558872805 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe9ec0b2910 0x7fe9ec0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.485+0000 7fea05b7a700 1 --2- 172.21.15.46:0/1558872805 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe9ec0b2910 0x7fe9ec0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.485+0000 7fe9f77fe700 1 -- 172.21.15.46:0/1558872805 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7fe9fc0e2390 con 0x7fea001054f0 2024-05-09T04:49:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.485+0000 7fe9f77fe700 1 --2- 172.21.15.46:0/1558872805 >> [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508] conn(0x7fe9ec0bc5a0 0x7fe9ec0bea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.485+0000 7fe9f77fe700 1 -- 172.21.15.46:0/1558872805 --> [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe9ec0bf160 con 0x7fe9ec0bc5a0 2024-05-09T04:49:18.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.485+0000 7fe9f77fe700 1 -- 172.21.15.46:0/1558872805 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_get_version_reply(handle=1 version=82) ==== 24+0+0 (secure 0 0 0) 0x7fe9fc0e27e0 con 0x7fea001054f0 2024-05-09T04:49:18.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.486+0000 7fea06b7c700 1 --2- 172.21.15.46:0/1558872805 >> [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508] conn(0x7fe9ec0bc5a0 0x7fe9ec0bea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:18.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.486+0000 7fea05b7a700 1 --2- 172.21.15.46:0/1558872805 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe9ec0b2910 0x7fe9ec0b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea00110700 tx=0x7fe9f000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:18.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.486+0000 7fea06b7c700 1 --2- 172.21.15.46:0/1558872805 >> [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508] conn(0x7fe9ec0bc5a0 0x7fe9ec0bea90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:18.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.487+0000 7fe9f77fe700 1 -- 172.21.15.46:0/1558872805 <== osd.0 v2:172.21.15.46:6802/1576436508 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe9ec0bf160 con 0x7fe9ec0bc5a0 2024-05-09T04:49:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.503+0000 7fea085df700 1 -- 172.21.15.46:0/1558872805 --> [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fea00064350 con 0x7fe9ec0bc5a0 2024-05-09T04:49:18.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.503+0000 7fe9f77fe700 1 -- 172.21.15.46:0/1558872805 <== osd.0 v2:172.21.15.46:6802/1576436508 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fea00064350 con 0x7fe9ec0bc5a0 2024-05-09T04:49:18.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.504+0000 7fea085df700 1 -- 172.21.15.46:0/1558872805 >> [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508] conn(0x7fe9ec0bc5a0 msgr2=0x7fe9ec0bea90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:18.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.504+0000 7fea085df700 1 --2- 172.21.15.46:0/1558872805 >> [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508] conn(0x7fe9ec0bc5a0 0x7fe9ec0bea90 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:18.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.504+0000 7fea085df700 1 -- 172.21.15.46:0/1558872805 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe9ec0b2910 msgr2=0x7fe9ec0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:18.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.504+0000 7fea085df700 1 --2- 172.21.15.46:0/1558872805 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe9ec0b2910 0x7fe9ec0b4e20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fea00110700 tx=0x7fe9f000b410 comp rx=0 tx=0).stop 2024-05-09T04:49:18.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.504+0000 7fea085df700 1 -- 172.21.15.46:0/1558872805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea001054f0 msgr2=0x7fea0019add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:18.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.504+0000 7fea085df700 1 --2- 172.21.15.46:0/1558872805 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea001054f0 0x7fea0019add0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc00d950 tx=0x7fe9fc00dc60 comp rx=0 tx=0).stop 2024-05-09T04:49:18.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.504+0000 7fea06b7c700 1 -- 172.21.15.46:0/1558872805 reap_dead start 2024-05-09T04:49:18.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.504+0000 7fea085df700 1 -- 172.21.15.46:0/1558872805 shutdown_connections 2024-05-09T04:49:18.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.504+0000 7fea085df700 1 -- 172.21.15.46:0/1558872805 >> 172.21.15.46:0/1558872805 conn(0x7fea00078e00 msgr2=0x7fea001000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:18.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.504+0000 7fea085df700 1 -- 172.21.15.46:0/1558872805 shutdown_connections 2024-05-09T04:49:18.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:18.504+0000 7fea085df700 1 -- 172.21.15.46:0/1558872805 wait complete. 2024-05-09T04:49:18.682 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:18 smithi072 ceph-mon[34088]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:49:18.804 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:18 smithi046 ceph-mon[30136]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:49:18.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:18 smithi110 ceph-mon[34904]: pgmap v337: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:49:19.019 INFO:teuthology.orchestra.run.smithi046.stdout:77309411408 2024-05-09T04:49:19.020 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-09T04:49:20.444 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:20 smithi046 ceph-mon[30136]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-05-09T04:49:20.652 INFO:teuthology.orchestra.run.smithi046.stdout:309237645346 2024-05-09T04:49:20.652 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-09T04:49:20.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:20 smithi110 ceph-mon[34904]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-05-09T04:49:20.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:20 smithi072 ceph-mon[34088]: pgmap v338: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2024-05-09T04:49:21.462 INFO:teuthology.orchestra.run.smithi046.stdout:34359738458 2024-05-09T04:49:21.463 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-09T04:49:21.684 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:21.688 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:21.694 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:21.696 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:22.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:21 smithi046 ceph-mon[30136]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T04:49:22.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:21 smithi110 ceph-mon[34904]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T04:49:22.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:21 smithi072 ceph-mon[34088]: pgmap v339: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-05-09T04:49:22.613 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:23.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:22 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1140955422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T04:49:23.306 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:22 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1140955422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T04:49:23.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:22 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1140955422' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-09T04:49:23.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.329+0000 7f982ad0d700 1 -- 172.21.15.46:0/2807535772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9824105500 msgr2=0x7f9824105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:23.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.329+0000 7f982ad0d700 1 --2- 172.21.15.46:0/2807535772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9824105500 0x7f9824105910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f9818009990 tx=0x7f9818009ca0 comp rx=0 tx=0).stop 2024-05-09T04:49:23.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.329+0000 7f982ad0d700 1 -- 172.21.15.46:0/2807535772 shutdown_connections 2024-05-09T04:49:23.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.329+0000 7f982ad0d700 1 --2- 172.21.15.46:0/2807535772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f982410e8e0 0x7f9824110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:23.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.329+0000 7f982ad0d700 1 --2- 172.21.15.46:0/2807535772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9824105500 0x7f9824105910 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:23.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.329+0000 7f982ad0d700 1 --2- 172.21.15.46:0/2807535772 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9824105e50 0x7f982410e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:23.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.329+0000 7f982ad0d700 1 -- 172.21.15.46:0/2807535772 >> 172.21.15.46:0/2807535772 conn(0x7f98240fedc0 msgr2=0x7f98241011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:23.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.329+0000 7f982ad0d700 1 -- 172.21.15.46:0/2807535772 shutdown_connections 2024-05-09T04:49:23.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.329+0000 7f982ad0d700 1 -- 172.21.15.46:0/2807535772 wait complete. 2024-05-09T04:49:23.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.329+0000 7f982ad0d700 1 Processor -- start 2024-05-09T04:49:23.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f982ad0d700 1 -- start start 2024-05-09T04:49:23.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f982ad0d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9824105500 0x7f982419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:23.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f982ad0d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9824105e50 0x7f982419b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:23.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f982ad0d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f982410e8e0 0x7f982419f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:23.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f982ad0d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f98241144b0 con 0x7f9824105500 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f982ad0d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9824114330 con 0x7f9824105e50 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f982ad0d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9824114630 con 0x7f982410e8e0 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f9828aa9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9824105500 0x7f982419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f9828aa9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9824105500 0x7f982419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54450/0 (socket says 172.21.15.46:54450) 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f9828aa9700 1 -- 172.21.15.46:0/1140955422 learned_addr learned my addr 172.21.15.46:0/1140955422 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f9823fff700 1 --2- 172.21.15.46:0/1140955422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9824105e50 0x7f982419b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f98292aa700 1 --2- 172.21.15.46:0/1140955422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f982410e8e0 0x7f982419f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f9828aa9700 1 -- 172.21.15.46:0/1140955422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f982410e8e0 msgr2=0x7f982419f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f9828aa9700 1 --2- 172.21.15.46:0/1140955422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f982410e8e0 0x7f982419f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f9828aa9700 1 -- 172.21.15.46:0/1140955422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9824105e50 msgr2=0x7f982419b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f9828aa9700 1 --2- 172.21.15.46:0/1140955422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9824105e50 0x7f982419b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f9828aa9700 1 -- 172.21.15.46:0/1140955422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982419fcf0 con 0x7f9824105500 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.330+0000 7f9828aa9700 1 --2- 172.21.15.46:0/1140955422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9824105500 0x7f982419ae20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9818000c00 tx=0x7f98180047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.331+0000 7f9821ffb700 1 -- 172.21.15.46:0/1140955422 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9818027070 con 0x7f9824105500 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.331+0000 7f9821ffb700 1 -- 172.21.15.46:0/1140955422 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98180040e0 con 0x7f9824105500 2024-05-09T04:49:23.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.331+0000 7f982ad0d700 1 -- 172.21.15.46:0/1140955422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982419ff80 con 0x7f9824105500 2024-05-09T04:49:23.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.331+0000 7f9821ffb700 1 -- 172.21.15.46:0/1140955422 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f981800f460 con 0x7f9824105500 2024-05-09T04:49:23.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.331+0000 7f982ad0d700 1 -- 172.21.15.46:0/1140955422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98241a7900 con 0x7f9824105500 2024-05-09T04:49:23.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.332+0000 7f982ad0d700 1 -- 172.21.15.46:0/1140955422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98240512c0 con 0x7f9824105500 2024-05-09T04:49:23.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.333+0000 7f9821ffb700 1 -- 172.21.15.46:0/1140955422 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f981802a070 con 0x7f9824105500 2024-05-09T04:49:23.445 INFO:teuthology.orchestra.run.smithi046.stdout:261993005100 2024-05-09T04:49:23.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.334+0000 7f9821ffb700 1 --2- 172.21.15.46:0/1140955422 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f98140b2990 0x7f98140b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:23.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.334+0000 7f9823fff700 1 --2- 172.21.15.46:0/1140955422 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f98140b2990 0x7f98140b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.334+0000 7f9821ffb700 1 -- 172.21.15.46:0/1140955422 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8269+0+0 (secure 0 0 0) 0x7f98180e3330 con 0x7f9824105500 2024-05-09T04:49:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.335+0000 7f9823fff700 1 --2- 172.21.15.46:0/1140955422 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f98140b2990 0x7f98140b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9824102270 tx=0x7f981000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.339+0000 7f9821ffb700 1 -- 172.21.15.46:0/1140955422 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9818013070 con 0x7f9824105500 2024-05-09T04:49:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.568+0000 7f982ad0d700 1 -- 172.21.15.46:0/1140955422 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f982402e5f0 con 0x7f9824105500 2024-05-09T04:49:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.568+0000 7f9821ffb700 1 -- 172.21.15.46:0/1140955422 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f98180aaf60 con 0x7f9824105500 2024-05-09T04:49:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.571+0000 7f982ad0d700 1 -- 172.21.15.46:0/1140955422 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f98140b2990 msgr2=0x7f98140b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.571+0000 7f982ad0d700 1 --2- 172.21.15.46:0/1140955422 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f98140b2990 0x7f98140b4ea0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9824102270 tx=0x7f981000a300 comp rx=0 tx=0).stop 2024-05-09T04:49:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.571+0000 7f982ad0d700 1 -- 172.21.15.46:0/1140955422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9824105500 msgr2=0x7f982419ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.571+0000 7f982ad0d700 1 --2- 172.21.15.46:0/1140955422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9824105500 0x7f982419ae20 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f9818000c00 tx=0x7f98180047b0 comp rx=0 tx=0).stop 2024-05-09T04:49:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.571+0000 7f982ad0d700 1 -- 172.21.15.46:0/1140955422 shutdown_connections 2024-05-09T04:49:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.571+0000 7f982ad0d700 1 --2- 172.21.15.46:0/1140955422 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f98140b2990 0x7f98140b4ea0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.571+0000 7f982ad0d700 1 --2- 172.21.15.46:0/1140955422 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f982410e8e0 0x7f982419f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:23.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.571+0000 7f982ad0d700 1 --2- 172.21.15.46:0/1140955422 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9824105500 0x7f982419ae20 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.571+0000 7f982ad0d700 1 --2- 172.21.15.46:0/1140955422 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9824105e50 0x7f982419b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.571+0000 7f982ad0d700 1 -- 172.21.15.46:0/1140955422 >> 172.21.15.46:0/1140955422 conn(0x7f98240fedc0 msgr2=0x7f98240ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.571+0000 7f982ad0d700 1 -- 172.21.15.46:0/1140955422 shutdown_connections 2024-05-09T04:49:23.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:22.571+0000 7f982ad0d700 1 -- 172.21.15.46:0/1140955422 wait complete. 2024-05-09T04:49:24.152 INFO:tasks.cephadm.ceph_manager.ceph:need seq 261993005096 got 261993005100 for osd.9 2024-05-09T04:49:24.152 DEBUG:teuthology.parallel:result is None 2024-05-09T04:49:24.230 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:23 smithi046 ceph-mon[30136]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:49:24.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:23 smithi110 ceph-mon[34904]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:49:24.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:23 smithi072 ceph-mon[34088]: pgmap v340: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:49:25.579 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:26.582 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:26 smithi046 ceph-mon[30136]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:49:26.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:26 smithi072 ceph-mon[34088]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:49:26.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:26 smithi110 ceph-mon[34904]: pgmap v341: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:49:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:26.886 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:28.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:28 smithi072 ceph-mon[34088]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:49:28.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:28 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-09T04:49:28.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:28 smithi110 ceph-mon[34904]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:49:28.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:28 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-09T04:49:28.824 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:28 smithi046 ceph-mon[30136]: pgmap v342: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:49:28.824 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:28 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]: dispatch 2024-05-09T04:49:29.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:29 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-09T04:49:29.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:29 smithi046 ceph-mon[30136]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T04:49:29.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:29 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-09T04:49:29.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:29 smithi110 ceph-mon[34904]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T04:49:29.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:29 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1c", "id": [8, 11]}]': finished 2024-05-09T04:49:29.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:29 smithi072 ceph-mon[34088]: osdmap e83: 12 total, 12 up, 12 in 2024-05-09T04:49:30.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:30 smithi046 ceph-mon[30136]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:49:30.642 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:30 smithi046 ceph-mon[30136]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T04:49:30.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:30 smithi110 ceph-mon[34904]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:49:30.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:30 smithi110 ceph-mon[34904]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T04:49:30.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:30 smithi072 ceph-mon[34088]: pgmap v344: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:49:30.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:30 smithi072 ceph-mon[34088]: osdmap e84: 12 total, 12 up, 12 in 2024-05-09T04:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.507+0000 7fded11b1700 1 -- 172.21.15.46:0/229207001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc1034c0 msgr2=0x7fdecc1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.507+0000 7fded11b1700 1 --2- 172.21.15.46:0/229207001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc1034c0 0x7fdecc1038d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0099c0 tx=0x7fdebc009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.507+0000 7fded11b1700 1 -- 172.21.15.46:0/229207001 shutdown_connections 2024-05-09T04:49:31.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.507+0000 7fded11b1700 1 --2- 172.21.15.46:0/229207001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdecc108740 0x7fdecc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.507+0000 7fded11b1700 1 --2- 172.21.15.46:0/229207001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc1034c0 0x7fdecc1038d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.507+0000 7fded11b1700 1 --2- 172.21.15.46:0/229207001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdecc107df0 0x7fdecc108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.507+0000 7fded11b1700 1 -- 172.21.15.46:0/229207001 >> 172.21.15.46:0/229207001 conn(0x7fdecc0fedc0 msgr2=0x7fdecc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.507+0000 7fded11b1700 1 -- 172.21.15.46:0/229207001 shutdown_connections 2024-05-09T04:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.507+0000 7fded11b1700 1 -- 172.21.15.46:0/229207001 wait complete. 2024-05-09T04:49:31.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.508+0000 7fded11b1700 1 Processor -- start 2024-05-09T04:49:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.508+0000 7fded11b1700 1 -- start start 2024-05-09T04:49:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.508+0000 7fded11b1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdecc1034c0 0x7fdecc19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:31.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.508+0000 7fded11b1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc107df0 0x7fdecc19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:31.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.508+0000 7fded11b1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdecc108740 0x7fdecc19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:31.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.508+0000 7fded11b1700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc113370 con 0x7fdecc107df0 2024-05-09T04:49:31.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.508+0000 7fded11b1700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc1131f0 con 0x7fdecc108740 2024-05-09T04:49:31.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.508+0000 7fded11b1700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdecc1134f0 con 0x7fdecc1034c0 2024-05-09T04:49:31.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.508+0000 7fdeca59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc107df0 0x7fdecc19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:31.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.508+0000 7fdeca59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc107df0 0x7fdecc19b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40898/0 (socket says 172.21.15.46:40898) 2024-05-09T04:49:31.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.508+0000 7fdeca59c700 1 -- 172.21.15.46:0/3267814889 learned_addr learned my addr 172.21.15.46:0/3267814889 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:31.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.508+0000 7fdecad9d700 1 --2- 172.21.15.46:0/3267814889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdecc1034c0 0x7fdecc19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:31.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.508+0000 7fdecb59e700 1 --2- 172.21.15.46:0/3267814889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdecc108740 0x7fdecc19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:31.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.509+0000 7fdeca59c700 1 -- 172.21.15.46:0/3267814889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdecc1034c0 msgr2=0x7fdecc19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:31.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.509+0000 7fdeca59c700 1 --2- 172.21.15.46:0/3267814889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdecc1034c0 0x7fdecc19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:31.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.509+0000 7fdeca59c700 1 -- 172.21.15.46:0/3267814889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdecc108740 msgr2=0x7fdecc19f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:31.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.509+0000 7fdeca59c700 1 --2- 172.21.15.46:0/3267814889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdecc108740 0x7fdecc19f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.509+0000 7fdeca59c700 1 -- 172.21.15.46:0/3267814889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdecc19fce0 con 0x7fdecc107df0 2024-05-09T04:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.509+0000 7fdeca59c700 1 --2- 172.21.15.46:0/3267814889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc107df0 0x7fdecc19b3b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdec000e970 tx=0x7fdec000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.509+0000 7fdeb3fff700 1 -- 172.21.15.46:0/3267814889 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0004a30 con 0x7fdecc107df0 2024-05-09T04:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.509+0000 7fdeb3fff700 1 -- 172.21.15.46:0/3267814889 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdec000cc90 con 0x7fdecc107df0 2024-05-09T04:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.509+0000 7fded11b1700 1 -- 172.21.15.46:0/3267814889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdecc19ffd0 con 0x7fdecc107df0 2024-05-09T04:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.509+0000 7fdeb3fff700 1 -- 172.21.15.46:0/3267814889 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdec0005610 con 0x7fdecc107df0 2024-05-09T04:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.509+0000 7fded11b1700 1 -- 172.21.15.46:0/3267814889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdecc1a78f0 con 0x7fdecc107df0 2024-05-09T04:49:31.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.510+0000 7fded11b1700 1 -- 172.21.15.46:0/3267814889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdecc0512c0 con 0x7fdecc107df0 2024-05-09T04:49:31.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.513+0000 7fdeb3fff700 1 -- 172.21.15.46:0/3267814889 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fdec000c7e0 con 0x7fdecc107df0 2024-05-09T04:49:31.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.514+0000 7fdeb3fff700 1 --2- 172.21.15.46:0/3267814889 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fdeb40b2910 0x7fdeb40b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:31.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.514+0000 7fdeb3fff700 1 -- 172.21.15.46:0/3267814889 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fdec00e2810 con 0x7fdecc107df0 2024-05-09T04:49:31.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.514+0000 7fdecad9d700 1 --2- 172.21.15.46:0/3267814889 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fdeb40b2910 0x7fdeb40b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:31.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.515+0000 7fdecad9d700 1 --2- 172.21.15.46:0/3267814889 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fdeb40b2910 0x7fdeb40b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdebc000c00 tx=0x7fdebc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:31.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.517+0000 7fdeb3fff700 1 -- 172.21.15.46:0/3267814889 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdec0054df0 con 0x7fdecc107df0 2024-05-09T04:49:31.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.743+0000 7fded11b1700 1 -- 172.21.15.46:0/3267814889 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fdecc19c170 con 0x7fdecc107df0 2024-05-09T04:49:31.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.743+0000 7fdeb3fff700 1 -- 172.21.15.46:0/3267814889 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fdec00aa330 con 0x7fdecc107df0 2024-05-09T04:49:31.743 INFO:teuthology.orchestra.run.smithi046.stdout:163208757316 2024-05-09T04:49:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.746+0000 7fded11b1700 1 -- 172.21.15.46:0/3267814889 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fdeb40b2910 msgr2=0x7fdeb40b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.746+0000 7fded11b1700 1 --2- 172.21.15.46:0/3267814889 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fdeb40b2910 0x7fdeb40b4e20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fdebc000c00 tx=0x7fdebc01a040 comp rx=0 tx=0).stop 2024-05-09T04:49:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.746+0000 7fded11b1700 1 -- 172.21.15.46:0/3267814889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc107df0 msgr2=0x7fdecc19b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.746+0000 7fded11b1700 1 --2- 172.21.15.46:0/3267814889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc107df0 0x7fdecc19b3b0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fdec000e970 tx=0x7fdec000ec80 comp rx=0 tx=0).stop 2024-05-09T04:49:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.746+0000 7fded11b1700 1 -- 172.21.15.46:0/3267814889 shutdown_connections 2024-05-09T04:49:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.746+0000 7fded11b1700 1 --2- 172.21.15.46:0/3267814889 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fdeb40b2910 0x7fdeb40b4e20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:31.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.746+0000 7fded11b1700 1 --2- 172.21.15.46:0/3267814889 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdecc108740 0x7fdecc19f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.746+0000 7fded11b1700 1 --2- 172.21.15.46:0/3267814889 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdecc1034c0 0x7fdecc19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.746+0000 7fded11b1700 1 --2- 172.21.15.46:0/3267814889 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdecc107df0 0x7fdecc19b3b0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.746+0000 7fded11b1700 1 -- 172.21.15.46:0/3267814889 >> 172.21.15.46:0/3267814889 conn(0x7fdecc0fedc0 msgr2=0x7fdecc06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.746+0000 7fded11b1700 1 -- 172.21.15.46:0/3267814889 shutdown_connections 2024-05-09T04:49:31.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:31.746+0000 7fded11b1700 1 -- 172.21.15.46:0/3267814889 wait complete. 2024-05-09T04:49:32.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.159+0000 7feb5d57c700 1 -- 172.21.15.46:0/1865999141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb58105e50 msgr2=0x7feb5810e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:32.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.159+0000 7feb5d57c700 1 --2- 172.21.15.46:0/1865999141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb58105e50 0x7feb5810e3a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7feb4c009990 tx=0x7feb4c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.159+0000 7feb5d57c700 1 -- 172.21.15.46:0/1865999141 shutdown_connections 2024-05-09T04:49:32.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.159+0000 7feb5d57c700 1 --2- 172.21.15.46:0/1865999141 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb5810e930 0x7feb58110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.159+0000 7feb5d57c700 1 --2- 172.21.15.46:0/1865999141 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb58105500 0x7feb58105910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.159+0000 7feb5d57c700 1 --2- 172.21.15.46:0/1865999141 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb58105e50 0x7feb5810e3a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.159+0000 7feb5d57c700 1 -- 172.21.15.46:0/1865999141 >> 172.21.15.46:0/1865999141 conn(0x7feb580fedf0 msgr2=0x7feb58101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:32.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.159+0000 7feb5d57c700 1 -- 172.21.15.46:0/1865999141 shutdown_connections 2024-05-09T04:49:32.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.159+0000 7feb5d57c700 1 -- 172.21.15.46:0/1865999141 wait complete. 2024-05-09T04:49:32.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.160+0000 7feb5d57c700 1 Processor -- start 2024-05-09T04:49:32.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.160+0000 7feb5d57c700 1 -- start start 2024-05-09T04:49:32.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.160+0000 7feb5d57c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb58105500 0x7feb5819ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:32.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.160+0000 7feb5d57c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb58105e50 0x7feb5819b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:32.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.160+0000 7feb5d57c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb5810e930 0x7feb5819f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:32.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.160+0000 7feb5d57c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7feb581144e0 con 0x7feb58105e50 2024-05-09T04:49:32.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.160+0000 7feb5d57c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7feb58114360 con 0x7feb5810e930 2024-05-09T04:49:32.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.160+0000 7feb5d57c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7feb58114660 con 0x7feb58105500 2024-05-09T04:49:32.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.160+0000 7feb567fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb58105e50 0x7feb5819b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:32.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.160+0000 7feb567fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb58105e50 0x7feb5819b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40914/0 (socket says 172.21.15.46:40914) 2024-05-09T04:49:32.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.160+0000 7feb567fc700 1 -- 172.21.15.46:0/2225605786 learned_addr learned my addr 172.21.15.46:0/2225605786 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:32.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.161+0000 7feb56ffd700 1 --2- 172.21.15.46:0/2225605786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb58105500 0x7feb5819ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:32.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.161+0000 7feb577fe700 1 --2- 172.21.15.46:0/2225605786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb5810e930 0x7feb5819f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:32.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.161+0000 7feb567fc700 1 -- 172.21.15.46:0/2225605786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb58105500 msgr2=0x7feb5819ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:32.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.161+0000 7feb567fc700 1 --2- 172.21.15.46:0/2225605786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb58105500 0x7feb5819ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.161+0000 7feb567fc700 1 -- 172.21.15.46:0/2225605786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb5810e930 msgr2=0x7feb5819f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:32.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.161+0000 7feb567fc700 1 --2- 172.21.15.46:0/2225605786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb5810e930 0x7feb5819f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.161+0000 7feb567fc700 1 -- 172.21.15.46:0/2225605786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb5819fd20 con 0x7feb58105e50 2024-05-09T04:49:32.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.161+0000 7feb567fc700 1 --2- 172.21.15.46:0/2225605786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb58105e50 0x7feb5819b3f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feb4c000c00 tx=0x7feb4c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:32.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.161+0000 7feb3ffff700 1 -- 172.21.15.46:0/2225605786 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c027070 con 0x7feb58105e50 2024-05-09T04:49:32.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.161+0000 7feb5d57c700 1 -- 172.21.15.46:0/2225605786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb5819ffb0 con 0x7feb58105e50 2024-05-09T04:49:32.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.161+0000 7feb3ffff700 1 -- 172.21.15.46:0/2225605786 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb4c003f20 con 0x7feb58105e50 2024-05-09T04:49:32.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.161+0000 7feb3ffff700 1 -- 172.21.15.46:0/2225605786 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb4c022ca0 con 0x7feb58105e50 2024-05-09T04:49:32.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.161+0000 7feb5d57c700 1 -- 172.21.15.46:0/2225605786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb581a0490 con 0x7feb58105e50 2024-05-09T04:49:32.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.163+0000 7feb5d57c700 1 -- 172.21.15.46:0/2225605786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb580512c0 con 0x7feb58105e50 2024-05-09T04:49:32.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.164+0000 7feb3ffff700 1 -- 172.21.15.46:0/2225605786 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7feb4c02a070 con 0x7feb58105e50 2024-05-09T04:49:32.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.165+0000 7feb3ffff700 1 --2- 172.21.15.46:0/2225605786 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7feb400b2910 0x7feb400b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:32.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.165+0000 7feb56ffd700 1 --2- 172.21.15.46:0/2225605786 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7feb400b2910 0x7feb400b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:32.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.165+0000 7feb3ffff700 1 -- 172.21.15.46:0/2225605786 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7feb4c0e32b0 con 0x7feb58105e50 2024-05-09T04:49:32.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.165+0000 7feb56ffd700 1 --2- 172.21.15.46:0/2225605786 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7feb400b2910 0x7feb400b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feb5806d150 tx=0x7feb48009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:32.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.170+0000 7feb3ffff700 1 -- 172.21.15.46:0/2225605786 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feb4c013070 con 0x7feb58105e50 2024-05-09T04:49:32.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.392+0000 7feb5d57c700 1 -- 172.21.15.46:0/2225605786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7feb5802e5f0 con 0x7feb58105e50 2024-05-09T04:49:32.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.393+0000 7feb3ffff700 1 -- 172.21.15.46:0/2225605786 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7feb4c0aae60 con 0x7feb58105e50 2024-05-09T04:49:32.393 INFO:teuthology.orchestra.run.smithi046.stdout:210453397562 2024-05-09T04:49:32.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.395+0000 7feb5d57c700 1 -- 172.21.15.46:0/2225605786 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7feb400b2910 msgr2=0x7feb400b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:32.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.395+0000 7feb5d57c700 1 --2- 172.21.15.46:0/2225605786 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7feb400b2910 0x7feb400b4e20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7feb5806d150 tx=0x7feb48009190 comp rx=0 tx=0).stop 2024-05-09T04:49:32.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.395+0000 7feb5d57c700 1 -- 172.21.15.46:0/2225605786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb58105e50 msgr2=0x7feb5819b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:32.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.395+0000 7feb5d57c700 1 --2- 172.21.15.46:0/2225605786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb58105e50 0x7feb5819b3f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7feb4c000c00 tx=0x7feb4c0047b0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.395+0000 7feb5d57c700 1 -- 172.21.15.46:0/2225605786 shutdown_connections 2024-05-09T04:49:32.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.395+0000 7feb5d57c700 1 --2- 172.21.15.46:0/2225605786 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7feb400b2910 0x7feb400b4e20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.395+0000 7feb5d57c700 1 --2- 172.21.15.46:0/2225605786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7feb5810e930 0x7feb5819f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.395+0000 7feb5d57c700 1 --2- 172.21.15.46:0/2225605786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7feb58105500 0x7feb5819ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.395+0000 7feb5d57c700 1 --2- 172.21.15.46:0/2225605786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7feb58105e50 0x7feb5819b3f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.395+0000 7feb5d57c700 1 -- 172.21.15.46:0/2225605786 >> 172.21.15.46:0/2225605786 conn(0x7feb580fedf0 msgr2=0x7feb58100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:32.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.395+0000 7feb5d57c700 1 -- 172.21.15.46:0/2225605786 shutdown_connections 2024-05-09T04:49:32.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.395+0000 7feb5d57c700 1 -- 172.21.15.46:0/2225605786 wait complete. 2024-05-09T04:49:32.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.563+0000 7fb3a607c700 1 -- 172.21.15.46:0/3476662830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3a0108f10 msgr2=0x7fb3a0109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:32.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.563+0000 7fb3a607c700 1 --2- 172.21.15.46:0/3476662830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3a0108f10 0x7fb3a0109320 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb3940099c0 tx=0x7fb394009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.563+0000 7fb3a607c700 1 -- 172.21.15.46:0/3476662830 shutdown_connections 2024-05-09T04:49:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.563+0000 7fb3a607c700 1 --2- 172.21.15.46:0/3476662830 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb3a0109860 0x7fb3a01111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.563+0000 7fb3a607c700 1 --2- 172.21.15.46:0/3476662830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3a01045e0 0x7fb3a01049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.563+0000 7fb3a607c700 1 --2- 172.21.15.46:0/3476662830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3a0108f10 0x7fb3a0109320 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.563+0000 7fb3a607c700 1 -- 172.21.15.46:0/3476662830 >> 172.21.15.46:0/3476662830 conn(0x7fb3a00ffed0 msgr2=0x7fb3a0102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.563+0000 7fb3a607c700 1 -- 172.21.15.46:0/3476662830 shutdown_connections 2024-05-09T04:49:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.563+0000 7fb3a607c700 1 -- 172.21.15.46:0/3476662830 wait complete. 2024-05-09T04:49:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.564+0000 7fb3a607c700 1 Processor -- start 2024-05-09T04:49:32.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.564+0000 7fb3a607c700 1 -- start start 2024-05-09T04:49:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb3a607c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb3a01045e0 0x7fb3a0076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb3a607c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3a0108f10 0x7fb3a0077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb3a607c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3a0109860 0x7fb3a0077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb3a607c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a0114470 con 0x7fb3a0109860 2024-05-09T04:49:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb3a607c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a01142f0 con 0x7fb3a01045e0 2024-05-09T04:49:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb3a607c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb3a01145f0 con 0x7fb3a0108f10 2024-05-09T04:49:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb39ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3a0109860 0x7fb3a0077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:32.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb39effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3a0108f10 0x7fb3a0077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb39f7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb3a01045e0 0x7fb3a0076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb39ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3a0109860 0x7fb3a0077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40932/0 (socket says 172.21.15.46:40932) 2024-05-09T04:49:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb39ffff700 1 -- 172.21.15.46:0/1628613308 learned_addr learned my addr 172.21.15.46:0/1628613308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb39ffff700 1 -- 172.21.15.46:0/1628613308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3a0108f10 msgr2=0x7fb3a0077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb39ffff700 1 --2- 172.21.15.46:0/1628613308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3a0108f10 0x7fb3a0077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb39ffff700 1 -- 172.21.15.46:0/1628613308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb3a01045e0 msgr2=0x7fb3a0076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb39ffff700 1 --2- 172.21.15.46:0/1628613308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb3a01045e0 0x7fb3a0076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb39ffff700 1 -- 172.21.15.46:0/1628613308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3a01a91b0 con 0x7fb3a0109860 2024-05-09T04:49:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb39effd700 1 --2- 172.21.15.46:0/1628613308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3a0108f10 0x7fb3a0077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:49:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.565+0000 7fb39ffff700 1 --2- 172.21.15.46:0/1628613308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3a0109860 0x7fb3a0077650 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb398007d70 tx=0x7fb39800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.566+0000 7fb39cff9700 1 -- 172.21.15.46:0/1628613308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb398016020 con 0x7fb3a0109860 2024-05-09T04:49:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.566+0000 7fb3a607c700 1 -- 172.21.15.46:0/1628613308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3a01a94a0 con 0x7fb3a0109860 2024-05-09T04:49:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.566+0000 7fb39cff9700 1 -- 172.21.15.46:0/1628613308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb39800ac90 con 0x7fb3a0109860 2024-05-09T04:49:32.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.566+0000 7fb39cff9700 1 -- 172.21.15.46:0/1628613308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3980115e0 con 0x7fb3a0109860 2024-05-09T04:49:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.566+0000 7fb3a607c700 1 -- 172.21.15.46:0/1628613308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3a01a99b0 con 0x7fb3a0109860 2024-05-09T04:49:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.567+0000 7fb3a607c700 1 -- 172.21.15.46:0/1628613308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3a00512c0 con 0x7fb3a0109860 2024-05-09T04:49:32.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.569+0000 7fb39cff9700 1 -- 172.21.15.46:0/1628613308 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb398019020 con 0x7fb3a0109860 2024-05-09T04:49:32.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.570+0000 7fb39cff9700 1 --2- 172.21.15.46:0/1628613308 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb3880b2910 0x7fb3880b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:32.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.570+0000 7fb39f7fe700 1 --2- 172.21.15.46:0/1628613308 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb3880b2910 0x7fb3880b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:32.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.571+0000 7fb39cff9700 1 -- 172.21.15.46:0/1628613308 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fb3980e26b0 con 0x7fb3a0109860 2024-05-09T04:49:32.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.571+0000 7fb39f7fe700 1 --2- 172.21.15.46:0/1628613308 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb3880b2910 0x7fb3880b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb390009c60 tx=0x7fb390009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:32.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.577+0000 7fb39cff9700 1 -- 172.21.15.46:0/1628613308 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb398020020 con 0x7fb3a0109860 2024-05-09T04:49:32.657 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:32 smithi046 ceph-mon[30136]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T04:49:32.657 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:32 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3267814889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T04:49:32.657 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:32 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2225605786' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T04:49:32.795 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.795+0000 7fb3a607c700 1 -- 172.21.15.46:0/1628613308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fb3a0078100 con 0x7fb3a0109860 2024-05-09T04:49:32.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.796+0000 7fb39cff9700 1 -- 172.21.15.46:0/1628613308 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fb3980aa340 con 0x7fb3a0109860 2024-05-09T04:49:32.796 INFO:teuthology.orchestra.run.smithi046.stdout:283467841576 2024-05-09T04:49:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.798+0000 7fb3a607c700 1 -- 172.21.15.46:0/1628613308 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb3880b2910 msgr2=0x7fb3880b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.798+0000 7fb3a607c700 1 --2- 172.21.15.46:0/1628613308 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb3880b2910 0x7fb3880b4e20 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb390009c60 tx=0x7fb390009260 comp rx=0 tx=0).stop 2024-05-09T04:49:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.798+0000 7fb3a607c700 1 -- 172.21.15.46:0/1628613308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3a0109860 msgr2=0x7fb3a0077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.798+0000 7fb3a607c700 1 --2- 172.21.15.46:0/1628613308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3a0109860 0x7fb3a0077650 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb398007d70 tx=0x7fb39800a340 comp rx=0 tx=0).stop 2024-05-09T04:49:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.799+0000 7fb3a607c700 1 -- 172.21.15.46:0/1628613308 shutdown_connections 2024-05-09T04:49:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.799+0000 7fb3a607c700 1 --2- 172.21.15.46:0/1628613308 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb3880b2910 0x7fb3880b4e20 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.799+0000 7fb3a607c700 1 --2- 172.21.15.46:0/1628613308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb3a0109860 0x7fb3a0077650 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.799+0000 7fb3a607c700 1 --2- 172.21.15.46:0/1628613308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb3a01045e0 0x7fb3a0076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.799+0000 7fb3a607c700 1 --2- 172.21.15.46:0/1628613308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb3a0108f10 0x7fb3a0077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.799+0000 7fb3a607c700 1 -- 172.21.15.46:0/1628613308 >> 172.21.15.46:0/1628613308 conn(0x7fb3a00ffed0 msgr2=0x7fb3a01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:32.799 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.799+0000 7fb3a607c700 1 -- 172.21.15.46:0/1628613308 shutdown_connections 2024-05-09T04:49:32.800 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:32.799+0000 7fb3a607c700 1 -- 172.21.15.46:0/1628613308 wait complete. 2024-05-09T04:49:32.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:32 smithi110 ceph-mon[34904]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T04:49:32.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:32 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3267814889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T04:49:32.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:32 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2225605786' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T04:49:32.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:32 smithi072 ceph-mon[34088]: pgmap v346: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T04:49:32.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:32 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3267814889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-09T04:49:32.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:32 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2225605786' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-09T04:49:33.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:33 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1628613308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T04:49:33.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:33 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1628613308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T04:49:33.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:33 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1628613308' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-09T04:49:34.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.488+0000 7fd0fbafd700 1 -- 172.21.15.46:0/1654857976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0f4107df0 msgr2=0x7fd0f4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:34.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.488+0000 7fd0fbafd700 1 --2- 172.21.15.46:0/1654857976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0f4107df0 0x7fd0f4108200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd0e40099c0 tx=0x7fd0e4009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:34.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.488+0000 7fd0fbafd700 1 -- 172.21.15.46:0/1654857976 shutdown_connections 2024-05-09T04:49:34.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.488+0000 7fd0fbafd700 1 --2- 172.21.15.46:0/1654857976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0f4108740 0x7fd0f41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:34.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.488+0000 7fd0fbafd700 1 --2- 172.21.15.46:0/1654857976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f41034c0 0x7fd0f41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.488+0000 7fd0fbafd700 1 --2- 172.21.15.46:0/1654857976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0f4107df0 0x7fd0f4108200 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:34.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.488+0000 7fd0fbafd700 1 -- 172.21.15.46:0/1654857976 >> 172.21.15.46:0/1654857976 conn(0x7fd0f40fedc0 msgr2=0x7fd0f41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.489+0000 7fd0fbafd700 1 -- 172.21.15.46:0/1654857976 shutdown_connections 2024-05-09T04:49:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.489+0000 7fd0fbafd700 1 -- 172.21.15.46:0/1654857976 wait complete. 2024-05-09T04:49:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.489+0000 7fd0fbafd700 1 Processor -- start 2024-05-09T04:49:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.489+0000 7fd0fbafd700 1 -- start start 2024-05-09T04:49:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0fbafd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0f41034c0 0x7fd0f4199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:34.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0fbafd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0f4107df0 0x7fd0f419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0fbafd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f4108740 0x7fd0f419e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0fbafd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f4113350 con 0x7fd0f4107df0 2024-05-09T04:49:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0fbafd700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f41131d0 con 0x7fd0f4108740 2024-05-09T04:49:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0fbafd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd0f41134d0 con 0x7fd0f41034c0 2024-05-09T04:49:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0f9098700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0f4107df0 0x7fd0f419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0f9098700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0f4107df0 0x7fd0f419a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51978/0 (socket says 172.21.15.46:51978) 2024-05-09T04:49:34.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0f9098700 1 -- 172.21.15.46:0/1307535434 learned_addr learned my addr 172.21.15.46:0/1307535434 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0f9899700 1 --2- 172.21.15.46:0/1307535434 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0f41034c0 0x7fd0f4199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0fa09a700 1 --2- 172.21.15.46:0/1307535434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f4108740 0x7fd0f419e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0f9098700 1 -- 172.21.15.46:0/1307535434 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0f41034c0 msgr2=0x7fd0f4199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0f9098700 1 --2- 172.21.15.46:0/1307535434 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0f41034c0 0x7fd0f4199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0f9098700 1 -- 172.21.15.46:0/1307535434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f4108740 msgr2=0x7fd0f419e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0f9098700 1 --2- 172.21.15.46:0/1307535434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f4108740 0x7fd0f419e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0f9098700 1 -- 172.21.15.46:0/1307535434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd0f419ebf0 con 0x7fd0f4107df0 2024-05-09T04:49:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.490+0000 7fd0f9098700 1 --2- 172.21.15.46:0/1307535434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0f4107df0 0x7fd0f419a2c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4006010 tx=0x7fd0e40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.491+0000 7fd0eaffd700 1 -- 172.21.15.46:0/1307535434 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4027070 con 0x7fd0f4107df0 2024-05-09T04:49:34.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.491+0000 7fd0eaffd700 1 -- 172.21.15.46:0/1307535434 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0e4003fa0 con 0x7fd0f4107df0 2024-05-09T04:49:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.491+0000 7fd0eaffd700 1 -- 172.21.15.46:0/1307535434 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0e4022dd0 con 0x7fd0f4107df0 2024-05-09T04:49:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.491+0000 7fd0fbafd700 1 -- 172.21.15.46:0/1307535434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0f419ee80 con 0x7fd0f4107df0 2024-05-09T04:49:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.491+0000 7fd0fbafd700 1 -- 172.21.15.46:0/1307535434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0f419f360 con 0x7fd0f4107df0 2024-05-09T04:49:34.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.492+0000 7fd0fbafd700 1 -- 172.21.15.46:0/1307535434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd0f40512c0 con 0x7fd0f4107df0 2024-05-09T04:49:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.493+0000 7fd0eaffd700 1 -- 172.21.15.46:0/1307535434 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fd0e402a070 con 0x7fd0f4107df0 2024-05-09T04:49:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.494+0000 7fd0eaffd700 1 --2- 172.21.15.46:0/1307535434 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd0e00b2910 0x7fd0e00b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.494+0000 7fd0f9899700 1 --2- 172.21.15.46:0/1307535434 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd0e00b2910 0x7fd0e00b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.495+0000 7fd0eaffd700 1 -- 172.21.15.46:0/1307535434 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fd0e40e3650 con 0x7fd0f4107df0 2024-05-09T04:49:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.495+0000 7fd0f9899700 1 --2- 172.21.15.46:0/1307535434 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd0e00b2910 0x7fd0e00b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0005fd0 tx=0x7fd0f0005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:34.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.498+0000 7fd0eaffd700 1 -- 172.21.15.46:0/1307535434 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd0e4013070 con 0x7fd0f4107df0 2024-05-09T04:49:34.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.714+0000 7fd0fbafd700 1 -- 172.21.15.46:0/1307535434 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fd0f402e5f0 con 0x7fd0f4107df0 2024-05-09T04:49:34.715 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:34 smithi046 ceph-mon[30136]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:34.715 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:34 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2979418845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T04:49:34.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.715+0000 7fd0eaffd700 1 -- 172.21.15.46:0/1307535434 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fd0e40ab2f0 con 0x7fd0f4107df0 2024-05-09T04:49:34.715 INFO:teuthology.orchestra.run.smithi046.stdout:115964117071 2024-05-09T04:49:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.717+0000 7fd0fbafd700 1 -- 172.21.15.46:0/1307535434 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd0e00b2910 msgr2=0x7fd0e00b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.717+0000 7fd0fbafd700 1 --2- 172.21.15.46:0/1307535434 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd0e00b2910 0x7fd0e00b4e20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fd0f0005fd0 tx=0x7fd0f0005f60 comp rx=0 tx=0).stop 2024-05-09T04:49:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.717+0000 7fd0fbafd700 1 -- 172.21.15.46:0/1307535434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0f4107df0 msgr2=0x7fd0f419a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.717+0000 7fd0fbafd700 1 --2- 172.21.15.46:0/1307535434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0f4107df0 0x7fd0f419a2c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd0e4006010 tx=0x7fd0e40047b0 comp rx=0 tx=0).stop 2024-05-09T04:49:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.717+0000 7fd0fbafd700 1 -- 172.21.15.46:0/1307535434 shutdown_connections 2024-05-09T04:49:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.717+0000 7fd0fbafd700 1 --2- 172.21.15.46:0/1307535434 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd0e00b2910 0x7fd0e00b4e20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.717+0000 7fd0fbafd700 1 --2- 172.21.15.46:0/1307535434 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd0f4108740 0x7fd0f419e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.717+0000 7fd0fbafd700 1 --2- 172.21.15.46:0/1307535434 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd0f41034c0 0x7fd0f4199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.717+0000 7fd0fbafd700 1 --2- 172.21.15.46:0/1307535434 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd0f4107df0 0x7fd0f419a2c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:34.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.717+0000 7fd0fbafd700 1 -- 172.21.15.46:0/1307535434 >> 172.21.15.46:0/1307535434 conn(0x7fd0f40fedc0 msgr2=0x7fd0f41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.717+0000 7fd0fbafd700 1 -- 172.21.15.46:0/1307535434 shutdown_connections 2024-05-09T04:49:34.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:34.717+0000 7fd0fbafd700 1 -- 172.21.15.46:0/1307535434 wait complete. 2024-05-09T04:49:34.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:34 smithi110 ceph-mon[34904]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:34.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:34 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2979418845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T04:49:34.883 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:34 smithi072 ceph-mon[34088]: pgmap v347: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:34.883 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:34 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2979418845' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-09T04:49:34.997 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397559 got 210453397562 for osd.7 2024-05-09T04:49:34.997 DEBUG:teuthology.parallel:result is None 2024-05-09T04:49:35.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:35 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1307535434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T04:49:35.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:35 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1307535434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T04:49:35.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:35 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1307535434' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-09T04:49:36.538 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:36 smithi046 ceph-mon[30136]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:36.573 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117065 got 115964117071 for osd.3 2024-05-09T04:49:36.573 DEBUG:teuthology.parallel:result is None 2024-05-09T04:49:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.426+0000 7f207f7d9700 1 -- 172.21.15.46:0/1218858642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2078105dd0 msgr2=0x7f207810e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.426+0000 7f207f7d9700 1 --2- 172.21.15.46:0/1218858642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2078105dd0 0x7f207810e320 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f206c009990 tx=0x7f206c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.426+0000 7f207f7d9700 1 -- 172.21.15.46:0/1218858642 shutdown_connections 2024-05-09T04:49:36.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.426+0000 7f207f7d9700 1 --2- 172.21.15.46:0/1218858642 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f207810e8f0 0x7f2078110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.426+0000 7f207f7d9700 1 --2- 172.21.15.46:0/1218858642 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2078105480 0x7f2078105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.426+0000 7f207f7d9700 1 --2- 172.21.15.46:0/1218858642 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2078105dd0 0x7f207810e320 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.426+0000 7f207f7d9700 1 -- 172.21.15.46:0/1218858642 >> 172.21.15.46:0/1218858642 conn(0x7f20780fef30 msgr2=0x7f2078101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.426+0000 7f207f7d9700 1 -- 172.21.15.46:0/1218858642 shutdown_connections 2024-05-09T04:49:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.426+0000 7f207f7d9700 1 -- 172.21.15.46:0/1218858642 wait complete. 2024-05-09T04:49:36.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207f7d9700 1 Processor -- start 2024-05-09T04:49:36.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207f7d9700 1 -- start start 2024-05-09T04:49:36.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207f7d9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2078105480 0x7f207819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:36.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207f7d9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2078105dd0 0x7f207819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:36.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207f7d9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f207810e8f0 0x7f207819f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:36.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207f7d9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2078114450 con 0x7f2078105dd0 2024-05-09T04:49:36.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207f7d9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f20781142d0 con 0x7f2078105480 2024-05-09T04:49:36.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207f7d9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f20781145d0 con 0x7f207810e8f0 2024-05-09T04:49:36.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207cd74700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2078105dd0 0x7f207819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:36.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207cd74700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2078105dd0 0x7f207819b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40944/0 (socket says 172.21.15.46:40944) 2024-05-09T04:49:36.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207cd74700 1 -- 172.21.15.46:0/2979418845 learned_addr learned my addr 172.21.15.46:0/2979418845 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:36.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207dd76700 1 --2- 172.21.15.46:0/2979418845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f207810e8f0 0x7f207819f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:36.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207d575700 1 --2- 172.21.15.46:0/2979418845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2078105480 0x7f207819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:36.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207cd74700 1 -- 172.21.15.46:0/2979418845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f207810e8f0 msgr2=0x7f207819f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207cd74700 1 --2- 172.21.15.46:0/2979418845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f207810e8f0 0x7f207819f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207cd74700 1 -- 172.21.15.46:0/2979418845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2078105480 msgr2=0x7f207819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.427+0000 7f207cd74700 1 --2- 172.21.15.46:0/2979418845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2078105480 0x7f207819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.428+0000 7f207cd74700 1 -- 172.21.15.46:0/2979418845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f207819fcf0 con 0x7f2078105dd0 2024-05-09T04:49:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.428+0000 7f207d575700 1 --2- 172.21.15.46:0/2979418845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2078105480 0x7f207819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:49:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.428+0000 7f207cd74700 1 --2- 172.21.15.46:0/2979418845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2078105dd0 0x7f207819b3c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f206c000c00 tx=0x7f206c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:36.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.428+0000 7f206a7fc700 1 -- 172.21.15.46:0/2979418845 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c027070 con 0x7f2078105dd0 2024-05-09T04:49:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.428+0000 7f207f7d9700 1 -- 172.21.15.46:0/2979418845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f207819ff80 con 0x7f2078105dd0 2024-05-09T04:49:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.428+0000 7f206a7fc700 1 -- 172.21.15.46:0/2979418845 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f206c003f20 con 0x7f2078105dd0 2024-05-09T04:49:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.428+0000 7f206a7fc700 1 -- 172.21.15.46:0/2979418845 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f206c022cf0 con 0x7f2078105dd0 2024-05-09T04:49:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.428+0000 7f207f7d9700 1 -- 172.21.15.46:0/2979418845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20781a7900 con 0x7f2078105dd0 2024-05-09T04:49:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.429+0000 7f207f7d9700 1 -- 172.21.15.46:0/2979418845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20780512c0 con 0x7f2078105dd0 2024-05-09T04:49:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.430+0000 7f206a7fc700 1 -- 172.21.15.46:0/2979418845 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f206c02a070 con 0x7f2078105dd0 2024-05-09T04:49:36.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.431+0000 7f206a7fc700 1 --2- 172.21.15.46:0/2979418845 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f20640b2910 0x7f20640b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.431+0000 7f207d575700 1 --2- 172.21.15.46:0/2979418845 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f20640b2910 0x7f20640b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.431+0000 7f206a7fc700 1 -- 172.21.15.46:0/2979418845 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f206c0e33c0 con 0x7f2078105dd0 2024-05-09T04:49:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.432+0000 7f207d575700 1 --2- 172.21.15.46:0/2979418845 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f20640b2910 0x7f20640b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2074005950 tx=0x7f2074005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.436+0000 7f206a7fc700 1 -- 172.21.15.46:0/2979418845 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f206c013070 con 0x7f2078105dd0 2024-05-09T04:49:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.655+0000 7f207f7d9700 1 -- 172.21.15.46:0/2979418845 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f207802e5f0 con 0x7f2078105dd0 2024-05-09T04:49:36.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.655+0000 7f206a7fc700 1 -- 172.21.15.46:0/2979418845 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f206c0aaf70 con 0x7f2078105dd0 2024-05-09T04:49:36.646 INFO:teuthology.orchestra.run.smithi046.stdout:188978561087 2024-05-09T04:49:36.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.658+0000 7f207f7d9700 1 -- 172.21.15.46:0/2979418845 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f20640b2910 msgr2=0x7f20640b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:36.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.658+0000 7f207f7d9700 1 --2- 172.21.15.46:0/2979418845 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f20640b2910 0x7f20640b4e20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2074005950 tx=0x7f2074005f50 comp rx=0 tx=0).stop 2024-05-09T04:49:36.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.658+0000 7f207f7d9700 1 -- 172.21.15.46:0/2979418845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2078105dd0 msgr2=0x7f207819b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:36.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.658+0000 7f207f7d9700 1 --2- 172.21.15.46:0/2979418845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2078105dd0 0x7f207819b3c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f206c000c00 tx=0x7f206c0047b0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.658+0000 7f207f7d9700 1 -- 172.21.15.46:0/2979418845 shutdown_connections 2024-05-09T04:49:36.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.658+0000 7f207f7d9700 1 --2- 172.21.15.46:0/2979418845 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f20640b2910 0x7f20640b4e20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.658+0000 7f207f7d9700 1 --2- 172.21.15.46:0/2979418845 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f207810e8f0 0x7f207819f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.658+0000 7f207f7d9700 1 --2- 172.21.15.46:0/2979418845 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2078105480 0x7f207819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.658+0000 7f207f7d9700 1 --2- 172.21.15.46:0/2979418845 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2078105dd0 0x7f207819b3c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:36.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.658+0000 7f207f7d9700 1 -- 172.21.15.46:0/2979418845 >> 172.21.15.46:0/2979418845 conn(0x7f20780fef30 msgr2=0x7f20780ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:36.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.658+0000 7f207f7d9700 1 -- 172.21.15.46:0/2979418845 shutdown_connections 2024-05-09T04:49:36.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:33.658+0000 7f207f7d9700 1 -- 172.21.15.46:0/2979418845 wait complete. 2024-05-09T04:49:36.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:36 smithi110 ceph-mon[34904]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:36.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:36 smithi072 ceph-mon[34088]: pgmap v348: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:37.279 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561082 got 188978561087 for osd.6 2024-05-09T04:49:37.279 DEBUG:teuthology.parallel:result is None 2024-05-09T04:49:37.379 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:37.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.504+0000 7f5d1e535700 1 -- 172.21.15.46:0/3692826723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18108f00 msgr2=0x7f5d18109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:37.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.504+0000 7f5d1e535700 1 --2- 172.21.15.46:0/3692826723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18108f00 0x7f5d18109310 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f5d000099c0 tx=0x7f5d00009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.505+0000 7f5d1e535700 1 -- 172.21.15.46:0/3692826723 shutdown_connections 2024-05-09T04:49:37.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.505+0000 7f5d1e535700 1 --2- 172.21.15.46:0/3692826723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d18109850 0x7f5d181111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.505+0000 7f5d1e535700 1 --2- 172.21.15.46:0/3692826723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d181045d0 0x7f5d181049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.505+0000 7f5d1e535700 1 --2- 172.21.15.46:0/3692826723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18108f00 0x7f5d18109310 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.505+0000 7f5d1e535700 1 -- 172.21.15.46:0/3692826723 >> 172.21.15.46:0/3692826723 conn(0x7f5d180ffea0 msgr2=0x7f5d181022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:37.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.505+0000 7f5d1e535700 1 -- 172.21.15.46:0/3692826723 shutdown_connections 2024-05-09T04:49:37.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.505+0000 7f5d1e535700 1 -- 172.21.15.46:0/3692826723 wait complete. 2024-05-09T04:49:37.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.505+0000 7f5d1e535700 1 Processor -- start 2024-05-09T04:49:37.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.505+0000 7f5d1e535700 1 -- start start 2024-05-09T04:49:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d1e535700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d181045d0 0x7f5d18076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d1e535700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d18108f00 0x7f5d18077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d1e535700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18109850 0x7f5d18077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:37.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d1e535700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d18114480 con 0x7f5d18109850 2024-05-09T04:49:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d1e535700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5d18114300 con 0x7f5d18108f00 2024-05-09T04:49:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d1e535700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5d18114600 con 0x7f5d181045d0 2024-05-09T04:49:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d1cad2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18109850 0x7f5d18077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d1cad2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18109850 0x7f5d18077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51990/0 (socket says 172.21.15.46:51990) 2024-05-09T04:49:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d1cad2700 1 -- 172.21.15.46:0/3813028712 learned_addr learned my addr 172.21.15.46:0/3813028712 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d0ffff700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d18108f00 0x7f5d18077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d17fff700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d181045d0 0x7f5d18076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d1cad2700 1 -- 172.21.15.46:0/3813028712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d181045d0 msgr2=0x7f5d18076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d1cad2700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d181045d0 0x7f5d18076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d1cad2700 1 -- 172.21.15.46:0/3813028712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d18108f00 msgr2=0x7f5d18077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d1cad2700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d18108f00 0x7f5d18077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d1cad2700 1 -- 172.21.15.46:0/3813028712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d181a9190 con 0x7f5d18109850 2024-05-09T04:49:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d17fff700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d181045d0 0x7f5d18076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:49:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d0ffff700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d18108f00 0x7f5d18077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:49:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.506+0000 7f5d1cad2700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18109850 0x7f5d18077650 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d10007ee0 tx=0x7f5d1000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.507+0000 7f5d15ffb700 1 -- 172.21.15.46:0/3813028712 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1000dbd0 con 0x7f5d18109850 2024-05-09T04:49:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.507+0000 7f5d15ffb700 1 -- 172.21.15.46:0/3813028712 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d1000dd70 con 0x7f5d18109850 2024-05-09T04:49:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.507+0000 7f5d15ffb700 1 -- 172.21.15.46:0/3813028712 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10005990 con 0x7f5d18109850 2024-05-09T04:49:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.507+0000 7f5d1e535700 1 -- 172.21.15.46:0/3813028712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d181a9480 con 0x7f5d18109850 2024-05-09T04:49:37.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.507+0000 7f5d1e535700 1 -- 172.21.15.46:0/3813028712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d181a9990 con 0x7f5d18109850 2024-05-09T04:49:37.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.508+0000 7f5d1e535700 1 -- 172.21.15.46:0/3813028712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d180512c0 con 0x7f5d18109850 2024-05-09T04:49:37.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.509+0000 7f5d15ffb700 1 -- 172.21.15.46:0/3813028712 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f5d1001a020 con 0x7f5d18109850 2024-05-09T04:49:37.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.510+0000 7f5d15ffb700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5d040b2990 0x7f5d040b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:37.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.510+0000 7f5d17fff700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5d040b2990 0x7f5d040b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:37.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.510+0000 7f5d15ffb700 1 -- 172.21.15.46:0/3813028712 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f5d100e1b10 con 0x7f5d18109850 2024-05-09T04:49:37.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.511+0000 7f5d17fff700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5d040b2990 0x7f5d040b4ea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800bc10 tx=0x7f5d0800b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:37.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.516+0000 7f5d15ffb700 1 -- 172.21.15.46:0/3813028712 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5d10010070 con 0x7f5d18109850 2024-05-09T04:49:37.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.735+0000 7f5d1e535700 1 -- 172.21.15.46:0/3813028712 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f5d1802e7b0 con 0x7f5d18109850 2024-05-09T04:49:37.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.736+0000 7f5d15ffb700 1 -- 172.21.15.46:0/3813028712 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f5d100a9690 con 0x7f5d18109850 2024-05-09T04:49:37.736 INFO:teuthology.orchestra.run.smithi046.stdout:236223201334 2024-05-09T04:49:37.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.738+0000 7f5d1e535700 1 -- 172.21.15.46:0/3813028712 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5d040b2990 msgr2=0x7f5d040b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:37.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.738+0000 7f5d1e535700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5d040b2990 0x7f5d040b4ea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800bc10 tx=0x7f5d0800b5f0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.738+0000 7f5d1e535700 1 -- 172.21.15.46:0/3813028712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18109850 msgr2=0x7f5d18077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:37.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.738+0000 7f5d1e535700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18109850 0x7f5d18077650 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f5d10007ee0 tx=0x7f5d1000d3b0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.738+0000 7f5d1e535700 1 -- 172.21.15.46:0/3813028712 shutdown_connections 2024-05-09T04:49:37.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.738+0000 7f5d1e535700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f5d040b2990 0x7f5d040b4ea0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.738+0000 7f5d1e535700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d18109850 0x7f5d18077650 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.738+0000 7f5d1e535700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d181045d0 0x7f5d18076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.738+0000 7f5d1e535700 1 --2- 172.21.15.46:0/3813028712 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d18108f00 0x7f5d18077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:37.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.738+0000 7f5d1e535700 1 -- 172.21.15.46:0/3813028712 >> 172.21.15.46:0/3813028712 conn(0x7f5d180ffea0 msgr2=0x7f5d1810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:37.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.738+0000 7f5d1e535700 1 -- 172.21.15.46:0/3813028712 shutdown_connections 2024-05-09T04:49:37.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:37.738+0000 7f5d1e535700 1 -- 172.21.15.46:0/3813028712 wait complete. 2024-05-09T04:49:37.960 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757313 got 163208757316 for osd.5 2024-05-09T04:49:37.961 DEBUG:teuthology.parallel:result is None 2024-05-09T04:49:38.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:38 smithi110 ceph-mon[34904]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:38.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:38 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3813028712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T04:49:38.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:38 smithi046 ceph-mon[30136]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:38.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:38 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3813028712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T04:49:38.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:38 smithi072 ceph-mon[34088]: pgmap v349: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:38.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:38 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3813028712' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-09T04:49:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.690+0000 7fb150af0700 1 -- 172.21.15.46:0/2520297396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb14c107e10 msgr2=0x7fb14c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.690+0000 7fb150af0700 1 --2- 172.21.15.46:0/2520297396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb14c107e10 0x7fb14c108220 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fb1400099c0 tx=0x7fb140009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.691+0000 7fb150af0700 1 -- 172.21.15.46:0/2520297396 shutdown_connections 2024-05-09T04:49:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.691+0000 7fb150af0700 1 --2- 172.21.15.46:0/2520297396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb14c108760 0x7fb14c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.691+0000 7fb150af0700 1 --2- 172.21.15.46:0/2520297396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb14c1034e0 0x7fb14c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.691+0000 7fb150af0700 1 --2- 172.21.15.46:0/2520297396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb14c107e10 0x7fb14c108220 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.691+0000 7fb150af0700 1 -- 172.21.15.46:0/2520297396 >> 172.21.15.46:0/2520297396 conn(0x7fb14c0fef30 msgr2=0x7fb14c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.691+0000 7fb150af0700 1 -- 172.21.15.46:0/2520297396 shutdown_connections 2024-05-09T04:49:39.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.691+0000 7fb150af0700 1 -- 172.21.15.46:0/2520297396 wait complete. 2024-05-09T04:49:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb150af0700 1 Processor -- start 2024-05-09T04:49:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb150af0700 1 -- start start 2024-05-09T04:49:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb150af0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb14c1034e0 0x7fb14c19bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb150af0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb14c107e10 0x7fb14c19c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb150af0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb14c108760 0x7fb14c19c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb150af0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c113370 con 0x7fb14c1034e0 2024-05-09T04:49:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb150af0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c1131f0 con 0x7fb14c107e10 2024-05-09T04:49:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb150af0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb14c1134f0 con 0x7fb14c108760 2024-05-09T04:49:39.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb14a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb14c1034e0 0x7fb14c19bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb14a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb14c1034e0 0x7fb14c19bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52008/0 (socket says 172.21.15.46:52008) 2024-05-09T04:49:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb14a59c700 1 -- 172.21.15.46:0/1428221111 learned_addr learned my addr 172.21.15.46:0/1428221111 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb149d9b700 1 --2- 172.21.15.46:0/1428221111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb14c107e10 0x7fb14c19c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb14ad9d700 1 --2- 172.21.15.46:0/1428221111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb14c108760 0x7fb14c19c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb14a59c700 1 -- 172.21.15.46:0/1428221111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb14c108760 msgr2=0x7fb14c19c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb14a59c700 1 --2- 172.21.15.46:0/1428221111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb14c108760 0x7fb14c19c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb14a59c700 1 -- 172.21.15.46:0/1428221111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb14c107e10 msgr2=0x7fb14c19c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb14a59c700 1 --2- 172.21.15.46:0/1428221111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb14c107e10 0x7fb14c19c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.692+0000 7fb14a59c700 1 -- 172.21.15.46:0/1428221111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb14c1a7000 con 0x7fb14c1034e0 2024-05-09T04:49:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.693+0000 7fb14a59c700 1 --2- 172.21.15.46:0/1428221111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb14c1034e0 0x7fb14c19bc90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00d980 tx=0x7fb13c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.693+0000 7fb13b7fe700 1 -- 172.21.15.46:0/1428221111 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c016070 con 0x7fb14c1034e0 2024-05-09T04:49:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.693+0000 7fb150af0700 1 -- 172.21.15.46:0/1428221111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14c1a72f0 con 0x7fb14c1034e0 2024-05-09T04:49:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.693+0000 7fb13b7fe700 1 -- 172.21.15.46:0/1428221111 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb13c004830 con 0x7fb14c1034e0 2024-05-09T04:49:39.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.693+0000 7fb13b7fe700 1 -- 172.21.15.46:0/1428221111 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb13c005540 con 0x7fb14c1034e0 2024-05-09T04:49:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.693+0000 7fb150af0700 1 -- 172.21.15.46:0/1428221111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14c1a7800 con 0x7fb14c1034e0 2024-05-09T04:49:39.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.694+0000 7fb150af0700 1 -- 172.21.15.46:0/1428221111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb14c0512c0 con 0x7fb14c1034e0 2024-05-09T04:49:39.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.696+0000 7fb13b7fe700 1 -- 172.21.15.46:0/1428221111 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb13c004220 con 0x7fb14c1034e0 2024-05-09T04:49:39.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.697+0000 7fb13b7fe700 1 --2- 172.21.15.46:0/1428221111 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb1340b2910 0x7fb1340b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:39.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.697+0000 7fb149d9b700 1 --2- 172.21.15.46:0/1428221111 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb1340b2910 0x7fb1340b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:39.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.697+0000 7fb13b7fe700 1 -- 172.21.15.46:0/1428221111 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fb13c0e2760 con 0x7fb14c1034e0 2024-05-09T04:49:39.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.698+0000 7fb149d9b700 1 --2- 172.21.15.46:0/1428221111 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb1340b2910 0x7fb1340b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb140005b40 tx=0x7fb14001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:39.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.701+0000 7fb13b7fe700 1 -- 172.21.15.46:0/1428221111 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb13c01f020 con 0x7fb14c1034e0 2024-05-09T04:49:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.921+0000 7fb150af0700 1 -- 172.21.15.46:0/1428221111 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fb14c02e7b0 con 0x7fb14c1034e0 2024-05-09T04:49:39.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.922+0000 7fb13b7fe700 1 -- 172.21.15.46:0/1428221111 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fb13c0aa280 con 0x7fb14c1034e0 2024-05-09T04:49:39.922 INFO:teuthology.orchestra.run.smithi046.stdout:55834574938 2024-05-09T04:49:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.925+0000 7fb150af0700 1 -- 172.21.15.46:0/1428221111 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb1340b2910 msgr2=0x7fb1340b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.925+0000 7fb150af0700 1 --2- 172.21.15.46:0/1428221111 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb1340b2910 0x7fb1340b4e20 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb140005b40 tx=0x7fb14001a040 comp rx=0 tx=0).stop 2024-05-09T04:49:39.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.925+0000 7fb150af0700 1 -- 172.21.15.46:0/1428221111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb14c1034e0 msgr2=0x7fb14c19bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.925+0000 7fb150af0700 1 --2- 172.21.15.46:0/1428221111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb14c1034e0 0x7fb14c19bc90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00d980 tx=0x7fb13c00dc90 comp rx=0 tx=0).stop 2024-05-09T04:49:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.925+0000 7fb150af0700 1 -- 172.21.15.46:0/1428221111 shutdown_connections 2024-05-09T04:49:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.925+0000 7fb150af0700 1 --2- 172.21.15.46:0/1428221111 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fb1340b2910 0x7fb1340b4e20 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.925+0000 7fb150af0700 1 --2- 172.21.15.46:0/1428221111 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb14c108760 0x7fb14c19c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.925+0000 7fb150af0700 1 --2- 172.21.15.46:0/1428221111 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb14c1034e0 0x7fb14c19bc90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.925+0000 7fb150af0700 1 --2- 172.21.15.46:0/1428221111 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb14c107e10 0x7fb14c19c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.925+0000 7fb150af0700 1 -- 172.21.15.46:0/1428221111 >> 172.21.15.46:0/1428221111 conn(0x7fb14c0fef30 msgr2=0x7fb14c1096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.925+0000 7fb150af0700 1 -- 172.21.15.46:0/1428221111 shutdown_connections 2024-05-09T04:49:39.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:39.926+0000 7fb150af0700 1 -- 172.21.15.46:0/1428221111 wait complete. 2024-05-09T04:49:40.738 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:40 smithi046 ceph-mon[30136]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:40.738 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:40 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1428221111' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T04:49:40.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:40 smithi110 ceph-mon[34904]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:40.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:40 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1428221111' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T04:49:40.863 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:40.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:40 smithi072 ceph-mon[34088]: pgmap v350: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:40.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:40 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1428221111' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-09T04:49:40.894 INFO:tasks.cephadm.ceph_manager.ceph:need seq 283467841571 got 283467841576 for osd.10 2024-05-09T04:49:40.894 DEBUG:teuthology.parallel:result is None 2024-05-09T04:49:41.589 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574931 got 55834574938 for osd.1 2024-05-09T04:49:41.590 DEBUG:teuthology.parallel:result is None 2024-05-09T04:49:42.035 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201328 got 236223201334 for osd.8 2024-05-09T04:49:42.035 DEBUG:teuthology.parallel:result is None 2024-05-09T04:49:42.788 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:42 smithi046 ceph-mon[30136]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:42.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:42 smithi110 ceph-mon[34904]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:42.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:42 smithi072 ceph-mon[34088]: pgmap v351: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:42.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.909+0000 7f7e2a394700 1 -- 172.21.15.46:0/1431416504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e24108740 msgr2=0x7f7e241100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:42.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.909+0000 7f7e2a394700 1 --2- 172.21.15.46:0/1431416504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e24108740 0x7f7e241100a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f7e1c00b0a0 tx=0x7f7e1c00b3b0 comp rx=0 tx=0).stop 2024-05-09T04:49:42.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.910+0000 7f7e2a394700 1 -- 172.21.15.46:0/1431416504 shutdown_connections 2024-05-09T04:49:42.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.910+0000 7f7e2a394700 1 --2- 172.21.15.46:0/1431416504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e24108740 0x7f7e241100a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:42.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.910+0000 7f7e2a394700 1 --2- 172.21.15.46:0/1431416504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e241034c0 0x7f7e241038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:42.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.910+0000 7f7e2a394700 1 --2- 172.21.15.46:0/1431416504 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e24107df0 0x7f7e24108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:42.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.910+0000 7f7e2a394700 1 -- 172.21.15.46:0/1431416504 >> 172.21.15.46:0/1431416504 conn(0x7f7e240fedf0 msgr2=0x7f7e24101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:42.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.910+0000 7f7e2a394700 1 -- 172.21.15.46:0/1431416504 shutdown_connections 2024-05-09T04:49:42.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.910+0000 7f7e2a394700 1 -- 172.21.15.46:0/1431416504 wait complete. 2024-05-09T04:49:42.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e2a394700 1 Processor -- start 2024-05-09T04:49:42.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e2a394700 1 -- start start 2024-05-09T04:49:42.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e2a394700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e241034c0 0x7f7e2419adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:42.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e2a394700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e24107df0 0x7f7e2419b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e2a394700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e24108740 0x7f7e2419f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e2a394700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7e241144d0 con 0x7f7e24107df0 2024-05-09T04:49:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e2a394700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24114350 con 0x7f7e241034c0 2024-05-09T04:49:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e2a394700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7e24114650 con 0x7f7e24108740 2024-05-09T04:49:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e1bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e24107df0 0x7f7e2419b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e1bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e24107df0 0x7f7e2419b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52022/0 (socket says 172.21.15.46:52022) 2024-05-09T04:49:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e1bfff700 1 -- 172.21.15.46:0/4216635586 learned_addr learned my addr 172.21.15.46:0/4216635586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e23fff700 1 --2- 172.21.15.46:0/4216635586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e241034c0 0x7f7e2419adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e28931700 1 --2- 172.21.15.46:0/4216635586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e24108740 0x7f7e2419f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e1bfff700 1 -- 172.21.15.46:0/4216635586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e24108740 msgr2=0x7f7e2419f5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e1bfff700 1 --2- 172.21.15.46:0/4216635586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e24108740 0x7f7e2419f5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e1bfff700 1 -- 172.21.15.46:0/4216635586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e241034c0 msgr2=0x7f7e2419adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.911+0000 7f7e1bfff700 1 --2- 172.21.15.46:0/4216635586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e241034c0 0x7f7e2419adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.912+0000 7f7e1bfff700 1 -- 172.21.15.46:0/4216635586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e2419fc90 con 0x7f7e24107df0 2024-05-09T04:49:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.912+0000 7f7e1bfff700 1 --2- 172.21.15.46:0/4216635586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e24107df0 0x7f7e2419b360 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c00d980 tx=0x7f7e0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.912+0000 7f7e21ffb700 1 -- 172.21.15.46:0/4216635586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c017070 con 0x7f7e24107df0 2024-05-09T04:49:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.912+0000 7f7e21ffb700 1 -- 172.21.15.46:0/4216635586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e0c009de0 con 0x7f7e24107df0 2024-05-09T04:49:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.912+0000 7f7e2a394700 1 -- 172.21.15.46:0/4216635586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e2419ff80 con 0x7f7e24107df0 2024-05-09T04:49:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.912+0000 7f7e21ffb700 1 -- 172.21.15.46:0/4216635586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e0c00f3c0 con 0x7f7e24107df0 2024-05-09T04:49:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.912+0000 7f7e2a394700 1 -- 172.21.15.46:0/4216635586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e241a78a0 con 0x7f7e24107df0 2024-05-09T04:49:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.913+0000 7f7e2a394700 1 -- 172.21.15.46:0/4216635586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e240512c0 con 0x7f7e24107df0 2024-05-09T04:49:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.914+0000 7f7e21ffb700 1 -- 172.21.15.46:0/4216635586 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7e0c009930 con 0x7f7e24107df0 2024-05-09T04:49:42.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.915+0000 7f7e21ffb700 1 --2- 172.21.15.46:0/4216635586 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7e100b2990 0x7f7e100b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:42.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.915+0000 7f7e23fff700 1 --2- 172.21.15.46:0/4216635586 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7e100b2990 0x7f7e100b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:42.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.915+0000 7f7e21ffb700 1 -- 172.21.15.46:0/4216635586 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f7e0c0e1e20 con 0x7f7e24107df0 2024-05-09T04:49:42.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.916+0000 7f7e23fff700 1 --2- 172.21.15.46:0/4216635586 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7e100b2990 0x7f7e100b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7e14007be0 tx=0x7f7e140073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:42.920+0000 7f7e21ffb700 1 -- 172.21.15.46:0/4216635586 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e0c0a9940 con 0x7f7e24107df0 2024-05-09T04:49:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.098+0000 7f4650d63700 1 -- 172.21.15.46:0/84136797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f464c108740 msgr2=0x7f464c1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.098+0000 7f4650d63700 1 --2- 172.21.15.46:0/84136797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f464c108740 0x7f464c1100a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f464400b0d0 tx=0x7f464400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.098+0000 7f4650d63700 1 -- 172.21.15.46:0/84136797 shutdown_connections 2024-05-09T04:49:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.098+0000 7f4650d63700 1 --2- 172.21.15.46:0/84136797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f464c108740 0x7f464c1100a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.098+0000 7f4650d63700 1 --2- 172.21.15.46:0/84136797 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f464c1034c0 0x7f464c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.098+0000 7f4650d63700 1 --2- 172.21.15.46:0/84136797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f464c107df0 0x7f464c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.098+0000 7f4650d63700 1 -- 172.21.15.46:0/84136797 >> 172.21.15.46:0/84136797 conn(0x7f464c0fedf0 msgr2=0x7f464c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:43.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.098+0000 7f4650d63700 1 -- 172.21.15.46:0/84136797 shutdown_connections 2024-05-09T04:49:43.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.098+0000 7f4650d63700 1 -- 172.21.15.46:0/84136797 wait complete. 2024-05-09T04:49:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.099+0000 7f4650d63700 1 Processor -- start 2024-05-09T04:49:43.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.099+0000 7f4650d63700 1 -- start start 2024-05-09T04:49:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.099+0000 7f4650d63700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f464c1034c0 0x7f464c19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.099+0000 7f4650d63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f464c107df0 0x7f464c19b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.099+0000 7f4650d63700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f464c108740 0x7f464c19f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.099+0000 7f4650d63700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f464c113370 con 0x7f464c107df0 2024-05-09T04:49:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.099+0000 7f4650d63700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f464c1131f0 con 0x7f464c1034c0 2024-05-09T04:49:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.099+0000 7f4650d63700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f464c1134f0 con 0x7f464c108740 2024-05-09T04:49:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.099+0000 7f4649d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f464c107df0 0x7f464c19b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.099+0000 7f4649d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f464c107df0 0x7f464c19b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52046/0 (socket says 172.21.15.46:52046) 2024-05-09T04:49:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.099+0000 7f4649d9b700 1 -- 172.21.15.46:0/106493473 learned_addr learned my addr 172.21.15.46:0/106493473 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.099+0000 7f464a59c700 1 --2- 172.21.15.46:0/106493473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f464c1034c0 0x7f464c19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.099+0000 7f464ad9d700 1 --2- 172.21.15.46:0/106493473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f464c108740 0x7f464c19f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:43.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.100+0000 7f4649d9b700 1 -- 172.21.15.46:0/106493473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f464c108740 msgr2=0x7f464c19f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.100+0000 7f4649d9b700 1 --2- 172.21.15.46:0/106493473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f464c108740 0x7f464c19f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.100+0000 7f4649d9b700 1 -- 172.21.15.46:0/106493473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f464c1034c0 msgr2=0x7f464c19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.100+0000 7f4649d9b700 1 --2- 172.21.15.46:0/106493473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f464c1034c0 0x7f464c19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.100+0000 7f4649d9b700 1 -- 172.21.15.46:0/106493473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f464c19fca0 con 0x7f464c107df0 2024-05-09T04:49:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.100+0000 7f464a59c700 1 --2- 172.21.15.46:0/106493473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f464c1034c0 0x7f464c19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:49:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.100+0000 7f4649d9b700 1 --2- 172.21.15.46:0/106493473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f464c107df0 0x7f464c19b370 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f464000d980 tx=0x7f464000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.100+0000 7f463b7fe700 1 -- 172.21.15.46:0/106493473 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4640017070 con 0x7f464c107df0 2024-05-09T04:49:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.100+0000 7f463b7fe700 1 -- 172.21.15.46:0/106493473 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f464000fd40 con 0x7f464c107df0 2024-05-09T04:49:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.100+0000 7f4650d63700 1 -- 172.21.15.46:0/106493473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f464c19ff90 con 0x7f464c107df0 2024-05-09T04:49:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.100+0000 7f463b7fe700 1 -- 172.21.15.46:0/106493473 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f464000e3b0 con 0x7f464c107df0 2024-05-09T04:49:43.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.100+0000 7f4650d63700 1 -- 172.21.15.46:0/106493473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f464c1a78b0 con 0x7f464c107df0 2024-05-09T04:49:43.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.101+0000 7f4650d63700 1 -- 172.21.15.46:0/106493473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f464c0512c0 con 0x7f464c107df0 2024-05-09T04:49:43.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.103+0000 7f463b7fe700 1 -- 172.21.15.46:0/106493473 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f4640010610 con 0x7f464c107df0 2024-05-09T04:49:43.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.104+0000 7f463b7fe700 1 --2- 172.21.15.46:0/106493473 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f46340b2910 0x7f46340b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:43.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.104+0000 7f464a59c700 1 --2- 172.21.15.46:0/106493473 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f46340b2910 0x7f46340b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:43.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.104+0000 7f463b7fe700 1 -- 172.21.15.46:0/106493473 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f46400e1de0 con 0x7f464c107df0 2024-05-09T04:49:43.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.104+0000 7f464a59c700 1 --2- 172.21.15.46:0/106493473 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f46340b2910 0x7f46340b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f463c007fd0 tx=0x7f463c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:43.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.109+0000 7f463b7fe700 1 -- 172.21.15.46:0/106493473 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f46400a9960 con 0x7f464c107df0 2024-05-09T04:49:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.142+0000 7f7e2a394700 1 -- 172.21.15.46:0/4216635586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f7e2402e5f0 con 0x7f7e24107df0 2024-05-09T04:49:43.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.142+0000 7f7e21ffb700 1 -- 172.21.15.46:0/4216635586 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f7e0c0a9240 con 0x7f7e24107df0 2024-05-09T04:49:43.145 INFO:teuthology.orchestra.run.smithi046.stdout:309237645350 2024-05-09T04:49:43.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.146+0000 7f7e2a394700 1 -- 172.21.15.46:0/4216635586 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7e100b2990 msgr2=0x7f7e100b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:43.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.146+0000 7f7e2a394700 1 --2- 172.21.15.46:0/4216635586 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7e100b2990 0x7f7e100b4ea0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f7e14007be0 tx=0x7f7e140073d0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.146+0000 7f7e2a394700 1 -- 172.21.15.46:0/4216635586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e24107df0 msgr2=0x7f7e2419b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:43.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.146+0000 7f7e2a394700 1 --2- 172.21.15.46:0/4216635586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e24107df0 0x7f7e2419b360 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f7e0c00d980 tx=0x7f7e0c00dc90 comp rx=0 tx=0).stop 2024-05-09T04:49:43.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.146+0000 7f7e2a394700 1 -- 172.21.15.46:0/4216635586 shutdown_connections 2024-05-09T04:49:43.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.146+0000 7f7e2a394700 1 --2- 172.21.15.46:0/4216635586 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f7e100b2990 0x7f7e100b4ea0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.146+0000 7f7e2a394700 1 --2- 172.21.15.46:0/4216635586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7e24108740 0x7f7e2419f5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.146+0000 7f7e2a394700 1 --2- 172.21.15.46:0/4216635586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7e241034c0 0x7f7e2419adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.146+0000 7f7e2a394700 1 --2- 172.21.15.46:0/4216635586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7e24107df0 0x7f7e2419b360 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.146+0000 7f7e2a394700 1 -- 172.21.15.46:0/4216635586 >> 172.21.15.46:0/4216635586 conn(0x7f7e240fedf0 msgr2=0x7f7e24107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:43.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.146+0000 7f7e2a394700 1 -- 172.21.15.46:0/4216635586 shutdown_connections 2024-05-09T04:49:43.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.146+0000 7f7e2a394700 1 -- 172.21.15.46:0/4216635586 wait complete. 2024-05-09T04:49:43.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.340+0000 7f4650d63700 1 -- 172.21.15.46:0/106493473 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f464c02e5f0 con 0x7f464c107df0 2024-05-09T04:49:43.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.340+0000 7f463b7fe700 1 -- 172.21.15.46:0/106493473 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f46400a9260 con 0x7f464c107df0 2024-05-09T04:49:43.341 INFO:teuthology.orchestra.run.smithi046.stdout:141733920842 2024-05-09T04:49:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.343+0000 7f4650d63700 1 -- 172.21.15.46:0/106493473 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f46340b2910 msgr2=0x7f46340b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.343+0000 7f4650d63700 1 --2- 172.21.15.46:0/106493473 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f46340b2910 0x7f46340b4e20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f463c007fd0 tx=0x7f463c0073d0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.343+0000 7f4650d63700 1 -- 172.21.15.46:0/106493473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f464c107df0 msgr2=0x7f464c19b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.343+0000 7f4650d63700 1 --2- 172.21.15.46:0/106493473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f464c107df0 0x7f464c19b370 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f464000d980 tx=0x7f464000dc90 comp rx=0 tx=0).stop 2024-05-09T04:49:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.343+0000 7f4650d63700 1 -- 172.21.15.46:0/106493473 shutdown_connections 2024-05-09T04:49:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.343+0000 7f4650d63700 1 --2- 172.21.15.46:0/106493473 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f46340b2910 0x7f46340b4e20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.343+0000 7f4650d63700 1 --2- 172.21.15.46:0/106493473 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f464c108740 0x7f464c19f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.343+0000 7f4650d63700 1 --2- 172.21.15.46:0/106493473 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f464c1034c0 0x7f464c19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.343+0000 7f4650d63700 1 --2- 172.21.15.46:0/106493473 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f464c107df0 0x7f464c19b370 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.343+0000 7f4650d63700 1 -- 172.21.15.46:0/106493473 >> 172.21.15.46:0/106493473 conn(0x7f464c0fedf0 msgr2=0x7f464c075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.343+0000 7f4650d63700 1 -- 172.21.15.46:0/106493473 shutdown_connections 2024-05-09T04:49:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:43.343+0000 7f4650d63700 1 -- 172.21.15.46:0/106493473 wait complete. 2024-05-09T04:49:43.625 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:43 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/4216635586' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T04:49:43.625 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:43 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/106493473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T04:49:43.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:43 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/4216635586' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T04:49:43.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:43 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/106493473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T04:49:43.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:43 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/4216635586' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-09T04:49:43.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:43 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/106493473' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-09T04:49:44.050 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645346 got 309237645350 for osd.11 2024-05-09T04:49:44.051 DEBUG:teuthology.parallel:result is None 2024-05-09T04:49:44.576 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:44 smithi046 ceph-mon[30136]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.705+0000 7f3fd0674700 1 -- 172.21.15.46:0/2021187471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc8108f00 msgr2=0x7f3fc8109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.705+0000 7f3fd0674700 1 --2- 172.21.15.46:0/2021187471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc8108f00 0x7f3fc8109310 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc0099c0 tx=0x7f3fbc009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.705+0000 7f3fd0674700 1 -- 172.21.15.46:0/2021187471 shutdown_connections 2024-05-09T04:49:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.705+0000 7f3fd0674700 1 --2- 172.21.15.46:0/2021187471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3fc8109850 0x7f3fc81111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.705+0000 7f3fd0674700 1 --2- 172.21.15.46:0/2021187471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fc81045d0 0x7f3fc81049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.705+0000 7f3fd0674700 1 --2- 172.21.15.46:0/2021187471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc8108f00 0x7f3fc8109310 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.705+0000 7f3fd0674700 1 -- 172.21.15.46:0/2021187471 >> 172.21.15.46:0/2021187471 conn(0x7f3fc80ffe80 msgr2=0x7f3fc81022d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.706+0000 7f3fd0674700 1 -- 172.21.15.46:0/2021187471 shutdown_connections 2024-05-09T04:49:44.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.706+0000 7f3fd0674700 1 -- 172.21.15.46:0/2021187471 wait complete. 2024-05-09T04:49:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.706+0000 7f3fd0674700 1 Processor -- start 2024-05-09T04:49:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.706+0000 7f3fd0674700 1 -- start start 2024-05-09T04:49:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fd0674700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fc81045d0 0x7f3fc819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fd0674700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc8108f00 0x7f3fc819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fd0674700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3fc8109850 0x7f3fc819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fd0674700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc8114460 con 0x7f3fc8108f00 2024-05-09T04:49:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fd0674700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc81142e0 con 0x7f3fc8109850 2024-05-09T04:49:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fd0674700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3fc81145e0 con 0x7f3fc81045d0 2024-05-09T04:49:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fcdc0f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc8108f00 0x7f3fc819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fcdc0f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc8108f00 0x7f3fc819b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49726/0 (socket says 172.21.15.46:49726) 2024-05-09T04:49:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fcdc0f700 1 -- 172.21.15.46:0/1472027366 learned_addr learned my addr 172.21.15.46:0/1472027366 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:44.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fce410700 1 --2- 172.21.15.46:0/1472027366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fc81045d0 0x7f3fc819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fcec11700 1 --2- 172.21.15.46:0/1472027366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3fc8109850 0x7f3fc819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fcdc0f700 1 -- 172.21.15.46:0/1472027366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fc81045d0 msgr2=0x7f3fc819ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fcdc0f700 1 --2- 172.21.15.46:0/1472027366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fc81045d0 0x7f3fc819ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fcdc0f700 1 -- 172.21.15.46:0/1472027366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3fc8109850 msgr2=0x7f3fc819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fcdc0f700 1 --2- 172.21.15.46:0/1472027366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3fc8109850 0x7f3fc819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fcdc0f700 1 -- 172.21.15.46:0/1472027366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fc819fd60 con 0x7f3fc8108f00 2024-05-09T04:49:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.707+0000 7f3fcdc0f700 1 --2- 172.21.15.46:0/1472027366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc8108f00 0x7f3fc819b430 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc005f50 tx=0x7f3fbc004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.708+0000 7f3fc77fe700 1 -- 172.21.15.46:0/1472027366 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc026070 con 0x7f3fc8108f00 2024-05-09T04:49:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.708+0000 7f3fc77fe700 1 -- 172.21.15.46:0/1472027366 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fbc003c70 con 0x7f3fc8108f00 2024-05-09T04:49:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.708+0000 7f3fd0674700 1 -- 172.21.15.46:0/1472027366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fc819fff0 con 0x7f3fc8108f00 2024-05-09T04:49:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.708+0000 7f3fc77fe700 1 -- 172.21.15.46:0/1472027366 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc00fad0 con 0x7f3fc8108f00 2024-05-09T04:49:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.708+0000 7f3fd0674700 1 -- 172.21.15.46:0/1472027366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fc81a7970 con 0x7f3fc8108f00 2024-05-09T04:49:44.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.709+0000 7f3fd0674700 1 -- 172.21.15.46:0/1472027366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fc80512c0 con 0x7f3fc8108f00 2024-05-09T04:49:44.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.710+0000 7f3fc77fe700 1 -- 172.21.15.46:0/1472027366 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3fbc029070 con 0x7f3fc8108f00 2024-05-09T04:49:44.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.711+0000 7f3fc77fe700 1 --2- 172.21.15.46:0/1472027366 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3fb00b2990 0x7f3fb00b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:44.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.712+0000 7f3fce410700 1 --2- 172.21.15.46:0/1472027366 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3fb00b2990 0x7f3fb00b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:44.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.712+0000 7f3fc77fe700 1 -- 172.21.15.46:0/1472027366 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f3fbc0e3350 con 0x7f3fc8108f00 2024-05-09T04:49:44.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.712+0000 7f3fce410700 1 --2- 172.21.15.46:0/1472027366 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3fb00b2990 0x7f3fb00b4ea0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800f540 tx=0x7f3fb800b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:44.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.715+0000 7f3fc77fe700 1 -- 172.21.15.46:0/1472027366 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3fbc030080 con 0x7f3fc8108f00 2024-05-09T04:49:44.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:44 smithi110 ceph-mon[34904]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:44.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:44 smithi072 ceph-mon[34088]: pgmap v352: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 0 B/s, 0 objects/s recovering 2024-05-09T04:49:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.931+0000 7f3fd0674700 1 -- 172.21.15.46:0/1472027366 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f3fc802e5f0 con 0x7f3fc8108f00 2024-05-09T04:49:44.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.931+0000 7f3fc77fe700 1 -- 172.21.15.46:0/1472027366 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3fbc013070 con 0x7f3fc8108f00 2024-05-09T04:49:44.931 INFO:teuthology.orchestra.run.smithi046.stdout:34359738463 2024-05-09T04:49:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.933+0000 7f3fd0674700 1 -- 172.21.15.46:0/1472027366 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3fb00b2990 msgr2=0x7f3fb00b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:44.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.933+0000 7f3fd0674700 1 --2- 172.21.15.46:0/1472027366 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3fb00b2990 0x7f3fb00b4ea0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3fb800f540 tx=0x7f3fb800b040 comp rx=0 tx=0).stop 2024-05-09T04:49:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.933+0000 7f3fd0674700 1 -- 172.21.15.46:0/1472027366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc8108f00 msgr2=0x7f3fc819b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.933+0000 7f3fd0674700 1 --2- 172.21.15.46:0/1472027366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc8108f00 0x7f3fc819b430 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc005f50 tx=0x7f3fbc004270 comp rx=0 tx=0).stop 2024-05-09T04:49:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.933+0000 7f3fd0674700 1 -- 172.21.15.46:0/1472027366 shutdown_connections 2024-05-09T04:49:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.934+0000 7f3fd0674700 1 --2- 172.21.15.46:0/1472027366 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3fb00b2990 0x7f3fb00b4ea0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.934+0000 7f3fd0674700 1 --2- 172.21.15.46:0/1472027366 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3fc8109850 0x7f3fc819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.934+0000 7f3fd0674700 1 --2- 172.21.15.46:0/1472027366 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3fc81045d0 0x7f3fc819ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.934+0000 7f3fd0674700 1 --2- 172.21.15.46:0/1472027366 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3fc8108f00 0x7f3fc819b430 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.934+0000 7f3fd0674700 1 -- 172.21.15.46:0/1472027366 >> 172.21.15.46:0/1472027366 conn(0x7f3fc80ffe80 msgr2=0x7f3fc810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.934+0000 7f3fd0674700 1 -- 172.21.15.46:0/1472027366 shutdown_connections 2024-05-09T04:49:44.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:44.934+0000 7f3fd0674700 1 -- 172.21.15.46:0/1472027366 wait complete. 2024-05-09T04:49:45.385 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920837 got 141733920842 for osd.4 2024-05-09T04:49:45.386 DEBUG:teuthology.parallel:result is None 2024-05-09T04:49:45.717 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:45 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1472027366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T04:49:45.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:45 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1472027366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T04:49:45.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:45 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1472027366' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-09T04:49:46.566 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738458 got 34359738463 for osd.0 2024-05-09T04:49:46.566 DEBUG:teuthology.parallel:result is None 2024-05-09T04:49:46.747 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:46 smithi046 ceph-mon[30136]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:49:46.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:46 smithi110 ceph-mon[34904]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:49:46.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:46 smithi072 ceph-mon[34088]: pgmap v353: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:49:47.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.127+0000 7f3c6563f700 1 -- 172.21.15.46:0/817906441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c60108740 msgr2=0x7f3c601100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:47.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.127+0000 7f3c6563f700 1 --2- 172.21.15.46:0/817906441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c60108740 0x7f3c601100a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3c5800b0d0 tx=0x7f3c5800b3e0 comp rx=0 tx=0).stop 2024-05-09T04:49:47.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.127+0000 7f3c6563f700 1 -- 172.21.15.46:0/817906441 shutdown_connections 2024-05-09T04:49:47.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.127+0000 7f3c6563f700 1 --2- 172.21.15.46:0/817906441 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c60108740 0x7f3c601100a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:47.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.127+0000 7f3c6563f700 1 --2- 172.21.15.46:0/817906441 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c601034c0 0x7f3c601038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:47.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.127+0000 7f3c6563f700 1 --2- 172.21.15.46:0/817906441 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c60107df0 0x7f3c60108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:47.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.127+0000 7f3c6563f700 1 -- 172.21.15.46:0/817906441 >> 172.21.15.46:0/817906441 conn(0x7f3c600fedf0 msgr2=0x7f3c60101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:47.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.127+0000 7f3c6563f700 1 -- 172.21.15.46:0/817906441 shutdown_connections 2024-05-09T04:49:47.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.127+0000 7f3c6563f700 1 -- 172.21.15.46:0/817906441 wait complete. 2024-05-09T04:49:47.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.128+0000 7f3c6563f700 1 Processor -- start 2024-05-09T04:49:47.129 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.128+0000 7f3c6563f700 1 -- start start 2024-05-09T04:49:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c6563f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c601034c0 0x7f3c6019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c6563f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c60107df0 0x7f3c6019c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c6563f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c60108740 0x7f3c6019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c6563f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3c60113370 con 0x7f3c601034c0 2024-05-09T04:49:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c6563f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601131f0 con 0x7f3c60108740 2024-05-09T04:49:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c6563f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3c601134f0 con 0x7f3c60107df0 2024-05-09T04:49:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c5effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c601034c0 0x7f3c6019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c5effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c601034c0 0x7f3c6019bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49744/0 (socket says 172.21.15.46:49744) 2024-05-09T04:49:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c5effd700 1 -- 172.21.15.46:0/2885286032 learned_addr learned my addr 172.21.15.46:0/2885286032 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c5f7fe700 1 --2- 172.21.15.46:0/2885286032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c60108740 0x7f3c6019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c5e7fc700 1 --2- 172.21.15.46:0/2885286032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c60107df0 0x7f3c6019c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c5effd700 1 -- 172.21.15.46:0/2885286032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c60107df0 msgr2=0x7f3c6019c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c5effd700 1 --2- 172.21.15.46:0/2885286032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c60107df0 0x7f3c6019c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:47.130 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c5effd700 1 -- 172.21.15.46:0/2885286032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c60108740 msgr2=0x7f3c6019c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c5effd700 1 --2- 172.21.15.46:0/2885286032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c60108740 0x7f3c6019c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c5effd700 1 -- 172.21.15.46:0/2885286032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c601a6fc0 con 0x7f3c601034c0 2024-05-09T04:49:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c5effd700 1 --2- 172.21.15.46:0/2885286032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c601034c0 0x7f3c6019bc70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000d950 tx=0x7f3c5000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c47fff700 1 -- 172.21.15.46:0/2885286032 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c50016070 con 0x7f3c601034c0 2024-05-09T04:49:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.129+0000 7f3c6563f700 1 -- 172.21.15.46:0/2885286032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c601a72b0 con 0x7f3c601034c0 2024-05-09T04:49:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.130+0000 7f3c6563f700 1 -- 172.21.15.46:0/2885286032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c601a7840 con 0x7f3c601034c0 2024-05-09T04:49:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.130+0000 7f3c47fff700 1 -- 172.21.15.46:0/2885286032 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c50009df0 con 0x7f3c601034c0 2024-05-09T04:49:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.130+0000 7f3c47fff700 1 -- 172.21.15.46:0/2885286032 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c5000f3c0 con 0x7f3c601034c0 2024-05-09T04:49:47.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.130+0000 7f3c6563f700 1 -- 172.21.15.46:0/2885286032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c4c005330 con 0x7f3c601034c0 2024-05-09T04:49:47.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.132+0000 7f3c47fff700 1 -- 172.21.15.46:0/2885286032 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3c50019070 con 0x7f3c601034c0 2024-05-09T04:49:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.133+0000 7f3c47fff700 1 --2- 172.21.15.46:0/2885286032 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3c480b2910 0x7f3c480b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.133+0000 7f3c5e7fc700 1 --2- 172.21.15.46:0/2885286032 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3c480b2910 0x7f3c480b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.133+0000 7f3c47fff700 1 -- 172.21.15.46:0/2885286032 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f3c500e1e70 con 0x7f3c601034c0 2024-05-09T04:49:47.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.134+0000 7f3c5e7fc700 1 --2- 172.21.15.46:0/2885286032 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3c480b2910 0x7f3c480b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3c54005950 tx=0x7f3c540058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:47.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.137+0000 7f3c47fff700 1 -- 172.21.15.46:0/2885286032 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3c500a99f0 con 0x7f3c601034c0 2024-05-09T04:49:47.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.350+0000 7f3c6563f700 1 -- 172.21.15.46:0/2885286032 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f3c4c0058f0 con 0x7f3c601034c0 2024-05-09T04:49:47.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.350+0000 7f3c47fff700 1 -- 172.21.15.46:0/2885286032 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f3c500a92f0 con 0x7f3c601034c0 2024-05-09T04:49:47.350 INFO:teuthology.orchestra.run.smithi046.stdout:77309411414 2024-05-09T04:49:47.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.352+0000 7f3c6563f700 1 -- 172.21.15.46:0/2885286032 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3c480b2910 msgr2=0x7f3c480b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:47.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.352+0000 7f3c6563f700 1 --2- 172.21.15.46:0/2885286032 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3c480b2910 0x7f3c480b4e20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3c54005950 tx=0x7f3c540058e0 comp rx=0 tx=0).stop 2024-05-09T04:49:47.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.352+0000 7f3c6563f700 1 -- 172.21.15.46:0/2885286032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c601034c0 msgr2=0x7f3c6019bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:47.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.352+0000 7f3c6563f700 1 --2- 172.21.15.46:0/2885286032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c601034c0 0x7f3c6019bc70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f3c5000d950 tx=0x7f3c5000dc60 comp rx=0 tx=0).stop 2024-05-09T04:49:47.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.352+0000 7f3c6563f700 1 -- 172.21.15.46:0/2885286032 shutdown_connections 2024-05-09T04:49:47.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.352+0000 7f3c6563f700 1 --2- 172.21.15.46:0/2885286032 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3c480b2910 0x7f3c480b4e20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:47.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.352+0000 7f3c6563f700 1 --2- 172.21.15.46:0/2885286032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3c60108740 0x7f3c6019c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:47.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.352+0000 7f3c6563f700 1 --2- 172.21.15.46:0/2885286032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3c601034c0 0x7f3c6019bc70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:47.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.352+0000 7f3c6563f700 1 --2- 172.21.15.46:0/2885286032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3c60107df0 0x7f3c6019c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:47.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.352+0000 7f3c6563f700 1 -- 172.21.15.46:0/2885286032 >> 172.21.15.46:0/2885286032 conn(0x7f3c600fedf0 msgr2=0x7f3c60107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:47.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.353+0000 7f3c6563f700 1 -- 172.21.15.46:0/2885286032 shutdown_connections 2024-05-09T04:49:47.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:47.353+0000 7f3c6563f700 1 -- 172.21.15.46:0/2885286032 wait complete. 2024-05-09T04:49:47.670 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:47 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2885286032' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T04:49:47.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:47 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2885286032' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T04:49:47.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:47 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2885286032' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-09T04:49:48.036 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411408 got 77309411414 for osd.2 2024-05-09T04:49:48.037 DEBUG:teuthology.parallel:result is None 2024-05-09T04:49:48.037 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-09T04:49:48.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T04:49:48.293 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:48.692 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:48 smithi046 ceph-mon[30136]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:49:48.781 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:48 smithi072 ceph-mon[34088]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:49:48.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:48 smithi110 ceph-mon[34904]: pgmap v354: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:49:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.011+0000 7f2320af9700 1 -- 172.21.15.46:0/1337142796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f231c1034c0 msgr2=0x7f231c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.011+0000 7f2320af9700 1 --2- 172.21.15.46:0/1337142796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f231c1034c0 0x7f231c1038d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f230c0099c0 tx=0x7f230c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.011+0000 7f2320af9700 1 -- 172.21.15.46:0/1337142796 shutdown_connections 2024-05-09T04:49:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.011+0000 7f2320af9700 1 --2- 172.21.15.46:0/1337142796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f231c108740 0x7f231c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.011+0000 7f2320af9700 1 --2- 172.21.15.46:0/1337142796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f231c1034c0 0x7f231c1038d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.011+0000 7f2320af9700 1 --2- 172.21.15.46:0/1337142796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f231c107df0 0x7f231c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.011+0000 7f2320af9700 1 -- 172.21.15.46:0/1337142796 >> 172.21.15.46:0/1337142796 conn(0x7f231c0fedf0 msgr2=0x7f231c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.011+0000 7f2320af9700 1 -- 172.21.15.46:0/1337142796 shutdown_connections 2024-05-09T04:49:50.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.011+0000 7f2320af9700 1 -- 172.21.15.46:0/1337142796 wait complete. 2024-05-09T04:49:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.012+0000 7f2320af9700 1 Processor -- start 2024-05-09T04:49:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.012+0000 7f2320af9700 1 -- start start 2024-05-09T04:49:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.012+0000 7f2320af9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f231c1034c0 0x7f231c0ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.012+0000 7f2320af9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f231c107df0 0x7f231c0ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.012+0000 7f2320af9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f231c108740 0x7f231c0ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.012+0000 7f2320af9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f231c113370 con 0x7f231c1034c0 2024-05-09T04:49:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.012+0000 7f2320af9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f231c1131f0 con 0x7f231c107df0 2024-05-09T04:49:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.012+0000 7f2320af9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f231c1134f0 con 0x7f231c108740 2024-05-09T04:49:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.013+0000 7f231a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f231c1034c0 0x7f231c0ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.013+0000 7f231a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f231c1034c0 0x7f231c0ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49758/0 (socket says 172.21.15.46:49758) 2024-05-09T04:49:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.013+0000 7f231a59c700 1 -- 172.21.15.46:0/2861463130 learned_addr learned my addr 172.21.15.46:0/2861463130 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:50.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.013+0000 7f2319d9b700 1 --2- 172.21.15.46:0/2861463130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f231c107df0 0x7f231c0ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.013+0000 7f231ad9d700 1 --2- 172.21.15.46:0/2861463130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f231c108740 0x7f231c0ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.013+0000 7f231a59c700 1 -- 172.21.15.46:0/2861463130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f231c108740 msgr2=0x7f231c0ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.013+0000 7f231a59c700 1 --2- 172.21.15.46:0/2861463130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f231c108740 0x7f231c0ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.013+0000 7f231a59c700 1 -- 172.21.15.46:0/2861463130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f231c107df0 msgr2=0x7f231c0ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.013+0000 7f231a59c700 1 --2- 172.21.15.46:0/2861463130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f231c107df0 0x7f231c0ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.013+0000 7f231a59c700 1 -- 172.21.15.46:0/2861463130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231c1a9210 con 0x7f231c1034c0 2024-05-09T04:49:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.013+0000 7f231a59c700 1 --2- 172.21.15.46:0/2861463130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f231c1034c0 0x7f231c0ff2d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f230c00b580 tx=0x7f230c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.013+0000 7f230b7fe700 1 -- 172.21.15.46:0/2861463130 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c027070 con 0x7f231c1034c0 2024-05-09T04:49:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.014+0000 7f2320af9700 1 -- 172.21.15.46:0/2861463130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231c1a94a0 con 0x7f231c1034c0 2024-05-09T04:49:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.014+0000 7f230b7fe700 1 -- 172.21.15.46:0/2861463130 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f230c003f20 con 0x7f231c1034c0 2024-05-09T04:49:50.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.014+0000 7f2320af9700 1 -- 172.21.15.46:0/2861463130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f231c1a9950 con 0x7f231c1034c0 2024-05-09T04:49:50.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.014+0000 7f230b7fe700 1 -- 172.21.15.46:0/2861463130 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230c022d10 con 0x7f231c1034c0 2024-05-09T04:49:50.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.015+0000 7f2320af9700 1 -- 172.21.15.46:0/2861463130 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22fc005330 con 0x7f231c1034c0 2024-05-09T04:49:50.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.016+0000 7f230b7fe700 1 -- 172.21.15.46:0/2861463130 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f230c022470 con 0x7f231c1034c0 2024-05-09T04:49:50.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.017+0000 7f230b7fe700 1 --2- 172.21.15.46:0/2861463130 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f23040b2860 0x7f23040b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:50.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.017+0000 7f230b7fe700 1 -- 172.21.15.46:0/2861463130 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f230c0e2ec0 con 0x7f231c1034c0 2024-05-09T04:49:50.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.017+0000 7f2319d9b700 1 --2- 172.21.15.46:0/2861463130 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f23040b2860 0x7f23040b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:50.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.018+0000 7f2319d9b700 1 --2- 172.21.15.46:0/2861463130 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f23040b2860 0x7f23040b4d70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f231c1022d0 tx=0x7f231000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:50.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.022+0000 7f230b7fe700 1 -- 172.21.15.46:0/2861463130 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f230c013070 con 0x7f231c1034c0 2024-05-09T04:49:50.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.232+0000 7f2320af9700 1 -- 172.21.15.46:0/2861463130 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f22fc000bc0 con 0x7f23040b2860 2024-05-09T04:49:50.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.240+0000 7f230b7fe700 1 -- 172.21.15.46:0/2861463130 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258994 (secure 0 0 0) 0x7f22fc000bc0 con 0x7f23040b2860 2024-05-09T04:49:50.243 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:49:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.243+0000 7f2320af9700 1 -- 172.21.15.46:0/2861463130 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f23040b2860 msgr2=0x7f23040b4d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.243+0000 7f2320af9700 1 --2- 172.21.15.46:0/2861463130 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f23040b2860 0x7f23040b4d70 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f231c1022d0 tx=0x7f231000b410 comp rx=0 tx=0).stop 2024-05-09T04:49:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.243+0000 7f2320af9700 1 -- 172.21.15.46:0/2861463130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f231c1034c0 msgr2=0x7f231c0ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.243+0000 7f2320af9700 1 --2- 172.21.15.46:0/2861463130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f231c1034c0 0x7f231c0ff2d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f230c00b580 tx=0x7f230c0047b0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.243+0000 7f2320af9700 1 -- 172.21.15.46:0/2861463130 shutdown_connections 2024-05-09T04:49:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.243+0000 7f2320af9700 1 --2- 172.21.15.46:0/2861463130 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f23040b2860 0x7f23040b4d70 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.243+0000 7f2320af9700 1 --2- 172.21.15.46:0/2861463130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f231c108740 0x7f231c0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.243+0000 7f2320af9700 1 --2- 172.21.15.46:0/2861463130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f231c1034c0 0x7f231c0ff2d0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.243+0000 7f2320af9700 1 --2- 172.21.15.46:0/2861463130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f231c107df0 0x7f231c0ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.243+0000 7f2320af9700 1 -- 172.21.15.46:0/2861463130 >> 172.21.15.46:0/2861463130 conn(0x7f231c0fedf0 msgr2=0x7f231c107420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:50.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.243+0000 7f2320af9700 1 -- 172.21.15.46:0/2861463130 shutdown_connections 2024-05-09T04:49:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:50.243+0000 7f2320af9700 1 -- 172.21.15.46:0/2861463130 wait complete. 2024-05-09T04:49:50.247 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T04:49:50.565 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:50 smithi046 ceph-mon[30136]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:49:50.565 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:50 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:49:50.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:50 smithi110 ceph-mon[34904]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:49:50.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:50 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:49:50.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:50 smithi072 ceph-mon[34088]: pgmap v355: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:49:50.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:50 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:49:51.000 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2024-05-09T04:49:49.329841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654950,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":704,"num_read_kb":439,"num_write":289,"num_write_kb":2263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":3002535,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364488,"kb_used_data":11100,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577336,"statfs":{"total":1151940427776,"available":1151567192064,"internally_reserved":0,"allocated":11366400,"data_stored":7346178,"data_compressed":255830,"data_compressed_allocated":344064,"data_compressed_original":2211840,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003630"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.397417+0000","last_change":"2024-05-09T04:47:48.408878+0000","last_active":"2024-05-09T04:49:29.397417+0000","last_peered":"2024-05-09T04:49:29.397417+0000","last_clean":"2024-05-09T04:49:29.397417+0000","last_became_active":"2024-05-09T04:47:48.408479+0000","last_became_peered":"2024-05-09T04:47:48.408479+0000","last_unstale":"2024-05-09T04:49:29.397417+0000","last_undegraded":"2024-05-09T04:49:29.397417+0000","last_fullsized":"2024-05-09T04:49:29.397417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:02.494768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.782165+0000","last_change":"2024-05-09T04:47:48.415650+0000","last_active":"2024-05-09T04:49:29.782165+0000","last_peered":"2024-05-09T04:49:29.782165+0000","last_clean":"2024-05-09T04:49:29.782165+0000","last_became_active":"2024-05-09T04:47:48.415322+0000","last_became_peered":"2024-05-09T04:47:48.415322+0000","last_unstale":"2024-05-09T04:49:29.782165+0000","last_undegraded":"2024-05-09T04:49:29.782165+0000","last_fullsized":"2024-05-09T04:49:29.782165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.142513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781957+0000","last_change":"2024-05-09T04:47:48.417706+0000","last_active":"2024-05-09T04:49:29.781957+0000","last_peered":"2024-05-09T04:49:29.781957+0000","last_clean":"2024-05-09T04:49:29.781957+0000","last_became_active":"2024-05-09T04:47:48.412068+0000","last_became_peered":"2024-05-09T04:47:48.412068+0000","last_unstale":"2024-05-09T04:49:29.781957+0000","last_undegraded":"2024-05-09T04:49:29.781957+0000","last_fullsized":"2024-05-09T04:49:29.781957+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:44.602567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.632393+0000","last_change":"2024-05-09T04:47:48.415751+0000","last_active":"2024-05-09T04:49:29.632393+0000","last_peered":"2024-05-09T04:49:29.632393+0000","last_clean":"2024-05-09T04:49:29.632393+0000","last_became_active":"2024-05-09T04:47:48.414312+0000","last_became_peered":"2024-05-09T04:47:48.414312+0000","last_unstale":"2024-05-09T04:49:29.632393+0000","last_undegraded":"2024-05-09T04:49:29.632393+0000","last_fullsized":"2024-05-09T04:49:29.632393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:12.202761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399222+0000","last_change":"2024-05-09T04:47:48.415919+0000","last_active":"2024-05-09T04:49:29.399222+0000","last_peered":"2024-05-09T04:49:29.399222+0000","last_clean":"2024-05-09T04:49:29.399222+0000","last_became_active":"2024-05-09T04:47:48.415533+0000","last_became_peered":"2024-05-09T04:47:48.415533+0000","last_unstale":"2024-05-09T04:49:29.399222+0000","last_undegraded":"2024-05-09T04:49:29.399222+0000","last_fullsized":"2024-05-09T04:49:29.399222+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:32.262416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394968+0000","last_change":"2024-05-09T04:47:48.419705+0000","last_active":"2024-05-09T04:49:29.394968+0000","last_peered":"2024-05-09T04:49:29.394968+0000","last_clean":"2024-05-09T04:49:29.394968+0000","last_became_active":"2024-05-09T04:47:48.419339+0000","last_became_peered":"2024-05-09T04:47:48.419339+0000","last_unstale":"2024-05-09T04:49:29.394968+0000","last_undegraded":"2024-05-09T04:49:29.394968+0000","last_fullsized":"2024-05-09T04:49:29.394968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:50:23.754071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780603+0000","last_change":"2024-05-09T04:47:48.404106+0000","last_active":"2024-05-09T04:49:29.780603+0000","last_peered":"2024-05-09T04:49:29.780603+0000","last_clean":"2024-05-09T04:49:29.780603+0000","last_became_active":"2024-05-09T04:47:48.403724+0000","last_became_peered":"2024-05-09T04:47:48.403724+0000","last_unstale":"2024-05-09T04:49:29.780603+0000","last_undegraded":"2024-05-09T04:49:29.780603+0000","last_fullsized":"2024-05-09T04:49:29.780603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:41.134752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780621+0000","last_change":"2024-05-09T04:48:28.562515+0000","last_active":"2024-05-09T04:49:29.780621+0000","last_peered":"2024-05-09T04:49:29.780621+0000","last_clean":"2024-05-09T04:49:29.780621+0000","last_became_active":"2024-05-09T04:48:28.562104+0000","last_became_peered":"2024-05-09T04:48:28.562104+0000","last_unstale":"2024-05-09T04:49:29.780621+0000","last_undegraded":"2024-05-09T04:49:29.780621+0000","last_fullsized":"2024-05-09T04:49:29.780621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:33.342877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399309+0000","last_change":"2024-05-09T04:47:48.400467+0000","last_active":"2024-05-09T04:49:29.399309+0000","last_peered":"2024-05-09T04:49:29.399309+0000","last_clean":"2024-05-09T04:49:29.399309+0000","last_became_active":"2024-05-09T04:47:48.399960+0000","last_became_peered":"2024-05-09T04:47:48.399960+0000","last_unstale":"2024-05-09T04:49:29.399309+0000","last_undegraded":"2024-05-09T04:49:29.399309+0000","last_fullsized":"2024-05-09T04:49:29.399309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:23:44.473053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.395358+0000","last_change":"2024-05-09T04:47:48.418970+0000","last_active":"2024-05-09T04:49:29.395358+0000","last_peered":"2024-05-09T04:49:29.395358+0000","last_clean":"2024-05-09T04:49:29.395358+0000","last_became_active":"2024-05-09T04:47:48.418574+0000","last_became_peered":"2024-05-09T04:47:48.418574+0000","last_unstale":"2024-05-09T04:49:29.395358+0000","last_undegraded":"2024-05-09T04:49:29.395358+0000","last_fullsized":"2024-05-09T04:49:29.395358+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:48.075893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780929+0000","last_change":"2024-05-09T04:47:48.409879+0000","last_active":"2024-05-09T04:49:29.780929+0000","last_peered":"2024-05-09T04:49:29.780929+0000","last_clean":"2024-05-09T04:49:29.780929+0000","last_became_active":"2024-05-09T04:47:48.409515+0000","last_became_peered":"2024-05-09T04:47:48.409515+0000","last_unstale":"2024-05-09T04:49:29.780929+0000","last_undegraded":"2024-05-09T04:49:29.780929+0000","last_fullsized":"2024-05-09T04:49:29.780929+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:15.546214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.632290+0000","last_change":"2024-05-09T04:47:48.417556+0000","last_active":"2024-05-09T04:49:29.632290+0000","last_peered":"2024-05-09T04:49:29.632290+0000","last_clean":"2024-05-09T04:49:29.632290+0000","last_became_active":"2024-05-09T04:47:48.416787+0000","last_became_peered":"2024-05-09T04:47:48.416787+0000","last_unstale":"2024-05-09T04:49:29.632290+0000","last_undegraded":"2024-05-09T04:49:29.632290+0000","last_fullsized":"2024-05-09T04:49:29.632290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:20.360232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394343+0000","last_change":"2024-05-09T04:47:48.415145+0000","last_active":"2024-05-09T04:49:29.394343+0000","last_peered":"2024-05-09T04:49:29.394343+0000","last_clean":"2024-05-09T04:49:29.394343+0000","last_became_active":"2024-05-09T04:47:48.414830+0000","last_became_peered":"2024-05-09T04:47:48.414830+0000","last_unstale":"2024-05-09T04:49:29.394343+0000","last_undegraded":"2024-05-09T04:49:29.394343+0000","last_fullsized":"2024-05-09T04:49:29.394343+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:55.936409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781684+0000","last_change":"2024-05-09T04:47:48.413977+0000","last_active":"2024-05-09T04:49:29.781684+0000","last_peered":"2024-05-09T04:49:29.781684+0000","last_clean":"2024-05-09T04:49:29.781684+0000","last_became_active":"2024-05-09T04:47:48.413604+0000","last_became_peered":"2024-05-09T04:47:48.413604+0000","last_unstale":"2024-05-09T04:49:29.781684+0000","last_undegraded":"2024-05-09T04:49:29.781684+0000","last_fullsized":"2024-05-09T04:49:29.781684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:25.462958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781580+0000","last_change":"2024-05-09T04:47:48.419000+0000","last_active":"2024-05-09T04:49:29.781580+0000","last_peered":"2024-05-09T04:49:29.781580+0000","last_clean":"2024-05-09T04:49:29.781580+0000","last_became_active":"2024-05-09T04:47:48.418457+0000","last_became_peered":"2024-05-09T04:47:48.418457+0000","last_unstale":"2024-05-09T04:49:29.781580+0000","last_undegraded":"2024-05-09T04:49:29.781580+0000","last_fullsized":"2024-05-09T04:49:29.781580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:36.325460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.398747+0000","last_change":"2024-05-09T04:47:48.406754+0000","last_active":"2024-05-09T04:49:29.398747+0000","last_peered":"2024-05-09T04:49:29.398747+0000","last_clean":"2024-05-09T04:49:29.398747+0000","last_became_active":"2024-05-09T04:47:48.406396+0000","last_became_peered":"2024-05-09T04:47:48.406396+0000","last_unstale":"2024-05-09T04:49:29.398747+0000","last_undegraded":"2024-05-09T04:49:29.398747+0000","last_fullsized":"2024-05-09T04:49:29.398747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:46.421061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.397568+0000","last_change":"2024-05-09T04:47:48.409556+0000","last_active":"2024-05-09T04:49:29.397568+0000","last_peered":"2024-05-09T04:49:29.397568+0000","last_clean":"2024-05-09T04:49:29.397568+0000","last_became_active":"2024-05-09T04:47:48.409145+0000","last_became_peered":"2024-05-09T04:47:48.409145+0000","last_unstale":"2024-05-09T04:49:29.397568+0000","last_undegraded":"2024-05-09T04:49:29.397568+0000","last_fullsized":"2024-05-09T04:49:29.397568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:25.486257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781325+0000","last_change":"2024-05-09T04:47:48.415546+0000","last_active":"2024-05-09T04:49:29.781325+0000","last_peered":"2024-05-09T04:49:29.781325+0000","last_clean":"2024-05-09T04:49:29.781325+0000","last_became_active":"2024-05-09T04:47:48.415221+0000","last_became_peered":"2024-05-09T04:47:48.415221+0000","last_unstale":"2024-05-09T04:49:29.781325+0000","last_undegraded":"2024-05-09T04:49:29.781325+0000","last_fullsized":"2024-05-09T04:49:29.781325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:57:29.112409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781020+0000","last_change":"2024-05-09T04:47:48.413106+0000","last_active":"2024-05-09T04:49:29.781020+0000","last_peered":"2024-05-09T04:49:29.781020+0000","last_clean":"2024-05-09T04:49:29.781020+0000","last_became_active":"2024-05-09T04:47:48.412700+0000","last_became_peered":"2024-05-09T04:47:48.412700+0000","last_unstale":"2024-05-09T04:49:29.781020+0000","last_undegraded":"2024-05-09T04:49:29.781020+0000","last_fullsized":"2024-05-09T04:49:29.781020+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:37:31.682008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781180+0000","last_change":"2024-05-09T04:47:45.386931+0000","last_active":"2024-05-09T04:49:29.781180+0000","last_peered":"2024-05-09T04:49:29.781180+0000","last_clean":"2024-05-09T04:49:29.781180+0000","last_became_active":"2024-05-09T04:47:45.386562+0000","last_became_peered":"2024-05-09T04:47:45.386562+0000","last_unstale":"2024-05-09T04:49:29.781180+0000","last_undegraded":"2024-05-09T04:49:29.781180+0000","last_fullsized":"2024-05-09T04:49:29.781180+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:59.335687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781714+0000","last_change":"2024-05-09T04:47:48.411064+0000","last_active":"2024-05-09T04:49:29.781714+0000","last_peered":"2024-05-09T04:49:29.781714+0000","last_clean":"2024-05-09T04:49:29.781714+0000","last_became_active":"2024-05-09T04:47:48.410689+0000","last_became_peered":"2024-05-09T04:47:48.410689+0000","last_unstale":"2024-05-09T04:49:29.781714+0000","last_undegraded":"2024-05-09T04:49:29.781714+0000","last_fullsized":"2024-05-09T04:49:29.781714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:21.178831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781574+0000","last_change":"2024-05-09T04:47:45.612837+0000","last_active":"2024-05-09T04:49:29.781574+0000","last_peered":"2024-05-09T04:49:29.781574+0000","last_clean":"2024-05-09T04:49:29.781574+0000","last_became_active":"2024-05-09T04:47:45.612307+0000","last_became_peered":"2024-05-09T04:47:45.612307+0000","last_unstale":"2024-05-09T04:49:29.781574+0000","last_undegraded":"2024-05-09T04:49:29.781574+0000","last_fullsized":"2024-05-09T04:49:29.781574+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:59.686391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.782117+0000","last_change":"2024-05-09T04:47:48.412781+0000","last_active":"2024-05-09T04:49:29.782117+0000","last_peered":"2024-05-09T04:49:29.782117+0000","last_clean":"2024-05-09T04:49:29.782117+0000","last_became_active":"2024-05-09T04:47:48.411396+0000","last_became_peered":"2024-05-09T04:47:48.411396+0000","last_unstale":"2024-05-09T04:49:29.782117+0000","last_undegraded":"2024-05-09T04:49:29.782117+0000","last_fullsized":"2024-05-09T04:49:29.782117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:41:22.775432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.397190+0000","last_change":"2024-05-09T04:48:28.568024+0000","last_active":"2024-05-09T04:49:29.397190+0000","last_peered":"2024-05-09T04:49:29.397190+0000","last_clean":"2024-05-09T04:49:29.397190+0000","last_became_active":"2024-05-09T04:48:28.559441+0000","last_became_peered":"2024-05-09T04:48:28.559441+0000","last_unstale":"2024-05-09T04:49:29.397190+0000","last_undegraded":"2024-05-09T04:49:29.397190+0000","last_fullsized":"2024-05-09T04:49:29.397190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:56.257589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.398866+0000","last_change":"2024-05-09T04:47:48.421361+0000","last_active":"2024-05-09T04:49:29.398866+0000","last_peered":"2024-05-09T04:49:29.398866+0000","last_clean":"2024-05-09T04:49:29.398866+0000","last_became_active":"2024-05-09T04:47:48.421018+0000","last_became_peered":"2024-05-09T04:47:48.421018+0000","last_unstale":"2024-05-09T04:49:29.398866+0000","last_undegraded":"2024-05-09T04:49:29.398866+0000","last_fullsized":"2024-05-09T04:49:29.398866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:00.634983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.403356+0000","last_change":"2024-05-09T04:49:29.402935+0000","last_active":"2024-05-09T04:49:29.403356+0000","last_peered":"2024-05-09T04:49:29.403356+0000","last_clean":"2024-05-09T04:49:29.403356+0000","last_became_active":"2024-05-09T04:49:29.400084+0000","last_became_peered":"2024-05-09T04:49:29.400084+0000","last_unstale":"2024-05-09T04:49:29.403356+0000","last_undegraded":"2024-05-09T04:49:29.403356+0000","last_fullsized":"2024-05-09T04:49:29.403356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:06:06.683122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394894+0000","last_change":"2024-05-09T04:48:28.564243+0000","last_active":"2024-05-09T04:49:29.394894+0000","last_peered":"2024-05-09T04:49:29.394894+0000","last_clean":"2024-05-09T04:49:29.394894+0000","last_became_active":"2024-05-09T04:48:28.563732+0000","last_became_peered":"2024-05-09T04:48:28.563732+0000","last_unstale":"2024-05-09T04:49:29.394894+0000","last_undegraded":"2024-05-09T04:49:29.394894+0000","last_fullsized":"2024-05-09T04:49:29.394894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:26.982339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":137,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:47.655992+0000","last_change":"2024-05-09T04:47:45.389868+0000","last_active":"2024-05-09T04:49:47.655992+0000","last_peered":"2024-05-09T04:49:47.655992+0000","last_clean":"2024-05-09T04:49:47.655992+0000","last_became_active":"2024-05-09T04:47:45.389508+0000","last_became_peered":"2024-05-09T04:47:45.389508+0000","last_unstale":"2024-05-09T04:49:47.655992+0000","last_undegraded":"2024-05-09T04:49:47.655992+0000","last_fullsized":"2024-05-09T04:49:47.655992+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:45.050187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399110+0000","last_change":"2024-05-09T04:47:48.404457+0000","last_active":"2024-05-09T04:49:29.399110+0000","last_peered":"2024-05-09T04:49:29.399110+0000","last_clean":"2024-05-09T04:49:29.399110+0000","last_became_active":"2024-05-09T04:47:48.403968+0000","last_became_peered":"2024-05-09T04:47:48.403968+0000","last_unstale":"2024-05-09T04:49:29.399110+0000","last_undegraded":"2024-05-09T04:49:29.399110+0000","last_fullsized":"2024-05-09T04:49:29.399110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:23:40.364916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394856+0000","last_change":"2024-05-09T04:47:45.388183+0000","last_active":"2024-05-09T04:49:29.394856+0000","last_peered":"2024-05-09T04:49:29.394856+0000","last_clean":"2024-05-09T04:49:29.394856+0000","last_became_active":"2024-05-09T04:47:45.386603+0000","last_became_peered":"2024-05-09T04:47:45.386603+0000","last_unstale":"2024-05-09T04:49:29.394856+0000","last_undegraded":"2024-05-09T04:49:29.394856+0000","last_fullsized":"2024-05-09T04:49:29.394856+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:53.433619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780754+0000","last_change":"2024-05-09T04:48:28.663120+0000","last_active":"2024-05-09T04:49:29.780754+0000","last_peered":"2024-05-09T04:49:29.780754+0000","last_clean":"2024-05-09T04:49:29.780754+0000","last_became_active":"2024-05-09T04:48:28.662748+0000","last_became_peered":"2024-05-09T04:48:28.662748+0000","last_unstale":"2024-05-09T04:49:29.780754+0000","last_undegraded":"2024-05-09T04:49:29.780754+0000","last_fullsized":"2024-05-09T04:49:29.780754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:57.677876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394206+0000","last_change":"2024-05-09T04:47:45.384192+0000","last_active":"2024-05-09T04:49:29.394206+0000","last_peered":"2024-05-09T04:49:29.394206+0000","last_clean":"2024-05-09T04:49:29.394206+0000","last_became_active":"2024-05-09T04:47:45.383843+0000","last_became_peered":"2024-05-09T04:47:45.383843+0000","last_unstale":"2024-05-09T04:49:29.394206+0000","last_undegraded":"2024-05-09T04:49:29.394206+0000","last_fullsized":"2024-05-09T04:49:29.394206+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:28.815207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780961+0000","last_change":"2024-05-09T04:47:48.409203+0000","last_active":"2024-05-09T04:49:29.780961+0000","last_peered":"2024-05-09T04:49:29.780961+0000","last_clean":"2024-05-09T04:49:29.780961+0000","last_became_active":"2024-05-09T04:47:48.408861+0000","last_became_peered":"2024-05-09T04:47:48.408861+0000","last_unstale":"2024-05-09T04:49:29.780961+0000","last_undegraded":"2024-05-09T04:49:29.780961+0000","last_fullsized":"2024-05-09T04:49:29.780961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:07.160397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394642+0000","last_change":"2024-05-09T04:47:45.392723+0000","last_active":"2024-05-09T04:49:29.394642+0000","last_peered":"2024-05-09T04:49:29.394642+0000","last_clean":"2024-05-09T04:49:29.394642+0000","last_became_active":"2024-05-09T04:47:45.392313+0000","last_became_peered":"2024-05-09T04:47:45.392313+0000","last_unstale":"2024-05-09T04:49:29.394642+0000","last_undegraded":"2024-05-09T04:49:29.394642+0000","last_fullsized":"2024-05-09T04:49:29.394642+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:44.358702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780790+0000","last_change":"2024-05-09T04:47:48.414000+0000","last_active":"2024-05-09T04:49:29.780790+0000","last_peered":"2024-05-09T04:49:29.780790+0000","last_clean":"2024-05-09T04:49:29.780790+0000","last_became_active":"2024-05-09T04:47:48.413532+0000","last_became_peered":"2024-05-09T04:47:48.413532+0000","last_unstale":"2024-05-09T04:49:29.780790+0000","last_undegraded":"2024-05-09T04:49:29.780790+0000","last_fullsized":"2024-05-09T04:49:29.780790+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:22.774814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":133,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:47.655215+0000","last_change":"2024-05-09T04:47:45.388661+0000","last_active":"2024-05-09T04:49:47.655215+0000","last_peered":"2024-05-09T04:49:47.655215+0000","last_clean":"2024-05-09T04:49:47.655215+0000","last_became_active":"2024-05-09T04:47:45.386010+0000","last_became_peered":"2024-05-09T04:47:45.386010+0000","last_unstale":"2024-05-09T04:49:47.655215+0000","last_undegraded":"2024-05-09T04:49:47.655215+0000","last_fullsized":"2024-05-09T04:49:47.655215+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:23.250463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399287+0000","last_change":"2024-05-09T04:47:48.421706+0000","last_active":"2024-05-09T04:49:29.399287+0000","last_peered":"2024-05-09T04:49:29.399287+0000","last_clean":"2024-05-09T04:49:29.399287+0000","last_became_active":"2024-05-09T04:47:48.420529+0000","last_became_peered":"2024-05-09T04:47:48.420529+0000","last_unstale":"2024-05-09T04:49:29.399287+0000","last_undegraded":"2024-05-09T04:49:29.399287+0000","last_fullsized":"2024-05-09T04:49:29.399287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:10.110366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781039+0000","last_change":"2024-05-09T04:47:45.386625+0000","last_active":"2024-05-09T04:49:29.781039+0000","last_peered":"2024-05-09T04:49:29.781039+0000","last_clean":"2024-05-09T04:49:29.781039+0000","last_became_active":"2024-05-09T04:47:45.385087+0000","last_became_peered":"2024-05-09T04:47:45.385087+0000","last_unstale":"2024-05-09T04:49:29.781039+0000","last_undegraded":"2024-05-09T04:49:29.781039+0000","last_fullsized":"2024-05-09T04:49:29.781039+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:15.546894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.782090+0000","last_change":"2024-05-09T04:48:28.562517+0000","last_active":"2024-05-09T04:49:29.782090+0000","last_peered":"2024-05-09T04:49:29.782090+0000","last_clean":"2024-05-09T04:49:29.782090+0000","last_became_active":"2024-05-09T04:48:28.562047+0000","last_became_peered":"2024-05-09T04:48:28.562047+0000","last_unstale":"2024-05-09T04:49:29.782090+0000","last_undegraded":"2024-05-09T04:49:29.782090+0000","last_fullsized":"2024-05-09T04:49:29.782090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:22.769844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.395036+0000","last_change":"2024-05-09T04:47:45.613713+0000","last_active":"2024-05-09T04:49:29.395036+0000","last_peered":"2024-05-09T04:49:29.395036+0000","last_clean":"2024-05-09T04:49:29.395036+0000","last_became_active":"2024-05-09T04:47:45.613313+0000","last_became_peered":"2024-05-09T04:47:45.613313+0000","last_unstale":"2024-05-09T04:49:29.395036+0000","last_undegraded":"2024-05-09T04:49:29.395036+0000","last_fullsized":"2024-05-09T04:49:29.395036+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:32:44.825695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399234+0000","last_change":"2024-05-09T04:47:48.411234+0000","last_active":"2024-05-09T04:49:29.399234+0000","last_peered":"2024-05-09T04:49:29.399234+0000","last_clean":"2024-05-09T04:49:29.399234+0000","last_became_active":"2024-05-09T04:47:48.410881+0000","last_became_peered":"2024-05-09T04:47:48.410881+0000","last_unstale":"2024-05-09T04:49:29.399234+0000","last_undegraded":"2024-05-09T04:49:29.399234+0000","last_fullsized":"2024-05-09T04:49:29.399234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.747871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'29","reported_seq":287,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:48.681241+0000","last_change":"2024-05-09T04:48:28.570426+0000","last_active":"2024-05-09T04:49:48.681241+0000","last_peered":"2024-05-09T04:49:48.681241+0000","last_clean":"2024-05-09T04:49:48.681241+0000","last_became_active":"2024-05-09T04:48:28.561786+0000","last_became_peered":"2024-05-09T04:48:28.561786+0000","last_unstale":"2024-05-09T04:49:48.681241+0000","last_undegraded":"2024-05-09T04:49:48.681241+0000","last_fullsized":"2024-05-09T04:49:48.681241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:32.367142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":206,"num_read_kb":153,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":50695,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399373+0000","last_change":"2024-05-09T04:47:48.406510+0000","last_active":"2024-05-09T04:49:29.399373+0000","last_peered":"2024-05-09T04:49:29.399373+0000","last_clean":"2024-05-09T04:49:29.399373+0000","last_became_active":"2024-05-09T04:47:48.406166+0000","last_became_peered":"2024-05-09T04:47:48.406166+0000","last_unstale":"2024-05-09T04:49:29.399373+0000","last_undegraded":"2024-05-09T04:49:29.399373+0000","last_fullsized":"2024-05-09T04:49:29.399373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:18.579620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780774+0000","last_change":"2024-05-09T04:47:45.387164+0000","last_active":"2024-05-09T04:49:29.780774+0000","last_peered":"2024-05-09T04:49:29.780774+0000","last_clean":"2024-05-09T04:49:29.780774+0000","last_became_active":"2024-05-09T04:47:45.386755+0000","last_became_peered":"2024-05-09T04:47:45.386755+0000","last_unstale":"2024-05-09T04:49:29.780774+0000","last_undegraded":"2024-05-09T04:49:29.780774+0000","last_fullsized":"2024-05-09T04:49:29.780774+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:07.838782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394112+0000","last_change":"2024-05-09T04:47:48.405436+0000","last_active":"2024-05-09T04:49:29.394112+0000","last_peered":"2024-05-09T04:49:29.394112+0000","last_clean":"2024-05-09T04:49:29.394112+0000","last_became_active":"2024-05-09T04:47:48.405032+0000","last_became_peered":"2024-05-09T04:47:48.405032+0000","last_unstale":"2024-05-09T04:49:29.394112+0000","last_undegraded":"2024-05-09T04:49:29.394112+0000","last_fullsized":"2024-05-09T04:49:29.394112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:27.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394859+0000","last_change":"2024-05-09T04:47:45.391970+0000","last_active":"2024-05-09T04:49:29.394859+0000","last_peered":"2024-05-09T04:49:29.394859+0000","last_clean":"2024-05-09T04:49:29.394859+0000","last_became_active":"2024-05-09T04:47:45.391597+0000","last_became_peered":"2024-05-09T04:47:45.391597+0000","last_unstale":"2024-05-09T04:49:29.394859+0000","last_undegraded":"2024-05-09T04:49:29.394859+0000","last_fullsized":"2024-05-09T04:49:29.394859+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:27.076631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780897+0000","last_change":"2024-05-09T04:47:48.419328+0000","last_active":"2024-05-09T04:49:29.780897+0000","last_peered":"2024-05-09T04:49:29.780897+0000","last_clean":"2024-05-09T04:49:29.780897+0000","last_became_active":"2024-05-09T04:47:48.418972+0000","last_became_peered":"2024-05-09T04:47:48.418972+0000","last_unstale":"2024-05-09T04:49:29.780897+0000","last_undegraded":"2024-05-09T04:49:29.780897+0000","last_fullsized":"2024-05-09T04:49:29.780897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:15.164747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":136,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:45.664451+0000","last_change":"2024-05-09T04:47:45.386173+0000","last_active":"2024-05-09T04:49:45.664451+0000","last_peered":"2024-05-09T04:49:45.664451+0000","last_clean":"2024-05-09T04:49:45.664451+0000","last_became_active":"2024-05-09T04:47:45.384296+0000","last_became_peered":"2024-05-09T04:47:45.384296+0000","last_unstale":"2024-05-09T04:49:45.664451+0000","last_undegraded":"2024-05-09T04:49:45.664451+0000","last_fullsized":"2024-05-09T04:49:45.664451+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:03.246187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780422+0000","last_change":"2024-05-09T04:47:48.415645+0000","last_active":"2024-05-09T04:49:29.780422+0000","last_peered":"2024-05-09T04:49:29.780422+0000","last_clean":"2024-05-09T04:49:29.780422+0000","last_became_active":"2024-05-09T04:47:48.415303+0000","last_became_peered":"2024-05-09T04:47:48.415303+0000","last_unstale":"2024-05-09T04:49:29.780422+0000","last_undegraded":"2024-05-09T04:49:29.780422+0000","last_fullsized":"2024-05-09T04:49:29.780422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:39.703948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":46,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399050+0000","last_change":"2024-05-09T04:47:45.613320+0000","last_active":"2024-05-09T04:49:29.399050+0000","last_peered":"2024-05-09T04:49:29.399050+0000","last_clean":"2024-05-09T04:49:29.399050+0000","last_became_active":"2024-05-09T04:47:45.612871+0000","last_became_peered":"2024-05-09T04:47:45.612871+0000","last_unstale":"2024-05-09T04:49:29.399050+0000","last_undegraded":"2024-05-09T04:49:29.399050+0000","last_fullsized":"2024-05-09T04:49:29.399050+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:31.516652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780915+0000","last_change":"2024-05-09T04:48:28.563376+0000","last_active":"2024-05-09T04:49:29.780915+0000","last_peered":"2024-05-09T04:49:29.780915+0000","last_clean":"2024-05-09T04:49:29.780915+0000","last_became_active":"2024-05-09T04:48:28.562973+0000","last_became_peered":"2024-05-09T04:48:28.562973+0000","last_unstale":"2024-05-09T04:49:29.780915+0000","last_undegraded":"2024-05-09T04:49:29.780915+0000","last_fullsized":"2024-05-09T04:49:29.780915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:20.068517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399757+0000","last_change":"2024-05-09T04:47:45.389785+0000","last_active":"2024-05-09T04:49:29.399757+0000","last_peered":"2024-05-09T04:49:29.399757+0000","last_clean":"2024-05-09T04:49:29.399757+0000","last_became_active":"2024-05-09T04:47:45.389159+0000","last_became_peered":"2024-05-09T04:47:45.389159+0000","last_unstale":"2024-05-09T04:49:29.399757+0000","last_undegraded":"2024-05-09T04:49:29.399757+0000","last_fullsized":"2024-05-09T04:49:29.399757+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:28:51.804123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.397253+0000","last_change":"2024-05-09T04:48:28.563170+0000","last_active":"2024-05-09T04:49:29.397253+0000","last_peered":"2024-05-09T04:49:29.397253+0000","last_clean":"2024-05-09T04:49:29.397253+0000","last_became_active":"2024-05-09T04:48:28.562189+0000","last_became_peered":"2024-05-09T04:48:28.562189+0000","last_unstale":"2024-05-09T04:49:29.397253+0000","last_undegraded":"2024-05-09T04:49:29.397253+0000","last_fullsized":"2024-05-09T04:49:29.397253+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:00.643292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780838+0000","last_change":"2024-05-09T04:47:45.613739+0000","last_active":"2024-05-09T04:49:29.780838+0000","last_peered":"2024-05-09T04:49:29.780838+0000","last_clean":"2024-05-09T04:49:29.780838+0000","last_became_active":"2024-05-09T04:47:45.613365+0000","last_became_peered":"2024-05-09T04:47:45.613365+0000","last_unstale":"2024-05-09T04:49:29.780838+0000","last_undegraded":"2024-05-09T04:49:29.780838+0000","last_fullsized":"2024-05-09T04:49:29.780838+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:57.138068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.395050+0000","last_change":"2024-05-09T04:47:48.413343+0000","last_active":"2024-05-09T04:49:29.395050+0000","last_peered":"2024-05-09T04:49:29.395050+0000","last_clean":"2024-05-09T04:49:29.395050+0000","last_became_active":"2024-05-09T04:47:48.412941+0000","last_became_peered":"2024-05-09T04:47:48.412941+0000","last_unstale":"2024-05-09T04:49:29.395050+0000","last_undegraded":"2024-05-09T04:49:29.395050+0000","last_fullsized":"2024-05-09T04:49:29.395050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:28.764553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780691+0000","last_change":"2024-05-09T04:47:45.386487+0000","last_active":"2024-05-09T04:49:29.780691+0000","last_peered":"2024-05-09T04:49:29.780691+0000","last_clean":"2024-05-09T04:49:29.780691+0000","last_became_active":"2024-05-09T04:47:45.386078+0000","last_became_peered":"2024-05-09T04:47:45.386078+0000","last_unstale":"2024-05-09T04:49:29.780691+0000","last_undegraded":"2024-05-09T04:49:29.780691+0000","last_fullsized":"2024-05-09T04:49:29.780691+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:19.998977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394993+0000","last_change":"2024-05-09T04:47:48.418955+0000","last_active":"2024-05-09T04:49:29.394993+0000","last_peered":"2024-05-09T04:49:29.394993+0000","last_clean":"2024-05-09T04:49:29.394993+0000","last_became_active":"2024-05-09T04:47:48.418528+0000","last_became_peered":"2024-05-09T04:47:48.418528+0000","last_unstale":"2024-05-09T04:49:29.394993+0000","last_undegraded":"2024-05-09T04:49:29.394993+0000","last_fullsized":"2024-05-09T04:49:29.394993+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:51:02.386342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.395407+0000","last_change":"2024-05-09T04:47:48.413304+0000","last_active":"2024-05-09T04:49:29.395407+0000","last_peered":"2024-05-09T04:49:29.395407+0000","last_clean":"2024-05-09T04:49:29.395407+0000","last_became_active":"2024-05-09T04:47:48.412582+0000","last_became_peered":"2024-05-09T04:47:48.412582+0000","last_unstale":"2024-05-09T04:49:29.395407+0000","last_undegraded":"2024-05-09T04:49:29.395407+0000","last_fullsized":"2024-05-09T04:49:29.395407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:40.027082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":131,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:43.659958+0000","last_change":"2024-05-09T04:47:45.612742+0000","last_active":"2024-05-09T04:49:43.659958+0000","last_peered":"2024-05-09T04:49:43.659958+0000","last_clean":"2024-05-09T04:49:43.659958+0000","last_became_active":"2024-05-09T04:47:45.612325+0000","last_became_peered":"2024-05-09T04:47:45.612325+0000","last_unstale":"2024-05-09T04:49:43.659958+0000","last_undegraded":"2024-05-09T04:49:43.659958+0000","last_fullsized":"2024-05-09T04:49:43.659958+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:00.573996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781304+0000","last_change":"2024-05-09T04:47:48.412894+0000","last_active":"2024-05-09T04:49:29.781304+0000","last_peered":"2024-05-09T04:49:29.781304+0000","last_clean":"2024-05-09T04:49:29.781304+0000","last_became_active":"2024-05-09T04:47:48.412265+0000","last_became_peered":"2024-05-09T04:47:48.412265+0000","last_unstale":"2024-05-09T04:49:29.781304+0000","last_undegraded":"2024-05-09T04:49:29.781304+0000","last_fullsized":"2024-05-09T04:49:29.781304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:23.517566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":35,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.632081+0000","last_change":"2024-05-09T04:47:45.387715+0000","last_active":"2024-05-09T04:49:29.632081+0000","last_peered":"2024-05-09T04:49:29.632081+0000","last_clean":"2024-05-09T04:49:29.632081+0000","last_became_active":"2024-05-09T04:47:45.387229+0000","last_became_peered":"2024-05-09T04:47:45.387229+0000","last_unstale":"2024-05-09T04:49:29.632081+0000","last_undegraded":"2024-05-09T04:49:29.632081+0000","last_fullsized":"2024-05-09T04:49:29.632081+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:22.540214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":104,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780701+0000","last_change":"2024-05-09T04:46:21.632579+0000","last_active":"2024-05-09T04:49:29.780701+0000","last_peered":"2024-05-09T04:49:29.780701+0000","last_clean":"2024-05-09T04:49:29.780701+0000","last_became_active":"2024-05-09T04:46:21.627483+0000","last_became_peered":"2024-05-09T04:46:21.627483+0000","last_unstale":"2024-05-09T04:49:29.780701+0000","last_undegraded":"2024-05-09T04:49:29.780701+0000","last_fullsized":"2024-05-09T04:49:29.780701+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:42:54.179739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:42:54.179739+0000","last_clean_scrub_stamp":"2024-05-09T04:42:54.179739+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:44.331526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394006+0000","last_change":"2024-05-09T04:47:48.418570+0000","last_active":"2024-05-09T04:49:29.394006+0000","last_peered":"2024-05-09T04:49:29.394006+0000","last_clean":"2024-05-09T04:49:29.394006+0000","last_became_active":"2024-05-09T04:47:48.418270+0000","last_became_peered":"2024-05-09T04:47:48.418270+0000","last_unstale":"2024-05-09T04:49:29.394006+0000","last_undegraded":"2024-05-09T04:49:29.394006+0000","last_fullsized":"2024-05-09T04:49:29.394006+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:28.406846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780915+0000","last_change":"2024-05-09T04:47:45.387161+0000","last_active":"2024-05-09T04:49:29.780915+0000","last_peered":"2024-05-09T04:49:29.780915+0000","last_clean":"2024-05-09T04:49:29.780915+0000","last_became_active":"2024-05-09T04:47:45.386753+0000","last_became_peered":"2024-05-09T04:47:45.386753+0000","last_unstale":"2024-05-09T04:49:29.780915+0000","last_undegraded":"2024-05-09T04:49:29.780915+0000","last_fullsized":"2024-05-09T04:49:29.780915+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:15.402418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399564+0000","last_change":"2024-05-09T04:47:45.613551+0000","last_active":"2024-05-09T04:49:29.399564+0000","last_peered":"2024-05-09T04:49:29.399564+0000","last_clean":"2024-05-09T04:49:29.399564+0000","last_became_active":"2024-05-09T04:47:45.613086+0000","last_became_peered":"2024-05-09T04:47:45.613086+0000","last_unstale":"2024-05-09T04:49:29.399564+0000","last_undegraded":"2024-05-09T04:49:29.399564+0000","last_fullsized":"2024-05-09T04:49:29.399564+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:04.350960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394435+0000","last_change":"2024-05-09T04:47:48.403701+0000","last_active":"2024-05-09T04:49:29.394435+0000","last_peered":"2024-05-09T04:49:29.394435+0000","last_clean":"2024-05-09T04:49:29.394435+0000","last_became_active":"2024-05-09T04:47:48.403337+0000","last_became_peered":"2024-05-09T04:47:48.403337+0000","last_unstale":"2024-05-09T04:49:29.394435+0000","last_undegraded":"2024-05-09T04:49:29.394435+0000","last_fullsized":"2024-05-09T04:49:29.394435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:53.032348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781201+0000","last_change":"2024-05-09T04:47:45.385935+0000","last_active":"2024-05-09T04:49:29.781201+0000","last_peered":"2024-05-09T04:49:29.781201+0000","last_clean":"2024-05-09T04:49:29.781201+0000","last_became_active":"2024-05-09T04:47:45.385533+0000","last_became_peered":"2024-05-09T04:47:45.385533+0000","last_unstale":"2024-05-09T04:49:29.781201+0000","last_undegraded":"2024-05-09T04:49:29.781201+0000","last_fullsized":"2024-05-09T04:49:29.781201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:34:38.014201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.782032+0000","last_change":"2024-05-09T04:47:48.409590+0000","last_active":"2024-05-09T04:49:29.782032+0000","last_peered":"2024-05-09T04:49:29.782032+0000","last_clean":"2024-05-09T04:49:29.782032+0000","last_became_active":"2024-05-09T04:47:48.409191+0000","last_became_peered":"2024-05-09T04:47:48.409191+0000","last_unstale":"2024-05-09T04:49:29.782032+0000","last_undegraded":"2024-05-09T04:49:29.782032+0000","last_fullsized":"2024-05-09T04:49:29.782032+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:50.440609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.632285+0000","last_change":"2024-05-09T04:47:45.389147+0000","last_active":"2024-05-09T04:49:29.632285+0000","last_peered":"2024-05-09T04:49:29.632285+0000","last_clean":"2024-05-09T04:49:29.632285+0000","last_became_active":"2024-05-09T04:47:45.388797+0000","last_became_peered":"2024-05-09T04:47:45.388797+0000","last_unstale":"2024-05-09T04:49:29.632285+0000","last_undegraded":"2024-05-09T04:49:29.632285+0000","last_fullsized":"2024-05-09T04:49:29.632285+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:19.265278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.782218+0000","last_change":"2024-05-09T04:47:48.419100+0000","last_active":"2024-05-09T04:49:29.782218+0000","last_peered":"2024-05-09T04:49:29.782218+0000","last_clean":"2024-05-09T04:49:29.782218+0000","last_became_active":"2024-05-09T04:47:48.418499+0000","last_became_peered":"2024-05-09T04:47:48.418499+0000","last_unstale":"2024-05-09T04:49:29.782218+0000","last_undegraded":"2024-05-09T04:49:29.782218+0000","last_fullsized":"2024-05-09T04:49:29.782218+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:23.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.398765+0000","last_change":"2024-05-09T04:47:45.613269+0000","last_active":"2024-05-09T04:49:29.398765+0000","last_peered":"2024-05-09T04:49:29.398765+0000","last_clean":"2024-05-09T04:49:29.398765+0000","last_became_active":"2024-05-09T04:47:45.612895+0000","last_became_peered":"2024-05-09T04:47:45.612895+0000","last_unstale":"2024-05-09T04:49:29.398765+0000","last_undegraded":"2024-05-09T04:49:29.398765+0000","last_fullsized":"2024-05-09T04:49:29.398765+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:52.966974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399055+0000","last_change":"2024-05-09T04:47:48.416658+0000","last_active":"2024-05-09T04:49:29.399055+0000","last_peered":"2024-05-09T04:49:29.399055+0000","last_clean":"2024-05-09T04:49:29.399055+0000","last_became_active":"2024-05-09T04:47:48.416238+0000","last_became_peered":"2024-05-09T04:47:48.416238+0000","last_unstale":"2024-05-09T04:49:29.399055+0000","last_undegraded":"2024-05-09T04:49:29.399055+0000","last_fullsized":"2024-05-09T04:49:29.399055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:35:50.126723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.782067+0000","last_change":"2024-05-09T04:47:45.384018+0000","last_active":"2024-05-09T04:49:29.782067+0000","last_peered":"2024-05-09T04:49:29.782067+0000","last_clean":"2024-05-09T04:49:29.782067+0000","last_became_active":"2024-05-09T04:47:45.383647+0000","last_became_peered":"2024-05-09T04:47:45.383647+0000","last_unstale":"2024-05-09T04:49:29.782067+0000","last_undegraded":"2024-05-09T04:49:29.782067+0000","last_fullsized":"2024-05-09T04:49:29.782067+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:06.431233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.396884+0000","last_change":"2024-05-09T04:47:48.409089+0000","last_active":"2024-05-09T04:49:29.396884+0000","last_peered":"2024-05-09T04:49:29.396884+0000","last_clean":"2024-05-09T04:49:29.396884+0000","last_became_active":"2024-05-09T04:47:48.408548+0000","last_became_peered":"2024-05-09T04:47:48.408548+0000","last_unstale":"2024-05-09T04:49:29.396884+0000","last_undegraded":"2024-05-09T04:49:29.396884+0000","last_fullsized":"2024-05-09T04:49:29.396884+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:13.633008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780638+0000","last_change":"2024-05-09T04:47:45.387580+0000","last_active":"2024-05-09T04:49:29.780638+0000","last_peered":"2024-05-09T04:49:29.780638+0000","last_clean":"2024-05-09T04:49:29.780638+0000","last_became_active":"2024-05-09T04:47:45.386929+0000","last_became_peered":"2024-05-09T04:47:45.386929+0000","last_unstale":"2024-05-09T04:49:29.780638+0000","last_undegraded":"2024-05-09T04:49:29.780638+0000","last_fullsized":"2024-05-09T04:49:29.780638+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:19.661763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781686+0000","last_change":"2024-05-09T04:47:48.416311+0000","last_active":"2024-05-09T04:49:29.781686+0000","last_peered":"2024-05-09T04:49:29.781686+0000","last_clean":"2024-05-09T04:49:29.781686+0000","last_became_active":"2024-05-09T04:47:48.415969+0000","last_became_peered":"2024-05-09T04:47:48.415969+0000","last_unstale":"2024-05-09T04:49:29.781686+0000","last_undegraded":"2024-05-09T04:49:29.781686+0000","last_fullsized":"2024-05-09T04:49:29.781686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:55.751210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399299+0000","last_change":"2024-05-09T04:48:28.573916+0000","last_active":"2024-05-09T04:49:29.399299+0000","last_peered":"2024-05-09T04:49:29.399299+0000","last_clean":"2024-05-09T04:49:29.399299+0000","last_became_active":"2024-05-09T04:48:28.562872+0000","last_became_peered":"2024-05-09T04:48:28.562872+0000","last_unstale":"2024-05-09T04:49:29.399299+0000","last_undegraded":"2024-05-09T04:49:29.399299+0000","last_fullsized":"2024-05-09T04:49:29.399299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:02.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780680+0000","last_change":"2024-05-09T04:47:48.408364+0000","last_active":"2024-05-09T04:49:29.780680+0000","last_peered":"2024-05-09T04:49:29.780680+0000","last_clean":"2024-05-09T04:49:29.780680+0000","last_became_active":"2024-05-09T04:47:48.407982+0000","last_became_peered":"2024-05-09T04:47:48.407982+0000","last_unstale":"2024-05-09T04:49:29.780680+0000","last_undegraded":"2024-05-09T04:49:29.780680+0000","last_fullsized":"2024-05-09T04:49:29.780680+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:14.103171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780402+0000","last_change":"2024-05-09T04:47:45.614637+0000","last_active":"2024-05-09T04:49:29.780402+0000","last_peered":"2024-05-09T04:49:29.780402+0000","last_clean":"2024-05-09T04:49:29.780402+0000","last_became_active":"2024-05-09T04:47:45.613290+0000","last_became_peered":"2024-05-09T04:47:45.613290+0000","last_unstale":"2024-05-09T04:49:29.780402+0000","last_undegraded":"2024-05-09T04:49:29.780402+0000","last_fullsized":"2024-05-09T04:49:29.780402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:48.547382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781075+0000","last_change":"2024-05-09T04:47:48.405537+0000","last_active":"2024-05-09T04:49:29.781075+0000","last_peered":"2024-05-09T04:49:29.781075+0000","last_clean":"2024-05-09T04:49:29.781075+0000","last_became_active":"2024-05-09T04:47:48.405148+0000","last_became_peered":"2024-05-09T04:47:48.405148+0000","last_unstale":"2024-05-09T04:49:29.781075+0000","last_undegraded":"2024-05-09T04:49:29.781075+0000","last_fullsized":"2024-05-09T04:49:29.781075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:46.981401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394998+0000","last_change":"2024-05-09T04:47:45.389151+0000","last_active":"2024-05-09T04:49:29.394998+0000","last_peered":"2024-05-09T04:49:29.394998+0000","last_clean":"2024-05-09T04:49:29.394998+0000","last_became_active":"2024-05-09T04:47:45.388735+0000","last_became_peered":"2024-05-09T04:47:45.388735+0000","last_unstale":"2024-05-09T04:49:29.394998+0000","last_undegraded":"2024-05-09T04:49:29.394998+0000","last_fullsized":"2024-05-09T04:49:29.394998+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:26.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780893+0000","last_change":"2024-05-09T04:47:48.413881+0000","last_active":"2024-05-09T04:49:29.780893+0000","last_peered":"2024-05-09T04:49:29.780893+0000","last_clean":"2024-05-09T04:49:29.780893+0000","last_became_active":"2024-05-09T04:47:48.413559+0000","last_became_peered":"2024-05-09T04:47:48.413559+0000","last_unstale":"2024-05-09T04:49:29.780893+0000","last_undegraded":"2024-05-09T04:49:29.780893+0000","last_fullsized":"2024-05-09T04:49:29.780893+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:40:40.305319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394608+0000","last_change":"2024-05-09T04:47:45.383406+0000","last_active":"2024-05-09T04:49:29.394608+0000","last_peered":"2024-05-09T04:49:29.394608+0000","last_clean":"2024-05-09T04:49:29.394608+0000","last_became_active":"2024-05-09T04:47:45.383025+0000","last_became_peered":"2024-05-09T04:47:45.383025+0000","last_unstale":"2024-05-09T04:49:29.394608+0000","last_undegraded":"2024-05-09T04:49:29.394608+0000","last_fullsized":"2024-05-09T04:49:29.394608+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:46.007454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781143+0000","last_change":"2024-05-09T04:47:48.412481+0000","last_active":"2024-05-09T04:49:29.781143+0000","last_peered":"2024-05-09T04:49:29.781143+0000","last_clean":"2024-05-09T04:49:29.781143+0000","last_became_active":"2024-05-09T04:47:48.412034+0000","last_became_peered":"2024-05-09T04:47:48.412034+0000","last_unstale":"2024-05-09T04:49:29.781143+0000","last_undegraded":"2024-05-09T04:49:29.781143+0000","last_fullsized":"2024-05-09T04:49:29.781143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:14.030981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.396935+0000","last_change":"2024-05-09T04:47:48.406958+0000","last_active":"2024-05-09T04:49:29.396935+0000","last_peered":"2024-05-09T04:49:29.396935+0000","last_clean":"2024-05-09T04:49:29.396935+0000","last_became_active":"2024-05-09T04:47:48.406575+0000","last_became_peered":"2024-05-09T04:47:48.406575+0000","last_unstale":"2024-05-09T04:49:29.396935+0000","last_undegraded":"2024-05-09T04:49:29.396935+0000","last_fullsized":"2024-05-09T04:49:29.396935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:23.389676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780617+0000","last_change":"2024-05-09T04:48:28.562376+0000","last_active":"2024-05-09T04:49:29.780617+0000","last_peered":"2024-05-09T04:49:29.780617+0000","last_clean":"2024-05-09T04:49:29.780617+0000","last_became_active":"2024-05-09T04:48:28.561866+0000","last_became_peered":"2024-05-09T04:48:28.561866+0000","last_unstale":"2024-05-09T04:49:29.780617+0000","last_undegraded":"2024-05-09T04:49:29.780617+0000","last_fullsized":"2024-05-09T04:49:29.780617+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:51.974146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780780+0000","last_change":"2024-05-09T04:47:48.413908+0000","last_active":"2024-05-09T04:49:29.780780+0000","last_peered":"2024-05-09T04:49:29.780780+0000","last_clean":"2024-05-09T04:49:29.780780+0000","last_became_active":"2024-05-09T04:47:48.413575+0000","last_became_peered":"2024-05-09T04:47:48.413575+0000","last_unstale":"2024-05-09T04:49:29.780780+0000","last_undegraded":"2024-05-09T04:49:29.780780+0000","last_fullsized":"2024-05-09T04:49:29.780780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:40.742510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781821+0000","last_change":"2024-05-09T04:47:48.406607+0000","last_active":"2024-05-09T04:49:29.781821+0000","last_peered":"2024-05-09T04:49:29.781821+0000","last_clean":"2024-05-09T04:49:29.781821+0000","last_became_active":"2024-05-09T04:47:48.406290+0000","last_became_peered":"2024-05-09T04:47:48.406290+0000","last_unstale":"2024-05-09T04:49:29.781821+0000","last_undegraded":"2024-05-09T04:49:29.781821+0000","last_fullsized":"2024-05-09T04:49:29.781821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:22.107953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394552+0000","last_change":"2024-05-09T04:47:48.415153+0000","last_active":"2024-05-09T04:49:29.394552+0000","last_peered":"2024-05-09T04:49:29.394552+0000","last_clean":"2024-05-09T04:49:29.394552+0000","last_became_active":"2024-05-09T04:47:48.414793+0000","last_became_peered":"2024-05-09T04:47:48.414793+0000","last_unstale":"2024-05-09T04:49:29.394552+0000","last_undegraded":"2024-05-09T04:49:29.394552+0000","last_fullsized":"2024-05-09T04:49:29.394552+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:57.564091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.397195+0000","last_change":"2024-05-09T04:47:48.408646+0000","last_active":"2024-05-09T04:49:29.397195+0000","last_peered":"2024-05-09T04:49:29.397195+0000","last_clean":"2024-05-09T04:49:29.397195+0000","last_became_active":"2024-05-09T04:47:48.408250+0000","last_became_peered":"2024-05-09T04:47:48.408250+0000","last_unstale":"2024-05-09T04:49:29.397195+0000","last_undegraded":"2024-05-09T04:49:29.397195+0000","last_fullsized":"2024-05-09T04:49:29.397195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:42:50.622937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781033+0000","last_change":"2024-05-09T04:47:48.408686+0000","last_active":"2024-05-09T04:49:29.781033+0000","last_peered":"2024-05-09T04:49:29.781033+0000","last_clean":"2024-05-09T04:49:29.781033+0000","last_became_active":"2024-05-09T04:47:48.408318+0000","last_became_peered":"2024-05-09T04:47:48.408318+0000","last_unstale":"2024-05-09T04:49:29.781033+0000","last_undegraded":"2024-05-09T04:49:29.781033+0000","last_fullsized":"2024-05-09T04:49:29.781033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:51.240485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.782028+0000","last_change":"2024-05-09T04:47:48.421562+0000","last_active":"2024-05-09T04:49:29.782028+0000","last_peered":"2024-05-09T04:49:29.782028+0000","last_clean":"2024-05-09T04:49:29.782028+0000","last_became_active":"2024-05-09T04:47:48.421068+0000","last_became_peered":"2024-05-09T04:47:48.421068+0000","last_unstale":"2024-05-09T04:49:29.782028+0000","last_undegraded":"2024-05-09T04:49:29.782028+0000","last_fullsized":"2024-05-09T04:49:29.782028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:18:41.455187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.395161+0000","last_change":"2024-05-09T04:47:48.413942+0000","last_active":"2024-05-09T04:49:29.395161+0000","last_peered":"2024-05-09T04:49:29.395161+0000","last_clean":"2024-05-09T04:49:29.395161+0000","last_became_active":"2024-05-09T04:47:48.413561+0000","last_became_peered":"2024-05-09T04:47:48.413561+0000","last_unstale":"2024-05-09T04:49:29.395161+0000","last_undegraded":"2024-05-09T04:49:29.395161+0000","last_fullsized":"2024-05-09T04:49:29.395161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:33:48.148834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781292+0000","last_change":"2024-05-09T04:47:48.408752+0000","last_active":"2024-05-09T04:49:29.781292+0000","last_peered":"2024-05-09T04:49:29.781292+0000","last_clean":"2024-05-09T04:49:29.781292+0000","last_became_active":"2024-05-09T04:47:48.408393+0000","last_became_peered":"2024-05-09T04:47:48.408393+0000","last_unstale":"2024-05-09T04:49:29.781292+0000","last_undegraded":"2024-05-09T04:49:29.781292+0000","last_fullsized":"2024-05-09T04:49:29.781292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:06.778002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.398972+0000","last_change":"2024-05-09T04:47:48.409303+0000","last_active":"2024-05-09T04:49:29.398972+0000","last_peered":"2024-05-09T04:49:29.398972+0000","last_clean":"2024-05-09T04:49:29.398972+0000","last_became_active":"2024-05-09T04:47:48.408971+0000","last_became_peered":"2024-05-09T04:47:48.408971+0000","last_unstale":"2024-05-09T04:49:29.398972+0000","last_undegraded":"2024-05-09T04:49:29.398972+0000","last_fullsized":"2024-05-09T04:49:29.398972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:45.830647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394427+0000","last_change":"2024-05-09T04:47:48.404961+0000","last_active":"2024-05-09T04:49:29.394427+0000","last_peered":"2024-05-09T04:49:29.394427+0000","last_clean":"2024-05-09T04:49:29.394427+0000","last_became_active":"2024-05-09T04:47:48.404577+0000","last_became_peered":"2024-05-09T04:47:48.404577+0000","last_unstale":"2024-05-09T04:49:29.394427+0000","last_undegraded":"2024-05-09T04:49:29.394427+0000","last_fullsized":"2024-05-09T04:49:29.394427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:40.528066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.397745+0000","last_change":"2024-05-09T04:47:48.424364+0000","last_active":"2024-05-09T04:49:29.397745+0000","last_peered":"2024-05-09T04:49:29.397745+0000","last_clean":"2024-05-09T04:49:29.397745+0000","last_became_active":"2024-05-09T04:47:48.423991+0000","last_became_peered":"2024-05-09T04:47:48.423991+0000","last_unstale":"2024-05-09T04:49:29.397745+0000","last_undegraded":"2024-05-09T04:49:29.397745+0000","last_fullsized":"2024-05-09T04:49:29.397745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:14.040560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249829,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64348,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":623,"num_read_kb":370,"num_write":164,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":50695,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":193044,"data_compressed":8483,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645352,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":905216,"data_stored":449026,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841580,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":1101245,"data_compressed":47727,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005106,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":449262,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201336,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":449182,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397565,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":1041112,"data_compressed":47727,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561090,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":448651,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":508791,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738464,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":449922,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":449794,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117074,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":925696,"data_stored":448962,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920844,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":509826,"data_compressed":20057,"data_compressed_allocated":28672,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":983040,"data_stored":1040405,"data_compressed":47727,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34532,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34532,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":61382,"data_compressed":8483,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:49:51.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-09T04:49:51.629 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:51 smithi072 ceph-mon[34088]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:49:51.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:51 smithi110 ceph-mon[34904]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:49:51.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:51 smithi046 ceph-mon[30136]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:49:51.880 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:52.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:52 smithi046 ceph-mon[30136]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:49:52.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:52 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:52.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:52 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:52.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:52 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:52.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:52 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:52.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:52 smithi110 ceph-mon[34904]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:49:52.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:52 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:52.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:52 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:52.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:52 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:52.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:52 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:52.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:52 smithi072 ceph-mon[34088]: pgmap v356: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:49:52.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:52 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:52.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:52 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:52.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:52 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:52.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:52 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:53.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.596+0000 7f1cdab35700 1 -- 172.21.15.46:0/1170255013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cd4108760 msgr2=0x7f1cd41100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:53.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.596+0000 7f1cdab35700 1 --2- 172.21.15.46:0/1170255013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cd4108760 0x7f1cd41100c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1ccc00b0d0 tx=0x7f1ccc00b3e0 comp rx=0 tx=0).stop 2024-05-09T04:49:53.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.596+0000 7f1cdab35700 1 -- 172.21.15.46:0/1170255013 shutdown_connections 2024-05-09T04:49:53.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.596+0000 7f1cdab35700 1 --2- 172.21.15.46:0/1170255013 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cd4108760 0x7f1cd41100c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:53.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.596+0000 7f1cdab35700 1 --2- 172.21.15.46:0/1170255013 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cd41034e0 0x7f1cd41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:53.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.596+0000 7f1cdab35700 1 --2- 172.21.15.46:0/1170255013 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cd4107e10 0x7f1cd4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:53.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.596+0000 7f1cdab35700 1 -- 172.21.15.46:0/1170255013 >> 172.21.15.46:0/1170255013 conn(0x7f1cd40fef30 msgr2=0x7f1cd4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:53.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.596+0000 7f1cdab35700 1 -- 172.21.15.46:0/1170255013 shutdown_connections 2024-05-09T04:49:53.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.596+0000 7f1cdab35700 1 -- 172.21.15.46:0/1170255013 wait complete. 2024-05-09T04:49:53.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.597+0000 7f1cdab35700 1 Processor -- start 2024-05-09T04:49:53.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.597+0000 7f1cdab35700 1 -- start start 2024-05-09T04:49:53.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.597+0000 7f1cdab35700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cd41034e0 0x7f1cd419d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:53.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.597+0000 7f1cdab35700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cd4107e10 0x7f1cd419d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:53.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.597+0000 7f1cdab35700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cd4108760 0x7f1cd41a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:53.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.597+0000 7f1cdab35700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd4113400 con 0x7f1cd4107e10 2024-05-09T04:49:53.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.597+0000 7f1cdab35700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd4113280 con 0x7f1cd4108760 2024-05-09T04:49:53.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.597+0000 7f1cdab35700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1cd4113580 con 0x7f1cd41034e0 2024-05-09T04:49:53.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.597+0000 7f1ccbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cd4107e10 0x7f1cd419d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:53.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.597+0000 7f1ccbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cd4107e10 0x7f1cd419d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:49774/0 (socket says 172.21.15.46:49774) 2024-05-09T04:49:53.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.597+0000 7f1ccbfff700 1 -- 172.21.15.46:0/2990612530 learned_addr learned my addr 172.21.15.46:0/2990612530 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:53.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.597+0000 7f1cd88d1700 1 --2- 172.21.15.46:0/2990612530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cd41034e0 0x7f1cd419d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:53.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.597+0000 7f1cd90d2700 1 --2- 172.21.15.46:0/2990612530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cd4108760 0x7f1cd41a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:53.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.598+0000 7f1ccbfff700 1 -- 172.21.15.46:0/2990612530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cd41034e0 msgr2=0x7f1cd419d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:53.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.598+0000 7f1ccbfff700 1 --2- 172.21.15.46:0/2990612530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cd41034e0 0x7f1cd419d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:53.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.598+0000 7f1ccbfff700 1 -- 172.21.15.46:0/2990612530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cd4108760 msgr2=0x7f1cd41a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:53.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.598+0000 7f1ccbfff700 1 --2- 172.21.15.46:0/2990612530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cd4108760 0x7f1cd41a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:53.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.598+0000 7f1ccbfff700 1 -- 172.21.15.46:0/2990612530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cd41a2050 con 0x7f1cd4107e10 2024-05-09T04:49:53.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.598+0000 7f1ccbfff700 1 --2- 172.21.15.46:0/2990612530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cd4107e10 0x7f1cd419d720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400b500 tx=0x7f1cc400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:53.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.598+0000 7f1cc9ffb700 1 -- 172.21.15.46:0/2990612530 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4015020 con 0x7f1cd4107e10 2024-05-09T04:49:53.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.598+0000 7f1cdab35700 1 -- 172.21.15.46:0/2990612530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cd41a2340 con 0x7f1cd4107e10 2024-05-09T04:49:53.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.598+0000 7f1cc9ffb700 1 -- 172.21.15.46:0/2990612530 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cc4011ce0 con 0x7f1cd4107e10 2024-05-09T04:49:53.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.598+0000 7f1cc9ffb700 1 -- 172.21.15.46:0/2990612530 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cc4010cb0 con 0x7f1cd4107e10 2024-05-09T04:49:53.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.598+0000 7f1cdab35700 1 -- 172.21.15.46:0/2990612530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cd41a9c60 con 0x7f1cd4107e10 2024-05-09T04:49:53.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.599+0000 7f1cdab35700 1 -- 172.21.15.46:0/2990612530 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1cd40512c0 con 0x7f1cd4107e10 2024-05-09T04:49:53.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.601+0000 7f1cc9ffb700 1 -- 172.21.15.46:0/2990612530 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1cc4018020 con 0x7f1cd4107e10 2024-05-09T04:49:53.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.601+0000 7f1cc9ffb700 1 --2- 172.21.15.46:0/2990612530 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1cbc0b2910 0x7f1cbc0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:53.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.602+0000 7f1cd88d1700 1 --2- 172.21.15.46:0/2990612530 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1cbc0b2910 0x7f1cbc0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:53.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.602+0000 7f1cc9ffb700 1 -- 172.21.15.46:0/2990612530 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f1cc40e1f20 con 0x7f1cd4107e10 2024-05-09T04:49:53.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.602+0000 7f1cd88d1700 1 --2- 172.21.15.46:0/2990612530 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1cbc0b2910 0x7f1cbc0b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0005950 tx=0x7f1cd000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:53.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.605+0000 7f1cc9ffb700 1 -- 172.21.15.46:0/2990612530 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1cc401f020 con 0x7f1cd4107e10 2024-05-09T04:49:53.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.859+0000 7f1cdab35700 1 -- 172.21.15.46:0/2990612530 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1cd419e4e0 con 0x7f1cbc0b2910 2024-05-09T04:49:53.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.868+0000 7f1cc9ffb700 1 -- 172.21.15.46:0/2990612530 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258996 (secure 0 0 0) 0x7f1cd419e4e0 con 0x7f1cbc0b2910 2024-05-09T04:49:53.871 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:49:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.871+0000 7f1cdab35700 1 -- 172.21.15.46:0/2990612530 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1cbc0b2910 msgr2=0x7f1cbc0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.871+0000 7f1cdab35700 1 --2- 172.21.15.46:0/2990612530 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1cbc0b2910 0x7f1cbc0b4e20 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1cd0005950 tx=0x7f1cd000a300 comp rx=0 tx=0).stop 2024-05-09T04:49:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.871+0000 7f1cdab35700 1 -- 172.21.15.46:0/2990612530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cd4107e10 msgr2=0x7f1cd419d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.871+0000 7f1cdab35700 1 --2- 172.21.15.46:0/2990612530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cd4107e10 0x7f1cd419d720 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400b500 tx=0x7f1cc400b810 comp rx=0 tx=0).stop 2024-05-09T04:49:53.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.871+0000 7f1cdab35700 1 -- 172.21.15.46:0/2990612530 shutdown_connections 2024-05-09T04:49:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.871+0000 7f1cdab35700 1 --2- 172.21.15.46:0/2990612530 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f1cbc0b2910 0x7f1cbc0b4e20 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.871+0000 7f1cdab35700 1 --2- 172.21.15.46:0/2990612530 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1cd4108760 0x7f1cd41a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.871+0000 7f1cdab35700 1 --2- 172.21.15.46:0/2990612530 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1cd41034e0 0x7f1cd419d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.871+0000 7f1cdab35700 1 --2- 172.21.15.46:0/2990612530 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1cd4107e10 0x7f1cd419d720 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.871+0000 7f1cdab35700 1 -- 172.21.15.46:0/2990612530 >> 172.21.15.46:0/2990612530 conn(0x7f1cd40fef30 msgr2=0x7f1cd410ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.871+0000 7f1cdab35700 1 -- 172.21.15.46:0/2990612530 shutdown_connections 2024-05-09T04:49:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:53.871+0000 7f1cdab35700 1 -- 172.21.15.46:0/2990612530 wait complete. 2024-05-09T04:49:53.874 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T04:49:54.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:54 smithi046 ceph-mon[30136]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:49:54.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:54 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:54.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:54 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:54.612 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2024-05-09T04:49:53.330947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654950,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":714,"num_read_kb":444,"num_write":289,"num_write_kb":2263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":3002535,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":233,"ondisk_log_size":233,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":364488,"kb_used_data":11100,"kb_used_omap":18,"kb_used_meta":353197,"kb_avail":1124577336,"statfs":{"total":1151940427776,"available":1151567192064,"internally_reserved":0,"allocated":11366400,"data_stored":7346178,"data_compressed":255830,"data_compressed_allocated":344064,"data_compressed_original":2211840,"omap_allocated":19142,"internal_metadata":361674042},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003382"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.397417+0000","last_change":"2024-05-09T04:47:48.408878+0000","last_active":"2024-05-09T04:49:29.397417+0000","last_peered":"2024-05-09T04:49:29.397417+0000","last_clean":"2024-05-09T04:49:29.397417+0000","last_became_active":"2024-05-09T04:47:48.408479+0000","last_became_peered":"2024-05-09T04:47:48.408479+0000","last_unstale":"2024-05-09T04:49:29.397417+0000","last_undegraded":"2024-05-09T04:49:29.397417+0000","last_fullsized":"2024-05-09T04:49:29.397417+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:02.494768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.782165+0000","last_change":"2024-05-09T04:47:48.415650+0000","last_active":"2024-05-09T04:49:29.782165+0000","last_peered":"2024-05-09T04:49:29.782165+0000","last_clean":"2024-05-09T04:49:29.782165+0000","last_became_active":"2024-05-09T04:47:48.415322+0000","last_became_peered":"2024-05-09T04:47:48.415322+0000","last_unstale":"2024-05-09T04:49:29.782165+0000","last_undegraded":"2024-05-09T04:49:29.782165+0000","last_fullsized":"2024-05-09T04:49:29.782165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.142513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781957+0000","last_change":"2024-05-09T04:47:48.417706+0000","last_active":"2024-05-09T04:49:29.781957+0000","last_peered":"2024-05-09T04:49:29.781957+0000","last_clean":"2024-05-09T04:49:29.781957+0000","last_became_active":"2024-05-09T04:47:48.412068+0000","last_became_peered":"2024-05-09T04:47:48.412068+0000","last_unstale":"2024-05-09T04:49:29.781957+0000","last_undegraded":"2024-05-09T04:49:29.781957+0000","last_fullsized":"2024-05-09T04:49:29.781957+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:44.602567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.632393+0000","last_change":"2024-05-09T04:47:48.415751+0000","last_active":"2024-05-09T04:49:29.632393+0000","last_peered":"2024-05-09T04:49:29.632393+0000","last_clean":"2024-05-09T04:49:29.632393+0000","last_became_active":"2024-05-09T04:47:48.414312+0000","last_became_peered":"2024-05-09T04:47:48.414312+0000","last_unstale":"2024-05-09T04:49:29.632393+0000","last_undegraded":"2024-05-09T04:49:29.632393+0000","last_fullsized":"2024-05-09T04:49:29.632393+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:12.202761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399222+0000","last_change":"2024-05-09T04:47:48.415919+0000","last_active":"2024-05-09T04:49:29.399222+0000","last_peered":"2024-05-09T04:49:29.399222+0000","last_clean":"2024-05-09T04:49:29.399222+0000","last_became_active":"2024-05-09T04:47:48.415533+0000","last_became_peered":"2024-05-09T04:47:48.415533+0000","last_unstale":"2024-05-09T04:49:29.399222+0000","last_undegraded":"2024-05-09T04:49:29.399222+0000","last_fullsized":"2024-05-09T04:49:29.399222+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:32.262416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394968+0000","last_change":"2024-05-09T04:47:48.419705+0000","last_active":"2024-05-09T04:49:29.394968+0000","last_peered":"2024-05-09T04:49:29.394968+0000","last_clean":"2024-05-09T04:49:29.394968+0000","last_became_active":"2024-05-09T04:47:48.419339+0000","last_became_peered":"2024-05-09T04:47:48.419339+0000","last_unstale":"2024-05-09T04:49:29.394968+0000","last_undegraded":"2024-05-09T04:49:29.394968+0000","last_fullsized":"2024-05-09T04:49:29.394968+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:50:23.754071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780603+0000","last_change":"2024-05-09T04:47:48.404106+0000","last_active":"2024-05-09T04:49:29.780603+0000","last_peered":"2024-05-09T04:49:29.780603+0000","last_clean":"2024-05-09T04:49:29.780603+0000","last_became_active":"2024-05-09T04:47:48.403724+0000","last_became_peered":"2024-05-09T04:47:48.403724+0000","last_unstale":"2024-05-09T04:49:29.780603+0000","last_undegraded":"2024-05-09T04:49:29.780603+0000","last_fullsized":"2024-05-09T04:49:29.780603+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:41.134752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780621+0000","last_change":"2024-05-09T04:48:28.562515+0000","last_active":"2024-05-09T04:49:29.780621+0000","last_peered":"2024-05-09T04:49:29.780621+0000","last_clean":"2024-05-09T04:49:29.780621+0000","last_became_active":"2024-05-09T04:48:28.562104+0000","last_became_peered":"2024-05-09T04:48:28.562104+0000","last_unstale":"2024-05-09T04:49:29.780621+0000","last_undegraded":"2024-05-09T04:49:29.780621+0000","last_fullsized":"2024-05-09T04:49:29.780621+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:33.342877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399309+0000","last_change":"2024-05-09T04:47:48.400467+0000","last_active":"2024-05-09T04:49:29.399309+0000","last_peered":"2024-05-09T04:49:29.399309+0000","last_clean":"2024-05-09T04:49:29.399309+0000","last_became_active":"2024-05-09T04:47:48.399960+0000","last_became_peered":"2024-05-09T04:47:48.399960+0000","last_unstale":"2024-05-09T04:49:29.399309+0000","last_undegraded":"2024-05-09T04:49:29.399309+0000","last_fullsized":"2024-05-09T04:49:29.399309+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:23:44.473053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.395358+0000","last_change":"2024-05-09T04:47:48.418970+0000","last_active":"2024-05-09T04:49:29.395358+0000","last_peered":"2024-05-09T04:49:29.395358+0000","last_clean":"2024-05-09T04:49:29.395358+0000","last_became_active":"2024-05-09T04:47:48.418574+0000","last_became_peered":"2024-05-09T04:47:48.418574+0000","last_unstale":"2024-05-09T04:49:29.395358+0000","last_undegraded":"2024-05-09T04:49:29.395358+0000","last_fullsized":"2024-05-09T04:49:29.395358+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:48.075893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780929+0000","last_change":"2024-05-09T04:47:48.409879+0000","last_active":"2024-05-09T04:49:29.780929+0000","last_peered":"2024-05-09T04:49:29.780929+0000","last_clean":"2024-05-09T04:49:29.780929+0000","last_became_active":"2024-05-09T04:47:48.409515+0000","last_became_peered":"2024-05-09T04:47:48.409515+0000","last_unstale":"2024-05-09T04:49:29.780929+0000","last_undegraded":"2024-05-09T04:49:29.780929+0000","last_fullsized":"2024-05-09T04:49:29.780929+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:15.546214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.632290+0000","last_change":"2024-05-09T04:47:48.417556+0000","last_active":"2024-05-09T04:49:29.632290+0000","last_peered":"2024-05-09T04:49:29.632290+0000","last_clean":"2024-05-09T04:49:29.632290+0000","last_became_active":"2024-05-09T04:47:48.416787+0000","last_became_peered":"2024-05-09T04:47:48.416787+0000","last_unstale":"2024-05-09T04:49:29.632290+0000","last_undegraded":"2024-05-09T04:49:29.632290+0000","last_fullsized":"2024-05-09T04:49:29.632290+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:20.360232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394343+0000","last_change":"2024-05-09T04:47:48.415145+0000","last_active":"2024-05-09T04:49:29.394343+0000","last_peered":"2024-05-09T04:49:29.394343+0000","last_clean":"2024-05-09T04:49:29.394343+0000","last_became_active":"2024-05-09T04:47:48.414830+0000","last_became_peered":"2024-05-09T04:47:48.414830+0000","last_unstale":"2024-05-09T04:49:29.394343+0000","last_undegraded":"2024-05-09T04:49:29.394343+0000","last_fullsized":"2024-05-09T04:49:29.394343+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:55.936409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781684+0000","last_change":"2024-05-09T04:47:48.413977+0000","last_active":"2024-05-09T04:49:29.781684+0000","last_peered":"2024-05-09T04:49:29.781684+0000","last_clean":"2024-05-09T04:49:29.781684+0000","last_became_active":"2024-05-09T04:47:48.413604+0000","last_became_peered":"2024-05-09T04:47:48.413604+0000","last_unstale":"2024-05-09T04:49:29.781684+0000","last_undegraded":"2024-05-09T04:49:29.781684+0000","last_fullsized":"2024-05-09T04:49:29.781684+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:25.462958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781580+0000","last_change":"2024-05-09T04:47:48.419000+0000","last_active":"2024-05-09T04:49:29.781580+0000","last_peered":"2024-05-09T04:49:29.781580+0000","last_clean":"2024-05-09T04:49:29.781580+0000","last_became_active":"2024-05-09T04:47:48.418457+0000","last_became_peered":"2024-05-09T04:47:48.418457+0000","last_unstale":"2024-05-09T04:49:29.781580+0000","last_undegraded":"2024-05-09T04:49:29.781580+0000","last_fullsized":"2024-05-09T04:49:29.781580+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:36.325460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.398747+0000","last_change":"2024-05-09T04:47:48.406754+0000","last_active":"2024-05-09T04:49:29.398747+0000","last_peered":"2024-05-09T04:49:29.398747+0000","last_clean":"2024-05-09T04:49:29.398747+0000","last_became_active":"2024-05-09T04:47:48.406396+0000","last_became_peered":"2024-05-09T04:47:48.406396+0000","last_unstale":"2024-05-09T04:49:29.398747+0000","last_undegraded":"2024-05-09T04:49:29.398747+0000","last_fullsized":"2024-05-09T04:49:29.398747+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:46.421061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.397568+0000","last_change":"2024-05-09T04:47:48.409556+0000","last_active":"2024-05-09T04:49:29.397568+0000","last_peered":"2024-05-09T04:49:29.397568+0000","last_clean":"2024-05-09T04:49:29.397568+0000","last_became_active":"2024-05-09T04:47:48.409145+0000","last_became_peered":"2024-05-09T04:47:48.409145+0000","last_unstale":"2024-05-09T04:49:29.397568+0000","last_undegraded":"2024-05-09T04:49:29.397568+0000","last_fullsized":"2024-05-09T04:49:29.397568+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:25.486257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781325+0000","last_change":"2024-05-09T04:47:48.415546+0000","last_active":"2024-05-09T04:49:29.781325+0000","last_peered":"2024-05-09T04:49:29.781325+0000","last_clean":"2024-05-09T04:49:29.781325+0000","last_became_active":"2024-05-09T04:47:48.415221+0000","last_became_peered":"2024-05-09T04:47:48.415221+0000","last_unstale":"2024-05-09T04:49:29.781325+0000","last_undegraded":"2024-05-09T04:49:29.781325+0000","last_fullsized":"2024-05-09T04:49:29.781325+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:57:29.112409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781020+0000","last_change":"2024-05-09T04:47:48.413106+0000","last_active":"2024-05-09T04:49:29.781020+0000","last_peered":"2024-05-09T04:49:29.781020+0000","last_clean":"2024-05-09T04:49:29.781020+0000","last_became_active":"2024-05-09T04:47:48.412700+0000","last_became_peered":"2024-05-09T04:47:48.412700+0000","last_unstale":"2024-05-09T04:49:29.781020+0000","last_undegraded":"2024-05-09T04:49:29.781020+0000","last_fullsized":"2024-05-09T04:49:29.781020+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:37:31.682008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":39,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781180+0000","last_change":"2024-05-09T04:47:45.386931+0000","last_active":"2024-05-09T04:49:29.781180+0000","last_peered":"2024-05-09T04:49:29.781180+0000","last_clean":"2024-05-09T04:49:29.781180+0000","last_became_active":"2024-05-09T04:47:45.386562+0000","last_became_peered":"2024-05-09T04:47:45.386562+0000","last_unstale":"2024-05-09T04:49:29.781180+0000","last_undegraded":"2024-05-09T04:49:29.781180+0000","last_fullsized":"2024-05-09T04:49:29.781180+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:59.335687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781714+0000","last_change":"2024-05-09T04:47:48.411064+0000","last_active":"2024-05-09T04:49:29.781714+0000","last_peered":"2024-05-09T04:49:29.781714+0000","last_clean":"2024-05-09T04:49:29.781714+0000","last_became_active":"2024-05-09T04:47:48.410689+0000","last_became_peered":"2024-05-09T04:47:48.410689+0000","last_unstale":"2024-05-09T04:49:29.781714+0000","last_undegraded":"2024-05-09T04:49:29.781714+0000","last_fullsized":"2024-05-09T04:49:29.781714+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:21.178831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781574+0000","last_change":"2024-05-09T04:47:45.612837+0000","last_active":"2024-05-09T04:49:29.781574+0000","last_peered":"2024-05-09T04:49:29.781574+0000","last_clean":"2024-05-09T04:49:29.781574+0000","last_became_active":"2024-05-09T04:47:45.612307+0000","last_became_peered":"2024-05-09T04:47:45.612307+0000","last_unstale":"2024-05-09T04:49:29.781574+0000","last_undegraded":"2024-05-09T04:49:29.781574+0000","last_fullsized":"2024-05-09T04:49:29.781574+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:59.686391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.782117+0000","last_change":"2024-05-09T04:47:48.412781+0000","last_active":"2024-05-09T04:49:29.782117+0000","last_peered":"2024-05-09T04:49:29.782117+0000","last_clean":"2024-05-09T04:49:29.782117+0000","last_became_active":"2024-05-09T04:47:48.411396+0000","last_became_peered":"2024-05-09T04:47:48.411396+0000","last_unstale":"2024-05-09T04:49:29.782117+0000","last_undegraded":"2024-05-09T04:49:29.782117+0000","last_fullsized":"2024-05-09T04:49:29.782117+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:41:22.775432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":21,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.397190+0000","last_change":"2024-05-09T04:48:28.568024+0000","last_active":"2024-05-09T04:49:29.397190+0000","last_peered":"2024-05-09T04:49:29.397190+0000","last_clean":"2024-05-09T04:49:29.397190+0000","last_became_active":"2024-05-09T04:48:28.559441+0000","last_became_peered":"2024-05-09T04:48:28.559441+0000","last_unstale":"2024-05-09T04:49:29.397190+0000","last_undegraded":"2024-05-09T04:49:29.397190+0000","last_fullsized":"2024-05-09T04:49:29.397190+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:56.257589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.398866+0000","last_change":"2024-05-09T04:47:48.421361+0000","last_active":"2024-05-09T04:49:29.398866+0000","last_peered":"2024-05-09T04:49:29.398866+0000","last_clean":"2024-05-09T04:49:29.398866+0000","last_became_active":"2024-05-09T04:47:48.421018+0000","last_became_peered":"2024-05-09T04:47:48.421018+0000","last_unstale":"2024-05-09T04:49:29.398866+0000","last_undegraded":"2024-05-09T04:49:29.398866+0000","last_fullsized":"2024-05-09T04:49:29.398866+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:00.634983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.403356+0000","last_change":"2024-05-09T04:49:29.402935+0000","last_active":"2024-05-09T04:49:29.403356+0000","last_peered":"2024-05-09T04:49:29.403356+0000","last_clean":"2024-05-09T04:49:29.403356+0000","last_became_active":"2024-05-09T04:49:29.400084+0000","last_became_peered":"2024-05-09T04:49:29.400084+0000","last_unstale":"2024-05-09T04:49:29.403356+0000","last_undegraded":"2024-05-09T04:49:29.403356+0000","last_fullsized":"2024-05-09T04:49:29.403356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:06:06.683122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":28,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394894+0000","last_change":"2024-05-09T04:48:28.564243+0000","last_active":"2024-05-09T04:49:29.394894+0000","last_peered":"2024-05-09T04:49:29.394894+0000","last_clean":"2024-05-09T04:49:29.394894+0000","last_became_active":"2024-05-09T04:48:28.563732+0000","last_became_peered":"2024-05-09T04:48:28.563732+0000","last_unstale":"2024-05-09T04:49:29.394894+0000","last_undegraded":"2024-05-09T04:49:29.394894+0000","last_fullsized":"2024-05-09T04:49:29.394894+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:26.982339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"80'5","reported_seq":137,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:47.655992+0000","last_change":"2024-05-09T04:47:45.389868+0000","last_active":"2024-05-09T04:49:47.655992+0000","last_peered":"2024-05-09T04:49:47.655992+0000","last_clean":"2024-05-09T04:49:47.655992+0000","last_became_active":"2024-05-09T04:47:45.389508+0000","last_became_peered":"2024-05-09T04:47:45.389508+0000","last_unstale":"2024-05-09T04:49:47.655992+0000","last_undegraded":"2024-05-09T04:49:47.655992+0000","last_fullsized":"2024-05-09T04:49:47.655992+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:45.050187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":622,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399110+0000","last_change":"2024-05-09T04:47:48.404457+0000","last_active":"2024-05-09T04:49:29.399110+0000","last_peered":"2024-05-09T04:49:29.399110+0000","last_clean":"2024-05-09T04:49:29.399110+0000","last_became_active":"2024-05-09T04:47:48.403968+0000","last_became_peered":"2024-05-09T04:47:48.403968+0000","last_unstale":"2024-05-09T04:49:29.399110+0000","last_undegraded":"2024-05-09T04:49:29.399110+0000","last_fullsized":"2024-05-09T04:49:29.399110+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:23:40.364916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394856+0000","last_change":"2024-05-09T04:47:45.388183+0000","last_active":"2024-05-09T04:49:29.394856+0000","last_peered":"2024-05-09T04:49:29.394856+0000","last_clean":"2024-05-09T04:49:29.394856+0000","last_became_active":"2024-05-09T04:47:45.386603+0000","last_became_peered":"2024-05-09T04:47:45.386603+0000","last_unstale":"2024-05-09T04:49:29.394856+0000","last_undegraded":"2024-05-09T04:49:29.394856+0000","last_fullsized":"2024-05-09T04:49:29.394856+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:53.433619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780754+0000","last_change":"2024-05-09T04:48:28.663120+0000","last_active":"2024-05-09T04:49:29.780754+0000","last_peered":"2024-05-09T04:49:29.780754+0000","last_clean":"2024-05-09T04:49:29.780754+0000","last_became_active":"2024-05-09T04:48:28.662748+0000","last_became_peered":"2024-05-09T04:48:28.662748+0000","last_unstale":"2024-05-09T04:49:29.780754+0000","last_undegraded":"2024-05-09T04:49:29.780754+0000","last_fullsized":"2024-05-09T04:49:29.780754+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:57.677876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394206+0000","last_change":"2024-05-09T04:47:45.384192+0000","last_active":"2024-05-09T04:49:29.394206+0000","last_peered":"2024-05-09T04:49:29.394206+0000","last_clean":"2024-05-09T04:49:29.394206+0000","last_became_active":"2024-05-09T04:47:45.383843+0000","last_became_peered":"2024-05-09T04:47:45.383843+0000","last_unstale":"2024-05-09T04:49:29.394206+0000","last_undegraded":"2024-05-09T04:49:29.394206+0000","last_fullsized":"2024-05-09T04:49:29.394206+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:28.815207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780961+0000","last_change":"2024-05-09T04:47:48.409203+0000","last_active":"2024-05-09T04:49:29.780961+0000","last_peered":"2024-05-09T04:49:29.780961+0000","last_clean":"2024-05-09T04:49:29.780961+0000","last_became_active":"2024-05-09T04:47:48.408861+0000","last_became_peered":"2024-05-09T04:47:48.408861+0000","last_unstale":"2024-05-09T04:49:29.780961+0000","last_undegraded":"2024-05-09T04:49:29.780961+0000","last_fullsized":"2024-05-09T04:49:29.780961+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:07.160397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394642+0000","last_change":"2024-05-09T04:47:45.392723+0000","last_active":"2024-05-09T04:49:29.394642+0000","last_peered":"2024-05-09T04:49:29.394642+0000","last_clean":"2024-05-09T04:49:29.394642+0000","last_became_active":"2024-05-09T04:47:45.392313+0000","last_became_peered":"2024-05-09T04:47:45.392313+0000","last_unstale":"2024-05-09T04:49:29.394642+0000","last_undegraded":"2024-05-09T04:49:29.394642+0000","last_fullsized":"2024-05-09T04:49:29.394642+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:44.358702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780790+0000","last_change":"2024-05-09T04:47:48.414000+0000","last_active":"2024-05-09T04:49:29.780790+0000","last_peered":"2024-05-09T04:49:29.780790+0000","last_clean":"2024-05-09T04:49:29.780790+0000","last_became_active":"2024-05-09T04:47:48.413532+0000","last_became_peered":"2024-05-09T04:47:48.413532+0000","last_unstale":"2024-05-09T04:49:29.780790+0000","last_undegraded":"2024-05-09T04:49:29.780790+0000","last_fullsized":"2024-05-09T04:49:29.780790+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:22.774814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"80'3","reported_seq":133,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:47.655215+0000","last_change":"2024-05-09T04:47:45.388661+0000","last_active":"2024-05-09T04:49:47.655215+0000","last_peered":"2024-05-09T04:49:47.655215+0000","last_clean":"2024-05-09T04:49:47.655215+0000","last_became_active":"2024-05-09T04:47:45.386010+0000","last_became_peered":"2024-05-09T04:47:45.386010+0000","last_unstale":"2024-05-09T04:49:47.655215+0000","last_undegraded":"2024-05-09T04:49:47.655215+0000","last_fullsized":"2024-05-09T04:49:47.655215+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:23.250463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399287+0000","last_change":"2024-05-09T04:47:48.421706+0000","last_active":"2024-05-09T04:49:29.399287+0000","last_peered":"2024-05-09T04:49:29.399287+0000","last_clean":"2024-05-09T04:49:29.399287+0000","last_became_active":"2024-05-09T04:47:48.420529+0000","last_became_peered":"2024-05-09T04:47:48.420529+0000","last_unstale":"2024-05-09T04:49:29.399287+0000","last_undegraded":"2024-05-09T04:49:29.399287+0000","last_fullsized":"2024-05-09T04:49:29.399287+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:10.110366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781039+0000","last_change":"2024-05-09T04:47:45.386625+0000","last_active":"2024-05-09T04:49:29.781039+0000","last_peered":"2024-05-09T04:49:29.781039+0000","last_clean":"2024-05-09T04:49:29.781039+0000","last_became_active":"2024-05-09T04:47:45.385087+0000","last_became_peered":"2024-05-09T04:47:45.385087+0000","last_unstale":"2024-05-09T04:49:29.781039+0000","last_undegraded":"2024-05-09T04:49:29.781039+0000","last_fullsized":"2024-05-09T04:49:29.781039+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:15.546894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.782090+0000","last_change":"2024-05-09T04:48:28.562517+0000","last_active":"2024-05-09T04:49:29.782090+0000","last_peered":"2024-05-09T04:49:29.782090+0000","last_clean":"2024-05-09T04:49:29.782090+0000","last_became_active":"2024-05-09T04:48:28.562047+0000","last_became_peered":"2024-05-09T04:48:28.562047+0000","last_unstale":"2024-05-09T04:49:29.782090+0000","last_undegraded":"2024-05-09T04:49:29.782090+0000","last_fullsized":"2024-05-09T04:49:29.782090+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:22.769844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.395036+0000","last_change":"2024-05-09T04:47:45.613713+0000","last_active":"2024-05-09T04:49:29.395036+0000","last_peered":"2024-05-09T04:49:29.395036+0000","last_clean":"2024-05-09T04:49:29.395036+0000","last_became_active":"2024-05-09T04:47:45.613313+0000","last_became_peered":"2024-05-09T04:47:45.613313+0000","last_unstale":"2024-05-09T04:49:29.395036+0000","last_undegraded":"2024-05-09T04:49:29.395036+0000","last_fullsized":"2024-05-09T04:49:29.395036+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:32:44.825695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399234+0000","last_change":"2024-05-09T04:47:48.411234+0000","last_active":"2024-05-09T04:49:29.399234+0000","last_peered":"2024-05-09T04:49:29.399234+0000","last_clean":"2024-05-09T04:49:29.399234+0000","last_became_active":"2024-05-09T04:47:48.410881+0000","last_became_peered":"2024-05-09T04:47:48.410881+0000","last_unstale":"2024-05-09T04:49:29.399234+0000","last_undegraded":"2024-05-09T04:49:29.399234+0000","last_fullsized":"2024-05-09T04:49:29.399234+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.747871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"82'29","reported_seq":287,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:48.681241+0000","last_change":"2024-05-09T04:48:28.570426+0000","last_active":"2024-05-09T04:49:48.681241+0000","last_peered":"2024-05-09T04:49:48.681241+0000","last_clean":"2024-05-09T04:49:48.681241+0000","last_became_active":"2024-05-09T04:48:28.561786+0000","last_became_peered":"2024-05-09T04:48:28.561786+0000","last_unstale":"2024-05-09T04:49:48.681241+0000","last_undegraded":"2024-05-09T04:49:48.681241+0000","last_fullsized":"2024-05-09T04:49:48.681241+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:32.367142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60347,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":206,"num_read_kb":153,"num_write":30,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":50695,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399373+0000","last_change":"2024-05-09T04:47:48.406510+0000","last_active":"2024-05-09T04:49:29.399373+0000","last_peered":"2024-05-09T04:49:29.399373+0000","last_clean":"2024-05-09T04:49:29.399373+0000","last_became_active":"2024-05-09T04:47:48.406166+0000","last_became_peered":"2024-05-09T04:47:48.406166+0000","last_unstale":"2024-05-09T04:49:29.399373+0000","last_undegraded":"2024-05-09T04:49:29.399373+0000","last_fullsized":"2024-05-09T04:49:29.399373+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:18.579620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780774+0000","last_change":"2024-05-09T04:47:45.387164+0000","last_active":"2024-05-09T04:49:29.780774+0000","last_peered":"2024-05-09T04:49:29.780774+0000","last_clean":"2024-05-09T04:49:29.780774+0000","last_became_active":"2024-05-09T04:47:45.386755+0000","last_became_peered":"2024-05-09T04:47:45.386755+0000","last_unstale":"2024-05-09T04:49:29.780774+0000","last_undegraded":"2024-05-09T04:49:29.780774+0000","last_fullsized":"2024-05-09T04:49:29.780774+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:07.838782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394112+0000","last_change":"2024-05-09T04:47:48.405436+0000","last_active":"2024-05-09T04:49:29.394112+0000","last_peered":"2024-05-09T04:49:29.394112+0000","last_clean":"2024-05-09T04:49:29.394112+0000","last_became_active":"2024-05-09T04:47:48.405032+0000","last_became_peered":"2024-05-09T04:47:48.405032+0000","last_unstale":"2024-05-09T04:49:29.394112+0000","last_undegraded":"2024-05-09T04:49:29.394112+0000","last_fullsized":"2024-05-09T04:49:29.394112+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:27.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394859+0000","last_change":"2024-05-09T04:47:45.391970+0000","last_active":"2024-05-09T04:49:29.394859+0000","last_peered":"2024-05-09T04:49:29.394859+0000","last_clean":"2024-05-09T04:49:29.394859+0000","last_became_active":"2024-05-09T04:47:45.391597+0000","last_became_peered":"2024-05-09T04:47:45.391597+0000","last_unstale":"2024-05-09T04:49:29.394859+0000","last_undegraded":"2024-05-09T04:49:29.394859+0000","last_fullsized":"2024-05-09T04:49:29.394859+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:27.076631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780897+0000","last_change":"2024-05-09T04:47:48.419328+0000","last_active":"2024-05-09T04:49:29.780897+0000","last_peered":"2024-05-09T04:49:29.780897+0000","last_clean":"2024-05-09T04:49:29.780897+0000","last_became_active":"2024-05-09T04:47:48.418972+0000","last_became_peered":"2024-05-09T04:47:48.418972+0000","last_unstale":"2024-05-09T04:49:29.780897+0000","last_undegraded":"2024-05-09T04:49:29.780897+0000","last_fullsized":"2024-05-09T04:49:29.780897+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:15.164747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"80'4","reported_seq":141,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:50.673128+0000","last_change":"2024-05-09T04:47:45.386173+0000","last_active":"2024-05-09T04:49:50.673128+0000","last_peered":"2024-05-09T04:49:50.673128+0000","last_clean":"2024-05-09T04:49:50.673128+0000","last_became_active":"2024-05-09T04:47:45.384296+0000","last_became_peered":"2024-05-09T04:47:45.384296+0000","last_unstale":"2024-05-09T04:49:50.673128+0000","last_undegraded":"2024-05-09T04:49:50.673128+0000","last_fullsized":"2024-05-09T04:49:50.673128+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:03.246187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780422+0000","last_change":"2024-05-09T04:47:48.415645+0000","last_active":"2024-05-09T04:49:29.780422+0000","last_peered":"2024-05-09T04:49:29.780422+0000","last_clean":"2024-05-09T04:49:29.780422+0000","last_became_active":"2024-05-09T04:47:48.415303+0000","last_became_peered":"2024-05-09T04:47:48.415303+0000","last_unstale":"2024-05-09T04:49:29.780422+0000","last_undegraded":"2024-05-09T04:49:29.780422+0000","last_fullsized":"2024-05-09T04:49:29.780422+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:39.703948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":46,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399050+0000","last_change":"2024-05-09T04:47:45.613320+0000","last_active":"2024-05-09T04:49:29.399050+0000","last_peered":"2024-05-09T04:49:29.399050+0000","last_clean":"2024-05-09T04:49:29.399050+0000","last_became_active":"2024-05-09T04:47:45.612871+0000","last_became_peered":"2024-05-09T04:47:45.612871+0000","last_unstale":"2024-05-09T04:49:29.399050+0000","last_undegraded":"2024-05-09T04:49:29.399050+0000","last_fullsized":"2024-05-09T04:49:29.399050+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:31.516652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":15,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780915+0000","last_change":"2024-05-09T04:48:28.563376+0000","last_active":"2024-05-09T04:49:29.780915+0000","last_peered":"2024-05-09T04:49:29.780915+0000","last_clean":"2024-05-09T04:49:29.780915+0000","last_became_active":"2024-05-09T04:48:28.562973+0000","last_became_peered":"2024-05-09T04:48:28.562973+0000","last_unstale":"2024-05-09T04:49:29.780915+0000","last_undegraded":"2024-05-09T04:49:29.780915+0000","last_fullsized":"2024-05-09T04:49:29.780915+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:20.068517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399757+0000","last_change":"2024-05-09T04:47:45.389785+0000","last_active":"2024-05-09T04:49:29.399757+0000","last_peered":"2024-05-09T04:49:29.399757+0000","last_clean":"2024-05-09T04:49:29.399757+0000","last_became_active":"2024-05-09T04:47:45.389159+0000","last_became_peered":"2024-05-09T04:47:45.389159+0000","last_unstale":"2024-05-09T04:49:29.399757+0000","last_undegraded":"2024-05-09T04:49:29.399757+0000","last_fullsized":"2024-05-09T04:49:29.399757+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:28:51.804123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.397253+0000","last_change":"2024-05-09T04:48:28.563170+0000","last_active":"2024-05-09T04:49:29.397253+0000","last_peered":"2024-05-09T04:49:29.397253+0000","last_clean":"2024-05-09T04:49:29.397253+0000","last_became_active":"2024-05-09T04:48:28.562189+0000","last_became_peered":"2024-05-09T04:48:28.562189+0000","last_unstale":"2024-05-09T04:49:29.397253+0000","last_undegraded":"2024-05-09T04:49:29.397253+0000","last_fullsized":"2024-05-09T04:49:29.397253+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:00.643292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780838+0000","last_change":"2024-05-09T04:47:45.613739+0000","last_active":"2024-05-09T04:49:29.780838+0000","last_peered":"2024-05-09T04:49:29.780838+0000","last_clean":"2024-05-09T04:49:29.780838+0000","last_became_active":"2024-05-09T04:47:45.613365+0000","last_became_peered":"2024-05-09T04:47:45.613365+0000","last_unstale":"2024-05-09T04:49:29.780838+0000","last_undegraded":"2024-05-09T04:49:29.780838+0000","last_fullsized":"2024-05-09T04:49:29.780838+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:57.138068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"80'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.395050+0000","last_change":"2024-05-09T04:47:48.413343+0000","last_active":"2024-05-09T04:49:29.395050+0000","last_peered":"2024-05-09T04:49:29.395050+0000","last_clean":"2024-05-09T04:49:29.395050+0000","last_became_active":"2024-05-09T04:47:48.412941+0000","last_became_peered":"2024-05-09T04:47:48.412941+0000","last_unstale":"2024-05-09T04:49:29.395050+0000","last_undegraded":"2024-05-09T04:49:29.395050+0000","last_fullsized":"2024-05-09T04:49:29.395050+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:28.764553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780761+0000","last_change":"2024-05-09T04:47:45.386487+0000","last_active":"2024-05-09T04:49:29.780761+0000","last_peered":"2024-05-09T04:49:29.780761+0000","last_clean":"2024-05-09T04:49:29.780761+0000","last_became_active":"2024-05-09T04:47:45.386078+0000","last_became_peered":"2024-05-09T04:47:45.386078+0000","last_unstale":"2024-05-09T04:49:29.780761+0000","last_undegraded":"2024-05-09T04:49:29.780761+0000","last_fullsized":"2024-05-09T04:49:29.780761+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:19.998977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394993+0000","last_change":"2024-05-09T04:47:48.418955+0000","last_active":"2024-05-09T04:49:29.394993+0000","last_peered":"2024-05-09T04:49:29.394993+0000","last_clean":"2024-05-09T04:49:29.394993+0000","last_became_active":"2024-05-09T04:47:48.418528+0000","last_became_peered":"2024-05-09T04:47:48.418528+0000","last_unstale":"2024-05-09T04:49:29.394993+0000","last_undegraded":"2024-05-09T04:49:29.394993+0000","last_fullsized":"2024-05-09T04:49:29.394993+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:51:02.386342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.395407+0000","last_change":"2024-05-09T04:47:48.413304+0000","last_active":"2024-05-09T04:49:29.395407+0000","last_peered":"2024-05-09T04:49:29.395407+0000","last_clean":"2024-05-09T04:49:29.395407+0000","last_became_active":"2024-05-09T04:47:48.412582+0000","last_became_peered":"2024-05-09T04:47:48.412582+0000","last_unstale":"2024-05-09T04:49:29.395407+0000","last_undegraded":"2024-05-09T04:49:29.395407+0000","last_fullsized":"2024-05-09T04:49:29.395407+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:40.027082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"80'4","reported_seq":137,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:49.670712+0000","last_change":"2024-05-09T04:47:45.612742+0000","last_active":"2024-05-09T04:49:49.670712+0000","last_peered":"2024-05-09T04:49:49.670712+0000","last_clean":"2024-05-09T04:49:49.670712+0000","last_became_active":"2024-05-09T04:47:45.612325+0000","last_became_peered":"2024-05-09T04:47:45.612325+0000","last_unstale":"2024-05-09T04:49:49.670712+0000","last_undegraded":"2024-05-09T04:49:49.670712+0000","last_fullsized":"2024-05-09T04:49:49.670712+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:00.573996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781355+0000","last_change":"2024-05-09T04:47:48.412894+0000","last_active":"2024-05-09T04:49:29.781355+0000","last_peered":"2024-05-09T04:49:29.781355+0000","last_clean":"2024-05-09T04:49:29.781355+0000","last_became_active":"2024-05-09T04:47:48.412265+0000","last_became_peered":"2024-05-09T04:47:48.412265+0000","last_unstale":"2024-05-09T04:49:29.781355+0000","last_undegraded":"2024-05-09T04:49:29.781355+0000","last_fullsized":"2024-05-09T04:49:29.781355+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:23.517566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":36,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.632150+0000","last_change":"2024-05-09T04:47:45.387715+0000","last_active":"2024-05-09T04:49:29.632150+0000","last_peered":"2024-05-09T04:49:29.632150+0000","last_clean":"2024-05-09T04:49:29.632150+0000","last_became_active":"2024-05-09T04:47:45.387229+0000","last_became_peered":"2024-05-09T04:47:45.387229+0000","last_unstale":"2024-05-09T04:49:29.632150+0000","last_undegraded":"2024-05-09T04:49:29.632150+0000","last_fullsized":"2024-05-09T04:49:29.632150+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:22.540214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":104,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780701+0000","last_change":"2024-05-09T04:46:21.632579+0000","last_active":"2024-05-09T04:49:29.780701+0000","last_peered":"2024-05-09T04:49:29.780701+0000","last_clean":"2024-05-09T04:49:29.780701+0000","last_became_active":"2024-05-09T04:46:21.627483+0000","last_became_peered":"2024-05-09T04:46:21.627483+0000","last_unstale":"2024-05-09T04:49:29.780701+0000","last_undegraded":"2024-05-09T04:49:29.780701+0000","last_fullsized":"2024-05-09T04:49:29.780701+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:42:54.179739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:42:54.179739+0000","last_clean_scrub_stamp":"2024-05-09T04:42:54.179739+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:44.331526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394006+0000","last_change":"2024-05-09T04:47:48.418570+0000","last_active":"2024-05-09T04:49:29.394006+0000","last_peered":"2024-05-09T04:49:29.394006+0000","last_clean":"2024-05-09T04:49:29.394006+0000","last_became_active":"2024-05-09T04:47:48.418270+0000","last_became_peered":"2024-05-09T04:47:48.418270+0000","last_unstale":"2024-05-09T04:49:29.394006+0000","last_undegraded":"2024-05-09T04:49:29.394006+0000","last_fullsized":"2024-05-09T04:49:29.394006+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:28.406846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780915+0000","last_change":"2024-05-09T04:47:45.387161+0000","last_active":"2024-05-09T04:49:29.780915+0000","last_peered":"2024-05-09T04:49:29.780915+0000","last_clean":"2024-05-09T04:49:29.780915+0000","last_became_active":"2024-05-09T04:47:45.386753+0000","last_became_peered":"2024-05-09T04:47:45.386753+0000","last_unstale":"2024-05-09T04:49:29.780915+0000","last_undegraded":"2024-05-09T04:49:29.780915+0000","last_fullsized":"2024-05-09T04:49:29.780915+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:15.402418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399631+0000","last_change":"2024-05-09T04:47:45.613551+0000","last_active":"2024-05-09T04:49:29.399631+0000","last_peered":"2024-05-09T04:49:29.399631+0000","last_clean":"2024-05-09T04:49:29.399631+0000","last_became_active":"2024-05-09T04:47:45.613086+0000","last_became_peered":"2024-05-09T04:47:45.613086+0000","last_unstale":"2024-05-09T04:49:29.399631+0000","last_undegraded":"2024-05-09T04:49:29.399631+0000","last_fullsized":"2024-05-09T04:49:29.399631+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:04.350960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394435+0000","last_change":"2024-05-09T04:47:48.403701+0000","last_active":"2024-05-09T04:49:29.394435+0000","last_peered":"2024-05-09T04:49:29.394435+0000","last_clean":"2024-05-09T04:49:29.394435+0000","last_became_active":"2024-05-09T04:47:48.403337+0000","last_became_peered":"2024-05-09T04:47:48.403337+0000","last_unstale":"2024-05-09T04:49:29.394435+0000","last_undegraded":"2024-05-09T04:49:29.394435+0000","last_fullsized":"2024-05-09T04:49:29.394435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:53.032348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"82'11","reported_seq":40,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781201+0000","last_change":"2024-05-09T04:47:45.385935+0000","last_active":"2024-05-09T04:49:29.781201+0000","last_peered":"2024-05-09T04:49:29.781201+0000","last_clean":"2024-05-09T04:49:29.781201+0000","last_became_active":"2024-05-09T04:47:45.385533+0000","last_became_peered":"2024-05-09T04:47:45.385533+0000","last_unstale":"2024-05-09T04:49:29.781201+0000","last_undegraded":"2024-05-09T04:49:29.781201+0000","last_fullsized":"2024-05-09T04:49:29.781201+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:34:38.014201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.782032+0000","last_change":"2024-05-09T04:47:48.409590+0000","last_active":"2024-05-09T04:49:29.782032+0000","last_peered":"2024-05-09T04:49:29.782032+0000","last_clean":"2024-05-09T04:49:29.782032+0000","last_became_active":"2024-05-09T04:47:48.409191+0000","last_became_peered":"2024-05-09T04:47:48.409191+0000","last_unstale":"2024-05-09T04:49:29.782032+0000","last_undegraded":"2024-05-09T04:49:29.782032+0000","last_fullsized":"2024-05-09T04:49:29.782032+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:50.440609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":34,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.632285+0000","last_change":"2024-05-09T04:47:45.389147+0000","last_active":"2024-05-09T04:49:29.632285+0000","last_peered":"2024-05-09T04:49:29.632285+0000","last_clean":"2024-05-09T04:49:29.632285+0000","last_became_active":"2024-05-09T04:47:45.388797+0000","last_became_peered":"2024-05-09T04:47:45.388797+0000","last_unstale":"2024-05-09T04:49:29.632285+0000","last_undegraded":"2024-05-09T04:49:29.632285+0000","last_fullsized":"2024-05-09T04:49:29.632285+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:19.265278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.782218+0000","last_change":"2024-05-09T04:47:48.419100+0000","last_active":"2024-05-09T04:49:29.782218+0000","last_peered":"2024-05-09T04:49:29.782218+0000","last_clean":"2024-05-09T04:49:29.782218+0000","last_became_active":"2024-05-09T04:47:48.418499+0000","last_became_peered":"2024-05-09T04:47:48.418499+0000","last_unstale":"2024-05-09T04:49:29.782218+0000","last_undegraded":"2024-05-09T04:49:29.782218+0000","last_fullsized":"2024-05-09T04:49:29.782218+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:23.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.398765+0000","last_change":"2024-05-09T04:47:45.613269+0000","last_active":"2024-05-09T04:49:29.398765+0000","last_peered":"2024-05-09T04:49:29.398765+0000","last_clean":"2024-05-09T04:49:29.398765+0000","last_became_active":"2024-05-09T04:47:45.612895+0000","last_became_peered":"2024-05-09T04:47:45.612895+0000","last_unstale":"2024-05-09T04:49:29.398765+0000","last_undegraded":"2024-05-09T04:49:29.398765+0000","last_fullsized":"2024-05-09T04:49:29.398765+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:52.966974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399055+0000","last_change":"2024-05-09T04:47:48.416658+0000","last_active":"2024-05-09T04:49:29.399055+0000","last_peered":"2024-05-09T04:49:29.399055+0000","last_clean":"2024-05-09T04:49:29.399055+0000","last_became_active":"2024-05-09T04:47:48.416238+0000","last_became_peered":"2024-05-09T04:47:48.416238+0000","last_unstale":"2024-05-09T04:49:29.399055+0000","last_undegraded":"2024-05-09T04:49:29.399055+0000","last_fullsized":"2024-05-09T04:49:29.399055+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:35:50.126723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.782067+0000","last_change":"2024-05-09T04:47:45.384018+0000","last_active":"2024-05-09T04:49:29.782067+0000","last_peered":"2024-05-09T04:49:29.782067+0000","last_clean":"2024-05-09T04:49:29.782067+0000","last_became_active":"2024-05-09T04:47:45.383647+0000","last_became_peered":"2024-05-09T04:47:45.383647+0000","last_unstale":"2024-05-09T04:49:29.782067+0000","last_undegraded":"2024-05-09T04:49:29.782067+0000","last_fullsized":"2024-05-09T04:49:29.782067+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:06.431233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.396884+0000","last_change":"2024-05-09T04:47:48.409089+0000","last_active":"2024-05-09T04:49:29.396884+0000","last_peered":"2024-05-09T04:49:29.396884+0000","last_clean":"2024-05-09T04:49:29.396884+0000","last_became_active":"2024-05-09T04:47:48.408548+0000","last_became_peered":"2024-05-09T04:47:48.408548+0000","last_unstale":"2024-05-09T04:49:29.396884+0000","last_undegraded":"2024-05-09T04:49:29.396884+0000","last_fullsized":"2024-05-09T04:49:29.396884+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:13.633008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"80'1","reported_seq":30,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780638+0000","last_change":"2024-05-09T04:47:45.387580+0000","last_active":"2024-05-09T04:49:29.780638+0000","last_peered":"2024-05-09T04:49:29.780638+0000","last_clean":"2024-05-09T04:49:29.780638+0000","last_became_active":"2024-05-09T04:47:45.386929+0000","last_became_peered":"2024-05-09T04:47:45.386929+0000","last_unstale":"2024-05-09T04:49:29.780638+0000","last_undegraded":"2024-05-09T04:49:29.780638+0000","last_fullsized":"2024-05-09T04:49:29.780638+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:19.661763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781686+0000","last_change":"2024-05-09T04:47:48.416311+0000","last_active":"2024-05-09T04:49:29.781686+0000","last_peered":"2024-05-09T04:49:29.781686+0000","last_clean":"2024-05-09T04:49:29.781686+0000","last_became_active":"2024-05-09T04:47:48.415969+0000","last_became_peered":"2024-05-09T04:47:48.415969+0000","last_unstale":"2024-05-09T04:49:29.781686+0000","last_undegraded":"2024-05-09T04:49:29.781686+0000","last_fullsized":"2024-05-09T04:49:29.781686+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:55.751210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":47,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.399299+0000","last_change":"2024-05-09T04:48:28.573916+0000","last_active":"2024-05-09T04:49:29.399299+0000","last_peered":"2024-05-09T04:49:29.399299+0000","last_clean":"2024-05-09T04:49:29.399299+0000","last_became_active":"2024-05-09T04:48:28.562872+0000","last_became_peered":"2024-05-09T04:48:28.562872+0000","last_unstale":"2024-05-09T04:49:29.399299+0000","last_undegraded":"2024-05-09T04:49:29.399299+0000","last_fullsized":"2024-05-09T04:49:29.399299+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:02.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780680+0000","last_change":"2024-05-09T04:47:48.408364+0000","last_active":"2024-05-09T04:49:29.780680+0000","last_peered":"2024-05-09T04:49:29.780680+0000","last_clean":"2024-05-09T04:49:29.780680+0000","last_became_active":"2024-05-09T04:47:48.407982+0000","last_became_peered":"2024-05-09T04:47:48.407982+0000","last_unstale":"2024-05-09T04:49:29.780680+0000","last_undegraded":"2024-05-09T04:49:29.780680+0000","last_fullsized":"2024-05-09T04:49:29.780680+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:14.103171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":33,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780402+0000","last_change":"2024-05-09T04:47:45.614637+0000","last_active":"2024-05-09T04:49:29.780402+0000","last_peered":"2024-05-09T04:49:29.780402+0000","last_clean":"2024-05-09T04:49:29.780402+0000","last_became_active":"2024-05-09T04:47:45.613290+0000","last_became_peered":"2024-05-09T04:47:45.613290+0000","last_unstale":"2024-05-09T04:49:29.780402+0000","last_undegraded":"2024-05-09T04:49:29.780402+0000","last_fullsized":"2024-05-09T04:49:29.780402+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:48.547382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781075+0000","last_change":"2024-05-09T04:47:48.405537+0000","last_active":"2024-05-09T04:49:29.781075+0000","last_peered":"2024-05-09T04:49:29.781075+0000","last_clean":"2024-05-09T04:49:29.781075+0000","last_became_active":"2024-05-09T04:47:48.405148+0000","last_became_peered":"2024-05-09T04:47:48.405148+0000","last_unstale":"2024-05-09T04:49:29.781075+0000","last_undegraded":"2024-05-09T04:49:29.781075+0000","last_fullsized":"2024-05-09T04:49:29.781075+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:46.981401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"80'1","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394998+0000","last_change":"2024-05-09T04:47:45.389151+0000","last_active":"2024-05-09T04:49:29.394998+0000","last_peered":"2024-05-09T04:49:29.394998+0000","last_clean":"2024-05-09T04:49:29.394998+0000","last_became_active":"2024-05-09T04:47:45.388735+0000","last_became_peered":"2024-05-09T04:47:45.388735+0000","last_unstale":"2024-05-09T04:49:29.394998+0000","last_undegraded":"2024-05-09T04:49:29.394998+0000","last_fullsized":"2024-05-09T04:49:29.394998+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:26.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"80'1","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780893+0000","last_change":"2024-05-09T04:47:48.413881+0000","last_active":"2024-05-09T04:49:29.780893+0000","last_peered":"2024-05-09T04:49:29.780893+0000","last_clean":"2024-05-09T04:49:29.780893+0000","last_became_active":"2024-05-09T04:47:48.413559+0000","last_became_peered":"2024-05-09T04:47:48.413559+0000","last_unstale":"2024-05-09T04:49:29.780893+0000","last_undegraded":"2024-05-09T04:49:29.780893+0000","last_fullsized":"2024-05-09T04:49:29.780893+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:40:40.305319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394608+0000","last_change":"2024-05-09T04:47:45.383406+0000","last_active":"2024-05-09T04:49:29.394608+0000","last_peered":"2024-05-09T04:49:29.394608+0000","last_clean":"2024-05-09T04:49:29.394608+0000","last_became_active":"2024-05-09T04:47:45.383025+0000","last_became_peered":"2024-05-09T04:47:45.383025+0000","last_unstale":"2024-05-09T04:49:29.394608+0000","last_undegraded":"2024-05-09T04:49:29.394608+0000","last_fullsized":"2024-05-09T04:49:29.394608+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:46.007454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781143+0000","last_change":"2024-05-09T04:47:48.412481+0000","last_active":"2024-05-09T04:49:29.781143+0000","last_peered":"2024-05-09T04:49:29.781143+0000","last_clean":"2024-05-09T04:49:29.781143+0000","last_became_active":"2024-05-09T04:47:48.412034+0000","last_became_peered":"2024-05-09T04:47:48.412034+0000","last_unstale":"2024-05-09T04:49:29.781143+0000","last_undegraded":"2024-05-09T04:49:29.781143+0000","last_fullsized":"2024-05-09T04:49:29.781143+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:14.030981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.396935+0000","last_change":"2024-05-09T04:47:48.406958+0000","last_active":"2024-05-09T04:49:29.396935+0000","last_peered":"2024-05-09T04:49:29.396935+0000","last_clean":"2024-05-09T04:49:29.396935+0000","last_became_active":"2024-05-09T04:47:48.406575+0000","last_became_peered":"2024-05-09T04:47:48.406575+0000","last_unstale":"2024-05-09T04:49:29.396935+0000","last_undegraded":"2024-05-09T04:49:29.396935+0000","last_fullsized":"2024-05-09T04:49:29.396935+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:23.389676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780717+0000","last_change":"2024-05-09T04:48:28.562376+0000","last_active":"2024-05-09T04:49:29.780717+0000","last_peered":"2024-05-09T04:49:29.780717+0000","last_clean":"2024-05-09T04:49:29.780717+0000","last_became_active":"2024-05-09T04:48:28.561866+0000","last_became_peered":"2024-05-09T04:48:28.561866+0000","last_unstale":"2024-05-09T04:49:29.780717+0000","last_undegraded":"2024-05-09T04:49:29.780717+0000","last_fullsized":"2024-05-09T04:49:29.780717+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:51.974146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.780902+0000","last_change":"2024-05-09T04:47:48.413908+0000","last_active":"2024-05-09T04:49:29.780902+0000","last_peered":"2024-05-09T04:49:29.780902+0000","last_clean":"2024-05-09T04:49:29.780902+0000","last_became_active":"2024-05-09T04:47:48.413575+0000","last_became_peered":"2024-05-09T04:47:48.413575+0000","last_unstale":"2024-05-09T04:49:29.780902+0000","last_undegraded":"2024-05-09T04:49:29.780902+0000","last_fullsized":"2024-05-09T04:49:29.780902+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:40.742510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781821+0000","last_change":"2024-05-09T04:47:48.406607+0000","last_active":"2024-05-09T04:49:29.781821+0000","last_peered":"2024-05-09T04:49:29.781821+0000","last_clean":"2024-05-09T04:49:29.781821+0000","last_became_active":"2024-05-09T04:47:48.406290+0000","last_became_peered":"2024-05-09T04:47:48.406290+0000","last_unstale":"2024-05-09T04:49:29.781821+0000","last_undegraded":"2024-05-09T04:49:29.781821+0000","last_fullsized":"2024-05-09T04:49:29.781821+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:22.107953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394552+0000","last_change":"2024-05-09T04:47:48.415153+0000","last_active":"2024-05-09T04:49:29.394552+0000","last_peered":"2024-05-09T04:49:29.394552+0000","last_clean":"2024-05-09T04:49:29.394552+0000","last_became_active":"2024-05-09T04:47:48.414793+0000","last_became_peered":"2024-05-09T04:47:48.414793+0000","last_unstale":"2024-05-09T04:49:29.394552+0000","last_undegraded":"2024-05-09T04:49:29.394552+0000","last_fullsized":"2024-05-09T04:49:29.394552+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:57.564091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.397195+0000","last_change":"2024-05-09T04:47:48.408646+0000","last_active":"2024-05-09T04:49:29.397195+0000","last_peered":"2024-05-09T04:49:29.397195+0000","last_clean":"2024-05-09T04:49:29.397195+0000","last_became_active":"2024-05-09T04:47:48.408250+0000","last_became_peered":"2024-05-09T04:47:48.408250+0000","last_unstale":"2024-05-09T04:49:29.397195+0000","last_undegraded":"2024-05-09T04:49:29.397195+0000","last_fullsized":"2024-05-09T04:49:29.397195+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:42:50.622937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781033+0000","last_change":"2024-05-09T04:47:48.408686+0000","last_active":"2024-05-09T04:49:29.781033+0000","last_peered":"2024-05-09T04:49:29.781033+0000","last_clean":"2024-05-09T04:49:29.781033+0000","last_became_active":"2024-05-09T04:47:48.408318+0000","last_became_peered":"2024-05-09T04:47:48.408318+0000","last_unstale":"2024-05-09T04:49:29.781033+0000","last_undegraded":"2024-05-09T04:49:29.781033+0000","last_fullsized":"2024-05-09T04:49:29.781033+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:51.240485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.782028+0000","last_change":"2024-05-09T04:47:48.421562+0000","last_active":"2024-05-09T04:49:29.782028+0000","last_peered":"2024-05-09T04:49:29.782028+0000","last_clean":"2024-05-09T04:49:29.782028+0000","last_became_active":"2024-05-09T04:47:48.421068+0000","last_became_peered":"2024-05-09T04:47:48.421068+0000","last_unstale":"2024-05-09T04:49:29.782028+0000","last_undegraded":"2024-05-09T04:49:29.782028+0000","last_fullsized":"2024-05-09T04:49:29.782028+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:18:41.455187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.395161+0000","last_change":"2024-05-09T04:47:48.413942+0000","last_active":"2024-05-09T04:49:29.395161+0000","last_peered":"2024-05-09T04:49:29.395161+0000","last_clean":"2024-05-09T04:49:29.395161+0000","last_became_active":"2024-05-09T04:47:48.413561+0000","last_became_peered":"2024-05-09T04:47:48.413561+0000","last_unstale":"2024-05-09T04:49:29.395161+0000","last_undegraded":"2024-05-09T04:49:29.395161+0000","last_fullsized":"2024-05-09T04:49:29.395161+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:33:48.148834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.781292+0000","last_change":"2024-05-09T04:47:48.408752+0000","last_active":"2024-05-09T04:49:29.781292+0000","last_peered":"2024-05-09T04:49:29.781292+0000","last_clean":"2024-05-09T04:49:29.781292+0000","last_became_active":"2024-05-09T04:47:48.408393+0000","last_became_peered":"2024-05-09T04:47:48.408393+0000","last_unstale":"2024-05-09T04:49:29.781292+0000","last_undegraded":"2024-05-09T04:49:29.781292+0000","last_fullsized":"2024-05-09T04:49:29.781292+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:06.778002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.398972+0000","last_change":"2024-05-09T04:47:48.409303+0000","last_active":"2024-05-09T04:49:29.398972+0000","last_peered":"2024-05-09T04:49:29.398972+0000","last_clean":"2024-05-09T04:49:29.398972+0000","last_became_active":"2024-05-09T04:47:48.408971+0000","last_became_peered":"2024-05-09T04:47:48.408971+0000","last_unstale":"2024-05-09T04:49:29.398972+0000","last_undegraded":"2024-05-09T04:49:29.398972+0000","last_fullsized":"2024-05-09T04:49:29.398972+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:45.830647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.394427+0000","last_change":"2024-05-09T04:47:48.404961+0000","last_active":"2024-05-09T04:49:29.394427+0000","last_peered":"2024-05-09T04:49:29.394427+0000","last_clean":"2024-05-09T04:49:29.394427+0000","last_became_active":"2024-05-09T04:47:48.404577+0000","last_became_peered":"2024-05-09T04:47:48.404577+0000","last_unstale":"2024-05-09T04:49:29.394427+0000","last_undegraded":"2024-05-09T04:49:29.394427+0000","last_fullsized":"2024-05-09T04:49:29.394427+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:40.528066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-05-09T04:49:29.397745+0000","last_change":"2024-05-09T04:47:48.424364+0000","last_active":"2024-05-09T04:49:29.397745+0000","last_peered":"2024-05-09T04:49:29.397745+0000","last_clean":"2024-05-09T04:49:29.397745+0000","last_became_active":"2024-05-09T04:47:48.423991+0000","last_became_peered":"2024-05-09T04:47:48.423991+0000","last_unstale":"2024-05-09T04:49:29.397745+0000","last_undegraded":"2024-05-09T04:49:29.397745+0000","last_fullsized":"2024-05-09T04:49:29.397745+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:14.040560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":249829,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64348,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":633,"num_read_kb":375,"num_write":164,"num_write_kb":111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":50695,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":193044,"data_compressed":8483,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645353,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36228,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708924,"statfs":{"total":95995035648,"available":95957938176,"internally_reserved":0,"allocated":905216,"data_stored":449026,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841580,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28052,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717100,"statfs":{"total":95995035648,"available":95966310400,"internally_reserved":0,"allocated":1056768,"data_stored":1101245,"data_compressed":47727,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005106,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":449262,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201337,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27924,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717228,"statfs":{"total":95995035648,"available":95966441472,"internally_reserved":0,"allocated":921600,"data_stored":449182,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397566,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27972,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717180,"statfs":{"total":95995035648,"available":95966392320,"internally_reserved":0,"allocated":970752,"data_stored":1041112,"data_compressed":47727,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561091,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":888,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":909312,"data_stored":448651,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32132,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713020,"statfs":{"total":95995035648,"available":95962132480,"internally_reserved":0,"allocated":970752,"data_stored":508791,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":908,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":929792,"data_stored":449922,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27928,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717224,"statfs":{"total":95995035648,"available":95966437376,"internally_reserved":0,"allocated":925696,"data_stored":449794,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117075,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36244,"kb_used_data":904,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708908,"statfs":{"total":95995035648,"available":95957921792,"internally_reserved":0,"allocated":925696,"data_stored":448962,"data_compressed":11574,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920845,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":946176,"data_stored":509826,"data_compressed":20057,"data_compressed_allocated":28672,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36300,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708852,"statfs":{"total":95995035648,"available":95957864448,"internally_reserved":0,"allocated":983040,"data_stored":1040405,"data_compressed":47727,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34532,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34532,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60347,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":61382,"data_compressed":8483,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2183,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:49:54.616 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-09T04:49:54.617 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-09T04:49:54.617 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-09T04:49:54.617 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-09T04:49:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:54 smithi110 ceph-mon[34904]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:49:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:54 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:54.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:54 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:54.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:54 smithi072 ceph-mon[34088]: pgmap v357: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:49:54.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:54 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:54.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:54 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:54.889 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:55.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:55 smithi046 ceph-mon[30136]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:49:55.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:55 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:55.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:55 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:49:55.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:55 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:55.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:55 smithi110 ceph-mon[34904]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:49:55.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:55 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:55.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:55 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:49:55.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:55 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:55.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:55 smithi072 ceph-mon[34088]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:49:55.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:55 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:49:55.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:55 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:49:55.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:55 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:49:56.615 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:56 smithi046 ceph-mon[30136]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:49:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.624+0000 7fee8aa40700 1 -- 172.21.15.46:0/1665746847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee841034e0 msgr2=0x7fee841038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.624+0000 7fee8aa40700 1 --2- 172.21.15.46:0/1665746847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee841034e0 0x7fee841038f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fee740099c0 tx=0x7fee74009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:56.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.624+0000 7fee8aa40700 1 -- 172.21.15.46:0/1665746847 shutdown_connections 2024-05-09T04:49:56.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.624+0000 7fee8aa40700 1 --2- 172.21.15.46:0/1665746847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee84108760 0x7fee841100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:56.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.624+0000 7fee8aa40700 1 --2- 172.21.15.46:0/1665746847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee841034e0 0x7fee841038f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:56.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.624+0000 7fee8aa40700 1 --2- 172.21.15.46:0/1665746847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee84107e10 0x7fee84108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:56.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.624+0000 7fee8aa40700 1 -- 172.21.15.46:0/1665746847 >> 172.21.15.46:0/1665746847 conn(0x7fee840fef30 msgr2=0x7fee84101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:56.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.624+0000 7fee8aa40700 1 -- 172.21.15.46:0/1665746847 shutdown_connections 2024-05-09T04:49:56.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.624+0000 7fee8aa40700 1 -- 172.21.15.46:0/1665746847 wait complete. 2024-05-09T04:49:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.625+0000 7fee8aa40700 1 Processor -- start 2024-05-09T04:49:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.625+0000 7fee8aa40700 1 -- start start 2024-05-09T04:49:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.625+0000 7fee8aa40700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee841034e0 0x7fee840ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:56.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.625+0000 7fee8aa40700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee84107e10 0x7fee840ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.625+0000 7fee8aa40700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee84108760 0x7fee840ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.625+0000 7fee8aa40700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee84113400 con 0x7fee84107e10 2024-05-09T04:49:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.625+0000 7fee8aa40700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fee84113280 con 0x7fee841034e0 2024-05-09T04:49:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.625+0000 7fee8aa40700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee84113580 con 0x7fee84108760 2024-05-09T04:49:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee84107e10 0x7fee840ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee84107e10 0x7fee840ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33102/0 (socket says 172.21.15.46:33102) 2024-05-09T04:49:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee837fe700 1 -- 172.21.15.46:0/429338957 learned_addr learned my addr 172.21.15.46:0/429338957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee83fff700 1 --2- 172.21.15.46:0/429338957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee841034e0 0x7fee840ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee88fdd700 1 --2- 172.21.15.46:0/429338957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee84108760 0x7fee840ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee837fe700 1 -- 172.21.15.46:0/429338957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee84108760 msgr2=0x7fee840ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee837fe700 1 --2- 172.21.15.46:0/429338957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee84108760 0x7fee840ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee837fe700 1 -- 172.21.15.46:0/429338957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee841034e0 msgr2=0x7fee840ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee837fe700 1 --2- 172.21.15.46:0/429338957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee841034e0 0x7fee840ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:56.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee837fe700 1 -- 172.21.15.46:0/429338957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee841a9260 con 0x7fee84107e10 2024-05-09T04:49:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee837fe700 1 --2- 172.21.15.46:0/429338957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee84107e10 0x7fee840ff950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fee7800b500 tx=0x7fee7800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee817fa700 1 -- 172.21.15.46:0/429338957 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78015020 con 0x7fee84107e10 2024-05-09T04:49:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee8aa40700 1 -- 172.21.15.46:0/429338957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee841a9550 con 0x7fee84107e10 2024-05-09T04:49:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee817fa700 1 -- 172.21.15.46:0/429338957 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee78011ce0 con 0x7fee84107e10 2024-05-09T04:49:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee817fa700 1 -- 172.21.15.46:0/429338957 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78010cb0 con 0x7fee84107e10 2024-05-09T04:49:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.626+0000 7fee8aa40700 1 -- 172.21.15.46:0/429338957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee841a9a90 con 0x7fee84107e10 2024-05-09T04:49:56.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.628+0000 7fee8aa40700 1 -- 172.21.15.46:0/429338957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee840512c0 con 0x7fee84107e10 2024-05-09T04:49:56.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.629+0000 7fee817fa700 1 -- 172.21.15.46:0/429338957 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fee78018020 con 0x7fee84107e10 2024-05-09T04:49:56.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.630+0000 7fee817fa700 1 --2- 172.21.15.46:0/429338957 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fee6c0b2910 0x7fee6c0b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:56.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.630+0000 7fee83fff700 1 --2- 172.21.15.46:0/429338957 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fee6c0b2910 0x7fee6c0b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:56.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.630+0000 7fee817fa700 1 -- 172.21.15.46:0/429338957 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fee780e19e0 con 0x7fee84107e10 2024-05-09T04:49:56.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.630+0000 7fee83fff700 1 --2- 172.21.15.46:0/429338957 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fee6c0b2910 0x7fee6c0b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fee74009990 tx=0x7fee740036e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:56.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.635+0000 7fee817fa700 1 -- 172.21.15.46:0/429338957 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee7801f020 con 0x7fee84107e10 2024-05-09T04:49:56.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:56 smithi110 ceph-mon[34904]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:49:56.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:56 smithi072 ceph-mon[34088]: pgmap v358: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:49:56.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.898+0000 7fee8aa40700 1 -- 172.21.15.46:0/429338957 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7fee84100940 con 0x7fee84107e10 2024-05-09T04:49:56.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.898+0000 7fee817fa700 1 -- 172.21.15.46:0/429338957 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7fee780a9610 con 0x7fee84107e10 2024-05-09T04:49:56.898 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:49:56.898 INFO:teuthology.orchestra.run.smithi046.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-09T04:49:56.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.900+0000 7fee8aa40700 1 -- 172.21.15.46:0/429338957 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fee6c0b2910 msgr2=0x7fee6c0b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:56.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.900+0000 7fee8aa40700 1 --2- 172.21.15.46:0/429338957 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fee6c0b2910 0x7fee6c0b4e20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fee74009990 tx=0x7fee740036e0 comp rx=0 tx=0).stop 2024-05-09T04:49:56.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.900+0000 7fee8aa40700 1 -- 172.21.15.46:0/429338957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee84107e10 msgr2=0x7fee840ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:56.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.900+0000 7fee8aa40700 1 --2- 172.21.15.46:0/429338957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee84107e10 0x7fee840ff950 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fee7800b500 tx=0x7fee7800b810 comp rx=0 tx=0).stop 2024-05-09T04:49:56.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.900+0000 7fee8aa40700 1 -- 172.21.15.46:0/429338957 shutdown_connections 2024-05-09T04:49:56.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.900+0000 7fee8aa40700 1 --2- 172.21.15.46:0/429338957 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fee6c0b2910 0x7fee6c0b4e20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:56.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.900+0000 7fee8aa40700 1 --2- 172.21.15.46:0/429338957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee84108760 0x7fee840ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:56.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.900+0000 7fee8aa40700 1 --2- 172.21.15.46:0/429338957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee841034e0 0x7fee840ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:56.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.900+0000 7fee8aa40700 1 --2- 172.21.15.46:0/429338957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee84107e10 0x7fee840ff950 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:56.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.900+0000 7fee8aa40700 1 -- 172.21.15.46:0/429338957 >> 172.21.15.46:0/429338957 conn(0x7fee840fef30 msgr2=0x7fee841063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:56.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.900+0000 7fee8aa40700 1 -- 172.21.15.46:0/429338957 shutdown_connections 2024-05-09T04:49:56.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:56.900+0000 7fee8aa40700 1 -- 172.21.15.46:0/429338957 wait complete. 2024-05-09T04:49:57.554 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:57 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/429338957' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T04:49:57.580 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-09T04:49:57.580 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:49:57.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:57 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/429338957' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T04:49:57.830 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:49:57.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:57 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/429338957' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-09T04:49:58.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:58 smithi046 ceph-mon[30136]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:49:58.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:49:58 smithi046 ceph-mon[30136]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T04:49:58.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:58 smithi110 ceph-mon[34904]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:49:58.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:49:58 smithi110 ceph-mon[34904]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T04:49:58.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:58 smithi072 ceph-mon[34088]: pgmap v359: 97 pgs: 97 active+clean; 640 KiB data, 356 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:49:58.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:49:58 smithi072 ceph-mon[34088]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-05-09T04:49:59.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.513+0000 7fd16f33a700 1 -- 172.21.15.46:0/427313245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd168109860 msgr2=0x7fd1681111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:59.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.513+0000 7fd16f33a700 1 --2- 172.21.15.46:0/427313245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd168109860 0x7fd1681111c0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd1600099c0 tx=0x7fd160009cd0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.513+0000 7fd16f33a700 1 -- 172.21.15.46:0/427313245 shutdown_connections 2024-05-09T04:49:59.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.513+0000 7fd16f33a700 1 --2- 172.21.15.46:0/427313245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd168109860 0x7fd1681111c0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.513+0000 7fd16f33a700 1 --2- 172.21.15.46:0/427313245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1681045e0 0x7fd1681049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.513+0000 7fd16f33a700 1 --2- 172.21.15.46:0/427313245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd168108f10 0x7fd168109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.513+0000 7fd16f33a700 1 -- 172.21.15.46:0/427313245 >> 172.21.15.46:0/427313245 conn(0x7fd1680ffe70 msgr2=0x7fd1681022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:59.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.513+0000 7fd16f33a700 1 -- 172.21.15.46:0/427313245 shutdown_connections 2024-05-09T04:49:59.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.513+0000 7fd16f33a700 1 -- 172.21.15.46:0/427313245 wait complete. 2024-05-09T04:49:59.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.513+0000 7fd16f33a700 1 Processor -- start 2024-05-09T04:49:59.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.514+0000 7fd16f33a700 1 -- start start 2024-05-09T04:49:59.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.514+0000 7fd16f33a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1681045e0 0x7fd16819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:59.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.514+0000 7fd16f33a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd168108f10 0x7fd16819b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:59.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.514+0000 7fd16f33a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd168109860 0x7fd16819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:59.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.514+0000 7fd16f33a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd168114490 con 0x7fd168108f10 2024-05-09T04:49:59.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.514+0000 7fd16f33a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd168114310 con 0x7fd1681045e0 2024-05-09T04:49:59.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.514+0000 7fd16f33a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd168114610 con 0x7fd168109860 2024-05-09T04:49:59.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.514+0000 7fd16c8d5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd168108f10 0x7fd16819b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:59.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.514+0000 7fd16d8d7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd168109860 0x7fd16819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:59.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.514+0000 7fd16d8d7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd168109860 0x7fd16819f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:57972/0 (socket says 172.21.15.46:57972) 2024-05-09T04:49:59.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.514+0000 7fd16d8d7700 1 -- 172.21.15.46:0/2052494132 learned_addr learned my addr 172.21.15.46:0/2052494132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:49:59.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.514+0000 7fd16d0d6700 1 --2- 172.21.15.46:0/2052494132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1681045e0 0x7fd16819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:59.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.514+0000 7fd16d8d7700 1 -- 172.21.15.46:0/2052494132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1681045e0 msgr2=0x7fd16819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:59.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.515+0000 7fd16d8d7700 1 --2- 172.21.15.46:0/2052494132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1681045e0 0x7fd16819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.515+0000 7fd16d8d7700 1 -- 172.21.15.46:0/2052494132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd168108f10 msgr2=0x7fd16819b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:59.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.515+0000 7fd16d8d7700 1 --2- 172.21.15.46:0/2052494132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd168108f10 0x7fd16819b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.515+0000 7fd16d8d7700 1 -- 172.21.15.46:0/2052494132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd16819fba0 con 0x7fd168109860 2024-05-09T04:49:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.515+0000 7fd16c8d5700 1 --2- 172.21.15.46:0/2052494132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd168108f10 0x7fd16819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:49:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.515+0000 7fd16d8d7700 1 --2- 172.21.15.46:0/2052494132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd168109860 0x7fd16819f5b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd1600059d0 tx=0x7fd1600057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.515+0000 7fd16d0d6700 1 --2- 172.21.15.46:0/2052494132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1681045e0 0x7fd16819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:49:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.515+0000 7fd15e7fc700 1 -- 172.21.15.46:0/2052494132 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160027070 con 0x7fd168109860 2024-05-09T04:49:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.515+0000 7fd15e7fc700 1 -- 172.21.15.46:0/2052494132 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1600040a0 con 0x7fd168109860 2024-05-09T04:49:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.515+0000 7fd15e7fc700 1 -- 172.21.15.46:0/2052494132 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd160022e60 con 0x7fd168109860 2024-05-09T04:49:59.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.515+0000 7fd16f33a700 1 -- 172.21.15.46:0/2052494132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd16819fe30 con 0x7fd168109860 2024-05-09T04:49:59.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.515+0000 7fd16f33a700 1 -- 172.21.15.46:0/2052494132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1681a02c0 con 0x7fd168109860 2024-05-09T04:49:59.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.517+0000 7fd16f33a700 1 -- 172.21.15.46:0/2052494132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1680512c0 con 0x7fd168109860 2024-05-09T04:49:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.518+0000 7fd15e7fc700 1 -- 172.21.15.46:0/2052494132 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd16002a070 con 0x7fd168109860 2024-05-09T04:49:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.519+0000 7fd15e7fc700 1 --2- 172.21.15.46:0/2052494132 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd1540b2900 0x7fd1540b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:49:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.519+0000 7fd16d0d6700 1 --2- 172.21.15.46:0/2052494132 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd1540b2900 0x7fd1540b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:49:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.519+0000 7fd15e7fc700 1 -- 172.21.15.46:0/2052494132 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fd1600e2ed0 con 0x7fd168109860 2024-05-09T04:49:59.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.519+0000 7fd16d0d6700 1 --2- 172.21.15.46:0/2052494132 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd1540b2900 0x7fd1540b4e10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd164007fd0 tx=0x7fd164007830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:49:59.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.524+0000 7fd15e7fc700 1 -- 172.21.15.46:0/2052494132 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd160013070 con 0x7fd168109860 2024-05-09T04:49:59.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.786+0000 7fd16f33a700 1 -- 172.21.15.46:0/2052494132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd16802e5f0 con 0x7fd168109860 2024-05-09T04:49:59.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.787+0000 7fd15e7fc700 1 -- 172.21.15.46:0/2052494132 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11862 (secure 0 0 0) 0x7fd1600aac40 con 0x7fd168109860 2024-05-09T04:49:59.787 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:49:59.787 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:48:11.653145+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24473,24503,24448]},"id":1}]} 2024-05-09T04:49:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.790+0000 7fd16f33a700 1 -- 172.21.15.46:0/2052494132 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd1540b2900 msgr2=0x7fd1540b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.790+0000 7fd16f33a700 1 --2- 172.21.15.46:0/2052494132 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd1540b2900 0x7fd1540b4e10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fd164007fd0 tx=0x7fd164007830 comp rx=0 tx=0).stop 2024-05-09T04:49:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.790+0000 7fd16f33a700 1 -- 172.21.15.46:0/2052494132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd168109860 msgr2=0x7fd16819f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:49:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.790+0000 7fd16f33a700 1 --2- 172.21.15.46:0/2052494132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd168109860 0x7fd16819f5b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd1600059d0 tx=0x7fd1600057c0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.790+0000 7fd16f33a700 1 -- 172.21.15.46:0/2052494132 shutdown_connections 2024-05-09T04:49:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.790+0000 7fd16f33a700 1 --2- 172.21.15.46:0/2052494132 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd1540b2900 0x7fd1540b4e10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.790+0000 7fd16f33a700 1 --2- 172.21.15.46:0/2052494132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd168109860 0x7fd16819f5b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.790+0000 7fd16f33a700 1 --2- 172.21.15.46:0/2052494132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd1681045e0 0x7fd16819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.790+0000 7fd16f33a700 1 --2- 172.21.15.46:0/2052494132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd168108f10 0x7fd16819b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:49:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.790+0000 7fd16f33a700 1 -- 172.21.15.46:0/2052494132 >> 172.21.15.46:0/2052494132 conn(0x7fd1680ffe70 msgr2=0x7fd168107010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:49:59.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.790+0000 7fd16f33a700 1 -- 172.21.15.46:0/2052494132 shutdown_connections 2024-05-09T04:49:59.791 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:49:59.790+0000 7fd16f33a700 1 -- 172.21.15.46:0/2052494132 wait complete. 2024-05-09T04:49:59.791 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-05-09T04:50:00.460 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:50:00.714 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:00.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:00 smithi110 ceph-mon[34904]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-09T04:50:00.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:00 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2052494132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:00.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:00 smithi110 ceph-mon[34904]: overall HEALTH_OK 2024-05-09T04:50:00.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:00 smithi046 ceph-mon[30136]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-09T04:50:00.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:00 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2052494132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:00.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:00 smithi046 ceph-mon[30136]: overall HEALTH_OK 2024-05-09T04:50:00.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:00 smithi072 ceph-mon[34088]: pgmap v360: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 341 B/s wr, 7 op/s 2024-05-09T04:50:00.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:00 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2052494132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:00.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:00 smithi072 ceph-mon[34088]: overall HEALTH_OK 2024-05-09T04:50:02.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.551+0000 7fae0e302700 1 -- 172.21.15.46:0/4274639777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae08107df0 msgr2=0x7fae08108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:02.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.551+0000 7fae0e302700 1 --2- 172.21.15.46:0/4274639777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae08107df0 0x7fae08108200 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fadf00099c0 tx=0x7fadf0009cd0 comp rx=0 tx=0).stop 2024-05-09T04:50:02.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.551+0000 7fae0e302700 1 -- 172.21.15.46:0/4274639777 shutdown_connections 2024-05-09T04:50:02.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.551+0000 7fae0e302700 1 --2- 172.21.15.46:0/4274639777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae08108740 0x7fae081100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:02.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.551+0000 7fae0e302700 1 --2- 172.21.15.46:0/4274639777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae081034c0 0x7fae081038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:02.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.551+0000 7fae0e302700 1 --2- 172.21.15.46:0/4274639777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae08107df0 0x7fae08108200 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:02.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.551+0000 7fae0e302700 1 -- 172.21.15.46:0/4274639777 >> 172.21.15.46:0/4274639777 conn(0x7fae080fedf0 msgr2=0x7fae08101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:02.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.551+0000 7fae0e302700 1 -- 172.21.15.46:0/4274639777 shutdown_connections 2024-05-09T04:50:02.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.551+0000 7fae0e302700 1 -- 172.21.15.46:0/4274639777 wait complete. 2024-05-09T04:50:02.552 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:02 smithi046 ceph-mon[30136]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T04:50:02.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.552+0000 7fae0e302700 1 Processor -- start 2024-05-09T04:50:02.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.552+0000 7fae0e302700 1 -- start start 2024-05-09T04:50:02.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae0e302700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae081034c0 0x7fae080ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:02.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae0e302700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae08107df0 0x7fae080ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:02.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae0e302700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae08108740 0x7fae080ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:02.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae0e302700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fae08113350 con 0x7fae08107df0 2024-05-09T04:50:02.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae0e302700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fae081131d0 con 0x7fae08108740 2024-05-09T04:50:02.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae0e302700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fae081134d0 con 0x7fae081034c0 2024-05-09T04:50:02.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae07fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae081034c0 0x7fae080ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:02.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fadfffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae08107df0 0x7fae080ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:02.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae07fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae081034c0 0x7fae080ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:57996/0 (socket says 172.21.15.46:57996) 2024-05-09T04:50:02.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fadfffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae08107df0 0x7fae080ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:33146/0 (socket says 172.21.15.46:33146) 2024-05-09T04:50:02.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae07fff700 1 -- 172.21.15.46:0/3652813895 learned_addr learned my addr 172.21.15.46:0/3652813895 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:02.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae0c89f700 1 --2- 172.21.15.46:0/3652813895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae08108740 0x7fae080ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:02.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae07fff700 1 -- 172.21.15.46:0/3652813895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae08108740 msgr2=0x7fae080ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:02.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae07fff700 1 --2- 172.21.15.46:0/3652813895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae08108740 0x7fae080ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:02.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae07fff700 1 -- 172.21.15.46:0/3652813895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae08107df0 msgr2=0x7fae080ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:02.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae07fff700 1 --2- 172.21.15.46:0/3652813895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae08107df0 0x7fae080ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:02.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae07fff700 1 -- 172.21.15.46:0/3652813895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae081a9210 con 0x7fae081034c0 2024-05-09T04:50:02.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fadfffff700 1 --2- 172.21.15.46:0/3652813895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae08107df0 0x7fae080ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:02.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.553+0000 7fae0c89f700 1 --2- 172.21.15.46:0/3652813895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae08108740 0x7fae080ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:02.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.554+0000 7fae07fff700 1 --2- 172.21.15.46:0/3652813895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae081034c0 0x7fae080ff2d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fadf800e9a0 tx=0x7fadf800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:02.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.554+0000 7fae05ffb700 1 -- 172.21.15.46:0/3652813895 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf800ca10 con 0x7fae081034c0 2024-05-09T04:50:02.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.554+0000 7fae0e302700 1 -- 172.21.15.46:0/3652813895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae081a9500 con 0x7fae081034c0 2024-05-09T04:50:02.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.554+0000 7fae05ffb700 1 -- 172.21.15.46:0/3652813895 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadf800cbb0 con 0x7fae081034c0 2024-05-09T04:50:02.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.554+0000 7fae05ffb700 1 -- 172.21.15.46:0/3652813895 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadf8018680 con 0x7fae081034c0 2024-05-09T04:50:02.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.554+0000 7fae0e302700 1 -- 172.21.15.46:0/3652813895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae081a9a10 con 0x7fae081034c0 2024-05-09T04:50:02.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.555+0000 7fae0e302700 1 -- 172.21.15.46:0/3652813895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae080512c0 con 0x7fae081034c0 2024-05-09T04:50:02.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.557+0000 7fae05ffb700 1 -- 172.21.15.46:0/3652813895 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fadf804aab0 con 0x7fae081034c0 2024-05-09T04:50:02.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.557+0000 7fae05ffb700 1 --2- 172.21.15.46:0/3652813895 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fadf40b2880 0x7fadf40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:02.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.557+0000 7fadfffff700 1 --2- 172.21.15.46:0/3652813895 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fadf40b2880 0x7fadf40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:02.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.558+0000 7fae05ffb700 1 -- 172.21.15.46:0/3652813895 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fadf80e24c0 con 0x7fae081034c0 2024-05-09T04:50:02.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.558+0000 7fadfffff700 1 --2- 172.21.15.46:0/3652813895 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fadf40b2880 0x7fadf40b4d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fadf0000c00 tx=0x7fadf001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:02.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.561+0000 7fae05ffb700 1 -- 172.21.15.46:0/3652813895 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fadf80aa1a0 con 0x7fae081034c0 2024-05-09T04:50:02.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:02 smithi110 ceph-mon[34904]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T04:50:02.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.827+0000 7fae0e302700 1 -- 172.21.15.46:0/3652813895 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae0802e5f0 con 0x7fae081034c0 2024-05-09T04:50:02.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.828+0000 7fae05ffb700 1 -- 172.21.15.46:0/3652813895 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11862 (secure 0 0 0) 0x7fadf80b2220 con 0x7fae081034c0 2024-05-09T04:50:02.829 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:50:02.829 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:48:11.653145+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24473,24503,24448]},"id":1}]} 2024-05-09T04:50:02.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.830+0000 7fae0e302700 1 -- 172.21.15.46:0/3652813895 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fadf40b2880 msgr2=0x7fadf40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:02.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.830+0000 7fae0e302700 1 --2- 172.21.15.46:0/3652813895 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fadf40b2880 0x7fadf40b4d90 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fadf0000c00 tx=0x7fadf001a040 comp rx=0 tx=0).stop 2024-05-09T04:50:02.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.831+0000 7fae0e302700 1 -- 172.21.15.46:0/3652813895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae081034c0 msgr2=0x7fae080ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:02.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.831+0000 7fae0e302700 1 --2- 172.21.15.46:0/3652813895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae081034c0 0x7fae080ff2d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fadf800e9a0 tx=0x7fadf800ecb0 comp rx=0 tx=0).stop 2024-05-09T04:50:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.831+0000 7fae0e302700 1 -- 172.21.15.46:0/3652813895 shutdown_connections 2024-05-09T04:50:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.831+0000 7fae0e302700 1 --2- 172.21.15.46:0/3652813895 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fadf40b2880 0x7fadf40b4d90 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.831+0000 7fae0e302700 1 --2- 172.21.15.46:0/3652813895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fae08108740 0x7fae080ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.831+0000 7fae0e302700 1 --2- 172.21.15.46:0/3652813895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fae081034c0 0x7fae080ff2d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.831+0000 7fae0e302700 1 --2- 172.21.15.46:0/3652813895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fae08107df0 0x7fae080ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:02.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.831+0000 7fae0e302700 1 -- 172.21.15.46:0/3652813895 >> 172.21.15.46:0/3652813895 conn(0x7fae080fedf0 msgr2=0x7fae08075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.831+0000 7fae0e302700 1 -- 172.21.15.46:0/3652813895 shutdown_connections 2024-05-09T04:50:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:02.831+0000 7fae0e302700 1 -- 172.21.15.46:0/3652813895 wait complete. 2024-05-09T04:50:02.833 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-05-09T04:50:02.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:02 smithi072 ceph-mon[34088]: pgmap v361: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T04:50:03.537 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:50:03.798 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:03.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:03 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3652813895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:03.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:03 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3652813895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:03.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:03 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3652813895' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:04.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:04 smithi110 ceph-mon[34904]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T04:50:04.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:04 smithi046 ceph-mon[30136]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T04:50:04.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:04 smithi072 ceph-mon[34088]: pgmap v362: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2024-05-09T04:50:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.494+0000 7f2d75b5d700 1 -- 172.21.15.46:0/835470521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d70107df0 msgr2=0x7f2d70108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.494+0000 7f2d75b5d700 1 --2- 172.21.15.46:0/835470521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d70107df0 0x7f2d70108200 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2d640099c0 tx=0x7f2d64009cd0 comp rx=0 tx=0).stop 2024-05-09T04:50:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.494+0000 7f2d75b5d700 1 -- 172.21.15.46:0/835470521 shutdown_connections 2024-05-09T04:50:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.495+0000 7f2d75b5d700 1 --2- 172.21.15.46:0/835470521 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d70108740 0x7f2d701100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.495+0000 7f2d75b5d700 1 --2- 172.21.15.46:0/835470521 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d701034c0 0x7f2d701038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.495+0000 7f2d75b5d700 1 --2- 172.21.15.46:0/835470521 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d70107df0 0x7f2d70108200 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.495+0000 7f2d75b5d700 1 -- 172.21.15.46:0/835470521 >> 172.21.15.46:0/835470521 conn(0x7f2d700fedc0 msgr2=0x7f2d701011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.495+0000 7f2d75b5d700 1 -- 172.21.15.46:0/835470521 shutdown_connections 2024-05-09T04:50:05.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.495+0000 7f2d75b5d700 1 -- 172.21.15.46:0/835470521 wait complete. 2024-05-09T04:50:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.495+0000 7f2d75b5d700 1 Processor -- start 2024-05-09T04:50:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d75b5d700 1 -- start start 2024-05-09T04:50:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d75b5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d701034c0 0x7f2d7019bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d75b5d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d70107df0 0x7f2d7019c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d75b5d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d70108740 0x7f2d7019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d75b5d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70113410 con 0x7f2d701034c0 2024-05-09T04:50:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d75b5d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70113290 con 0x7f2d70107df0 2024-05-09T04:50:05.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d75b5d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70113590 con 0x7f2d70108740 2024-05-09T04:50:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d6f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d701034c0 0x7f2d7019bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d6f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d701034c0 0x7f2d7019bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56022/0 (socket says 172.21.15.46:56022) 2024-05-09T04:50:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d6f7fe700 1 -- 172.21.15.46:0/3540350254 learned_addr learned my addr 172.21.15.46:0/3540350254 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d6effd700 1 --2- 172.21.15.46:0/3540350254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d70107df0 0x7f2d7019c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d6ffff700 1 --2- 172.21.15.46:0/3540350254 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d70108740 0x7f2d7019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d6f7fe700 1 -- 172.21.15.46:0/3540350254 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d70108740 msgr2=0x7f2d7019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d6f7fe700 1 --2- 172.21.15.46:0/3540350254 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d70108740 0x7f2d7019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d6f7fe700 1 -- 172.21.15.46:0/3540350254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d70107df0 msgr2=0x7f2d7019c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d6f7fe700 1 --2- 172.21.15.46:0/3540350254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d70107df0 0x7f2d7019c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.496+0000 7f2d6f7fe700 1 -- 172.21.15.46:0/3540350254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d701a7060 con 0x7f2d701034c0 2024-05-09T04:50:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.497+0000 7f2d6f7fe700 1 --2- 172.21.15.46:0/3540350254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d701034c0 0x7f2d7019bd10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000d950 tx=0x7f2d6000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.497+0000 7f2d6cff9700 1 -- 172.21.15.46:0/3540350254 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d60016070 con 0x7f2d701034c0 2024-05-09T04:50:05.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.497+0000 7f2d75b5d700 1 -- 172.21.15.46:0/3540350254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d701a7350 con 0x7f2d701034c0 2024-05-09T04:50:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.497+0000 7f2d6cff9700 1 -- 172.21.15.46:0/3540350254 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d60009df0 con 0x7f2d701034c0 2024-05-09T04:50:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.497+0000 7f2d6cff9700 1 -- 172.21.15.46:0/3540350254 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6000f3c0 con 0x7f2d701034c0 2024-05-09T04:50:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.497+0000 7f2d75b5d700 1 -- 172.21.15.46:0/3540350254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d701a7860 con 0x7f2d701034c0 2024-05-09T04:50:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.498+0000 7f2d75b5d700 1 -- 172.21.15.46:0/3540350254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d70104970 con 0x7f2d701034c0 2024-05-09T04:50:05.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.499+0000 7f2d6cff9700 1 -- 172.21.15.46:0/3540350254 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f2d60019070 con 0x7f2d701034c0 2024-05-09T04:50:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.500+0000 7f2d6cff9700 1 --2- 172.21.15.46:0/3540350254 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2d580b2880 0x7f2d580b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.500+0000 7f2d6effd700 1 --2- 172.21.15.46:0/3540350254 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2d580b2880 0x7f2d580b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.500+0000 7f2d6cff9700 1 -- 172.21.15.46:0/3540350254 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f2d600e1ed0 con 0x7f2d701034c0 2024-05-09T04:50:05.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.501+0000 7f2d6effd700 1 --2- 172.21.15.46:0/3540350254 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2d580b2880 0x7f2d580b4d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2d64006010 tx=0x7f2d64009710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:05.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.505+0000 7f2d6cff9700 1 -- 172.21.15.46:0/3540350254 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d600a9b50 con 0x7f2d701034c0 2024-05-09T04:50:05.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.722+0000 7f2d75b5d700 1 -- 172.21.15.46:0/3540350254 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d7002e5f0 con 0x7f2d701034c0 2024-05-09T04:50:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.723+0000 7f2d6cff9700 1 -- 172.21.15.46:0/3540350254 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22685 (secure 0 0 0) 0x7f2d600a9450 con 0x7f2d701034c0 2024-05-09T04:50:05.724 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:50:05.724 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"fsid":"ecf9ae2c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:25.633876+0000","modified":"2024-05-09T04:49:29.385454+0000","last_up_change":"2024-05-09T04:46:46.418597+0000","last_in_change":"2024-05-09T04:46:25.903582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:42:53.301400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:47:43.413569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:47:47.264915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d79cd3e7-6f82-4d60-96d9-26d8146c962f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6803","nonce":1576436508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6805","nonce":1576436508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6809","nonce":1576436508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6807","nonce":1576436508}]},"public_addr":"172.21.15.46:6803/1576436508","cluster_addr":"172.21.15.46:6805/1576436508","heartbeat_back_addr":"172.21.15.46:6809/1576436508","heartbeat_front_addr":"172.21.15.46:6807/1576436508","state":["exists","up"]},{"osd":1,"uuid":"698eba80-1ebd-4076-b6f9-953d2e7daf76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6801","nonce":379039775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6803","nonce":379039775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6807","nonce":379039775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6805","nonce":379039775}]},"public_addr":"172.21.15.72:6801/379039775","cluster_addr":"172.21.15.72:6803/379039775","heartbeat_back_addr":"172.21.15.72:6807/379039775","heartbeat_front_addr":"172.21.15.72:6805/379039775","state":["exists","up"]},{"osd":2,"uuid":"c549b131-cfa7-4724-926f-83e91a752d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6801","nonce":293447227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6803","nonce":293447227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6807","nonce":293447227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6805","nonce":293447227}]},"public_addr":"172.21.15.110:6801/293447227","cluster_addr":"172.21.15.110:6803/293447227","heartbeat_back_addr":"172.21.15.110:6807/293447227","heartbeat_front_addr":"172.21.15.110:6805/293447227","state":["exists","up"]},{"osd":3,"uuid":"549bb4f7-54d9-4cbc-ad40-7f3e717964f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6811","nonce":1115835653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6813","nonce":1115835653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6817","nonce":1115835653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6815","nonce":1115835653}]},"public_addr":"172.21.15.46:6811/1115835653","cluster_addr":"172.21.15.46:6813/1115835653","heartbeat_back_addr":"172.21.15.46:6817/1115835653","heartbeat_front_addr":"172.21.15.46:6815/1115835653","state":["exists","up"]},{"osd":4,"uuid":"444cd36b-6034-46cd-9a6b-29dce4ba6405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6809","nonce":59186693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6811","nonce":59186693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6815","nonce":59186693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6813","nonce":59186693}]},"public_addr":"172.21.15.72:6809/59186693","cluster_addr":"172.21.15.72:6811/59186693","heartbeat_back_addr":"172.21.15.72:6815/59186693","heartbeat_front_addr":"172.21.15.72:6813/59186693","state":["exists","up"]},{"osd":5,"uuid":"2c2bdd2f-43cf-4208-9ae1-f912807c84e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6809","nonce":1712162502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6811","nonce":1712162502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6815","nonce":1712162502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6813","nonce":1712162502}]},"public_addr":"172.21.15.110:6809/1712162502","cluster_addr":"172.21.15.110:6811/1712162502","heartbeat_back_addr":"172.21.15.110:6815/1712162502","heartbeat_front_addr":"172.21.15.110:6813/1712162502","state":["exists","up"]},{"osd":6,"uuid":"beb0b87f-c3a4-40dd-b224-1843e9301ca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6819","nonce":2429650065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6821","nonce":2429650065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6825","nonce":2429650065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6823","nonce":2429650065}]},"public_addr":"172.21.15.46:6819/2429650065","cluster_addr":"172.21.15.46:6821/2429650065","heartbeat_back_addr":"172.21.15.46:6825/2429650065","heartbeat_front_addr":"172.21.15.46:6823/2429650065","state":["exists","up"]},{"osd":7,"uuid":"4a634675-fad7-4a43-840c-8dec2631dec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6817","nonce":337796672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6819","nonce":337796672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6823","nonce":337796672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6821","nonce":337796672}]},"public_addr":"172.21.15.72:6817/337796672","cluster_addr":"172.21.15.72:6819/337796672","heartbeat_back_addr":"172.21.15.72:6823/337796672","heartbeat_front_addr":"172.21.15.72:6821/337796672","state":["exists","up"]},{"osd":8,"uuid":"01b5f638-90f0-4cf4-8042-20d20a725669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6817","nonce":189513203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6819","nonce":189513203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6823","nonce":189513203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6821","nonce":189513203}]},"public_addr":"172.21.15.110:6817/189513203","cluster_addr":"172.21.15.110:6819/189513203","heartbeat_back_addr":"172.21.15.110:6823/189513203","heartbeat_front_addr":"172.21.15.110:6821/189513203","state":["exists","up"]},{"osd":9,"uuid":"e423de41-50ec-48f4-a4ed-216de33c4fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6827","nonce":1762262822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6829","nonce":1762262822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6833","nonce":1762262822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6831","nonce":1762262822}]},"public_addr":"172.21.15.46:6827/1762262822","cluster_addr":"172.21.15.46:6829/1762262822","heartbeat_back_addr":"172.21.15.46:6833/1762262822","heartbeat_front_addr":"172.21.15.46:6831/1762262822","state":["exists","up"]},{"osd":10,"uuid":"f594d004-9f80-4508-b127-fb693d355f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6825","nonce":38453898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6827","nonce":38453898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6831","nonce":38453898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6829","nonce":38453898}]},"public_addr":"172.21.15.72:6825/38453898","cluster_addr":"172.21.15.72:6827/38453898","heartbeat_back_addr":"172.21.15.72:6831/38453898","heartbeat_front_addr":"172.21.15.72:6829/38453898","state":["exists","up"]},{"osd":11,"uuid":"f771d29f-89e3-4f96-9c29-00e424230c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6825","nonce":4153529632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6827","nonce":4153529632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6831","nonce":4153529632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6829","nonce":4153529632}]},"public_addr":"172.21.15.110:6825/4153529632","cluster_addr":"172.21.15.110:6827/4153529632","heartbeat_back_addr":"172.21.15.110:6831/4153529632","heartbeat_front_addr":"172.21.15.110:6829/4153529632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:02.615801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:26.453477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:50.334627+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:15.357320+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:40.403549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:06.030093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:31.904091+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:57.616372+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:24.045975+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:51.160898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:17.597845+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:44.636629+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1596672353":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1846482551":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/1069571878":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6800/2738158981":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/3625745146":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6801/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/564539954":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1797199863":"2024-05-10T04:40:27.201119+0000","172.21.15.46:6801/2738158981":"2024-05-10T04:39:59.515628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:50:05.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.726+0000 7f2d75b5d700 1 -- 172.21.15.46:0/3540350254 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2d580b2880 msgr2=0x7f2d580b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:05.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.726+0000 7f2d75b5d700 1 --2- 172.21.15.46:0/3540350254 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2d580b2880 0x7f2d580b4d90 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f2d64006010 tx=0x7f2d64009710 comp rx=0 tx=0).stop 2024-05-09T04:50:05.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.726+0000 7f2d75b5d700 1 -- 172.21.15.46:0/3540350254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d701034c0 msgr2=0x7f2d7019bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:05.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.726+0000 7f2d75b5d700 1 --2- 172.21.15.46:0/3540350254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d701034c0 0x7f2d7019bd10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2d6000d950 tx=0x7f2d6000dc60 comp rx=0 tx=0).stop 2024-05-09T04:50:05.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.726+0000 7f2d75b5d700 1 -- 172.21.15.46:0/3540350254 shutdown_connections 2024-05-09T04:50:05.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.726+0000 7f2d75b5d700 1 --2- 172.21.15.46:0/3540350254 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f2d580b2880 0x7f2d580b4d90 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:05.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.726+0000 7f2d75b5d700 1 --2- 172.21.15.46:0/3540350254 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2d70108740 0x7f2d7019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:05.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.726+0000 7f2d75b5d700 1 --2- 172.21.15.46:0/3540350254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2d701034c0 0x7f2d7019bd10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:05.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.726+0000 7f2d75b5d700 1 --2- 172.21.15.46:0/3540350254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2d70107df0 0x7f2d7019c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:05.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.726+0000 7f2d75b5d700 1 -- 172.21.15.46:0/3540350254 >> 172.21.15.46:0/3540350254 conn(0x7f2d700fedc0 msgr2=0x7f2d701096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:05.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.726+0000 7f2d75b5d700 1 -- 172.21.15.46:0/3540350254 shutdown_connections 2024-05-09T04:50:05.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:05.726+0000 7f2d75b5d700 1 -- 172.21.15.46:0/3540350254 wait complete. 2024-05-09T04:50:06.386 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:50:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:06.764 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:06 smithi046 ceph-mon[30136]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T04:50:06.764 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:06 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3540350254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:06.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:06 smithi110 ceph-mon[34904]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T04:50:06.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:06 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3540350254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:06.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:06 smithi072 ceph-mon[34088]: pgmap v363: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T04:50:06.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:06 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3540350254' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.365+0000 7f22e1ff7700 1 -- 172.21.15.46:0/2647864592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc108010 msgr2=0x7f22dc108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.365+0000 7f22e1ff7700 1 --2- 172.21.15.46:0/2647864592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc108010 0x7f22dc108420 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f22c40099c0 tx=0x7f22c4009cd0 comp rx=0 tx=0).stop 2024-05-09T04:50:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.365+0000 7f22e1ff7700 1 -- 172.21.15.46:0/2647864592 shutdown_connections 2024-05-09T04:50:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.365+0000 7f22e1ff7700 1 --2- 172.21.15.46:0/2647864592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22dc108960 0x7f22dc1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.365+0000 7f22e1ff7700 1 --2- 172.21.15.46:0/2647864592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22dc103c00 0x7f22dc104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.365+0000 7f22e1ff7700 1 --2- 172.21.15.46:0/2647864592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc108010 0x7f22dc108420 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.365+0000 7f22e1ff7700 1 -- 172.21.15.46:0/2647864592 >> 172.21.15.46:0/2647864592 conn(0x7f22dc0fe530 msgr2=0x7f22dc100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.365+0000 7f22e1ff7700 1 -- 172.21.15.46:0/2647864592 shutdown_connections 2024-05-09T04:50:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.365+0000 7f22e1ff7700 1 -- 172.21.15.46:0/2647864592 wait complete. 2024-05-09T04:50:08.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.366+0000 7f22e1ff7700 1 Processor -- start 2024-05-09T04:50:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.366+0000 7f22e1ff7700 1 -- start start 2024-05-09T04:50:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.366+0000 7f22e1ff7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22dc103c00 0x7f22dc110aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.366+0000 7f22e1ff7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22dc108010 0x7f22dc110fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.366+0000 7f22e1ff7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc108960 0x7f22dc111520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.366+0000 7f22e1ff7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc1146e0 con 0x7f22dc108960 2024-05-09T04:50:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.366+0000 7f22e1ff7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc114560 con 0x7f22dc103c00 2024-05-09T04:50:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.366+0000 7f22e1ff7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f22dc114860 con 0x7f22dc108010 2024-05-09T04:50:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.366+0000 7f22dbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc108960 0x7f22dc111520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22dbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc108960 0x7f22dc111520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56030/0 (socket says 172.21.15.46:56030) 2024-05-09T04:50:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22dbfff700 1 -- 172.21.15.46:0/3936123573 learned_addr learned my addr 172.21.15.46:0/3936123573 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22db7fe700 1 --2- 172.21.15.46:0/3936123573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22dc103c00 0x7f22dc110aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22daffd700 1 --2- 172.21.15.46:0/3936123573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22dc108010 0x7f22dc110fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22dbfff700 1 -- 172.21.15.46:0/3936123573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22dc108010 msgr2=0x7f22dc110fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:08.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22dbfff700 1 --2- 172.21.15.46:0/3936123573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22dc108010 0x7f22dc110fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22dbfff700 1 -- 172.21.15.46:0/3936123573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22dc103c00 msgr2=0x7f22dc110aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22dbfff700 1 --2- 172.21.15.46:0/3936123573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22dc103c00 0x7f22dc110aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22dbfff700 1 -- 172.21.15.46:0/3936123573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22dc1a92c0 con 0x7f22dc108960 2024-05-09T04:50:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22daffd700 1 --2- 172.21.15.46:0/3936123573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22dc108010 0x7f22dc110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:50:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22dbfff700 1 --2- 172.21.15.46:0/3936123573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc108960 0x7f22dc111520 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f22d4007d70 tx=0x7f22d400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22d8ff9700 1 -- 172.21.15.46:0/3936123573 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d4016020 con 0x7f22dc108960 2024-05-09T04:50:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22e1ff7700 1 -- 172.21.15.46:0/3936123573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22dc1a95b0 con 0x7f22dc108960 2024-05-09T04:50:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22d8ff9700 1 -- 172.21.15.46:0/3936123573 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22d400ad40 con 0x7f22dc108960 2024-05-09T04:50:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22d8ff9700 1 -- 172.21.15.46:0/3936123573 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22d40115e0 con 0x7f22dc108960 2024-05-09T04:50:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.367+0000 7f22e1ff7700 1 -- 172.21.15.46:0/3936123573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22dc1a9ac0 con 0x7f22dc108960 2024-05-09T04:50:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.368+0000 7f22e1ff7700 1 -- 172.21.15.46:0/3936123573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22dc0512c0 con 0x7f22dc108960 2024-05-09T04:50:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.370+0000 7f22d8ff9700 1 -- 172.21.15.46:0/3936123573 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f22d4019020 con 0x7f22dc108960 2024-05-09T04:50:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.370+0000 7f22d8ff9700 1 --2- 172.21.15.46:0/3936123573 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f22c80b2880 0x7f22c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.371+0000 7f22db7fe700 1 --2- 172.21.15.46:0/3936123573 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f22c80b2880 0x7f22c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:08.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.371+0000 7f22d8ff9700 1 -- 172.21.15.46:0/3936123573 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f22d40e26f0 con 0x7f22dc108960 2024-05-09T04:50:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.371+0000 7f22db7fe700 1 --2- 172.21.15.46:0/3936123573 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f22c80b2880 0x7f22c80b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00ac50 tx=0x7f22cc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:08.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.374+0000 7f22d8ff9700 1 -- 172.21.15.46:0/3936123573 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f22d4020020 con 0x7f22dc108960 2024-05-09T04:50:08.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.637+0000 7f22e1ff7700 1 -- 172.21.15.46:0/3936123573 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22dc06c180 con 0x7f22dc108960 2024-05-09T04:50:08.637 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:08 smithi046 ceph-mon[30136]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-09T04:50:08.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.638+0000 7f22d8ff9700 1 -- 172.21.15.46:0/3936123573 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11862 (secure 0 0 0) 0x7f22d40aa3d0 con 0x7f22dc108960 2024-05-09T04:50:08.638 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:50:08.638 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:48:11.653145+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24473,24503,24448]},"id":1}]} 2024-05-09T04:50:08.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.640+0000 7f22e1ff7700 1 -- 172.21.15.46:0/3936123573 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f22c80b2880 msgr2=0x7f22c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:08.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.640+0000 7f22e1ff7700 1 --2- 172.21.15.46:0/3936123573 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f22c80b2880 0x7f22c80b4d90 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f22cc00ac50 tx=0x7f22cc00a380 comp rx=0 tx=0).stop 2024-05-09T04:50:08.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.640+0000 7f22e1ff7700 1 -- 172.21.15.46:0/3936123573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc108960 msgr2=0x7f22dc111520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:08.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.640+0000 7f22e1ff7700 1 --2- 172.21.15.46:0/3936123573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc108960 0x7f22dc111520 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f22d4007d70 tx=0x7f22d400a340 comp rx=0 tx=0).stop 2024-05-09T04:50:08.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.640+0000 7f22e1ff7700 1 -- 172.21.15.46:0/3936123573 shutdown_connections 2024-05-09T04:50:08.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.640+0000 7f22e1ff7700 1 --2- 172.21.15.46:0/3936123573 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f22c80b2880 0x7f22c80b4d90 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:08.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.640+0000 7f22e1ff7700 1 --2- 172.21.15.46:0/3936123573 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22dc108960 0x7f22dc111520 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:08.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.640+0000 7f22e1ff7700 1 --2- 172.21.15.46:0/3936123573 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f22dc103c00 0x7f22dc110aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:08.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.640+0000 7f22e1ff7700 1 --2- 172.21.15.46:0/3936123573 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22dc108010 0x7f22dc110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:08.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.641+0000 7f22e1ff7700 1 -- 172.21.15.46:0/3936123573 >> 172.21.15.46:0/3936123573 conn(0x7f22dc0fe530 msgr2=0x7f22dc107230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:08.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.641+0000 7f22e1ff7700 1 -- 172.21.15.46:0/3936123573 shutdown_connections 2024-05-09T04:50:08.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:08.641+0000 7f22e1ff7700 1 -- 172.21.15.46:0/3936123573 wait complete. 2024-05-09T04:50:08.642 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-05-09T04:50:08.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:08 smithi110 ceph-mon[34904]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-09T04:50:08.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:08 smithi072 ceph-mon[34088]: pgmap v364: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-05-09T04:50:09.380 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T04:47:50.321554+0000', 'modified': '2024-05-09T04:48:11.653145+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24503, 'mds_1': 24473, 'mds_2': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24427': {'gid': 24427, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6835/2299300184', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6834', 'nonce': 2299300184}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 2299300184}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6837/2612088705', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6836', 'nonce': 2612088705}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 2612088705}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.72:6835/4256517546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6834', 'nonce': 4256517546}, {'type': 'v1', 'addr': '172.21.15.72:6835', 'nonce': 4256517546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.110:6839/556391603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 556391603}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 556391603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.72:6837/3365501578', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6836', 'nonce': 3365501578}, {'type': 'v1', 'addr': '172.21.15.72:6837', 'nonce': 3365501578}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.72:6839/2905856798', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6838', 'nonce': 2905856798}, {'type': 'v1', 'addr': '172.21.15.72:6839', 'nonce': 2905856798}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24503, 'qdb_cluster': [24473, 24503, 24448]} 2024-05-09T04:50:09.380 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-09T04:50:09.380 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T04:50:09.637 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:09.760 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:09 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3936123573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:09.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:09 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3936123573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:09.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:09 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3936123573' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:10.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:10 smithi046 ceph-mon[30136]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T04:50:10.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:10 smithi110 ceph-mon[34904]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T04:50:10.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:10 smithi072 ceph-mon[34088]: pgmap v365: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2024-05-09T04:50:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.415+0000 7f719616c700 1 -- 172.21.15.46:0/1553971364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7190107df0 msgr2=0x7f7190108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:11.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.415+0000 7f719616c700 1 --2- 172.21.15.46:0/1553971364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7190107df0 0x7f7190108200 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f71840099c0 tx=0x7f7184009cd0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.415+0000 7f719616c700 1 -- 172.21.15.46:0/1553971364 shutdown_connections 2024-05-09T04:50:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.415+0000 7f719616c700 1 --2- 172.21.15.46:0/1553971364 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7190108740 0x7f71901100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.415+0000 7f719616c700 1 --2- 172.21.15.46:0/1553971364 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71901034c0 0x7f71901038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.415+0000 7f719616c700 1 --2- 172.21.15.46:0/1553971364 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7190107df0 0x7f7190108200 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.415+0000 7f719616c700 1 -- 172.21.15.46:0/1553971364 >> 172.21.15.46:0/1553971364 conn(0x7f71900fedf0 msgr2=0x7f7190101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.415+0000 7f719616c700 1 -- 172.21.15.46:0/1553971364 shutdown_connections 2024-05-09T04:50:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.415+0000 7f719616c700 1 -- 172.21.15.46:0/1553971364 wait complete. 2024-05-09T04:50:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.416+0000 7f719616c700 1 Processor -- start 2024-05-09T04:50:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.416+0000 7f719616c700 1 -- start start 2024-05-09T04:50:11.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.416+0000 7f719616c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71901034c0 0x7f719019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f719616c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7190107df0 0x7f719019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f719616c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7190108740 0x7f719019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f719616c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7190113350 con 0x7f71901034c0 2024-05-09T04:50:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f719616c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f71901131d0 con 0x7f7190108740 2024-05-09T04:50:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f719616c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71901134d0 con 0x7f7190107df0 2024-05-09T04:50:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f718f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71901034c0 0x7f719019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f718f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71901034c0 0x7f719019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56058/0 (socket says 172.21.15.46:56058) 2024-05-09T04:50:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f718f7fe700 1 -- 172.21.15.46:0/3432968974 learned_addr learned my addr 172.21.15.46:0/3432968974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:11.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f718ffff700 1 --2- 172.21.15.46:0/3432968974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7190108740 0x7f719019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f718effd700 1 --2- 172.21.15.46:0/3432968974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7190107df0 0x7f719019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f718f7fe700 1 -- 172.21.15.46:0/3432968974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7190107df0 msgr2=0x7f719019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f718f7fe700 1 --2- 172.21.15.46:0/3432968974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7190107df0 0x7f719019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f718f7fe700 1 -- 172.21.15.46:0/3432968974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7190108740 msgr2=0x7f719019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f718f7fe700 1 --2- 172.21.15.46:0/3432968974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7190108740 0x7f719019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f718f7fe700 1 -- 172.21.15.46:0/3432968974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f719019fcc0 con 0x7f71901034c0 2024-05-09T04:50:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f718effd700 1 --2- 172.21.15.46:0/3432968974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7190107df0 0x7f719019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:50:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f718f7fe700 1 --2- 172.21.15.46:0/3432968974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71901034c0 0x7f719019adf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f718000d980 tx=0x7f718000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f718cff9700 1 -- 172.21.15.46:0/3432968974 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7180016070 con 0x7f71901034c0 2024-05-09T04:50:11.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f719616c700 1 -- 172.21.15.46:0/3432968974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f719019ffb0 con 0x7f71901034c0 2024-05-09T04:50:11.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f718cff9700 1 -- 172.21.15.46:0/3432968974 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7180004830 con 0x7f71901034c0 2024-05-09T04:50:11.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.417+0000 7f718cff9700 1 -- 172.21.15.46:0/3432968974 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71800054c0 con 0x7f71901034c0 2024-05-09T04:50:11.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.418+0000 7f719616c700 1 -- 172.21.15.46:0/3432968974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71900ff200 con 0x7f71901034c0 2024-05-09T04:50:11.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.420+0000 7f718cff9700 1 -- 172.21.15.46:0/3432968974 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f7180004220 con 0x7f71901034c0 2024-05-09T04:50:11.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.421+0000 7f718cff9700 1 --2- 172.21.15.46:0/3432968974 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f71780b2880 0x7f71780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.421+0000 7f718effd700 1 --2- 172.21.15.46:0/3432968974 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f71780b2880 0x7f71780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.421+0000 7f718cff9700 1 -- 172.21.15.46:0/3432968974 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f71800e2140 con 0x7f71901034c0 2024-05-09T04:50:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.422+0000 7f718effd700 1 --2- 172.21.15.46:0/3432968974 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f71780b2880 0x7f71780b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7184000c00 tx=0x7f718401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.422+0000 7f71a4b82b80 1 Processor -- start 2024-05-09T04:50:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.422+0000 7f71a4b82b80 1 -- start start 2024-05-09T04:50:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.422+0000 7f71a4b82b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb56b110 0x5592eb6044a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.422+0000 7f71a4b82b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb481450 0x5592eb6049e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.422+0000 7f71a4b82b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 0x5592eb5fea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.422+0000 7f71a4b82b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5592eb520fa0 con 0x5592eb481450 2024-05-09T04:50:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.422+0000 7f71a4b82b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5592eb4bf7a0 con 0x5592eb56b110 2024-05-09T04:50:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.422+0000 7f71a4b82b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5592eb51cf60 con 0x5592eb5fe650 2024-05-09T04:50:11.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.422+0000 7f718effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb481450 0x5592eb6049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.422+0000 7f718effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb481450 0x5592eb6049e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56072/0 (socket says 172.21.15.46:56072) 2024-05-09T04:50:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.422+0000 7f718effd700 1 -- 172.21.15.46:0/2998182734 learned_addr learned my addr 172.21.15.46:0/2998182734 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.423+0000 7f718f7fe700 1 --2- 172.21.15.46:0/2998182734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb56b110 0x5592eb6044a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.423+0000 7f718ffff700 1 --2- 172.21.15.46:0/2998182734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 0x5592eb5fea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.423+0000 7f718effd700 1 -- 172.21.15.46:0/2998182734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 msgr2=0x5592eb5fea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.423+0000 7f718effd700 1 --2- 172.21.15.46:0/2998182734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 0x5592eb5fea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.423+0000 7f718effd700 1 -- 172.21.15.46:0/2998182734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb56b110 msgr2=0x5592eb6044a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.423+0000 7f718effd700 1 --2- 172.21.15.46:0/2998182734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb56b110 0x5592eb6044a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.423+0000 7f718effd700 1 -- 172.21.15.46:0/2998182734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592eb4b7320 con 0x5592eb481450 2024-05-09T04:50:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.423+0000 7f718ffff700 1 --2- 172.21.15.46:0/2998182734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 0x5592eb5fea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:50:11.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.423+0000 7f718effd700 1 --2- 172.21.15.46:0/2998182734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb481450 0x5592eb6049e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f71840099c0 tx=0x7f718401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53703fb78f0eadba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.423+0000 7f7175ffb700 1 -- 172.21.15.46:0/2998182734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718400f8c0 con 0x5592eb481450 2024-05-09T04:50:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.423+0000 7f7175ffb700 1 -- 172.21.15.46:0/2998182734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7184003860 con 0x5592eb481450 2024-05-09T04:50:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.423+0000 7f7175ffb700 1 -- 172.21.15.46:0/2998182734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184005290 con 0x5592eb481450 2024-05-09T04:50:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.424+0000 7f71a4b82b80 1 -- 172.21.15.46:0/2998182734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb481450 msgr2=0x5592eb6049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.424+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/2998182734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb481450 0x5592eb6049e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f71840099c0 tx=0x7f718401e000 comp rx=0 tx=0).stop 2024-05-09T04:50:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.424+0000 7f71a4b82b80 1 -- 172.21.15.46:0/2998182734 shutdown_connections 2024-05-09T04:50:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.424+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/2998182734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 0x5592eb5fea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.424+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/2998182734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb56b110 0x5592eb6044a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.424+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/2998182734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb481450 0x5592eb6049e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.424+0000 7f71a4b82b80 1 -- 172.21.15.46:0/2998182734 >> 172.21.15.46:0/2998182734 conn(0x5592eb4e1650 msgr2=0x5592eb5f93b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.424+0000 7f71a4b82b80 1 -- 172.21.15.46:0/2998182734 shutdown_connections 2024-05-09T04:50:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.424+0000 7f71a4b82b80 1 -- 172.21.15.46:0/2998182734 wait complete. 2024-05-09T04:50:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.425+0000 7f71a4b82b80 1 -- start start 2024-05-09T04:50:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.425+0000 7f71a4b82b80 1 Processor -- start 2024-05-09T04:50:11.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.425+0000 7f71a4b82b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb481450 0x5592eb608270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.425+0000 7f71a4b82b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb56b110 0x5592eb6087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.425+0000 7f71a4b82b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb5fe650 0x5592eb608cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.425+0000 7f71a4b82b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5592eb39cb50 con 0x5592eb481450 2024-05-09T04:50:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.425+0000 7f71a4b82b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5592eb4db100 con 0x5592eb5fe650 2024-05-09T04:50:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.425+0000 7f71a4b82b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5592eb60b180 con 0x5592eb56b110 2024-05-09T04:50:11.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.425+0000 7f71a4b82b80 10 client.0 _ll_register_callbacks cb 0x5592eb392670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:50:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.425+0000 7f718f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb481450 0x5592eb608270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.425+0000 7f718f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb481450 0x5592eb608270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56074/0 (socket says 172.21.15.46:56074) 2024-05-09T04:50:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.425+0000 7f718f7fe700 1 -- 172.21.15.46:0/408072737 learned_addr learned my addr 172.21.15.46:0/408072737 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.425+0000 7f718effd700 1 --2- 172.21.15.46:0/408072737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb56b110 0x5592eb6087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.426+0000 7f718ffff700 1 --2- 172.21.15.46:0/408072737 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb5fe650 0x5592eb608cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.426+0000 7f718f7fe700 1 -- 172.21.15.46:0/408072737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb56b110 msgr2=0x5592eb6087b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.426+0000 7f718f7fe700 1 --2- 172.21.15.46:0/408072737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb56b110 0x5592eb6087b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.426+0000 7f718f7fe700 1 -- 172.21.15.46:0/408072737 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb5fe650 msgr2=0x5592eb608cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.426+0000 7f718f7fe700 1 --2- 172.21.15.46:0/408072737 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb5fe650 0x5592eb608cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.426+0000 7f718f7fe700 1 -- 172.21.15.46:0/408072737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592eb39c440 con 0x5592eb481450 2024-05-09T04:50:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.426+0000 7f718f7fe700 1 --2- 172.21.15.46:0/408072737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb481450 0x5592eb608270 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f71800dfba0 tx=0x7f71800a9280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=966dd9f4b24bc5ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:11.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.426+0000 7f716effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:50:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.426+0000 7f716effd700 1 -- 172.21.15.46:0/408072737 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71800c3060 con 0x5592eb481450 2024-05-09T04:50:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.426+0000 7f716effd700 1 -- 172.21.15.46:0/408072737 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f7180005660 con 0x5592eb481450 2024-05-09T04:50:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.426+0000 7f71a4b82b80 1 -- 172.21.15.46:0/408072737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7184003860 con 0x5592eb481450 2024-05-09T04:50:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.426+0000 7f716effd700 1 -- 172.21.15.46:0/408072737 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f71800a96c0 con 0x5592eb481450 2024-05-09T04:50:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.426+0000 7f71a4b82b80 10 client.14970 fetch_fsmap learned FSMap version 24 2024-05-09T04:50:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.426+0000 7f71a4b82b80 1 -- 172.21.15.46:0/408072737 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5592eb615ca0 con 0x5592eb481450 2024-05-09T04:50:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.426+0000 7f716effd700 1 -- 172.21.15.46:0/408072737 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f718006b030 con 0x5592eb481450 2024-05-09T04:50:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.427+0000 7f716effd700 1 -- 172.21.15.46:0/408072737 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f71800b2090 con 0x5592eb481450 2024-05-09T04:50:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.427+0000 7f71a4b82b80 10 client.14970 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T04:50:11.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.427+0000 7f71a4b82b80 10 client.14970 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T04:50:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.427+0000 7f71a4b82b80 20 client.14970 populate_metadata read hostname 'smithi046' 2024-05-09T04:50:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.427+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/408072737 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5592eb618210 0x5592eb619680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.427+0000 7f71a4b82b80 4 client.14970 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:50:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.427+0000 7f71a4b82b80 1 -- 172.21.15.46:0/408072737 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5592eb52e980 con 0x5592eb618210 2024-05-09T04:50:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.427+0000 7f718effd700 1 --2- 172.21.15.46:0/408072737 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5592eb618210 0x5592eb619680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.427+0000 7f716effd700 10 client.14970 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:50:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.427+0000 7f718effd700 1 --2- 172.21.15.46:0/408072737 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5592eb618210 0x5592eb619680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=819d112c5732cb36 server_cookie=b1e98d87f0f244ff in_seq=0 out_seq=0 2024-05-09T04:50:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.428+0000 7f716effd700 1 -- 172.21.15.46:0/408072737 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5592eb52e980 con 0x5592eb618210 2024-05-09T04:50:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.428+0000 7f716effd700 10 client.14970 handle_command_reply: tid=0 2024-05-09T04:50:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.428+0000 7f71a4b82b80 1 client.14970 shutdown 2024-05-09T04:50:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.429+0000 7f71a4b82b80 1 -- 172.21.15.46:0/408072737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb481450 msgr2=0x5592eb608270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.429+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/408072737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb481450 0x5592eb608270 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f71800dfba0 tx=0x7f71800a9280 comp rx=0 tx=0).stop 2024-05-09T04:50:11.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.429+0000 7f71a4b82b80 1 -- 172.21.15.46:0/408072737 shutdown_connections 2024-05-09T04:50:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.429+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/408072737 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb5fe650 0x5592eb608cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.429+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/408072737 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb481450 0x5592eb608270 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.429+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/408072737 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5592eb618210 0x5592eb619680 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.429+0000 7f716effd700 0 client.14970 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:50:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.429+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/408072737 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb56b110 0x5592eb6087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.429+0000 7f71a4b82b80 1 -- 172.21.15.46:0/408072737 >> 172.21.15.46:0/408072737 conn(0x5592eb4e1650 msgr2=0x5592eb5f9170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.429+0000 7f71a4b82b80 1 -- 172.21.15.46:0/408072737 shutdown_connections 2024-05-09T04:50:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.429+0000 7f71a4b82b80 1 -- 172.21.15.46:0/408072737 wait complete. 2024-05-09T04:50:11.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.429+0000 7f71a4b82b80 20 client.14970 trim_cache size 0 max 16384 2024-05-09T04:50:11.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.450+0000 7f71a4b82b80 1 Processor -- start 2024-05-09T04:50:11.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.450+0000 7f71a4b82b80 1 -- start start 2024-05-09T04:50:11.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.450+0000 7f71a4b82b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb481450 0x5592eb608b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.450+0000 7f71a4b82b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb56b110 0x5592eb603310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.450+0000 7f71a4b82b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 0x5592eb603850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.450+0000 7f71a4b82b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5592eb6179c0 con 0x5592eb56b110 2024-05-09T04:50:11.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.450+0000 7f71a4b82b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5592eb618090 con 0x5592eb481450 2024-05-09T04:50:11.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.450+0000 7f71a4b82b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5592eb618760 con 0x5592eb5fe650 2024-05-09T04:50:11.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.450+0000 7f718effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb56b110 0x5592eb603310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.450+0000 7f718effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb56b110 0x5592eb603310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56078/0 (socket says 172.21.15.46:56078) 2024-05-09T04:50:11.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.450+0000 7f718effd700 1 -- 172.21.15.46:0/3786452047 learned_addr learned my addr 172.21.15.46:0/3786452047 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:11.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.451+0000 7f718f7fe700 1 --2- 172.21.15.46:0/3786452047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb481450 0x5592eb608b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.451+0000 7f718ffff700 1 --2- 172.21.15.46:0/3786452047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 0x5592eb603850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.451+0000 7f718effd700 1 -- 172.21.15.46:0/3786452047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 msgr2=0x5592eb603850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:11.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.451+0000 7f718effd700 1 --2- 172.21.15.46:0/3786452047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 0x5592eb603850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.451+0000 7f718effd700 1 -- 172.21.15.46:0/3786452047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb481450 msgr2=0x5592eb608b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:11.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.451+0000 7f718effd700 1 --2- 172.21.15.46:0/3786452047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb481450 0x5592eb608b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.451+0000 7f718effd700 1 -- 172.21.15.46:0/3786452047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592eb609070 con 0x5592eb56b110 2024-05-09T04:50:11.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.451+0000 7f718effd700 1 --2- 172.21.15.46:0/3786452047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb56b110 0x5592eb603310 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f718400bbb0 tx=0x7f7184026040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d62cc406d1fa41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:11.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.451+0000 7f71757fa700 1 -- 172.21.15.46:0/3786452047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f718403d070 con 0x5592eb56b110 2024-05-09T04:50:11.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.451+0000 7f71757fa700 1 -- 172.21.15.46:0/3786452047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7184005190 con 0x5592eb56b110 2024-05-09T04:50:11.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.451+0000 7f71757fa700 1 -- 172.21.15.46:0/3786452047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7184038660 con 0x5592eb56b110 2024-05-09T04:50:11.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.452+0000 7f71a4b82b80 1 -- 172.21.15.46:0/3786452047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb56b110 msgr2=0x5592eb603310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:11.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.452+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/3786452047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb56b110 0x5592eb603310 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f718400bbb0 tx=0x7f7184026040 comp rx=0 tx=0).stop 2024-05-09T04:50:11.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.452+0000 7f71a4b82b80 1 -- 172.21.15.46:0/3786452047 shutdown_connections 2024-05-09T04:50:11.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.452+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/3786452047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 0x5592eb603850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.452+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/3786452047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb481450 0x5592eb608b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.452+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/3786452047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb56b110 0x5592eb603310 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.452+0000 7f71a4b82b80 1 -- 172.21.15.46:0/3786452047 >> 172.21.15.46:0/3786452047 conn(0x5592eb4e1650 msgr2=0x5592eb6044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:11.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.452+0000 7f71a4b82b80 1 -- 172.21.15.46:0/3786452047 shutdown_connections 2024-05-09T04:50:11.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.452+0000 7f71a4b82b80 1 -- 172.21.15.46:0/3786452047 wait complete. 2024-05-09T04:50:11.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.452+0000 7f71a4b82b80 1 -- start start 2024-05-09T04:50:11.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.453+0000 7f71a4b82b80 1 Processor -- start 2024-05-09T04:50:11.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.453+0000 7f71a4b82b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb481450 0x5592eb609770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.453+0000 7f71a4b82b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb56b110 0x5592eb609cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.453+0000 7f71a4b82b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 0x5592eb6220c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.453+0000 7f71a4b82b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5592eb619500 con 0x5592eb56b110 2024-05-09T04:50:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.453+0000 7f71a4b82b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5592eb618e30 con 0x5592eb481450 2024-05-09T04:50:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.453+0000 7f71a4b82b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5592eb619bd0 con 0x5592eb5fe650 2024-05-09T04:50:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.453+0000 7f71a4b82b80 10 client.0 _ll_register_callbacks cb 0x5592eb392670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:50:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.453+0000 7f718effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb56b110 0x5592eb609cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.453+0000 7f718effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb56b110 0x5592eb609cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56094/0 (socket says 172.21.15.46:56094) 2024-05-09T04:50:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.453+0000 7f718effd700 1 -- 172.21.15.46:0/4042024283 learned_addr learned my addr 172.21.15.46:0/4042024283 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:11.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.453+0000 7f718f7fe700 1 --2- 172.21.15.46:0/4042024283 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb481450 0x5592eb609770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.453+0000 7f718ffff700 1 --2- 172.21.15.46:0/4042024283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 0x5592eb6220c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.454+0000 7f718f7fe700 1 -- 172.21.15.46:0/4042024283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 msgr2=0x5592eb6220c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.454+0000 7f718f7fe700 1 --2- 172.21.15.46:0/4042024283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 0x5592eb6220c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.454+0000 7f718f7fe700 1 -- 172.21.15.46:0/4042024283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb56b110 msgr2=0x5592eb609cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.454+0000 7f718f7fe700 1 --2- 172.21.15.46:0/4042024283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb56b110 0x5592eb609cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.454+0000 7f718f7fe700 1 -- 172.21.15.46:0/4042024283 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592eb622600 con 0x5592eb481450 2024-05-09T04:50:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.454+0000 7f718f7fe700 1 --2- 172.21.15.46:0/4042024283 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb481450 0x5592eb609770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f718000b540 tx=0x7f71800dfc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ead87008a40a58a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.454+0000 7f716effd700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:50:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.454+0000 7f716effd700 1 -- 172.21.15.46:0/4042024283 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71800a9010 con 0x5592eb481450 2024-05-09T04:50:11.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.454+0000 7f716effd700 1 -- 172.21.15.46:0/4042024283 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f7180005660 con 0x5592eb481450 2024-05-09T04:50:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.454+0000 7f71a4b82b80 1 -- 172.21.15.46:0/4042024283 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7184005190 con 0x5592eb481450 2024-05-09T04:50:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.454+0000 7f716effd700 1 -- 172.21.15.46:0/4042024283 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f71800ada70 con 0x5592eb481450 2024-05-09T04:50:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.454+0000 7f71a4b82b80 10 client.24782 fetch_fsmap learned FSMap version 24 2024-05-09T04:50:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.454+0000 7f71a4b82b80 1 -- 172.21.15.46:0/4042024283 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5592eb683740 con 0x5592eb481450 2024-05-09T04:50:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.454+0000 7f716effd700 1 -- 172.21.15.46:0/4042024283 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7184005190 con 0x5592eb481450 2024-05-09T04:50:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.454+0000 7f716effd700 1 -- 172.21.15.46:0/4042024283 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f71800a5030 con 0x5592eb481450 2024-05-09T04:50:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.455+0000 7f71a4b82b80 10 client.24782 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T04:50:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.455+0000 7f71a4b82b80 10 client.24782 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T04:50:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.455+0000 7f71a4b82b80 20 client.24782 populate_metadata read hostname 'smithi046' 2024-05-09T04:50:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.455+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/4042024283 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5592eb618210 0x5592eb686fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:11.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.455+0000 7f71a4b82b80 4 client.24782 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:50:11.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.455+0000 7f71a4b82b80 1 -- 172.21.15.46:0/4042024283 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "status"}) -- 0x5592eb528a20 con 0x5592eb618210 2024-05-09T04:50:11.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.455+0000 7f718effd700 1 --2- 172.21.15.46:0/4042024283 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5592eb618210 0x5592eb686fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:11.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.455+0000 7f718effd700 1 --2- 172.21.15.46:0/4042024283 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5592eb618210 0x5592eb686fb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ce1f225d839bfda server_cookie=6f9581fa42bc7c9d in_seq=0 out_seq=0 2024-05-09T04:50:11.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.455+0000 7f716effd700 10 client.24782 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:50:11.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.456+0000 7f716effd700 1 -- 172.21.15.46:0/4042024283 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5592eb528a20 con 0x5592eb618210 2024-05-09T04:50:11.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.456+0000 7f716effd700 10 client.24782 handle_command_reply: tid=0 2024-05-09T04:50:11.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.456+0000 7f71a4b82b80 1 client.24782 shutdown 2024-05-09T04:50:11.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.456+0000 7f71a4b82b80 1 -- 172.21.15.46:0/4042024283 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb481450 msgr2=0x5592eb609770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:11.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.456+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/4042024283 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb481450 0x5592eb609770 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f718000b540 tx=0x7f71800dfc20 comp rx=0 tx=0).stop 2024-05-09T04:50:11.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.456+0000 7f71a4b82b80 1 -- 172.21.15.46:0/4042024283 shutdown_connections 2024-05-09T04:50:11.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.456+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/4042024283 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5592eb618210 0x5592eb686fb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.456+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/4042024283 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5592eb5fe650 0x5592eb6220c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.456+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/4042024283 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5592eb481450 0x5592eb609770 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.456+0000 7f71a4b82b80 1 --2- 172.21.15.46:0/4042024283 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5592eb56b110 0x5592eb609cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.456+0000 7f716effd700 0 client.24782 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:50:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.456+0000 7f71a4b82b80 1 -- 172.21.15.46:0/4042024283 >> 172.21.15.46:0/4042024283 conn(0x5592eb4e1650 msgr2=0x5592eb603310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.456+0000 7f71a4b82b80 1 -- 172.21.15.46:0/4042024283 shutdown_connections 2024-05-09T04:50:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.456+0000 7f71a4b82b80 1 -- 172.21.15.46:0/4042024283 wait complete. 2024-05-09T04:50:11.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.456+0000 7f71a4b82b80 20 client.24782 trim_cache size 0 max 16384 2024-05-09T04:50:11.463 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:50:11.464 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ecf9ae2c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:50:11.464 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 2, 2024-05-09T04:50:11.464 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24448, 2024-05-09T04:50:11.464 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-09T04:50:11.464 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-09T04:50:11.464 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-09T04:50:11.464 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 125.987860813, 2024-05-09T04:50:11.464 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 24, 2024-05-09T04:50:11.464 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 84, 2024-05-09T04:50:11.464 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 80, 2024-05-09T04:50:11.465 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 165.79050861299999 2024-05-09T04:50:11.465 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:50:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.457+0000 7f719616c700 1 -- 172.21.15.46:0/3432968974 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f71780b2880 msgr2=0x7f71780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.457+0000 7f719616c700 1 --2- 172.21.15.46:0/3432968974 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f71780b2880 0x7f71780b4d90 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f7184000c00 tx=0x7f718401a040 comp rx=0 tx=0).stop 2024-05-09T04:50:11.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.457+0000 7f719616c700 1 -- 172.21.15.46:0/3432968974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71901034c0 msgr2=0x7f719019adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.457+0000 7f719616c700 1 --2- 172.21.15.46:0/3432968974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71901034c0 0x7f719019adf0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f718000d980 tx=0x7f718000dc90 comp rx=0 tx=0).stop 2024-05-09T04:50:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.457+0000 7f719616c700 1 -- 172.21.15.46:0/3432968974 shutdown_connections 2024-05-09T04:50:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.457+0000 7f719616c700 1 --2- 172.21.15.46:0/3432968974 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f71780b2880 0x7f71780b4d90 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.457+0000 7f719616c700 1 --2- 172.21.15.46:0/3432968974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7190108740 0x7f719019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.457+0000 7f719616c700 1 --2- 172.21.15.46:0/3432968974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71901034c0 0x7f719019adf0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.457+0000 7f719616c700 1 --2- 172.21.15.46:0/3432968974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7190107df0 0x7f719019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.457+0000 7f719616c700 1 -- 172.21.15.46:0/3432968974 >> 172.21.15.46:0/3432968974 conn(0x7f71900fedf0 msgr2=0x7f7190075cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.457+0000 7f719616c700 1 -- 172.21.15.46:0/3432968974 shutdown_connections 2024-05-09T04:50:11.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:11.457+0000 7f719616c700 1 -- 172.21.15.46:0/3432968974 wait complete. 2024-05-09T04:50:12.178 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-09T04:50:12.430 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:12.803 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:12 smithi046 ceph-mon[30136]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T04:50:12.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:12 smithi110 ceph-mon[34904]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T04:50:12.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:12 smithi072 ceph-mon[34088]: pgmap v366: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T04:50:14.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.141+0000 7f567a6c0700 1 -- 172.21.15.46:0/2574896201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5674104650 msgr2=0x7f5674104a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:14.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.141+0000 7f567a6c0700 1 --2- 172.21.15.46:0/2574896201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5674104650 0x7f5674104a60 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f56640099c0 tx=0x7f5664009cd0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.141+0000 7f567a6c0700 1 -- 172.21.15.46:0/2574896201 shutdown_connections 2024-05-09T04:50:14.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.141+0000 7f567a6c0700 1 --2- 172.21.15.46:0/2574896201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5674109880 0x7f56741111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.141+0000 7f567a6c0700 1 --2- 172.21.15.46:0/2574896201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5674104650 0x7f5674104a60 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.141+0000 7f567a6c0700 1 --2- 172.21.15.46:0/2574896201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5674108f30 0x7f5674109340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.141+0000 7f567a6c0700 1 -- 172.21.15.46:0/2574896201 >> 172.21.15.46:0/2574896201 conn(0x7f5674079f00 msgr2=0x7f567407a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:14.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.141+0000 7f567a6c0700 1 -- 172.21.15.46:0/2574896201 shutdown_connections 2024-05-09T04:50:14.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.141+0000 7f567a6c0700 1 -- 172.21.15.46:0/2574896201 wait complete. 2024-05-09T04:50:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.142+0000 7f567a6c0700 1 Processor -- start 2024-05-09T04:50:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.142+0000 7f567a6c0700 1 -- start start 2024-05-09T04:50:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.142+0000 7f567a6c0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5674104650 0x7f567419d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.142+0000 7f567a6c0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5674108f30 0x7f567419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f567a6c0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5674109880 0x7f56741a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f567a6c0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5674114520 con 0x7f5674108f30 2024-05-09T04:50:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f567a6c0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f56741143a0 con 0x7f5674104650 2024-05-09T04:50:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f567a6c0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f56741146a0 con 0x7f5674109880 2024-05-09T04:50:14.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f566bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5674108f30 0x7f567419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f566bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5674108f30 0x7f567419d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43278/0 (socket says 172.21.15.46:43278) 2024-05-09T04:50:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f566bfff700 1 -- 172.21.15.46:0/2810102775 learned_addr learned my addr 172.21.15.46:0/2810102775 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f5673fff700 1 --2- 172.21.15.46:0/2810102775 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5674104650 0x7f567419d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f5678c5d700 1 --2- 172.21.15.46:0/2810102775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5674109880 0x7f56741a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f566bfff700 1 -- 172.21.15.46:0/2810102775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5674109880 msgr2=0x7f56741a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f566bfff700 1 --2- 172.21.15.46:0/2810102775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5674109880 0x7f56741a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f566bfff700 1 -- 172.21.15.46:0/2810102775 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5674104650 msgr2=0x7f567419d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f566bfff700 1 --2- 172.21.15.46:0/2810102775 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5674104650 0x7f567419d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f566bfff700 1 -- 172.21.15.46:0/2810102775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56741a1fa0 con 0x7f5674108f30 2024-05-09T04:50:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f5678c5d700 1 --2- 172.21.15.46:0/2810102775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5674109880 0x7f56741a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:50:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f566bfff700 1 --2- 172.21.15.46:0/2810102775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5674108f30 0x7f567419d670 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f565c00b500 tx=0x7f565c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f5671ffb700 1 -- 172.21.15.46:0/2810102775 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c015020 con 0x7f5674108f30 2024-05-09T04:50:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f5671ffb700 1 -- 172.21.15.46:0/2810102775 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f565c011ce0 con 0x7f5674108f30 2024-05-09T04:50:14.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f567a6c0700 1 -- 172.21.15.46:0/2810102775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56741a2290 con 0x7f5674108f30 2024-05-09T04:50:14.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.143+0000 7f5671ffb700 1 -- 172.21.15.46:0/2810102775 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f565c010d40 con 0x7f5674108f30 2024-05-09T04:50:14.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.144+0000 7f567a6c0700 1 -- 172.21.15.46:0/2810102775 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56741a9bb0 con 0x7f5674108f30 2024-05-09T04:50:14.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.146+0000 7f5671ffb700 1 -- 172.21.15.46:0/2810102775 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f565c018020 con 0x7f5674108f30 2024-05-09T04:50:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.147+0000 7f5671ffb700 1 --2- 172.21.15.46:0/2810102775 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f56600b2900 0x7f56600b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.147+0000 7f5673fff700 1 --2- 172.21.15.46:0/2810102775 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f56600b2900 0x7f56600b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.147+0000 7f5671ffb700 1 -- 172.21.15.46:0/2810102775 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f565c0e18e0 con 0x7f5674108f30 2024-05-09T04:50:14.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.147+0000 7f56890d6b80 1 Processor -- start 2024-05-09T04:50:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.147+0000 7f56890d6b80 1 -- start start 2024-05-09T04:50:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f5673fff700 1 --2- 172.21.15.46:0/2810102775 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f56600b2900 0x7f56600b4e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5664009990 tx=0x7f5664000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f56890d6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a743831f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f56890d6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a74383730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f56890d6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a7437d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f56890d6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562a742a0050 con 0x562a7437d3b0 2024-05-09T04:50:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f56890d6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562a74245950 con 0x562a74200450 2024-05-09T04:50:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f56890d6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562a7429c010 con 0x562a7427acb0 2024-05-09T04:50:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f5678c5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a7437d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f5678c5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a7437d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43284/0 (socket says 172.21.15.46:43284) 2024-05-09T04:50:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f5678c5d700 1 -- 172.21.15.46:0/3759137000 learned_addr learned my addr 172.21.15.46:0/3759137000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f566bfff700 1 --2- 172.21.15.46:0/3759137000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a74383730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f5673fff700 1 --2- 172.21.15.46:0/3759137000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a743831f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f5678c5d700 1 -- 172.21.15.46:0/3759137000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 msgr2=0x562a743831f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:14.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f5678c5d700 1 --2- 172.21.15.46:0/3759137000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a743831f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f5678c5d700 1 -- 172.21.15.46:0/3759137000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 msgr2=0x562a74383730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:14.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f5678c5d700 1 --2- 172.21.15.46:0/3759137000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a74383730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.148+0000 7f5678c5d700 1 -- 172.21.15.46:0/3759137000 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a742364b0 con 0x562a7437d3b0 2024-05-09T04:50:14.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.149+0000 7f5678c5d700 1 --2- 172.21.15.46:0/3759137000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a7437d7c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f566c0084f0 tx=0x7f566c008800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd22ff6314d1930e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:14.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.149+0000 7f566a7fc700 1 -- 172.21.15.46:0/3759137000 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c018070 con 0x562a7437d3b0 2024-05-09T04:50:14.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.149+0000 7f566a7fc700 1 -- 172.21.15.46:0/3759137000 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566c011bc0 con 0x562a7437d3b0 2024-05-09T04:50:14.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.149+0000 7f566a7fc700 1 -- 172.21.15.46:0/3759137000 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c013ab0 con 0x562a7437d3b0 2024-05-09T04:50:14.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.150+0000 7f56890d6b80 1 -- 172.21.15.46:0/3759137000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 msgr2=0x562a7437d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:14.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.150+0000 7f56890d6b80 1 --2- 172.21.15.46:0/3759137000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a7437d7c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f566c0084f0 tx=0x7f566c008800 comp rx=0 tx=0).stop 2024-05-09T04:50:14.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.150+0000 7f56890d6b80 1 -- 172.21.15.46:0/3759137000 shutdown_connections 2024-05-09T04:50:14.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.150+0000 7f56890d6b80 1 --2- 172.21.15.46:0/3759137000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a7437d7c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.150+0000 7f56890d6b80 1 --2- 172.21.15.46:0/3759137000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a743831f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.150+0000 7f56890d6b80 1 --2- 172.21.15.46:0/3759137000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a74383730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.150+0000 7f56890d6b80 1 -- 172.21.15.46:0/3759137000 >> 172.21.15.46:0/3759137000 conn(0x562a74203f20 msgr2=0x562a743781d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:14.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.150+0000 7f56890d6b80 1 -- 172.21.15.46:0/3759137000 shutdown_connections 2024-05-09T04:50:14.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.150+0000 7f56890d6b80 1 -- 172.21.15.46:0/3759137000 wait complete. 2024-05-09T04:50:14.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.150+0000 7f56890d6b80 1 -- start start 2024-05-09T04:50:14.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.150+0000 7f56890d6b80 1 Processor -- start 2024-05-09T04:50:14.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f56890d6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a74387150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f56890d6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a74387690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f56890d6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a74387bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f56890d6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562a7437e110 con 0x562a7437d3b0 2024-05-09T04:50:14.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f56890d6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562a7423aad0 con 0x562a74200450 2024-05-09T04:50:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f56890d6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562a74389fd0 con 0x562a7427acb0 2024-05-09T04:50:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f56890d6b80 10 client.0 _ll_register_callbacks cb 0x562a74111670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:50:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f5678c5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a74387bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f5678c5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a74387bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43294/0 (socket says 172.21.15.46:43294) 2024-05-09T04:50:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f5678c5d700 1 -- 172.21.15.46:0/4279949420 learned_addr learned my addr 172.21.15.46:0/4279949420 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f566bfff700 1 --2- 172.21.15.46:0/4279949420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a74387690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f5673fff700 1 --2- 172.21.15.46:0/4279949420 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a74387150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f5678c5d700 1 -- 172.21.15.46:0/4279949420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 msgr2=0x562a74387690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f5678c5d700 1 --2- 172.21.15.46:0/4279949420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a74387690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f5678c5d700 1 -- 172.21.15.46:0/4279949420 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 msgr2=0x562a74387150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f5678c5d700 1 --2- 172.21.15.46:0/4279949420 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a74387150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f5678c5d700 1 -- 172.21.15.46:0/4279949420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a7411b440 con 0x562a7437d3b0 2024-05-09T04:50:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f5673fff700 1 --2- 172.21.15.46:0/4279949420 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a74387150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:50:14.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f5678c5d700 1 --2- 172.21.15.46:0/4279949420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a74387bd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f566c0136d0 tx=0x7f566c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80ba2a32796a1809 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.151+0000 7f56537fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:50:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.152+0000 7f56537fe700 1 -- 172.21.15.46:0/4279949420 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c021070 con 0x562a7437d3b0 2024-05-09T04:50:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.152+0000 7f56537fe700 1 -- 172.21.15.46:0/4279949420 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f566c013930 con 0x562a7437d3b0 2024-05-09T04:50:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.152+0000 7f56890d6b80 1 -- 172.21.15.46:0/4279949420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f566c011bc0 con 0x562a7437d3b0 2024-05-09T04:50:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.152+0000 7f56537fe700 1 -- 172.21.15.46:0/4279949420 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f566c02d050 con 0x562a7437d3b0 2024-05-09T04:50:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.152+0000 7f56890d6b80 10 client.14988 fetch_fsmap learned FSMap version 24 2024-05-09T04:50:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.152+0000 7f56890d6b80 1 -- 172.21.15.46:0/4279949420 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562a74394ad0 con 0x562a7437d3b0 2024-05-09T04:50:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.152+0000 7f56537fe700 1 -- 172.21.15.46:0/4279949420 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566c011bc0 con 0x562a7437d3b0 2024-05-09T04:50:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.152+0000 7f56537fe700 1 -- 172.21.15.46:0/4279949420 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f566c01db20 con 0x562a7437d3b0 2024-05-09T04:50:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.152+0000 7f56890d6b80 10 client.14988 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T04:50:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.152+0000 7f56890d6b80 10 client.14988 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T04:50:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.152+0000 7f56890d6b80 20 client.14988 populate_metadata read hostname 'smithi046' 2024-05-09T04:50:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.152+0000 7f56890d6b80 1 --2- 172.21.15.46:0/4279949420 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x562a74396de0 0x562a74398250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.152+0000 7f56890d6b80 4 client.14988 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:50:14.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.152+0000 7f56890d6b80 1 -- 172.21.15.46:0/4279949420 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a742ada30 con 0x562a74396de0 2024-05-09T04:50:14.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.153+0000 7f5673fff700 1 --2- 172.21.15.46:0/4279949420 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x562a74396de0 0x562a74398250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.153+0000 7f56537fe700 10 client.14988 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:50:14.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.153+0000 7f5673fff700 1 --2- 172.21.15.46:0/4279949420 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x562a74396de0 0x562a74398250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d0a464cbef1ced79 server_cookie=323676046223219b in_seq=0 out_seq=0 2024-05-09T04:50:14.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.154+0000 7f56537fe700 1 -- 172.21.15.46:0/4279949420 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562a742ada30 con 0x562a74396de0 2024-05-09T04:50:14.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.154+0000 7f56537fe700 10 client.14988 handle_command_reply: tid=0 2024-05-09T04:50:14.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.154+0000 7f56890d6b80 1 client.14988 shutdown 2024-05-09T04:50:14.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.155+0000 7f56890d6b80 1 -- 172.21.15.46:0/4279949420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 msgr2=0x562a74387bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:14.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.155+0000 7f56890d6b80 1 --2- 172.21.15.46:0/4279949420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a74387bd0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f566c0136d0 tx=0x7f566c011ab0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.155+0000 7f56890d6b80 1 -- 172.21.15.46:0/4279949420 shutdown_connections 2024-05-09T04:50:14.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.155+0000 7f56890d6b80 1 --2- 172.21.15.46:0/4279949420 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x562a74396de0 0x562a74398250 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.155+0000 7f56890d6b80 1 --2- 172.21.15.46:0/4279949420 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a74387bd0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.155+0000 7f56890d6b80 1 --2- 172.21.15.46:0/4279949420 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a74387150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.155+0000 7f56890d6b80 1 --2- 172.21.15.46:0/4279949420 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a74387690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.155+0000 7f56537fe700 0 client.14988 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:50:14.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.155+0000 7f56890d6b80 1 -- 172.21.15.46:0/4279949420 >> 172.21.15.46:0/4279949420 conn(0x562a74203f20 msgr2=0x562a74371fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:14.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.155+0000 7f56890d6b80 1 -- 172.21.15.46:0/4279949420 shutdown_connections 2024-05-09T04:50:14.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.155+0000 7f56890d6b80 1 -- 172.21.15.46:0/4279949420 wait complete. 2024-05-09T04:50:14.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.155+0000 7f56890d6b80 20 client.14988 trim_cache size 0 max 16384 2024-05-09T04:50:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.176+0000 7f56890d6b80 1 Processor -- start 2024-05-09T04:50:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.176+0000 7f56890d6b80 1 -- start start 2024-05-09T04:50:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f56890d6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a74371fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f56890d6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a743724e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f56890d6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a743a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f56890d6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562a74396590 con 0x562a7437d3b0 2024-05-09T04:50:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f56890d6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562a74396c60 con 0x562a74200450 2024-05-09T04:50:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f56890d6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562a74397330 con 0x562a7427acb0 2024-05-09T04:50:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f5678c5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a743a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f5678c5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a743a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43302/0 (socket says 172.21.15.46:43302) 2024-05-09T04:50:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f5678c5d700 1 -- 172.21.15.46:0/404389066 learned_addr learned my addr 172.21.15.46:0/404389066 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f5673fff700 1 --2- 172.21.15.46:0/404389066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a74371fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f5678c5d700 1 -- 172.21.15.46:0/404389066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 msgr2=0x562a743724e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f566bfff700 1 --2- 172.21.15.46:0/404389066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a743724e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f5678c5d700 1 --2- 172.21.15.46:0/404389066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a743724e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f5678c5d700 1 -- 172.21.15.46:0/404389066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 msgr2=0x562a74371fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f5678c5d700 1 --2- 172.21.15.46:0/404389066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a74371fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f5678c5d700 1 -- 172.21.15.46:0/404389066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a74374980 con 0x562a7437d3b0 2024-05-09T04:50:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f5678c5d700 1 --2- 172.21.15.46:0/404389066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a743a16b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f566c008410 tx=0x7f566c008800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd326c7c1e19af9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.177+0000 7f5669ffb700 1 -- 172.21.15.46:0/404389066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c021050 con 0x562a7437d3b0 2024-05-09T04:50:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.178+0000 7f5669ffb700 1 -- 172.21.15.46:0/404389066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566c007a20 con 0x562a7437d3b0 2024-05-09T04:50:14.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.178+0000 7f5669ffb700 1 -- 172.21.15.46:0/404389066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c018040 con 0x562a7437d3b0 2024-05-09T04:50:14.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.178+0000 7f56890d6b80 1 -- 172.21.15.46:0/404389066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 msgr2=0x562a743a16b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.178+0000 7f56890d6b80 1 --2- 172.21.15.46:0/404389066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a743a16b0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f566c008410 tx=0x7f566c008800 comp rx=0 tx=0).stop 2024-05-09T04:50:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.178+0000 7f56890d6b80 1 -- 172.21.15.46:0/404389066 shutdown_connections 2024-05-09T04:50:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.178+0000 7f56890d6b80 1 --2- 172.21.15.46:0/404389066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a743a16b0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.178+0000 7f56890d6b80 1 --2- 172.21.15.46:0/404389066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a74371fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.178+0000 7f56890d6b80 1 --2- 172.21.15.46:0/404389066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a743724e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.178+0000 7f56890d6b80 1 -- 172.21.15.46:0/404389066 >> 172.21.15.46:0/404389066 conn(0x562a74203f20 msgr2=0x562a743821f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.179+0000 7f56890d6b80 1 -- 172.21.15.46:0/404389066 shutdown_connections 2024-05-09T04:50:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.179+0000 7f56890d6b80 1 -- 172.21.15.46:0/404389066 wait complete. 2024-05-09T04:50:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.179+0000 7f56890d6b80 1 -- start start 2024-05-09T04:50:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.179+0000 7f56890d6b80 1 Processor -- start 2024-05-09T04:50:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.179+0000 7f56890d6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a7439d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.179+0000 7f56890d6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a7439d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.179+0000 7f56890d6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a7439ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f56890d6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562a743980d0 con 0x562a7437d3b0 2024-05-09T04:50:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f56890d6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562a74397a00 con 0x562a74200450 2024-05-09T04:50:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f56890d6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562a743987a0 con 0x562a7427acb0 2024-05-09T04:50:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f56890d6b80 10 client.0 _ll_register_callbacks cb 0x562a74111670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:50:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f5678c5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a7439ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f5673fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a7439d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f5678c5d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a7439ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43318/0 (socket says 172.21.15.46:43318) 2024-05-09T04:50:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f5673fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a7439d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:42294/0 (socket says 172.21.15.46:42294) 2024-05-09T04:50:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f5678c5d700 1 -- 172.21.15.46:0/1578039143 learned_addr learned my addr 172.21.15.46:0/1578039143 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f566bfff700 1 --2- 172.21.15.46:0/1578039143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a7439d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f5678c5d700 1 -- 172.21.15.46:0/1578039143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 msgr2=0x562a7439d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f5678c5d700 1 --2- 172.21.15.46:0/1578039143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a7439d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f5678c5d700 1 -- 172.21.15.46:0/1578039143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 msgr2=0x562a7439d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f5678c5d700 1 --2- 172.21.15.46:0/1578039143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a7439d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f5678c5d700 1 -- 172.21.15.46:0/1578039143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a7437e5b0 con 0x562a7437d3b0 2024-05-09T04:50:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f566bfff700 1 --2- 172.21.15.46:0/1578039143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a7439d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:50:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f5678c5d700 1 --2- 172.21.15.46:0/1578039143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a7439ddf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f566c002230 tx=0x7f566c01c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7b7c8c4a6e480f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f56537fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:50:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f56537fe700 1 -- 172.21.15.46:0/1578039143 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c018070 con 0x562a7437d3b0 2024-05-09T04:50:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f56890d6b80 1 -- 172.21.15.46:0/1578039143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f566c007a20 con 0x562a7437d3b0 2024-05-09T04:50:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.180+0000 7f56537fe700 1 -- 172.21.15.46:0/1578039143 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f566c0138d0 con 0x562a7437d3b0 2024-05-09T04:50:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.181+0000 7f56537fe700 1 -- 172.21.15.46:0/1578039143 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f566c021070 con 0x562a7437d3b0 2024-05-09T04:50:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.181+0000 7f56890d6b80 10 client.14994 fetch_fsmap learned FSMap version 24 2024-05-09T04:50:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.181+0000 7f56890d6b80 1 -- 172.21.15.46:0/1578039143 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x562a744022f0 con 0x562a7437d3b0 2024-05-09T04:50:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.181+0000 7f56537fe700 1 -- 172.21.15.46:0/1578039143 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566c007a20 con 0x562a7437d3b0 2024-05-09T04:50:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.181+0000 7f56537fe700 1 -- 172.21.15.46:0/1578039143 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f566c01db10 con 0x562a7437d3b0 2024-05-09T04:50:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.181+0000 7f56890d6b80 10 client.14994 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T04:50:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.181+0000 7f56890d6b80 10 client.14994 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T04:50:14.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.181+0000 7f56890d6b80 20 client.14994 populate_metadata read hostname 'smithi046' 2024-05-09T04:50:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.181+0000 7f56890d6b80 1 --2- 172.21.15.46:0/1578039143 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x562a74404340 0x562a744057b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.181+0000 7f56890d6b80 4 client.14994 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:50:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.181+0000 7f56890d6b80 1 -- 172.21.15.46:0/1578039143 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "status"}) -- 0x562a742a7ad0 con 0x562a74404340 2024-05-09T04:50:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.181+0000 7f5673fff700 1 --2- 172.21.15.46:0/1578039143 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x562a74404340 0x562a744057b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56537fe700 10 client.14994 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:50:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f5673fff700 1 --2- 172.21.15.46:0/1578039143 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x562a74404340 0x562a744057b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7871b7a981dc9749 server_cookie=bb5522d3377118fc in_seq=0 out_seq=0 2024-05-09T04:50:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56537fe700 1 -- 172.21.15.46:0/1578039143 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x562a742a7ad0 con 0x562a74404340 2024-05-09T04:50:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56537fe700 10 client.14994 handle_command_reply: tid=0 2024-05-09T04:50:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56890d6b80 1 client.14994 shutdown 2024-05-09T04:50:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56890d6b80 1 -- 172.21.15.46:0/1578039143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 msgr2=0x562a7439ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56890d6b80 1 --2- 172.21.15.46:0/1578039143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a7439ddf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f566c002230 tx=0x7f566c01c570 comp rx=0 tx=0).stop 2024-05-09T04:50:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56890d6b80 1 -- 172.21.15.46:0/1578039143 shutdown_connections 2024-05-09T04:50:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56890d6b80 1 --2- 172.21.15.46:0/1578039143 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x562a74404340 0x562a744057b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56890d6b80 1 --2- 172.21.15.46:0/1578039143 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562a7437d3b0 0x562a7439ddf0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56890d6b80 1 --2- 172.21.15.46:0/1578039143 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562a74200450 0x562a7439d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56537fe700 0 client.14994 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:50:14.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56890d6b80 1 --2- 172.21.15.46:0/1578039143 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562a7427acb0 0x562a7439d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56890d6b80 1 -- 172.21.15.46:0/1578039143 >> 172.21.15.46:0/1578039143 conn(0x562a74203f20 msgr2=0x562a74371fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:14.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56890d6b80 1 -- 172.21.15.46:0/1578039143 shutdown_connections 2024-05-09T04:50:14.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56890d6b80 1 -- 172.21.15.46:0/1578039143 wait complete. 2024-05-09T04:50:14.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.182+0000 7f56890d6b80 20 client.14994 trim_cache size 0 max 16384 2024-05-09T04:50:14.188 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:50:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ecf9ae2c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:50:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 1, 2024-05-09T04:50:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24473, 2024-05-09T04:50:14.188 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-09T04:50:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-09T04:50:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-09T04:50:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 129.725630112, 2024-05-09T04:50:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 24, 2024-05-09T04:50:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 84, 2024-05-09T04:50:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 80, 2024-05-09T04:50:14.189 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 171.456451087 2024-05-09T04:50:14.189 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:50:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.183+0000 7f567a6c0700 1 -- 172.21.15.46:0/2810102775 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f56600b2900 msgr2=0x7f56600b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.183+0000 7f567a6c0700 1 --2- 172.21.15.46:0/2810102775 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f56600b2900 0x7f56600b4e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5664009990 tx=0x7f5664000bc0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.183+0000 7f567a6c0700 1 -- 172.21.15.46:0/2810102775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5674108f30 msgr2=0x7f567419d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.183+0000 7f567a6c0700 1 --2- 172.21.15.46:0/2810102775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5674108f30 0x7f567419d670 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f565c00b500 tx=0x7f565c00b810 comp rx=0 tx=0).stop 2024-05-09T04:50:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.183+0000 7f567a6c0700 1 -- 172.21.15.46:0/2810102775 shutdown_connections 2024-05-09T04:50:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.183+0000 7f567a6c0700 1 --2- 172.21.15.46:0/2810102775 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f56600b2900 0x7f56600b4e10 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.183+0000 7f567a6c0700 1 --2- 172.21.15.46:0/2810102775 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5674109880 0x7f56741a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.183+0000 7f567a6c0700 1 --2- 172.21.15.46:0/2810102775 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5674104650 0x7f567419d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.183+0000 7f567a6c0700 1 --2- 172.21.15.46:0/2810102775 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5674108f30 0x7f567419d670 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.183+0000 7f567a6c0700 1 -- 172.21.15.46:0/2810102775 >> 172.21.15.46:0/2810102775 conn(0x7f5674079f00 msgr2=0x7f5674102950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.183+0000 7f567a6c0700 1 -- 172.21.15.46:0/2810102775 shutdown_connections 2024-05-09T04:50:14.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:14.183+0000 7f567a6c0700 1 -- 172.21.15.46:0/2810102775 wait complete. 2024-05-09T04:50:14.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:14 smithi110 ceph-mon[34904]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T04:50:14.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:14 smithi046 ceph-mon[30136]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T04:50:14.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:14 smithi072 ceph-mon[34088]: pgmap v367: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T04:50:14.927 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T04:50:15.179 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:16.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:16 smithi046 ceph-mon[30136]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:50:16.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:16 smithi110 ceph-mon[34904]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:50:16.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:16 smithi072 ceph-mon[34088]: pgmap v368: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:50:16.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.911+0000 7fd609630700 1 -- 172.21.15.46:0/3483554352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6041045d0 msgr2=0x7fd6041049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:16.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.911+0000 7fd609630700 1 --2- 172.21.15.46:0/3483554352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6041045d0 0x7fd6041049e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fd5f40099c0 tx=0x7fd5f4009cd0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.911+0000 7fd609630700 1 -- 172.21.15.46:0/3483554352 shutdown_connections 2024-05-09T04:50:16.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.911+0000 7fd609630700 1 --2- 172.21.15.46:0/3483554352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd604109850 0x7fd6041111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.911+0000 7fd609630700 1 --2- 172.21.15.46:0/3483554352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6041045d0 0x7fd6041049e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.911+0000 7fd609630700 1 --2- 172.21.15.46:0/3483554352 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd604108f00 0x7fd604109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.911+0000 7fd609630700 1 -- 172.21.15.46:0/3483554352 >> 172.21.15.46:0/3483554352 conn(0x7fd6040ffea0 msgr2=0x7fd6041022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:16.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.911+0000 7fd609630700 1 -- 172.21.15.46:0/3483554352 shutdown_connections 2024-05-09T04:50:16.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.911+0000 7fd609630700 1 -- 172.21.15.46:0/3483554352 wait complete. 2024-05-09T04:50:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.912+0000 7fd609630700 1 Processor -- start 2024-05-09T04:50:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.912+0000 7fd609630700 1 -- start start 2024-05-09T04:50:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd609630700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6041045d0 0x7fd604076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd609630700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd604108f00 0x7fd604077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd609630700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd604109850 0x7fd604077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd609630700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd604114480 con 0x7fd6041045d0 2024-05-09T04:50:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd609630700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd604114300 con 0x7fd604108f00 2024-05-09T04:50:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd609630700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd604114600 con 0x7fd604109850 2024-05-09T04:50:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd602ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6041045d0 0x7fd604076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd602ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6041045d0 0x7fd604076bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43338/0 (socket says 172.21.15.46:43338) 2024-05-09T04:50:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd602ffd700 1 -- 172.21.15.46:0/630449414 learned_addr learned my addr 172.21.15.46:0/630449414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:16.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd6027fc700 1 --2- 172.21.15.46:0/630449414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd604108f00 0x7fd604077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd6037fe700 1 --2- 172.21.15.46:0/630449414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd604109850 0x7fd604077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd602ffd700 1 -- 172.21.15.46:0/630449414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd604109850 msgr2=0x7fd604077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:16.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd602ffd700 1 --2- 172.21.15.46:0/630449414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd604109850 0x7fd604077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd602ffd700 1 -- 172.21.15.46:0/630449414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd604108f00 msgr2=0x7fd604077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:16.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd602ffd700 1 --2- 172.21.15.46:0/630449414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd604108f00 0x7fd604077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd602ffd700 1 -- 172.21.15.46:0/630449414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6041a91a0 con 0x7fd6041045d0 2024-05-09T04:50:16.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.913+0000 7fd602ffd700 1 --2- 172.21.15.46:0/630449414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6041045d0 0x7fd604076bd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400b580 tx=0x7fd5f40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:16.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.914+0000 7fd5fbfff700 1 -- 172.21.15.46:0/630449414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4027070 con 0x7fd6041045d0 2024-05-09T04:50:16.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.914+0000 7fd609630700 1 -- 172.21.15.46:0/630449414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6041a9430 con 0x7fd6041045d0 2024-05-09T04:50:16.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.914+0000 7fd5fbfff700 1 -- 172.21.15.46:0/630449414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5f4003f20 con 0x7fd6041045d0 2024-05-09T04:50:16.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.914+0000 7fd5fbfff700 1 -- 172.21.15.46:0/630449414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5f4022d10 con 0x7fd6041045d0 2024-05-09T04:50:16.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.914+0000 7fd609630700 1 -- 172.21.15.46:0/630449414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6041a9890 con 0x7fd6041045d0 2024-05-09T04:50:16.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.916+0000 7fd5fbfff700 1 -- 172.21.15.46:0/630449414 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fd5f400f460 con 0x7fd6041045d0 2024-05-09T04:50:16.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.917+0000 7fd5fbfff700 1 --2- 172.21.15.46:0/630449414 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd5f00b2880 0x7fd5f00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.917+0000 7fd6027fc700 1 --2- 172.21.15.46:0/630449414 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd5f00b2880 0x7fd5f00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.917+0000 7fd5fbfff700 1 -- 172.21.15.46:0/630449414 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fd5f40e2ef0 con 0x7fd6041045d0 2024-05-09T04:50:16.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.918+0000 7fd6027fc700 1 --2- 172.21.15.46:0/630449414 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd5f00b2880 0x7fd5f00b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd604103520 tx=0x7fd5ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:16.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.918+0000 7fd618046b80 1 Processor -- start 2024-05-09T04:50:16.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.918+0000 7fd618046b80 1 -- start start 2024-05-09T04:50:16.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.918+0000 7fd618046b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564483a1acb0 0x564483b231f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.918+0000 7fd618046b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5644839a0450 0x564483b23730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.918+0000 7fd618046b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483b1d3b0 0x564483b1d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.918+0000 7fd618046b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564483a40050 con 0x564483b1d3b0 2024-05-09T04:50:16.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.918+0000 7fd618046b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5644839e5950 con 0x5644839a0450 2024-05-09T04:50:16.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.918+0000 7fd618046b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564483a3c010 con 0x564483a1acb0 2024-05-09T04:50:16.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.918+0000 7fd6037fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483b1d3b0 0x564483b1d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.918+0000 7fd6037fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483b1d3b0 0x564483b1d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43354/0 (socket says 172.21.15.46:43354) 2024-05-09T04:50:16.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.918+0000 7fd6037fe700 1 -- 172.21.15.46:0/4072994375 learned_addr learned my addr 172.21.15.46:0/4072994375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:16.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.918+0000 7fd6027fc700 1 --2- 172.21.15.46:0/4072994375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5644839a0450 0x564483b23730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.919+0000 7fd602ffd700 1 --2- 172.21.15.46:0/4072994375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564483a1acb0 0x564483b231f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.919+0000 7fd6037fe700 1 -- 172.21.15.46:0/4072994375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564483a1acb0 msgr2=0x564483b231f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:16.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.919+0000 7fd6037fe700 1 --2- 172.21.15.46:0/4072994375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564483a1acb0 0x564483b231f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.919+0000 7fd6037fe700 1 -- 172.21.15.46:0/4072994375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5644839a0450 msgr2=0x564483b23730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:16.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.919+0000 7fd6037fe700 1 --2- 172.21.15.46:0/4072994375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5644839a0450 0x564483b23730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.919+0000 7fd6037fe700 1 -- 172.21.15.46:0/4072994375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644839d64b0 con 0x564483b1d3b0 2024-05-09T04:50:16.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.919+0000 7fd602ffd700 1 --2- 172.21.15.46:0/4072994375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564483a1acb0 0x564483b231f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:50:16.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.919+0000 7fd6037fe700 1 --2- 172.21.15.46:0/4072994375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483b1d3b0 0x564483b1d7c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc003fa0 tx=0x7fd5fc009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b27aa3a459498ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:16.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.919+0000 7fd5f97fa700 1 -- 172.21.15.46:0/4072994375 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc004bb0 con 0x564483b1d3b0 2024-05-09T04:50:16.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.919+0000 7fd5f97fa700 1 -- 172.21.15.46:0/4072994375 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5fc004d50 con 0x564483b1d3b0 2024-05-09T04:50:16.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.919+0000 7fd5f97fa700 1 -- 172.21.15.46:0/4072994375 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc007d40 con 0x564483b1d3b0 2024-05-09T04:50:16.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.920+0000 7fd618046b80 1 -- 172.21.15.46:0/4072994375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483b1d3b0 msgr2=0x564483b1d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:16.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.920+0000 7fd618046b80 1 --2- 172.21.15.46:0/4072994375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483b1d3b0 0x564483b1d7c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc003fa0 tx=0x7fd5fc009ba0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.920+0000 7fd618046b80 1 -- 172.21.15.46:0/4072994375 shutdown_connections 2024-05-09T04:50:16.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.920+0000 7fd618046b80 1 --2- 172.21.15.46:0/4072994375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483b1d3b0 0x564483b1d7c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.920+0000 7fd618046b80 1 --2- 172.21.15.46:0/4072994375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564483a1acb0 0x564483b231f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.920+0000 7fd618046b80 1 --2- 172.21.15.46:0/4072994375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5644839a0450 0x564483b23730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.920+0000 7fd618046b80 1 -- 172.21.15.46:0/4072994375 >> 172.21.15.46:0/4072994375 conn(0x5644839a3f20 msgr2=0x564483b181d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:16.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.920+0000 7fd618046b80 1 -- 172.21.15.46:0/4072994375 shutdown_connections 2024-05-09T04:50:16.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.920+0000 7fd618046b80 1 -- 172.21.15.46:0/4072994375 wait complete. 2024-05-09T04:50:16.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.921+0000 7fd618046b80 1 -- start start 2024-05-09T04:50:16.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.921+0000 7fd618046b80 1 Processor -- start 2024-05-09T04:50:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.921+0000 7fd618046b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5644839a0450 0x564483b26f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.921+0000 7fd618046b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483a1acb0 0x564483b274c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.921+0000 7fd618046b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564483b1d3b0 0x564483b27a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.921+0000 7fd618046b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564483b1e110 con 0x564483a1acb0 2024-05-09T04:50:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.921+0000 7fd618046b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5644839daad0 con 0x564483b1d3b0 2024-05-09T04:50:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.921+0000 7fd618046b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564483b29fd0 con 0x5644839a0450 2024-05-09T04:50:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.921+0000 7fd618046b80 10 client.0 _ll_register_callbacks cb 0x5644838b1670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:50:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.921+0000 7fd6027fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483a1acb0 0x564483b274c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.921+0000 7fd6027fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483a1acb0 0x564483b274c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43370/0 (socket says 172.21.15.46:43370) 2024-05-09T04:50:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.921+0000 7fd6027fc700 1 -- 172.21.15.46:0/4146249407 learned_addr learned my addr 172.21.15.46:0/4146249407 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.921+0000 7fd6037fe700 1 --2- 172.21.15.46:0/4146249407 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564483b1d3b0 0x564483b27a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.921+0000 7fd602ffd700 1 --2- 172.21.15.46:0/4146249407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5644839a0450 0x564483b26f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.922+0000 7fd6027fc700 1 -- 172.21.15.46:0/4146249407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5644839a0450 msgr2=0x564483b26f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.922+0000 7fd6027fc700 1 --2- 172.21.15.46:0/4146249407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5644839a0450 0x564483b26f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.922+0000 7fd6027fc700 1 -- 172.21.15.46:0/4146249407 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564483b1d3b0 msgr2=0x564483b27a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.922+0000 7fd6027fc700 1 --2- 172.21.15.46:0/4146249407 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564483b1d3b0 0x564483b27a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.922+0000 7fd6027fc700 1 -- 172.21.15.46:0/4146249407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644838bb440 con 0x564483a1acb0 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.922+0000 7fd6027fc700 1 --2- 172.21.15.46:0/4146249407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483a1acb0 0x564483b274c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd60406c170 tx=0x7fd5ec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acfa91a8390d4e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.922+0000 7fd5ea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.922+0000 7fd5ea7fc700 1 -- 172.21.15.46:0/4146249407 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec004830 con 0x564483a1acb0 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.922+0000 7fd5ea7fc700 1 -- 172.21.15.46:0/4146249407 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fd5ec0049d0 con 0x564483a1acb0 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.922+0000 7fd618046b80 1 -- 172.21.15.46:0/4146249407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5fc004d50 con 0x564483a1acb0 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.922+0000 7fd5ea7fc700 1 -- 172.21.15.46:0/4146249407 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd5ec004bd0 con 0x564483a1acb0 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.922+0000 7fd618046b80 10 client.15006 fetch_fsmap learned FSMap version 24 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.922+0000 7fd618046b80 1 -- 172.21.15.46:0/4146249407 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564483b34a00 con 0x564483a1acb0 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.922+0000 7fd5ea7fc700 1 -- 172.21.15.46:0/4146249407 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5fc004d50 con 0x564483a1acb0 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.923+0000 7fd5ea7fc700 1 -- 172.21.15.46:0/4146249407 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd5ec01f5c0 con 0x564483a1acb0 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.923+0000 7fd618046b80 10 client.15006 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.923+0000 7fd618046b80 10 client.15006 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.923+0000 7fd618046b80 20 client.15006 populate_metadata read hostname 'smithi046' 2024-05-09T04:50:16.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.923+0000 7fd618046b80 1 --2- 172.21.15.46:0/4146249407 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564483b37020 0x564483b38490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.923+0000 7fd618046b80 4 client.15006 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.923+0000 7fd618046b80 1 -- 172.21.15.46:0/4146249407 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564483a4da30 con 0x564483b37020 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.923+0000 7fd602ffd700 1 --2- 172.21.15.46:0/4146249407 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564483b37020 0x564483b38490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.923+0000 7fd5ea7fc700 10 client.15006 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.923+0000 7fd602ffd700 1 --2- 172.21.15.46:0/4146249407 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564483b37020 0x564483b38490 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0766e295c9e7ad1 server_cookie=d9ccf90cd8b514a8 in_seq=0 out_seq=0 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.924+0000 7fd5ea7fc700 1 -- 172.21.15.46:0/4146249407 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564483a4da30 con 0x564483b37020 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.924+0000 7fd5ea7fc700 10 client.15006 handle_command_reply: tid=0 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.924+0000 7fd618046b80 1 client.15006 shutdown 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.924+0000 7fd618046b80 1 -- 172.21.15.46:0/4146249407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483a1acb0 msgr2=0x564483b274c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.924+0000 7fd618046b80 1 --2- 172.21.15.46:0/4146249407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483a1acb0 0x564483b274c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd60406c170 tx=0x7fd5ec0047c0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.924+0000 7fd618046b80 1 -- 172.21.15.46:0/4146249407 shutdown_connections 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.924+0000 7fd618046b80 1 --2- 172.21.15.46:0/4146249407 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564483b37020 0x564483b38490 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.924+0000 7fd618046b80 1 --2- 172.21.15.46:0/4146249407 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564483b1d3b0 0x564483b27a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.924+0000 7fd5ea7fc700 0 client.15006 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.924+0000 7fd618046b80 1 --2- 172.21.15.46:0/4146249407 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5644839a0450 0x564483b26f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.925+0000 7fd618046b80 1 --2- 172.21.15.46:0/4146249407 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483a1acb0 0x564483b274c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.925+0000 7fd618046b80 1 -- 172.21.15.46:0/4146249407 >> 172.21.15.46:0/4146249407 conn(0x5644839a3f20 msgr2=0x564483b11fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:16.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.925+0000 7fd618046b80 1 -- 172.21.15.46:0/4146249407 shutdown_connections 2024-05-09T04:50:16.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.925+0000 7fd618046b80 1 -- 172.21.15.46:0/4146249407 wait complete. 2024-05-09T04:50:16.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.925+0000 7fd618046b80 20 client.15006 trim_cache size 0 max 16384 2024-05-09T04:50:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.946+0000 7fd618046b80 1 Processor -- start 2024-05-09T04:50:16.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.946+0000 7fd618046b80 1 -- start start 2024-05-09T04:50:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.946+0000 7fd618046b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5644839a0450 0x564483b182e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.946+0000 7fd618046b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564483a1acb0 0x564483b27840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.946+0000 7fd618046b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483b1d3b0 0x564483b32fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.946+0000 7fd618046b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564483b367d0 con 0x564483b1d3b0 2024-05-09T04:50:16.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.946+0000 7fd618046b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x564483b36ea0 con 0x5644839a0450 2024-05-09T04:50:16.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.946+0000 7fd618046b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564483b37570 con 0x564483a1acb0 2024-05-09T04:50:16.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.946+0000 7fd6037fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483b1d3b0 0x564483b32fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.946+0000 7fd6037fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483b1d3b0 0x564483b32fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43384/0 (socket says 172.21.15.46:43384) 2024-05-09T04:50:16.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.946+0000 7fd6037fe700 1 -- 172.21.15.46:0/706082403 learned_addr learned my addr 172.21.15.46:0/706082403 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:16.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.947+0000 7fd6027fc700 1 --2- 172.21.15.46:0/706082403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564483a1acb0 0x564483b27840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.947+0000 7fd602ffd700 1 --2- 172.21.15.46:0/706082403 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5644839a0450 0x564483b182e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.947+0000 7fd6037fe700 1 -- 172.21.15.46:0/706082403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564483a1acb0 msgr2=0x564483b27840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:16.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.947+0000 7fd6037fe700 1 --2- 172.21.15.46:0/706082403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564483a1acb0 0x564483b27840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.947+0000 7fd6037fe700 1 -- 172.21.15.46:0/706082403 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5644839a0450 msgr2=0x564483b182e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:16.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.947+0000 7fd6037fe700 1 --2- 172.21.15.46:0/706082403 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5644839a0450 0x564483b182e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.947+0000 7fd6037fe700 1 -- 172.21.15.46:0/706082403 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564483b1e5b0 con 0x564483b1d3b0 2024-05-09T04:50:16.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.947+0000 7fd6027fc700 1 --2- 172.21.15.46:0/706082403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564483a1acb0 0x564483b27840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:50:16.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.947+0000 7fd6037fe700 1 --2- 172.21.15.46:0/706082403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483b1d3b0 0x564483b32fb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc018000 tx=0x7fd5fc003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86da64903c99177e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:16.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.947+0000 7fd5f8ff9700 1 -- 172.21.15.46:0/706082403 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc0079f0 con 0x564483b1d3b0 2024-05-09T04:50:16.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.947+0000 7fd5f8ff9700 1 -- 172.21.15.46:0/706082403 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5fc007b90 con 0x564483b1d3b0 2024-05-09T04:50:16.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.947+0000 7fd5f8ff9700 1 -- 172.21.15.46:0/706082403 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5fc005050 con 0x564483b1d3b0 2024-05-09T04:50:16.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.948+0000 7fd618046b80 1 -- 172.21.15.46:0/706082403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483b1d3b0 msgr2=0x564483b32fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:16.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.948+0000 7fd618046b80 1 --2- 172.21.15.46:0/706082403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483b1d3b0 0x564483b32fb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd5fc018000 tx=0x7fd5fc003f00 comp rx=0 tx=0).stop 2024-05-09T04:50:16.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.948+0000 7fd618046b80 1 -- 172.21.15.46:0/706082403 shutdown_connections 2024-05-09T04:50:16.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.948+0000 7fd618046b80 1 --2- 172.21.15.46:0/706082403 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483b1d3b0 0x564483b32fb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.948+0000 7fd618046b80 1 --2- 172.21.15.46:0/706082403 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5644839a0450 0x564483b182e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.948+0000 7fd618046b80 1 --2- 172.21.15.46:0/706082403 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564483a1acb0 0x564483b27840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.948+0000 7fd618046b80 1 -- 172.21.15.46:0/706082403 >> 172.21.15.46:0/706082403 conn(0x5644839a3f20 msgr2=0x564483b25890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.948+0000 7fd618046b80 1 -- 172.21.15.46:0/706082403 shutdown_connections 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.948+0000 7fd618046b80 1 -- 172.21.15.46:0/706082403 wait complete. 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.949+0000 7fd618046b80 1 -- start start 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.949+0000 7fd618046b80 1 Processor -- start 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.949+0000 7fd618046b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5644839a0450 0x564483b41910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.949+0000 7fd618046b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483a1acb0 0x564483b41e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.949+0000 7fd618046b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564483b1d3b0 0x564483b3c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.949+0000 7fd618046b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564483b38310 con 0x564483a1acb0 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.949+0000 7fd618046b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x564483b37c40 con 0x564483b1d3b0 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.949+0000 7fd618046b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564483b389e0 con 0x5644839a0450 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.949+0000 7fd618046b80 10 client.0 _ll_register_callbacks cb 0x5644838b1670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.949+0000 7fd6027fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483a1acb0 0x564483b41e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.949+0000 7fd6027fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483a1acb0 0x564483b41e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43392/0 (socket says 172.21.15.46:43392) 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.949+0000 7fd6027fc700 1 -- 172.21.15.46:0/603396657 learned_addr learned my addr 172.21.15.46:0/603396657 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:16.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.949+0000 7fd602ffd700 1 --2- 172.21.15.46:0/603396657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5644839a0450 0x564483b41910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.949+0000 7fd6037fe700 1 --2- 172.21.15.46:0/603396657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564483b1d3b0 0x564483b3c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.950+0000 7fd6027fc700 1 -- 172.21.15.46:0/603396657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5644839a0450 msgr2=0x564483b41910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.950+0000 7fd6027fc700 1 --2- 172.21.15.46:0/603396657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5644839a0450 0x564483b41910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.950+0000 7fd6027fc700 1 -- 172.21.15.46:0/603396657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564483b1d3b0 msgr2=0x564483b3c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.950+0000 7fd6027fc700 1 --2- 172.21.15.46:0/603396657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564483b1d3b0 0x564483b3c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.950+0000 7fd6027fc700 1 -- 172.21.15.46:0/603396657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564483b19e90 con 0x564483a1acb0 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.950+0000 7fd6027fc700 1 --2- 172.21.15.46:0/603396657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483a1acb0 0x564483b41e50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00ba80 tx=0x7fd5ec004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e204f007dc4daab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.950+0000 7fd5ea7fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.950+0000 7fd5ea7fc700 1 -- 172.21.15.46:0/603396657 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5ec036050 con 0x564483a1acb0 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.950+0000 7fd5ea7fc700 1 -- 172.21.15.46:0/603396657 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fd5ec004d70 con 0x564483a1acb0 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.950+0000 7fd618046b80 1 -- 172.21.15.46:0/603396657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd5fc007b90 con 0x564483a1acb0 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.950+0000 7fd5ea7fc700 1 -- 172.21.15.46:0/603396657 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fd5ec004f70 con 0x564483a1acb0 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.950+0000 7fd618046b80 10 client.15012 fetch_fsmap learned FSMap version 24 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.950+0000 7fd618046b80 1 -- 172.21.15.46:0/603396657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x564483b14980 con 0x564483a1acb0 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.950+0000 7fd5ea7fc700 1 -- 172.21.15.46:0/603396657 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5fc007b90 con 0x564483a1acb0 2024-05-09T04:50:16.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.951+0000 7fd5ea7fc700 1 -- 172.21.15.46:0/603396657 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fd5ec03b8f0 con 0x564483a1acb0 2024-05-09T04:50:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.951+0000 7fd618046b80 10 client.15012 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T04:50:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.951+0000 7fd618046b80 10 client.15012 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T04:50:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.951+0000 7fd618046b80 20 client.15012 populate_metadata read hostname 'smithi046' 2024-05-09T04:50:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.951+0000 7fd618046b80 1 --2- 172.21.15.46:0/603396657 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564483b37020 0x564483ba6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.951+0000 7fd618046b80 4 client.15012 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:50:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.951+0000 7fd618046b80 1 -- 172.21.15.46:0/603396657 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "status"}) -- 0x564483a47ad0 con 0x564483b37020 2024-05-09T04:50:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.951+0000 7fd602ffd700 1 --2- 172.21.15.46:0/603396657 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564483b37020 0x564483ba6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.951+0000 7fd602ffd700 1 --2- 172.21.15.46:0/603396657 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564483b37020 0x564483ba6370 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f15bf467d2cbc31 server_cookie=c6877c3337bb5bdf in_seq=0 out_seq=0 2024-05-09T04:50:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.951+0000 7fd5ea7fc700 10 client.15012 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:50:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.952+0000 7fd5ea7fc700 1 -- 172.21.15.46:0/603396657 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564483a47ad0 con 0x564483b37020 2024-05-09T04:50:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.952+0000 7fd5ea7fc700 10 client.15012 handle_command_reply: tid=0 2024-05-09T04:50:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.952+0000 7fd618046b80 1 client.15012 shutdown 2024-05-09T04:50:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.952+0000 7fd618046b80 1 -- 172.21.15.46:0/603396657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483a1acb0 msgr2=0x564483b41e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.952+0000 7fd618046b80 1 --2- 172.21.15.46:0/603396657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483a1acb0 0x564483b41e50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd5ec00ba80 tx=0x7fd5ec004690 comp rx=0 tx=0).stop 2024-05-09T04:50:16.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.952+0000 7fd618046b80 1 -- 172.21.15.46:0/603396657 shutdown_connections 2024-05-09T04:50:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.952+0000 7fd618046b80 1 --2- 172.21.15.46:0/603396657 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564483b37020 0x564483ba6370 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.952+0000 7fd5ea7fc700 0 client.15012 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:50:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.952+0000 7fd618046b80 1 --2- 172.21.15.46:0/603396657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564483b1d3b0 0x564483b3c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.952+0000 7fd618046b80 1 --2- 172.21.15.46:0/603396657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5644839a0450 0x564483b41910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.952+0000 7fd618046b80 1 --2- 172.21.15.46:0/603396657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564483a1acb0 0x564483b41e50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.952+0000 7fd618046b80 1 -- 172.21.15.46:0/603396657 >> 172.21.15.46:0/603396657 conn(0x5644839a3f20 msgr2=0x564483b22800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.952+0000 7fd618046b80 1 -- 172.21.15.46:0/603396657 shutdown_connections 2024-05-09T04:50:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.952+0000 7fd618046b80 1 -- 172.21.15.46:0/603396657 wait complete. 2024-05-09T04:50:16.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.952+0000 7fd618046b80 20 client.15012 trim_cache size 0 max 16384 2024-05-09T04:50:16.958 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:50:16.958 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ecf9ae2c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:50:16.958 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-05-09T04:50:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24503, 2024-05-09T04:50:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-09T04:50:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-09T04:50:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-09T04:50:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 146.54786889799999, 2024-05-09T04:50:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 24, 2024-05-09T04:50:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 84, 2024-05-09T04:50:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 80, 2024-05-09T04:50:16.959 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 161.036088423 2024-05-09T04:50:16.959 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:50:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.953+0000 7fd609630700 1 -- 172.21.15.46:0/630449414 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd5f00b2880 msgr2=0x7fd5f00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.953+0000 7fd609630700 1 --2- 172.21.15.46:0/630449414 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd5f00b2880 0x7fd5f00b4d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd604103520 tx=0x7fd5ec00b410 comp rx=0 tx=0).stop 2024-05-09T04:50:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.953+0000 7fd609630700 1 -- 172.21.15.46:0/630449414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6041045d0 msgr2=0x7fd604076bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.953+0000 7fd609630700 1 --2- 172.21.15.46:0/630449414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6041045d0 0x7fd604076bd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fd5f400b580 tx=0x7fd5f40047b0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.953+0000 7fd609630700 1 -- 172.21.15.46:0/630449414 shutdown_connections 2024-05-09T04:50:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.953+0000 7fd609630700 1 --2- 172.21.15.46:0/630449414 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fd5f00b2880 0x7fd5f00b4d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.953+0000 7fd609630700 1 --2- 172.21.15.46:0/630449414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd604109850 0x7fd604077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.953+0000 7fd609630700 1 --2- 172.21.15.46:0/630449414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd6041045d0 0x7fd604076bd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.953+0000 7fd609630700 1 --2- 172.21.15.46:0/630449414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd604108f00 0x7fd604077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.953+0000 7fd609630700 1 -- 172.21.15.46:0/630449414 >> 172.21.15.46:0/630449414 conn(0x7fd6040ffea0 msgr2=0x7fd60410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.953+0000 7fd609630700 1 -- 172.21.15.46:0/630449414 shutdown_connections 2024-05-09T04:50:16.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:16.953+0000 7fd609630700 1 -- 172.21.15.46:0/630449414 wait complete. 2024-05-09T04:50:17.749 INFO:tasks.cephadm:Setup complete, yielding 2024-05-09T04:50:17.750 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T04:50:17.760 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T04:50:17.760 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-09T04:50:18.015 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:18.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:18 smithi110 ceph-mon[34904]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:50:18.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:18 smithi046 ceph-mon[30136]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:50:18.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:18 smithi072 ceph-mon[34088]: pgmap v369: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:50:19.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.739+0000 7f3b4c1d7700 1 -- 172.21.15.46:0/3857068168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b4410a000 msgr2=0x7f3b4410c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.739+0000 7f3b4c1d7700 1 --2- 172.21.15.46:0/3857068168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b4410a000 0x7f3b4410c510 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f3b3c009990 tx=0x7f3b3c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:50:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.739+0000 7f3b4c1d7700 1 -- 172.21.15.46:0/3857068168 shutdown_connections 2024-05-09T04:50:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.739+0000 7f3b4c1d7700 1 --2- 172.21.15.46:0/3857068168 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b4410a000 0x7f3b4410c510 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.739+0000 7f3b4c1d7700 1 --2- 172.21.15.46:0/3857068168 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b440766b0 0x7f3b44106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.739+0000 7f3b4c1d7700 1 --2- 172.21.15.46:0/3857068168 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b44107490 0x7f3b44109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.739+0000 7f3b4c1d7700 1 -- 172.21.15.46:0/3857068168 >> 172.21.15.46:0/3857068168 conn(0x7f3b440fe890 msgr2=0x7f3b44100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.739+0000 7f3b4c1d7700 1 -- 172.21.15.46:0/3857068168 shutdown_connections 2024-05-09T04:50:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.739+0000 7f3b4c1d7700 1 -- 172.21.15.46:0/3857068168 wait complete. 2024-05-09T04:50:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.739+0000 7f3b4c1d7700 1 Processor -- start 2024-05-09T04:50:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.739+0000 7f3b4c1d7700 1 -- start start 2024-05-09T04:50:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b4c1d7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b440766b0 0x7f3b4419a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b4c1d7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b44107490 0x7f3b4419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:19.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b4c1d7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b4410a000 0x7f3b441a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b4c1d7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4410fdf0 con 0x7f3b44107490 2024-05-09T04:50:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b4c1d7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4410fc70 con 0x7f3b440766b0 2024-05-09T04:50:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b4c1d7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3b4410ff70 con 0x7f3b4410a000 2024-05-09T04:50:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b49772700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b44107490 0x7f3b4419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b49772700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b44107490 0x7f3b4419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43414/0 (socket says 172.21.15.46:43414) 2024-05-09T04:50:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b49772700 1 -- 172.21.15.46:0/1642833522 learned_addr learned my addr 172.21.15.46:0/1642833522 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b49f73700 1 --2- 172.21.15.46:0/1642833522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b440766b0 0x7f3b4419a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b4a774700 1 --2- 172.21.15.46:0/1642833522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b4410a000 0x7f3b441a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b49772700 1 -- 172.21.15.46:0/1642833522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b4410a000 msgr2=0x7f3b441a1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b49772700 1 --2- 172.21.15.46:0/1642833522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b4410a000 0x7f3b441a1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b49772700 1 -- 172.21.15.46:0/1642833522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b440766b0 msgr2=0x7f3b4419a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:19.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b49772700 1 --2- 172.21.15.46:0/1642833522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b440766b0 0x7f3b4419a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b49772700 1 -- 172.21.15.46:0/1642833522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3b441a2440 con 0x7f3b44107490 2024-05-09T04:50:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b4a774700 1 --2- 172.21.15.46:0/1642833522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b4410a000 0x7f3b441a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:50:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.740+0000 7f3b49772700 1 --2- 172.21.15.46:0/1642833522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b44107490 0x7f3b4419ae20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800d950 tx=0x7f3b3800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.741+0000 7f3b42ffd700 1 -- 172.21.15.46:0/1642833522 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b38016070 con 0x7f3b44107490 2024-05-09T04:50:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.741+0000 7f3b4c1d7700 1 -- 172.21.15.46:0/1642833522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3b441a2730 con 0x7f3b44107490 2024-05-09T04:50:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.741+0000 7f3b42ffd700 1 -- 172.21.15.46:0/1642833522 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3b38009df0 con 0x7f3b44107490 2024-05-09T04:50:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.741+0000 7f3b42ffd700 1 -- 172.21.15.46:0/1642833522 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b3800f3c0 con 0x7f3b44107490 2024-05-09T04:50:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.741+0000 7f3b4c1d7700 1 -- 172.21.15.46:0/1642833522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3b441a2d40 con 0x7f3b44107490 2024-05-09T04:50:19.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.742+0000 7f3b4c1d7700 1 -- 172.21.15.46:0/1642833522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3b440512c0 con 0x7f3b44107490 2024-05-09T04:50:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.743+0000 7f3b42ffd700 1 -- 172.21.15.46:0/1642833522 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3b38019070 con 0x7f3b44107490 2024-05-09T04:50:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.744+0000 7f3b42ffd700 1 --2- 172.21.15.46:0/1642833522 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b2c0b2880 0x7f3b2c0b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.745+0000 7f3b49f73700 1 --2- 172.21.15.46:0/1642833522 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b2c0b2880 0x7f3b2c0b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.745+0000 7f3b42ffd700 1 -- 172.21.15.46:0/1642833522 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f3b380e1e40 con 0x7f3b44107490 2024-05-09T04:50:19.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.745+0000 7f3b49f73700 1 --2- 172.21.15.46:0/1642833522 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b2c0b2880 0x7f3b2c0b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400a400 tx=0x7f3b3400d040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:19.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.748+0000 7f3b42ffd700 1 -- 172.21.15.46:0/1642833522 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3b380a9b20 con 0x7f3b44107490 2024-05-09T04:50:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.984+0000 7f3b4c1d7700 1 -- 172.21.15.46:0/1642833522 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f3b44063c00 con 0x7f3b2c0b2880 2024-05-09T04:50:19.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.985+0000 7f3b42ffd700 1 -- 172.21.15.46:0/1642833522 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f3b44063c00 con 0x7f3b2c0b2880 2024-05-09T04:50:19.986 INFO:teuthology.orchestra.run.smithi046.stdout:Backend: cephadm 2024-05-09T04:50:19.986 INFO:teuthology.orchestra.run.smithi046.stdout:Available: Yes 2024-05-09T04:50:19.986 INFO:teuthology.orchestra.run.smithi046.stdout:Paused: No 2024-05-09T04:50:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.987+0000 7f3b4c1d7700 1 -- 172.21.15.46:0/1642833522 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b2c0b2880 msgr2=0x7f3b2c0b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.987+0000 7f3b4c1d7700 1 --2- 172.21.15.46:0/1642833522 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b2c0b2880 0x7f3b2c0b4d90 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f3b3400a400 tx=0x7f3b3400d040 comp rx=0 tx=0).stop 2024-05-09T04:50:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.987+0000 7f3b4c1d7700 1 -- 172.21.15.46:0/1642833522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b44107490 msgr2=0x7f3b4419ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.987+0000 7f3b4c1d7700 1 --2- 172.21.15.46:0/1642833522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b44107490 0x7f3b4419ae20 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3b3800d950 tx=0x7f3b3800dc60 comp rx=0 tx=0).stop 2024-05-09T04:50:19.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.987+0000 7f3b4c1d7700 1 -- 172.21.15.46:0/1642833522 shutdown_connections 2024-05-09T04:50:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.987+0000 7f3b4c1d7700 1 --2- 172.21.15.46:0/1642833522 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3b2c0b2880 0x7f3b2c0b4d90 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.987+0000 7f3b4c1d7700 1 --2- 172.21.15.46:0/1642833522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3b4410a000 0x7f3b441a1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.987+0000 7f3b4c1d7700 1 --2- 172.21.15.46:0/1642833522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3b440766b0 0x7f3b4419a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.987+0000 7f3b4c1d7700 1 --2- 172.21.15.46:0/1642833522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3b44107490 0x7f3b4419ae20 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.987+0000 7f3b4c1d7700 1 -- 172.21.15.46:0/1642833522 >> 172.21.15.46:0/1642833522 conn(0x7f3b440fe890 msgr2=0x7f3b44105550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.987+0000 7f3b4c1d7700 1 -- 172.21.15.46:0/1642833522 shutdown_connections 2024-05-09T04:50:19.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:19.987+0000 7f3b4c1d7700 1 -- 172.21.15.46:0/1642833522 wait complete. 2024-05-09T04:50:20.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:20 smithi046 ceph-mon[30136]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:50:20.663 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-09T04:50:20.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:20 smithi110 ceph-mon[34904]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:50:20.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:20 smithi072 ceph-mon[34088]: pgmap v370: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:50:20.919 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:21.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:21 smithi046 ceph-mon[30136]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:50:21.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:21 smithi110 ceph-mon[34904]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:50:21.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:21 smithi072 ceph-mon[34088]: from='client.15018 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:50:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.681+0000 7fe81cb56700 1 -- 172.21.15.46:0/2911967590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe818104370 msgr2=0x7fe818104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.681+0000 7fe81cb56700 1 --2- 172.21.15.46:0/2911967590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe818104370 0x7fe818104780 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe80c009990 tx=0x7fe80c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.681+0000 7fe81cb56700 1 -- 172.21.15.46:0/2911967590 shutdown_connections 2024-05-09T04:50:22.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.681+0000 7fe81cb56700 1 --2- 172.21.15.46:0/2911967590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe818104d10 0x7fe81810b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.681+0000 7fe81cb56700 1 --2- 172.21.15.46:0/2911967590 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe818103240 0x7fe818103650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.681+0000 7fe81cb56700 1 --2- 172.21.15.46:0/2911967590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe818104370 0x7fe818104780 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.681+0000 7fe81cb56700 1 -- 172.21.15.46:0/2911967590 >> 172.21.15.46:0/2911967590 conn(0x7fe8180fe880 msgr2=0x7fe818100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:22.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.681+0000 7fe81cb56700 1 -- 172.21.15.46:0/2911967590 shutdown_connections 2024-05-09T04:50:22.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.681+0000 7fe81cb56700 1 -- 172.21.15.46:0/2911967590 wait complete. 2024-05-09T04:50:22.683 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:22 smithi046 ceph-mon[30136]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:50:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.682+0000 7fe81cb56700 1 Processor -- start 2024-05-09T04:50:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.682+0000 7fe81cb56700 1 -- start start 2024-05-09T04:50:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.682+0000 7fe81cb56700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe818103240 0x7fe81819cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.682+0000 7fe81cb56700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe818104370 0x7fe81819d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.682+0000 7fe81cb56700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe818104d10 0x7fe8181a40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.682+0000 7fe81cb56700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe81810ed00 con 0x7fe818104d10 2024-05-09T04:50:22.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.682+0000 7fe81cb56700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe81810eb80 con 0x7fe818103240 2024-05-09T04:50:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.682+0000 7fe81cb56700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe81810ee80 con 0x7fe818104370 2024-05-09T04:50:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.682+0000 7fe816d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe818104d10 0x7fe8181a40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.682+0000 7fe816d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe818104d10 0x7fe8181a40a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43432/0 (socket says 172.21.15.46:43432) 2024-05-09T04:50:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.682+0000 7fe816d9d700 1 -- 172.21.15.46:0/2139281456 learned_addr learned my addr 172.21.15.46:0/2139281456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.682+0000 7fe81659c700 1 --2- 172.21.15.46:0/2139281456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe818103240 0x7fe81819cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.682+0000 7fe815d9b700 1 --2- 172.21.15.46:0/2139281456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe818104370 0x7fe81819d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.683+0000 7fe816d9d700 1 -- 172.21.15.46:0/2139281456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe818104370 msgr2=0x7fe81819d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.683+0000 7fe816d9d700 1 --2- 172.21.15.46:0/2139281456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe818104370 0x7fe81819d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.683+0000 7fe816d9d700 1 -- 172.21.15.46:0/2139281456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe818103240 msgr2=0x7fe81819cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.683+0000 7fe816d9d700 1 --2- 172.21.15.46:0/2139281456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe818103240 0x7fe81819cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.683+0000 7fe816d9d700 1 -- 172.21.15.46:0/2139281456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8181a4720 con 0x7fe818104d10 2024-05-09T04:50:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.683+0000 7fe816d9d700 1 --2- 172.21.15.46:0/2139281456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe818104d10 0x7fe8181a40a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe81000ef30 tx=0x7fe81000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:22.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.683+0000 7fe8077fe700 1 -- 172.21.15.46:0/2139281456 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8100092e0 con 0x7fe818104d10 2024-05-09T04:50:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.683+0000 7fe8077fe700 1 -- 172.21.15.46:0/2139281456 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe810004bb0 con 0x7fe818104d10 2024-05-09T04:50:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.683+0000 7fe81cb56700 1 -- 172.21.15.46:0/2139281456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8181a49b0 con 0x7fe818104d10 2024-05-09T04:50:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.683+0000 7fe8077fe700 1 -- 172.21.15.46:0/2139281456 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe810004020 con 0x7fe818104d10 2024-05-09T04:50:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.683+0000 7fe81cb56700 1 -- 172.21.15.46:0/2139281456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8181a4ea0 con 0x7fe818104d10 2024-05-09T04:50:22.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.684+0000 7fe81cb56700 1 -- 172.21.15.46:0/2139281456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8180512c0 con 0x7fe818104d10 2024-05-09T04:50:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.687+0000 7fe8077fe700 1 -- 172.21.15.46:0/2139281456 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe81001b030 con 0x7fe818104d10 2024-05-09T04:50:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.688+0000 7fe8077fe700 1 --2- 172.21.15.46:0/2139281456 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe8000b2880 0x7fe8000b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.688+0000 7fe81659c700 1 --2- 172.21.15.46:0/2139281456 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe8000b2880 0x7fe8000b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:22.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.688+0000 7fe8077fe700 1 -- 172.21.15.46:0/2139281456 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fe8100e27b0 con 0x7fe818104d10 2024-05-09T04:50:22.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.688+0000 7fe81659c700 1 --2- 172.21.15.46:0/2139281456 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe8000b2880 0x7fe8000b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe818104270 tx=0x7fe80800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:22.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.692+0000 7fe8077fe700 1 -- 172.21.15.46:0/2139281456 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe810022020 con 0x7fe818104d10 2024-05-09T04:50:22.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:22 smithi110 ceph-mon[34904]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:50:22.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:22 smithi072 ceph-mon[34088]: pgmap v371: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:50:22.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.916+0000 7fe81cb56700 1 -- 172.21.15.46:0/2139281456 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fe8181008e0 con 0x7fe8000b2880 2024-05-09T04:50:22.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.930+0000 7fe8077fe700 1 -- 172.21.15.46:0/2139281456 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fe8181008e0 con 0x7fe8000b2880 2024-05-09T04:50:22.931 INFO:teuthology.orchestra.run.smithi046.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-09T04:50:22.931 INFO:teuthology.orchestra.run.smithi046.stdout:mds.a smithi046 running (2m) 29s ago 2m 81.9M - 19.0.0-3594-gc9da4049 db794d706f49 883e51440f91 2024-05-09T04:50:22.931 INFO:teuthology.orchestra.run.smithi046.stdout:mds.b smithi072 running (2m) 30s ago 2m 93.0M - 19.0.0-3594-gc9da4049 db794d706f49 b35740bd40ea 2024-05-09T04:50:22.932 INFO:teuthology.orchestra.run.smithi046.stdout:mds.c smithi110 running (3m) 30s ago 3m 85.7M - 19.0.0-3594-gc9da4049 db794d706f49 f31847814bf1 2024-05-09T04:50:22.932 INFO:teuthology.orchestra.run.smithi046.stdout:mds.d smithi046 running (3m) 29s ago 3m 81.8M - 19.0.0-3594-gc9da4049 db794d706f49 eeecbdc5532b 2024-05-09T04:50:22.932 INFO:teuthology.orchestra.run.smithi046.stdout:mds.e smithi072 running (3m) 30s ago 3m 90.3M - 19.0.0-3594-gc9da4049 db794d706f49 169e3dba0d29 2024-05-09T04:50:22.932 INFO:teuthology.orchestra.run.smithi046.stdout:mds.f smithi110 running (3m) 30s ago 3m 89.2M - 19.0.0-3594-gc9da4049 db794d706f49 6c87c4863332 2024-05-09T04:50:22.932 INFO:teuthology.orchestra.run.smithi046.stdout:mds.g smithi046 running (2m) 29s ago 2m 81.8M - 19.0.0-3594-gc9da4049 db794d706f49 89c1baac6199 2024-05-09T04:50:22.932 INFO:teuthology.orchestra.run.smithi046.stdout:mds.h smithi072 running (3m) 30s ago 3m 87.4M - 19.0.0-3594-gc9da4049 db794d706f49 9e50b6586128 2024-05-09T04:50:22.932 INFO:teuthology.orchestra.run.smithi046.stdout:mds.i smithi110 running (2m) 30s ago 2m 88.7M - 19.0.0-3594-gc9da4049 db794d706f49 d7005ada0039 2024-05-09T04:50:22.932 INFO:teuthology.orchestra.run.smithi046.stdout:mds.j smithi046 running (3m) 29s ago 3m 81.9M - 19.0.0-3594-gc9da4049 db794d706f49 03a8ee9e792c 2024-05-09T04:50:22.932 INFO:teuthology.orchestra.run.smithi046.stdout:mds.k smithi072 running (3m) 30s ago 3m 82.3M - 19.0.0-3594-gc9da4049 db794d706f49 83a0c56c6f0a 2024-05-09T04:50:22.932 INFO:teuthology.orchestra.run.smithi046.stdout:mds.l smithi110 running (3m) 30s ago 3m 81.8M - 19.0.0-3594-gc9da4049 db794d706f49 7f7cc7082be4 2024-05-09T04:50:22.932 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.x smithi046 *:9283,8765 running (10m) 29s ago 10m 598M - 19.0.0-3594-gc9da4049 db794d706f49 a3cb336f7930 2024-05-09T04:50:22.932 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.y smithi072 *:8443,8765 running (8m) 30s ago 8m 535M - 19.0.0-3594-gc9da4049 db794d706f49 feb402c9a467 2024-05-09T04:50:22.933 INFO:teuthology.orchestra.run.smithi046.stdout:mgr.z smithi110 *:8443,8765 running (8m) 30s ago 8m 533M - 19.0.0-3594-gc9da4049 db794d706f49 e9f541a521cf 2024-05-09T04:50:22.933 INFO:teuthology.orchestra.run.smithi046.stdout:mon.a smithi046 running (10m) 29s ago 10m 118M 2048M 19.0.0-3594-gc9da4049 db794d706f49 0809ad9b5ec4 2024-05-09T04:50:22.933 INFO:teuthology.orchestra.run.smithi046.stdout:mon.b smithi072 running (9m) 30s ago 9m 106M 2048M 19.0.0-3594-gc9da4049 db794d706f49 bb655ac8ed64 2024-05-09T04:50:22.933 INFO:teuthology.orchestra.run.smithi046.stdout:mon.c smithi110 running (9m) 30s ago 9m 107M 2048M 19.0.0-3594-gc9da4049 db794d706f49 f5239ba2c3bb 2024-05-09T04:50:22.933 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 smithi046 running (8m) 29s ago 8m 115M 4273M 19.0.0-3594-gc9da4049 db794d706f49 de012b99267c 2024-05-09T04:50:22.933 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 smithi072 running (8m) 30s ago 8m 124M 4272M 19.0.0-3594-gc9da4049 db794d706f49 1e8bf7d9484a 2024-05-09T04:50:22.933 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 smithi110 running (7m) 30s ago 7m 112M 4272M 19.0.0-3594-gc9da4049 db794d706f49 34b2bea055de 2024-05-09T04:50:22.933 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 smithi046 running (7m) 29s ago 7m 122M 4273M 19.0.0-3594-gc9da4049 db794d706f49 96fd5a286dd6 2024-05-09T04:50:22.933 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 smithi072 running (6m) 30s ago 6m 111M 4272M 19.0.0-3594-gc9da4049 db794d706f49 5e6569d86fd3 2024-05-09T04:50:22.933 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 smithi110 running (6m) 30s ago 6m 121M 4272M 19.0.0-3594-gc9da4049 db794d706f49 b7022f42db85 2024-05-09T04:50:22.933 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 smithi046 running (5m) 29s ago 5m 111M 4273M 19.0.0-3594-gc9da4049 db794d706f49 25b84c76eb26 2024-05-09T04:50:22.933 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 smithi072 running (5m) 30s ago 5m 111M 4272M 19.0.0-3594-gc9da4049 db794d706f49 35164b5d7257 2024-05-09T04:50:22.934 INFO:teuthology.orchestra.run.smithi046.stdout:osd.8 smithi110 running (5m) 30s ago 5m 113M 4272M 19.0.0-3594-gc9da4049 db794d706f49 490af2b2aeb1 2024-05-09T04:50:22.934 INFO:teuthology.orchestra.run.smithi046.stdout:osd.9 smithi046 running (4m) 29s ago 4m 112M 4273M 19.0.0-3594-gc9da4049 db794d706f49 04f7fd9bf78f 2024-05-09T04:50:22.934 INFO:teuthology.orchestra.run.smithi046.stdout:osd.10 smithi072 running (4m) 30s ago 4m 111M 4272M 19.0.0-3594-gc9da4049 db794d706f49 b6a877dba76d 2024-05-09T04:50:22.934 INFO:teuthology.orchestra.run.smithi046.stdout:osd.11 smithi110 running (3m) 30s ago 3m 119M 4272M 19.0.0-3594-gc9da4049 db794d706f49 e32e5e7e98f4 2024-05-09T04:50:22.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.933+0000 7fe81cb56700 1 -- 172.21.15.46:0/2139281456 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe8000b2880 msgr2=0x7fe8000b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:22.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.933+0000 7fe81cb56700 1 --2- 172.21.15.46:0/2139281456 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe8000b2880 0x7fe8000b4d90 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe818104270 tx=0x7fe80800b410 comp rx=0 tx=0).stop 2024-05-09T04:50:22.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.933+0000 7fe81cb56700 1 -- 172.21.15.46:0/2139281456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe818104d10 msgr2=0x7fe8181a40a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:22.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.933+0000 7fe81cb56700 1 --2- 172.21.15.46:0/2139281456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe818104d10 0x7fe8181a40a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe81000ef30 tx=0x7fe81000c3a0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.933+0000 7fe81cb56700 1 -- 172.21.15.46:0/2139281456 shutdown_connections 2024-05-09T04:50:22.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.933+0000 7fe81cb56700 1 --2- 172.21.15.46:0/2139281456 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fe8000b2880 0x7fe8000b4d90 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.933+0000 7fe81cb56700 1 --2- 172.21.15.46:0/2139281456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe818104d10 0x7fe8181a40a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.933+0000 7fe81cb56700 1 --2- 172.21.15.46:0/2139281456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe818103240 0x7fe81819cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.933+0000 7fe81cb56700 1 --2- 172.21.15.46:0/2139281456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe818104370 0x7fe81819d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:22.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.933+0000 7fe81cb56700 1 -- 172.21.15.46:0/2139281456 >> 172.21.15.46:0/2139281456 conn(0x7fe8180fe880 msgr2=0x7fe8181001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:22.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.933+0000 7fe81cb56700 1 -- 172.21.15.46:0/2139281456 shutdown_connections 2024-05-09T04:50:22.935 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:22.933+0000 7fe81cb56700 1 -- 172.21.15.46:0/2139281456 wait complete. 2024-05-09T04:50:23.563 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-09T04:50:23.813 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:24.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:24 smithi046 ceph-mon[30136]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:50:24.776 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:24 smithi046 ceph-mon[30136]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:50:24.806 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:24 smithi110 ceph-mon[34904]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:50:24.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:24 smithi110 ceph-mon[34904]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:50:24.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:24 smithi072 ceph-mon[34088]: from='client.15024 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:50:24.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:24 smithi072 ceph-mon[34088]: pgmap v372: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:50:25.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.768+0000 7f5a00cce700 1 -- 172.21.15.46:0/406609791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59fc104d20 msgr2=0x7f59fc10b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:25.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.768+0000 7f5a00cce700 1 --2- 172.21.15.46:0/406609791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59fc104d20 0x7f59fc10b5d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f59f400b0a0 tx=0x7f59f400b3b0 comp rx=0 tx=0).stop 2024-05-09T04:50:25.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.768+0000 7f5a00cce700 1 -- 172.21.15.46:0/406609791 shutdown_connections 2024-05-09T04:50:25.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.768+0000 7f5a00cce700 1 --2- 172.21.15.46:0/406609791 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59fc104d20 0x7f59fc10b5d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:25.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.768+0000 7f5a00cce700 1 --2- 172.21.15.46:0/406609791 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc103250 0x7f59fc103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:25.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.768+0000 7f5a00cce700 1 --2- 172.21.15.46:0/406609791 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59fc104380 0x7f59fc104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:25.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.768+0000 7f5a00cce700 1 -- 172.21.15.46:0/406609791 >> 172.21.15.46:0/406609791 conn(0x7f59fc0fe890 msgr2=0x7f59fc100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:25.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.768+0000 7f5a00cce700 1 -- 172.21.15.46:0/406609791 shutdown_connections 2024-05-09T04:50:25.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.768+0000 7f5a00cce700 1 -- 172.21.15.46:0/406609791 wait complete. 2024-05-09T04:50:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.769+0000 7f5a00cce700 1 Processor -- start 2024-05-09T04:50:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.769+0000 7f5a00cce700 1 -- start start 2024-05-09T04:50:25.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.769+0000 7f5a00cce700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59fc103250 0x7f59fc19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.769+0000 7f5a00cce700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59fc104380 0x7f59fc19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.769+0000 7f5a00cce700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc104d20 0x7f59fc1a1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.769+0000 7f5a00cce700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc10ecc0 con 0x7f59fc103250 2024-05-09T04:50:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.769+0000 7f5a00cce700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc10eb40 con 0x7f59fc104380 2024-05-09T04:50:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.769+0000 7f5a00cce700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f59fc10ee40 con 0x7f59fc104d20 2024-05-09T04:50:25.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f59fa59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59fc103250 0x7f59fc19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f59fa59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59fc103250 0x7f59fc19a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50966/0 (socket says 172.21.15.46:50966) 2024-05-09T04:50:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f59fa59c700 1 -- 172.21.15.46:0/721043895 learned_addr learned my addr 172.21.15.46:0/721043895 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f59fad9d700 1 --2- 172.21.15.46:0/721043895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc104d20 0x7f59fc1a1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f59f9d9b700 1 --2- 172.21.15.46:0/721043895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59fc104380 0x7f59fc19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f59fa59c700 1 -- 172.21.15.46:0/721043895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc104d20 msgr2=0x7f59fc1a1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:25.775 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f59fa59c700 1 --2- 172.21.15.46:0/721043895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc104d20 0x7f59fc1a1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f59fa59c700 1 -- 172.21.15.46:0/721043895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59fc104380 msgr2=0x7f59fc19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f59fa59c700 1 --2- 172.21.15.46:0/721043895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59fc104380 0x7f59fc19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f59fa59c700 1 -- 172.21.15.46:0/721043895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59fc1a2330 con 0x7f59fc103250 2024-05-09T04:50:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f59f9d9b700 1 --2- 172.21.15.46:0/721043895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59fc104380 0x7f59fc19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:50:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f59fa59c700 1 --2- 172.21.15.46:0/721043895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59fc103250 0x7f59fc19a860 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00c970 tx=0x7f59ec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:25.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f59eb7fe700 1 -- 172.21.15.46:0/721043895 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec007b00 con 0x7f59fc103250 2024-05-09T04:50:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f5a00cce700 1 -- 172.21.15.46:0/721043895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59fc1a2620 con 0x7f59fc103250 2024-05-09T04:50:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f59eb7fe700 1 -- 172.21.15.46:0/721043895 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59ec007ca0 con 0x7f59fc103250 2024-05-09T04:50:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f59eb7fe700 1 -- 172.21.15.46:0/721043895 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59ec018660 con 0x7f59fc103250 2024-05-09T04:50:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.770+0000 7f5a00cce700 1 -- 172.21.15.46:0/721043895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59fc1a2bb0 con 0x7f59fc103250 2024-05-09T04:50:25.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.772+0000 7f5a00cce700 1 -- 172.21.15.46:0/721043895 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59fc0512c0 con 0x7f59fc103250 2024-05-09T04:50:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.773+0000 7f59eb7fe700 1 -- 172.21.15.46:0/721043895 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f59ec00f450 con 0x7f59fc103250 2024-05-09T04:50:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.774+0000 7f59eb7fe700 1 --2- 172.21.15.46:0/721043895 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f59e40b2900 0x7f59e40b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.774+0000 7f59f9d9b700 1 --2- 172.21.15.46:0/721043895 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f59e40b2900 0x7f59e40b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.774+0000 7f59eb7fe700 1 -- 172.21.15.46:0/721043895 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f59ec0e2100 con 0x7f59fc103250 2024-05-09T04:50:25.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.774+0000 7f59f9d9b700 1 --2- 172.21.15.46:0/721043895 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f59e40b2900 0x7f59e40b4e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f59fc19b910 tx=0x7f59f000a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:25.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:25.778+0000 7f59eb7fe700 1 -- 172.21.15.46:0/721043895 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f59ec014030 con 0x7f59fc103250 2024-05-09T04:50:26.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:26.002+0000 7f5a00cce700 1 -- 172.21.15.46:0/721043895 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f59fc1a2e70 con 0x7f59e40b2900 2024-05-09T04:50:26.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:26.009+0000 7f59eb7fe700 1 -- 172.21.15.46:0/721043895 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f59fc1a2e70 con 0x7f59e40b2900 2024-05-09T04:50:26.011 INFO:teuthology.orchestra.run.smithi046.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-09T04:50:26.011 INFO:teuthology.orchestra.run.smithi046.stdout:mds.all 12/12 34s ago 3m smithi046=a;smithi046=d;smithi046=g;smithi046=j;smithi072=b;smithi072=e;smithi072=h;smithi072=k;smithi110=c;smithi110=f;smithi110=i;smithi110=l;count:12 2024-05-09T04:50:26.011 INFO:teuthology.orchestra.run.smithi046.stdout:mgr 3/3 34s ago 8m smithi046=x;smithi072=y;smithi110=z;count:3 2024-05-09T04:50:26.011 INFO:teuthology.orchestra.run.smithi046.stdout:mon 3/3 34s ago 9m smithi046:172.21.15.46=a;smithi072:172.21.15.72=b;smithi110:172.21.15.110=c;count:3 2024-05-09T04:50:26.011 INFO:teuthology.orchestra.run.smithi046.stdout:osd 12 34s ago - 2024-05-09T04:50:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:26.012+0000 7f5a00cce700 1 -- 172.21.15.46:0/721043895 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f59e40b2900 msgr2=0x7f59e40b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:26.012+0000 7f5a00cce700 1 --2- 172.21.15.46:0/721043895 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f59e40b2900 0x7f59e40b4e10 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f59fc19b910 tx=0x7f59f000a680 comp rx=0 tx=0).stop 2024-05-09T04:50:26.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:26.012+0000 7f5a00cce700 1 -- 172.21.15.46:0/721043895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59fc103250 msgr2=0x7f59fc19a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:26.012+0000 7f5a00cce700 1 --2- 172.21.15.46:0/721043895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59fc103250 0x7f59fc19a860 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f59ec00c970 tx=0x7f59ec00cc80 comp rx=0 tx=0).stop 2024-05-09T04:50:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:26.012+0000 7f5a00cce700 1 -- 172.21.15.46:0/721043895 shutdown_connections 2024-05-09T04:50:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:26.012+0000 7f5a00cce700 1 --2- 172.21.15.46:0/721043895 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f59e40b2900 0x7f59e40b4e10 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:26.012+0000 7f5a00cce700 1 --2- 172.21.15.46:0/721043895 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f59fc104d20 0x7f59fc1a1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:26.012+0000 7f5a00cce700 1 --2- 172.21.15.46:0/721043895 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f59fc103250 0x7f59fc19a860 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:26.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:26.012+0000 7f5a00cce700 1 --2- 172.21.15.46:0/721043895 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f59fc104380 0x7f59fc19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:26.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:26.012+0000 7f5a00cce700 1 -- 172.21.15.46:0/721043895 >> 172.21.15.46:0/721043895 conn(0x7f59fc0fe890 msgr2=0x7f59fc1000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:26.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:26.012+0000 7f5a00cce700 1 -- 172.21.15.46:0/721043895 shutdown_connections 2024-05-09T04:50:26.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:26.012+0000 7f5a00cce700 1 -- 172.21.15.46:0/721043895 wait complete. 2024-05-09T04:50:26.761 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:26 smithi046 ceph-mon[30136]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:50:26.792 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-09T04:50:26.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:26 smithi110 ceph-mon[34904]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:50:26.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:26 smithi072 ceph-mon[34088]: pgmap v373: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:50:27.045 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:27.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:27 smithi110 ceph-mon[34904]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:50:27.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:27 smithi046 ceph-mon[30136]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:50:27.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:27 smithi072 ceph-mon[34088]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:50:28.679 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:28 smithi046 ceph-mon[30136]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:50:28.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:28 smithi110 ceph-mon[34904]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:50:28.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:28 smithi072 ceph-mon[34088]: pgmap v374: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:50:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.934+0000 7f197c29d700 1 -- 172.21.15.46:0/1497163700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1974103250 msgr2=0x7f1974103660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.934+0000 7f197c29d700 1 --2- 172.21.15.46:0/1497163700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1974103250 0x7f1974103660 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f197000b0a0 tx=0x7f197000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:50:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.935+0000 7f197c29d700 1 -- 172.21.15.46:0/1497163700 shutdown_connections 2024-05-09T04:50:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.935+0000 7f197c29d700 1 --2- 172.21.15.46:0/1497163700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1974104d20 0x7f197410b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.935+0000 7f197c29d700 1 --2- 172.21.15.46:0/1497163700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1974103250 0x7f1974103660 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.935+0000 7f197c29d700 1 --2- 172.21.15.46:0/1497163700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1974104380 0x7f1974104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.935+0000 7f197c29d700 1 -- 172.21.15.46:0/1497163700 >> 172.21.15.46:0/1497163700 conn(0x7f19740fe890 msgr2=0x7f1974100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.935+0000 7f197c29d700 1 -- 172.21.15.46:0/1497163700 shutdown_connections 2024-05-09T04:50:28.936 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.935+0000 7f197c29d700 1 -- 172.21.15.46:0/1497163700 wait complete. 2024-05-09T04:50:28.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.936+0000 7f197c29d700 1 Processor -- start 2024-05-09T04:50:28.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.936+0000 7f197c29d700 1 -- start start 2024-05-09T04:50:28.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.936+0000 7f197c29d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1974103250 0x7f1974199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:28.938 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.936+0000 7f197c29d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1974104380 0x7f1974199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.936+0000 7f197c29d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1974104d20 0x7f19741a0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.936+0000 7f197c29d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f197410ec90 con 0x7f1974104380 2024-05-09T04:50:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.936+0000 7f197c29d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f197410eb10 con 0x7f1974103250 2024-05-09T04:50:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.936+0000 7f197c29d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f197410ee10 con 0x7f1974104d20 2024-05-09T04:50:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.936+0000 7f1979838700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1974104380 0x7f1974199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.936+0000 7f1979838700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1974104380 0x7f1974199d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50992/0 (socket says 172.21.15.46:50992) 2024-05-09T04:50:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.936+0000 7f1979838700 1 -- 172.21.15.46:0/727424785 learned_addr learned my addr 172.21.15.46:0/727424785 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.936+0000 7f197a83a700 1 --2- 172.21.15.46:0/727424785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1974104d20 0x7f19741a0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.936+0000 7f197a039700 1 --2- 172.21.15.46:0/727424785 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1974103250 0x7f1974199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.937+0000 7f1979838700 1 -- 172.21.15.46:0/727424785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1974104d20 msgr2=0x7f19741a0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.937+0000 7f1979838700 1 --2- 172.21.15.46:0/727424785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1974104d20 0x7f19741a0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.937+0000 7f1979838700 1 -- 172.21.15.46:0/727424785 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1974103250 msgr2=0x7f1974199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.937+0000 7f1979838700 1 --2- 172.21.15.46:0/727424785 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1974103250 0x7f1974199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.937+0000 7f1979838700 1 -- 172.21.15.46:0/727424785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19741a13a0 con 0x7f1974104380 2024-05-09T04:50:28.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.937+0000 7f1979838700 1 --2- 172.21.15.46:0/727424785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1974104380 0x7f1974199d80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f196400e9a0 tx=0x7f196400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.937+0000 7f196b7fe700 1 -- 172.21.15.46:0/727424785 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196400ca10 con 0x7f1974104380 2024-05-09T04:50:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.937+0000 7f197c29d700 1 -- 172.21.15.46:0/727424785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19741a1690 con 0x7f1974104380 2024-05-09T04:50:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.937+0000 7f196b7fe700 1 -- 172.21.15.46:0/727424785 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f196400cbb0 con 0x7f1974104380 2024-05-09T04:50:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.937+0000 7f196b7fe700 1 -- 172.21.15.46:0/727424785 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1964018680 con 0x7f1974104380 2024-05-09T04:50:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.937+0000 7f197c29d700 1 -- 172.21.15.46:0/727424785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19741a1bd0 con 0x7f1974104380 2024-05-09T04:50:28.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.938+0000 7f197c29d700 1 -- 172.21.15.46:0/727424785 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19740512c0 con 0x7f1974104380 2024-05-09T04:50:28.941 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.941+0000 7f196b7fe700 1 -- 172.21.15.46:0/727424785 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1964010430 con 0x7f1974104380 2024-05-09T04:50:28.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.941+0000 7f196b7fe700 1 --2- 172.21.15.46:0/727424785 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f19600b2880 0x7f19600b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:28.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.942+0000 7f197a039700 1 --2- 172.21.15.46:0/727424785 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f19600b2880 0x7f19600b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:28.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.942+0000 7f196b7fe700 1 -- 172.21.15.46:0/727424785 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f19640e2700 con 0x7f1974104380 2024-05-09T04:50:28.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.942+0000 7f197a039700 1 --2- 172.21.15.46:0/727424785 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f19600b2880 0x7f19600b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f197000b070 tx=0x7f19700060c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:28.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:28.945+0000 7f196b7fe700 1 -- 172.21.15.46:0/727424785 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f19640aa3e0 con 0x7f1974104380 2024-05-09T04:50:29.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:29.169+0000 7f197c29d700 1 -- 172.21.15.46:0/727424785 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f1974063c00 con 0x7f19600b2880 2024-05-09T04:50:29.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:29.170+0000 7f196b7fe700 1 -- 172.21.15.46:0/727424785 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f1974063c00 con 0x7f19600b2880 2024-05-09T04:50:29.171 INFO:teuthology.orchestra.run.smithi046.stdout:HOST ADDR LABELS STATUS 2024-05-09T04:50:29.171 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 172.21.15.46 2024-05-09T04:50:29.171 INFO:teuthology.orchestra.run.smithi046.stdout:smithi072 172.21.15.72 2024-05-09T04:50:29.171 INFO:teuthology.orchestra.run.smithi046.stdout:smithi110 172.21.15.110 2024-05-09T04:50:29.171 INFO:teuthology.orchestra.run.smithi046.stdout:3 hosts in cluster 2024-05-09T04:50:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:29.172+0000 7f197c29d700 1 -- 172.21.15.46:0/727424785 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f19600b2880 msgr2=0x7f19600b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:29.172+0000 7f197c29d700 1 --2- 172.21.15.46:0/727424785 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f19600b2880 0x7f19600b4d90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f197000b070 tx=0x7f19700060c0 comp rx=0 tx=0).stop 2024-05-09T04:50:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:29.172+0000 7f197c29d700 1 -- 172.21.15.46:0/727424785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1974104380 msgr2=0x7f1974199d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:29.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:29.172+0000 7f197c29d700 1 --2- 172.21.15.46:0/727424785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1974104380 0x7f1974199d80 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f196400e9a0 tx=0x7f196400ecb0 comp rx=0 tx=0).stop 2024-05-09T04:50:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:29.172+0000 7f197c29d700 1 -- 172.21.15.46:0/727424785 shutdown_connections 2024-05-09T04:50:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:29.172+0000 7f197c29d700 1 --2- 172.21.15.46:0/727424785 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f19600b2880 0x7f19600b4d90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:29.172+0000 7f197c29d700 1 --2- 172.21.15.46:0/727424785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1974104d20 0x7f19741a0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:29.172+0000 7f197c29d700 1 --2- 172.21.15.46:0/727424785 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1974103250 0x7f1974199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:29.172+0000 7f197c29d700 1 --2- 172.21.15.46:0/727424785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1974104380 0x7f1974199d80 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:29.172+0000 7f197c29d700 1 -- 172.21.15.46:0/727424785 >> 172.21.15.46:0/727424785 conn(0x7f19740fe890 msgr2=0x7f19741075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:29.172+0000 7f197c29d700 1 -- 172.21.15.46:0/727424785 shutdown_connections 2024-05-09T04:50:29.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:29.172+0000 7f197c29d700 1 -- 172.21.15.46:0/727424785 wait complete. 2024-05-09T04:50:29.891 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-09T04:50:30.144 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:30.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:30 smithi110 ceph-mon[34904]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:50:30.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:30 smithi110 ceph-mon[34904]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:50:30.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:30 smithi046 ceph-mon[30136]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:50:30.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:30 smithi046 ceph-mon[30136]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:50:30.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:30 smithi072 ceph-mon[34088]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:50:30.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:30 smithi072 ceph-mon[34088]: pgmap v375: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:50:32.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.010+0000 7fc5e2eb7700 1 -- 172.21.15.46:0/2459911354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc104370 msgr2=0x7fc5dc104780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.010+0000 7fc5e2eb7700 1 --2- 172.21.15.46:0/2459911354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc104370 0x7fc5dc104780 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00b0a0 tx=0x7fc5cc00b3b0 comp rx=0 tx=0).stop 2024-05-09T04:50:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.010+0000 7fc5e2eb7700 1 -- 172.21.15.46:0/2459911354 shutdown_connections 2024-05-09T04:50:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.010+0000 7fc5e2eb7700 1 --2- 172.21.15.46:0/2459911354 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5dc105e40 0x7fc5dc10c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.010+0000 7fc5e2eb7700 1 --2- 172.21.15.46:0/2459911354 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc104370 0x7fc5dc104780 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.010+0000 7fc5e2eb7700 1 --2- 172.21.15.46:0/2459911354 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5dc1054a0 0x7fc5dc1058b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.010+0000 7fc5e2eb7700 1 -- 172.21.15.46:0/2459911354 >> 172.21.15.46:0/2459911354 conn(0x7fc5dc0ff9d0 msgr2=0x7fc5dc101e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.010+0000 7fc5e2eb7700 1 -- 172.21.15.46:0/2459911354 shutdown_connections 2024-05-09T04:50:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.011+0000 7fc5e2eb7700 1 -- 172.21.15.46:0/2459911354 wait complete. 2024-05-09T04:50:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.011+0000 7fc5e2eb7700 1 Processor -- start 2024-05-09T04:50:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.011+0000 7fc5e2eb7700 1 -- start start 2024-05-09T04:50:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.011+0000 7fc5e2eb7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5dc104370 0x7fc5dc19a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:32.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.011+0000 7fc5e2eb7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc1054a0 0x7fc5dc19aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.011+0000 7fc5e2eb7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5dc105e40 0x7fc5dc1a1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.011+0000 7fc5e2eb7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc10fdc0 con 0x7fc5dc1054a0 2024-05-09T04:50:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.011+0000 7fc5e2eb7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc10fc40 con 0x7fc5dc104370 2024-05-09T04:50:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.011+0000 7fc5e2eb7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc10ff40 con 0x7fc5dc105e40 2024-05-09T04:50:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5dbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc1054a0 0x7fc5dc19aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5dbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc1054a0 0x7fc5dc19aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51014/0 (socket says 172.21.15.46:51014) 2024-05-09T04:50:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5dbfff700 1 -- 172.21.15.46:0/3552662128 learned_addr learned my addr 172.21.15.46:0/3552662128 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5e0c53700 1 --2- 172.21.15.46:0/3552662128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5dc104370 0x7fc5dc19a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5e1454700 1 --2- 172.21.15.46:0/3552662128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5dc105e40 0x7fc5dc1a1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5dbfff700 1 -- 172.21.15.46:0/3552662128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5dc105e40 msgr2=0x7fc5dc1a1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5dbfff700 1 --2- 172.21.15.46:0/3552662128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5dc105e40 0x7fc5dc1a1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5dbfff700 1 -- 172.21.15.46:0/3552662128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5dc104370 msgr2=0x7fc5dc19a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5dbfff700 1 --2- 172.21.15.46:0/3552662128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5dc104370 0x7fc5dc19a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:32.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5dbfff700 1 -- 172.21.15.46:0/3552662128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc1a2430 con 0x7fc5dc1054a0 2024-05-09T04:50:32.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5e1454700 1 --2- 172.21.15.46:0/3552662128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5dc105e40 0x7fc5dc1a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:50:32.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5dbfff700 1 --2- 172.21.15.46:0/3552662128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc1054a0 0x7fc5dc19aea0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400c920 tx=0x7fc5c400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:32.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5d9ffb700 1 -- 172.21.15.46:0/3552662128 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4007b00 con 0x7fc5dc1054a0 2024-05-09T04:50:32.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5d9ffb700 1 -- 172.21.15.46:0/3552662128 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5c4004ce0 con 0x7fc5dc1054a0 2024-05-09T04:50:32.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5d9ffb700 1 -- 172.21.15.46:0/3552662128 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5c4005550 con 0x7fc5dc1054a0 2024-05-09T04:50:32.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.012+0000 7fc5e2eb7700 1 -- 172.21.15.46:0/3552662128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc1a2720 con 0x7fc5dc1054a0 2024-05-09T04:50:32.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.013+0000 7fc5e2eb7700 1 -- 172.21.15.46:0/3552662128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc1a2c10 con 0x7fc5dc1054a0 2024-05-09T04:50:32.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.013+0000 7fc5e2eb7700 1 -- 172.21.15.46:0/3552662128 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5dc0512c0 con 0x7fc5dc1054a0 2024-05-09T04:50:32.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.015+0000 7fc5d9ffb700 1 -- 172.21.15.46:0/3552662128 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc5c4007ca0 con 0x7fc5dc1054a0 2024-05-09T04:50:32.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.016+0000 7fc5d9ffb700 1 --2- 172.21.15.46:0/3552662128 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc5c80b2880 0x7fc5c80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:32.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.016+0000 7fc5e0c53700 1 --2- 172.21.15.46:0/3552662128 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc5c80b2880 0x7fc5c80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.016+0000 7fc5d9ffb700 1 -- 172.21.15.46:0/3552662128 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fc5c40e21c0 con 0x7fc5dc1054a0 2024-05-09T04:50:32.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.017+0000 7fc5e0c53700 1 --2- 172.21.15.46:0/3552662128 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc5c80b2880 0x7fc5c80b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009fd0 tx=0x7fc5cc00b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:32.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.019+0000 7fc5d9ffb700 1 -- 172.21.15.46:0/3552662128 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc5c4014030 con 0x7fc5dc1054a0 2024-05-09T04:50:32.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.242+0000 7fc5e2eb7700 1 -- 172.21.15.46:0/3552662128 --> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fc5dc063c00 con 0x7fc5c80b2880 2024-05-09T04:50:32.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.244+0000 7fc5d9ffb700 1 -- 172.21.15.46:0/3552662128 <== mgr.14152 v2:172.21.15.46:6800/3691531627 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fc5dc063c00 con 0x7fc5c80b2880 2024-05-09T04:50:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-09T04:50:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:smithi046 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7512009A480DGN 447G No 4m ago Has a FileSystem, LVM detected 2024-05-09T04:50:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:smithi072 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400V7400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-09T04:50:32.245 INFO:teuthology.orchestra.run.smithi046.stdout:smithi110 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7476002T480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-05-09T04:50:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.247+0000 7fc5e2eb7700 1 -- 172.21.15.46:0/3552662128 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc5c80b2880 msgr2=0x7fc5c80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.247+0000 7fc5e2eb7700 1 --2- 172.21.15.46:0/3552662128 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc5c80b2880 0x7fc5c80b4d90 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009fd0 tx=0x7fc5cc00b7b0 comp rx=0 tx=0).stop 2024-05-09T04:50:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.247+0000 7fc5e2eb7700 1 -- 172.21.15.46:0/3552662128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc1054a0 msgr2=0x7fc5dc19aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.247+0000 7fc5e2eb7700 1 --2- 172.21.15.46:0/3552662128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc1054a0 0x7fc5dc19aea0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fc5c400c920 tx=0x7fc5c400cc30 comp rx=0 tx=0).stop 2024-05-09T04:50:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.247+0000 7fc5e2eb7700 1 -- 172.21.15.46:0/3552662128 shutdown_connections 2024-05-09T04:50:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.247+0000 7fc5e2eb7700 1 --2- 172.21.15.46:0/3552662128 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc5c80b2880 0x7fc5c80b4d90 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.247+0000 7fc5e2eb7700 1 --2- 172.21.15.46:0/3552662128 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5dc105e40 0x7fc5dc1a1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:32.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.247+0000 7fc5e2eb7700 1 --2- 172.21.15.46:0/3552662128 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5dc104370 0x7fc5dc19a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.247+0000 7fc5e2eb7700 1 --2- 172.21.15.46:0/3552662128 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc1054a0 0x7fc5dc19aea0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.247+0000 7fc5e2eb7700 1 -- 172.21.15.46:0/3552662128 >> 172.21.15.46:0/3552662128 conn(0x7fc5dc0ff9d0 msgr2=0x7fc5dc1086d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.247+0000 7fc5e2eb7700 1 -- 172.21.15.46:0/3552662128 shutdown_connections 2024-05-09T04:50:32.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:32.247+0000 7fc5e2eb7700 1 -- 172.21.15.46:0/3552662128 wait complete. 2024-05-09T04:50:32.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:32 smithi110 ceph-mon[34904]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:50:32.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:32 smithi046 ceph-mon[30136]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:50:32.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:32 smithi072 ceph-mon[34088]: pgmap v376: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:50:33.067 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-09T04:50:33.074 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T04:50:33.074 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-09T04:50:33.328 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:33.775 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:33 smithi046 ceph-mon[30136]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:50:33.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:33 smithi110 ceph-mon[34904]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:50:33.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:33 smithi072 ceph-mon[34088]: from='client.15042 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:50:34.764 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:34 smithi046 ceph-mon[30136]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:50:34.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:34 smithi110 ceph-mon[34904]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:50:34.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:34 smithi072 ceph-mon[34088]: pgmap v377: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:50:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.984+0000 7fec10b14700 1 -- 172.21.15.46:0/4142873579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec0c104380 msgr2=0x7fec0c104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.984+0000 7fec10b14700 1 --2- 172.21.15.46:0/4142873579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec0c104380 0x7fec0c104790 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7febfc009990 tx=0x7febfc009ca0 comp rx=0 tx=0).stop 2024-05-09T04:50:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.985+0000 7fec10b14700 1 -- 172.21.15.46:0/4142873579 shutdown_connections 2024-05-09T04:50:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.985+0000 7fec10b14700 1 --2- 172.21.15.46:0/4142873579 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec0c104d20 0x7fec0c10b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:34.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.985+0000 7fec10b14700 1 --2- 172.21.15.46:0/4142873579 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec0c103250 0x7fec0c103660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.985+0000 7fec10b14700 1 --2- 172.21.15.46:0/4142873579 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec0c104380 0x7fec0c104790 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.985+0000 7fec10b14700 1 -- 172.21.15.46:0/4142873579 >> 172.21.15.46:0/4142873579 conn(0x7fec0c0fe890 msgr2=0x7fec0c100d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.985+0000 7fec10b14700 1 -- 172.21.15.46:0/4142873579 shutdown_connections 2024-05-09T04:50:34.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.985+0000 7fec10b14700 1 -- 172.21.15.46:0/4142873579 wait complete. 2024-05-09T04:50:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.985+0000 7fec10b14700 1 Processor -- start 2024-05-09T04:50:34.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec10b14700 1 -- start start 2024-05-09T04:50:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec10b14700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec0c103250 0x7fec0c076450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec10b14700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec0c104380 0x7fec0c19f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec10b14700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec0c104d20 0x7fec0c1a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec10b14700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c10ecb0 con 0x7fec0c104380 2024-05-09T04:50:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec10b14700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c10eb30 con 0x7fec0c103250 2024-05-09T04:50:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec10b14700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fec0c10ee30 con 0x7fec0c104d20 2024-05-09T04:50:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec09d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec0c104380 0x7fec0c19f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec09d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec0c104380 0x7fec0c19f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53380/0 (socket says 172.21.15.46:53380) 2024-05-09T04:50:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec09d9b700 1 -- 172.21.15.46:0/3068201944 learned_addr learned my addr 172.21.15.46:0/3068201944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:34.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec0a59c700 1 --2- 172.21.15.46:0/3068201944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec0c103250 0x7fec0c076450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec0ad9d700 1 --2- 172.21.15.46:0/3068201944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec0c104d20 0x7fec0c1a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec09d9b700 1 -- 172.21.15.46:0/3068201944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec0c104d20 msgr2=0x7fec0c1a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec09d9b700 1 --2- 172.21.15.46:0/3068201944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec0c104d20 0x7fec0c1a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec09d9b700 1 -- 172.21.15.46:0/3068201944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec0c103250 msgr2=0x7fec0c076450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec09d9b700 1 --2- 172.21.15.46:0/3068201944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec0c103250 0x7fec0c076450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec09d9b700 1 -- 172.21.15.46:0/3068201944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec0c1a24e0 con 0x7fec0c104380 2024-05-09T04:50:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.986+0000 7fec09d9b700 1 --2- 172.21.15.46:0/3068201944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec0c104380 0x7fec0c19f740 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7febfc006010 tx=0x7febfc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.987+0000 7fec037fe700 1 -- 172.21.15.46:0/3068201944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc027070 con 0x7fec0c104380 2024-05-09T04:50:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.987+0000 7fec10b14700 1 -- 172.21.15.46:0/3068201944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec0c1a2770 con 0x7fec0c104380 2024-05-09T04:50:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.987+0000 7fec037fe700 1 -- 172.21.15.46:0/3068201944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febfc004000 con 0x7fec0c104380 2024-05-09T04:50:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.987+0000 7fec10b14700 1 -- 172.21.15.46:0/3068201944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec0c1a2c20 con 0x7fec0c104380 2024-05-09T04:50:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.987+0000 7fec037fe700 1 -- 172.21.15.46:0/3068201944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febfc022d30 con 0x7fec0c104380 2024-05-09T04:50:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.988+0000 7fec10b14700 1 -- 172.21.15.46:0/3068201944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec0c0512c0 con 0x7fec0c104380 2024-05-09T04:50:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.990+0000 7fec037fe700 1 -- 172.21.15.46:0/3068201944 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7febfc02a070 con 0x7fec0c104380 2024-05-09T04:50:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.991+0000 7fec037fe700 1 --2- 172.21.15.46:0/3068201944 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7febf80b2900 0x7febf80b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.991+0000 7fec0a59c700 1 --2- 172.21.15.46:0/3068201944 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7febf80b2900 0x7febf80b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.991+0000 7fec037fe700 1 -- 172.21.15.46:0/3068201944 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7febfc0e35c0 con 0x7fec0c104380 2024-05-09T04:50:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.992+0000 7fec0a59c700 1 --2- 172.21.15.46:0/3068201944 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7febf80b2900 0x7febf80b4e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fec0c104280 tx=0x7febf400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:34.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:34.994+0000 7fec037fe700 1 -- 172.21.15.46:0/3068201944 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7febfc031080 con 0x7fec0c104380 2024-05-09T04:50:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:35.260+0000 7fec10b14700 1 -- 172.21.15.46:0/3068201944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fec0c02e5f0 con 0x7fec0c104380 2024-05-09T04:50:35.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:35.260+0000 7fec037fe700 1 -- 172.21.15.46:0/3068201944 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) ==== 76+0+3044 (secure 0 0 0) 0x7febfc013070 con 0x7fec0c104380 2024-05-09T04:50:35.262 INFO:teuthology.orchestra.run.smithi046.stdout:e24 2024-05-09T04:50:35.262 INFO:teuthology.orchestra.run.smithi046.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-09T04:50:35.262 INFO:teuthology.orchestra.run.smithi046.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T04:50:35.262 INFO:teuthology.orchestra.run.smithi046.stdout:legacy client fscid: 1 2024-05-09T04:50:35.262 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:50:35.262 INFO:teuthology.orchestra.run.smithi046.stdout:Filesystem 'cephfs' (1) 2024-05-09T04:50:35.262 INFO:teuthology.orchestra.run.smithi046.stdout:fs_name cephfs 2024-05-09T04:50:35.262 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 24 2024-05-09T04:50:35.262 INFO:teuthology.orchestra.run.smithi046.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-09T04:50:35.262 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-05-09T04:47:50.321554+0000 2024-05-09T04:50:35.262 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-05-09T04:48:11.653145+0000 2024-05-09T04:50:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:tableserver 0 2024-05-09T04:50:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:root 0 2024-05-09T04:50:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:session_timeout 300 2024-05-09T04:50:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:session_autoclose 300 2024-05-09T04:50:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:max_file_size 1099511627776 2024-05-09T04:50:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:max_xattr_size 65536 2024-05-09T04:50:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:required_client_features {} 2024-05-09T04:50:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure 0 2024-05-09T04:50:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:last_failure_osd_epoch 0 2024-05-09T04:50:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-09T04:50:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:max_mds 3 2024-05-09T04:50:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:in 0,1,2 2024-05-09T04:50:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:up {0=24503,1=24473,2=24448} 2024-05-09T04:50:35.263 INFO:teuthology.orchestra.run.smithi046.stdout:failed 2024-05-09T04:50:35.264 INFO:teuthology.orchestra.run.smithi046.stdout:damaged 2024-05-09T04:50:35.264 INFO:teuthology.orchestra.run.smithi046.stdout:stopped 2024-05-09T04:50:35.264 INFO:teuthology.orchestra.run.smithi046.stdout:data_pools [3] 2024-05-09T04:50:35.264 INFO:teuthology.orchestra.run.smithi046.stdout:metadata_pool 2 2024-05-09T04:50:35.264 INFO:teuthology.orchestra.run.smithi046.stdout:inline_data disabled 2024-05-09T04:50:35.264 INFO:teuthology.orchestra.run.smithi046.stdout:balancer 2024-05-09T04:50:35.264 INFO:teuthology.orchestra.run.smithi046.stdout:bal_rank_mask -1 2024-05-09T04:50:35.264 INFO:teuthology.orchestra.run.smithi046.stdout:standby_count_wanted 0 2024-05-09T04:50:35.264 INFO:teuthology.orchestra.run.smithi046.stdout:qdb_cluster leader: 24503 members: 24473,24503,24448 2024-05-09T04:50:35.264 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.b{0:24503} state up:active seq 5 addr [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:50:35.264 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.e{0:24443} state up:standby-replay seq 1 addr [v2:172.21.15.72:6834/4256517546,v1:172.21.15.72:6835/4256517546] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:50:35.264 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.h{1:24473} state up:active seq 12 addr [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:50:35.264 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.c{1:24433} state up:standby-replay seq 1 addr [v2:172.21.15.110:6836/2612088705,v1:172.21.15.110:6837/2612088705] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:50:35.265 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.i{2:24448} state up:active seq 11 addr [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:50:35.265 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.f{2:24427} state up:standby-replay seq 1 addr [v2:172.21.15.110:6834/2299300184,v1:172.21.15.110:6835/2299300184] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:50:35.265 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:50:35.265 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:50:35.265 INFO:teuthology.orchestra.run.smithi046.stdout:Standby daemons: 2024-05-09T04:50:35.265 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:50:35.265 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.j{-1:14541} state up:standby seq 1 addr [v2:172.21.15.46:6834/3023074192,v1:172.21.15.46:6835/3023074192] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:50:35.265 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.d{-1:14553} state up:standby seq 1 addr [v2:172.21.15.46:6836/3965071509,v1:172.21.15.46:6837/3965071509] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:50:35.265 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.g{-1:14607} state up:standby seq 1 addr [v2:172.21.15.46:6838/952386829,v1:172.21.15.46:6839/952386829] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:50:35.265 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.a{-1:14619} state up:standby seq 1 addr [v2:172.21.15.46:6840/204132247,v1:172.21.15.46:6841/204132247] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:50:35.265 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.l{-1:24400} state up:standby seq 1 addr [v2:172.21.15.110:6832/2118887975,v1:172.21.15.110:6833/2118887975] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:50:35.265 INFO:teuthology.orchestra.run.smithi046.stdout:[mds.k{-1:24419} state up:standby seq 1 addr [v2:172.21.15.72:6832/815890099,v1:172.21.15.72:6833/815890099] compat {c=[1],r=[1],i=[1fff]}] 2024-05-09T04:50:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:35.263+0000 7fec10b14700 1 -- 172.21.15.46:0/3068201944 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7febf80b2900 msgr2=0x7febf80b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:35.263+0000 7fec10b14700 1 --2- 172.21.15.46:0/3068201944 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7febf80b2900 0x7febf80b4e10 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fec0c104280 tx=0x7febf400b410 comp rx=0 tx=0).stop 2024-05-09T04:50:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:35.263+0000 7fec10b14700 1 -- 172.21.15.46:0/3068201944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec0c104380 msgr2=0x7fec0c19f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:35.263+0000 7fec10b14700 1 --2- 172.21.15.46:0/3068201944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec0c104380 0x7fec0c19f740 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7febfc006010 tx=0x7febfc0047b0 comp rx=0 tx=0).stop 2024-05-09T04:50:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:35.263+0000 7fec10b14700 1 -- 172.21.15.46:0/3068201944 shutdown_connections 2024-05-09T04:50:35.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:35.263+0000 7fec10b14700 1 --2- 172.21.15.46:0/3068201944 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7febf80b2900 0x7febf80b4e10 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:35.263+0000 7fec10b14700 1 --2- 172.21.15.46:0/3068201944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fec0c104d20 0x7fec0c1a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:35.263+0000 7fec10b14700 1 --2- 172.21.15.46:0/3068201944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fec0c103250 0x7fec0c076450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:35.263+0000 7fec10b14700 1 --2- 172.21.15.46:0/3068201944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fec0c104380 0x7fec0c19f740 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:35.263+0000 7fec10b14700 1 -- 172.21.15.46:0/3068201944 >> 172.21.15.46:0/3068201944 conn(0x7fec0c0fe890 msgr2=0x7fec0c1075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:35.263+0000 7fec10b14700 1 -- 172.21.15.46:0/3068201944 shutdown_connections 2024-05-09T04:50:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:35.263+0000 7fec10b14700 1 -- 172.21.15.46:0/3068201944 wait complete. 2024-05-09T04:50:35.267 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-05-09T04:50:35.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:35 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3068201944' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T04:50:35.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:35 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3068201944' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T04:50:35.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:35 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3068201944' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-09T04:50:36.019 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-09T04:50:36.284 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:36.676 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:36 smithi046 ceph-mon[30136]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:50:36.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:36 smithi110 ceph-mon[34904]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:50:36.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:36 smithi072 ceph-mon[34088]: pgmap v378: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:50:38.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.081+0000 7f75eb1b7700 1 -- 172.21.15.46:0/2386019959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75e4104350 msgr2=0x7f75e4104760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.081+0000 7f75eb1b7700 1 --2- 172.21.15.46:0/2386019959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75e4104350 0x7f75e4104760 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f75d8009990 tx=0x7f75d8009ca0 comp rx=0 tx=0).stop 2024-05-09T04:50:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.081+0000 7f75eb1b7700 1 -- 172.21.15.46:0/2386019959 shutdown_connections 2024-05-09T04:50:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.081+0000 7f75eb1b7700 1 --2- 172.21.15.46:0/2386019959 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75e4105e20 0x7f75e410c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.081+0000 7f75eb1b7700 1 --2- 172.21.15.46:0/2386019959 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75e4104350 0x7f75e4104760 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.081+0000 7f75eb1b7700 1 --2- 172.21.15.46:0/2386019959 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75e4105480 0x7f75e4105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.081+0000 7f75eb1b7700 1 -- 172.21.15.46:0/2386019959 >> 172.21.15.46:0/2386019959 conn(0x7f75e40ff990 msgr2=0x7f75e4101e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.081+0000 7f75eb1b7700 1 -- 172.21.15.46:0/2386019959 shutdown_connections 2024-05-09T04:50:38.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.081+0000 7f75eb1b7700 1 -- 172.21.15.46:0/2386019959 wait complete. 2024-05-09T04:50:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.082+0000 7f75eb1b7700 1 Processor -- start 2024-05-09T04:50:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.082+0000 7f75eb1b7700 1 -- start start 2024-05-09T04:50:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.082+0000 7f75eb1b7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75e4104350 0x7f75e419a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:38.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.082+0000 7f75eb1b7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75e4105480 0x7f75e419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.082+0000 7f75eb1b7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75e4105e20 0x7f75e41a1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.082+0000 7f75eb1b7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75e410fda0 con 0x7f75e4104350 2024-05-09T04:50:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.082+0000 7f75eb1b7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f75e410fc20 con 0x7f75e4105e20 2024-05-09T04:50:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.082+0000 7f75eb1b7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f75e410ff20 con 0x7f75e4105480 2024-05-09T04:50:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.082+0000 7f75e8f53700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75e4104350 0x7f75e419a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.082+0000 7f75e8f53700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75e4104350 0x7f75e419a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53388/0 (socket says 172.21.15.46:53388) 2024-05-09T04:50:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.083+0000 7f75e8f53700 1 -- 172.21.15.46:0/3660577801 learned_addr learned my addr 172.21.15.46:0/3660577801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.083+0000 7f75e3fff700 1 --2- 172.21.15.46:0/3660577801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75e4105480 0x7f75e419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.083+0000 7f75e9754700 1 --2- 172.21.15.46:0/3660577801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75e4105e20 0x7f75e41a1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.083+0000 7f75e8f53700 1 -- 172.21.15.46:0/3660577801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75e4105480 msgr2=0x7f75e419ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.083+0000 7f75e8f53700 1 --2- 172.21.15.46:0/3660577801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75e4105480 0x7f75e419ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:38.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.083+0000 7f75e8f53700 1 -- 172.21.15.46:0/3660577801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75e4105e20 msgr2=0x7f75e41a1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.083+0000 7f75e8f53700 1 --2- 172.21.15.46:0/3660577801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75e4105e20 0x7f75e41a1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.083+0000 7f75e8f53700 1 -- 172.21.15.46:0/3660577801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75e41a2410 con 0x7f75e4104350 2024-05-09T04:50:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.083+0000 7f75e8f53700 1 --2- 172.21.15.46:0/3660577801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75e4104350 0x7f75e419a940 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f75d80057c0 tx=0x7f75d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.083+0000 7f75e1ffb700 1 -- 172.21.15.46:0/3660577801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8027070 con 0x7f75e4104350 2024-05-09T04:50:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.083+0000 7f75eb1b7700 1 -- 172.21.15.46:0/3660577801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75e41a26a0 con 0x7f75e4104350 2024-05-09T04:50:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.083+0000 7f75e1ffb700 1 -- 172.21.15.46:0/3660577801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75d8004000 con 0x7f75e4104350 2024-05-09T04:50:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.083+0000 7f75e1ffb700 1 -- 172.21.15.46:0/3660577801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75d8022be0 con 0x7f75e4104350 2024-05-09T04:50:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.083+0000 7f75eb1b7700 1 -- 172.21.15.46:0/3660577801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75e41a2b50 con 0x7f75e4104350 2024-05-09T04:50:38.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.085+0000 7f75eb1b7700 1 -- 172.21.15.46:0/3660577801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75e40512c0 con 0x7f75e4104350 2024-05-09T04:50:38.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.086+0000 7f75e1ffb700 1 -- 172.21.15.46:0/3660577801 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f75d802a070 con 0x7f75e4104350 2024-05-09T04:50:38.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.087+0000 7f75e1ffb700 1 --2- 172.21.15.46:0/3660577801 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75d40b2880 0x7f75d40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:38.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.087+0000 7f75e3fff700 1 --2- 172.21.15.46:0/3660577801 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75d40b2880 0x7f75d40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:38.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.087+0000 7f75e1ffb700 1 -- 172.21.15.46:0/3660577801 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f75d80e3240 con 0x7f75e4104350 2024-05-09T04:50:38.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.087+0000 7f75e3fff700 1 --2- 172.21.15.46:0/3660577801 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75d40b2880 0x7f75d40b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f75e419b9f0 tx=0x7f75d000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:38.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.092+0000 7f75e1ffb700 1 -- 172.21.15.46:0/3660577801 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f75d8013070 con 0x7f75e4104350 2024-05-09T04:50:38.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.306+0000 7f75eb1b7700 1 -- 172.21.15.46:0/3660577801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f75e402e5f0 con 0x7f75e4104350 2024-05-09T04:50:38.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.307+0000 7f75e1ffb700 1 -- 172.21.15.46:0/3660577801 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v84) ==== 56+0+5325 (secure 0 0 0) 0x7f75d80041b0 con 0x7f75e4104350 2024-05-09T04:50:38.307 INFO:teuthology.orchestra.run.smithi046.stdout:epoch 84 2024-05-09T04:50:38.307 INFO:teuthology.orchestra.run.smithi046.stdout:fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T04:50:38.307 INFO:teuthology.orchestra.run.smithi046.stdout:created 2024-05-09T04:39:25.633876+0000 2024-05-09T04:50:38.307 INFO:teuthology.orchestra.run.smithi046.stdout:modified 2024-05-09T04:49:29.385454+0000 2024-05-09T04:50:38.307 INFO:teuthology.orchestra.run.smithi046.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-09T04:50:38.307 INFO:teuthology.orchestra.run.smithi046.stdout:crush_version 26 2024-05-09T04:50:38.307 INFO:teuthology.orchestra.run.smithi046.stdout:full_ratio 0.95 2024-05-09T04:50:38.307 INFO:teuthology.orchestra.run.smithi046.stdout:backfillfull_ratio 0.9 2024-05-09T04:50:38.307 INFO:teuthology.orchestra.run.smithi046.stdout:nearfull_ratio 0.85 2024-05-09T04:50:38.308 INFO:teuthology.orchestra.run.smithi046.stdout:require_min_compat_client luminous 2024-05-09T04:50:38.308 INFO:teuthology.orchestra.run.smithi046.stdout:min_compat_client luminous 2024-05-09T04:50:38.308 INFO:teuthology.orchestra.run.smithi046.stdout:require_osd_release squid 2024-05-09T04:50:38.308 INFO:teuthology.orchestra.run.smithi046.stdout:stretch_mode_enabled false 2024-05-09T04:50:38.308 INFO:teuthology.orchestra.run.smithi046.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-09T04:50:38.308 INFO:teuthology.orchestra.run.smithi046.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-09T04:50:38.308 INFO:teuthology.orchestra.run.smithi046.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-05-09T04:50:38.308 INFO:teuthology.orchestra.run.smithi046.stdout:max_osd 12 2024-05-09T04:50:38.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.0 up in weight 1 up_from 8 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6802/1576436508,v1:172.21.15.46:6803/1576436508] [v2:172.21.15.46:6804/1576436508,v1:172.21.15.46:6805/1576436508] exists,up d79cd3e7-6f82-4d60-96d9-26d8146c962f 2024-05-09T04:50:38.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.1 up in weight 1 up_from 13 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6800/379039775,v1:172.21.15.72:6801/379039775] [v2:172.21.15.72:6802/379039775,v1:172.21.15.72:6803/379039775] exists,up 698eba80-1ebd-4076-b6f9-953d2e7daf76 2024-05-09T04:50:38.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.2 up in weight 1 up_from 18 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6800/293447227,v1:172.21.15.110:6801/293447227] [v2:172.21.15.110:6802/293447227,v1:172.21.15.110:6803/293447227] exists,up c549b131-cfa7-4724-926f-83e91a752d44 2024-05-09T04:50:38.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.3 up in weight 1 up_from 27 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6810/1115835653,v1:172.21.15.46:6811/1115835653] [v2:172.21.15.46:6812/1115835653,v1:172.21.15.46:6813/1115835653] exists,up 549bb4f7-54d9-4cbc-ad40-7f3e717964f3 2024-05-09T04:50:38.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.4 up in weight 1 up_from 33 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6808/59186693,v1:172.21.15.72:6809/59186693] [v2:172.21.15.72:6810/59186693,v1:172.21.15.72:6811/59186693] exists,up 444cd36b-6034-46cd-9a6b-29dce4ba6405 2024-05-09T04:50:38.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.5 up in weight 1 up_from 38 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6808/1712162502,v1:172.21.15.110:6809/1712162502] [v2:172.21.15.110:6810/1712162502,v1:172.21.15.110:6811/1712162502] exists,up 2c2bdd2f-43cf-4208-9ae1-f912807c84e0 2024-05-09T04:50:38.308 INFO:teuthology.orchestra.run.smithi046.stdout:osd.6 up in weight 1 up_from 44 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6818/2429650065,v1:172.21.15.46:6819/2429650065] [v2:172.21.15.46:6820/2429650065,v1:172.21.15.46:6821/2429650065] exists,up beb0b87f-c3a4-40dd-b224-1843e9301ca9 2024-05-09T04:50:38.309 INFO:teuthology.orchestra.run.smithi046.stdout:osd.7 up in weight 1 up_from 49 up_thru 77 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6816/337796672,v1:172.21.15.72:6817/337796672] [v2:172.21.15.72:6818/337796672,v1:172.21.15.72:6819/337796672] exists,up 4a634675-fad7-4a43-840c-8dec2631dec9 2024-05-09T04:50:38.309 INFO:teuthology.orchestra.run.smithi046.stdout:osd.8 up in weight 1 up_from 55 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6816/189513203,v1:172.21.15.110:6817/189513203] [v2:172.21.15.110:6818/189513203,v1:172.21.15.110:6819/189513203] exists,up 01b5f638-90f0-4cf4-8042-20d20a725669 2024-05-09T04:50:38.309 INFO:teuthology.orchestra.run.smithi046.stdout:osd.9 up in weight 1 up_from 61 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.46:6826/1762262822,v1:172.21.15.46:6827/1762262822] [v2:172.21.15.46:6828/1762262822,v1:172.21.15.46:6829/1762262822] exists,up e423de41-50ec-48f4-a4ed-216de33c4fb5 2024-05-09T04:50:38.309 INFO:teuthology.orchestra.run.smithi046.stdout:osd.10 up in weight 1 up_from 66 up_thru 81 down_at 0 last_clean_interval [0,0) [v2:172.21.15.72:6824/38453898,v1:172.21.15.72:6825/38453898] [v2:172.21.15.72:6826/38453898,v1:172.21.15.72:6827/38453898] exists,up f594d004-9f80-4508-b127-fb693d355f40 2024-05-09T04:50:38.309 INFO:teuthology.orchestra.run.smithi046.stdout:osd.11 up in weight 1 up_from 72 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.110:6824/4153529632,v1:172.21.15.110:6825/4153529632] [v2:172.21.15.110:6826/4153529632,v1:172.21.15.110:6827/4153529632] exists,up f771d29f-89e3-4f96-9c29-00e424230c1a 2024-05-09T04:50:38.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.b [2,1] 2024-05-09T04:50:38.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.14 [2,4] 2024-05-09T04:50:38.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.1c [8,11] 2024-05-09T04:50:38.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 2.1d [8,11] 2024-05-09T04:50:38.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.e [4,6] 2024-05-09T04:50:38.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.11 [5,9] 2024-05-09T04:50:38.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.17 [4,8] 2024-05-09T04:50:38.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.1b [5,9] 2024-05-09T04:50:38.309 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.1d [4,8] 2024-05-09T04:50:38.310 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.21 [4,7] 2024-05-09T04:50:38.310 INFO:teuthology.orchestra.run.smithi046.stdout:pg_upmap_items 3.38 [3,9] 2024-05-09T04:50:38.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6800/123334554 expires 2024-05-10T04:40:27.201119+0000 2024-05-09T04:50:38.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/1596672353 expires 2024-05-10T04:40:27.201119+0000 2024-05-09T04:50:38.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/1846482551 expires 2024-05-10T04:39:59.515628+0000 2024-05-09T04:50:38.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/1069571878 expires 2024-05-10T04:39:59.515628+0000 2024-05-09T04:50:38.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6800/2738158981 expires 2024-05-10T04:39:59.515628+0000 2024-05-09T04:50:38.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/3625745146 expires 2024-05-10T04:39:59.515628+0000 2024-05-09T04:50:38.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6801/123334554 expires 2024-05-10T04:40:27.201119+0000 2024-05-09T04:50:38.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/564539954 expires 2024-05-10T04:40:27.201119+0000 2024-05-09T04:50:38.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:0/1797199863 expires 2024-05-10T04:40:27.201119+0000 2024-05-09T04:50:38.310 INFO:teuthology.orchestra.run.smithi046.stdout:blocklist 172.21.15.46:6801/2738158981 expires 2024-05-10T04:39:59.515628+0000 2024-05-09T04:50:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.310+0000 7f75eb1b7700 1 -- 172.21.15.46:0/3660577801 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75d40b2880 msgr2=0x7f75d40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.310+0000 7f75eb1b7700 1 --2- 172.21.15.46:0/3660577801 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75d40b2880 0x7f75d40b4d90 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f75e419b9f0 tx=0x7f75d000a380 comp rx=0 tx=0).stop 2024-05-09T04:50:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.310+0000 7f75eb1b7700 1 -- 172.21.15.46:0/3660577801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75e4104350 msgr2=0x7f75e419a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.310+0000 7f75eb1b7700 1 --2- 172.21.15.46:0/3660577801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75e4104350 0x7f75e419a940 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f75d80057c0 tx=0x7f75d80057f0 comp rx=0 tx=0).stop 2024-05-09T04:50:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.310+0000 7f75eb1b7700 1 -- 172.21.15.46:0/3660577801 shutdown_connections 2024-05-09T04:50:38.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.310+0000 7f75eb1b7700 1 --2- 172.21.15.46:0/3660577801 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75d40b2880 0x7f75d40b4d90 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.310+0000 7f75eb1b7700 1 --2- 172.21.15.46:0/3660577801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75e4105e20 0x7f75e41a1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.310+0000 7f75eb1b7700 1 --2- 172.21.15.46:0/3660577801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75e4104350 0x7f75e419a940 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.310+0000 7f75eb1b7700 1 --2- 172.21.15.46:0/3660577801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75e4105480 0x7f75e419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.310+0000 7f75eb1b7700 1 -- 172.21.15.46:0/3660577801 >> 172.21.15.46:0/3660577801 conn(0x7f75e40ff990 msgr2=0x7f75e41086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.310+0000 7f75eb1b7700 1 -- 172.21.15.46:0/3660577801 shutdown_connections 2024-05-09T04:50:38.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:38.311+0000 7f75eb1b7700 1 -- 172.21.15.46:0/3660577801 wait complete. 2024-05-09T04:50:38.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:38 smithi110 ceph-mon[34904]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:50:38.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:38 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3660577801' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T04:50:38.829 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:38 smithi046 ceph-mon[30136]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:50:38.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:38 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3660577801' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T04:50:38.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:38 smithi072 ceph-mon[34088]: pgmap v379: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:50:38.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:38 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3660577801' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-09T04:50:38.990 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-09T04:50:39.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:50:39.260 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:40.745 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:40 smithi046 ceph-mon[30136]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:50:40.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:40 smithi072 ceph-mon[34088]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:50:41.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.011+0000 7f75a7b4e700 1 -- 172.21.15.46:0/333963453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a0108f20 msgr2=0x7f75a0109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:41.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.011+0000 7f75a7b4e700 1 --2- 172.21.15.46:0/333963453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a0108f20 0x7f75a0109330 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f75940099c0 tx=0x7f7594009cd0 comp rx=0 tx=0).stop 2024-05-09T04:50:41.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.011+0000 7f75a7b4e700 1 -- 172.21.15.46:0/333963453 shutdown_connections 2024-05-09T04:50:41.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.011+0000 7f75a7b4e700 1 --2- 172.21.15.46:0/333963453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75a0109870 0x7f75a01111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:41.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.011+0000 7f75a7b4e700 1 --2- 172.21.15.46:0/333963453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75a01045f0 0x7f75a0104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:41.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.011+0000 7f75a7b4e700 1 --2- 172.21.15.46:0/333963453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a0108f20 0x7f75a0109330 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:41.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.011+0000 7f75a7b4e700 1 -- 172.21.15.46:0/333963453 >> 172.21.15.46:0/333963453 conn(0x7f75a00ffec0 msgr2=0x7f75a0102310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:41.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.011+0000 7f75a7b4e700 1 -- 172.21.15.46:0/333963453 shutdown_connections 2024-05-09T04:50:41.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.011+0000 7f75a7b4e700 1 -- 172.21.15.46:0/333963453 wait complete. 2024-05-09T04:50:41.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.012+0000 7f75a7b4e700 1 Processor -- start 2024-05-09T04:50:41.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.012+0000 7f75a7b4e700 1 -- start start 2024-05-09T04:50:41.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a7b4e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a01045f0 0x7f75a019d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:41.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a7b4e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75a0108f20 0x7f75a019d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:41.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a7b4e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75a0109870 0x7f75a01a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:41.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a7b4e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75a0114480 con 0x7f75a01045f0 2024-05-09T04:50:41.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a7b4e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f75a0114300 con 0x7f75a0108f20 2024-05-09T04:50:41.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a7b4e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f75a0114600 con 0x7f75a0109870 2024-05-09T04:50:41.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a58ea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a01045f0 0x7f75a019d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:41.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a58ea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a01045f0 0x7f75a019d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53398/0 (socket says 172.21.15.46:53398) 2024-05-09T04:50:41.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a58ea700 1 -- 172.21.15.46:0/3709572328 learned_addr learned my addr 172.21.15.46:0/3709572328 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:41.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a50e9700 1 --2- 172.21.15.46:0/3709572328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75a0108f20 0x7f75a019d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:41.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a60eb700 1 --2- 172.21.15.46:0/3709572328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75a0109870 0x7f75a01a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:41.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a58ea700 1 -- 172.21.15.46:0/3709572328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75a0109870 msgr2=0x7f75a01a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:41.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a58ea700 1 --2- 172.21.15.46:0/3709572328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75a0109870 0x7f75a01a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:41.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a58ea700 1 -- 172.21.15.46:0/3709572328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75a0108f20 msgr2=0x7f75a019d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:41.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a58ea700 1 --2- 172.21.15.46:0/3709572328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75a0108f20 0x7f75a019d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:41.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a58ea700 1 -- 172.21.15.46:0/3709572328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75a01a1f20 con 0x7f75a01045f0 2024-05-09T04:50:41.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f75a58ea700 1 --2- 172.21.15.46:0/3709572328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a01045f0 0x7f75a019d050 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f75900060b0 tx=0x7f7590007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:41.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.013+0000 7f759effd700 1 -- 172.21.15.46:0/3709572328 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759001a070 con 0x7f75a01045f0 2024-05-09T04:50:41.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.014+0000 7f75a7b4e700 1 -- 172.21.15.46:0/3709572328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75a01a2210 con 0x7f75a01045f0 2024-05-09T04:50:41.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.014+0000 7f759effd700 1 -- 172.21.15.46:0/3709572328 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f759000de20 con 0x7f75a01045f0 2024-05-09T04:50:41.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.014+0000 7f759effd700 1 -- 172.21.15.46:0/3709572328 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7590014720 con 0x7f75a01045f0 2024-05-09T04:50:41.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.014+0000 7f75a7b4e700 1 -- 172.21.15.46:0/3709572328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75a01a9b30 con 0x7f75a01045f0 2024-05-09T04:50:41.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.015+0000 7f75a7b4e700 1 -- 172.21.15.46:0/3709572328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75a0105aa0 con 0x7f75a01045f0 2024-05-09T04:50:41.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.016+0000 7f759effd700 1 -- 172.21.15.46:0/3709572328 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f759001d070 con 0x7f75a01045f0 2024-05-09T04:50:41.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.018+0000 7f759effd700 1 --2- 172.21.15.46:0/3709572328 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75880b2880 0x7f75880b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:41.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.018+0000 7f75a50e9700 1 --2- 172.21.15.46:0/3709572328 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75880b2880 0x7f75880b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:41.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.018+0000 7f759effd700 1 -- 172.21.15.46:0/3709572328 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f75900e2030 con 0x7f75a01045f0 2024-05-09T04:50:41.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.018+0000 7f75a50e9700 1 --2- 172.21.15.46:0/3709572328 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75880b2880 0x7f75880b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7594000c00 tx=0x7f759401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:41.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.021+0000 7f759effd700 1 -- 172.21.15.46:0/3709572328 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7590010070 con 0x7f75a01045f0 2024-05-09T04:50:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:40 smithi110 ceph-mon[34904]: pgmap v380: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:50:41.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.285+0000 7f75a7b4e700 1 -- 172.21.15.46:0/3709572328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f75a00512c0 con 0x7f75a01045f0 2024-05-09T04:50:41.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.286+0000 7f759effd700 1 -- 172.21.15.46:0/3709572328 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11862 (secure 0 0 0) 0x7f75900a9d10 con 0x7f75a01045f0 2024-05-09T04:50:41.286 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:50:41.286 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:48:11.653145+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24473,24503,24448]},"id":1}]} 2024-05-09T04:50:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.288+0000 7f75a7b4e700 1 -- 172.21.15.46:0/3709572328 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75880b2880 msgr2=0x7f75880b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.288+0000 7f75a7b4e700 1 --2- 172.21.15.46:0/3709572328 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75880b2880 0x7f75880b4d90 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7594000c00 tx=0x7f759401a040 comp rx=0 tx=0).stop 2024-05-09T04:50:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.288+0000 7f75a7b4e700 1 -- 172.21.15.46:0/3709572328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a01045f0 msgr2=0x7f75a019d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.288+0000 7f75a7b4e700 1 --2- 172.21.15.46:0/3709572328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a01045f0 0x7f75a019d050 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f75900060b0 tx=0x7f7590007f90 comp rx=0 tx=0).stop 2024-05-09T04:50:41.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.288+0000 7f75a7b4e700 1 -- 172.21.15.46:0/3709572328 shutdown_connections 2024-05-09T04:50:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.288+0000 7f75a7b4e700 1 --2- 172.21.15.46:0/3709572328 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f75880b2880 0x7f75880b4d90 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.288+0000 7f75a7b4e700 1 --2- 172.21.15.46:0/3709572328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75a0109870 0x7f75a01a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.288+0000 7f75a7b4e700 1 --2- 172.21.15.46:0/3709572328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75a01045f0 0x7f75a019d050 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.288+0000 7f75a7b4e700 1 --2- 172.21.15.46:0/3709572328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75a0108f20 0x7f75a019d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.288+0000 7f75a7b4e700 1 -- 172.21.15.46:0/3709572328 >> 172.21.15.46:0/3709572328 conn(0x7f75a00ffec0 msgr2=0x7f75a010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.288+0000 7f75a7b4e700 1 -- 172.21.15.46:0/3709572328 shutdown_connections 2024-05-09T04:50:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:41.288+0000 7f75a7b4e700 1 -- 172.21.15.46:0/3709572328 wait complete. 2024-05-09T04:50:41.289 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-05-09T04:50:41.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:41 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3709572328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:41.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:41 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3709572328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:42.037 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:50:42.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:41 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3709572328' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:42.291 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:42.761 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:42 smithi046 ceph-mon[30136]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T04:50:42.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:42 smithi072 ceph-mon[34088]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T04:50:43.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:42 smithi110 ceph-mon[34904]: pgmap v381: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T04:50:44.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.055+0000 7f82c3a7e700 1 -- 172.21.15.46:0/1690883632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc109850 msgr2=0x7f82bc1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:44.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.055+0000 7f82c3a7e700 1 --2- 172.21.15.46:0/1690883632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc109850 0x7f82bc1111b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f82b400b0d0 tx=0x7f82b400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:50:44.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.055+0000 7f82c3a7e700 1 -- 172.21.15.46:0/1690883632 shutdown_connections 2024-05-09T04:50:44.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.055+0000 7f82c3a7e700 1 --2- 172.21.15.46:0/1690883632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc109850 0x7f82bc1111b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:44.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.055+0000 7f82c3a7e700 1 --2- 172.21.15.46:0/1690883632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82bc1045d0 0x7f82bc1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:44.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.055+0000 7f82c3a7e700 1 --2- 172.21.15.46:0/1690883632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82bc108f00 0x7f82bc109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:44.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.055+0000 7f82c3a7e700 1 -- 172.21.15.46:0/1690883632 >> 172.21.15.46:0/1690883632 conn(0x7f82bc0ffea0 msgr2=0x7f82bc1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:44.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.055+0000 7f82c3a7e700 1 -- 172.21.15.46:0/1690883632 shutdown_connections 2024-05-09T04:50:44.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.055+0000 7f82c3a7e700 1 -- 172.21.15.46:0/1690883632 wait complete. 2024-05-09T04:50:44.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.056+0000 7f82c3a7e700 1 Processor -- start 2024-05-09T04:50:44.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.056+0000 7f82c3a7e700 1 -- start start 2024-05-09T04:50:44.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.056+0000 7f82c3a7e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82bc1045d0 0x7f82bc075340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:44.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.056+0000 7f82c3a7e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc108f00 0x7f82bc06e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:44.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.056+0000 7f82c3a7e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82bc109850 0x7f82bc06ea30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:44.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.056+0000 7f82c3a7e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc114480 con 0x7f82bc108f00 2024-05-09T04:50:44.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.056+0000 7f82c3a7e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc114300 con 0x7f82bc1045d0 2024-05-09T04:50:44.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.056+0000 7f82c3a7e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f82bc114600 con 0x7f82bc109850 2024-05-09T04:50:44.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.056+0000 7f82c1019700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc108f00 0x7f82bc06e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:44.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.056+0000 7f82c1019700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc108f00 0x7f82bc06e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57846/0 (socket says 172.21.15.46:57846) 2024-05-09T04:50:44.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.056+0000 7f82c1019700 1 -- 172.21.15.46:0/38732158 learned_addr learned my addr 172.21.15.46:0/38732158 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:44.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.057+0000 7f82c181a700 1 --2- 172.21.15.46:0/38732158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82bc1045d0 0x7f82bc075340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:44.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.057+0000 7f82c201b700 1 --2- 172.21.15.46:0/38732158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82bc109850 0x7f82bc06ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:44.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.057+0000 7f82c1019700 1 -- 172.21.15.46:0/38732158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82bc109850 msgr2=0x7f82bc06ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:44.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.057+0000 7f82c1019700 1 --2- 172.21.15.46:0/38732158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82bc109850 0x7f82bc06ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:44.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.057+0000 7f82c1019700 1 -- 172.21.15.46:0/38732158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82bc1045d0 msgr2=0x7f82bc075340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:44.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.057+0000 7f82c1019700 1 --2- 172.21.15.46:0/38732158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82bc1045d0 0x7f82bc075340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:44.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.057+0000 7f82c1019700 1 -- 172.21.15.46:0/38732158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82bc06f290 con 0x7f82bc108f00 2024-05-09T04:50:44.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.057+0000 7f82c1019700 1 --2- 172.21.15.46:0/38732158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc108f00 0x7f82bc06e470 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00c8f0 tx=0x7f82ac00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:44.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.057+0000 7f82b2ffd700 1 -- 172.21.15.46:0/38732158 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac0078d0 con 0x7f82bc108f00 2024-05-09T04:50:44.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.057+0000 7f82c3a7e700 1 -- 172.21.15.46:0/38732158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82bc072f90 con 0x7f82bc108f00 2024-05-09T04:50:44.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.057+0000 7f82b2ffd700 1 -- 172.21.15.46:0/38732158 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82ac00f450 con 0x7f82bc108f00 2024-05-09T04:50:44.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.057+0000 7f82b2ffd700 1 -- 172.21.15.46:0/38732158 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82ac018660 con 0x7f82bc108f00 2024-05-09T04:50:44.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.057+0000 7f82c3a7e700 1 -- 172.21.15.46:0/38732158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82bc073520 con 0x7f82bc108f00 2024-05-09T04:50:44.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.059+0000 7f82c3a7e700 1 -- 172.21.15.46:0/38732158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82bc105a80 con 0x7f82bc108f00 2024-05-09T04:50:44.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.061+0000 7f82b2ffd700 1 -- 172.21.15.46:0/38732158 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f82ac00f900 con 0x7f82bc108f00 2024-05-09T04:50:44.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.062+0000 7f82b2ffd700 1 --2- 172.21.15.46:0/38732158 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f82a80b2880 0x7f82a80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:44.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.062+0000 7f82c181a700 1 --2- 172.21.15.46:0/38732158 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f82a80b2880 0x7f82a80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:44.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.062+0000 7f82b2ffd700 1 -- 172.21.15.46:0/38732158 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f82ac0e20d0 con 0x7f82bc108f00 2024-05-09T04:50:44.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.063+0000 7f82c181a700 1 --2- 172.21.15.46:0/38732158 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f82a80b2880 0x7f82a80b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f82bc06fcc0 tx=0x7f82b800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:44.065 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.065+0000 7f82b2ffd700 1 -- 172.21.15.46:0/38732158 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f82ac014030 con 0x7f82bc108f00 2024-05-09T04:50:44.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.332+0000 7f82c3a7e700 1 -- 172.21.15.46:0/38732158 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82bc06f9a0 con 0x7f82bc108f00 2024-05-09T04:50:44.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.333+0000 7f82b2ffd700 1 -- 172.21.15.46:0/38732158 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) ==== 94+0+11862 (secure 0 0 0) 0x7f82ac0a9de0 con 0x7f82bc108f00 2024-05-09T04:50:44.333 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:50:44.333 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:48:11.653145+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24473,24503,24448]},"id":1}]} 2024-05-09T04:50:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.335+0000 7f82c3a7e700 1 -- 172.21.15.46:0/38732158 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f82a80b2880 msgr2=0x7f82a80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.335+0000 7f82c3a7e700 1 --2- 172.21.15.46:0/38732158 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f82a80b2880 0x7f82a80b4d90 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f82bc06fcc0 tx=0x7f82b800b410 comp rx=0 tx=0).stop 2024-05-09T04:50:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.335+0000 7f82c3a7e700 1 -- 172.21.15.46:0/38732158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc108f00 msgr2=0x7f82bc06e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.335+0000 7f82c3a7e700 1 --2- 172.21.15.46:0/38732158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc108f00 0x7f82bc06e470 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f82ac00c8f0 tx=0x7f82ac00cc00 comp rx=0 tx=0).stop 2024-05-09T04:50:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.335+0000 7f82c3a7e700 1 -- 172.21.15.46:0/38732158 shutdown_connections 2024-05-09T04:50:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.335+0000 7f82c3a7e700 1 --2- 172.21.15.46:0/38732158 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f82a80b2880 0x7f82a80b4d90 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.335+0000 7f82c3a7e700 1 --2- 172.21.15.46:0/38732158 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f82bc109850 0x7f82bc06ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.335+0000 7f82c3a7e700 1 --2- 172.21.15.46:0/38732158 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f82bc1045d0 0x7f82bc075340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.335+0000 7f82c3a7e700 1 --2- 172.21.15.46:0/38732158 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f82bc108f00 0x7f82bc06e470 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.335+0000 7f82c3a7e700 1 -- 172.21.15.46:0/38732158 >> 172.21.15.46:0/38732158 conn(0x7f82bc0ffea0 msgr2=0x7f82bc108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:44.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.335+0000 7f82c3a7e700 1 -- 172.21.15.46:0/38732158 shutdown_connections 2024-05-09T04:50:44.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:44.335+0000 7f82c3a7e700 1 -- 172.21.15.46:0/38732158 wait complete. 2024-05-09T04:50:44.336 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 24 2024-05-09T04:50:44.678 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:44 smithi046 ceph-mon[30136]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-09T04:50:44.678 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:44 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/38732158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:44.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:44 smithi072 ceph-mon[34088]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-09T04:50:44.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:44 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/38732158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:45.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:44 smithi110 ceph-mon[34904]: pgmap v382: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 5 op/s 2024-05-09T04:50:45.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:44 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/38732158' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:50:45.092 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:50:45.344 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:46.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:46 smithi046 ceph-mon[30136]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T04:50:46.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:46 smithi072 ceph-mon[34088]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T04:50:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:46 smithi110 ceph-mon[34904]: pgmap v383: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T04:50:47.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.158+0000 7fbf93491700 1 -- 172.21.15.46:0/1057314582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8c1034c0 msgr2=0x7fbf8c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:47.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.158+0000 7fbf93491700 1 --2- 172.21.15.46:0/1057314582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8c1034c0 0x7fbf8c1038d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fbf880099c0 tx=0x7fbf88009cd0 comp rx=0 tx=0).stop 2024-05-09T04:50:47.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.159+0000 7fbf93491700 1 -- 172.21.15.46:0/1057314582 shutdown_connections 2024-05-09T04:50:47.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.159+0000 7fbf93491700 1 --2- 172.21.15.46:0/1057314582 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf8c108740 0x7fbf8c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:47.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.159+0000 7fbf93491700 1 --2- 172.21.15.46:0/1057314582 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8c1034c0 0x7fbf8c1038d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:47.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.159+0000 7fbf93491700 1 --2- 172.21.15.46:0/1057314582 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf8c107df0 0x7fbf8c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:47.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.159+0000 7fbf93491700 1 -- 172.21.15.46:0/1057314582 >> 172.21.15.46:0/1057314582 conn(0x7fbf8c0fedc0 msgr2=0x7fbf8c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:47.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.159+0000 7fbf93491700 1 -- 172.21.15.46:0/1057314582 shutdown_connections 2024-05-09T04:50:47.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.159+0000 7fbf93491700 1 -- 172.21.15.46:0/1057314582 wait complete. 2024-05-09T04:50:47.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf93491700 1 Processor -- start 2024-05-09T04:50:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf93491700 1 -- start start 2024-05-09T04:50:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf93491700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8c1034c0 0x7fbf8c19aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf93491700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf8c107df0 0x7fbf8c19b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf93491700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf8c108740 0x7fbf8c19f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf93491700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c1144c0 con 0x7fbf8c1034c0 2024-05-09T04:50:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf93491700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c114340 con 0x7fbf8c107df0 2024-05-09T04:50:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf93491700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbf8c114640 con 0x7fbf8c108740 2024-05-09T04:50:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf9122d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8c1034c0 0x7fbf8c19aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf9122d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8c1034c0 0x7fbf8c19aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57866/0 (socket says 172.21.15.46:57866) 2024-05-09T04:50:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf9122d700 1 -- 172.21.15.46:0/426000209 learned_addr learned my addr 172.21.15.46:0/426000209 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf90a2c700 1 --2- 172.21.15.46:0/426000209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf8c107df0 0x7fbf8c19b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf91a2e700 1 --2- 172.21.15.46:0/426000209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf8c108740 0x7fbf8c19f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:47.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf9122d700 1 -- 172.21.15.46:0/426000209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf8c108740 msgr2=0x7fbf8c19f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:47.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf9122d700 1 --2- 172.21.15.46:0/426000209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf8c108740 0x7fbf8c19f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:47.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf9122d700 1 -- 172.21.15.46:0/426000209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf8c107df0 msgr2=0x7fbf8c19b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:47.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf9122d700 1 --2- 172.21.15.46:0/426000209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf8c107df0 0x7fbf8c19b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:47.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.160+0000 7fbf9122d700 1 -- 172.21.15.46:0/426000209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf8c19fda0 con 0x7fbf8c1034c0 2024-05-09T04:50:47.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.161+0000 7fbf9122d700 1 --2- 172.21.15.46:0/426000209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8c1034c0 0x7fbf8c19aed0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800b580 tx=0x7fbf880047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:47.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.161+0000 7fbf827fc700 1 -- 172.21.15.46:0/426000209 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88027070 con 0x7fbf8c1034c0 2024-05-09T04:50:47.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.161+0000 7fbf827fc700 1 -- 172.21.15.46:0/426000209 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf88003f20 con 0x7fbf8c1034c0 2024-05-09T04:50:47.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.161+0000 7fbf827fc700 1 -- 172.21.15.46:0/426000209 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf88022d70 con 0x7fbf8c1034c0 2024-05-09T04:50:47.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.161+0000 7fbf93491700 1 -- 172.21.15.46:0/426000209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf8c1a0030 con 0x7fbf8c1034c0 2024-05-09T04:50:47.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.161+0000 7fbf93491700 1 -- 172.21.15.46:0/426000209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf8c1a04c0 con 0x7fbf8c1034c0 2024-05-09T04:50:47.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.162+0000 7fbf93491700 1 -- 172.21.15.46:0/426000209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf8c0512c0 con 0x7fbf8c1034c0 2024-05-09T04:50:47.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.164+0000 7fbf827fc700 1 -- 172.21.15.46:0/426000209 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fbf88022500 con 0x7fbf8c1034c0 2024-05-09T04:50:47.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.165+0000 7fbf827fc700 1 --2- 172.21.15.46:0/426000209 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbf780b2880 0x7fbf780b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:47.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.165+0000 7fbf90a2c700 1 --2- 172.21.15.46:0/426000209 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbf780b2880 0x7fbf780b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:47.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.165+0000 7fbf827fc700 1 -- 172.21.15.46:0/426000209 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fbf880e3280 con 0x7fbf8c1034c0 2024-05-09T04:50:47.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.166+0000 7fbf90a2c700 1 --2- 172.21.15.46:0/426000209 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbf780b2880 0x7fbf780b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c1063e0 tx=0x7fbf7c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:47.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.169+0000 7fbf827fc700 1 -- 172.21.15.46:0/426000209 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbf88013070 con 0x7fbf8c1034c0 2024-05-09T04:50:47.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.382+0000 7fbf93491700 1 -- 172.21.15.46:0/426000209 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbf8c02e5f0 con 0x7fbf8c1034c0 2024-05-09T04:50:47.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.383+0000 7fbf827fc700 1 -- 172.21.15.46:0/426000209 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22685 (secure 0 0 0) 0x7fbf880aaff0 con 0x7fbf8c1034c0 2024-05-09T04:50:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:50:47.384 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"fsid":"ecf9ae2c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:25.633876+0000","modified":"2024-05-09T04:49:29.385454+0000","last_up_change":"2024-05-09T04:46:46.418597+0000","last_in_change":"2024-05-09T04:46:25.903582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:42:53.301400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:47:43.413569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:47:47.264915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d79cd3e7-6f82-4d60-96d9-26d8146c962f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6803","nonce":1576436508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6805","nonce":1576436508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6809","nonce":1576436508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6807","nonce":1576436508}]},"public_addr":"172.21.15.46:6803/1576436508","cluster_addr":"172.21.15.46:6805/1576436508","heartbeat_back_addr":"172.21.15.46:6809/1576436508","heartbeat_front_addr":"172.21.15.46:6807/1576436508","state":["exists","up"]},{"osd":1,"uuid":"698eba80-1ebd-4076-b6f9-953d2e7daf76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6801","nonce":379039775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6803","nonce":379039775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6807","nonce":379039775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6805","nonce":379039775}]},"public_addr":"172.21.15.72:6801/379039775","cluster_addr":"172.21.15.72:6803/379039775","heartbeat_back_addr":"172.21.15.72:6807/379039775","heartbeat_front_addr":"172.21.15.72:6805/379039775","state":["exists","up"]},{"osd":2,"uuid":"c549b131-cfa7-4724-926f-83e91a752d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6801","nonce":293447227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6803","nonce":293447227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6807","nonce":293447227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6805","nonce":293447227}]},"public_addr":"172.21.15.110:6801/293447227","cluster_addr":"172.21.15.110:6803/293447227","heartbeat_back_addr":"172.21.15.110:6807/293447227","heartbeat_front_addr":"172.21.15.110:6805/293447227","state":["exists","up"]},{"osd":3,"uuid":"549bb4f7-54d9-4cbc-ad40-7f3e717964f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6811","nonce":1115835653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6813","nonce":1115835653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6817","nonce":1115835653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6815","nonce":1115835653}]},"public_addr":"172.21.15.46:6811/1115835653","cluster_addr":"172.21.15.46:6813/1115835653","heartbeat_back_addr":"172.21.15.46:6817/1115835653","heartbeat_front_addr":"172.21.15.46:6815/1115835653","state":["exists","up"]},{"osd":4,"uuid":"444cd36b-6034-46cd-9a6b-29dce4ba6405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6809","nonce":59186693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6811","nonce":59186693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6815","nonce":59186693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6813","nonce":59186693}]},"public_addr":"172.21.15.72:6809/59186693","cluster_addr":"172.21.15.72:6811/59186693","heartbeat_back_addr":"172.21.15.72:6815/59186693","heartbeat_front_addr":"172.21.15.72:6813/59186693","state":["exists","up"]},{"osd":5,"uuid":"2c2bdd2f-43cf-4208-9ae1-f912807c84e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6809","nonce":1712162502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6811","nonce":1712162502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6815","nonce":1712162502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6813","nonce":1712162502}]},"public_addr":"172.21.15.110:6809/1712162502","cluster_addr":"172.21.15.110:6811/1712162502","heartbeat_back_addr":"172.21.15.110:6815/1712162502","heartbeat_front_addr":"172.21.15.110:6813/1712162502","state":["exists","up"]},{"osd":6,"uuid":"beb0b87f-c3a4-40dd-b224-1843e9301ca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6819","nonce":2429650065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6821","nonce":2429650065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6825","nonce":2429650065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6823","nonce":2429650065}]},"public_addr":"172.21.15.46:6819/2429650065","cluster_addr":"172.21.15.46:6821/2429650065","heartbeat_back_addr":"172.21.15.46:6825/2429650065","heartbeat_front_addr":"172.21.15.46:6823/2429650065","state":["exists","up"]},{"osd":7,"uuid":"4a634675-fad7-4a43-840c-8dec2631dec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6817","nonce":337796672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6819","nonce":337796672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6823","nonce":337796672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6821","nonce":337796672}]},"public_addr":"172.21.15.72:6817/337796672","cluster_addr":"172.21.15.72:6819/337796672","heartbeat_back_addr":"172.21.15.72:6823/337796672","heartbeat_front_addr":"172.21.15.72:6821/337796672","state":["exists","up"]},{"osd":8,"uuid":"01b5f638-90f0-4cf4-8042-20d20a725669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6817","nonce":189513203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6819","nonce":189513203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6823","nonce":189513203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6821","nonce":189513203}]},"public_addr":"172.21.15.110:6817/189513203","cluster_addr":"172.21.15.110:6819/189513203","heartbeat_back_addr":"172.21.15.110:6823/189513203","heartbeat_front_addr":"172.21.15.110:6821/189513203","state":["exists","up"]},{"osd":9,"uuid":"e423de41-50ec-48f4-a4ed-216de33c4fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6827","nonce":1762262822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6829","nonce":1762262822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6833","nonce":1762262822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6831","nonce":1762262822}]},"public_addr":"172.21.15.46:6827/1762262822","cluster_addr":"172.21.15.46:6829/1762262822","heartbeat_back_addr":"172.21.15.46:6833/1762262822","heartbeat_front_addr":"172.21.15.46:6831/1762262822","state":["exists","up"]},{"osd":10,"uuid":"f594d004-9f80-4508-b127-fb693d355f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6825","nonce":38453898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6827","nonce":38453898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6831","nonce":38453898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6829","nonce":38453898}]},"public_addr":"172.21.15.72:6825/38453898","cluster_addr":"172.21.15.72:6827/38453898","heartbeat_back_addr":"172.21.15.72:6831/38453898","heartbeat_front_addr":"172.21.15.72:6829/38453898","state":["exists","up"]},{"osd":11,"uuid":"f771d29f-89e3-4f96-9c29-00e424230c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6825","nonce":4153529632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6827","nonce":4153529632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6831","nonce":4153529632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6829","nonce":4153529632}]},"public_addr":"172.21.15.110:6825/4153529632","cluster_addr":"172.21.15.110:6827/4153529632","heartbeat_back_addr":"172.21.15.110:6831/4153529632","heartbeat_front_addr":"172.21.15.110:6829/4153529632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:02.615801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:26.453477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:50.334627+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:15.357320+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:40.403549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:06.030093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:31.904091+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:57.616372+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:24.045975+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:51.160898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:17.597845+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:44.636629+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1596672353":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1846482551":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/1069571878":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6800/2738158981":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/3625745146":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6801/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/564539954":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1797199863":"2024-05-10T04:40:27.201119+0000","172.21.15.46:6801/2738158981":"2024-05-10T04:39:59.515628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:50:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.385+0000 7fbf93491700 1 -- 172.21.15.46:0/426000209 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbf780b2880 msgr2=0x7fbf780b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.385+0000 7fbf93491700 1 --2- 172.21.15.46:0/426000209 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbf780b2880 0x7fbf780b4d90 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fbf8c1063e0 tx=0x7fbf7c00a300 comp rx=0 tx=0).stop 2024-05-09T04:50:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.385+0000 7fbf93491700 1 -- 172.21.15.46:0/426000209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8c1034c0 msgr2=0x7fbf8c19aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.385+0000 7fbf93491700 1 --2- 172.21.15.46:0/426000209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8c1034c0 0x7fbf8c19aed0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800b580 tx=0x7fbf880047b0 comp rx=0 tx=0).stop 2024-05-09T04:50:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.386+0000 7fbf93491700 1 -- 172.21.15.46:0/426000209 shutdown_connections 2024-05-09T04:50:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.386+0000 7fbf93491700 1 --2- 172.21.15.46:0/426000209 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fbf780b2880 0x7fbf780b4d90 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.386+0000 7fbf93491700 1 --2- 172.21.15.46:0/426000209 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbf8c108740 0x7fbf8c19f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.386+0000 7fbf93491700 1 --2- 172.21.15.46:0/426000209 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbf8c1034c0 0x7fbf8c19aed0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:47.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.386+0000 7fbf93491700 1 --2- 172.21.15.46:0/426000209 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbf8c107df0 0x7fbf8c19b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:47.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.386+0000 7fbf93491700 1 -- 172.21.15.46:0/426000209 >> 172.21.15.46:0/426000209 conn(0x7fbf8c0fedc0 msgr2=0x7fbf8c101db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:47.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.386+0000 7fbf93491700 1 -- 172.21.15.46:0/426000209 shutdown_connections 2024-05-09T04:50:47.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:47.386+0000 7fbf93491700 1 -- 172.21.15.46:0/426000209 wait complete. 2024-05-09T04:50:47.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:47 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/426000209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:47.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:47 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/426000209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:48.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:47 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/426000209' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:50:48.191 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-09T04:47:50.321554+0000', 'modified': '2024-05-09T04:48:11.653145+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24503, 'mds_1': 24473, 'mds_2': 24448}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24427': {'gid': 24427, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6835/2299300184', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6834', 'nonce': 2299300184}, {'type': 'v1', 'addr': '172.21.15.110:6835', 'nonce': 2299300184}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.110:6837/2612088705', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6836', 'nonce': 2612088705}, {'type': 'v1', 'addr': '172.21.15.110:6837', 'nonce': 2612088705}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.72:6835/4256517546', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6834', 'nonce': 4256517546}, {'type': 'v1', 'addr': '172.21.15.72:6835', 'nonce': 4256517546}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'i', 'rank': 2, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.110:6839/556391603', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.110:6838', 'nonce': 556391603}, {'type': 'v1', 'addr': '172.21.15.110:6839', 'nonce': 556391603}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'h', 'rank': 1, 'incarnation': 18, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.72:6837/3365501578', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6836', 'nonce': 3365501578}, {'type': 'v1', 'addr': '172.21.15.72:6837', 'nonce': 3365501578}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24503': {'gid': 24503, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.72:6839/2905856798', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.72:6838', 'nonce': 2905856798}, {'type': 'v1', 'addr': '172.21.15.72:6839', 'nonce': 2905856798}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24503, 'qdb_cluster': [24473, 24503, 24448]} 2024-05-09T04:50:48.191 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-05-09T04:50:48.191 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-09T04:50:48.450 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:48.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:48 smithi046 ceph-mon[30136]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-09T04:50:48.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:48 smithi072 ceph-mon[34088]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-09T04:50:49.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:48 smithi110 ceph-mon[34904]: pgmap v384: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-09T04:50:50.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.336+0000 7f15050e3700 1 -- 172.21.15.46:0/280255073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1500108f20 msgr2=0x7f1500109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:50.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.336+0000 7f15050e3700 1 --2- 172.21.15.46:0/280255073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1500108f20 0x7f1500109330 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f14f40099c0 tx=0x7f14f4009cd0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.336+0000 7f15050e3700 1 -- 172.21.15.46:0/280255073 shutdown_connections 2024-05-09T04:50:50.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.336+0000 7f15050e3700 1 --2- 172.21.15.46:0/280255073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1500109870 0x7f15001111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.336+0000 7f15050e3700 1 --2- 172.21.15.46:0/280255073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15001045f0 0x7f1500104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.336+0000 7f15050e3700 1 --2- 172.21.15.46:0/280255073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1500108f20 0x7f1500109330 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.336+0000 7f15050e3700 1 -- 172.21.15.46:0/280255073 >> 172.21.15.46:0/280255073 conn(0x7f1500100040 msgr2=0x7f1500102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:50.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.336+0000 7f15050e3700 1 -- 172.21.15.46:0/280255073 shutdown_connections 2024-05-09T04:50:50.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.336+0000 7f15050e3700 1 -- 172.21.15.46:0/280255073 wait complete. 2024-05-09T04:50:50.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.337+0000 7f15050e3700 1 Processor -- start 2024-05-09T04:50:50.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.337+0000 7f15050e3700 1 -- start start 2024-05-09T04:50:50.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.337+0000 7f15050e3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15001045f0 0x7f150019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f15050e3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1500108f20 0x7f150019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f15050e3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1500109870 0x7f150019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f15050e3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1500114480 con 0x7f1500109870 2024-05-09T04:50:50.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f15050e3700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1500114300 con 0x7f1500108f20 2024-05-09T04:50:50.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f15050e3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1500114600 con 0x7f15001045f0 2024-05-09T04:50:50.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f14fe59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1500108f20 0x7f150019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f14ff59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1500109870 0x7f150019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f14ff59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1500109870 0x7f150019f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57884/0 (socket says 172.21.15.46:57884) 2024-05-09T04:50:50.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f14fe59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1500108f20 0x7f150019b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:47320/0 (socket says 172.21.15.46:47320) 2024-05-09T04:50:50.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f14ff59e700 1 -- 172.21.15.46:0/4097588576 learned_addr learned my addr 172.21.15.46:0/4097588576 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:50.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f14fed9d700 1 --2- 172.21.15.46:0/4097588576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15001045f0 0x7f150019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f14fe59c700 1 -- 172.21.15.46:0/4097588576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15001045f0 msgr2=0x7f150019aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:50.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f14fe59c700 1 --2- 172.21.15.46:0/4097588576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15001045f0 0x7f150019aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f14fe59c700 1 -- 172.21.15.46:0/4097588576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1500109870 msgr2=0x7f150019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:50.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f14fe59c700 1 --2- 172.21.15.46:0/4097588576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1500109870 0x7f150019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f14fe59c700 1 -- 172.21.15.46:0/4097588576 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f150019fda0 con 0x7f1500108f20 2024-05-09T04:50:50.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f14ff59e700 1 --2- 172.21.15.46:0/4097588576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1500109870 0x7f150019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:50.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.338+0000 7f14fe59c700 1 --2- 172.21.15.46:0/4097588576 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1500108f20 0x7f150019b470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f14f40098e0 tx=0x7f14f40045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:50.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.339+0000 7f14e7fff700 1 -- 172.21.15.46:0/4097588576 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4026070 con 0x7f1500108f20 2024-05-09T04:50:50.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.339+0000 7f15050e3700 1 -- 172.21.15.46:0/4097588576 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15001a0030 con 0x7f1500108f20 2024-05-09T04:50:50.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.339+0000 7f14e7fff700 1 -- 172.21.15.46:0/4097588576 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14f400f460 con 0x7f1500108f20 2024-05-09T04:50:50.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.339+0000 7f14e7fff700 1 -- 172.21.15.46:0/4097588576 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f4009e60 con 0x7f1500108f20 2024-05-09T04:50:50.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.339+0000 7f15050e3700 1 -- 172.21.15.46:0/4097588576 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15001a0510 con 0x7f1500108f20 2024-05-09T04:50:50.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.342+0000 7f14e7fff700 1 -- 172.21.15.46:0/4097588576 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f14f4029070 con 0x7f1500108f20 2024-05-09T04:50:50.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.342+0000 7f1513af9b80 1 Processor -- start 2024-05-09T04:50:50.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.342+0000 7f14e7fff700 1 --2- 172.21.15.46:0/4097588576 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f14e80b2880 0x7f14e80b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.342+0000 7f1513af9b80 1 -- start start 2024-05-09T04:50:50.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f14e7fff700 1 -- 172.21.15.46:0/4097588576 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f14f40e2960 con 0x7f1500108f20 2024-05-09T04:50:50.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f14ff59e700 1 --2- 172.21.15.46:0/4097588576 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f14e80b2880 0x7f14e80b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f1513af9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd192cb0 0x55a5dd2a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f1513af9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 0x55a5dd2a3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f1513af9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd29dbc0 0x55a5dd29dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f1513af9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd1b8050 con 0x55a5dd29dbc0 2024-05-09T04:50:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f1513af9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd15d950 con 0x55a5dd118450 2024-05-09T04:50:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f1513af9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd1b4010 con 0x55a5dd192cb0 2024-05-09T04:50:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f14fe59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 0x55a5dd2a3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f14fe59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 0x55a5dd2a3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:47330/0 (socket says 172.21.15.46:47330) 2024-05-09T04:50:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f14fe59c700 1 -- 172.21.15.46:0/734171222 learned_addr learned my addr 172.21.15.46:0/734171222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f14ff59e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd29dbc0 0x55a5dd29dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f14fed9d700 1 --2- 172.21.15.46:0/734171222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd192cb0 0x55a5dd2a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f14fe59c700 1 -- 172.21.15.46:0/734171222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd192cb0 msgr2=0x55a5dd2a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f14fe59c700 1 --2- 172.21.15.46:0/734171222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd192cb0 0x55a5dd2a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f14fe59c700 1 -- 172.21.15.46:0/734171222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd29dbc0 msgr2=0x55a5dd29dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f14fe59c700 1 --2- 172.21.15.46:0/734171222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd29dbc0 0x55a5dd29dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f14fe59c700 1 -- 172.21.15.46:0/734171222 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5dd14e4b0 con 0x55a5dd118450 2024-05-09T04:50:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f14fed9d700 1 --2- 172.21.15.46:0/734171222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd192cb0 0x55a5dd2a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:50:50.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.343+0000 7f14ff59e700 1 --2- 172.21.15.46:0/734171222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd29dbc0 0x55a5dd29dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:50.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.344+0000 7f14fe59c700 1 --2- 172.21.15.46:0/734171222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 0x55a5dd2a3f70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f14f40a6050 tx=0x7f14f40a99f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb229fcdefcbfc67 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:50.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.344+0000 7f14ff59e700 1 --2- 172.21.15.46:0/4097588576 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f14e80b2880 0x7f14e80b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009550 tx=0x7f14f80091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:50.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.344+0000 7f14e57fa700 1 -- 172.21.15.46:0/734171222 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f400f910 con 0x55a5dd118450 2024-05-09T04:50:50.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.344+0000 7f14e57fa700 1 -- 172.21.15.46:0/734171222 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14f40a9ae0 con 0x55a5dd118450 2024-05-09T04:50:50.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.344+0000 7f14e57fa700 1 -- 172.21.15.46:0/734171222 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f40aa470 con 0x55a5dd118450 2024-05-09T04:50:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.345+0000 7f1513af9b80 1 -- 172.21.15.46:0/734171222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 msgr2=0x55a5dd2a3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.345+0000 7f1513af9b80 1 --2- 172.21.15.46:0/734171222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 0x55a5dd2a3f70 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f14f40a6050 tx=0x7f14f40a99f0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.345+0000 7f1513af9b80 1 -- 172.21.15.46:0/734171222 shutdown_connections 2024-05-09T04:50:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.345+0000 7f1513af9b80 1 --2- 172.21.15.46:0/734171222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd29dbc0 0x55a5dd29dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.345+0000 7f1513af9b80 1 --2- 172.21.15.46:0/734171222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd192cb0 0x55a5dd2a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.345+0000 7f1513af9b80 1 --2- 172.21.15.46:0/734171222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 0x55a5dd2a3f70 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.345+0000 7f1513af9b80 1 -- 172.21.15.46:0/734171222 >> 172.21.15.46:0/734171222 conn(0x55a5dd11bf20 msgr2=0x55a5dd2901d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.345+0000 7f1513af9b80 1 -- 172.21.15.46:0/734171222 shutdown_connections 2024-05-09T04:50:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.345+0000 7f1513af9b80 1 -- 172.21.15.46:0/734171222 wait complete. 2024-05-09T04:50:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.345+0000 7f1513af9b80 1 -- start start 2024-05-09T04:50:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f1513af9b80 1 Processor -- start 2024-05-09T04:50:50.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f1513af9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd118450 0x55a5dd2ad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f1513af9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd192cb0 0x55a5dd2ad7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f1513af9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd29dbc0 0x55a5dd2a7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f1513af9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd29e770 con 0x55a5dd118450 2024-05-09T04:50:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f1513af9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd152ad0 con 0x55a5dd192cb0 2024-05-09T04:50:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f1513af9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd29e8f0 con 0x55a5dd29dbc0 2024-05-09T04:50:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f1513af9b80 10 client.0 _ll_register_callbacks cb 0x55a5dd029670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:50:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f14fed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd118450 0x55a5dd2ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f14fed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd118450 0x55a5dd2ad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57896/0 (socket says 172.21.15.46:57896) 2024-05-09T04:50:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f14fed9d700 1 -- 172.21.15.46:0/4129100132 learned_addr learned my addr 172.21.15.46:0/4129100132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f14fe59c700 1 --2- 172.21.15.46:0/4129100132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd192cb0 0x55a5dd2ad7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f14ff59e700 1 --2- 172.21.15.46:0/4129100132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd29dbc0 0x55a5dd2a7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f14fed9d700 1 -- 172.21.15.46:0/4129100132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd29dbc0 msgr2=0x55a5dd2a7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:50.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f14fed9d700 1 --2- 172.21.15.46:0/4129100132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd29dbc0 0x55a5dd2a7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f14fed9d700 1 -- 172.21.15.46:0/4129100132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd192cb0 msgr2=0x55a5dd2ad7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.346+0000 7f14fed9d700 1 --2- 172.21.15.46:0/4129100132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd192cb0 0x55a5dd2ad7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.347+0000 7f14fed9d700 1 -- 172.21.15.46:0/4129100132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5dd033440 con 0x55a5dd118450 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.347+0000 7f14fed9d700 1 --2- 172.21.15.46:0/4129100132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd118450 0x55a5dd2ad260 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f14f0013780 tx=0x7f14f0013a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44b4ff787b379c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.347+0000 7f14e2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.347+0000 7f14e2ff5700 1 -- 172.21.15.46:0/4129100132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f0020060 con 0x55a5dd118450 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.347+0000 7f1513af9b80 1 -- 172.21.15.46:0/4129100132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14f40a9ae0 con 0x55a5dd118450 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.347+0000 7f14e2ff5700 1 -- 172.21.15.46:0/4129100132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f14f00118b0 con 0x55a5dd118450 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.347+0000 7f14e2ff5700 1 -- 172.21.15.46:0/4129100132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f14f0011ab0 con 0x55a5dd118450 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.347+0000 7f1513af9b80 10 client.15084 fetch_fsmap learned FSMap version 24 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.347+0000 7f1513af9b80 1 -- 172.21.15.46:0/4129100132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a5dd2b4e50 con 0x55a5dd118450 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.347+0000 7f14e2ff5700 1 -- 172.21.15.46:0/4129100132 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14f40a9ae0 con 0x55a5dd118450 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.348+0000 7f14e2ff5700 1 -- 172.21.15.46:0/4129100132 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f14f000f3c0 con 0x55a5dd118450 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.348+0000 7f1513af9b80 10 client.15084 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.348+0000 7f1513af9b80 10 client.15084 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T04:50:50.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.348+0000 7f1513af9b80 20 client.15084 populate_metadata read hostname 'smithi046' 2024-05-09T04:50:50.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.348+0000 7f1513af9b80 1 --2- 172.21.15.46:0/4129100132 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a5dd2b7460 0x55a5dd2b88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.348+0000 7f1513af9b80 4 client.15084 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:50:50.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.348+0000 7f1513af9b80 1 -- 172.21.15.46:0/4129100132 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5dd1c5a30 con 0x55a5dd2b7460 2024-05-09T04:50:50.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.348+0000 7f14fe59c700 1 --2- 172.21.15.46:0/4129100132 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a5dd2b7460 0x55a5dd2b88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.348+0000 7f14fe59c700 1 --2- 172.21.15.46:0/4129100132 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a5dd2b7460 0x55a5dd2b88d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d9f3da37c92b2c3e server_cookie=5170ddea4c307f6c in_seq=0 out_seq=0 2024-05-09T04:50:50.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.348+0000 7f14e2ff5700 10 client.15084 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:50:50.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.349+0000 7f14e2ff5700 1 -- 172.21.15.46:0/4129100132 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a5dd1c5a30 con 0x55a5dd2b7460 2024-05-09T04:50:50.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.349+0000 7f14e2ff5700 10 client.15084 handle_command_reply: tid=0 2024-05-09T04:50:50.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.349+0000 7f1513af9b80 1 client.15084 shutdown 2024-05-09T04:50:50.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.349+0000 7f1513af9b80 1 -- 172.21.15.46:0/4129100132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd118450 msgr2=0x55a5dd2ad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:50.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.349+0000 7f1513af9b80 1 --2- 172.21.15.46:0/4129100132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd118450 0x55a5dd2ad260 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f14f0013780 tx=0x7f14f0013a90 comp rx=0 tx=0).stop 2024-05-09T04:50:50.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.350+0000 7f1513af9b80 1 -- 172.21.15.46:0/4129100132 shutdown_connections 2024-05-09T04:50:50.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.350+0000 7f1513af9b80 1 --2- 172.21.15.46:0/4129100132 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a5dd2b7460 0x55a5dd2b88d0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.350+0000 7f1513af9b80 1 --2- 172.21.15.46:0/4129100132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd29dbc0 0x55a5dd2a7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.350+0000 7f1513af9b80 1 --2- 172.21.15.46:0/4129100132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd118450 0x55a5dd2ad260 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.350+0000 7f1513af9b80 1 --2- 172.21.15.46:0/4129100132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd192cb0 0x55a5dd2ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.350+0000 7f1513af9b80 1 -- 172.21.15.46:0/4129100132 >> 172.21.15.46:0/4129100132 conn(0x55a5dd11bf20 msgr2=0x55a5dd2943a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:50.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.350+0000 7f14e2ff5700 0 client.15084 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:50:50.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.350+0000 7f1513af9b80 1 -- 172.21.15.46:0/4129100132 shutdown_connections 2024-05-09T04:50:50.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.350+0000 7f1513af9b80 1 -- 172.21.15.46:0/4129100132 wait complete. 2024-05-09T04:50:50.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.350+0000 7f1513af9b80 20 client.15084 trim_cache size 0 max 16384 2024-05-09T04:50:50.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.372+0000 7f1513af9b80 1 Processor -- start 2024-05-09T04:50:50.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.372+0000 7f1513af9b80 1 -- start start 2024-05-09T04:50:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.372+0000 7f1513af9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 0x55a5dd2ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.372+0000 7f1513af9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd192cb0 0x55a5dd2a7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.372+0000 7f1513af9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd29dbc0 0x55a5dd2a2860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.372+0000 7f1513af9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd2b6c10 con 0x55a5dd192cb0 2024-05-09T04:50:50.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.372+0000 7f1513af9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd2b72e0 con 0x55a5dd118450 2024-05-09T04:50:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.372+0000 7f1513af9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd2b79b0 con 0x55a5dd29dbc0 2024-05-09T04:50:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.372+0000 7f14fe59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd192cb0 0x55a5dd2a7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.372+0000 7f14fe59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd192cb0 0x55a5dd2a7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57908/0 (socket says 172.21.15.46:57908) 2024-05-09T04:50:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.372+0000 7f14fe59c700 1 -- 172.21.15.46:0/3128000231 learned_addr learned my addr 172.21.15.46:0/3128000231 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.373+0000 7f14ff59e700 1 --2- 172.21.15.46:0/3128000231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd29dbc0 0x55a5dd2a2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.373+0000 7f14fed9d700 1 --2- 172.21.15.46:0/3128000231 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 0x55a5dd2ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.373+0000 7f14fe59c700 1 -- 172.21.15.46:0/3128000231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd29dbc0 msgr2=0x55a5dd2a2860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.373+0000 7f14fe59c700 1 --2- 172.21.15.46:0/3128000231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd29dbc0 0x55a5dd2a2860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.373+0000 7f14fe59c700 1 -- 172.21.15.46:0/3128000231 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 msgr2=0x55a5dd2ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.373+0000 7f14fe59c700 1 --2- 172.21.15.46:0/3128000231 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 0x55a5dd2ad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.373+0000 7f14fe59c700 1 -- 172.21.15.46:0/3128000231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5dd28c980 con 0x55a5dd192cb0 2024-05-09T04:50:50.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.373+0000 7f14fe59c700 1 --2- 172.21.15.46:0/3128000231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd192cb0 0x55a5dd2a7620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f14f40e0470 tx=0x7f14f40094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7817554aa79e787 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:50.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.373+0000 7f14e4ff9700 1 -- 172.21.15.46:0/3128000231 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f40b2da0 con 0x55a5dd192cb0 2024-05-09T04:50:50.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.373+0000 7f14e4ff9700 1 -- 172.21.15.46:0/3128000231 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14f40ae380 con 0x55a5dd192cb0 2024-05-09T04:50:50.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.373+0000 7f14e4ff9700 1 -- 172.21.15.46:0/3128000231 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f40a5940 con 0x55a5dd192cb0 2024-05-09T04:50:50.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.374+0000 7f1513af9b80 1 -- 172.21.15.46:0/3128000231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd192cb0 msgr2=0x55a5dd2a7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.374+0000 7f1513af9b80 1 --2- 172.21.15.46:0/3128000231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd192cb0 0x55a5dd2a7620 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f14f40e0470 tx=0x7f14f40094b0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.374+0000 7f1513af9b80 1 -- 172.21.15.46:0/3128000231 shutdown_connections 2024-05-09T04:50:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.374+0000 7f1513af9b80 1 --2- 172.21.15.46:0/3128000231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd29dbc0 0x55a5dd2a2860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.374+0000 7f1513af9b80 1 --2- 172.21.15.46:0/3128000231 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 0x55a5dd2ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.374+0000 7f1513af9b80 1 --2- 172.21.15.46:0/3128000231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd192cb0 0x55a5dd2a7620 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.374+0000 7f1513af9b80 1 -- 172.21.15.46:0/3128000231 >> 172.21.15.46:0/3128000231 conn(0x55a5dd11bf20 msgr2=0x55a5dd2a6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.374+0000 7f1513af9b80 1 -- 172.21.15.46:0/3128000231 shutdown_connections 2024-05-09T04:50:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.374+0000 7f1513af9b80 1 -- 172.21.15.46:0/3128000231 wait complete. 2024-05-09T04:50:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.375+0000 7f1513af9b80 1 -- start start 2024-05-09T04:50:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.375+0000 7f1513af9b80 1 Processor -- start 2024-05-09T04:50:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.375+0000 7f1513af9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 0x55a5dd294e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.375+0000 7f1513af9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd192cb0 0x55a5dd2aff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.375+0000 7f1513af9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd29dbc0 0x55a5dd2b0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.375+0000 7f1513af9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd2b8750 con 0x55a5dd192cb0 2024-05-09T04:50:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.375+0000 7f1513af9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd2b8080 con 0x55a5dd118450 2024-05-09T04:50:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.375+0000 7f1513af9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a5dd2b8e20 con 0x55a5dd29dbc0 2024-05-09T04:50:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.375+0000 7f1513af9b80 10 client.0 _ll_register_callbacks cb 0x55a5dd029670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:50:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.375+0000 7f14fe59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd192cb0 0x55a5dd2aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.375+0000 7f14fe59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd192cb0 0x55a5dd2aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57912/0 (socket says 172.21.15.46:57912) 2024-05-09T04:50:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.375+0000 7f14fe59c700 1 -- 172.21.15.46:0/4003431266 learned_addr learned my addr 172.21.15.46:0/4003431266 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.375+0000 7f14fed9d700 1 --2- 172.21.15.46:0/4003431266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 0x55a5dd294e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.376+0000 7f14fe59c700 1 -- 172.21.15.46:0/4003431266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd29dbc0 msgr2=0x55a5dd2b0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.376+0000 7f14fe59c700 1 --2- 172.21.15.46:0/4003431266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd29dbc0 0x55a5dd2b0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.376+0000 7f14fe59c700 1 -- 172.21.15.46:0/4003431266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 msgr2=0x55a5dd294e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.376+0000 7f14fe59c700 1 --2- 172.21.15.46:0/4003431266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 0x55a5dd294e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.376+0000 7f14fe59c700 1 -- 172.21.15.46:0/4003431266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5dd290ef0 con 0x55a5dd192cb0 2024-05-09T04:50:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.376+0000 7f14fe59c700 1 --2- 172.21.15.46:0/4003431266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd192cb0 0x55a5dd2aff00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f14f40a9940 tx=0x7f14f40aa820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=523dd3d16944ca04 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.376+0000 7f14e2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:50:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.376+0000 7f14e2ff5700 1 -- 172.21.15.46:0/4003431266 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14f40b2da0 con 0x55a5dd192cb0 2024-05-09T04:50:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.376+0000 7f14e2ff5700 1 -- 172.21.15.46:0/4003431266 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f14f40b2b20 con 0x55a5dd192cb0 2024-05-09T04:50:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.376+0000 7f1513af9b80 1 -- 172.21.15.46:0/4003431266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14f40ae380 con 0x55a5dd192cb0 2024-05-09T04:50:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.376+0000 7f14e2ff5700 1 -- 172.21.15.46:0/4003431266 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7f14f40aa230 con 0x55a5dd192cb0 2024-05-09T04:50:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.376+0000 7f1513af9b80 10 client.15090 fetch_fsmap learned FSMap version 24 2024-05-09T04:50:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.376+0000 7f1513af9b80 1 -- 172.21.15.46:0/4003431266 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55a5dd322640 con 0x55a5dd192cb0 2024-05-09T04:50:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.376+0000 7f14e2ff5700 1 -- 172.21.15.46:0/4003431266 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14f40ae380 con 0x55a5dd192cb0 2024-05-09T04:50:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.377+0000 7f14e2ff5700 1 -- 172.21.15.46:0/4003431266 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7f14f40a8e10 con 0x55a5dd192cb0 2024-05-09T04:50:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.377+0000 7f1513af9b80 10 client.15090 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T04:50:50.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.377+0000 7f1513af9b80 10 client.15090 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-09T04:50:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.377+0000 7f1513af9b80 20 client.15090 populate_metadata read hostname 'smithi046' 2024-05-09T04:50:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.377+0000 7f1513af9b80 1 --2- 172.21.15.46:0/4003431266 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a5dd324660 0x55a5dd325ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.377+0000 7f1513af9b80 4 client.15090 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:50:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.377+0000 7f1513af9b80 1 -- 172.21.15.46:0/4003431266 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "status"}) -- 0x55a5dd1bfad0 con 0x55a5dd324660 2024-05-09T04:50:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.377+0000 7f14fed9d700 1 --2- 172.21.15.46:0/4003431266 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a5dd324660 0x55a5dd325ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f14e2ff5700 10 client.15090 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:50:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f14fed9d700 1 --2- 172.21.15.46:0/4003431266 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a5dd324660 0x55a5dd325ad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a10b485253479015 server_cookie=6f60073dd5788c69 in_seq=0 out_seq=0 2024-05-09T04:50:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f14e2ff5700 1 -- 172.21.15.46:0/4003431266 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55a5dd1bfad0 con 0x55a5dd324660 2024-05-09T04:50:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f14e2ff5700 10 client.15090 handle_command_reply: tid=0 2024-05-09T04:50:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f1513af9b80 1 client.15090 shutdown 2024-05-09T04:50:50.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f1513af9b80 1 -- 172.21.15.46:0/4003431266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd192cb0 msgr2=0x55a5dd2aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f1513af9b80 1 --2- 172.21.15.46:0/4003431266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd192cb0 0x55a5dd2aff00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f14f40a9940 tx=0x7f14f40aa820 comp rx=0 tx=0).stop 2024-05-09T04:50:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f1513af9b80 1 -- 172.21.15.46:0/4003431266 shutdown_connections 2024-05-09T04:50:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f1513af9b80 1 --2- 172.21.15.46:0/4003431266 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a5dd324660 0x55a5dd325ad0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f1513af9b80 1 --2- 172.21.15.46:0/4003431266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a5dd29dbc0 0x55a5dd2b0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f1513af9b80 1 --2- 172.21.15.46:0/4003431266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a5dd118450 0x55a5dd294e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f14e2ff5700 0 client.15090 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:50:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f1513af9b80 1 --2- 172.21.15.46:0/4003431266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a5dd192cb0 0x55a5dd2aff00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f1513af9b80 1 -- 172.21.15.46:0/4003431266 >> 172.21.15.46:0/4003431266 conn(0x55a5dd11bf20 msgr2=0x55a5dd2a1460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f1513af9b80 1 -- 172.21.15.46:0/4003431266 shutdown_connections 2024-05-09T04:50:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f1513af9b80 1 -- 172.21.15.46:0/4003431266 wait complete. 2024-05-09T04:50:50.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.378+0000 7f1513af9b80 20 client.15090 trim_cache size 0 max 16384 2024-05-09T04:50:50.384 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:50:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ecf9ae2c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:50:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 2, 2024-05-09T04:50:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24448, 2024-05-09T04:50:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-09T04:50:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-09T04:50:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-09T04:50:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 164.91005463100001, 2024-05-09T04:50:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 24, 2024-05-09T04:50:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 84, 2024-05-09T04:50:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 80, 2024-05-09T04:50:50.384 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 204.712702576 2024-05-09T04:50:50.385 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:50:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.379+0000 7f15050e3700 1 -- 172.21.15.46:0/4097588576 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f14e80b2880 msgr2=0x7f14e80b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.379+0000 7f15050e3700 1 --2- 172.21.15.46:0/4097588576 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f14e80b2880 0x7f14e80b4d90 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f14f8009550 tx=0x7f14f80091b0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.379+0000 7f15050e3700 1 -- 172.21.15.46:0/4097588576 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1500108f20 msgr2=0x7f150019b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.379+0000 7f15050e3700 1 --2- 172.21.15.46:0/4097588576 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1500108f20 0x7f150019b470 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f14f40098e0 tx=0x7f14f40045e0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.379+0000 7f15050e3700 1 -- 172.21.15.46:0/4097588576 shutdown_connections 2024-05-09T04:50:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.379+0000 7f15050e3700 1 --2- 172.21.15.46:0/4097588576 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f14e80b2880 0x7f14e80b4d90 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.379+0000 7f15050e3700 1 --2- 172.21.15.46:0/4097588576 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1500109870 0x7f150019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.379+0000 7f15050e3700 1 --2- 172.21.15.46:0/4097588576 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15001045f0 0x7f150019aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.379+0000 7f15050e3700 1 --2- 172.21.15.46:0/4097588576 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1500108f20 0x7f150019b470 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.379+0000 7f15050e3700 1 -- 172.21.15.46:0/4097588576 >> 172.21.15.46:0/4097588576 conn(0x7f1500100040 msgr2=0x7f150010a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.379+0000 7f15050e3700 1 -- 172.21.15.46:0/4097588576 shutdown_connections 2024-05-09T04:50:50.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:50.379+0000 7f15050e3700 1 -- 172.21.15.46:0/4097588576 wait complete. 2024-05-09T04:50:50.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:50 smithi046 ceph-mon[30136]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-09T04:50:50.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:50 smithi072 ceph-mon[34088]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-09T04:50:51.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:50 smithi110 ceph-mon[34904]: pgmap v385: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 6 op/s 2024-05-09T04:50:51.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-09T04:50:51.457 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:52.875 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:52 smithi046 ceph-mon[30136]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T04:50:52.879 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:52 smithi072 ceph-mon[34088]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T04:50:53.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:52 smithi110 ceph-mon[34904]: pgmap v386: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 5 op/s 2024-05-09T04:50:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.290+0000 7fc7bc7b2700 1 -- 172.21.15.46:0/304575668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b41045f0 msgr2=0x7fc7b4104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.290+0000 7fc7bc7b2700 1 --2- 172.21.15.46:0/304575668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b41045f0 0x7fc7b4104a00 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00099c0 tx=0x7fc7b0009cd0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.291+0000 7fc7bc7b2700 1 -- 172.21.15.46:0/304575668 shutdown_connections 2024-05-09T04:50:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.291+0000 7fc7bc7b2700 1 --2- 172.21.15.46:0/304575668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7b4109870 0x7fc7b41111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.291+0000 7fc7bc7b2700 1 --2- 172.21.15.46:0/304575668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b41045f0 0x7fc7b4104a00 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.291+0000 7fc7bc7b2700 1 --2- 172.21.15.46:0/304575668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7b4108f20 0x7fc7b4109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.291+0000 7fc7bc7b2700 1 -- 172.21.15.46:0/304575668 >> 172.21.15.46:0/304575668 conn(0x7fc7b4100040 msgr2=0x7fc7b4102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.291+0000 7fc7bc7b2700 1 -- 172.21.15.46:0/304575668 shutdown_connections 2024-05-09T04:50:53.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.291+0000 7fc7bc7b2700 1 -- 172.21.15.46:0/304575668 wait complete. 2024-05-09T04:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.292+0000 7fc7bc7b2700 1 Processor -- start 2024-05-09T04:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.292+0000 7fc7bc7b2700 1 -- start start 2024-05-09T04:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.292+0000 7fc7bc7b2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b41045f0 0x7fc7b419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.292+0000 7fc7bc7b2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7b4108f20 0x7fc7b419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.292+0000 7fc7bc7b2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7b4109870 0x7fc7b419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.292+0000 7fc7bc7b2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4114510 con 0x7fc7b41045f0 2024-05-09T04:50:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.292+0000 7fc7bc7b2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4114390 con 0x7fc7b4109870 2024-05-09T04:50:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.292+0000 7fc7bc7b2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc7b4114690 con 0x7fc7b4108f20 2024-05-09T04:50:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.292+0000 7fc7ba54e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b41045f0 0x7fc7b419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.292+0000 7fc7ba54e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b41045f0 0x7fc7b419ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57926/0 (socket says 172.21.15.46:57926) 2024-05-09T04:50:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.292+0000 7fc7ba54e700 1 -- 172.21.15.46:0/2519928100 learned_addr learned my addr 172.21.15.46:0/2519928100 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.292+0000 7fc7b9d4d700 1 --2- 172.21.15.46:0/2519928100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7b4108f20 0x7fc7b419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.292+0000 7fc7bad4f700 1 --2- 172.21.15.46:0/2519928100 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7b4109870 0x7fc7b419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.293+0000 7fc7ba54e700 1 -- 172.21.15.46:0/2519928100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7b4108f20 msgr2=0x7fc7b419b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.293+0000 7fc7ba54e700 1 --2- 172.21.15.46:0/2519928100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7b4108f20 0x7fc7b419b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.293+0000 7fc7ba54e700 1 -- 172.21.15.46:0/2519928100 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7b4109870 msgr2=0x7fc7b419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.293+0000 7fc7ba54e700 1 --2- 172.21.15.46:0/2519928100 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7b4109870 0x7fc7b419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.293+0000 7fc7ba54e700 1 -- 172.21.15.46:0/2519928100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7b419fd50 con 0x7fc7b41045f0 2024-05-09T04:50:53.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.293+0000 7fc7ba54e700 1 --2- 172.21.15.46:0/2519928100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b41045f0 0x7fc7b419ae80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000b580 tx=0x7fc7b000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.293+0000 7fc7ab7fe700 1 -- 172.21.15.46:0/2519928100 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b0003ae0 con 0x7fc7b41045f0 2024-05-09T04:50:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.293+0000 7fc7ab7fe700 1 -- 172.21.15.46:0/2519928100 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b0004130 con 0x7fc7b41045f0 2024-05-09T04:50:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.293+0000 7fc7bc7b2700 1 -- 172.21.15.46:0/2519928100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7b419ffe0 con 0x7fc7b41045f0 2024-05-09T04:50:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.293+0000 7fc7ab7fe700 1 -- 172.21.15.46:0/2519928100 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b000f460 con 0x7fc7b41045f0 2024-05-09T04:50:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.293+0000 7fc7bc7b2700 1 -- 172.21.15.46:0/2519928100 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7b41a04c0 con 0x7fc7b41045f0 2024-05-09T04:50:53.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.296+0000 7fc7ab7fe700 1 -- 172.21.15.46:0/2519928100 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc7b001f780 con 0x7fc7b41045f0 2024-05-09T04:50:53.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.296+0000 7fc7ab7fe700 1 --2- 172.21.15.46:0/2519928100 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc7a00b2880 0x7fc7a00b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.297+0000 7fc7b9d4d700 1 --2- 172.21.15.46:0/2519928100 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc7a00b2880 0x7fc7a00b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.297+0000 7fc7ab7fe700 1 -- 172.21.15.46:0/2519928100 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fc7b00e7970 con 0x7fc7b41045f0 2024-05-09T04:50:53.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.297+0000 7fc7cb1c8b80 1 Processor -- start 2024-05-09T04:50:53.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.297+0000 7fc7cb1c8b80 1 -- start start 2024-05-09T04:50:53.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.297+0000 7fc7b9d4d700 1 --2- 172.21.15.46:0/2519928100 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc7a00b2880 0x7fc7a00b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4006fd0 tx=0x7fc7a4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.297+0000 7fc7cb1c8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 0x5569e1ddb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.297+0000 7fc7cb1c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.297+0000 7fc7cb1c8b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1dd57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.297+0000 7fc7cb1c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5569e1cf8050 con 0x5569e1c58450 2024-05-09T04:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.297+0000 7fc7cb1c8b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5569e1c9d950 con 0x5569e1dd53b0 2024-05-09T04:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.297+0000 7fc7cb1c8b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5569e1cf4010 con 0x5569e1cd2cb0 2024-05-09T04:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.298+0000 7fc7b9d4d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.298+0000 7fc7b9d4d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57928/0 (socket says 172.21.15.46:57928) 2024-05-09T04:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.298+0000 7fc7b9d4d700 1 -- 172.21.15.46:0/2536548503 learned_addr learned my addr 172.21.15.46:0/2536548503 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.298+0000 7fc7ba54e700 1 --2- 172.21.15.46:0/2536548503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 0x5569e1ddb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.298+0000 7fc7bad4f700 1 --2- 172.21.15.46:0/2536548503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1dd57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.298+0000 7fc7b9d4d700 1 -- 172.21.15.46:0/2536548503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 msgr2=0x5569e1ddb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.298+0000 7fc7b9d4d700 1 --2- 172.21.15.46:0/2536548503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 0x5569e1ddb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.298+0000 7fc7b9d4d700 1 -- 172.21.15.46:0/2536548503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 msgr2=0x5569e1dd57c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.298+0000 7fc7b9d4d700 1 --2- 172.21.15.46:0/2536548503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1dd57c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.298+0000 7fc7b9d4d700 1 -- 172.21.15.46:0/2536548503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569e1c8e4b0 con 0x5569e1c58450 2024-05-09T04:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.298+0000 7fc7ba54e700 1 --2- 172.21.15.46:0/2536548503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 0x5569e1ddb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.298+0000 7fc7b9d4d700 1 --2- 172.21.15.46:0/2536548503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddb730 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400d6a0 tx=0x7fc7a4010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb5d6eae98a8ab08 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.298+0000 7fc7a8ff9700 1 -- 172.21.15.46:0/2536548503 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4024070 con 0x5569e1c58450 2024-05-09T04:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.298+0000 7fc7a8ff9700 1 -- 172.21.15.46:0/2536548503 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a4014af0 con 0x5569e1c58450 2024-05-09T04:50:53.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.298+0000 7fc7a8ff9700 1 -- 172.21.15.46:0/2536548503 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a401fb40 con 0x5569e1c58450 2024-05-09T04:50:53.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.299+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/2536548503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 msgr2=0x5569e1ddb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:53.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.299+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/2536548503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddb730 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc7a400d6a0 tx=0x7fc7a4010000 comp rx=0 tx=0).stop 2024-05-09T04:50:53.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.299+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/2536548503 shutdown_connections 2024-05-09T04:50:53.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.299+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/2536548503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1dd57c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.299+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/2536548503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 0x5569e1ddb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.299+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/2536548503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddb730 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.299+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/2536548503 >> 172.21.15.46:0/2536548503 conn(0x5569e1c5bf20 msgr2=0x5569e1dd01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.299+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/2536548503 shutdown_connections 2024-05-09T04:50:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.299+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/2536548503 wait complete. 2024-05-09T04:50:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.300+0000 7fc7cb1c8b80 1 -- start start 2024-05-09T04:50:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.300+0000 7fc7cb1c8b80 1 Processor -- start 2024-05-09T04:50:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.300+0000 7fc7cb1c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.300+0000 7fc7cb1c8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 0x5569e1ddf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.300+0000 7fc7cb1c8b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1ddfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.300+0000 7fc7cb1c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5569e1dd6110 con 0x5569e1c58450 2024-05-09T04:50:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.300+0000 7fc7cb1c8b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5569e1c92ad0 con 0x5569e1dd53b0 2024-05-09T04:50:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.300+0000 7fc7cb1c8b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5569e1de1fd0 con 0x5569e1cd2cb0 2024-05-09T04:50:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.300+0000 7fc7cb1c8b80 10 client.0 _ll_register_callbacks cb 0x5569e1b69670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:50:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.300+0000 7fc7ba54e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.300+0000 7fc7ba54e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57932/0 (socket says 172.21.15.46:57932) 2024-05-09T04:50:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.300+0000 7fc7ba54e700 1 -- 172.21.15.46:0/1301006245 learned_addr learned my addr 172.21.15.46:0/1301006245 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:53.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.300+0000 7fc7b9d4d700 1 --2- 172.21.15.46:0/1301006245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 0x5569e1ddf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.300+0000 7fc7bad4f700 1 --2- 172.21.15.46:0/1301006245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1ddfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.301+0000 7fc7ba54e700 1 -- 172.21.15.46:0/1301006245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 msgr2=0x5569e1ddf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.301+0000 7fc7ba54e700 1 --2- 172.21.15.46:0/1301006245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 0x5569e1ddf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.301+0000 7fc7ba54e700 1 -- 172.21.15.46:0/1301006245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 msgr2=0x5569e1ddfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.301+0000 7fc7ba54e700 1 --2- 172.21.15.46:0/1301006245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1ddfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.301+0000 7fc7ba54e700 1 -- 172.21.15.46:0/1301006245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569e1b73440 con 0x5569e1c58450 2024-05-09T04:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.301+0000 7fc7ba54e700 1 --2- 172.21.15.46:0/1301006245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddef80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00e57a0 tx=0x7fc7b00ae880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b9f20f649ea05d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.301+0000 7fc79dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.301+0000 7fc79dffb700 1 -- 172.21.15.46:0/1301006245 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00af6e0 con 0x5569e1c58450 2024-05-09T04:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.301+0000 7fc79dffb700 1 -- 172.21.15.46:0/1301006245 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fc7b00b7920 con 0x5569e1c58450 2024-05-09T04:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.301+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/1301006245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7a4014af0 con 0x5569e1c58450 2024-05-09T04:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.301+0000 7fc79dffb700 1 -- 172.21.15.46:0/1301006245 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc7b00aefe0 con 0x5569e1c58450 2024-05-09T04:50:53.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.301+0000 7fc7cb1c8b80 10 client.15102 fetch_fsmap learned FSMap version 24 2024-05-09T04:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.301+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/1301006245 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5569e1dec960 con 0x5569e1c58450 2024-05-09T04:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.301+0000 7fc79dffb700 1 -- 172.21.15.46:0/1301006245 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7a4014af0 con 0x5569e1c58450 2024-05-09T04:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.302+0000 7fc79dffb700 1 -- 172.21.15.46:0/1301006245 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc7b00ae360 con 0x5569e1c58450 2024-05-09T04:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.302+0000 7fc7cb1c8b80 10 client.15102 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T04:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.302+0000 7fc7cb1c8b80 10 client.15102 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T04:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.302+0000 7fc7cb1c8b80 20 client.15102 populate_metadata read hostname 'smithi046' 2024-05-09T04:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.302+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/1301006245 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5569e1deef60 0x5569e1df03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.302+0000 7fc7cb1c8b80 4 client.15102 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.302+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/1301006245 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569e1d05a30 con 0x5569e1deef60 2024-05-09T04:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.302+0000 7fc7b9d4d700 1 --2- 172.21.15.46:0/1301006245 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5569e1deef60 0x5569e1df03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.302+0000 7fc7b9d4d700 1 --2- 172.21.15.46:0/1301006245 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5569e1deef60 0x5569e1df03d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6d3850ff63bbe0a2 server_cookie=442c7d7b7913c4ba in_seq=0 out_seq=0 2024-05-09T04:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.302+0000 7fc79dffb700 10 client.15102 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.303+0000 7fc79dffb700 1 -- 172.21.15.46:0/1301006245 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5569e1d05a30 con 0x5569e1deef60 2024-05-09T04:50:53.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.303+0000 7fc79dffb700 10 client.15102 handle_command_reply: tid=0 2024-05-09T04:50:53.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.303+0000 7fc7cb1c8b80 1 client.15102 shutdown 2024-05-09T04:50:53.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.304+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/1301006245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 msgr2=0x5569e1ddef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.304+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/1301006245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddef80 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00e57a0 tx=0x7fc7b00ae880 comp rx=0 tx=0).stop 2024-05-09T04:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.304+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/1301006245 shutdown_connections 2024-05-09T04:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.304+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/1301006245 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5569e1deef60 0x5569e1df03d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.304+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/1301006245 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1ddfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.304+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/1301006245 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddef80 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.304+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/1301006245 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 0x5569e1ddf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.304+0000 7fc79dffb700 0 client.15102 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.304+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/1301006245 >> 172.21.15.46:0/1301006245 conn(0x5569e1c5bf20 msgr2=0x5569e1dc9fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.304+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/1301006245 shutdown_connections 2024-05-09T04:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.304+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/1301006245 wait complete. 2024-05-09T04:50:53.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.304+0000 7fc7cb1c8b80 20 client.15102 trim_cache size 0 max 16384 2024-05-09T04:50:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7cb1c8b80 1 Processor -- start 2024-05-09T04:50:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7cb1c8b80 1 -- start start 2024-05-09T04:50:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7cb1c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7cb1c8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 0x5569e1dddfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7cb1c8b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1dead70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7cb1c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5569e1dee710 con 0x5569e1c58450 2024-05-09T04:50:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7cb1c8b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5569e1deede0 con 0x5569e1dd53b0 2024-05-09T04:50:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7cb1c8b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5569e1def4b0 con 0x5569e1cd2cb0 2024-05-09T04:50:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7ba54e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7ba54e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57944/0 (socket says 172.21.15.46:57944) 2024-05-09T04:50:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7ba54e700 1 -- 172.21.15.46:0/3525605689 learned_addr learned my addr 172.21.15.46:0/3525605689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:53.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7bad4f700 1 --2- 172.21.15.46:0/3525605689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1dead70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7b9d4d700 1 --2- 172.21.15.46:0/3525605689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 0x5569e1dddfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7ba54e700 1 -- 172.21.15.46:0/3525605689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 msgr2=0x5569e1dddfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7ba54e700 1 --2- 172.21.15.46:0/3525605689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 0x5569e1dddfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7ba54e700 1 -- 172.21.15.46:0/3525605689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 msgr2=0x5569e1dead70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7ba54e700 1 --2- 172.21.15.46:0/3525605689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1dead70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.325+0000 7fc7ba54e700 1 -- 172.21.15.46:0/3525605689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569e1de1820 con 0x5569e1c58450 2024-05-09T04:50:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.326+0000 7fc7bad4f700 1 --2- 172.21.15.46:0/3525605689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1dead70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:50:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.326+0000 7fc7ba54e700 1 --2- 172.21.15.46:0/3525605689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddf840 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00e7c90 tx=0x7fc7b001fb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55a9ad44972609 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.326+0000 7fc79ffff700 1 -- 172.21.15.46:0/3525605689 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00b3390 con 0x5569e1c58450 2024-05-09T04:50:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.326+0000 7fc79ffff700 1 -- 172.21.15.46:0/3525605689 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b00b7db0 con 0x5569e1c58450 2024-05-09T04:50:53.327 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.326+0000 7fc79ffff700 1 -- 172.21.15.46:0/3525605689 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7b00adf40 con 0x5569e1c58450 2024-05-09T04:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/3525605689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 msgr2=0x5569e1ddf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/3525605689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddf840 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00e7c90 tx=0x7fc7b001fb20 comp rx=0 tx=0).stop 2024-05-09T04:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/3525605689 shutdown_connections 2024-05-09T04:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/3525605689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1dead70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/3525605689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1c58450 0x5569e1ddf840 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/3525605689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1cd2cb0 0x5569e1dddfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/3525605689 >> 172.21.15.46:0/3525605689 conn(0x5569e1c5bf20 msgr2=0x5569e1ddd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/3525605689 shutdown_connections 2024-05-09T04:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/3525605689 wait complete. 2024-05-09T04:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 -- start start 2024-05-09T04:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 Processor -- start 2024-05-09T04:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1c58450 0x5569e1df3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1cd2cb0 0x5569e1df4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1df8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5569e1df0250 con 0x5569e1cd2cb0 2024-05-09T04:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5569e1defb80 con 0x5569e1dd53b0 2024-05-09T04:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.327+0000 7fc7cb1c8b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5569e1df0920 con 0x5569e1c58450 2024-05-09T04:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7cb1c8b80 10 client.0 _ll_register_callbacks cb 0x5569e1b69670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7b9d4d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1cd2cb0 0x5569e1df4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7b9d4d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1cd2cb0 0x5569e1df4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57958/0 (socket says 172.21.15.46:57958) 2024-05-09T04:50:53.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7b9d4d700 1 -- 172.21.15.46:0/2749315640 learned_addr learned my addr 172.21.15.46:0/2749315640 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7b9d4d700 1 -- 172.21.15.46:0/2749315640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1c58450 msgr2=0x5569e1df3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7ba54e700 1 --2- 172.21.15.46:0/2749315640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1c58450 0x5569e1df3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7bad4f700 1 --2- 172.21.15.46:0/2749315640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1df8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7b9d4d700 1 --2- 172.21.15.46:0/2749315640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1c58450 0x5569e1df3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7b9d4d700 1 -- 172.21.15.46:0/2749315640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 msgr2=0x5569e1df8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7b9d4d700 1 --2- 172.21.15.46:0/2749315640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1df8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7b9d4d700 1 -- 172.21.15.46:0/2749315640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569e1df8830 con 0x5569e1cd2cb0 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7ba54e700 1 --2- 172.21.15.46:0/2749315640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1c58450 0x5569e1df3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7b9d4d700 1 --2- 172.21.15.46:0/2749315640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1cd2cb0 0x5569e1df4200 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4008010 tx=0x7fc7a4004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30626977586b57f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc79dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc79dffb700 1 -- 172.21.15.46:0/2749315640 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7a4024040 con 0x5569e1cd2cb0 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc79dffb700 1 -- 172.21.15.46:0/2749315640 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fc7a400f040 con 0x5569e1cd2cb0 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/2749315640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7b00b7db0 con 0x5569e1cd2cb0 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc79dffb700 1 -- 172.21.15.46:0/2749315640 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7fc7a4040050 con 0x5569e1cd2cb0 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7cb1c8b80 10 client.15108 fetch_fsmap learned FSMap version 24 2024-05-09T04:50:53.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.328+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/2749315640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x5569e1e5abf0 con 0x5569e1cd2cb0 2024-05-09T04:50:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.329+0000 7fc79dffb700 1 -- 172.21.15.46:0/2749315640 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7b00b7db0 con 0x5569e1cd2cb0 2024-05-09T04:50:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.329+0000 7fc79dffb700 1 -- 172.21.15.46:0/2749315640 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7fc7a4013070 con 0x5569e1cd2cb0 2024-05-09T04:50:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.329+0000 7fc7cb1c8b80 10 client.15108 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T04:50:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.329+0000 7fc7cb1c8b80 10 client.15108 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-09T04:50:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.329+0000 7fc7cb1c8b80 20 client.15108 populate_metadata read hostname 'smithi046' 2024-05-09T04:50:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.329+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/2749315640 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5569e1e5ce80 0x5569e1e5e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.329+0000 7fc7cb1c8b80 4 client.15108 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:50:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.329+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/2749315640 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "status"}) -- 0x5569e1cffad0 con 0x5569e1e5ce80 2024-05-09T04:50:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.329+0000 7fc7ba54e700 1 --2- 172.21.15.46:0/2749315640 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5569e1e5ce80 0x5569e1e5e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc79dffb700 10 client.15108 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:50:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc7ba54e700 1 --2- 172.21.15.46:0/2749315640 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5569e1e5ce80 0x5569e1e5e2f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6495fceb53c90c59 server_cookie=26406329815b9108 in_seq=0 out_seq=0 2024-05-09T04:50:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc79dffb700 1 -- 172.21.15.46:0/2749315640 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5569e1cffad0 con 0x5569e1e5ce80 2024-05-09T04:50:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc79dffb700 10 client.15108 handle_command_reply: tid=0 2024-05-09T04:50:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc7cb1c8b80 1 client.15108 shutdown 2024-05-09T04:50:53.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/2749315640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1cd2cb0 msgr2=0x5569e1df4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/2749315640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1cd2cb0 0x5569e1df4200 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4008010 tx=0x7fc7a4004860 comp rx=0 tx=0).stop 2024-05-09T04:50:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/2749315640 shutdown_connections 2024-05-09T04:50:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/2749315640 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5569e1e5ce80 0x5569e1e5e2f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/2749315640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5569e1dd53b0 0x5569e1df8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/2749315640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5569e1c58450 0x5569e1df3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc7cb1c8b80 1 --2- 172.21.15.46:0/2749315640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5569e1cd2cb0 0x5569e1df4200 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/2749315640 >> 172.21.15.46:0/2749315640 conn(0x5569e1c5bf20 msgr2=0x5569e1dd8bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc79dffb700 0 client.15108 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:50:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/2749315640 shutdown_connections 2024-05-09T04:50:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc7cb1c8b80 1 -- 172.21.15.46:0/2749315640 wait complete. 2024-05-09T04:50:53.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.330+0000 7fc7cb1c8b80 20 client.15108 trim_cache size 0 max 16384 2024-05-09T04:50:53.336 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:50:53.336 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ecf9ae2c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:50:53.336 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 1, 2024-05-09T04:50:53.336 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24473, 2024-05-09T04:50:53.336 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-09T04:50:53.336 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-09T04:50:53.336 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-09T04:50:53.336 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 168.87257685599999, 2024-05-09T04:50:53.336 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 24, 2024-05-09T04:50:53.336 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 84, 2024-05-09T04:50:53.336 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 80, 2024-05-09T04:50:53.336 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 210.60339798199999 2024-05-09T04:50:53.336 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.331+0000 7fc7bc7b2700 1 -- 172.21.15.46:0/2519928100 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc7a00b2880 msgr2=0x7fc7a00b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.331+0000 7fc7bc7b2700 1 --2- 172.21.15.46:0/2519928100 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc7a00b2880 0x7fc7a00b4d90 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc7a4006fd0 tx=0x7fc7a4008040 comp rx=0 tx=0).stop 2024-05-09T04:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.331+0000 7fc7bc7b2700 1 -- 172.21.15.46:0/2519928100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b41045f0 msgr2=0x7fc7b419ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.331+0000 7fc7bc7b2700 1 --2- 172.21.15.46:0/2519928100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b41045f0 0x7fc7b419ae80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fc7b000b580 tx=0x7fc7b000be20 comp rx=0 tx=0).stop 2024-05-09T04:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.331+0000 7fc7bc7b2700 1 -- 172.21.15.46:0/2519928100 shutdown_connections 2024-05-09T04:50:53.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.331+0000 7fc7bc7b2700 1 --2- 172.21.15.46:0/2519928100 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fc7a00b2880 0x7fc7a00b4d90 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.331+0000 7fc7bc7b2700 1 --2- 172.21.15.46:0/2519928100 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc7b4109870 0x7fc7b419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.331+0000 7fc7bc7b2700 1 --2- 172.21.15.46:0/2519928100 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc7b41045f0 0x7fc7b419ae80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.331+0000 7fc7bc7b2700 1 --2- 172.21.15.46:0/2519928100 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc7b4108f20 0x7fc7b419b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:53.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.331+0000 7fc7bc7b2700 1 -- 172.21.15.46:0/2519928100 >> 172.21.15.46:0/2519928100 conn(0x7fc7b4100040 msgr2=0x7fc7b410a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:53.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.331+0000 7fc7bc7b2700 1 -- 172.21.15.46:0/2519928100 shutdown_connections 2024-05-09T04:50:53.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:53.331+0000 7fc7bc7b2700 1 -- 172.21.15.46:0/2519928100 wait complete. 2024-05-09T04:50:54.110 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-09T04:50:54.365 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:54.772 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:54 smithi046 ceph-mon[30136]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:50:54.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:54 smithi072 ceph-mon[34088]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:50:55.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:54 smithi110 ceph-mon[34904]: pgmap v387: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-05-09T04:50:55.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:55 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:50:55.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:55 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:50:55.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:55 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:50:55.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:55 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:50:55.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:55 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:50:55.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:55 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:50:56.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:55 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:50:56.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:55 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:50:56.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:55 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:50:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.184+0000 7ff2dacdb700 1 -- 172.21.15.46:0/4232027686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d4107df0 msgr2=0x7ff2d4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.184+0000 7ff2dacdb700 1 --2- 172.21.15.46:0/4232027686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d4107df0 0x7ff2d4108200 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00099c0 tx=0x7ff2c0009cd0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.184+0000 7ff2dacdb700 1 -- 172.21.15.46:0/4232027686 shutdown_connections 2024-05-09T04:50:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.184+0000 7ff2dacdb700 1 --2- 172.21.15.46:0/4232027686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2d4108740 0x7ff2d41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.184+0000 7ff2dacdb700 1 --2- 172.21.15.46:0/4232027686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2d41034c0 0x7ff2d41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.184+0000 7ff2dacdb700 1 --2- 172.21.15.46:0/4232027686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d4107df0 0x7ff2d4108200 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.184+0000 7ff2dacdb700 1 -- 172.21.15.46:0/4232027686 >> 172.21.15.46:0/4232027686 conn(0x7ff2d40fedf0 msgr2=0x7ff2d4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.184+0000 7ff2dacdb700 1 -- 172.21.15.46:0/4232027686 shutdown_connections 2024-05-09T04:50:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.184+0000 7ff2dacdb700 1 -- 172.21.15.46:0/4232027686 wait complete. 2024-05-09T04:50:56.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.185+0000 7ff2dacdb700 1 Processor -- start 2024-05-09T04:50:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.185+0000 7ff2dacdb700 1 -- start start 2024-05-09T04:50:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.185+0000 7ff2dacdb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2d41034c0 0x7ff2d419bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.185+0000 7ff2dacdb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2d4107df0 0x7ff2d419c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.185+0000 7ff2dacdb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d4108740 0x7ff2d419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.185+0000 7ff2dacdb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d41133c0 con 0x7ff2d4108740 2024-05-09T04:50:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.185+0000 7ff2dacdb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4113240 con 0x7ff2d41034c0 2024-05-09T04:50:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.185+0000 7ff2dacdb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d4113540 con 0x7ff2d4107df0 2024-05-09T04:50:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.185+0000 7ff2d9278700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d4108740 0x7ff2d419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.185+0000 7ff2d9278700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d4108740 0x7ff2d419c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37082/0 (socket says 172.21.15.46:37082) 2024-05-09T04:50:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.185+0000 7ff2d9278700 1 -- 172.21.15.46:0/1998290603 learned_addr learned my addr 172.21.15.46:0/1998290603 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.185+0000 7ff2d8a77700 1 --2- 172.21.15.46:0/1998290603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2d41034c0 0x7ff2d419bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.186+0000 7ff2d3fff700 1 --2- 172.21.15.46:0/1998290603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2d4107df0 0x7ff2d419c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.186+0000 7ff2d9278700 1 -- 172.21.15.46:0/1998290603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2d4107df0 msgr2=0x7ff2d419c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.186+0000 7ff2d9278700 1 --2- 172.21.15.46:0/1998290603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2d4107df0 0x7ff2d419c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.186+0000 7ff2d9278700 1 -- 172.21.15.46:0/1998290603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2d41034c0 msgr2=0x7ff2d419bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:56.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.186+0000 7ff2d9278700 1 --2- 172.21.15.46:0/1998290603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2d41034c0 0x7ff2d419bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.186+0000 7ff2d9278700 1 -- 172.21.15.46:0/1998290603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d41a7050 con 0x7ff2d4108740 2024-05-09T04:50:56.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.186+0000 7ff2d8a77700 1 --2- 172.21.15.46:0/1998290603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2d41034c0 0x7ff2d419bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:56.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.186+0000 7ff2d9278700 1 --2- 172.21.15.46:0/1998290603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d4108740 0x7ff2d419c7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc007ee0 tx=0x7ff2cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:56.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.186+0000 7ff2d1ffb700 1 -- 172.21.15.46:0/1998290603 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc00dbd0 con 0x7ff2d4108740 2024-05-09T04:50:56.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.186+0000 7ff2d1ffb700 1 -- 172.21.15.46:0/1998290603 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2cc00dd70 con 0x7ff2d4108740 2024-05-09T04:50:56.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.186+0000 7ff2dacdb700 1 -- 172.21.15.46:0/1998290603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d41a7340 con 0x7ff2d4108740 2024-05-09T04:50:56.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.186+0000 7ff2d1ffb700 1 -- 172.21.15.46:0/1998290603 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc005990 con 0x7ff2d4108740 2024-05-09T04:50:56.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.186+0000 7ff2dacdb700 1 -- 172.21.15.46:0/1998290603 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d41a7800 con 0x7ff2d4108740 2024-05-09T04:50:56.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.189+0000 7ff2d1ffb700 1 -- 172.21.15.46:0/1998290603 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff2cc01a020 con 0x7ff2d4108740 2024-05-09T04:50:56.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.189+0000 7ff2d1ffb700 1 --2- 172.21.15.46:0/1998290603 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff2c40b2880 0x7ff2c40b4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.190+0000 7ff2d8a77700 1 --2- 172.21.15.46:0/1998290603 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff2c40b2880 0x7ff2c40b4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.190+0000 7ff2d1ffb700 1 -- 172.21.15.46:0/1998290603 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7ff2cc0e1990 con 0x7ff2d4108740 2024-05-09T04:50:56.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.190+0000 7ff2d8a77700 1 --2- 172.21.15.46:0/1998290603 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff2c40b2880 0x7ff2c40b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8011490 tx=0x7ff2c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:56.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.190+0000 7ff2e96f1b80 1 Processor -- start 2024-05-09T04:50:56.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.190+0000 7ff2e96f1b80 1 -- start start 2024-05-09T04:50:56.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.190+0000 7ff2e96f1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 0x55693ae984a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.190+0000 7ff2e96f1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 0x55693ae989e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.190+0000 7ff2e96f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693ae92a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.190+0000 7ff2e96f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55693adb4fa0 con 0x55693ae92650 2024-05-09T04:50:56.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.190+0000 7ff2e96f1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55693ad537a0 con 0x55693adff110 2024-05-09T04:50:56.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.190+0000 7ff2e96f1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55693adb0f60 con 0x55693ad15450 2024-05-09T04:50:56.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.191+0000 7ff2d9278700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693ae92a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.191+0000 7ff2d9278700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693ae92a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37092/0 (socket says 172.21.15.46:37092) 2024-05-09T04:50:56.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.191+0000 7ff2d9278700 1 -- 172.21.15.46:0/3564070250 learned_addr learned my addr 172.21.15.46:0/3564070250 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:56.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.191+0000 7ff2d3fff700 1 --2- 172.21.15.46:0/3564070250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 0x55693ae989e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.191+0000 7ff2d8a77700 1 --2- 172.21.15.46:0/3564070250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 0x55693ae984a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.191+0000 7ff2d9278700 1 -- 172.21.15.46:0/3564070250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 msgr2=0x55693ae989e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:56.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.191+0000 7ff2d9278700 1 --2- 172.21.15.46:0/3564070250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 0x55693ae989e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.191+0000 7ff2d9278700 1 -- 172.21.15.46:0/3564070250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 msgr2=0x55693ae984a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:56.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.191+0000 7ff2d9278700 1 --2- 172.21.15.46:0/3564070250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 0x55693ae984a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.191+0000 7ff2d9278700 1 -- 172.21.15.46:0/3564070250 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55693ad4b320 con 0x55693ae92650 2024-05-09T04:50:56.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.191+0000 7ff2d8a77700 1 --2- 172.21.15.46:0/3564070250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 0x55693ae984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:50:56.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.191+0000 7ff2d9278700 1 --2- 172.21.15.46:0/3564070250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693ae92a80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0a8f00 tx=0x7ff2cc0a8940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7847f7d0189ed5bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:56.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.191+0000 7ff2beffd700 1 -- 172.21.15.46:0/3564070250 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0a9670 con 0x55693ae92650 2024-05-09T04:50:56.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.191+0000 7ff2beffd700 1 -- 172.21.15.46:0/3564070250 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2cc0a8f70 con 0x55693ae92650 2024-05-09T04:50:56.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.191+0000 7ff2beffd700 1 -- 172.21.15.46:0/3564070250 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0b1940 con 0x55693ae92650 2024-05-09T04:50:56.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.192+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3564070250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 msgr2=0x55693ae92a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:56.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.192+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/3564070250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693ae92a80 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0a8f00 tx=0x7ff2cc0a8940 comp rx=0 tx=0).stop 2024-05-09T04:50:56.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.192+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3564070250 shutdown_connections 2024-05-09T04:50:56.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.192+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/3564070250 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693ae92a80 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.192+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/3564070250 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 0x55693ae984a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.192+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/3564070250 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 0x55693ae989e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.192+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3564070250 >> 172.21.15.46:0/3564070250 conn(0x55693ad75650 msgr2=0x55693ae8d3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:56.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.192+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3564070250 shutdown_connections 2024-05-09T04:50:56.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.192+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3564070250 wait complete. 2024-05-09T04:50:56.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.193+0000 7ff2e96f1b80 1 -- start start 2024-05-09T04:50:56.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.193+0000 7ff2e96f1b80 1 Processor -- start 2024-05-09T04:50:56.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.193+0000 7ff2e96f1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 0x55693ae9c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.193+0000 7ff2e96f1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 0x55693ae9c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.193+0000 7ff2e96f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693ae9cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.193+0000 7ff2e96f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55693ac30b50 con 0x55693ae92650 2024-05-09T04:50:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.193+0000 7ff2e96f1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55693ad6f100 con 0x55693adff110 2024-05-09T04:50:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.193+0000 7ff2e96f1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55693ae9f180 con 0x55693ad15450 2024-05-09T04:50:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.193+0000 7ff2e96f1b80 10 client.0 _ll_register_callbacks cb 0x55693ac26670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:50:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.193+0000 7ff2d9278700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693ae9cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.193+0000 7ff2d9278700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693ae9cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37100/0 (socket says 172.21.15.46:37100) 2024-05-09T04:50:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.193+0000 7ff2d9278700 1 -- 172.21.15.46:0/2717262774 learned_addr learned my addr 172.21.15.46:0/2717262774 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.193+0000 7ff2d8a77700 1 --2- 172.21.15.46:0/2717262774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 0x55693ae9c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2d3fff700 1 --2- 172.21.15.46:0/2717262774 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 0x55693ae9c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2d9278700 1 -- 172.21.15.46:0/2717262774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 msgr2=0x55693ae9c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2d9278700 1 --2- 172.21.15.46:0/2717262774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 0x55693ae9c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2d9278700 1 -- 172.21.15.46:0/2717262774 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 msgr2=0x55693ae9c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2d9278700 1 --2- 172.21.15.46:0/2717262774 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 0x55693ae9c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2d9278700 1 -- 172.21.15.46:0/2717262774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55693ac30440 con 0x55693ae92650 2024-05-09T04:50:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2d3fff700 1 --2- 172.21.15.46:0/2717262774 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 0x55693ae9c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:50:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2d9278700 1 --2- 172.21.15.46:0/2717262774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693ae9cc20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0a8c70 tx=0x7ff2cc0a9400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0880c9b25c9b812 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:50:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2b3fff700 1 -- 172.21.15.46:0/2717262774 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2cc0a9670 con 0x55693ae92650 2024-05-09T04:50:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/2717262774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2cc0a8f70 con 0x55693ae92650 2024-05-09T04:50:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2b3fff700 1 -- 172.21.15.46:0/2717262774 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7ff2cc0a5560 con 0x55693ae92650 2024-05-09T04:50:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2b3fff700 1 -- 172.21.15.46:0/2717262774 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff2cc0b1b50 con 0x55693ae92650 2024-05-09T04:50:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2e96f1b80 10 client.15120 fetch_fsmap learned FSMap version 24 2024-05-09T04:50:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/2717262774 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55693aea9b70 con 0x55693ae92650 2024-05-09T04:50:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.194+0000 7ff2b3fff700 1 -- 172.21.15.46:0/2717262774 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2cc0a8f70 con 0x55693ae92650 2024-05-09T04:50:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.195+0000 7ff2b3fff700 1 -- 172.21.15.46:0/2717262774 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff2cc0ac030 con 0x55693ae92650 2024-05-09T04:50:56.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.195+0000 7ff2e96f1b80 10 client.15120 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T04:50:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.195+0000 7ff2e96f1b80 10 client.15120 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T04:50:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.195+0000 7ff2e96f1b80 20 client.15120 populate_metadata read hostname 'smithi046' 2024-05-09T04:50:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.195+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/2717262774 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55693aeaaec0 0x55693aeacfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.195+0000 7ff2e96f1b80 4 client.15120 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:50:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.195+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/2717262774 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55693adc2980 con 0x55693aeaaec0 2024-05-09T04:50:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.195+0000 7ff2d8a77700 1 --2- 172.21.15.46:0/2717262774 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55693aeaaec0 0x55693aeacfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.195+0000 7ff2b3fff700 10 client.15120 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:50:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.195+0000 7ff2d8a77700 1 --2- 172.21.15.46:0/2717262774 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55693aeaaec0 0x55693aeacfc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f54c2bc6626f08a1 server_cookie=32586a6bcc04be29 in_seq=0 out_seq=0 2024-05-09T04:50:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.196+0000 7ff2b3fff700 1 -- 172.21.15.46:0/2717262774 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55693adc2980 con 0x55693aeaaec0 2024-05-09T04:50:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.196+0000 7ff2b3fff700 10 client.15120 handle_command_reply: tid=0 2024-05-09T04:50:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.196+0000 7ff2e96f1b80 1 client.15120 shutdown 2024-05-09T04:50:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.196+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/2717262774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 msgr2=0x55693ae9cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.196+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/2717262774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693ae9cc20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc0a8c70 tx=0x7ff2cc0a9400 comp rx=0 tx=0).stop 2024-05-09T04:50:56.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.196+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/2717262774 shutdown_connections 2024-05-09T04:50:56.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.196+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/2717262774 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693ae9cc20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.196+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/2717262774 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 0x55693ae9c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.196+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/2717262774 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55693aeaaec0 0x55693aeacfc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.196+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/2717262774 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 0x55693ae9c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.196+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/2717262774 >> 172.21.15.46:0/2717262774 conn(0x55693ad75650 msgr2=0x55693ae8d170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:56.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.196+0000 7ff2b3fff700 0 client.15120 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:50:56.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.197+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/2717262774 shutdown_connections 2024-05-09T04:50:56.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.197+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/2717262774 wait complete. 2024-05-09T04:50:56.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.197+0000 7ff2e96f1b80 20 client.15120 trim_cache size 0 max 16384 2024-05-09T04:50:56.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.223+0000 7ff2e96f1b80 1 Processor -- start 2024-05-09T04:50:56.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.223+0000 7ff2e96f1b80 1 -- start start 2024-05-09T04:50:56.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.223+0000 7ff2e96f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ad15450 0x55693ae91d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.223+0000 7ff2e96f1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693adff110 0x55693aeb6420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.223+0000 7ff2e96f1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693ae92650 0x55693aeb6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.223+0000 7ff2e96f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55693aeab300 con 0x55693ad15450 2024-05-09T04:50:56.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.223+0000 7ff2e96f1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55693aeab9d0 con 0x55693ae92650 2024-05-09T04:50:56.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.223+0000 7ff2e96f1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55693aeac0a0 con 0x55693adff110 2024-05-09T04:50:56.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.224+0000 7ff2d8a77700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ad15450 0x55693ae91d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.224+0000 7ff2d8a77700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ad15450 0x55693ae91d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37114/0 (socket says 172.21.15.46:37114) 2024-05-09T04:50:56.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.224+0000 7ff2d8a77700 1 -- 172.21.15.46:0/3323451069 learned_addr learned my addr 172.21.15.46:0/3323451069 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.224+0000 7ff2d3fff700 1 --2- 172.21.15.46:0/3323451069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693adff110 0x55693aeb6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.224+0000 7ff2d9278700 1 --2- 172.21.15.46:0/3323451069 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693ae92650 0x55693aeb6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.224+0000 7ff2d8a77700 1 -- 172.21.15.46:0/3323451069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693adff110 msgr2=0x55693aeb6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.224+0000 7ff2d8a77700 1 --2- 172.21.15.46:0/3323451069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693adff110 0x55693aeb6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.224+0000 7ff2d8a77700 1 -- 172.21.15.46:0/3323451069 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693ae92650 msgr2=0x55693aeb6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.224+0000 7ff2d8a77700 1 --2- 172.21.15.46:0/3323451069 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693ae92650 0x55693aeb6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.224+0000 7ff2d8a77700 1 -- 172.21.15.46:0/3323451069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55693aeab110 con 0x55693ad15450 2024-05-09T04:50:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.224+0000 7ff2d8a77700 1 --2- 172.21.15.46:0/3323451069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ad15450 0x55693ae91d50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80043e0 tx=0x7ff2c8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4bbbc998fdbaa1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.224+0000 7ff2be7fc700 1 -- 172.21.15.46:0/3323451069 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8031070 con 0x55693ad15450 2024-05-09T04:50:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.224+0000 7ff2be7fc700 1 -- 172.21.15.46:0/3323451069 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c8011e50 con 0x55693ad15450 2024-05-09T04:50:56.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.224+0000 7ff2be7fc700 1 -- 172.21.15.46:0/3323451069 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c801fbc0 con 0x55693ad15450 2024-05-09T04:50:56.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.225+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3323451069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ad15450 msgr2=0x55693ae91d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:56.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.225+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/3323451069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ad15450 0x55693ae91d50 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80043e0 tx=0x7ff2c8014000 comp rx=0 tx=0).stop 2024-05-09T04:50:56.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.225+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3323451069 shutdown_connections 2024-05-09T04:50:56.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.225+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/3323451069 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693ae92650 0x55693aeb6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.225+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/3323451069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ad15450 0x55693ae91d50 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.225+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/3323451069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693adff110 0x55693aeb6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.225+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3323451069 >> 172.21.15.46:0/3323451069 conn(0x55693ad75650 msgr2=0x55693ae91640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.225+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3323451069 shutdown_connections 2024-05-09T04:50:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.225+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3323451069 wait complete. 2024-05-09T04:50:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.226+0000 7ff2e96f1b80 1 -- start start 2024-05-09T04:50:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.226+0000 7ff2e96f1b80 1 Processor -- start 2024-05-09T04:50:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.226+0000 7ff2e96f1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 0x55693aeb6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.226+0000 7ff2e96f1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 0x55693aeb2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.226+0000 7ff2e96f1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693aeb2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.226+0000 7ff2e96f1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55693aeace40 con 0x55693ae92650 2024-05-09T04:50:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.226+0000 7ff2e96f1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55693aeac770 con 0x55693adff110 2024-05-09T04:50:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.226+0000 7ff2e96f1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55693aead510 con 0x55693ad15450 2024-05-09T04:50:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2e96f1b80 10 client.0 _ll_register_callbacks cb 0x55693ac26670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:50:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2d9278700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693aeb2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2d8a77700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 0x55693aeb6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2d3fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 0x55693aeb2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2d9278700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693aeb2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37130/0 (socket says 172.21.15.46:37130) 2024-05-09T04:50:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2d9278700 1 -- 172.21.15.46:0/3225381606 learned_addr learned my addr 172.21.15.46:0/3225381606 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2d8a77700 1 -- 172.21.15.46:0/3225381606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 msgr2=0x55693aeb2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2d8a77700 1 --2- 172.21.15.46:0/3225381606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 0x55693aeb2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2d8a77700 1 -- 172.21.15.46:0/3225381606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 msgr2=0x55693aeb2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:50:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2d8a77700 1 --2- 172.21.15.46:0/3225381606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693aeb2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2d8a77700 1 -- 172.21.15.46:0/3225381606 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55693aeb34f0 con 0x55693ad15450 2024-05-09T04:50:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2d3fff700 1 --2- 172.21.15.46:0/3225381606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 0x55693aeb2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2d9278700 1 --2- 172.21.15.46:0/3225381606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693aeb2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:50:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2d8a77700 1 --2- 172.21.15.46:0/3225381606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 0x55693aeb6700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80044b0 tx=0x7ff2c801f400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ac7b6b8703c9e8c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2b3fff700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:50:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2b3fff700 1 -- 172.21.15.46:0/3225381606 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c8031040 con 0x55693ad15450 2024-05-09T04:50:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2b3fff700 1 -- 172.21.15.46:0/3225381606 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7ff2c801fa00 con 0x55693ad15450 2024-05-09T04:50:56.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.227+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3225381606 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2c8011e50 con 0x55693ad15450 2024-05-09T04:50:56.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.228+0000 7ff2b3fff700 1 -- 172.21.15.46:0/3225381606 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) ==== 24+0+0 (secure 0 0 0) 0x7ff2c801fc00 con 0x55693ad15450 2024-05-09T04:50:56.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.228+0000 7ff2e96f1b80 10 client.24862 fetch_fsmap learned FSMap version 24 2024-05-09T04:50:56.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.228+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3225381606 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) -- 0x55693af17c60 con 0x55693ad15450 2024-05-09T04:50:56.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.228+0000 7ff2b3fff700 1 -- 172.21.15.46:0/3225381606 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c8011e50 con 0x55693ad15450 2024-05-09T04:50:56.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.228+0000 7ff2b3fff700 1 -- 172.21.15.46:0/3225381606 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 24) ==== 8338+0+0 (secure 0 0 0) 0x7ff2c8034070 con 0x55693ad15450 2024-05-09T04:50:56.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.228+0000 7ff2e96f1b80 10 client.24862 fetch_fsmap finished waiting for FSMap version 24 2024-05-09T04:50:56.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.228+0000 7ff2e96f1b80 10 client.24862 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-09T04:50:56.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.228+0000 7ff2e96f1b80 20 client.24862 populate_metadata read hostname 'smithi046' 2024-05-09T04:50:56.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.228+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/3225381606 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55693af19cc0 0x55693af1b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:56.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.228+0000 7ff2e96f1b80 4 client.24862 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-09T04:50:56.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.228+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3225381606 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "status"}) -- 0x55693adbca20 con 0x55693af19cc0 2024-05-09T04:50:56.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.229+0000 7ff2d3fff700 1 --2- 172.21.15.46:0/3225381606 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55693af19cc0 0x55693af1b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:56.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.229+0000 7ff2b3fff700 10 client.24862 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:50:56.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.229+0000 7ff2d3fff700 1 --2- 172.21.15.46:0/3225381606 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55693af19cc0 0x55693af1b130 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f0ad43743ba8282 server_cookie=f2a97ce2dfea0daa in_seq=0 out_seq=0 2024-05-09T04:50:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.230+0000 7ff2b3fff700 1 -- 172.21.15.46:0/3225381606 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55693adbca20 con 0x55693af19cc0 2024-05-09T04:50:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.230+0000 7ff2b3fff700 10 client.24862 handle_command_reply: tid=0 2024-05-09T04:50:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.230+0000 7ff2e96f1b80 1 client.24862 shutdown 2024-05-09T04:50:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.230+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3225381606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 msgr2=0x55693aeb6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.230+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/3225381606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 0x55693aeb6700 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff2c80044b0 tx=0x7ff2c801f400 comp rx=0 tx=0).stop 2024-05-09T04:50:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.230+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3225381606 shutdown_connections 2024-05-09T04:50:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.230+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/3225381606 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55693ae92650 0x55693aeb2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.230+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/3225381606 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55693ad15450 0x55693aeb6700 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.230+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/3225381606 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55693af19cc0 0x55693af1b130 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.230+0000 7ff2b3fff700 0 client.24862 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:50:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.230+0000 7ff2e96f1b80 1 --2- 172.21.15.46:0/3225381606 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55693adff110 0x55693aeb2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.230+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3225381606 >> 172.21.15.46:0/3225381606 conn(0x55693ad75650 msgr2=0x55693ae97a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.230+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3225381606 shutdown_connections 2024-05-09T04:50:56.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.230+0000 7ff2e96f1b80 1 -- 172.21.15.46:0/3225381606 wait complete. 2024-05-09T04:50:56.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.230+0000 7ff2e96f1b80 20 client.24862 trim_cache size 0 max 16384 2024-05-09T04:50:56.235 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:50:56.235 INFO:teuthology.orchestra.run.smithi046.stdout: "cluster_fsid": "ecf9ae2c-0dbd-11ef-bc97-c7b262605968", 2024-05-09T04:50:56.235 INFO:teuthology.orchestra.run.smithi046.stdout: "whoami": 0, 2024-05-09T04:50:56.235 INFO:teuthology.orchestra.run.smithi046.stdout: "id": 24503, 2024-05-09T04:50:56.235 INFO:teuthology.orchestra.run.smithi046.stdout: "want_state": "up:active", 2024-05-09T04:50:56.235 INFO:teuthology.orchestra.run.smithi046.stdout: "state": "up:active", 2024-05-09T04:50:56.235 INFO:teuthology.orchestra.run.smithi046.stdout: "fs_name": "cephfs", 2024-05-09T04:50:56.235 INFO:teuthology.orchestra.run.smithi046.stdout: "rank_uptime": 185.824803737, 2024-05-09T04:50:56.236 INFO:teuthology.orchestra.run.smithi046.stdout: "mdsmap_epoch": 24, 2024-05-09T04:50:56.236 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch": 84, 2024-05-09T04:50:56.236 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_epoch_barrier": 80, 2024-05-09T04:50:56.236 INFO:teuthology.orchestra.run.smithi046.stdout: "uptime": 200.313024184 2024-05-09T04:50:56.236 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:50:56.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.231+0000 7ff2dacdb700 1 -- 172.21.15.46:0/1998290603 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff2c40b2880 msgr2=0x7ff2c40b4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:56.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.231+0000 7ff2dacdb700 1 --2- 172.21.15.46:0/1998290603 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff2c40b2880 0x7ff2c40b4d90 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8011490 tx=0x7ff2c8009260 comp rx=0 tx=0).stop 2024-05-09T04:50:56.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.231+0000 7ff2dacdb700 1 -- 172.21.15.46:0/1998290603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d4108740 msgr2=0x7ff2d419c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:56.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.231+0000 7ff2dacdb700 1 --2- 172.21.15.46:0/1998290603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d4108740 0x7ff2d419c7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7ff2cc007ee0 tx=0x7ff2cc00d3b0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.231+0000 7ff2dacdb700 1 -- 172.21.15.46:0/1998290603 shutdown_connections 2024-05-09T04:50:56.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.231+0000 7ff2dacdb700 1 --2- 172.21.15.46:0/1998290603 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7ff2c40b2880 0x7ff2c40b4d90 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.231+0000 7ff2dacdb700 1 --2- 172.21.15.46:0/1998290603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2d4108740 0x7ff2d419c7a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.231+0000 7ff2dacdb700 1 --2- 172.21.15.46:0/1998290603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff2d41034c0 0x7ff2d419bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.231+0000 7ff2dacdb700 1 --2- 172.21.15.46:0/1998290603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2d4107df0 0x7ff2d419c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:56.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.231+0000 7ff2dacdb700 1 -- 172.21.15.46:0/1998290603 >> 172.21.15.46:0/1998290603 conn(0x7ff2d40fedf0 msgr2=0x7ff2d41096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:56.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.231+0000 7ff2dacdb700 1 -- 172.21.15.46:0/1998290603 shutdown_connections 2024-05-09T04:50:56.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:56.231+0000 7ff2dacdb700 1 -- 172.21.15.46:0/1998290603 wait complete. 2024-05-09T04:50:56.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:56 smithi046 ceph-mon[30136]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T04:50:56.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:50:56.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:50:56.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:50:56.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:50:56.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:56 smithi046 ceph-mon[30136]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:50:57.006 INFO:teuthology.run_tasks:Running task sequential... 2024-05-09T04:50:57.016 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-09T04:50:57.016 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-09T04:50:57.017 INFO:teuthology.task.print:Enabling mgr modules 2024-05-09T04:50:57.017 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-09T04:50:57.017 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T04:50:57.018 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T04:50:57.018 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-09T04:50:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:56 smithi110 ceph-mon[34904]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T04:50:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:50:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:50:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:50:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:50:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:56 smithi110 ceph-mon[34904]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:50:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:56 smithi072 ceph-mon[34088]: pgmap v388: 97 pgs: 97 active+clean; 640 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T04:50:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:50:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:50:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:50:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:50:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:56 smithi072 ceph-mon[34088]: from='mgr.14152 172.21.15.46:0/792409104' entity='mgr.x' 2024-05-09T04:50:57.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.134+0000 7f801c0f9640 1 -- 172.21.15.46:0/1163592604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8014108270 msgr2=0x7f801410a690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:57.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.134+0000 7f801c0f9640 1 --2- 172.21.15.46:0/1163592604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8014108270 0x7f801410a690 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f800c009ca0 tx=0x7f800c02a8c0 comp rx=0 tx=0).stop 2024-05-09T04:50:57.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.134+0000 7f801c0f9640 1 -- 172.21.15.46:0/1163592604 shutdown_connections 2024-05-09T04:50:57.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.134+0000 7f801c0f9640 1 --2- 172.21.15.46:0/1163592604 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80141026b0 0x7f80141035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.134+0000 7f801c0f9640 1 --2- 172.21.15.46:0/1163592604 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8014108270 0x7f801410a690 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.134+0000 7f801c0f9640 1 --2- 172.21.15.46:0/1163592604 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f80141021f0 0x7f8014102fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.134+0000 7f801c0f9640 1 -- 172.21.15.46:0/1163592604 >> 172.21.15.46:0/1163592604 conn(0x7f80140fd4f0 msgr2=0x7f80140ff9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.135+0000 7f801c0f9640 1 -- 172.21.15.46:0/1163592604 shutdown_connections 2024-05-09T04:50:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.135+0000 7f801c0f9640 1 -- 172.21.15.46:0/1163592604 wait complete. 2024-05-09T04:50:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.135+0000 7f801c0f9640 1 Processor -- start 2024-05-09T04:50:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.135+0000 7f801c0f9640 1 -- start start 2024-05-09T04:50:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.135+0000 7f801c0f9640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80141021f0 0x7f8014199f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.135+0000 7f801c0f9640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8014108270 0x7f801419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.135+0000 7f801c0f9640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801419f440 0x7f80141a1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.135+0000 7f801c0f9640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f801410d5f0 con 0x7f801419f440 2024-05-09T04:50:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.135+0000 7f801c0f9640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f801410d470 con 0x7f8014108270 2024-05-09T04:50:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.135+0000 7f801c0f9640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f801410d770 con 0x7f80141021f0 2024-05-09T04:50:57.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f801a696640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801419f440 0x7f80141a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f801a696640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801419f440 0x7f80141a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37152/0 (socket says 172.21.15.46:37152) 2024-05-09T04:50:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f801a696640 1 -- 172.21.15.46:0/2442803818 learned_addr learned my addr 172.21.15.46:0/2442803818 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f8019694640 1 --2- 172.21.15.46:0/2442803818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8014108270 0x7f801419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f8019e95640 1 --2- 172.21.15.46:0/2442803818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80141021f0 0x7f8014199f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f801a696640 1 -- 172.21.15.46:0/2442803818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80141021f0 msgr2=0x7f8014199f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f801a696640 1 --2- 172.21.15.46:0/2442803818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80141021f0 0x7f8014199f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f801a696640 1 -- 172.21.15.46:0/2442803818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8014108270 msgr2=0x7f801419a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f801a696640 1 --2- 172.21.15.46:0/2442803818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8014108270 0x7f801419a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f801a696640 1 -- 172.21.15.46:0/2442803818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80141a1ed0 con 0x7f801419f440 2024-05-09T04:50:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f801a696640 1 --2- 172.21.15.46:0/2442803818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801419f440 0x7f80141a1860 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f800c0097b0 tx=0x7f800c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f800affd640 1 -- 172.21.15.46:0/2442803818 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c004240 con 0x7f801419f440 2024-05-09T04:50:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f800affd640 1 -- 172.21.15.46:0/2442803818 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f800c02f380 con 0x7f801419f440 2024-05-09T04:50:57.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f801c0f9640 1 -- 172.21.15.46:0/2442803818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80141a2160 con 0x7f801419f440 2024-05-09T04:50:57.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f800affd640 1 -- 172.21.15.46:0/2442803818 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c02fc30 con 0x7f801419f440 2024-05-09T04:50:57.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.136+0000 7f801c0f9640 1 -- 172.21.15.46:0/2442803818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80141a26b0 con 0x7f801419f440 2024-05-09T04:50:57.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.137+0000 7f801c0f9640 1 -- 172.21.15.46:0/2442803818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8014106970 con 0x7f801419f440 2024-05-09T04:50:57.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.139+0000 7f800affd640 1 -- 172.21.15.46:0/2442803818 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f800c0043e0 con 0x7f801419f440 2024-05-09T04:50:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.140+0000 7f800affd640 1 --2- 172.21.15.46:0/2442803818 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f80000b27e0 0x7f80000b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.140+0000 7f8019e95640 1 --2- 172.21.15.46:0/2442803818 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f80000b27e0 0x7f80000b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.140+0000 7f800affd640 1 -- 172.21.15.46:0/2442803818 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f800c103ad0 con 0x7f801419f440 2024-05-09T04:50:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.141+0000 7f8019e95640 1 --2- 172.21.15.46:0/2442803818 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f80000b27e0 0x7f80000b4cf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8010005d00 tx=0x7f801000b510 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:57.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.144+0000 7f800affd640 1 -- 172.21.15.46:0/2442803818 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f800c0cb750 con 0x7f801419f440 2024-05-09T04:50:57.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.388+0000 7f801c0f9640 1 -- 172.21.15.46:0/2442803818 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f80140638d0 con 0x7f801419f440 2024-05-09T04:50:57.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.662+0000 7f800affd640 1 -- 172.21.15.46:0/2442803818 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f800c0cb050 con 0x7f801419f440 2024-05-09T04:50:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.665+0000 7f801c0f9640 1 -- 172.21.15.46:0/2442803818 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f80000b27e0 msgr2=0x7f80000b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.665+0000 7f801c0f9640 1 --2- 172.21.15.46:0/2442803818 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f80000b27e0 0x7f80000b4cf0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f8010005d00 tx=0x7f801000b510 comp rx=0 tx=0).stop 2024-05-09T04:50:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.665+0000 7f801c0f9640 1 -- 172.21.15.46:0/2442803818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801419f440 msgr2=0x7f80141a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.665+0000 7f801c0f9640 1 --2- 172.21.15.46:0/2442803818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801419f440 0x7f80141a1860 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f800c0097b0 tx=0x7f800c003c20 comp rx=0 tx=0).stop 2024-05-09T04:50:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.665+0000 7f801c0f9640 1 -- 172.21.15.46:0/2442803818 shutdown_connections 2024-05-09T04:50:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.665+0000 7f801c0f9640 1 --2- 172.21.15.46:0/2442803818 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f80000b27e0 0x7f80000b4cf0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.665+0000 7f801c0f9640 1 --2- 172.21.15.46:0/2442803818 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f801419f440 0x7f80141a1860 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.665+0000 7f801c0f9640 1 --2- 172.21.15.46:0/2442803818 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8014108270 0x7f801419a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:57.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.665+0000 7f801c0f9640 1 --2- 172.21.15.46:0/2442803818 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f80141021f0 0x7f8014199f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:57.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.665+0000 7f801c0f9640 1 -- 172.21.15.46:0/2442803818 >> 172.21.15.46:0/2442803818 conn(0x7f80140fd4f0 msgr2=0x7f80140ff910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:57.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.665+0000 7f801c0f9640 1 -- 172.21.15.46:0/2442803818 shutdown_connections 2024-05-09T04:50:57.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:57.665+0000 7f801c0f9640 1 -- 172.21.15.46:0/2442803818 wait complete. 2024-05-09T04:50:57.683 INFO:teuthology.run_tasks:Running task kclient... 2024-05-09T04:50:57.712 INFO:tasks.kclient:Mounting kernel clients... 2024-05-09T04:50:57.713 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-09T04:50:57.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:50:57.713 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-05-09T04:50:57.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:50:57.730 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-05-09T04:50:57.799 INFO:teuthology.orchestra.run.smithi046.stderr:Cannot find device "ceph-brx" 2024-05-09T04:50:57.800 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T04:50:57.801 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-09T04:50:57.801 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-09T04:50:57.802 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T04:50:57.802 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-09T04:50:58.051 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:57 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2442803818' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-09T04:50:58.052 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:50:57 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: ignoring --setuser ceph since I am not root 2024-05-09T04:50:58.052 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:50:57 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: ignoring --setgroup ceph since I am not root 2024-05-09T04:50:58.052 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:50:57 smithi072 ceph-mgr[35473]: -- 172.21.15.72:0/3441628935 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e5dfac45a0 con 0x55e5e484e900 2024-05-09T04:50:58.052 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:50:57 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:50:57.865+0000 7ff3fe2f5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T04:50:58.052 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:50:58 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:50:58.051+0000 7ff3fe2f5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:50:58.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:57 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2442803818' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-09T04:50:58.057 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:50:57 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: ignoring --setuser ceph since I am not root 2024-05-09T04:50:58.057 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:50:57 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: ignoring --setgroup ceph since I am not root 2024-05-09T04:50:58.057 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:50:57 smithi110 ceph-mgr[36788]: -- 172.21.15.110:0/3838571 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562e105a65a0 con 0x562e15330480 2024-05-09T04:50:58.057 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:50:57 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:50:57.872+0000 7f2296def200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T04:50:58.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:57 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2442803818' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-09T04:50:58.080 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:50:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: ignoring --setuser ceph since I am not root 2024-05-09T04:50:58.080 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:50:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: ignoring --setgroup ceph since I am not root 2024-05-09T04:50:58.080 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:50:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:50:57.889+0000 7f890b550200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-09T04:50:58.106 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:50:58.379 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:50:58 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:50:58.154+0000 7ff3fe2f5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T04:50:58.392 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:50:58 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:50:58.106+0000 7f890b550200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:50:58.392 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:50:58 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:50:58.222+0000 7f890b550200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T04:50:58.468 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:50:58 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:50:58.059+0000 7f2296def200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:50:58.468 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:50:58 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:50:58.162+0000 7f2296def200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-09T04:50:58.468 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:50:58 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:50:58.468+0000 7f2296def200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T04:50:58.648 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:50:58 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:50:58.555+0000 7f890b550200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T04:50:58.649 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:50:58 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:50:58.648+0000 7f890b550200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T04:50:58.807 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:50:58 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:50:58.555+0000 7f2296def200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T04:50:58.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:58 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2442803818' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-09T04:50:58.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:50:58 smithi110 ceph-mon[34904]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-09T04:50:58.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:58 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2442803818' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-09T04:50:58.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:50:58 smithi072 ceph-mon[34088]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-09T04:50:58.880 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:50:58 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:50:58.463+0000 7ff3fe2f5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-09T04:50:58.880 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:50:58 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:50:58.550+0000 7ff3fe2f5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-09T04:50:59.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:58 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2442803818' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-09T04:50:59.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:50:58 smithi046 ceph-mon[30136]: mgrmap e19: x(active, since 10m), standbys: z, y 2024-05-09T04:50:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.875+0000 7faf9754d700 1 -- 172.21.15.46:0/4227786307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf901034c0 msgr2=0x7faf901038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.875+0000 7faf9754d700 1 --2- 172.21.15.46:0/4227786307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf901034c0 0x7faf901038d0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7faf8c00b0d0 tx=0x7faf8c00b3e0 comp rx=0 tx=0).stop 2024-05-09T04:50:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.875+0000 7faf9754d700 1 -- 172.21.15.46:0/4227786307 shutdown_connections 2024-05-09T04:50:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.875+0000 7faf9754d700 1 --2- 172.21.15.46:0/4227786307 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf90108740 0x7faf901100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:59.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.875+0000 7faf9754d700 1 --2- 172.21.15.46:0/4227786307 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf901034c0 0x7faf901038d0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.875+0000 7faf9754d700 1 --2- 172.21.15.46:0/4227786307 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf90107df0 0x7faf90108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.875+0000 7faf9754d700 1 -- 172.21.15.46:0/4227786307 >> 172.21.15.46:0/4227786307 conn(0x7faf900fedc0 msgr2=0x7faf901011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:50:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.875+0000 7faf9754d700 1 -- 172.21.15.46:0/4227786307 shutdown_connections 2024-05-09T04:50:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.876+0000 7faf9754d700 1 -- 172.21.15.46:0/4227786307 wait complete. 2024-05-09T04:50:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.876+0000 7faf9754d700 1 Processor -- start 2024-05-09T04:50:59.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.876+0000 7faf9754d700 1 -- start start 2024-05-09T04:50:59.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf9754d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf901034c0 0x7faf9019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf9754d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf90107df0 0x7faf9019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf9754d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90108740 0x7faf901a6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf9754d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf90113370 con 0x7faf90108740 2024-05-09T04:50:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf9754d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7faf901131f0 con 0x7faf901034c0 2024-05-09T04:50:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf9754d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faf901134f0 con 0x7faf90107df0 2024-05-09T04:50:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf94ae8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf90107df0 0x7faf9019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf952e9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf901034c0 0x7faf9019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf94ae8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf90107df0 0x7faf9019c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:56544/0 (socket says 172.21.15.46:56544) 2024-05-09T04:50:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf952e9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf901034c0 0x7faf9019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:48492/0 (socket says 172.21.15.46:48492) 2024-05-09T04:50:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf94ae8700 1 -- 172.21.15.46:0/3330930619 learned_addr learned my addr 172.21.15.46:0/3330930619 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:50:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf95aea700 1 --2- 172.21.15.46:0/3330930619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90108740 0x7faf901a6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:50:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf94ae8700 1 -- 172.21.15.46:0/3330930619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf901034c0 msgr2=0x7faf9019bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:59.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf94ae8700 1 --2- 172.21.15.46:0/3330930619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf901034c0 0x7faf9019bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf94ae8700 1 -- 172.21.15.46:0/3330930619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90108740 msgr2=0x7faf901a6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:50:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf94ae8700 1 --2- 172.21.15.46:0/3330930619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90108740 0x7faf901a6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:50:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.877+0000 7faf94ae8700 1 -- 172.21.15.46:0/3330930619 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9019ca40 con 0x7faf90107df0 2024-05-09T04:50:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.878+0000 7faf94ae8700 1 --2- 172.21.15.46:0/3330930619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf90107df0 0x7faf9019c190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faf8000d950 tx=0x7faf8000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:50:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.878+0000 7faf867fc700 1 -- 172.21.15.46:0/3330930619 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80016070 con 0x7faf90107df0 2024-05-09T04:50:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.878+0000 7faf9754d700 1 -- 172.21.15.46:0/3330930619 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf90199aa0 con 0x7faf90107df0 2024-05-09T04:50:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.878+0000 7faf867fc700 1 -- 172.21.15.46:0/3330930619 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf80004830 con 0x7faf90107df0 2024-05-09T04:50:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.878+0000 7faf9754d700 1 -- 172.21.15.46:0/3330930619 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf90199fe0 con 0x7faf90107df0 2024-05-09T04:50:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.878+0000 7faf867fc700 1 -- 172.21.15.46:0/3330930619 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf80005520 con 0x7faf90107df0 2024-05-09T04:50:59.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.879+0000 7faf9754d700 1 -- 172.21.15.46:0/3330930619 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf900512c0 con 0x7faf90107df0 2024-05-09T04:50:59.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.881+0000 7faf867fc700 1 -- 172.21.15.46:0/3330930619 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7faf80004220 con 0x7faf90107df0 2024-05-09T04:50:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.882+0000 7faf867fc700 1 --2- 172.21.15.46:0/3330930619 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7faf7c0b2950 0x7faf7c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:50:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.882+0000 7faf952e9700 1 -- 172.21.15.46:0/3330930619 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7faf7c0b2950 msgr2=0x7faf7c0b4e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3691531627 2024-05-09T04:50:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.882+0000 7faf867fc700 1 -- 172.21.15.46:0/3330930619 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7faf800e2440 con 0x7faf90107df0 2024-05-09T04:50:59.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.882+0000 7faf952e9700 1 --2- 172.21.15.46:0/3330930619 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7faf7c0b2950 0x7faf7c0b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:50:59.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:50:59.885+0000 7faf867fc700 1 -- 172.21.15.46:0/3330930619 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faf8001f020 con 0x7faf90107df0 2024-05-09T04:51:00.057 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:50:59 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:50:59.735+0000 7f2296def200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T04:51:00.057 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:50:59 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:50:59.910+0000 7f2296def200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T04:51:00.078 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:50:59 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:50:59.734+0000 7ff3fe2f5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T04:51:00.078 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:50:59 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:50:59.908+0000 7ff3fe2f5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T04:51:00.078 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:00 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:00.078+0000 7ff3fe2f5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T04:51:00.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.083+0000 7faf952e9700 1 -- 172.21.15.46:0/3330930619 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7faf7c0b2950 msgr2=0x7faf7c0b4e60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3691531627 2024-05-09T04:51:00.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.083+0000 7faf952e9700 1 --2- 172.21.15.46:0/3330930619 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7faf7c0b2950 0x7faf7c0b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:51:00.127 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:50:59 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:50:59.943+0000 7f890b550200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-09T04:51:00.128 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:00 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:00.127+0000 7f890b550200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-09T04:51:00.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.150+0000 7faf9754d700 1 -- 172.21.15.46:0/3330930619 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7faf9002e5f0 con 0x7faf90107df0 2024-05-09T04:51:00.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.151+0000 7faf867fc700 1 -- 172.21.15.46:0/3330930619 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) ==== 53+0+73 (secure 0 0 0) 0x7faf800aa120 con 0x7faf90107df0 2024-05-09T04:51:00.151 INFO:teuthology.orchestra.run.smithi046.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-05-09T04:51:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.154+0000 7faf9754d700 1 -- 172.21.15.46:0/3330930619 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7faf7c0b2950 msgr2=0x7faf7c0b4e60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:51:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.154+0000 7faf9754d700 1 --2- 172.21.15.46:0/3330930619 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7faf7c0b2950 0x7faf7c0b4e60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.154+0000 7faf9754d700 1 -- 172.21.15.46:0/3330930619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf90107df0 msgr2=0x7faf9019c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.154+0000 7faf9754d700 1 --2- 172.21.15.46:0/3330930619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf90107df0 0x7faf9019c190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7faf8000d950 tx=0x7faf8000dc60 comp rx=0 tx=0).stop 2024-05-09T04:51:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.154+0000 7faf9754d700 1 -- 172.21.15.46:0/3330930619 shutdown_connections 2024-05-09T04:51:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.154+0000 7faf9754d700 1 --2- 172.21.15.46:0/3330930619 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7faf7c0b2950 0x7faf7c0b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.154+0000 7faf9754d700 1 --2- 172.21.15.46:0/3330930619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf90108740 0x7faf901a6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:00.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.154+0000 7faf9754d700 1 --2- 172.21.15.46:0/3330930619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf901034c0 0x7faf9019bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:00.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.154+0000 7faf9754d700 1 --2- 172.21.15.46:0/3330930619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf90107df0 0x7faf9019c190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:00.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.154+0000 7faf9754d700 1 -- 172.21.15.46:0/3330930619 >> 172.21.15.46:0/3330930619 conn(0x7faf900fedc0 msgr2=0x7faf901096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:00.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.154+0000 7faf9754d700 1 -- 172.21.15.46:0/3330930619 shutdown_connections 2024-05-09T04:51:00.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00.154+0000 7faf9754d700 1 -- 172.21.15.46:0/3330930619 wait complete. 2024-05-09T04:51:00.371 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:00 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3330930619' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T04:51:00.372 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:00 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3330930619' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T04:51:00.372 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:00 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:00.080+0000 7f2296def200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T04:51:00.480 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:00 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3330930619' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-09T04:51:00.480 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:00 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:00.311+0000 7f890b550200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-09T04:51:00.630 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:00 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:00.371+0000 7ff3fe2f5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T04:51:00.630 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:00 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:00.469+0000 7ff3fe2f5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T04:51:00.671 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:00 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:00.372+0000 7f2296def200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T04:51:00.671 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:00 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:00.470+0000 7f2296def200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T04:51:00.671 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:00 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:00.672+0000 7f2296def200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T04:51:00.798 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:00 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:00.627+0000 7f890b550200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-09T04:51:00.798 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:00 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:00.728+0000 7f890b550200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-09T04:51:00.833 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-09T04:51:00.833 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-09T04:51:00.833 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi046.front.sepia.ceph.com 2024-05-09T04:51:00.833 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-09T04:51:00.833 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-09T04:51:00.833 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-09T04:51:00.833 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-09T04:51:00.834 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-09T04:51:00.834 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-09T04:51:00.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:51:00.834 DEBUG:teuthology.orchestra.run.smithi046:> ip addr 2024-05-09T04:51:00.852 INFO:teuthology.orchestra.run.smithi046.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-09T04:51:00.852 INFO:teuthology.orchestra.run.smithi046.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-09T04:51:00.853 INFO:teuthology.orchestra.run.smithi046.stdout: inet 127.0.0.1/8 scope host lo 2024-05-09T04:51:00.853 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:51:00.853 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 ::1/128 scope host 2024-05-09T04:51:00.853 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:51:00.853 INFO:teuthology.orchestra.run.smithi046.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:51:00.853 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:10 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:51:00.853 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f0 2024-05-09T04:51:00.853 INFO:teuthology.orchestra.run.smithi046.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:51:00.853 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:6b:f6:11 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:51:00.853 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp7s0f1 2024-05-09T04:51:00.853 INFO:teuthology.orchestra.run.smithi046.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-09T04:51:00.853 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f8 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:51:00.853 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f0 2024-05-09T04:51:00.853 INFO:teuthology.orchestra.run.smithi046.stdout: inet 172.21.15.46/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-09T04:51:00.854 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft 42019sec preferred_lft 42019sec 2024-05-09T04:51:00.854 INFO:teuthology.orchestra.run.smithi046.stdout: inet6 fe80::ec4:7aff:febd:15f8/64 scope link noprefixroute 2024-05-09T04:51:00.854 INFO:teuthology.orchestra.run.smithi046.stdout: valid_lft forever preferred_lft forever 2024-05-09T04:51:00.854 INFO:teuthology.orchestra.run.smithi046.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-09T04:51:00.854 INFO:teuthology.orchestra.run.smithi046.stdout: link/ether 0c:c4:7a:bd:15:f9 brd ff:ff:ff:ff:ff:ff 2024-05-09T04:51:00.854 INFO:teuthology.orchestra.run.smithi046.stdout: altname enp1s0f1 2024-05-09T04:51:00.854 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-09T04:51:00.854 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:51:00.854 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-09T04:51:00.855 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add name ceph-brx type bridge 2024-05-09T04:51:00.855 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr flush dev ceph-brx 2024-05-09T04:51:00.855 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set ceph-brx up 2024-05-09T04:51:00.855 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-09T04:51:00.855 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-09T04:51:00.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:51:01.048 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:00 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:00.960+0000 7f890b550200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T04:51:01.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:51:01.056 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:00 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:00.867+0000 7f2296def200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:51:01.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:51:01.059 DEBUG:teuthology.orchestra.run.smithi046:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-09T04:51:01.090 INFO:teuthology.orchestra.run.smithi046.stdout:1 2024-05-09T04:51:01.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:51:01.090 DEBUG:teuthology.orchestra.run.smithi046:> route 2024-05-09T04:51:01.130 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:00 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:00.674+0000 7ff3fe2f5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-09T04:51:01.130 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:00 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:00.868+0000 7ff3fe2f5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:51:01.165 INFO:teuthology.orchestra.run.smithi046.stdout:Kernel IP routing table 2024-05-09T04:51:01.165 INFO:teuthology.orchestra.run.smithi046.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-09T04:51:01.165 INFO:teuthology.orchestra.run.smithi046.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-09T04:51:01.165 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-09T04:51:01.165 INFO:teuthology.orchestra.run.smithi046.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-09T04:51:01.165 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:51:01.166 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-09T04:51:01.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-09T04:51:01.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-09T04:51:01.166 DEBUG:teuthology.orchestra.run.smithi046:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-09T04:51:01.166 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-09T04:51:01.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:51:01.330 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:01 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:01.186+0000 7f890b550200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-09T04:51:01.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:51:01.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:51:01.477 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-05-09T04:51:01.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:51:01.495 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list-id 2024-05-09T04:51:01.552 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:51:01.552 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-09T04:51:01.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T04:51:01.552 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-09T04:51:01.552 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-09T04:51:01.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:51:01.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:51:01.676 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-09T04:51:01.676 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:51:01.676 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-09T04:51:01.676 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-09T04:51:01.676 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-09T04:51:01.676 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-09T04:51:01.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-09T04:51:01.677 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-09T04:51:01.677 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-09T04:51:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:51:01.807 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:01 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:01.462+0000 7f2296def200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T04:51:01.807 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:01 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:01.683+0000 7f2296def200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T04:51:01.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:51:01.814 DEBUG:teuthology.orchestra.run.smithi046:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-09T04:51:01.814 DEBUG:teuthology.orchestra.run.smithi046:> set -e 2024-05-09T04:51:01.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set brx.0 up 2024-05-09T04:51:01.814 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link set dev brx.0 master ceph-brx 2024-05-09T04:51:01.815 DEBUG:teuthology.orchestra.run.smithi046:> ') 2024-05-09T04:51:01.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:01 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:51:01.880 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:01 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:01.467+0000 7ff3fe2f5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T04:51:01.880 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:01 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:01.691+0000 7ff3fe2f5200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T04:51:01.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:51:01.899 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:51:01.899 DEBUG:teuthology.orchestra.run.smithi046:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-09T04:51:01.927 INFO:teuthology.orchestra.run.smithi046.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-09T04:51:01.927 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-09T04:51:01.927 DEBUG:teuthology.orchestra.run.smithi046:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-09T04:51:01.985 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-09T04:51:01.985 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-09T04:51:01.985 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/6e0b2d8b-2a4b-496a-a206-fa632876bce3 2024-05-09T04:51:01.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:51:01.985 DEBUG:teuthology.orchestra.run.smithi046:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/6e0b2d8b-2a4b-496a-a206-fa632876bce3 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-09T04:51:02.048 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:01 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:01.857+0000 7f890b550200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-09T04:51:02.136 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-09T04:51:02.136 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-09T04:51:02.136 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-05-09T04:51:02.295 INFO:teuthology.orchestra.run.smithi046.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-05-09T04:51:02.295 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-05-09T04:51:02.295 INFO:teuthology.orchestra.run.smithi046.stdout:invalid new device string format 2024-05-09T04:51:02.296 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: resolved to: "172.21.15.46:3300,172.21.15.72:3300,172.21.15.110:3300" 2024-05-09T04:51:02.296 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.46:3300,172.21.15.72:3300,172.21.15.110:3300:/volumes/qa/sv_1/6e0b2d8b-2a4b-496a-a206-fa632876bce3 2024-05-09T04:51:02.296 INFO:teuthology.orchestra.run.smithi046.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=ecf9ae2c-0dbd-11ef-bc97-c7b262605968" will pass to kernel 2024-05-09T04:51:02.296 INFO:teuthology.orchestra.run.smithi046.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-09T04:51:02.297 INFO:teuthology.orchestra.run.smithi046.stdout: You just mounted a file system that supports labels which does not 2024-05-09T04:51:02.297 INFO:teuthology.orchestra.run.smithi046.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-09T04:51:02.297 INFO:teuthology.orchestra.run.smithi046.stdout: applications will generate AVC messages and not be allowed access to 2024-05-09T04:51:02.297 INFO:teuthology.orchestra.run.smithi046.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-09T04:51:02.298 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-09T04:51:02.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T04:51:02.299 DEBUG:teuthology.orchestra.run.smithi046:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-09T04:51:02.330 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:02 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:02.109+0000 7f890b550200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-09T04:51:02.358 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T04:51:02.380 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:02 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:02.118+0000 7ff3fe2f5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T04:51:02.380 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:02 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:02.214+0000 7ff3fe2f5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T04:51:02.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T04:51:02.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:51:02.459 INFO:teuthology.orchestra.run.smithi046.stdout:client15159 2024-05-09T04:51:02.459 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:51:02.459 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T04:51:02.476 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T04:51:02.557 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:02 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:02.107+0000 7f2296def200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T04:51:02.557 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:02 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:02.202+0000 7f2296def200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T04:51:02.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T04:51:02.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:51:02.577 INFO:teuthology.orchestra.run.smithi046.stdout:client15159 2024-05-09T04:51:02.577 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-09T04:51:02.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-09T04:51:02.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:51:02.638 INFO:teuthology.orchestra.run.smithi046.stdout:client15159 2024-05-09T04:51:02.639 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968.client15159/status') 2024-05-09T04:51:02.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-09T04:51:02.700 INFO:teuthology.orchestra.run.smithi046.stdout:instance: client.15159 (3)192.168.0.1:0/2565138792 2024-05-09T04:51:02.700 INFO:teuthology.orchestra.run.smithi046.stdout:blocklisted: false 2024-05-09T04:51:02.701 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records in 2024-05-09T04:51:02.701 INFO:teuthology.orchestra.run.smithi046.stderr:0+1 records out 2024-05-09T04:51:02.701 INFO:teuthology.orchestra.run.smithi046.stderr:70 bytes copied, 2.5376e-05 s, 2.8 MB/s 2024-05-09T04:51:02.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-09T04:51:02.708 INFO:teuthology.run_tasks:Running task exec... 2024-05-09T04:51:02.718 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T04:51:02.718 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T04:51:02.718 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-05-09T04:51:02.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.837+0000 7f4172619640 1 -- 172.21.15.46:0/432813736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f416c102b70 msgr2=0x7f416c103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:02.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.837+0000 7f4172619640 1 --2- 172.21.15.46:0/432813736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f416c102b70 0x7f416c103870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f415c009900 tx=0x7f415c02a410 comp rx=0 tx=0).stop 2024-05-09T04:51:02.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.837+0000 7f4172619640 1 -- 172.21.15.46:0/432813736 shutdown_connections 2024-05-09T04:51:02.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.837+0000 7f4172619640 1 --2- 172.21.15.46:0/432813736 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c108ab0 0x7f416c10aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:02.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.837+0000 7f4172619640 1 --2- 172.21.15.46:0/432813736 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c103db0 0x7f416c1041c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:02.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.837+0000 7f4172619640 1 --2- 172.21.15.46:0/432813736 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f416c102b70 0x7f416c103870 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:02.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.837+0000 7f4172619640 1 -- 172.21.15.46:0/432813736 >> 172.21.15.46:0/432813736 conn(0x7f416c0fdd60 msgr2=0x7f416c100210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:02.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.837+0000 7f4172619640 1 -- 172.21.15.46:0/432813736 shutdown_connections 2024-05-09T04:51:02.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.837+0000 7f4172619640 1 -- 172.21.15.46:0/432813736 wait complete. 2024-05-09T04:51:02.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.837+0000 7f4172619640 1 Processor -- start 2024-05-09T04:51:02.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.837+0000 7f4172619640 1 -- start start 2024-05-09T04:51:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f4172619640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c102b70 0x7f416c0ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f4172619640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c103db0 0x7f416c0fe290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f4172619640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f416c108ab0 0x7f416c0fe7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f4172619640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f416c10e650 con 0x7f416c108ab0 2024-05-09T04:51:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f4172619640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f416c10e4d0 con 0x7f416c103db0 2024-05-09T04:51:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f4172619640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f416c10e7d0 con 0x7f416c102b70 2024-05-09T04:51:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f4170bb6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f416c108ab0 0x7f416c0fe7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f416b7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c103db0 0x7f416c0fe290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f416b7fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c103db0 0x7f416c0fe290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:48520/0 (socket says 172.21.15.46:48520) 2024-05-09T04:51:02.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f4170bb6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f416c108ab0 0x7f416c0fe7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37226/0 (socket says 172.21.15.46:37226) 2024-05-09T04:51:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f416b7fe640 1 -- 172.21.15.46:0/2424632545 learned_addr learned my addr 172.21.15.46:0/2424632545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f416bfff640 1 --2- 172.21.15.46:0/2424632545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c102b70 0x7f416c0ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f416b7fe640 1 -- 172.21.15.46:0/2424632545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c102b70 msgr2=0x7f416c0ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f416b7fe640 1 --2- 172.21.15.46:0/2424632545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c102b70 0x7f416c0ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f416b7fe640 1 -- 172.21.15.46:0/2424632545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f416c108ab0 msgr2=0x7f416c0fe7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f416b7fe640 1 --2- 172.21.15.46:0/2424632545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f416c108ab0 0x7f416c0fe7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f416b7fe640 1 -- 172.21.15.46:0/2424632545 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f416c0fef00 con 0x7f416c103db0 2024-05-09T04:51:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.838+0000 7f4170bb6640 1 --2- 172.21.15.46:0/2424632545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f416c108ab0 0x7f416c0fe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.839+0000 7f416b7fe640 1 --2- 172.21.15.46:0/2424632545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c103db0 0x7f416c0fe290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f416000e9f0 tx=0x7f416000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.839+0000 7f41697fa640 1 -- 172.21.15.46:0/2424632545 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f416000cd80 con 0x7f416c103db0 2024-05-09T04:51:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.839+0000 7f41697fa640 1 -- 172.21.15.46:0/2424632545 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41600108e0 con 0x7f416c103db0 2024-05-09T04:51:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.839+0000 7f41697fa640 1 -- 172.21.15.46:0/2424632545 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4160018980 con 0x7f416c103db0 2024-05-09T04:51:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.839+0000 7f4172619640 1 -- 172.21.15.46:0/2424632545 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f416c1a4430 con 0x7f416c103db0 2024-05-09T04:51:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.839+0000 7f4172619640 1 -- 172.21.15.46:0/2424632545 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f416c1a4920 con 0x7f416c103db0 2024-05-09T04:51:02.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.840+0000 7f4172619640 1 -- 172.21.15.46:0/2424632545 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f416c1a4b80 con 0x7f416c103db0 2024-05-09T04:51:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.841+0000 7f41697fa640 1 -- 172.21.15.46:0/2424632545 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f4160010430 con 0x7f416c103db0 2024-05-09T04:51:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.842+0000 7f41697fa640 1 --2- 172.21.15.46:0/2424632545 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f41540b28b0 0x7f41540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.842+0000 7f416bfff640 1 -- 172.21.15.46:0/2424632545 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f41540b28b0 msgr2=0x7f41540b4dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3691531627 2024-05-09T04:51:02.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.842+0000 7f41697fa640 1 -- 172.21.15.46:0/2424632545 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f41600e2a80 con 0x7f416c103db0 2024-05-09T04:51:02.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.842+0000 7f416bfff640 1 --2- 172.21.15.46:0/2424632545 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f41540b28b0 0x7f41540b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:51:02.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:02.846+0000 7f41697fa640 1 -- 172.21.15.46:0/2424632545 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f41600aa0b0 con 0x7f416c103db0 2024-05-09T04:51:02.879 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:02 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:02.588+0000 7ff3fe2f5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T04:51:02.927 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:02 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:02.570+0000 7f2296def200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T04:51:02.927 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:02 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:02.926+0000 7f2296def200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T04:51:03.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.043+0000 7f416bfff640 1 -- 172.21.15.46:0/2424632545 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f41540b28b0 msgr2=0x7f41540b4dc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3691531627 2024-05-09T04:51:03.043 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.043+0000 7f416bfff640 1 --2- 172.21.15.46:0/2424632545 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f41540b28b0 0x7f41540b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:51:03.080 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:02 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:02.603+0000 7f890b550200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-09T04:51:03.080 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:02 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:02.712+0000 7f890b550200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-09T04:51:03.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.083+0000 7f4172619640 1 -- 172.21.15.46:0/2424632545 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7f416c1a4d50 con 0x7f416c103db0 2024-05-09T04:51:03.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.139+0000 7f41697fa640 1 -- 172.21.15.46:0/2424632545 <== mon.2 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v25) ==== 117+0+0 (secure 0 0 0) 0x7f41600ae3b0 con 0x7f416c103db0 2024-05-09T04:51:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.142+0000 7f4172619640 1 -- 172.21.15.46:0/2424632545 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f41540b28b0 msgr2=0x7f41540b4dc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:51:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.143+0000 7f4172619640 1 --2- 172.21.15.46:0/2424632545 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f41540b28b0 0x7f41540b4dc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.143+0000 7f4172619640 1 -- 172.21.15.46:0/2424632545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c103db0 msgr2=0x7f416c0fe290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.143+0000 7f4172619640 1 --2- 172.21.15.46:0/2424632545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c103db0 0x7f416c0fe290 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f416000e9f0 tx=0x7f416000eeb0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.143+0000 7f4172619640 1 -- 172.21.15.46:0/2424632545 shutdown_connections 2024-05-09T04:51:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.143+0000 7f4172619640 1 --2- 172.21.15.46:0/2424632545 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f41540b28b0 0x7f41540b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.143+0000 7f4172619640 1 --2- 172.21.15.46:0/2424632545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f416c108ab0 0x7f416c0fe7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.143+0000 7f4172619640 1 --2- 172.21.15.46:0/2424632545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f416c103db0 0x7f416c0fe290 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.143+0000 7f4172619640 1 --2- 172.21.15.46:0/2424632545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f416c102b70 0x7f416c0ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.143+0000 7f4172619640 1 -- 172.21.15.46:0/2424632545 >> 172.21.15.46:0/2424632545 conn(0x7f416c0fdd60 msgr2=0x7f416c109fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:03.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.143+0000 7f4172619640 1 -- 172.21.15.46:0/2424632545 shutdown_connections 2024-05-09T04:51:03.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.143+0000 7f4172619640 1 -- 172.21.15.46:0/2424632545 wait complete. 2024-05-09T04:51:03.154 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-09T04:51:03.165 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-05-09T04:51:03.213 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T04:51:03.264 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:03 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:03.020+0000 7f2296def200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T04:51:03.264 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:03 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2424632545' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T04:51:03.264 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:03 smithi110 ceph-mon[34904]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T04:51:03.284 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:03 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2424632545' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T04:51:03.284 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:03 smithi072 ceph-mon[34088]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T04:51:03.285 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:02 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:02.946+0000 7ff3fe2f5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T04:51:03.285 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:03 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:03.040+0000 7ff3fe2f5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T04:51:03.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.330+0000 7fcbd7fb2640 1 -- 172.21.15.46:0/2548805671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd0103780 msgr2=0x7fcbd0103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:03.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.330+0000 7fcbd7fb2640 1 --2- 172.21.15.46:0/2548805671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd0103780 0x7fcbd0103b90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcbc8009cd0 tx=0x7fcbc802a8e0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.330+0000 7fcbd7fb2640 1 -- 172.21.15.46:0/2548805671 shutdown_connections 2024-05-09T04:51:03.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.330+0000 7fcbd7fb2640 1 --2- 172.21.15.46:0/2548805671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd0102e30 0x7fcbd0103240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.330+0000 7fcbd7fb2640 1 --2- 172.21.15.46:0/2548805671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd0103780 0x7fcbd0103b90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.330+0000 7fcbd7fb2640 1 --2- 172.21.15.46:0/2548805671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbd01024e0 0x7fcbd01028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.330+0000 7fcbd7fb2640 1 -- 172.21.15.46:0/2548805671 >> 172.21.15.46:0/2548805671 conn(0x7fcbd00fdcd0 msgr2=0x7fcbd01001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.330+0000 7fcbd7fb2640 1 -- 172.21.15.46:0/2548805671 shutdown_connections 2024-05-09T04:51:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.330+0000 7fcbd7fb2640 1 -- 172.21.15.46:0/2548805671 wait complete. 2024-05-09T04:51:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.331+0000 7fcbd7fb2640 1 Processor -- start 2024-05-09T04:51:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.331+0000 7fcbd7fb2640 1 -- start start 2024-05-09T04:51:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.331+0000 7fcbd7fb2640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbd01024e0 0x7fcbd019c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.331+0000 7fcbd7fb2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd0102e30 0x7fcbd019ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.331+0000 7fcbd7fb2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd0103780 0x7fcbd01a39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.331+0000 7fcbd7fb2640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd0111920 con 0x7fcbd0103780 2024-05-09T04:51:03.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.331+0000 7fcbd7fb2640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd01117a0 con 0x7fcbd01024e0 2024-05-09T04:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.331+0000 7fcbd7fb2640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcbd0111aa0 con 0x7fcbd0102e30 2024-05-09T04:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.331+0000 7fcbd654f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd0103780 0x7fcbd01a39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.331+0000 7fcbd5d4e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbd01024e0 0x7fcbd019c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.331+0000 7fcbd554d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd0102e30 0x7fcbd019ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.331+0000 7fcbd654f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd0103780 0x7fcbd01a39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:37244/0 (socket says 172.21.15.46:37244) 2024-05-09T04:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.331+0000 7fcbd654f640 1 -- 172.21.15.46:0/2108113597 learned_addr learned my addr 172.21.15.46:0/2108113597 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.332+0000 7fcbd5d4e640 1 -- 172.21.15.46:0/2108113597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd0102e30 msgr2=0x7fcbd019ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.332+0000 7fcbd5d4e640 1 --2- 172.21.15.46:0/2108113597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd0102e30 0x7fcbd019ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.332+0000 7fcbd5d4e640 1 -- 172.21.15.46:0/2108113597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd0103780 msgr2=0x7fcbd01a39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.332+0000 7fcbd5d4e640 1 --2- 172.21.15.46:0/2108113597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd0103780 0x7fcbd01a39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.332+0000 7fcbd5d4e640 1 -- 172.21.15.46:0/2108113597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcbd01a4060 con 0x7fcbd01024e0 2024-05-09T04:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.332+0000 7fcbd5d4e640 1 --2- 172.21.15.46:0/2108113597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbd01024e0 0x7fcbd019c520 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00e9c0 tx=0x7fcbcc00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:03.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.332+0000 7fcbc6ffd640 1 -- 172.21.15.46:0/2108113597 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc00ccc0 con 0x7fcbd01024e0 2024-05-09T04:51:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.332+0000 7fcbd7fb2640 1 -- 172.21.15.46:0/2108113597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcbd01a4350 con 0x7fcbd01024e0 2024-05-09T04:51:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.332+0000 7fcbc6ffd640 1 -- 172.21.15.46:0/2108113597 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbcc0108e0 con 0x7fcbd01024e0 2024-05-09T04:51:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.332+0000 7fcbc6ffd640 1 -- 172.21.15.46:0/2108113597 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbcc0187a0 con 0x7fcbd01024e0 2024-05-09T04:51:03.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.332+0000 7fcbd7fb2640 1 -- 172.21.15.46:0/2108113597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcbd01a48f0 con 0x7fcbd01024e0 2024-05-09T04:51:03.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.333+0000 7fcbd7fb2640 1 -- 172.21.15.46:0/2108113597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcbd010d520 con 0x7fcbd01024e0 2024-05-09T04:51:03.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.335+0000 7fcbc6ffd640 1 -- 172.21.15.46:0/2108113597 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7fcbcc010430 con 0x7fcbd01024e0 2024-05-09T04:51:03.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.335+0000 7fcbc6ffd640 1 --2- 172.21.15.46:0/2108113597 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fcbbc0b2840 0x7fcbbc0b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:03.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.335+0000 7fcbd554d640 1 -- 172.21.15.46:0/2108113597 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fcbbc0b2840 msgr2=0x7fcbbc0b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3691531627 2024-05-09T04:51:03.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.335+0000 7fcbc6ffd640 1 -- 172.21.15.46:0/2108113597 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7fcbcc0e2670 con 0x7fcbd01024e0 2024-05-09T04:51:03.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.335+0000 7fcbd554d640 1 --2- 172.21.15.46:0/2108113597 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fcbbc0b2840 0x7fcbbc0b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:51:03.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.340+0000 7fcbc6ffd640 1 -- 172.21.15.46:0/2108113597 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcbcc0a9ca0 con 0x7fcbd01024e0 2024-05-09T04:51:03.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:03 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2424632545' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T04:51:03.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:03 smithi046 ceph-mon[30136]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-05-09T04:51:03.509 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:03 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:03.114+0000 7f890b550200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-09T04:51:03.509 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:03 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:03.508+0000 7f890b550200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-09T04:51:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.536+0000 7fcbd554d640 1 -- 172.21.15.46:0/2108113597 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fcbbc0b2840 msgr2=0x7fcbbc0b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3691531627 2024-05-09T04:51:03.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.536+0000 7fcbd554d640 1 --2- 172.21.15.46:0/2108113597 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fcbbc0b2840 0x7fcbbc0b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:51:03.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.537+0000 7fcbd7fb2640 1 -- 172.21.15.46:0/2108113597 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcbd00638d0 con 0x7fcbd01024e0 2024-05-09T04:51:03.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.539+0000 7fcbc6ffd640 1 -- 172.21.15.46:0/2108113597 <== mon.2 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) ==== 74+0+22685 (secure 0 0 0) 0x7fcbcc0adfa0 con 0x7fcbd01024e0 2024-05-09T04:51:03.539 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:51:03.540 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":84,"fsid":"ecf9ae2c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:25.633876+0000","modified":"2024-05-09T04:49:29.385454+0000","last_up_change":"2024-05-09T04:46:46.418597+0000","last_in_change":"2024-05-09T04:46:25.903582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:42:53.301400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:47:43.413569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:47:47.264915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d79cd3e7-6f82-4d60-96d9-26d8146c962f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6803","nonce":1576436508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6805","nonce":1576436508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6809","nonce":1576436508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6807","nonce":1576436508}]},"public_addr":"172.21.15.46:6803/1576436508","cluster_addr":"172.21.15.46:6805/1576436508","heartbeat_back_addr":"172.21.15.46:6809/1576436508","heartbeat_front_addr":"172.21.15.46:6807/1576436508","state":["exists","up"]},{"osd":1,"uuid":"698eba80-1ebd-4076-b6f9-953d2e7daf76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6801","nonce":379039775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6803","nonce":379039775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6807","nonce":379039775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6805","nonce":379039775}]},"public_addr":"172.21.15.72:6801/379039775","cluster_addr":"172.21.15.72:6803/379039775","heartbeat_back_addr":"172.21.15.72:6807/379039775","heartbeat_front_addr":"172.21.15.72:6805/379039775","state":["exists","up"]},{"osd":2,"uuid":"c549b131-cfa7-4724-926f-83e91a752d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6801","nonce":293447227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6803","nonce":293447227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6807","nonce":293447227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6805","nonce":293447227}]},"public_addr":"172.21.15.110:6801/293447227","cluster_addr":"172.21.15.110:6803/293447227","heartbeat_back_addr":"172.21.15.110:6807/293447227","heartbeat_front_addr":"172.21.15.110:6805/293447227","state":["exists","up"]},{"osd":3,"uuid":"549bb4f7-54d9-4cbc-ad40-7f3e717964f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6811","nonce":1115835653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6813","nonce":1115835653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6817","nonce":1115835653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6815","nonce":1115835653}]},"public_addr":"172.21.15.46:6811/1115835653","cluster_addr":"172.21.15.46:6813/1115835653","heartbeat_back_addr":"172.21.15.46:6817/1115835653","heartbeat_front_addr":"172.21.15.46:6815/1115835653","state":["exists","up"]},{"osd":4,"uuid":"444cd36b-6034-46cd-9a6b-29dce4ba6405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6809","nonce":59186693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6811","nonce":59186693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6815","nonce":59186693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6813","nonce":59186693}]},"public_addr":"172.21.15.72:6809/59186693","cluster_addr":"172.21.15.72:6811/59186693","heartbeat_back_addr":"172.21.15.72:6815/59186693","heartbeat_front_addr":"172.21.15.72:6813/59186693","state":["exists","up"]},{"osd":5,"uuid":"2c2bdd2f-43cf-4208-9ae1-f912807c84e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6809","nonce":1712162502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6811","nonce":1712162502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6815","nonce":1712162502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6813","nonce":1712162502}]},"public_addr":"172.21.15.110:6809/1712162502","cluster_addr":"172.21.15.110:6811/1712162502","heartbeat_back_addr":"172.21.15.110:6815/1712162502","heartbeat_front_addr":"172.21.15.110:6813/1712162502","state":["exists","up"]},{"osd":6,"uuid":"beb0b87f-c3a4-40dd-b224-1843e9301ca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6819","nonce":2429650065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6821","nonce":2429650065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6825","nonce":2429650065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6823","nonce":2429650065}]},"public_addr":"172.21.15.46:6819/2429650065","cluster_addr":"172.21.15.46:6821/2429650065","heartbeat_back_addr":"172.21.15.46:6825/2429650065","heartbeat_front_addr":"172.21.15.46:6823/2429650065","state":["exists","up"]},{"osd":7,"uuid":"4a634675-fad7-4a43-840c-8dec2631dec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6817","nonce":337796672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6819","nonce":337796672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6823","nonce":337796672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6821","nonce":337796672}]},"public_addr":"172.21.15.72:6817/337796672","cluster_addr":"172.21.15.72:6819/337796672","heartbeat_back_addr":"172.21.15.72:6823/337796672","heartbeat_front_addr":"172.21.15.72:6821/337796672","state":["exists","up"]},{"osd":8,"uuid":"01b5f638-90f0-4cf4-8042-20d20a725669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6817","nonce":189513203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6819","nonce":189513203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6823","nonce":189513203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6821","nonce":189513203}]},"public_addr":"172.21.15.110:6817/189513203","cluster_addr":"172.21.15.110:6819/189513203","heartbeat_back_addr":"172.21.15.110:6823/189513203","heartbeat_front_addr":"172.21.15.110:6821/189513203","state":["exists","up"]},{"osd":9,"uuid":"e423de41-50ec-48f4-a4ed-216de33c4fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6827","nonce":1762262822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6829","nonce":1762262822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6833","nonce":1762262822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6831","nonce":1762262822}]},"public_addr":"172.21.15.46:6827/1762262822","cluster_addr":"172.21.15.46:6829/1762262822","heartbeat_back_addr":"172.21.15.46:6833/1762262822","heartbeat_front_addr":"172.21.15.46:6831/1762262822","state":["exists","up"]},{"osd":10,"uuid":"f594d004-9f80-4508-b127-fb693d355f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6825","nonce":38453898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6827","nonce":38453898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6831","nonce":38453898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6829","nonce":38453898}]},"public_addr":"172.21.15.72:6825/38453898","cluster_addr":"172.21.15.72:6827/38453898","heartbeat_back_addr":"172.21.15.72:6831/38453898","heartbeat_front_addr":"172.21.15.72:6829/38453898","state":["exists","up"]},{"osd":11,"uuid":"f771d29f-89e3-4f96-9c29-00e424230c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6825","nonce":4153529632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6827","nonce":4153529632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6831","nonce":4153529632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6829","nonce":4153529632}]},"public_addr":"172.21.15.110:6825/4153529632","cluster_addr":"172.21.15.110:6827/4153529632","heartbeat_back_addr":"172.21.15.110:6831/4153529632","heartbeat_front_addr":"172.21.15.110:6829/4153529632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:02.615801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:26.453477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:50.334627+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:15.357320+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:40.403549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:06.030093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:31.904091+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:57.616372+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:24.045975+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:51.160898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:17.597845+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:44.636629+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:6800/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1596672353":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1846482551":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/1069571878":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6800/2738158981":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/3625745146":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6801/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/564539954":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1797199863":"2024-05-10T04:40:27.201119+0000","172.21.15.46:6801/2738158981":"2024-05-10T04:39:59.515628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:51:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.541+0000 7fcbd7fb2640 1 -- 172.21.15.46:0/2108113597 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fcbbc0b2840 msgr2=0x7fcbbc0b4d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:51:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.541+0000 7fcbd7fb2640 1 --2- 172.21.15.46:0/2108113597 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fcbbc0b2840 0x7fcbbc0b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.541+0000 7fcbd7fb2640 1 -- 172.21.15.46:0/2108113597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbd01024e0 msgr2=0x7fcbd019c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:03.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.541+0000 7fcbd7fb2640 1 --2- 172.21.15.46:0/2108113597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbd01024e0 0x7fcbd019c520 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcbcc00e9c0 tx=0x7fcbcc00ee80 comp rx=0 tx=0).stop 2024-05-09T04:51:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.541+0000 7fcbd7fb2640 1 -- 172.21.15.46:0/2108113597 shutdown_connections 2024-05-09T04:51:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.541+0000 7fcbd7fb2640 1 --2- 172.21.15.46:0/2108113597 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7fcbbc0b2840 0x7fcbbc0b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.541+0000 7fcbd7fb2640 1 --2- 172.21.15.46:0/2108113597 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcbd0102e30 0x7fcbd019ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.541+0000 7fcbd7fb2640 1 --2- 172.21.15.46:0/2108113597 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcbd0103780 0x7fcbd01a39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.541+0000 7fcbd7fb2640 1 --2- 172.21.15.46:0/2108113597 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcbd01024e0 0x7fcbd019c520 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.541+0000 7fcbd7fb2640 1 -- 172.21.15.46:0/2108113597 >> 172.21.15.46:0/2108113597 conn(0x7fcbd00fdcd0 msgr2=0x7fcbd010a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.541+0000 7fcbd7fb2640 1 -- 172.21.15.46:0/2108113597 shutdown_connections 2024-05-09T04:51:03.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.541+0000 7fcbd7fb2640 1 -- 172.21.15.46:0/2108113597 wait complete. 2024-05-09T04:51:03.554 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T04:42:53.301400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T04:47:43.413569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T04:47:47.264915+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T04:51:03.555 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T04:51:03.557 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:03 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:03.263+0000 7f2296def200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T04:51:03.557 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:03 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:03.398+0000 7f2296def200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T04:51:03.557 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:03 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:03.493+0000 7f2296def200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T04:51:03.630 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:03 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:03.284+0000 7ff3fe2f5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T04:51:03.630 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:03 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:03.417+0000 7ff3fe2f5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T04:51:03.630 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:03 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:03.514+0000 7ff3fe2f5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T04:51:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.676+0000 7f3bb6f9d640 1 -- 172.21.15.46:0/2269761850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb0102b10 msgr2=0x7f3bb0103810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.676+0000 7f3bb6f9d640 1 --2- 172.21.15.46:0/2269761850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb0102b10 0x7f3bb0103810 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3ba4009900 tx=0x7f3ba402a410 comp rx=0 tx=0).stop 2024-05-09T04:51:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.676+0000 7f3bb6f9d640 1 -- 172.21.15.46:0/2269761850 shutdown_connections 2024-05-09T04:51:03.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.676+0000 7f3bb6f9d640 1 --2- 172.21.15.46:0/2269761850 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bb0103d50 0x7f3bb0104160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.676+0000 7f3bb6f9d640 1 --2- 172.21.15.46:0/2269761850 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bb0108a50 0x7f3bb010ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.676+0000 7f3bb6f9d640 1 --2- 172.21.15.46:0/2269761850 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb0102b10 0x7f3bb0103810 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.676+0000 7f3bb6f9d640 1 -- 172.21.15.46:0/2269761850 >> 172.21.15.46:0/2269761850 conn(0x7f3bb00fdd00 msgr2=0x7f3bb01001b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.676+0000 7f3bb6f9d640 1 -- 172.21.15.46:0/2269761850 shutdown_connections 2024-05-09T04:51:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.676+0000 7f3bb6f9d640 1 -- 172.21.15.46:0/2269761850 wait complete. 2024-05-09T04:51:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb6f9d640 1 Processor -- start 2024-05-09T04:51:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb6f9d640 1 -- start start 2024-05-09T04:51:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb6f9d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bb0102b10 0x7f3bb019a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb6f9d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bb0103d50 0x7f3bb019ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb6f9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb0108a50 0x7f3bb01a1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb6f9d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb010e610 con 0x7f3bb0108a50 2024-05-09T04:51:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb6f9d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb010e490 con 0x7f3bb0103d50 2024-05-09T04:51:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb6f9d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3bb010e790 con 0x7f3bb0102b10 2024-05-09T04:51:03.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb553a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb0108a50 0x7f3bb01a1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb4d39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bb0102b10 0x7f3bb019a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb4d39640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bb0102b10 0x7f3bb019a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:56596/0 (socket says 172.21.15.46:56596) 2024-05-09T04:51:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb4d39640 1 -- 172.21.15.46:0/1949873949 learned_addr learned my addr 172.21.15.46:0/1949873949 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3baffff640 1 --2- 172.21.15.46:0/1949873949 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bb0103d50 0x7f3bb019ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb553a640 1 -- 172.21.15.46:0/1949873949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bb0102b10 msgr2=0x7f3bb019a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb553a640 1 --2- 172.21.15.46:0/1949873949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bb0102b10 0x7f3bb019a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb553a640 1 -- 172.21.15.46:0/1949873949 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bb0103d50 msgr2=0x7f3bb019ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb553a640 1 --2- 172.21.15.46:0/1949873949 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bb0103d50 0x7f3bb019ac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb553a640 1 -- 172.21.15.46:0/1949873949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bb01a2240 con 0x7f3bb0108a50 2024-05-09T04:51:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3bb4d39640 1 --2- 172.21.15.46:0/1949873949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bb0102b10 0x7f3bb019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.677+0000 7f3baffff640 1 --2- 172.21.15.46:0/1949873949 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bb0103d50 0x7f3bb019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.678+0000 7f3bb553a640 1 --2- 172.21.15.46:0/1949873949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb0108a50 0x7f3bb01a1bd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009fd0 tx=0x7f3ba800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:03.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.678+0000 7f3badffb640 1 -- 172.21.15.46:0/1949873949 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba800ce20 con 0x7f3bb0108a50 2024-05-09T04:51:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.678+0000 7f3bb6f9d640 1 -- 172.21.15.46:0/1949873949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bb01a2530 con 0x7f3bb0108a50 2024-05-09T04:51:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.678+0000 7f3badffb640 1 -- 172.21.15.46:0/1949873949 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ba8005020 con 0x7f3bb0108a50 2024-05-09T04:51:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.678+0000 7f3badffb640 1 -- 172.21.15.46:0/1949873949 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ba8010690 con 0x7f3bb0108a50 2024-05-09T04:51:03.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.678+0000 7f3bb6f9d640 1 -- 172.21.15.46:0/1949873949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bb01a2a30 con 0x7f3bb0108a50 2024-05-09T04:51:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.679+0000 7f3bb6f9d640 1 -- 172.21.15.46:0/1949873949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3bb006c3d0 con 0x7f3bb0108a50 2024-05-09T04:51:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.681+0000 7f3badffb640 1 -- 172.21.15.46:0/1949873949 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f3ba80054d0 con 0x7f3bb0108a50 2024-05-09T04:51:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.681+0000 7f3badffb640 1 --2- 172.21.15.46:0/1949873949 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3ba00b2840 0x7f3ba00b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.682+0000 7f3bb4d39640 1 -- 172.21.15.46:0/1949873949 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3ba00b2840 msgr2=0x7f3ba00b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3691531627 2024-05-09T04:51:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.682+0000 7f3badffb640 1 -- 172.21.15.46:0/1949873949 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f3ba80e2960 con 0x7f3bb0108a50 2024-05-09T04:51:03.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.682+0000 7f3bb4d39640 1 --2- 172.21.15.46:0/1949873949 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3ba00b2840 0x7f3ba00b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:51:03.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.687+0000 7f3badffb640 1 -- 172.21.15.46:0/1949873949 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3ba80aa640 con 0x7f3bb0108a50 2024-05-09T04:51:03.830 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:03 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:03.614+0000 7f890b550200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-09T04:51:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.883+0000 7f3bb4d39640 1 -- 172.21.15.46:0/1949873949 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3ba00b2840 msgr2=0x7f3ba00b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3691531627 2024-05-09T04:51:03.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.883+0000 7f3bb4d39640 1 --2- 172.21.15.46:0/1949873949 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3ba00b2840 0x7f3ba00b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:51:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.883+0000 7f3bb6f9d640 1 -- 172.21.15.46:0/1949873949 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3bb01a2cf0 con 0x7f3bb0108a50 2024-05-09T04:51:03.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.884+0000 7f3badffb640 1 -- 172.21.15.46:0/1949873949 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) ==== 93+0+10 (secure 0 0 0) 0x7f3ba80a9f40 con 0x7f3bb0108a50 2024-05-09T04:51:03.885 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-09T04:51:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.886+0000 7f3bb6f9d640 1 -- 172.21.15.46:0/1949873949 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3ba00b2840 msgr2=0x7f3ba00b4d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:51:03.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.886+0000 7f3bb6f9d640 1 --2- 172.21.15.46:0/1949873949 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3ba00b2840 0x7f3ba00b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.887+0000 7f3bb6f9d640 1 -- 172.21.15.46:0/1949873949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb0108a50 msgr2=0x7f3bb01a1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.887+0000 7f3bb6f9d640 1 --2- 172.21.15.46:0/1949873949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb0108a50 0x7f3bb01a1bd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009fd0 tx=0x7f3ba800ef10 comp rx=0 tx=0).stop 2024-05-09T04:51:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.887+0000 7f3bb6f9d640 1 -- 172.21.15.46:0/1949873949 shutdown_connections 2024-05-09T04:51:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.887+0000 7f3bb6f9d640 1 --2- 172.21.15.46:0/1949873949 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3ba00b2840 0x7f3ba00b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.887+0000 7f3bb6f9d640 1 --2- 172.21.15.46:0/1949873949 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3bb0108a50 0x7f3bb01a1bd0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.887+0000 7f3bb6f9d640 1 --2- 172.21.15.46:0/1949873949 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3bb0103d50 0x7f3bb019ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.887+0000 7f3bb6f9d640 1 --2- 172.21.15.46:0/1949873949 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3bb0102b10 0x7f3bb019a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.887+0000 7f3bb6f9d640 1 -- 172.21.15.46:0/1949873949 >> 172.21.15.46:0/1949873949 conn(0x7f3bb00fdd00 msgr2=0x7f3bb0109f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.887+0000 7f3bb6f9d640 1 -- 172.21.15.46:0/1949873949 shutdown_connections 2024-05-09T04:51:03.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:03.887+0000 7f3bb6f9d640 1 -- 172.21.15.46:0/1949873949 wait complete. 2024-05-09T04:51:03.899 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T04:51:04.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.019+0000 7f8e6aa9a640 1 -- 172.21.15.46:0/1875685032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e64109a30 msgr2=0x7f8e6410be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:04.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.019+0000 7f8e6aa9a640 1 --2- 172.21.15.46:0/1875685032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e64109a30 0x7f8e6410be50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8e5c009ca0 tx=0x7f8e5c02a8c0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.019+0000 7f8e6aa9a640 1 -- 172.21.15.46:0/1875685032 shutdown_connections 2024-05-09T04:51:04.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.019+0000 7f8e6aa9a640 1 --2- 172.21.15.46:0/1875685032 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e64109a30 0x7f8e6410be50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.019+0000 7f8e6aa9a640 1 --2- 172.21.15.46:0/1875685032 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e641070d0 0x7f8e641094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.019+0000 7f8e6aa9a640 1 --2- 172.21.15.46:0/1875685032 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e64104770 0x7f8e64106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.019+0000 7f8e6aa9a640 1 -- 172.21.15.46:0/1875685032 >> 172.21.15.46:0/1875685032 conn(0x7f8e640fdd70 msgr2=0x7f8e64100220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:04.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.019+0000 7f8e6aa9a640 1 -- 172.21.15.46:0/1875685032 shutdown_connections 2024-05-09T04:51:04.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.019+0000 7f8e6aa9a640 1 -- 172.21.15.46:0/1875685032 wait complete. 2024-05-09T04:51:04.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e6aa9a640 1 Processor -- start 2024-05-09T04:51:04.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e6aa9a640 1 -- start start 2024-05-09T04:51:04.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e6aa9a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e64104770 0x7f8e6419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:04.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e6aa9a640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e641070d0 0x7f8e6419a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:04.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e6aa9a640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e64109a30 0x7f8e641a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:04.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e6aa9a640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6410f740 con 0x7f8e64104770 2024-05-09T04:51:04.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e6aa9a640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6410f5c0 con 0x7f8e64109a30 2024-05-09T04:51:04.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e6aa9a640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8e6410f8c0 con 0x7f8e641070d0 2024-05-09T04:51:04.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e68836640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e64104770 0x7f8e6419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:04.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e68836640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e64104770 0x7f8e6419a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36220/0 (socket says 172.21.15.46:36220) 2024-05-09T04:51:04.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e68836640 1 -- 172.21.15.46:0/778004765 learned_addr learned my addr 172.21.15.46:0/778004765 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:04.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e69037640 1 --2- 172.21.15.46:0/778004765 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e64109a30 0x7f8e641a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:04.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e63fff640 1 --2- 172.21.15.46:0/778004765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e641070d0 0x7f8e6419a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:04.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e68836640 1 -- 172.21.15.46:0/778004765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e641070d0 msgr2=0x7f8e6419a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:04.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e68836640 1 --2- 172.21.15.46:0/778004765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e641070d0 0x7f8e6419a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e68836640 1 -- 172.21.15.46:0/778004765 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e64109a30 msgr2=0x7f8e641a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:04.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e68836640 1 --2- 172.21.15.46:0/778004765 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e64109a30 0x7f8e641a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.020+0000 7f8e68836640 1 -- 172.21.15.46:0/778004765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e641a1e80 con 0x7f8e64104770 2024-05-09T04:51:04.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.021+0000 7f8e68836640 1 --2- 172.21.15.46:0/778004765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e64104770 0x7f8e6419a340 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800e9f0 tx=0x7f8e5800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:04.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.021+0000 7f8e61ffb640 1 -- 172.21.15.46:0/778004765 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e5800ccc0 con 0x7f8e64104770 2024-05-09T04:51:04.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.021+0000 7f8e61ffb640 1 -- 172.21.15.46:0/778004765 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e580054d0 con 0x7f8e64104770 2024-05-09T04:51:04.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.021+0000 7f8e61ffb640 1 -- 172.21.15.46:0/778004765 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e58010430 con 0x7f8e64104770 2024-05-09T04:51:04.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.021+0000 7f8e6aa9a640 1 -- 172.21.15.46:0/778004765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e641a2170 con 0x7f8e64104770 2024-05-09T04:51:04.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.021+0000 7f8e6aa9a640 1 -- 172.21.15.46:0/778004765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e641a2670 con 0x7f8e64104770 2024-05-09T04:51:04.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.022+0000 7f8e6aa9a640 1 -- 172.21.15.46:0/778004765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e640ff890 con 0x7f8e64104770 2024-05-09T04:51:04.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.024+0000 7f8e61ffb640 1 -- 172.21.15.46:0/778004765 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f8e580046c0 con 0x7f8e64104770 2024-05-09T04:51:04.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.024+0000 7f8e61ffb640 1 --2- 172.21.15.46:0/778004765 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f8e540b2840 0x7f8e540b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:04.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.024+0000 7f8e63fff640 1 -- 172.21.15.46:0/778004765 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f8e540b2840 msgr2=0x7f8e540b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3691531627 2024-05-09T04:51:04.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.025+0000 7f8e61ffb640 1 -- 172.21.15.46:0/778004765 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f8e580e2850 con 0x7f8e64104770 2024-05-09T04:51:04.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.025+0000 7f8e63fff640 1 --2- 172.21.15.46:0/778004765 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f8e540b2840 0x7f8e540b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:51:04.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.032+0000 7f8e61ffb640 1 -- 172.21.15.46:0/778004765 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8e58054df0 con 0x7f8e64104770 2024-05-09T04:51:04.130 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:03 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:03.841+0000 7ff3fe2f5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T04:51:04.130 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:03 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:03.933+0000 7ff3fe2f5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T04:51:04.141 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:03 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:03.889+0000 7f890b550200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-09T04:51:04.141 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:04.046+0000 7f890b550200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-09T04:51:04.141 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:03 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:03.815+0000 7f2296def200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T04:51:04.141 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:03 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:03.908+0000 7f2296def200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T04:51:04.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.226+0000 7f8e63fff640 1 -- 172.21.15.46:0/778004765 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f8e540b2840 msgr2=0x7f8e540b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3691531627 2024-05-09T04:51:04.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.226+0000 7f8e63fff640 1 --2- 172.21.15.46:0/778004765 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f8e540b2840 0x7f8e540b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-09T04:51:04.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.226+0000 7f8e6aa9a640 1 -- 172.21.15.46:0/778004765 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f8e640638d0 con 0x7f8e64104770 2024-05-09T04:51:04.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.227+0000 7f8e61ffb640 1 -- 172.21.15.46:0/778004765 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) ==== 104+0+11 (secure 0 0 0) 0x7f8e580aa530 con 0x7f8e64104770 2024-05-09T04:51:04.227 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-09T04:51:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.229+0000 7f8e6aa9a640 1 -- 172.21.15.46:0/778004765 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f8e540b2840 msgr2=0x7f8e540b4d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:51:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.229+0000 7f8e6aa9a640 1 --2- 172.21.15.46:0/778004765 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f8e540b2840 0x7f8e540b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.229+0000 7f8e6aa9a640 1 -- 172.21.15.46:0/778004765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e64104770 msgr2=0x7f8e6419a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:04.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.229+0000 7f8e6aa9a640 1 --2- 172.21.15.46:0/778004765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e64104770 0x7f8e6419a340 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8e5800e9f0 tx=0x7f8e5800eeb0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.229+0000 7f8e6aa9a640 1 -- 172.21.15.46:0/778004765 shutdown_connections 2024-05-09T04:51:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.229+0000 7f8e6aa9a640 1 --2- 172.21.15.46:0/778004765 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f8e540b2840 0x7f8e540b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.229+0000 7f8e6aa9a640 1 --2- 172.21.15.46:0/778004765 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8e641070d0 0x7f8e6419a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.229+0000 7f8e6aa9a640 1 --2- 172.21.15.46:0/778004765 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8e64109a30 0x7f8e641a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.229+0000 7f8e6aa9a640 1 --2- 172.21.15.46:0/778004765 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8e64104770 0x7f8e6419a340 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.229+0000 7f8e6aa9a640 1 -- 172.21.15.46:0/778004765 >> 172.21.15.46:0/778004765 conn(0x7f8e640fdd70 msgr2=0x7f8e64105710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.229+0000 7f8e6aa9a640 1 -- 172.21.15.46:0/778004765 shutdown_connections 2024-05-09T04:51:04.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.229+0000 7f8e6aa9a640 1 -- 172.21.15.46:0/778004765 wait complete. 2024-05-09T04:51:04.241 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T04:51:04.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.365+0000 7f3dae534640 1 -- 172.21.15.46:0/2522644738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da8103c10 msgr2=0x7f3da8104910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:04.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.365+0000 7f3dae534640 1 --2- 172.21.15.46:0/2522644738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da8103c10 0x7f3da8104910 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009900 tx=0x7f3d9802a410 comp rx=0 tx=0).stop 2024-05-09T04:51:04.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.365+0000 7f3dae534640 1 -- 172.21.15.46:0/2522644738 shutdown_connections 2024-05-09T04:51:04.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.365+0000 7f3dae534640 1 --2- 172.21.15.46:0/2522644738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3da8109b50 0x7f3da810bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.365+0000 7f3dae534640 1 --2- 172.21.15.46:0/2522644738 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3da8104e50 0x7f3da8105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.365+0000 7f3dae534640 1 --2- 172.21.15.46:0/2522644738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da8103c10 0x7f3da8104910 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.365+0000 7f3dae534640 1 -- 172.21.15.46:0/2522644738 >> 172.21.15.46:0/2522644738 conn(0x7f3da80fee00 msgr2=0x7f3da81012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:04.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.365+0000 7f3dae534640 1 -- 172.21.15.46:0/2522644738 shutdown_connections 2024-05-09T04:51:04.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.365+0000 7f3dae534640 1 -- 172.21.15.46:0/2522644738 wait complete. 2024-05-09T04:51:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.366+0000 7f3dae534640 1 Processor -- start 2024-05-09T04:51:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.366+0000 7f3dae534640 1 -- start start 2024-05-09T04:51:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.366+0000 7f3dae534640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3da8103c10 0x7f3da819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:04.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.366+0000 7f3dae534640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3da8104e50 0x7f3da819a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.366+0000 7f3dae534640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da8109b50 0x7f3da81a1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.366+0000 7f3dae534640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3da810f710 con 0x7f3da8109b50 2024-05-09T04:51:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.366+0000 7f3dae534640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3da810f590 con 0x7f3da8104e50 2024-05-09T04:51:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.366+0000 7f3dae534640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3da810f890 con 0x7f3da8103c10 2024-05-09T04:51:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.366+0000 7f3dacad1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da8109b50 0x7f3da81a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.366+0000 7f3dacad1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da8109b50 0x7f3da81a1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36238/0 (socket says 172.21.15.46:36238) 2024-05-09T04:51:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.366+0000 7f3dacad1640 1 -- 172.21.15.46:0/3711789415 learned_addr learned my addr 172.21.15.46:0/3711789415 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.366+0000 7f3da7fff640 1 --2- 172.21.15.46:0/3711789415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3da8103c10 0x7f3da819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.366+0000 7f3d9ffff640 1 --2- 172.21.15.46:0/3711789415 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3da8104e50 0x7f3da819a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.367+0000 7f3dacad1640 1 -- 172.21.15.46:0/3711789415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3da8103c10 msgr2=0x7f3da819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:04.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.367+0000 7f3dacad1640 1 --2- 172.21.15.46:0/3711789415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3da8103c10 0x7f3da819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.367+0000 7f3dacad1640 1 -- 172.21.15.46:0/3711789415 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3da8104e50 msgr2=0x7f3da819a880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:04.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.367+0000 7f3dacad1640 1 --2- 172.21.15.46:0/3711789415 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3da8104e50 0x7f3da819a880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.367+0000 7f3dacad1640 1 -- 172.21.15.46:0/3711789415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da81a1e80 con 0x7f3da8109b50 2024-05-09T04:51:04.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.367+0000 7f3dacad1640 1 --2- 172.21.15.46:0/3711789415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da8109b50 0x7f3da81a1810 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3da000eb10 tx=0x7f3da000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:04.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.367+0000 7f3da5ffb640 1 -- 172.21.15.46:0/3711789415 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0004b20 con 0x7f3da8109b50 2024-05-09T04:51:04.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.367+0000 7f3dae534640 1 -- 172.21.15.46:0/3711789415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da81a2170 con 0x7f3da8109b50 2024-05-09T04:51:04.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.367+0000 7f3da5ffb640 1 -- 172.21.15.46:0/3711789415 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3da0004cc0 con 0x7f3da8109b50 2024-05-09T04:51:04.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.367+0000 7f3da5ffb640 1 -- 172.21.15.46:0/3711789415 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3da0010690 con 0x7f3da8109b50 2024-05-09T04:51:04.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.367+0000 7f3dae534640 1 -- 172.21.15.46:0/3711789415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da81a2710 con 0x7f3da8109b50 2024-05-09T04:51:04.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.368+0000 7f3dae534640 1 -- 172.21.15.46:0/3711789415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3da8108960 con 0x7f3da8109b50 2024-05-09T04:51:04.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.370+0000 7f3da5ffb640 1 -- 172.21.15.46:0/3711789415 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f3da0005000 con 0x7f3da8109b50 2024-05-09T04:51:04.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.371+0000 7f3da5ffb640 1 --2- 172.21.15.46:0/3711789415 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3d940b2840 0x7f3d940b4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:04.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.371+0000 7f3da7fff640 1 -- 172.21.15.46:0/3711789415 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3d940b2840 msgr2=0x7f3d940b4d50 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.46:6800/3691531627 2024-05-09T04:51:04.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.371+0000 7f3da5ffb640 1 -- 172.21.15.46:0/3711789415 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8298+0+0 (secure 0 0 0) 0x7f3da00e2a50 con 0x7f3da8109b50 2024-05-09T04:51:04.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.371+0000 7f3da7fff640 1 --2- 172.21.15.46:0/3711789415 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3d940b2840 0x7f3d940b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-09T04:51:04.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.374+0000 7f3da5ffb640 1 -- 172.21.15.46:0/3711789415 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3da0054df0 con 0x7f3da8109b50 2024-05-09T04:51:04.507 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:04 smithi046 ceph-mon[30136]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T04:51:04.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:04 smithi046 ceph-mon[30136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T04:51:04.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:04 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2108113597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:51:04.508 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:04 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1949873949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T04:51:04.508 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:04.155+0000 7f890b550200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-09T04:51:04.508 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:04.507+0000 7f890b550200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-09T04:51:04.510 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:04 smithi110 ceph-mon[34904]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T04:51:04.510 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:04 smithi110 ceph-mon[34904]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T04:51:04.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:04 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2108113597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:51:04.511 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:04 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1949873949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T04:51:04.534 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:04 smithi072 ceph-mon[34088]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-05-09T04:51:04.534 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:04 smithi072 ceph-mon[34088]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2024-05-09T04:51:04.534 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:04 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2108113597' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:51:04.534 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:04 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1949873949' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T04:51:04.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.568+0000 7f3dae534640 1 -- 172.21.15.46:0/3711789415 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3da802dde0 con 0x7f3da8109b50 2024-05-09T04:51:04.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.569+0000 7f3da5ffb640 1 -- 172.21.15.46:0/3711789415 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) ==== 100+0+11 (secure 0 0 0) 0x7f3da00aa6d0 con 0x7f3da8109b50 2024-05-09T04:51:04.568 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-05-09T04:51:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.571+0000 7f3dae534640 1 -- 172.21.15.46:0/3711789415 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3d940b2840 msgr2=0x7f3d940b4d50 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-09T04:51:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.571+0000 7f3dae534640 1 --2- 172.21.15.46:0/3711789415 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3d940b2840 0x7f3d940b4d50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.571+0000 7f3dae534640 1 -- 172.21.15.46:0/3711789415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da8109b50 msgr2=0x7f3da81a1810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.571+0000 7f3dae534640 1 --2- 172.21.15.46:0/3711789415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da8109b50 0x7f3da81a1810 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f3da000eb10 tx=0x7f3da000c2d0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.571+0000 7f3dae534640 1 -- 172.21.15.46:0/3711789415 shutdown_connections 2024-05-09T04:51:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.571+0000 7f3dae534640 1 --2- 172.21.15.46:0/3711789415 >> [v2:172.21.15.46:6800/3691531627,v1:172.21.15.46:6801/3691531627] conn(0x7f3d940b2840 0x7f3d940b4d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.571+0000 7f3dae534640 1 --2- 172.21.15.46:0/3711789415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3da8109b50 0x7f3da81a1810 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.571+0000 7f3dae534640 1 --2- 172.21.15.46:0/3711789415 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3da8104e50 0x7f3da819a880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.571+0000 7f3dae534640 1 --2- 172.21.15.46:0/3711789415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3da8103c10 0x7f3da819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.571+0000 7f3dae534640 1 -- 172.21.15.46:0/3711789415 >> 172.21.15.46:0/3711789415 conn(0x7f3da80fee00 msgr2=0x7f3da8105b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.571+0000 7f3dae534640 1 -- 172.21.15.46:0/3711789415 shutdown_connections 2024-05-09T04:51:04.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:04.571+0000 7f3dae534640 1 -- 172.21.15.46:0/3711789415 wait complete. 2024-05-09T04:51:04.583 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-05-09T04:51:04.583 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:51:04.807 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:04 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:04.510+0000 7f2296def200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T04:51:04.807 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 04:51:04 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z[36783]: 2024-05-09T04:51:04.728+0000 7f2296def200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T04:51:04.830 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:04.612+0000 7f890b550200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-09T04:51:04.843 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:04.880 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:04 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:04.533+0000 7ff3fe2f5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T04:51:04.880 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 04:51:04 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y[35469]: 2024-05-09T04:51:04.751+0000 7ff3fe2f5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T04:51:05.246 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:05 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/778004765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T04:51:05.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:05 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3711789415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T04:51:05.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:05 smithi046 ceph-mon[30136]: Standby manager daemon z restarted 2024-05-09T04:51:05.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:05 smithi046 ceph-mon[30136]: Standby manager daemon z started 2024-05-09T04:51:05.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:05 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.110:0/4078494292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T04:51:05.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:05 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.110:0/4078494292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:51:05.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:05 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.110:0/4078494292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T04:51:05.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:05 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.110:0/4078494292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:51:05.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:05 smithi046 ceph-mon[30136]: Standby manager daemon y restarted 2024-05-09T04:51:05.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:05 smithi046 ceph-mon[30136]: Standby manager daemon y started 2024-05-09T04:51:05.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:05 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.72:0/2949483566' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T04:51:05.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:05 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.72:0/2949483566' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:51:05.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:05 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.72:0/2949483566' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T04:51:05.247 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:05 smithi046 ceph-mon[30136]: from='mgr.? 172.21.15.72:0/2949483566' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:51:05.537 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:05.298+0000 7f890b550200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-09T04:51:05.537 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:05.536+0000 7f890b550200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-09T04:51:05.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:05 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/778004765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T04:51:05.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:05 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3711789415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T04:51:05.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:05 smithi110 ceph-mon[34904]: Standby manager daemon z restarted 2024-05-09T04:51:05.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:05 smithi110 ceph-mon[34904]: Standby manager daemon z started 2024-05-09T04:51:05.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:05 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.110:0/4078494292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T04:51:05.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:05 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.110:0/4078494292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:51:05.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:05 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.110:0/4078494292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T04:51:05.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:05 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.110:0/4078494292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:51:05.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:05 smithi110 ceph-mon[34904]: Standby manager daemon y restarted 2024-05-09T04:51:05.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:05 smithi110 ceph-mon[34904]: Standby manager daemon y started 2024-05-09T04:51:05.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:05 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.72:0/2949483566' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T04:51:05.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:05 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.72:0/2949483566' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:51:05.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:05 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.72:0/2949483566' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T04:51:05.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:05 smithi110 ceph-mon[34904]: from='mgr.? 172.21.15.72:0/2949483566' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:51:05.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:05 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/778004765' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T04:51:05.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:05 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3711789415' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T04:51:05.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:05 smithi072 ceph-mon[34088]: Standby manager daemon z restarted 2024-05-09T04:51:05.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:05 smithi072 ceph-mon[34088]: Standby manager daemon z started 2024-05-09T04:51:05.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:05 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.110:0/4078494292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-09T04:51:05.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:05 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.110:0/4078494292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:51:05.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:05 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.110:0/4078494292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-09T04:51:05.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:05 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.110:0/4078494292' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:51:05.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:05 smithi072 ceph-mon[34088]: Standby manager daemon y restarted 2024-05-09T04:51:05.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:05 smithi072 ceph-mon[34088]: Standby manager daemon y started 2024-05-09T04:51:05.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:05 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.72:0/2949483566' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-09T04:51:05.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:05 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.72:0/2949483566' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-09T04:51:05.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:05 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.72:0/2949483566' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-09T04:51:05.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:05 smithi072 ceph-mon[34088]: from='mgr.? 172.21.15.72:0/2949483566' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-09T04:51:06.080 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:05.804+0000 7f8869b53700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:51:06.080 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:05.804+0000 7f8869b53700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:51:06.080 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:05.804+0000 7f8869b53700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:51:06.080 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:05.804+0000 7f8869b53700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:51:06.080 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:05.804+0000 7f8869b53700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:51:06.081 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:05.804+0000 7f886cbd9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:51:06.081 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:05.804+0000 7f886cbd9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:51:06.081 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:05.804+0000 7f886cbd9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:51:06.081 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:05.804+0000 7f886cbd9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:51:06.081 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:51:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:51:05.804+0000 7f886cbd9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:51:06.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-05-09T04:51:06.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: Active manager daemon x restarted 2024-05-09T04:51:06.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: Activating manager daemon x 2024-05-09T04:51:06.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T04:51:06.491 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: mgrmap e21: x(active, starting, since 0.0133761s), standbys: z, y 2024-05-09T04:51:06.492 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:51:06.492 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:51:06.492 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:51:06.492 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T04:51:06.492 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T04:51:06.492 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T04:51:06.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T04:51:06.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T04:51:06.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T04:51:06.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T04:51:06.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T04:51:06.493 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T04:51:06.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T04:51:06.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T04:51:06.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T04:51:06.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T04:51:06.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T04:51:06.494 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T04:51:06.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:51:06.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:51:06.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:51:06.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:51:06.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:51:06.495 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:51:06.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:51:06.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:51:06.496 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:51:06.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:51:06.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:51:06.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:51:06.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T04:51:06.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T04:51:06.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T04:51:06.497 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: Manager daemon x is now available 2024-05-09T04:51:06.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:06.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T04:51:06.498 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:06 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T04:51:06.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-05-09T04:51:06.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: Active manager daemon x restarted 2024-05-09T04:51:06.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: Activating manager daemon x 2024-05-09T04:51:06.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T04:51:06.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: mgrmap e21: x(active, starting, since 0.0133761s), standbys: z, y 2024-05-09T04:51:06.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:51:06.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:51:06.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:51:06.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T04:51:06.559 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T04:51:06.559 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T04:51:06.559 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T04:51:06.559 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T04:51:06.559 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T04:51:06.559 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T04:51:06.559 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T04:51:06.559 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T04:51:06.559 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T04:51:06.559 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T04:51:06.559 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T04:51:06.559 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T04:51:06.559 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T04:51:06.559 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T04:51:06.560 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:51:06.560 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:51:06.560 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:51:06.560 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:51:06.560 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:51:06.560 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:51:06.560 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:51:06.560 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:51:06.560 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:51:06.560 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:51:06.560 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:51:06.560 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:51:06.560 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T04:51:06.560 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T04:51:06.560 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T04:51:06.561 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: Manager daemon x is now available 2024-05-09T04:51:06.561 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:06.561 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T04:51:06.561 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:06 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T04:51:06.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-05-09T04:51:06.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: Active manager daemon x restarted 2024-05-09T04:51:06.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: Activating manager daemon x 2024-05-09T04:51:06.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: osdmap e85: 12 total, 12 up, 12 in 2024-05-09T04:51:06.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: mgrmap e21: x(active, starting, since 0.0133761s), standbys: z, y 2024-05-09T04:51:06.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-09T04:51:06.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-09T04:51:06.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-09T04:51:06.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-09T04:51:06.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-09T04:51:06.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-09T04:51:06.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-09T04:51:06.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-09T04:51:06.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-09T04:51:06.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T04:51:06.632 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T04:51:06.632 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T04:51:06.632 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T04:51:06.632 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T04:51:06.632 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T04:51:06.632 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-09T04:51:06.632 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-09T04:51:06.632 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-09T04:51:06.632 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-09T04:51:06.632 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-09T04:51:06.632 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-09T04:51:06.632 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-09T04:51:06.632 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-09T04:51:06.633 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-09T04:51:06.633 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-09T04:51:06.633 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-09T04:51:06.633 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-09T04:51:06.633 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-09T04:51:06.633 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-09T04:51:06.633 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-09T04:51:06.633 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-09T04:51:06.633 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-09T04:51:06.633 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-09T04:51:06.633 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: Manager daemon x is now available 2024-05-09T04:51:06.633 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:06.633 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-09T04:51:06.633 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:06 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-09T04:51:06.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.630+0000 7fa1eff94700 1 -- 172.21.15.46:0/446363290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8107df0 msgr2=0x7fa1e8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:06.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.630+0000 7fa1eff94700 1 --2- 172.21.15.46:0/446363290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8107df0 0x7fa1e8108200 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa1d80099c0 tx=0x7fa1d8009cd0 comp rx=0 tx=0).stop 2024-05-09T04:51:06.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.631+0000 7fa1eff94700 1 -- 172.21.15.46:0/446363290 shutdown_connections 2024-05-09T04:51:06.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.631+0000 7fa1eff94700 1 --2- 172.21.15.46:0/446363290 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1e8108740 0x7fa1e81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:06.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.631+0000 7fa1eff94700 1 --2- 172.21.15.46:0/446363290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1e81034c0 0x7fa1e81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:06.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.631+0000 7fa1eff94700 1 --2- 172.21.15.46:0/446363290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8107df0 0x7fa1e8108200 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:06.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.631+0000 7fa1eff94700 1 -- 172.21.15.46:0/446363290 >> 172.21.15.46:0/446363290 conn(0x7fa1e80fedc0 msgr2=0x7fa1e81011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:06.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.632+0000 7fa1eff94700 1 -- 172.21.15.46:0/446363290 shutdown_connections 2024-05-09T04:51:06.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.632+0000 7fa1eff94700 1 -- 172.21.15.46:0/446363290 wait complete. 2024-05-09T04:51:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.632+0000 7fa1eff94700 1 Processor -- start 2024-05-09T04:51:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.632+0000 7fa1eff94700 1 -- start start 2024-05-09T04:51:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1eff94700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1e81034c0 0x7fa1e8199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1eff94700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1e8107df0 0x7fa1e8199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1eff94700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108740 0x7fa1e819a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1eff94700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e81133a0 con 0x7fa1e8108740 2024-05-09T04:51:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1eff94700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e8113220 con 0x7fa1e8107df0 2024-05-09T04:51:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1eff94700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e8113520 con 0x7fa1e81034c0 2024-05-09T04:51:06.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1ed52f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1e8107df0 0x7fa1e8199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1ee531700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108740 0x7fa1e819a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1ee531700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108740 0x7fa1e819a510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36382/0 (socket says 172.21.15.46:36382) 2024-05-09T04:51:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1ed52f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1e8107df0 0x7fa1e8199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:45368/0 (socket says 172.21.15.46:45368) 2024-05-09T04:51:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1ee531700 1 -- 172.21.15.46:0/2572520887 learned_addr learned my addr 172.21.15.46:0/2572520887 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1edd30700 1 --2- 172.21.15.46:0/2572520887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1e81034c0 0x7fa1e8199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1ee531700 1 -- 172.21.15.46:0/2572520887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1e81034c0 msgr2=0x7fa1e8199a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1ee531700 1 --2- 172.21.15.46:0/2572520887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1e81034c0 0x7fa1e8199a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1ee531700 1 -- 172.21.15.46:0/2572520887 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1e8107df0 msgr2=0x7fa1e8199fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1ee531700 1 --2- 172.21.15.46:0/2572520887 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1e8107df0 0x7fa1e8199fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1ee531700 1 -- 172.21.15.46:0/2572520887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e81a9190 con 0x7fa1e8108740 2024-05-09T04:51:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.633+0000 7fa1ed52f700 1 --2- 172.21.15.46:0/2572520887 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1e8107df0 0x7fa1e8199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:06.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.634+0000 7fa1ee531700 1 --2- 172.21.15.46:0/2572520887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108740 0x7fa1e819a510 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000ef30 tx=0x7fa1e000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.634+0000 7fa1deffd700 1 -- 172.21.15.46:0/2572520887 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e00092e0 con 0x7fa1e8108740 2024-05-09T04:51:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.634+0000 7fa1deffd700 1 -- 172.21.15.46:0/2572520887 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1e0004bb0 con 0x7fa1e8108740 2024-05-09T04:51:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.634+0000 7fa1deffd700 1 -- 172.21.15.46:0/2572520887 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0004020 con 0x7fa1e8108740 2024-05-09T04:51:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.634+0000 7fa1eff94700 1 -- 172.21.15.46:0/2572520887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e81a9420 con 0x7fa1e8108740 2024-05-09T04:51:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.634+0000 7fa1eff94700 1 -- 172.21.15.46:0/2572520887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e81a98e0 con 0x7fa1e8108740 2024-05-09T04:51:06.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.636+0000 7fa1eff94700 1 -- 172.21.15.46:0/2572520887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e8104970 con 0x7fa1e8108740 2024-05-09T04:51:06.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.637+0000 7fa1deffd700 1 -- 172.21.15.46:0/2572520887 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fa1e001b030 con 0x7fa1e8108740 2024-05-09T04:51:06.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.638+0000 7fa1deffd700 1 --2- 172.21.15.46:0/2572520887 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa1d40b26f0 0x7fa1d40b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:06.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.638+0000 7fa1deffd700 1 -- 172.21.15.46:0/2572520887 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa1e00e1940 con 0x7fa1e8108740 2024-05-09T04:51:06.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.638+0000 7fa1edd30700 1 --2- 172.21.15.46:0/2572520887 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa1d40b26f0 0x7fa1d40b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:06.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.639+0000 7fa1edd30700 1 --2- 172.21.15.46:0/2572520887 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa1d40b26f0 0x7fa1d40b4c00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4007be0 tx=0x7fa1e40073d0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:06.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.644+0000 7fa1deffd700 1 -- 172.21.15.46:0/2572520887 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa1e0022020 con 0x7fa1e8108740 2024-05-09T04:51:06.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.914+0000 7fa1eff94700 1 -- 172.21.15.46:0/2572520887 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1e802e5f0 con 0x7fa1e8108740 2024-05-09T04:51:06.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.915+0000 7fa1deffd700 1 -- 172.21.15.46:0/2572520887 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7fa1e0010040 con 0x7fa1e8108740 2024-05-09T04:51:06.915 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:51:06.915 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:51:06.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.917+0000 7fa1eff94700 1 -- 172.21.15.46:0/2572520887 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa1d40b26f0 msgr2=0x7fa1d40b4c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:06.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.917+0000 7fa1eff94700 1 --2- 172.21.15.46:0/2572520887 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa1d40b26f0 0x7fa1d40b4c00 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4007be0 tx=0x7fa1e40073d0 comp rx=0 tx=0).stop 2024-05-09T04:51:06.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.917+0000 7fa1eff94700 1 -- 172.21.15.46:0/2572520887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108740 msgr2=0x7fa1e819a510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:06.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.917+0000 7fa1eff94700 1 --2- 172.21.15.46:0/2572520887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108740 0x7fa1e819a510 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa1e000ef30 tx=0x7fa1e000c3a0 comp rx=0 tx=0).stop 2024-05-09T04:51:06.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.917+0000 7fa1eff94700 1 -- 172.21.15.46:0/2572520887 shutdown_connections 2024-05-09T04:51:06.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.917+0000 7fa1eff94700 1 --2- 172.21.15.46:0/2572520887 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa1d40b26f0 0x7fa1d40b4c00 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:06.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.917+0000 7fa1eff94700 1 --2- 172.21.15.46:0/2572520887 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1e8108740 0x7fa1e819a510 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:06.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.917+0000 7fa1eff94700 1 --2- 172.21.15.46:0/2572520887 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1e81034c0 0x7fa1e8199a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:06.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.917+0000 7fa1eff94700 1 --2- 172.21.15.46:0/2572520887 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1e8107df0 0x7fa1e8199fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:06.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.917+0000 7fa1eff94700 1 -- 172.21.15.46:0/2572520887 >> 172.21.15.46:0/2572520887 conn(0x7fa1e80fedc0 msgr2=0x7fa1e81096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:06.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.917+0000 7fa1eff94700 1 -- 172.21.15.46:0/2572520887 shutdown_connections 2024-05-09T04:51:06.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:06.917+0000 7fa1eff94700 1 -- 172.21.15.46:0/2572520887 wait complete. 2024-05-09T04:51:06.919 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:51:07.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:07 smithi046 ceph-mon[30136]: [09/May/2024:04:51:05] ENGINE Bus STARTING 2024-05-09T04:51:07.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:07 smithi046 ceph-mon[30136]: [09/May/2024:04:51:06] ENGINE Serving on https://172.21.15.46:7150 2024-05-09T04:51:07.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:07 smithi046 ceph-mon[30136]: [09/May/2024:04:51:06] ENGINE Serving on http://172.21.15.46:8765 2024-05-09T04:51:07.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:07 smithi046 ceph-mon[30136]: [09/May/2024:04:51:06] ENGINE Bus STARTED 2024-05-09T04:51:07.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:07 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:07.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:07 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:07.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:07 smithi046 ceph-mon[30136]: mgrmap e22: x(active, since 1.02734s), standbys: z, y 2024-05-09T04:51:07.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:07 smithi046 ceph-mon[30136]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:51:07.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:07 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2572520887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:51:07.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:07 smithi110 ceph-mon[34904]: [09/May/2024:04:51:05] ENGINE Bus STARTING 2024-05-09T04:51:07.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:07 smithi110 ceph-mon[34904]: [09/May/2024:04:51:06] ENGINE Serving on https://172.21.15.46:7150 2024-05-09T04:51:07.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:07 smithi110 ceph-mon[34904]: [09/May/2024:04:51:06] ENGINE Serving on http://172.21.15.46:8765 2024-05-09T04:51:07.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:07 smithi110 ceph-mon[34904]: [09/May/2024:04:51:06] ENGINE Bus STARTED 2024-05-09T04:51:07.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:07 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:07.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:07 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:07.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:07 smithi110 ceph-mon[34904]: mgrmap e22: x(active, since 1.02734s), standbys: z, y 2024-05-09T04:51:07.807 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:07 smithi110 ceph-mon[34904]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:51:07.808 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:07 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2572520887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:51:07.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:07 smithi072 ceph-mon[34088]: [09/May/2024:04:51:05] ENGINE Bus STARTING 2024-05-09T04:51:07.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:07 smithi072 ceph-mon[34088]: [09/May/2024:04:51:06] ENGINE Serving on https://172.21.15.46:7150 2024-05-09T04:51:07.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:07 smithi072 ceph-mon[34088]: [09/May/2024:04:51:06] ENGINE Serving on http://172.21.15.46:8765 2024-05-09T04:51:07.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:07 smithi072 ceph-mon[34088]: [09/May/2024:04:51:06] ENGINE Bus STARTED 2024-05-09T04:51:07.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:07 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:07.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:07 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:07.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:07 smithi072 ceph-mon[34088]: mgrmap e22: x(active, since 1.02734s), standbys: z, y 2024-05-09T04:51:07.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:07 smithi072 ceph-mon[34088]: pgmap v3: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:51:07.881 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:07 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2572520887' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:51:08.039 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-05-09T04:51:08.039 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-05-09T04:51:08.039 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:51:08.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.166+0000 7f30772c8640 1 -- 172.21.15.46:0/3270345800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3070107080 msgr2=0x7f30701094a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:08.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.166+0000 7f30772c8640 1 --2- 172.21.15.46:0/3270345800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3070107080 0x7f30701094a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f3060009900 tx=0x7f306002a410 comp rx=0 tx=0).stop 2024-05-09T04:51:08.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.167+0000 7f30772c8640 1 -- 172.21.15.46:0/3270345800 shutdown_connections 2024-05-09T04:51:08.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.167+0000 7f30772c8640 1 --2- 172.21.15.46:0/3270345800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f30701099e0 0x7f307010be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.167+0000 7f30772c8640 1 --2- 172.21.15.46:0/3270345800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3070107080 0x7f30701094a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.167+0000 7f30772c8640 1 --2- 172.21.15.46:0/3270345800 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3070104720 0x7f3070106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.167+0000 7f30772c8640 1 -- 172.21.15.46:0/3270345800 >> 172.21.15.46:0/3270345800 conn(0x7f30700fdd20 msgr2=0x7f30701001d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:08.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.167+0000 7f30772c8640 1 -- 172.21.15.46:0/3270345800 shutdown_connections 2024-05-09T04:51:08.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.167+0000 7f30772c8640 1 -- 172.21.15.46:0/3270345800 wait complete. 2024-05-09T04:51:08.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f30772c8640 1 Processor -- start 2024-05-09T04:51:08.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f30772c8640 1 -- start start 2024-05-09T04:51:08.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f30772c8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3070104720 0x7f307019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:08.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f30772c8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3070107080 0x7f307019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:08.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f30772c8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30701099e0 0x7f30701a1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:08.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f30772c8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f307010f760 con 0x7f3070107080 2024-05-09T04:51:08.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f30772c8640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f307010f5e0 con 0x7f30701099e0 2024-05-09T04:51:08.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f30772c8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f307010f8e0 con 0x7f3070104720 2024-05-09T04:51:08.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f3074863640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3070107080 0x7f307019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:08.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f3075064640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3070104720 0x7f307019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:08.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f3074863640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3070107080 0x7f307019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36408/0 (socket says 172.21.15.46:36408) 2024-05-09T04:51:08.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f3074863640 1 -- 172.21.15.46:0/3455164697 learned_addr learned my addr 172.21.15.46:0/3455164697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:08.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f3075865640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30701099e0 0x7f30701a1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:08.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f3074863640 1 -- 172.21.15.46:0/3455164697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3070104720 msgr2=0x7f307019a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:08.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f3074863640 1 --2- 172.21.15.46:0/3455164697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3070104720 0x7f307019a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f3074863640 1 -- 172.21.15.46:0/3455164697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30701099e0 msgr2=0x7f30701a1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:08.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f3074863640 1 --2- 172.21.15.46:0/3455164697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30701099e0 0x7f30701a1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f3074863640 1 -- 172.21.15.46:0/3455164697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30701a2170 con 0x7f3070107080 2024-05-09T04:51:08.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.168+0000 7f3075865640 1 --2- 172.21.15.46:0/3455164697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30701099e0 0x7f30701a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:08.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.169+0000 7f3074863640 1 --2- 172.21.15.46:0/3455164697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3070107080 0x7f307019ab70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3060006010 tx=0x7f30600042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:08.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.169+0000 7f30667fc640 1 -- 172.21.15.46:0/3455164697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f306003f030 con 0x7f3070107080 2024-05-09T04:51:08.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.169+0000 7f30667fc640 1 -- 172.21.15.46:0/3455164697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3060031910 con 0x7f3070107080 2024-05-09T04:51:08.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.169+0000 7f30772c8640 1 -- 172.21.15.46:0/3455164697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30701a2400 con 0x7f3070107080 2024-05-09T04:51:08.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.169+0000 7f30667fc640 1 -- 172.21.15.46:0/3455164697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3060035070 con 0x7f3070107080 2024-05-09T04:51:08.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.169+0000 7f30772c8640 1 -- 172.21.15.46:0/3455164697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30701a28f0 con 0x7f3070107080 2024-05-09T04:51:08.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.170+0000 7f30772c8640 1 -- 172.21.15.46:0/3455164697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3070069ca0 con 0x7f3070107080 2024-05-09T04:51:08.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.173+0000 7f30667fc640 1 -- 172.21.15.46:0/3455164697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f3060042030 con 0x7f3070107080 2024-05-09T04:51:08.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.173+0000 7f30667fc640 1 --2- 172.21.15.46:0/3455164697 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f305c0b25e0 0x7f305c0b4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:08.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.174+0000 7f30667fc640 1 -- 172.21.15.46:0/3455164697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f3060102a80 con 0x7f3070107080 2024-05-09T04:51:08.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.174+0000 7f3075064640 1 --2- 172.21.15.46:0/3455164697 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f305c0b25e0 0x7f305c0b4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:08.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.174+0000 7f3075064640 1 --2- 172.21.15.46:0/3455164697 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f305c0b25e0 0x7f305c0b4af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f306c0095f0 tx=0x7f306c005f90 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:08.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.178+0000 7f30667fc640 1 -- 172.21.15.46:0/3455164697 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f30600ca300 con 0x7f3070107080 2024-05-09T04:51:08.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.431+0000 7f30772c8640 1 -- 172.21.15.46:0/3455164697 --> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f30701066d0 con 0x7f305c0b25e0 2024-05-09T04:51:08.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.440+0000 7f30667fc640 1 -- 172.21.15.46:0/3455164697 <== mgr.15150 v2:172.21.15.46:6800/2595796783 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258023 (secure 0 0 0) 0x7f30701066d0 con 0x7f305c0b25e0 2024-05-09T04:51:08.443 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:51:08.445 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T04:51:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.444+0000 7f30772c8640 1 -- 172.21.15.46:0/3455164697 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f305c0b25e0 msgr2=0x7f305c0b4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.444+0000 7f30772c8640 1 --2- 172.21.15.46:0/3455164697 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f305c0b25e0 0x7f305c0b4af0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f306c0095f0 tx=0x7f306c005f90 comp rx=0 tx=0).stop 2024-05-09T04:51:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.444+0000 7f30772c8640 1 -- 172.21.15.46:0/3455164697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3070107080 msgr2=0x7f307019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.444+0000 7f30772c8640 1 --2- 172.21.15.46:0/3455164697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3070107080 0x7f307019ab70 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f3060006010 tx=0x7f30600042f0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.444+0000 7f30772c8640 1 -- 172.21.15.46:0/3455164697 shutdown_connections 2024-05-09T04:51:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.444+0000 7f30772c8640 1 --2- 172.21.15.46:0/3455164697 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f305c0b25e0 0x7f305c0b4af0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.445+0000 7f30772c8640 1 --2- 172.21.15.46:0/3455164697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f30701099e0 0x7f30701a1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.445+0000 7f30772c8640 1 --2- 172.21.15.46:0/3455164697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3070107080 0x7f307019ab70 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.445+0000 7f30772c8640 1 --2- 172.21.15.46:0/3455164697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3070104720 0x7f307019a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.445+0000 7f30772c8640 1 -- 172.21.15.46:0/3455164697 >> 172.21.15.46:0/3455164697 conn(0x7f30700fdd20 msgr2=0x7f3070105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:08.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.445+0000 7f30772c8640 1 -- 172.21.15.46:0/3455164697 shutdown_connections 2024-05-09T04:51:08.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.445+0000 7f30772c8640 1 -- 172.21.15.46:0/3455164697 wait complete. 2024-05-09T04:51:08.462 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-05-09T04:51:07.560145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188100,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1268,"num_read_kb":757,"num_write":418,"num_write_kb":3591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":3002535,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":823,"ondisk_log_size":823,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368860,"kb_used_data":11356,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572964,"statfs":{"total":1151940427776,"available":1151562715136,"internally_reserved":0,"allocated":11628544,"data_stored":8645020,"data_compressed":344655,"data_compressed_allocated":462848,"data_compressed_original":3457024,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.989646"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.892107+0000","last_change":"2024-05-09T04:47:48.408878+0000","last_active":"2024-05-09T04:51:05.892107+0000","last_peered":"2024-05-09T04:51:05.892107+0000","last_clean":"2024-05-09T04:51:05.892107+0000","last_became_active":"2024-05-09T04:47:48.408479+0000","last_became_peered":"2024-05-09T04:47:48.408479+0000","last_unstale":"2024-05-09T04:51:05.892107+0000","last_undegraded":"2024-05-09T04:51:05.892107+0000","last_fullsized":"2024-05-09T04:51:05.892107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:02.494768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814499+0000","last_change":"2024-05-09T04:47:48.415650+0000","last_active":"2024-05-09T04:51:05.814499+0000","last_peered":"2024-05-09T04:51:05.814499+0000","last_clean":"2024-05-09T04:51:05.814499+0000","last_became_active":"2024-05-09T04:47:48.415322+0000","last_became_peered":"2024-05-09T04:47:48.415322+0000","last_unstale":"2024-05-09T04:51:05.814499+0000","last_undegraded":"2024-05-09T04:51:05.814499+0000","last_fullsized":"2024-05-09T04:51:05.814499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.142513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814332+0000","last_change":"2024-05-09T04:47:48.417706+0000","last_active":"2024-05-09T04:51:05.814332+0000","last_peered":"2024-05-09T04:51:05.814332+0000","last_clean":"2024-05-09T04:51:05.814332+0000","last_became_active":"2024-05-09T04:47:48.412068+0000","last_became_peered":"2024-05-09T04:47:48.412068+0000","last_unstale":"2024-05-09T04:51:05.814332+0000","last_undegraded":"2024-05-09T04:51:05.814332+0000","last_fullsized":"2024-05-09T04:51:05.814332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:44.602567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.805515+0000","last_change":"2024-05-09T04:47:48.415751+0000","last_active":"2024-05-09T04:51:05.805515+0000","last_peered":"2024-05-09T04:51:05.805515+0000","last_clean":"2024-05-09T04:51:05.805515+0000","last_became_active":"2024-05-09T04:47:48.414312+0000","last_became_peered":"2024-05-09T04:47:48.414312+0000","last_unstale":"2024-05-09T04:51:05.805515+0000","last_undegraded":"2024-05-09T04:51:05.805515+0000","last_fullsized":"2024-05-09T04:51:05.805515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:12.202761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651437+0000","last_change":"2024-05-09T04:47:48.415919+0000","last_active":"2024-05-09T04:51:05.651437+0000","last_peered":"2024-05-09T04:51:05.651437+0000","last_clean":"2024-05-09T04:51:05.651437+0000","last_became_active":"2024-05-09T04:47:48.415533+0000","last_became_peered":"2024-05-09T04:47:48.415533+0000","last_unstale":"2024-05-09T04:51:05.651437+0000","last_undegraded":"2024-05-09T04:51:05.651437+0000","last_fullsized":"2024-05-09T04:51:05.651437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:32.262416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.662001+0000","last_change":"2024-05-09T04:47:48.419705+0000","last_active":"2024-05-09T04:51:05.662001+0000","last_peered":"2024-05-09T04:51:05.662001+0000","last_clean":"2024-05-09T04:51:05.662001+0000","last_became_active":"2024-05-09T04:47:48.419339+0000","last_became_peered":"2024-05-09T04:47:48.419339+0000","last_unstale":"2024-05-09T04:51:05.662001+0000","last_undegraded":"2024-05-09T04:51:05.662001+0000","last_fullsized":"2024-05-09T04:51:05.662001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:50:23.754071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810211+0000","last_change":"2024-05-09T04:47:48.404106+0000","last_active":"2024-05-09T04:51:05.810211+0000","last_peered":"2024-05-09T04:51:05.810211+0000","last_clean":"2024-05-09T04:51:05.810211+0000","last_became_active":"2024-05-09T04:47:48.403724+0000","last_became_peered":"2024-05-09T04:47:48.403724+0000","last_unstale":"2024-05-09T04:51:05.810211+0000","last_undegraded":"2024-05-09T04:51:05.810211+0000","last_fullsized":"2024-05-09T04:51:05.810211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:41.134752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.648971+0000","last_change":"2024-05-09T04:48:28.562515+0000","last_active":"2024-05-09T04:51:05.648971+0000","last_peered":"2024-05-09T04:51:05.648971+0000","last_clean":"2024-05-09T04:51:05.648971+0000","last_became_active":"2024-05-09T04:48:28.562104+0000","last_became_peered":"2024-05-09T04:48:28.562104+0000","last_unstale":"2024-05-09T04:51:05.648971+0000","last_undegraded":"2024-05-09T04:51:05.648971+0000","last_fullsized":"2024-05-09T04:51:05.648971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:33.342877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548850+0000","last_change":"2024-05-09T04:47:48.400467+0000","last_active":"2024-05-09T04:51:05.548850+0000","last_peered":"2024-05-09T04:51:05.548850+0000","last_clean":"2024-05-09T04:51:05.548850+0000","last_became_active":"2024-05-09T04:47:48.399960+0000","last_became_peered":"2024-05-09T04:47:48.399960+0000","last_unstale":"2024-05-09T04:51:05.548850+0000","last_undegraded":"2024-05-09T04:51:05.548850+0000","last_fullsized":"2024-05-09T04:51:05.548850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:23:44.473053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.662377+0000","last_change":"2024-05-09T04:47:48.418970+0000","last_active":"2024-05-09T04:51:05.662377+0000","last_peered":"2024-05-09T04:51:05.662377+0000","last_clean":"2024-05-09T04:51:05.662377+0000","last_became_active":"2024-05-09T04:47:48.418574+0000","last_became_peered":"2024-05-09T04:47:48.418574+0000","last_unstale":"2024-05-09T04:51:05.662377+0000","last_undegraded":"2024-05-09T04:51:05.662377+0000","last_fullsized":"2024-05-09T04:51:05.662377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:48.075893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810307+0000","last_change":"2024-05-09T04:47:48.409879+0000","last_active":"2024-05-09T04:51:05.810307+0000","last_peered":"2024-05-09T04:51:05.810307+0000","last_clean":"2024-05-09T04:51:05.810307+0000","last_became_active":"2024-05-09T04:47:48.409515+0000","last_became_peered":"2024-05-09T04:47:48.409515+0000","last_unstale":"2024-05-09T04:51:05.810307+0000","last_undegraded":"2024-05-09T04:51:05.810307+0000","last_fullsized":"2024-05-09T04:51:05.810307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:15.546214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.805520+0000","last_change":"2024-05-09T04:47:48.417556+0000","last_active":"2024-05-09T04:51:05.805520+0000","last_peered":"2024-05-09T04:51:05.805520+0000","last_clean":"2024-05-09T04:51:05.805520+0000","last_became_active":"2024-05-09T04:47:48.416787+0000","last_became_peered":"2024-05-09T04:47:48.416787+0000","last_unstale":"2024-05-09T04:51:05.805520+0000","last_undegraded":"2024-05-09T04:51:05.805520+0000","last_fullsized":"2024-05-09T04:51:05.805520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:20.360232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544479+0000","last_change":"2024-05-09T04:47:48.415145+0000","last_active":"2024-05-09T04:51:05.544479+0000","last_peered":"2024-05-09T04:51:05.544479+0000","last_clean":"2024-05-09T04:51:05.544479+0000","last_became_active":"2024-05-09T04:47:48.414830+0000","last_became_peered":"2024-05-09T04:47:48.414830+0000","last_unstale":"2024-05-09T04:51:05.544479+0000","last_undegraded":"2024-05-09T04:51:05.544479+0000","last_fullsized":"2024-05-09T04:51:05.544479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:55.936409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814035+0000","last_change":"2024-05-09T04:47:48.413977+0000","last_active":"2024-05-09T04:51:05.814035+0000","last_peered":"2024-05-09T04:51:05.814035+0000","last_clean":"2024-05-09T04:51:05.814035+0000","last_became_active":"2024-05-09T04:47:48.413604+0000","last_became_peered":"2024-05-09T04:47:48.413604+0000","last_unstale":"2024-05-09T04:51:05.814035+0000","last_undegraded":"2024-05-09T04:51:05.814035+0000","last_fullsized":"2024-05-09T04:51:05.814035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:25.462958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.813924+0000","last_change":"2024-05-09T04:47:48.419000+0000","last_active":"2024-05-09T04:51:05.813924+0000","last_peered":"2024-05-09T04:51:05.813924+0000","last_clean":"2024-05-09T04:51:05.813924+0000","last_became_active":"2024-05-09T04:47:48.418457+0000","last_became_peered":"2024-05-09T04:47:48.418457+0000","last_unstale":"2024-05-09T04:51:05.813924+0000","last_undegraded":"2024-05-09T04:51:05.813924+0000","last_fullsized":"2024-05-09T04:51:05.813924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:36.325460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548387+0000","last_change":"2024-05-09T04:47:48.406754+0000","last_active":"2024-05-09T04:51:05.548387+0000","last_peered":"2024-05-09T04:51:05.548387+0000","last_clean":"2024-05-09T04:51:05.548387+0000","last_became_active":"2024-05-09T04:47:48.406396+0000","last_became_peered":"2024-05-09T04:47:48.406396+0000","last_unstale":"2024-05-09T04:51:05.548387+0000","last_undegraded":"2024-05-09T04:51:05.548387+0000","last_fullsized":"2024-05-09T04:51:05.548387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:46.421061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.892267+0000","last_change":"2024-05-09T04:47:48.409556+0000","last_active":"2024-05-09T04:51:05.892267+0000","last_peered":"2024-05-09T04:51:05.892267+0000","last_clean":"2024-05-09T04:51:05.892267+0000","last_became_active":"2024-05-09T04:47:48.409145+0000","last_became_peered":"2024-05-09T04:47:48.409145+0000","last_unstale":"2024-05-09T04:51:05.892267+0000","last_undegraded":"2024-05-09T04:51:05.892267+0000","last_fullsized":"2024-05-09T04:51:05.892267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:25.486257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.649670+0000","last_change":"2024-05-09T04:47:48.415546+0000","last_active":"2024-05-09T04:51:05.649670+0000","last_peered":"2024-05-09T04:51:05.649670+0000","last_clean":"2024-05-09T04:51:05.649670+0000","last_became_active":"2024-05-09T04:47:48.415221+0000","last_became_peered":"2024-05-09T04:47:48.415221+0000","last_unstale":"2024-05-09T04:51:05.649670+0000","last_undegraded":"2024-05-09T04:51:05.649670+0000","last_fullsized":"2024-05-09T04:51:05.649670+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:57:29.112409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545923+0000","last_change":"2024-05-09T04:47:48.413106+0000","last_active":"2024-05-09T04:51:05.545923+0000","last_peered":"2024-05-09T04:51:05.545923+0000","last_clean":"2024-05-09T04:51:05.545923+0000","last_became_active":"2024-05-09T04:47:48.412700+0000","last_became_peered":"2024-05-09T04:47:48.412700+0000","last_unstale":"2024-05-09T04:51:05.545923+0000","last_undegraded":"2024-05-09T04:51:05.545923+0000","last_fullsized":"2024-05-09T04:51:05.545923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:37:31.682008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.649600+0000","last_change":"2024-05-09T04:47:45.386931+0000","last_active":"2024-05-09T04:51:05.649600+0000","last_peered":"2024-05-09T04:51:05.649600+0000","last_clean":"2024-05-09T04:51:05.649600+0000","last_became_active":"2024-05-09T04:47:45.386562+0000","last_became_peered":"2024-05-09T04:47:45.386562+0000","last_unstale":"2024-05-09T04:51:05.649600+0000","last_undegraded":"2024-05-09T04:51:05.649600+0000","last_fullsized":"2024-05-09T04:51:05.649600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:59.335687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.652165+0000","last_change":"2024-05-09T04:47:48.411064+0000","last_active":"2024-05-09T04:51:05.652165+0000","last_peered":"2024-05-09T04:51:05.652165+0000","last_clean":"2024-05-09T04:51:05.652165+0000","last_became_active":"2024-05-09T04:47:48.410689+0000","last_became_peered":"2024-05-09T04:47:48.410689+0000","last_unstale":"2024-05-09T04:51:05.652165+0000","last_undegraded":"2024-05-09T04:51:05.652165+0000","last_fullsized":"2024-05-09T04:51:05.652165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:21.178831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.652322+0000","last_change":"2024-05-09T04:47:45.612837+0000","last_active":"2024-05-09T04:51:05.652322+0000","last_peered":"2024-05-09T04:51:05.652322+0000","last_clean":"2024-05-09T04:51:05.652322+0000","last_became_active":"2024-05-09T04:47:45.612307+0000","last_became_peered":"2024-05-09T04:47:45.612307+0000","last_unstale":"2024-05-09T04:51:05.652322+0000","last_undegraded":"2024-05-09T04:51:05.652322+0000","last_fullsized":"2024-05-09T04:51:05.652322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:59.686391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814704+0000","last_change":"2024-05-09T04:47:48.412781+0000","last_active":"2024-05-09T04:51:05.814704+0000","last_peered":"2024-05-09T04:51:05.814704+0000","last_clean":"2024-05-09T04:51:05.814704+0000","last_became_active":"2024-05-09T04:47:48.411396+0000","last_became_peered":"2024-05-09T04:47:48.411396+0000","last_unstale":"2024-05-09T04:51:05.814704+0000","last_undegraded":"2024-05-09T04:51:05.814704+0000","last_fullsized":"2024-05-09T04:51:05.814704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:41:22.775432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.892130+0000","last_change":"2024-05-09T04:48:28.568024+0000","last_active":"2024-05-09T04:51:05.892130+0000","last_peered":"2024-05-09T04:51:05.892130+0000","last_clean":"2024-05-09T04:51:05.892130+0000","last_became_active":"2024-05-09T04:48:28.559441+0000","last_became_peered":"2024-05-09T04:48:28.559441+0000","last_unstale":"2024-05-09T04:51:05.892130+0000","last_undegraded":"2024-05-09T04:51:05.892130+0000","last_fullsized":"2024-05-09T04:51:05.892130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:56.257589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548481+0000","last_change":"2024-05-09T04:47:48.421361+0000","last_active":"2024-05-09T04:51:05.548481+0000","last_peered":"2024-05-09T04:51:05.548481+0000","last_clean":"2024-05-09T04:51:05.548481+0000","last_became_active":"2024-05-09T04:47:48.421018+0000","last_became_peered":"2024-05-09T04:47:48.421018+0000","last_unstale":"2024-05-09T04:51:05.548481+0000","last_undegraded":"2024-05-09T04:51:05.548481+0000","last_fullsized":"2024-05-09T04:51:05.548481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:00.634983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.891836+0000","last_change":"2024-05-09T04:49:29.402935+0000","last_active":"2024-05-09T04:51:05.891836+0000","last_peered":"2024-05-09T04:51:05.891836+0000","last_clean":"2024-05-09T04:51:05.891836+0000","last_became_active":"2024-05-09T04:49:29.400084+0000","last_became_peered":"2024-05-09T04:49:29.400084+0000","last_unstale":"2024-05-09T04:51:05.891836+0000","last_undegraded":"2024-05-09T04:51:05.891836+0000","last_fullsized":"2024-05-09T04:51:05.891836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:06:06.683122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.659117+0000","last_change":"2024-05-09T04:48:28.564243+0000","last_active":"2024-05-09T04:51:05.659117+0000","last_peered":"2024-05-09T04:51:05.659117+0000","last_clean":"2024-05-09T04:51:05.659117+0000","last_became_active":"2024-05-09T04:48:28.563732+0000","last_became_peered":"2024-05-09T04:48:28.563732+0000","last_unstale":"2024-05-09T04:51:05.659117+0000","last_undegraded":"2024-05-09T04:51:05.659117+0000","last_fullsized":"2024-05-09T04:51:05.659117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:26.982339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'8","reported_seq":221,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810097+0000","last_change":"2024-05-09T04:47:45.389868+0000","last_active":"2024-05-09T04:51:05.810097+0000","last_peered":"2024-05-09T04:51:05.810097+0000","last_clean":"2024-05-09T04:51:05.810097+0000","last_became_active":"2024-05-09T04:47:45.389508+0000","last_became_peered":"2024-05-09T04:47:45.389508+0000","last_unstale":"2024-05-09T04:51:05.810097+0000","last_undegraded":"2024-05-09T04:51:05.810097+0000","last_fullsized":"2024-05-09T04:51:05.810097+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:45.050187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1712,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":181,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651364+0000","last_change":"2024-05-09T04:47:48.404457+0000","last_active":"2024-05-09T04:51:05.651364+0000","last_peered":"2024-05-09T04:51:05.651364+0000","last_clean":"2024-05-09T04:51:05.651364+0000","last_became_active":"2024-05-09T04:47:48.403968+0000","last_became_peered":"2024-05-09T04:47:48.403968+0000","last_unstale":"2024-05-09T04:51:05.651364+0000","last_undegraded":"2024-05-09T04:51:05.651364+0000","last_fullsized":"2024-05-09T04:51:05.651364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:23:40.364916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.661879+0000","last_change":"2024-05-09T04:47:45.388183+0000","last_active":"2024-05-09T04:51:05.661879+0000","last_peered":"2024-05-09T04:51:05.661879+0000","last_clean":"2024-05-09T04:51:05.661879+0000","last_became_active":"2024-05-09T04:47:45.386603+0000","last_became_peered":"2024-05-09T04:47:45.386603+0000","last_unstale":"2024-05-09T04:51:05.661879+0000","last_undegraded":"2024-05-09T04:51:05.661879+0000","last_fullsized":"2024-05-09T04:51:05.661879+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:53.433619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.649200+0000","last_change":"2024-05-09T04:48:28.663120+0000","last_active":"2024-05-09T04:51:05.649200+0000","last_peered":"2024-05-09T04:51:05.649200+0000","last_clean":"2024-05-09T04:51:05.649200+0000","last_became_active":"2024-05-09T04:48:28.662748+0000","last_became_peered":"2024-05-09T04:48:28.662748+0000","last_unstale":"2024-05-09T04:51:05.649200+0000","last_undegraded":"2024-05-09T04:51:05.649200+0000","last_fullsized":"2024-05-09T04:51:05.649200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:57.677876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544377+0000","last_change":"2024-05-09T04:47:45.384192+0000","last_active":"2024-05-09T04:51:05.544377+0000","last_peered":"2024-05-09T04:51:05.544377+0000","last_clean":"2024-05-09T04:51:05.544377+0000","last_became_active":"2024-05-09T04:47:45.383843+0000","last_became_peered":"2024-05-09T04:47:45.383843+0000","last_unstale":"2024-05-09T04:51:05.544377+0000","last_undegraded":"2024-05-09T04:51:05.544377+0000","last_fullsized":"2024-05-09T04:51:05.544377+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:28.815207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651282+0000","last_change":"2024-05-09T04:47:48.409203+0000","last_active":"2024-05-09T04:51:05.651282+0000","last_peered":"2024-05-09T04:51:05.651282+0000","last_clean":"2024-05-09T04:51:05.651282+0000","last_became_active":"2024-05-09T04:47:48.408861+0000","last_became_peered":"2024-05-09T04:47:48.408861+0000","last_unstale":"2024-05-09T04:51:05.651282+0000","last_undegraded":"2024-05-09T04:51:05.651282+0000","last_fullsized":"2024-05-09T04:51:05.651282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:07.160397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.658840+0000","last_change":"2024-05-09T04:47:45.392723+0000","last_active":"2024-05-09T04:51:05.658840+0000","last_peered":"2024-05-09T04:51:05.658840+0000","last_clean":"2024-05-09T04:51:05.658840+0000","last_became_active":"2024-05-09T04:47:45.392313+0000","last_became_peered":"2024-05-09T04:47:45.392313+0000","last_unstale":"2024-05-09T04:51:05.658840+0000","last_undegraded":"2024-05-09T04:51:05.658840+0000","last_fullsized":"2024-05-09T04:51:05.658840+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:44.358702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.648930+0000","last_change":"2024-05-09T04:47:48.414000+0000","last_active":"2024-05-09T04:51:05.648930+0000","last_peered":"2024-05-09T04:51:05.648930+0000","last_clean":"2024-05-09T04:51:05.648930+0000","last_became_active":"2024-05-09T04:47:48.413532+0000","last_became_peered":"2024-05-09T04:47:48.413532+0000","last_unstale":"2024-05-09T04:51:05.648930+0000","last_undegraded":"2024-05-09T04:51:05.648930+0000","last_fullsized":"2024-05-09T04:51:05.648930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:22.774814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":214,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.662399+0000","last_change":"2024-05-09T04:47:45.388661+0000","last_active":"2024-05-09T04:51:05.662399+0000","last_peered":"2024-05-09T04:51:05.662399+0000","last_clean":"2024-05-09T04:51:05.662399+0000","last_became_active":"2024-05-09T04:47:45.386010+0000","last_became_peered":"2024-05-09T04:47:45.386010+0000","last_unstale":"2024-05-09T04:51:05.662399+0000","last_undegraded":"2024-05-09T04:51:05.662399+0000","last_fullsized":"2024-05-09T04:51:05.662399+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:23.250463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":178,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548947+0000","last_change":"2024-05-09T04:47:48.421706+0000","last_active":"2024-05-09T04:51:05.548947+0000","last_peered":"2024-05-09T04:51:05.548947+0000","last_clean":"2024-05-09T04:51:05.548947+0000","last_became_active":"2024-05-09T04:47:48.420529+0000","last_became_peered":"2024-05-09T04:47:48.420529+0000","last_unstale":"2024-05-09T04:51:05.548947+0000","last_undegraded":"2024-05-09T04:51:05.548947+0000","last_fullsized":"2024-05-09T04:51:05.548947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:10.110366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810291+0000","last_change":"2024-05-09T04:47:45.386625+0000","last_active":"2024-05-09T04:51:05.810291+0000","last_peered":"2024-05-09T04:51:05.810291+0000","last_clean":"2024-05-09T04:51:05.810291+0000","last_became_active":"2024-05-09T04:47:45.385087+0000","last_became_peered":"2024-05-09T04:47:45.385087+0000","last_unstale":"2024-05-09T04:51:05.810291+0000","last_undegraded":"2024-05-09T04:51:05.810291+0000","last_fullsized":"2024-05-09T04:51:05.810291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:15.546894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814698+0000","last_change":"2024-05-09T04:48:28.562517+0000","last_active":"2024-05-09T04:51:05.814698+0000","last_peered":"2024-05-09T04:51:05.814698+0000","last_clean":"2024-05-09T04:51:05.814698+0000","last_became_active":"2024-05-09T04:48:28.562047+0000","last_became_peered":"2024-05-09T04:48:28.562047+0000","last_unstale":"2024-05-09T04:51:05.814698+0000","last_undegraded":"2024-05-09T04:51:05.814698+0000","last_fullsized":"2024-05-09T04:51:05.814698+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:22.769844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.659142+0000","last_change":"2024-05-09T04:47:45.613713+0000","last_active":"2024-05-09T04:51:05.659142+0000","last_peered":"2024-05-09T04:51:05.659142+0000","last_clean":"2024-05-09T04:51:05.659142+0000","last_became_active":"2024-05-09T04:47:45.613313+0000","last_became_peered":"2024-05-09T04:47:45.613313+0000","last_unstale":"2024-05-09T04:51:05.659142+0000","last_undegraded":"2024-05-09T04:51:05.659142+0000","last_fullsized":"2024-05-09T04:51:05.659142+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:32:44.825695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548722+0000","last_change":"2024-05-09T04:47:48.411234+0000","last_active":"2024-05-09T04:51:05.548722+0000","last_peered":"2024-05-09T04:51:05.548722+0000","last_clean":"2024-05-09T04:51:05.548722+0000","last_became_active":"2024-05-09T04:47:48.410881+0000","last_became_peered":"2024-05-09T04:47:48.410881+0000","last_unstale":"2024-05-09T04:51:05.548722+0000","last_undegraded":"2024-05-09T04:51:05.548722+0000","last_fullsized":"2024-05-09T04:51:05.548722+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.747871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'36","reported_seq":453,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.816697+0000","last_change":"2024-05-09T04:48:28.570426+0000","last_active":"2024-05-09T04:51:05.816697+0000","last_peered":"2024-05-09T04:51:05.816697+0000","last_clean":"2024-05-09T04:51:05.816697+0000","last_became_active":"2024-05-09T04:48:28.561786+0000","last_became_peered":"2024-05-09T04:48:28.561786+0000","last_unstale":"2024-05-09T04:51:05.816697+0000","last_undegraded":"2024-05-09T04:51:05.816697+0000","last_fullsized":"2024-05-09T04:51:05.816697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:32.367142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67059,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":362,"num_read_kb":232,"num_write":37,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":50695,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651672+0000","last_change":"2024-05-09T04:47:48.406510+0000","last_active":"2024-05-09T04:51:05.651672+0000","last_peered":"2024-05-09T04:51:05.651672+0000","last_clean":"2024-05-09T04:51:05.651672+0000","last_became_active":"2024-05-09T04:47:48.406166+0000","last_became_peered":"2024-05-09T04:47:48.406166+0000","last_unstale":"2024-05-09T04:51:05.651672+0000","last_undegraded":"2024-05-09T04:51:05.651672+0000","last_fullsized":"2024-05-09T04:51:05.651672+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:18.579620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545845+0000","last_change":"2024-05-09T04:47:45.387164+0000","last_active":"2024-05-09T04:51:05.545845+0000","last_peered":"2024-05-09T04:51:05.545845+0000","last_clean":"2024-05-09T04:51:05.545845+0000","last_became_active":"2024-05-09T04:47:45.386755+0000","last_became_peered":"2024-05-09T04:47:45.386755+0000","last_unstale":"2024-05-09T04:51:05.545845+0000","last_undegraded":"2024-05-09T04:51:05.545845+0000","last_fullsized":"2024-05-09T04:51:05.545845+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:07.838782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544274+0000","last_change":"2024-05-09T04:47:48.405436+0000","last_active":"2024-05-09T04:51:05.544274+0000","last_peered":"2024-05-09T04:51:05.544274+0000","last_clean":"2024-05-09T04:51:05.544274+0000","last_became_active":"2024-05-09T04:47:48.405032+0000","last_became_peered":"2024-05-09T04:47:48.405032+0000","last_unstale":"2024-05-09T04:51:05.544274+0000","last_undegraded":"2024-05-09T04:51:05.544274+0000","last_fullsized":"2024-05-09T04:51:05.544274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:27.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.658921+0000","last_change":"2024-05-09T04:47:45.391970+0000","last_active":"2024-05-09T04:51:05.658921+0000","last_peered":"2024-05-09T04:51:05.658921+0000","last_clean":"2024-05-09T04:51:05.658921+0000","last_became_active":"2024-05-09T04:47:45.391597+0000","last_became_peered":"2024-05-09T04:47:45.391597+0000","last_unstale":"2024-05-09T04:51:05.658921+0000","last_undegraded":"2024-05-09T04:51:05.658921+0000","last_fullsized":"2024-05-09T04:51:05.658921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:27.076631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810305+0000","last_change":"2024-05-09T04:47:48.419328+0000","last_active":"2024-05-09T04:51:05.810305+0000","last_peered":"2024-05-09T04:51:05.810305+0000","last_clean":"2024-05-09T04:51:05.810305+0000","last_became_active":"2024-05-09T04:47:48.418972+0000","last_became_peered":"2024-05-09T04:47:48.418972+0000","last_unstale":"2024-05-09T04:51:05.810305+0000","last_undegraded":"2024-05-09T04:51:05.810305+0000","last_fullsized":"2024-05-09T04:51:05.810305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:15.164747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":221,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.805695+0000","last_change":"2024-05-09T04:47:45.386173+0000","last_active":"2024-05-09T04:51:05.805695+0000","last_peered":"2024-05-09T04:51:05.805695+0000","last_clean":"2024-05-09T04:51:05.805695+0000","last_became_active":"2024-05-09T04:47:45.384296+0000","last_became_peered":"2024-05-09T04:47:45.384296+0000","last_unstale":"2024-05-09T04:51:05.805695+0000","last_undegraded":"2024-05-09T04:51:05.805695+0000","last_fullsized":"2024-05-09T04:51:05.805695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:03.246187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":183,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810042+0000","last_change":"2024-05-09T04:47:48.415645+0000","last_active":"2024-05-09T04:51:05.810042+0000","last_peered":"2024-05-09T04:51:05.810042+0000","last_clean":"2024-05-09T04:51:05.810042+0000","last_became_active":"2024-05-09T04:47:48.415303+0000","last_became_peered":"2024-05-09T04:47:48.415303+0000","last_unstale":"2024-05-09T04:51:05.810042+0000","last_undegraded":"2024-05-09T04:51:05.810042+0000","last_fullsized":"2024-05-09T04:51:05.810042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:39.703948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651268+0000","last_change":"2024-05-09T04:47:45.613320+0000","last_active":"2024-05-09T04:51:05.651268+0000","last_peered":"2024-05-09T04:51:05.651268+0000","last_clean":"2024-05-09T04:51:05.651268+0000","last_became_active":"2024-05-09T04:47:45.612871+0000","last_became_peered":"2024-05-09T04:47:45.612871+0000","last_unstale":"2024-05-09T04:51:05.651268+0000","last_undegraded":"2024-05-09T04:51:05.651268+0000","last_fullsized":"2024-05-09T04:51:05.651268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:31.516652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.648939+0000","last_change":"2024-05-09T04:48:28.563376+0000","last_active":"2024-05-09T04:51:05.648939+0000","last_peered":"2024-05-09T04:51:05.648939+0000","last_clean":"2024-05-09T04:51:05.648939+0000","last_became_active":"2024-05-09T04:48:28.562973+0000","last_became_peered":"2024-05-09T04:48:28.562973+0000","last_unstale":"2024-05-09T04:51:05.648939+0000","last_undegraded":"2024-05-09T04:51:05.648939+0000","last_fullsized":"2024-05-09T04:51:05.648939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:20.068517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651691+0000","last_change":"2024-05-09T04:47:45.389785+0000","last_active":"2024-05-09T04:51:05.651691+0000","last_peered":"2024-05-09T04:51:05.651691+0000","last_clean":"2024-05-09T04:51:05.651691+0000","last_became_active":"2024-05-09T04:47:45.389159+0000","last_became_peered":"2024-05-09T04:47:45.389159+0000","last_unstale":"2024-05-09T04:51:05.651691+0000","last_undegraded":"2024-05-09T04:51:05.651691+0000","last_fullsized":"2024-05-09T04:51:05.651691+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:28:51.804123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.892023+0000","last_change":"2024-05-09T04:48:28.563170+0000","last_active":"2024-05-09T04:51:05.892023+0000","last_peered":"2024-05-09T04:51:05.892023+0000","last_clean":"2024-05-09T04:51:05.892023+0000","last_became_active":"2024-05-09T04:48:28.562189+0000","last_became_peered":"2024-05-09T04:48:28.562189+0000","last_unstale":"2024-05-09T04:51:05.892023+0000","last_undegraded":"2024-05-09T04:51:05.892023+0000","last_fullsized":"2024-05-09T04:51:05.892023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:00.643292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.648790+0000","last_change":"2024-05-09T04:47:45.613739+0000","last_active":"2024-05-09T04:51:05.648790+0000","last_peered":"2024-05-09T04:51:05.648790+0000","last_clean":"2024-05-09T04:51:05.648790+0000","last_became_active":"2024-05-09T04:47:45.613365+0000","last_became_peered":"2024-05-09T04:47:45.613365+0000","last_unstale":"2024-05-09T04:51:05.648790+0000","last_undegraded":"2024-05-09T04:51:05.648790+0000","last_fullsized":"2024-05-09T04:51:05.648790+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:57.138068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.662064+0000","last_change":"2024-05-09T04:47:48.413343+0000","last_active":"2024-05-09T04:51:05.662064+0000","last_peered":"2024-05-09T04:51:05.662064+0000","last_clean":"2024-05-09T04:51:05.662064+0000","last_became_active":"2024-05-09T04:47:48.412941+0000","last_became_peered":"2024-05-09T04:47:48.412941+0000","last_unstale":"2024-05-09T04:51:05.662064+0000","last_undegraded":"2024-05-09T04:51:05.662064+0000","last_fullsized":"2024-05-09T04:51:05.662064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:28.764553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810369+0000","last_change":"2024-05-09T04:47:45.386487+0000","last_active":"2024-05-09T04:51:05.810369+0000","last_peered":"2024-05-09T04:51:05.810369+0000","last_clean":"2024-05-09T04:51:05.810369+0000","last_became_active":"2024-05-09T04:47:45.386078+0000","last_became_peered":"2024-05-09T04:47:45.386078+0000","last_unstale":"2024-05-09T04:51:05.810369+0000","last_undegraded":"2024-05-09T04:51:05.810369+0000","last_fullsized":"2024-05-09T04:51:05.810369+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:19.998977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.661951+0000","last_change":"2024-05-09T04:47:48.418955+0000","last_active":"2024-05-09T04:51:05.661951+0000","last_peered":"2024-05-09T04:51:05.661951+0000","last_clean":"2024-05-09T04:51:05.661951+0000","last_became_active":"2024-05-09T04:47:48.418528+0000","last_became_peered":"2024-05-09T04:47:48.418528+0000","last_unstale":"2024-05-09T04:51:05.661951+0000","last_undegraded":"2024-05-09T04:51:05.661951+0000","last_fullsized":"2024-05-09T04:51:05.661951+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:51:02.386342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.659089+0000","last_change":"2024-05-09T04:47:48.413304+0000","last_active":"2024-05-09T04:51:05.659089+0000","last_peered":"2024-05-09T04:51:05.659089+0000","last_clean":"2024-05-09T04:51:05.659089+0000","last_became_active":"2024-05-09T04:47:48.412582+0000","last_became_peered":"2024-05-09T04:47:48.412582+0000","last_unstale":"2024-05-09T04:51:05.659089+0000","last_undegraded":"2024-05-09T04:51:05.659089+0000","last_fullsized":"2024-05-09T04:51:05.659089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:40.027082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":217,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.803387+0000","last_change":"2024-05-09T04:47:45.612742+0000","last_active":"2024-05-09T04:51:05.803387+0000","last_peered":"2024-05-09T04:51:05.803387+0000","last_clean":"2024-05-09T04:51:05.803387+0000","last_became_active":"2024-05-09T04:47:45.612325+0000","last_became_peered":"2024-05-09T04:47:45.612325+0000","last_unstale":"2024-05-09T04:51:05.803387+0000","last_undegraded":"2024-05-09T04:51:05.803387+0000","last_fullsized":"2024-05-09T04:51:05.803387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:00.573996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":178,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651673+0000","last_change":"2024-05-09T04:47:48.412894+0000","last_active":"2024-05-09T04:51:05.651673+0000","last_peered":"2024-05-09T04:51:05.651673+0000","last_clean":"2024-05-09T04:51:05.651673+0000","last_became_active":"2024-05-09T04:47:48.412265+0000","last_became_peered":"2024-05-09T04:47:48.412265+0000","last_unstale":"2024-05-09T04:51:05.651673+0000","last_undegraded":"2024-05-09T04:51:05.651673+0000","last_fullsized":"2024-05-09T04:51:05.651673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:23.517566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.805325+0000","last_change":"2024-05-09T04:47:45.387715+0000","last_active":"2024-05-09T04:51:05.805325+0000","last_peered":"2024-05-09T04:51:05.805325+0000","last_clean":"2024-05-09T04:51:05.805325+0000","last_became_active":"2024-05-09T04:47:45.387229+0000","last_became_peered":"2024-05-09T04:47:45.387229+0000","last_unstale":"2024-05-09T04:51:05.805325+0000","last_undegraded":"2024-05-09T04:51:05.805325+0000","last_fullsized":"2024-05-09T04:51:05.805325+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:22.540214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":107,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810292+0000","last_change":"2024-05-09T04:46:21.632579+0000","last_active":"2024-05-09T04:51:05.810292+0000","last_peered":"2024-05-09T04:51:05.810292+0000","last_clean":"2024-05-09T04:51:05.810292+0000","last_became_active":"2024-05-09T04:46:21.627483+0000","last_became_peered":"2024-05-09T04:46:21.627483+0000","last_unstale":"2024-05-09T04:51:05.810292+0000","last_undegraded":"2024-05-09T04:51:05.810292+0000","last_fullsized":"2024-05-09T04:51:05.810292+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:42:54.179739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:42:54.179739+0000","last_clean_scrub_stamp":"2024-05-09T04:42:54.179739+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:44.331526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544198+0000","last_change":"2024-05-09T04:47:48.418570+0000","last_active":"2024-05-09T04:51:05.544198+0000","last_peered":"2024-05-09T04:51:05.544198+0000","last_clean":"2024-05-09T04:51:05.544198+0000","last_became_active":"2024-05-09T04:47:48.418270+0000","last_became_peered":"2024-05-09T04:47:48.418270+0000","last_unstale":"2024-05-09T04:51:05.544198+0000","last_undegraded":"2024-05-09T04:51:05.544198+0000","last_fullsized":"2024-05-09T04:51:05.544198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:28.406846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810387+0000","last_change":"2024-05-09T04:47:45.387161+0000","last_active":"2024-05-09T04:51:05.810387+0000","last_peered":"2024-05-09T04:51:05.810387+0000","last_clean":"2024-05-09T04:51:05.810387+0000","last_became_active":"2024-05-09T04:47:45.386753+0000","last_became_peered":"2024-05-09T04:47:45.386753+0000","last_unstale":"2024-05-09T04:51:05.810387+0000","last_undegraded":"2024-05-09T04:51:05.810387+0000","last_fullsized":"2024-05-09T04:51:05.810387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:15.402418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651727+0000","last_change":"2024-05-09T04:47:45.613551+0000","last_active":"2024-05-09T04:51:05.651727+0000","last_peered":"2024-05-09T04:51:05.651727+0000","last_clean":"2024-05-09T04:51:05.651727+0000","last_became_active":"2024-05-09T04:47:45.613086+0000","last_became_peered":"2024-05-09T04:47:45.613086+0000","last_unstale":"2024-05-09T04:51:05.651727+0000","last_undegraded":"2024-05-09T04:51:05.651727+0000","last_fullsized":"2024-05-09T04:51:05.651727+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:04.350960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544530+0000","last_change":"2024-05-09T04:47:48.403701+0000","last_active":"2024-05-09T04:51:05.544530+0000","last_peered":"2024-05-09T04:51:05.544530+0000","last_clean":"2024-05-09T04:51:05.544530+0000","last_became_active":"2024-05-09T04:47:48.403337+0000","last_became_peered":"2024-05-09T04:47:48.403337+0000","last_unstale":"2024-05-09T04:51:05.544530+0000","last_undegraded":"2024-05-09T04:51:05.544530+0000","last_fullsized":"2024-05-09T04:51:05.544530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:53.032348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'47","reported_seq":102,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.771184+0000","last_change":"2024-05-09T04:47:45.385935+0000","last_active":"2024-05-09T04:51:05.771184+0000","last_peered":"2024-05-09T04:51:05.771184+0000","last_clean":"2024-05-09T04:51:05.771184+0000","last_became_active":"2024-05-09T04:47:45.385533+0000","last_became_peered":"2024-05-09T04:47:45.385533+0000","last_unstale":"2024-05-09T04:51:05.771184+0000","last_undegraded":"2024-05-09T04:51:05.771184+0000","last_fullsized":"2024-05-09T04:51:05.771184+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:34:38.014201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":85,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.652135+0000","last_change":"2024-05-09T04:47:48.409590+0000","last_active":"2024-05-09T04:51:05.652135+0000","last_peered":"2024-05-09T04:51:05.652135+0000","last_clean":"2024-05-09T04:51:05.652135+0000","last_became_active":"2024-05-09T04:47:48.409191+0000","last_became_peered":"2024-05-09T04:47:48.409191+0000","last_unstale":"2024-05-09T04:51:05.652135+0000","last_undegraded":"2024-05-09T04:51:05.652135+0000","last_fullsized":"2024-05-09T04:51:05.652135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:50.440609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.805557+0000","last_change":"2024-05-09T04:47:45.389147+0000","last_active":"2024-05-09T04:51:05.805557+0000","last_peered":"2024-05-09T04:51:05.805557+0000","last_clean":"2024-05-09T04:51:05.805557+0000","last_became_active":"2024-05-09T04:47:45.388797+0000","last_became_peered":"2024-05-09T04:47:45.388797+0000","last_unstale":"2024-05-09T04:51:05.805557+0000","last_undegraded":"2024-05-09T04:51:05.805557+0000","last_fullsized":"2024-05-09T04:51:05.805557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:19.265278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814554+0000","last_change":"2024-05-09T04:47:48.419100+0000","last_active":"2024-05-09T04:51:05.814554+0000","last_peered":"2024-05-09T04:51:05.814554+0000","last_clean":"2024-05-09T04:51:05.814554+0000","last_became_active":"2024-05-09T04:47:48.418499+0000","last_became_peered":"2024-05-09T04:47:48.418499+0000","last_unstale":"2024-05-09T04:51:05.814554+0000","last_undegraded":"2024-05-09T04:51:05.814554+0000","last_fullsized":"2024-05-09T04:51:05.814554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:23.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548394+0000","last_change":"2024-05-09T04:47:45.613269+0000","last_active":"2024-05-09T04:51:05.548394+0000","last_peered":"2024-05-09T04:51:05.548394+0000","last_clean":"2024-05-09T04:51:05.548394+0000","last_became_active":"2024-05-09T04:47:45.612895+0000","last_became_peered":"2024-05-09T04:47:45.612895+0000","last_unstale":"2024-05-09T04:51:05.548394+0000","last_undegraded":"2024-05-09T04:51:05.548394+0000","last_fullsized":"2024-05-09T04:51:05.548394+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:52.966974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651326+0000","last_change":"2024-05-09T04:47:48.416658+0000","last_active":"2024-05-09T04:51:05.651326+0000","last_peered":"2024-05-09T04:51:05.651326+0000","last_clean":"2024-05-09T04:51:05.651326+0000","last_became_active":"2024-05-09T04:47:48.416238+0000","last_became_peered":"2024-05-09T04:47:48.416238+0000","last_unstale":"2024-05-09T04:51:05.651326+0000","last_undegraded":"2024-05-09T04:51:05.651326+0000","last_fullsized":"2024-05-09T04:51:05.651326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:35:50.126723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651564+0000","last_change":"2024-05-09T04:47:45.384018+0000","last_active":"2024-05-09T04:51:05.651564+0000","last_peered":"2024-05-09T04:51:05.651564+0000","last_clean":"2024-05-09T04:51:05.651564+0000","last_became_active":"2024-05-09T04:47:45.383647+0000","last_became_peered":"2024-05-09T04:47:45.383647+0000","last_unstale":"2024-05-09T04:51:05.651564+0000","last_undegraded":"2024-05-09T04:51:05.651564+0000","last_fullsized":"2024-05-09T04:51:05.651564+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:06.431233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.891798+0000","last_change":"2024-05-09T04:47:48.409089+0000","last_active":"2024-05-09T04:51:05.891798+0000","last_peered":"2024-05-09T04:51:05.891798+0000","last_clean":"2024-05-09T04:51:05.891798+0000","last_became_active":"2024-05-09T04:47:48.408548+0000","last_became_peered":"2024-05-09T04:47:48.408548+0000","last_unstale":"2024-05-09T04:51:05.891798+0000","last_undegraded":"2024-05-09T04:51:05.891798+0000","last_fullsized":"2024-05-09T04:51:05.891798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:13.633008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545683+0000","last_change":"2024-05-09T04:47:45.387580+0000","last_active":"2024-05-09T04:51:05.545683+0000","last_peered":"2024-05-09T04:51:05.545683+0000","last_clean":"2024-05-09T04:51:05.545683+0000","last_became_active":"2024-05-09T04:47:45.386929+0000","last_became_peered":"2024-05-09T04:47:45.386929+0000","last_unstale":"2024-05-09T04:51:05.545683+0000","last_undegraded":"2024-05-09T04:51:05.545683+0000","last_fullsized":"2024-05-09T04:51:05.545683+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:19.661763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651461+0000","last_change":"2024-05-09T04:47:48.416311+0000","last_active":"2024-05-09T04:51:05.651461+0000","last_peered":"2024-05-09T04:51:05.651461+0000","last_clean":"2024-05-09T04:51:05.651461+0000","last_became_active":"2024-05-09T04:47:48.415969+0000","last_became_peered":"2024-05-09T04:47:48.415969+0000","last_unstale":"2024-05-09T04:51:05.651461+0000","last_undegraded":"2024-05-09T04:51:05.651461+0000","last_fullsized":"2024-05-09T04:51:05.651461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:55.751210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651448+0000","last_change":"2024-05-09T04:48:28.573916+0000","last_active":"2024-05-09T04:51:05.651448+0000","last_peered":"2024-05-09T04:51:05.651448+0000","last_clean":"2024-05-09T04:51:05.651448+0000","last_became_active":"2024-05-09T04:48:28.562872+0000","last_became_peered":"2024-05-09T04:48:28.562872+0000","last_unstale":"2024-05-09T04:51:05.651448+0000","last_undegraded":"2024-05-09T04:51:05.651448+0000","last_fullsized":"2024-05-09T04:51:05.651448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:02.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810462+0000","last_change":"2024-05-09T04:47:48.408364+0000","last_active":"2024-05-09T04:51:05.810462+0000","last_peered":"2024-05-09T04:51:05.810462+0000","last_clean":"2024-05-09T04:51:05.810462+0000","last_became_active":"2024-05-09T04:47:48.407982+0000","last_became_peered":"2024-05-09T04:47:48.407982+0000","last_unstale":"2024-05-09T04:51:05.810462+0000","last_undegraded":"2024-05-09T04:51:05.810462+0000","last_fullsized":"2024-05-09T04:51:05.810462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:14.103171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545878+0000","last_change":"2024-05-09T04:47:45.614637+0000","last_active":"2024-05-09T04:51:05.545878+0000","last_peered":"2024-05-09T04:51:05.545878+0000","last_clean":"2024-05-09T04:51:05.545878+0000","last_became_active":"2024-05-09T04:47:45.613290+0000","last_became_peered":"2024-05-09T04:47:45.613290+0000","last_unstale":"2024-05-09T04:51:05.545878+0000","last_undegraded":"2024-05-09T04:51:05.545878+0000","last_fullsized":"2024-05-09T04:51:05.545878+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:48.547382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810433+0000","last_change":"2024-05-09T04:47:48.405537+0000","last_active":"2024-05-09T04:51:05.810433+0000","last_peered":"2024-05-09T04:51:05.810433+0000","last_clean":"2024-05-09T04:51:05.810433+0000","last_became_active":"2024-05-09T04:47:48.405148+0000","last_became_peered":"2024-05-09T04:47:48.405148+0000","last_unstale":"2024-05-09T04:51:05.810433+0000","last_undegraded":"2024-05-09T04:51:05.810433+0000","last_fullsized":"2024-05-09T04:51:05.810433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:46.981401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'39","reported_seq":79,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.768865+0000","last_change":"2024-05-09T04:47:45.389151+0000","last_active":"2024-05-09T04:51:05.768865+0000","last_peered":"2024-05-09T04:51:05.768865+0000","last_clean":"2024-05-09T04:51:05.768865+0000","last_became_active":"2024-05-09T04:47:45.388735+0000","last_became_peered":"2024-05-09T04:47:45.388735+0000","last_unstale":"2024-05-09T04:51:05.768865+0000","last_undegraded":"2024-05-09T04:51:05.768865+0000","last_fullsized":"2024-05-09T04:51:05.768865+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:26.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810206+0000","last_change":"2024-05-09T04:47:48.413881+0000","last_active":"2024-05-09T04:51:05.810206+0000","last_peered":"2024-05-09T04:51:05.810206+0000","last_clean":"2024-05-09T04:51:05.810206+0000","last_became_active":"2024-05-09T04:47:48.413559+0000","last_became_peered":"2024-05-09T04:47:48.413559+0000","last_unstale":"2024-05-09T04:51:05.810206+0000","last_undegraded":"2024-05-09T04:51:05.810206+0000","last_fullsized":"2024-05-09T04:51:05.810206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:40:40.305319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544766+0000","last_change":"2024-05-09T04:47:45.383406+0000","last_active":"2024-05-09T04:51:05.544766+0000","last_peered":"2024-05-09T04:51:05.544766+0000","last_clean":"2024-05-09T04:51:05.544766+0000","last_became_active":"2024-05-09T04:47:45.383025+0000","last_became_peered":"2024-05-09T04:47:45.383025+0000","last_unstale":"2024-05-09T04:51:05.544766+0000","last_undegraded":"2024-05-09T04:51:05.544766+0000","last_fullsized":"2024-05-09T04:51:05.544766+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:46.007454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810679+0000","last_change":"2024-05-09T04:47:48.412481+0000","last_active":"2024-05-09T04:51:05.810679+0000","last_peered":"2024-05-09T04:51:05.810679+0000","last_clean":"2024-05-09T04:51:05.810679+0000","last_became_active":"2024-05-09T04:47:48.412034+0000","last_became_peered":"2024-05-09T04:47:48.412034+0000","last_unstale":"2024-05-09T04:51:05.810679+0000","last_undegraded":"2024-05-09T04:51:05.810679+0000","last_fullsized":"2024-05-09T04:51:05.810679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:14.030981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.891860+0000","last_change":"2024-05-09T04:47:48.406958+0000","last_active":"2024-05-09T04:51:05.891860+0000","last_peered":"2024-05-09T04:51:05.891860+0000","last_clean":"2024-05-09T04:51:05.891860+0000","last_became_active":"2024-05-09T04:47:48.406575+0000","last_became_peered":"2024-05-09T04:47:48.406575+0000","last_unstale":"2024-05-09T04:51:05.891860+0000","last_undegraded":"2024-05-09T04:51:05.891860+0000","last_fullsized":"2024-05-09T04:51:05.891860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:23.389676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545618+0000","last_change":"2024-05-09T04:48:28.562376+0000","last_active":"2024-05-09T04:51:05.545618+0000","last_peered":"2024-05-09T04:51:05.545618+0000","last_clean":"2024-05-09T04:51:05.545618+0000","last_became_active":"2024-05-09T04:48:28.561866+0000","last_became_peered":"2024-05-09T04:48:28.561866+0000","last_unstale":"2024-05-09T04:51:05.545618+0000","last_undegraded":"2024-05-09T04:51:05.545618+0000","last_fullsized":"2024-05-09T04:51:05.545618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:51.974146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545565+0000","last_change":"2024-05-09T04:47:48.413908+0000","last_active":"2024-05-09T04:51:05.545565+0000","last_peered":"2024-05-09T04:51:05.545565+0000","last_clean":"2024-05-09T04:51:05.545565+0000","last_became_active":"2024-05-09T04:47:48.413575+0000","last_became_peered":"2024-05-09T04:47:48.413575+0000","last_unstale":"2024-05-09T04:51:05.545565+0000","last_undegraded":"2024-05-09T04:51:05.545565+0000","last_fullsized":"2024-05-09T04:51:05.545565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:40.742510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814127+0000","last_change":"2024-05-09T04:47:48.406607+0000","last_active":"2024-05-09T04:51:05.814127+0000","last_peered":"2024-05-09T04:51:05.814127+0000","last_clean":"2024-05-09T04:51:05.814127+0000","last_became_active":"2024-05-09T04:47:48.406290+0000","last_became_peered":"2024-05-09T04:47:48.406290+0000","last_unstale":"2024-05-09T04:51:05.814127+0000","last_undegraded":"2024-05-09T04:51:05.814127+0000","last_fullsized":"2024-05-09T04:51:05.814127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:22.107953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544591+0000","last_change":"2024-05-09T04:47:48.415153+0000","last_active":"2024-05-09T04:51:05.544591+0000","last_peered":"2024-05-09T04:51:05.544591+0000","last_clean":"2024-05-09T04:51:05.544591+0000","last_became_active":"2024-05-09T04:47:48.414793+0000","last_became_peered":"2024-05-09T04:47:48.414793+0000","last_unstale":"2024-05-09T04:51:05.544591+0000","last_undegraded":"2024-05-09T04:51:05.544591+0000","last_fullsized":"2024-05-09T04:51:05.544591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:57.564091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.892197+0000","last_change":"2024-05-09T04:47:48.408646+0000","last_active":"2024-05-09T04:51:05.892197+0000","last_peered":"2024-05-09T04:51:05.892197+0000","last_clean":"2024-05-09T04:51:05.892197+0000","last_became_active":"2024-05-09T04:47:48.408250+0000","last_became_peered":"2024-05-09T04:47:48.408250+0000","last_unstale":"2024-05-09T04:51:05.892197+0000","last_undegraded":"2024-05-09T04:51:05.892197+0000","last_fullsized":"2024-05-09T04:51:05.892197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:42:50.622937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.546131+0000","last_change":"2024-05-09T04:47:48.408686+0000","last_active":"2024-05-09T04:51:05.546131+0000","last_peered":"2024-05-09T04:51:05.546131+0000","last_clean":"2024-05-09T04:51:05.546131+0000","last_became_active":"2024-05-09T04:47:48.408318+0000","last_became_peered":"2024-05-09T04:47:48.408318+0000","last_unstale":"2024-05-09T04:51:05.546131+0000","last_undegraded":"2024-05-09T04:51:05.546131+0000","last_fullsized":"2024-05-09T04:51:05.546131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:51.240485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.652134+0000","last_change":"2024-05-09T04:47:48.421562+0000","last_active":"2024-05-09T04:51:05.652134+0000","last_peered":"2024-05-09T04:51:05.652134+0000","last_clean":"2024-05-09T04:51:05.652134+0000","last_became_active":"2024-05-09T04:47:48.421068+0000","last_became_peered":"2024-05-09T04:47:48.421068+0000","last_unstale":"2024-05-09T04:51:05.652134+0000","last_undegraded":"2024-05-09T04:51:05.652134+0000","last_fullsized":"2024-05-09T04:51:05.652134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:18:41.455187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.658896+0000","last_change":"2024-05-09T04:47:48.413942+0000","last_active":"2024-05-09T04:51:05.658896+0000","last_peered":"2024-05-09T04:51:05.658896+0000","last_clean":"2024-05-09T04:51:05.658896+0000","last_became_active":"2024-05-09T04:47:48.413561+0000","last_became_peered":"2024-05-09T04:47:48.413561+0000","last_unstale":"2024-05-09T04:51:05.658896+0000","last_undegraded":"2024-05-09T04:51:05.658896+0000","last_fullsized":"2024-05-09T04:51:05.658896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:33:48.148834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651456+0000","last_change":"2024-05-09T04:47:48.408752+0000","last_active":"2024-05-09T04:51:05.651456+0000","last_peered":"2024-05-09T04:51:05.651456+0000","last_clean":"2024-05-09T04:51:05.651456+0000","last_became_active":"2024-05-09T04:47:48.408393+0000","last_became_peered":"2024-05-09T04:47:48.408393+0000","last_unstale":"2024-05-09T04:51:05.651456+0000","last_undegraded":"2024-05-09T04:51:05.651456+0000","last_fullsized":"2024-05-09T04:51:05.651456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:06.778002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548656+0000","last_change":"2024-05-09T04:47:48.409303+0000","last_active":"2024-05-09T04:51:05.548656+0000","last_peered":"2024-05-09T04:51:05.548656+0000","last_clean":"2024-05-09T04:51:05.548656+0000","last_became_active":"2024-05-09T04:47:48.408971+0000","last_became_peered":"2024-05-09T04:47:48.408971+0000","last_unstale":"2024-05-09T04:51:05.548656+0000","last_undegraded":"2024-05-09T04:51:05.548656+0000","last_fullsized":"2024-05-09T04:51:05.548656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:45.830647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544522+0000","last_change":"2024-05-09T04:47:48.404961+0000","last_active":"2024-05-09T04:51:05.544522+0000","last_peered":"2024-05-09T04:51:05.544522+0000","last_clean":"2024-05-09T04:51:05.544522+0000","last_became_active":"2024-05-09T04:47:48.404577+0000","last_became_peered":"2024-05-09T04:47:48.404577+0000","last_unstale":"2024-05-09T04:51:05.544522+0000","last_undegraded":"2024-05-09T04:51:05.544522+0000","last_fullsized":"2024-05-09T04:51:05.544522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:40.528066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.891879+0000","last_change":"2024-05-09T04:47:48.424364+0000","last_active":"2024-05-09T04:51:05.891879+0000","last_peered":"2024-05-09T04:51:05.891879+0000","last_clean":"2024-05-09T04:51:05.891879+0000","last_became_active":"2024-05-09T04:47:48.423991+0000","last_became_peered":"2024-05-09T04:47:48.423991+0000","last_unstale":"2024-05-09T04:51:05.891879+0000","last_undegraded":"2024-05-09T04:51:05.891879+0000","last_fullsized":"2024-05-09T04:51:05.891879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:14.040560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108459,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597498,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1187,"num_read_kb":688,"num_write":293,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":50695,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":1792494,"data_compressed":84389,"data_compressed_allocated":110592,"data_compressed_original":1597440,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645367,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":917504,"data_stored":458040,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841594,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1249103,"data_compressed":56798,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005120,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":851492,"data_compressed":33635,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201351,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":458196,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397580,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":1051748,"data_compressed":50642,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561105,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":457665,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":524517,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":458936,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":459898,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117089,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":982792,"data_compressed":39791,"data_compressed_allocated":53248,"data_compressed_original":557056,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":510524,"data_compressed":20057,"data_compressed_allocated":28672,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757334,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":1182109,"data_compressed":56798,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":68626,"data_compressed":8483,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134166,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201160,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:51:08.467 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:51:08.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.632+0000 7f60f34d1640 1 -- 172.21.15.46:0/3430546847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec103d30 msgr2=0x7f60ec104140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:08.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.632+0000 7f60f34d1640 1 --2- 172.21.15.46:0/3430546847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec103d30 0x7f60ec104140 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009900 tx=0x7f60dc02a410 comp rx=0 tx=0).stop 2024-05-09T04:51:08.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.632+0000 7f60f34d1640 1 -- 172.21.15.46:0/3430546847 shutdown_connections 2024-05-09T04:51:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.632+0000 7f60f34d1640 1 --2- 172.21.15.46:0/3430546847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec108a30 0x7f60ec10ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.632+0000 7f60f34d1640 1 --2- 172.21.15.46:0/3430546847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec103d30 0x7f60ec104140 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.632+0000 7f60f34d1640 1 --2- 172.21.15.46:0/3430546847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ec102af0 0x7f60ec1037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.632+0000 7f60f34d1640 1 -- 172.21.15.46:0/3430546847 >> 172.21.15.46:0/3430546847 conn(0x7f60ec0fdd00 msgr2=0x7f60ec100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.632+0000 7f60f34d1640 1 -- 172.21.15.46:0/3430546847 shutdown_connections 2024-05-09T04:51:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.632+0000 7f60f34d1640 1 -- 172.21.15.46:0/3430546847 wait complete. 2024-05-09T04:51:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f34d1640 1 Processor -- start 2024-05-09T04:51:08.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f34d1640 1 -- start start 2024-05-09T04:51:08.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f34d1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ec102af0 0x7f60ec19c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:08.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f34d1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec103d30 0x7f60ec19ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:08.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f34d1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec108a30 0x7f60ec1a3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:08.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f34d1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec10e5f0 con 0x7f60ec108a30 2024-05-09T04:51:08.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f34d1640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec10e470 con 0x7f60ec102af0 2024-05-09T04:51:08.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f34d1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f60ec10e770 con 0x7f60ec103d30 2024-05-09T04:51:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f1a6e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec108a30 0x7f60ec1a3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f126d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ec102af0 0x7f60ec19c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f0a6c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec103d30 0x7f60ec19ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f1a6e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec108a30 0x7f60ec1a3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36428/0 (socket says 172.21.15.46:36428) 2024-05-09T04:51:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f126d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ec102af0 0x7f60ec19c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:45426/0 (socket says 172.21.15.46:45426) 2024-05-09T04:51:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f1a6e640 1 -- 172.21.15.46:0/3409662402 learned_addr learned my addr 172.21.15.46:0/3409662402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f126d640 1 -- 172.21.15.46:0/3409662402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec103d30 msgr2=0x7f60ec19ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f126d640 1 --2- 172.21.15.46:0/3409662402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec103d30 0x7f60ec19ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f126d640 1 -- 172.21.15.46:0/3409662402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec108a30 msgr2=0x7f60ec1a3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f126d640 1 --2- 172.21.15.46:0/3409662402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec108a30 0x7f60ec1a3a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f126d640 1 -- 172.21.15.46:0/3409662402 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60ec1a4080 con 0x7f60ec102af0 2024-05-09T04:51:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f0a6c640 1 --2- 172.21.15.46:0/3409662402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec103d30 0x7f60ec19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.633+0000 7f60f1a6e640 1 --2- 172.21.15.46:0/3409662402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec108a30 0x7f60ec1a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:08.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.634+0000 7f60f126d640 1 --2- 172.21.15.46:0/3409662402 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ec102af0 0x7f60ec19c540 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f60ec1023c0 tx=0x7f60e800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:08.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.634+0000 7f60e27fc640 1 -- 172.21.15.46:0/3409662402 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e800cd00 con 0x7f60ec102af0 2024-05-09T04:51:08.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.634+0000 7f60f34d1640 1 -- 172.21.15.46:0/3409662402 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60ec1a4370 con 0x7f60ec102af0 2024-05-09T04:51:08.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.634+0000 7f60e27fc640 1 -- 172.21.15.46:0/3409662402 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60e80108e0 con 0x7f60ec102af0 2024-05-09T04:51:08.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.634+0000 7f60e27fc640 1 -- 172.21.15.46:0/3409662402 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60e80187c0 con 0x7f60ec102af0 2024-05-09T04:51:08.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.634+0000 7f60f34d1640 1 -- 172.21.15.46:0/3409662402 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60ec1a48c0 con 0x7f60ec102af0 2024-05-09T04:51:08.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.635+0000 7f60f34d1640 1 -- 172.21.15.46:0/3409662402 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60ec106120 con 0x7f60ec102af0 2024-05-09T04:51:08.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.638+0000 7f60e27fc640 1 -- 172.21.15.46:0/3409662402 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 23) ==== 149772+0+0 (secure 0 0 0) 0x7f60e801f020 con 0x7f60ec102af0 2024-05-09T04:51:08.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.639+0000 7f60e27fc640 1 --2- 172.21.15.46:0/3409662402 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f60d80b2960 0x7f60d80b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:08.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.639+0000 7f60f0a6c640 1 --2- 172.21.15.46:0/3409662402 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f60d80b2960 0x7f60d80b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:08.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.639+0000 7f60e27fc640 1 -- 172.21.15.46:0/3409662402 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f60e80e2710 con 0x7f60ec102af0 2024-05-09T04:51:08.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.639+0000 7f60f0a6c640 1 --2- 172.21.15.46:0/3409662402 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f60d80b2960 0x7f60d80b4e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009230 tx=0x7f60dc031400 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:08.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.646+0000 7f60e27fc640 1 -- 172.21.15.46:0/3409662402 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f60e80aa0d0 con 0x7f60ec102af0 2024-05-09T04:51:08.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:08 smithi046 ceph-mon[30136]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:51:08.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:08 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:08.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:08 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:08.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:08 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:08.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:08 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:08.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:08 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:08.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:08 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:08.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:08 smithi046 ceph-mon[30136]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-05-09T04:51:08.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.910+0000 7f60f34d1640 1 -- 172.21.15.46:0/3409662402 --> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f60ec063110 con 0x7f60d80b2960 2024-05-09T04:51:08.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.918+0000 7f60e27fc640 1 -- 172.21.15.46:0/3409662402 <== mgr.15150 v2:172.21.15.46:6800/2595796783 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258023 (secure 0 0 0) 0x7f60ec063110 con 0x7f60d80b2960 2024-05-09T04:51:08.921 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:51:08.922 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T04:51:08.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.921+0000 7f60f34d1640 1 -- 172.21.15.46:0/3409662402 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f60d80b2960 msgr2=0x7f60d80b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:08.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.921+0000 7f60f34d1640 1 --2- 172.21.15.46:0/3409662402 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f60d80b2960 0x7f60d80b4e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f60dc009230 tx=0x7f60dc031400 comp rx=0 tx=0).stop 2024-05-09T04:51:08.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.921+0000 7f60f34d1640 1 -- 172.21.15.46:0/3409662402 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ec102af0 msgr2=0x7f60ec19c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:08.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.921+0000 7f60f34d1640 1 --2- 172.21.15.46:0/3409662402 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ec102af0 0x7f60ec19c540 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f60ec1023c0 tx=0x7f60e800ee50 comp rx=0 tx=0).stop 2024-05-09T04:51:08.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.921+0000 7f60f34d1640 1 -- 172.21.15.46:0/3409662402 shutdown_connections 2024-05-09T04:51:08.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.921+0000 7f60f34d1640 1 --2- 172.21.15.46:0/3409662402 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f60d80b2960 0x7f60d80b4e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.921+0000 7f60f34d1640 1 --2- 172.21.15.46:0/3409662402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f60ec103d30 0x7f60ec19ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.921+0000 7f60f34d1640 1 --2- 172.21.15.46:0/3409662402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f60ec108a30 0x7f60ec1a3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.921+0000 7f60f34d1640 1 --2- 172.21.15.46:0/3409662402 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f60ec102af0 0x7f60ec19c540 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:08.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.922+0000 7f60f34d1640 1 -- 172.21.15.46:0/3409662402 >> 172.21.15.46:0/3409662402 conn(0x7f60ec0fdd00 msgr2=0x7f60ec100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:08.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.922+0000 7f60f34d1640 1 -- 172.21.15.46:0/3409662402 shutdown_connections 2024-05-09T04:51:08.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:08.922+0000 7f60f34d1640 1 -- 172.21.15.46:0/3409662402 wait complete. 2024-05-09T04:51:08.933 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-05-09T04:51:07.560145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188100,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1268,"num_read_kb":757,"num_write":418,"num_write_kb":3591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":3002535,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":823,"ondisk_log_size":823,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368860,"kb_used_data":11356,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572964,"statfs":{"total":1151940427776,"available":1151562715136,"internally_reserved":0,"allocated":11628544,"data_stored":8645020,"data_compressed":344655,"data_compressed_allocated":462848,"data_compressed_original":3457024,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.989646"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.892107+0000","last_change":"2024-05-09T04:47:48.408878+0000","last_active":"2024-05-09T04:51:05.892107+0000","last_peered":"2024-05-09T04:51:05.892107+0000","last_clean":"2024-05-09T04:51:05.892107+0000","last_became_active":"2024-05-09T04:47:48.408479+0000","last_became_peered":"2024-05-09T04:47:48.408479+0000","last_unstale":"2024-05-09T04:51:05.892107+0000","last_undegraded":"2024-05-09T04:51:05.892107+0000","last_fullsized":"2024-05-09T04:51:05.892107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:02.494768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814499+0000","last_change":"2024-05-09T04:47:48.415650+0000","last_active":"2024-05-09T04:51:05.814499+0000","last_peered":"2024-05-09T04:51:05.814499+0000","last_clean":"2024-05-09T04:51:05.814499+0000","last_became_active":"2024-05-09T04:47:48.415322+0000","last_became_peered":"2024-05-09T04:47:48.415322+0000","last_unstale":"2024-05-09T04:51:05.814499+0000","last_undegraded":"2024-05-09T04:51:05.814499+0000","last_fullsized":"2024-05-09T04:51:05.814499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.142513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814332+0000","last_change":"2024-05-09T04:47:48.417706+0000","last_active":"2024-05-09T04:51:05.814332+0000","last_peered":"2024-05-09T04:51:05.814332+0000","last_clean":"2024-05-09T04:51:05.814332+0000","last_became_active":"2024-05-09T04:47:48.412068+0000","last_became_peered":"2024-05-09T04:47:48.412068+0000","last_unstale":"2024-05-09T04:51:05.814332+0000","last_undegraded":"2024-05-09T04:51:05.814332+0000","last_fullsized":"2024-05-09T04:51:05.814332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:44.602567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.805515+0000","last_change":"2024-05-09T04:47:48.415751+0000","last_active":"2024-05-09T04:51:05.805515+0000","last_peered":"2024-05-09T04:51:05.805515+0000","last_clean":"2024-05-09T04:51:05.805515+0000","last_became_active":"2024-05-09T04:47:48.414312+0000","last_became_peered":"2024-05-09T04:47:48.414312+0000","last_unstale":"2024-05-09T04:51:05.805515+0000","last_undegraded":"2024-05-09T04:51:05.805515+0000","last_fullsized":"2024-05-09T04:51:05.805515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:12.202761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651437+0000","last_change":"2024-05-09T04:47:48.415919+0000","last_active":"2024-05-09T04:51:05.651437+0000","last_peered":"2024-05-09T04:51:05.651437+0000","last_clean":"2024-05-09T04:51:05.651437+0000","last_became_active":"2024-05-09T04:47:48.415533+0000","last_became_peered":"2024-05-09T04:47:48.415533+0000","last_unstale":"2024-05-09T04:51:05.651437+0000","last_undegraded":"2024-05-09T04:51:05.651437+0000","last_fullsized":"2024-05-09T04:51:05.651437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:32.262416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.662001+0000","last_change":"2024-05-09T04:47:48.419705+0000","last_active":"2024-05-09T04:51:05.662001+0000","last_peered":"2024-05-09T04:51:05.662001+0000","last_clean":"2024-05-09T04:51:05.662001+0000","last_became_active":"2024-05-09T04:47:48.419339+0000","last_became_peered":"2024-05-09T04:47:48.419339+0000","last_unstale":"2024-05-09T04:51:05.662001+0000","last_undegraded":"2024-05-09T04:51:05.662001+0000","last_fullsized":"2024-05-09T04:51:05.662001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:50:23.754071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810211+0000","last_change":"2024-05-09T04:47:48.404106+0000","last_active":"2024-05-09T04:51:05.810211+0000","last_peered":"2024-05-09T04:51:05.810211+0000","last_clean":"2024-05-09T04:51:05.810211+0000","last_became_active":"2024-05-09T04:47:48.403724+0000","last_became_peered":"2024-05-09T04:47:48.403724+0000","last_unstale":"2024-05-09T04:51:05.810211+0000","last_undegraded":"2024-05-09T04:51:05.810211+0000","last_fullsized":"2024-05-09T04:51:05.810211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:41.134752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.648971+0000","last_change":"2024-05-09T04:48:28.562515+0000","last_active":"2024-05-09T04:51:05.648971+0000","last_peered":"2024-05-09T04:51:05.648971+0000","last_clean":"2024-05-09T04:51:05.648971+0000","last_became_active":"2024-05-09T04:48:28.562104+0000","last_became_peered":"2024-05-09T04:48:28.562104+0000","last_unstale":"2024-05-09T04:51:05.648971+0000","last_undegraded":"2024-05-09T04:51:05.648971+0000","last_fullsized":"2024-05-09T04:51:05.648971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:33.342877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548850+0000","last_change":"2024-05-09T04:47:48.400467+0000","last_active":"2024-05-09T04:51:05.548850+0000","last_peered":"2024-05-09T04:51:05.548850+0000","last_clean":"2024-05-09T04:51:05.548850+0000","last_became_active":"2024-05-09T04:47:48.399960+0000","last_became_peered":"2024-05-09T04:47:48.399960+0000","last_unstale":"2024-05-09T04:51:05.548850+0000","last_undegraded":"2024-05-09T04:51:05.548850+0000","last_fullsized":"2024-05-09T04:51:05.548850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:23:44.473053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.662377+0000","last_change":"2024-05-09T04:47:48.418970+0000","last_active":"2024-05-09T04:51:05.662377+0000","last_peered":"2024-05-09T04:51:05.662377+0000","last_clean":"2024-05-09T04:51:05.662377+0000","last_became_active":"2024-05-09T04:47:48.418574+0000","last_became_peered":"2024-05-09T04:47:48.418574+0000","last_unstale":"2024-05-09T04:51:05.662377+0000","last_undegraded":"2024-05-09T04:51:05.662377+0000","last_fullsized":"2024-05-09T04:51:05.662377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:48.075893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810307+0000","last_change":"2024-05-09T04:47:48.409879+0000","last_active":"2024-05-09T04:51:05.810307+0000","last_peered":"2024-05-09T04:51:05.810307+0000","last_clean":"2024-05-09T04:51:05.810307+0000","last_became_active":"2024-05-09T04:47:48.409515+0000","last_became_peered":"2024-05-09T04:47:48.409515+0000","last_unstale":"2024-05-09T04:51:05.810307+0000","last_undegraded":"2024-05-09T04:51:05.810307+0000","last_fullsized":"2024-05-09T04:51:05.810307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:15.546214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.805520+0000","last_change":"2024-05-09T04:47:48.417556+0000","last_active":"2024-05-09T04:51:05.805520+0000","last_peered":"2024-05-09T04:51:05.805520+0000","last_clean":"2024-05-09T04:51:05.805520+0000","last_became_active":"2024-05-09T04:47:48.416787+0000","last_became_peered":"2024-05-09T04:47:48.416787+0000","last_unstale":"2024-05-09T04:51:05.805520+0000","last_undegraded":"2024-05-09T04:51:05.805520+0000","last_fullsized":"2024-05-09T04:51:05.805520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:20.360232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544479+0000","last_change":"2024-05-09T04:47:48.415145+0000","last_active":"2024-05-09T04:51:05.544479+0000","last_peered":"2024-05-09T04:51:05.544479+0000","last_clean":"2024-05-09T04:51:05.544479+0000","last_became_active":"2024-05-09T04:47:48.414830+0000","last_became_peered":"2024-05-09T04:47:48.414830+0000","last_unstale":"2024-05-09T04:51:05.544479+0000","last_undegraded":"2024-05-09T04:51:05.544479+0000","last_fullsized":"2024-05-09T04:51:05.544479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:55.936409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814035+0000","last_change":"2024-05-09T04:47:48.413977+0000","last_active":"2024-05-09T04:51:05.814035+0000","last_peered":"2024-05-09T04:51:05.814035+0000","last_clean":"2024-05-09T04:51:05.814035+0000","last_became_active":"2024-05-09T04:47:48.413604+0000","last_became_peered":"2024-05-09T04:47:48.413604+0000","last_unstale":"2024-05-09T04:51:05.814035+0000","last_undegraded":"2024-05-09T04:51:05.814035+0000","last_fullsized":"2024-05-09T04:51:05.814035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:25.462958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.813924+0000","last_change":"2024-05-09T04:47:48.419000+0000","last_active":"2024-05-09T04:51:05.813924+0000","last_peered":"2024-05-09T04:51:05.813924+0000","last_clean":"2024-05-09T04:51:05.813924+0000","last_became_active":"2024-05-09T04:47:48.418457+0000","last_became_peered":"2024-05-09T04:47:48.418457+0000","last_unstale":"2024-05-09T04:51:05.813924+0000","last_undegraded":"2024-05-09T04:51:05.813924+0000","last_fullsized":"2024-05-09T04:51:05.813924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:36.325460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548387+0000","last_change":"2024-05-09T04:47:48.406754+0000","last_active":"2024-05-09T04:51:05.548387+0000","last_peered":"2024-05-09T04:51:05.548387+0000","last_clean":"2024-05-09T04:51:05.548387+0000","last_became_active":"2024-05-09T04:47:48.406396+0000","last_became_peered":"2024-05-09T04:47:48.406396+0000","last_unstale":"2024-05-09T04:51:05.548387+0000","last_undegraded":"2024-05-09T04:51:05.548387+0000","last_fullsized":"2024-05-09T04:51:05.548387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:46.421061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.892267+0000","last_change":"2024-05-09T04:47:48.409556+0000","last_active":"2024-05-09T04:51:05.892267+0000","last_peered":"2024-05-09T04:51:05.892267+0000","last_clean":"2024-05-09T04:51:05.892267+0000","last_became_active":"2024-05-09T04:47:48.409145+0000","last_became_peered":"2024-05-09T04:47:48.409145+0000","last_unstale":"2024-05-09T04:51:05.892267+0000","last_undegraded":"2024-05-09T04:51:05.892267+0000","last_fullsized":"2024-05-09T04:51:05.892267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:25.486257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.649670+0000","last_change":"2024-05-09T04:47:48.415546+0000","last_active":"2024-05-09T04:51:05.649670+0000","last_peered":"2024-05-09T04:51:05.649670+0000","last_clean":"2024-05-09T04:51:05.649670+0000","last_became_active":"2024-05-09T04:47:48.415221+0000","last_became_peered":"2024-05-09T04:47:48.415221+0000","last_unstale":"2024-05-09T04:51:05.649670+0000","last_undegraded":"2024-05-09T04:51:05.649670+0000","last_fullsized":"2024-05-09T04:51:05.649670+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:57:29.112409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545923+0000","last_change":"2024-05-09T04:47:48.413106+0000","last_active":"2024-05-09T04:51:05.545923+0000","last_peered":"2024-05-09T04:51:05.545923+0000","last_clean":"2024-05-09T04:51:05.545923+0000","last_became_active":"2024-05-09T04:47:48.412700+0000","last_became_peered":"2024-05-09T04:47:48.412700+0000","last_unstale":"2024-05-09T04:51:05.545923+0000","last_undegraded":"2024-05-09T04:51:05.545923+0000","last_fullsized":"2024-05-09T04:51:05.545923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:37:31.682008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.649600+0000","last_change":"2024-05-09T04:47:45.386931+0000","last_active":"2024-05-09T04:51:05.649600+0000","last_peered":"2024-05-09T04:51:05.649600+0000","last_clean":"2024-05-09T04:51:05.649600+0000","last_became_active":"2024-05-09T04:47:45.386562+0000","last_became_peered":"2024-05-09T04:47:45.386562+0000","last_unstale":"2024-05-09T04:51:05.649600+0000","last_undegraded":"2024-05-09T04:51:05.649600+0000","last_fullsized":"2024-05-09T04:51:05.649600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:59.335687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.652165+0000","last_change":"2024-05-09T04:47:48.411064+0000","last_active":"2024-05-09T04:51:05.652165+0000","last_peered":"2024-05-09T04:51:05.652165+0000","last_clean":"2024-05-09T04:51:05.652165+0000","last_became_active":"2024-05-09T04:47:48.410689+0000","last_became_peered":"2024-05-09T04:47:48.410689+0000","last_unstale":"2024-05-09T04:51:05.652165+0000","last_undegraded":"2024-05-09T04:51:05.652165+0000","last_fullsized":"2024-05-09T04:51:05.652165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:21.178831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.652322+0000","last_change":"2024-05-09T04:47:45.612837+0000","last_active":"2024-05-09T04:51:05.652322+0000","last_peered":"2024-05-09T04:51:05.652322+0000","last_clean":"2024-05-09T04:51:05.652322+0000","last_became_active":"2024-05-09T04:47:45.612307+0000","last_became_peered":"2024-05-09T04:47:45.612307+0000","last_unstale":"2024-05-09T04:51:05.652322+0000","last_undegraded":"2024-05-09T04:51:05.652322+0000","last_fullsized":"2024-05-09T04:51:05.652322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:59.686391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814704+0000","last_change":"2024-05-09T04:47:48.412781+0000","last_active":"2024-05-09T04:51:05.814704+0000","last_peered":"2024-05-09T04:51:05.814704+0000","last_clean":"2024-05-09T04:51:05.814704+0000","last_became_active":"2024-05-09T04:47:48.411396+0000","last_became_peered":"2024-05-09T04:47:48.411396+0000","last_unstale":"2024-05-09T04:51:05.814704+0000","last_undegraded":"2024-05-09T04:51:05.814704+0000","last_fullsized":"2024-05-09T04:51:05.814704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:41:22.775432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.892130+0000","last_change":"2024-05-09T04:48:28.568024+0000","last_active":"2024-05-09T04:51:05.892130+0000","last_peered":"2024-05-09T04:51:05.892130+0000","last_clean":"2024-05-09T04:51:05.892130+0000","last_became_active":"2024-05-09T04:48:28.559441+0000","last_became_peered":"2024-05-09T04:48:28.559441+0000","last_unstale":"2024-05-09T04:51:05.892130+0000","last_undegraded":"2024-05-09T04:51:05.892130+0000","last_fullsized":"2024-05-09T04:51:05.892130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:56.257589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548481+0000","last_change":"2024-05-09T04:47:48.421361+0000","last_active":"2024-05-09T04:51:05.548481+0000","last_peered":"2024-05-09T04:51:05.548481+0000","last_clean":"2024-05-09T04:51:05.548481+0000","last_became_active":"2024-05-09T04:47:48.421018+0000","last_became_peered":"2024-05-09T04:47:48.421018+0000","last_unstale":"2024-05-09T04:51:05.548481+0000","last_undegraded":"2024-05-09T04:51:05.548481+0000","last_fullsized":"2024-05-09T04:51:05.548481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:00.634983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.891836+0000","last_change":"2024-05-09T04:49:29.402935+0000","last_active":"2024-05-09T04:51:05.891836+0000","last_peered":"2024-05-09T04:51:05.891836+0000","last_clean":"2024-05-09T04:51:05.891836+0000","last_became_active":"2024-05-09T04:49:29.400084+0000","last_became_peered":"2024-05-09T04:49:29.400084+0000","last_unstale":"2024-05-09T04:51:05.891836+0000","last_undegraded":"2024-05-09T04:51:05.891836+0000","last_fullsized":"2024-05-09T04:51:05.891836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:06:06.683122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.659117+0000","last_change":"2024-05-09T04:48:28.564243+0000","last_active":"2024-05-09T04:51:05.659117+0000","last_peered":"2024-05-09T04:51:05.659117+0000","last_clean":"2024-05-09T04:51:05.659117+0000","last_became_active":"2024-05-09T04:48:28.563732+0000","last_became_peered":"2024-05-09T04:48:28.563732+0000","last_unstale":"2024-05-09T04:51:05.659117+0000","last_undegraded":"2024-05-09T04:51:05.659117+0000","last_fullsized":"2024-05-09T04:51:05.659117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:26.982339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'8","reported_seq":221,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810097+0000","last_change":"2024-05-09T04:47:45.389868+0000","last_active":"2024-05-09T04:51:05.810097+0000","last_peered":"2024-05-09T04:51:05.810097+0000","last_clean":"2024-05-09T04:51:05.810097+0000","last_became_active":"2024-05-09T04:47:45.389508+0000","last_became_peered":"2024-05-09T04:47:45.389508+0000","last_unstale":"2024-05-09T04:51:05.810097+0000","last_undegraded":"2024-05-09T04:51:05.810097+0000","last_fullsized":"2024-05-09T04:51:05.810097+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:45.050187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1712,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":181,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651364+0000","last_change":"2024-05-09T04:47:48.404457+0000","last_active":"2024-05-09T04:51:05.651364+0000","last_peered":"2024-05-09T04:51:05.651364+0000","last_clean":"2024-05-09T04:51:05.651364+0000","last_became_active":"2024-05-09T04:47:48.403968+0000","last_became_peered":"2024-05-09T04:47:48.403968+0000","last_unstale":"2024-05-09T04:51:05.651364+0000","last_undegraded":"2024-05-09T04:51:05.651364+0000","last_fullsized":"2024-05-09T04:51:05.651364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:23:40.364916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.661879+0000","last_change":"2024-05-09T04:47:45.388183+0000","last_active":"2024-05-09T04:51:05.661879+0000","last_peered":"2024-05-09T04:51:05.661879+0000","last_clean":"2024-05-09T04:51:05.661879+0000","last_became_active":"2024-05-09T04:47:45.386603+0000","last_became_peered":"2024-05-09T04:47:45.386603+0000","last_unstale":"2024-05-09T04:51:05.661879+0000","last_undegraded":"2024-05-09T04:51:05.661879+0000","last_fullsized":"2024-05-09T04:51:05.661879+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:53.433619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.649200+0000","last_change":"2024-05-09T04:48:28.663120+0000","last_active":"2024-05-09T04:51:05.649200+0000","last_peered":"2024-05-09T04:51:05.649200+0000","last_clean":"2024-05-09T04:51:05.649200+0000","last_became_active":"2024-05-09T04:48:28.662748+0000","last_became_peered":"2024-05-09T04:48:28.662748+0000","last_unstale":"2024-05-09T04:51:05.649200+0000","last_undegraded":"2024-05-09T04:51:05.649200+0000","last_fullsized":"2024-05-09T04:51:05.649200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:57.677876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544377+0000","last_change":"2024-05-09T04:47:45.384192+0000","last_active":"2024-05-09T04:51:05.544377+0000","last_peered":"2024-05-09T04:51:05.544377+0000","last_clean":"2024-05-09T04:51:05.544377+0000","last_became_active":"2024-05-09T04:47:45.383843+0000","last_became_peered":"2024-05-09T04:47:45.383843+0000","last_unstale":"2024-05-09T04:51:05.544377+0000","last_undegraded":"2024-05-09T04:51:05.544377+0000","last_fullsized":"2024-05-09T04:51:05.544377+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:28.815207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651282+0000","last_change":"2024-05-09T04:47:48.409203+0000","last_active":"2024-05-09T04:51:05.651282+0000","last_peered":"2024-05-09T04:51:05.651282+0000","last_clean":"2024-05-09T04:51:05.651282+0000","last_became_active":"2024-05-09T04:47:48.408861+0000","last_became_peered":"2024-05-09T04:47:48.408861+0000","last_unstale":"2024-05-09T04:51:05.651282+0000","last_undegraded":"2024-05-09T04:51:05.651282+0000","last_fullsized":"2024-05-09T04:51:05.651282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:07.160397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.658840+0000","last_change":"2024-05-09T04:47:45.392723+0000","last_active":"2024-05-09T04:51:05.658840+0000","last_peered":"2024-05-09T04:51:05.658840+0000","last_clean":"2024-05-09T04:51:05.658840+0000","last_became_active":"2024-05-09T04:47:45.392313+0000","last_became_peered":"2024-05-09T04:47:45.392313+0000","last_unstale":"2024-05-09T04:51:05.658840+0000","last_undegraded":"2024-05-09T04:51:05.658840+0000","last_fullsized":"2024-05-09T04:51:05.658840+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:44.358702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.648930+0000","last_change":"2024-05-09T04:47:48.414000+0000","last_active":"2024-05-09T04:51:05.648930+0000","last_peered":"2024-05-09T04:51:05.648930+0000","last_clean":"2024-05-09T04:51:05.648930+0000","last_became_active":"2024-05-09T04:47:48.413532+0000","last_became_peered":"2024-05-09T04:47:48.413532+0000","last_unstale":"2024-05-09T04:51:05.648930+0000","last_undegraded":"2024-05-09T04:51:05.648930+0000","last_fullsized":"2024-05-09T04:51:05.648930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:22.774814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":214,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.662399+0000","last_change":"2024-05-09T04:47:45.388661+0000","last_active":"2024-05-09T04:51:05.662399+0000","last_peered":"2024-05-09T04:51:05.662399+0000","last_clean":"2024-05-09T04:51:05.662399+0000","last_became_active":"2024-05-09T04:47:45.386010+0000","last_became_peered":"2024-05-09T04:47:45.386010+0000","last_unstale":"2024-05-09T04:51:05.662399+0000","last_undegraded":"2024-05-09T04:51:05.662399+0000","last_fullsized":"2024-05-09T04:51:05.662399+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:23.250463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":178,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548947+0000","last_change":"2024-05-09T04:47:48.421706+0000","last_active":"2024-05-09T04:51:05.548947+0000","last_peered":"2024-05-09T04:51:05.548947+0000","last_clean":"2024-05-09T04:51:05.548947+0000","last_became_active":"2024-05-09T04:47:48.420529+0000","last_became_peered":"2024-05-09T04:47:48.420529+0000","last_unstale":"2024-05-09T04:51:05.548947+0000","last_undegraded":"2024-05-09T04:51:05.548947+0000","last_fullsized":"2024-05-09T04:51:05.548947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:10.110366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810291+0000","last_change":"2024-05-09T04:47:45.386625+0000","last_active":"2024-05-09T04:51:05.810291+0000","last_peered":"2024-05-09T04:51:05.810291+0000","last_clean":"2024-05-09T04:51:05.810291+0000","last_became_active":"2024-05-09T04:47:45.385087+0000","last_became_peered":"2024-05-09T04:47:45.385087+0000","last_unstale":"2024-05-09T04:51:05.810291+0000","last_undegraded":"2024-05-09T04:51:05.810291+0000","last_fullsized":"2024-05-09T04:51:05.810291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:15.546894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814698+0000","last_change":"2024-05-09T04:48:28.562517+0000","last_active":"2024-05-09T04:51:05.814698+0000","last_peered":"2024-05-09T04:51:05.814698+0000","last_clean":"2024-05-09T04:51:05.814698+0000","last_became_active":"2024-05-09T04:48:28.562047+0000","last_became_peered":"2024-05-09T04:48:28.562047+0000","last_unstale":"2024-05-09T04:51:05.814698+0000","last_undegraded":"2024-05-09T04:51:05.814698+0000","last_fullsized":"2024-05-09T04:51:05.814698+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:22.769844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.659142+0000","last_change":"2024-05-09T04:47:45.613713+0000","last_active":"2024-05-09T04:51:05.659142+0000","last_peered":"2024-05-09T04:51:05.659142+0000","last_clean":"2024-05-09T04:51:05.659142+0000","last_became_active":"2024-05-09T04:47:45.613313+0000","last_became_peered":"2024-05-09T04:47:45.613313+0000","last_unstale":"2024-05-09T04:51:05.659142+0000","last_undegraded":"2024-05-09T04:51:05.659142+0000","last_fullsized":"2024-05-09T04:51:05.659142+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:32:44.825695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548722+0000","last_change":"2024-05-09T04:47:48.411234+0000","last_active":"2024-05-09T04:51:05.548722+0000","last_peered":"2024-05-09T04:51:05.548722+0000","last_clean":"2024-05-09T04:51:05.548722+0000","last_became_active":"2024-05-09T04:47:48.410881+0000","last_became_peered":"2024-05-09T04:47:48.410881+0000","last_unstale":"2024-05-09T04:51:05.548722+0000","last_undegraded":"2024-05-09T04:51:05.548722+0000","last_fullsized":"2024-05-09T04:51:05.548722+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.747871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'36","reported_seq":453,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.816697+0000","last_change":"2024-05-09T04:48:28.570426+0000","last_active":"2024-05-09T04:51:05.816697+0000","last_peered":"2024-05-09T04:51:05.816697+0000","last_clean":"2024-05-09T04:51:05.816697+0000","last_became_active":"2024-05-09T04:48:28.561786+0000","last_became_peered":"2024-05-09T04:48:28.561786+0000","last_unstale":"2024-05-09T04:51:05.816697+0000","last_undegraded":"2024-05-09T04:51:05.816697+0000","last_fullsized":"2024-05-09T04:51:05.816697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:32.367142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67059,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":362,"num_read_kb":232,"num_write":37,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":50695,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651672+0000","last_change":"2024-05-09T04:47:48.406510+0000","last_active":"2024-05-09T04:51:05.651672+0000","last_peered":"2024-05-09T04:51:05.651672+0000","last_clean":"2024-05-09T04:51:05.651672+0000","last_became_active":"2024-05-09T04:47:48.406166+0000","last_became_peered":"2024-05-09T04:47:48.406166+0000","last_unstale":"2024-05-09T04:51:05.651672+0000","last_undegraded":"2024-05-09T04:51:05.651672+0000","last_fullsized":"2024-05-09T04:51:05.651672+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:18.579620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545845+0000","last_change":"2024-05-09T04:47:45.387164+0000","last_active":"2024-05-09T04:51:05.545845+0000","last_peered":"2024-05-09T04:51:05.545845+0000","last_clean":"2024-05-09T04:51:05.545845+0000","last_became_active":"2024-05-09T04:47:45.386755+0000","last_became_peered":"2024-05-09T04:47:45.386755+0000","last_unstale":"2024-05-09T04:51:05.545845+0000","last_undegraded":"2024-05-09T04:51:05.545845+0000","last_fullsized":"2024-05-09T04:51:05.545845+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:07.838782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544274+0000","last_change":"2024-05-09T04:47:48.405436+0000","last_active":"2024-05-09T04:51:05.544274+0000","last_peered":"2024-05-09T04:51:05.544274+0000","last_clean":"2024-05-09T04:51:05.544274+0000","last_became_active":"2024-05-09T04:47:48.405032+0000","last_became_peered":"2024-05-09T04:47:48.405032+0000","last_unstale":"2024-05-09T04:51:05.544274+0000","last_undegraded":"2024-05-09T04:51:05.544274+0000","last_fullsized":"2024-05-09T04:51:05.544274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:27.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.658921+0000","last_change":"2024-05-09T04:47:45.391970+0000","last_active":"2024-05-09T04:51:05.658921+0000","last_peered":"2024-05-09T04:51:05.658921+0000","last_clean":"2024-05-09T04:51:05.658921+0000","last_became_active":"2024-05-09T04:47:45.391597+0000","last_became_peered":"2024-05-09T04:47:45.391597+0000","last_unstale":"2024-05-09T04:51:05.658921+0000","last_undegraded":"2024-05-09T04:51:05.658921+0000","last_fullsized":"2024-05-09T04:51:05.658921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:27.076631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810305+0000","last_change":"2024-05-09T04:47:48.419328+0000","last_active":"2024-05-09T04:51:05.810305+0000","last_peered":"2024-05-09T04:51:05.810305+0000","last_clean":"2024-05-09T04:51:05.810305+0000","last_became_active":"2024-05-09T04:47:48.418972+0000","last_became_peered":"2024-05-09T04:47:48.418972+0000","last_unstale":"2024-05-09T04:51:05.810305+0000","last_undegraded":"2024-05-09T04:51:05.810305+0000","last_fullsized":"2024-05-09T04:51:05.810305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:15.164747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":221,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.805695+0000","last_change":"2024-05-09T04:47:45.386173+0000","last_active":"2024-05-09T04:51:05.805695+0000","last_peered":"2024-05-09T04:51:05.805695+0000","last_clean":"2024-05-09T04:51:05.805695+0000","last_became_active":"2024-05-09T04:47:45.384296+0000","last_became_peered":"2024-05-09T04:47:45.384296+0000","last_unstale":"2024-05-09T04:51:05.805695+0000","last_undegraded":"2024-05-09T04:51:05.805695+0000","last_fullsized":"2024-05-09T04:51:05.805695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:03.246187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":183,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810042+0000","last_change":"2024-05-09T04:47:48.415645+0000","last_active":"2024-05-09T04:51:05.810042+0000","last_peered":"2024-05-09T04:51:05.810042+0000","last_clean":"2024-05-09T04:51:05.810042+0000","last_became_active":"2024-05-09T04:47:48.415303+0000","last_became_peered":"2024-05-09T04:47:48.415303+0000","last_unstale":"2024-05-09T04:51:05.810042+0000","last_undegraded":"2024-05-09T04:51:05.810042+0000","last_fullsized":"2024-05-09T04:51:05.810042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:39.703948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651268+0000","last_change":"2024-05-09T04:47:45.613320+0000","last_active":"2024-05-09T04:51:05.651268+0000","last_peered":"2024-05-09T04:51:05.651268+0000","last_clean":"2024-05-09T04:51:05.651268+0000","last_became_active":"2024-05-09T04:47:45.612871+0000","last_became_peered":"2024-05-09T04:47:45.612871+0000","last_unstale":"2024-05-09T04:51:05.651268+0000","last_undegraded":"2024-05-09T04:51:05.651268+0000","last_fullsized":"2024-05-09T04:51:05.651268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:31.516652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.648939+0000","last_change":"2024-05-09T04:48:28.563376+0000","last_active":"2024-05-09T04:51:05.648939+0000","last_peered":"2024-05-09T04:51:05.648939+0000","last_clean":"2024-05-09T04:51:05.648939+0000","last_became_active":"2024-05-09T04:48:28.562973+0000","last_became_peered":"2024-05-09T04:48:28.562973+0000","last_unstale":"2024-05-09T04:51:05.648939+0000","last_undegraded":"2024-05-09T04:51:05.648939+0000","last_fullsized":"2024-05-09T04:51:05.648939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:20.068517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651691+0000","last_change":"2024-05-09T04:47:45.389785+0000","last_active":"2024-05-09T04:51:05.651691+0000","last_peered":"2024-05-09T04:51:05.651691+0000","last_clean":"2024-05-09T04:51:05.651691+0000","last_became_active":"2024-05-09T04:47:45.389159+0000","last_became_peered":"2024-05-09T04:47:45.389159+0000","last_unstale":"2024-05-09T04:51:05.651691+0000","last_undegraded":"2024-05-09T04:51:05.651691+0000","last_fullsized":"2024-05-09T04:51:05.651691+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:28:51.804123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.892023+0000","last_change":"2024-05-09T04:48:28.563170+0000","last_active":"2024-05-09T04:51:05.892023+0000","last_peered":"2024-05-09T04:51:05.892023+0000","last_clean":"2024-05-09T04:51:05.892023+0000","last_became_active":"2024-05-09T04:48:28.562189+0000","last_became_peered":"2024-05-09T04:48:28.562189+0000","last_unstale":"2024-05-09T04:51:05.892023+0000","last_undegraded":"2024-05-09T04:51:05.892023+0000","last_fullsized":"2024-05-09T04:51:05.892023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:00.643292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.648790+0000","last_change":"2024-05-09T04:47:45.613739+0000","last_active":"2024-05-09T04:51:05.648790+0000","last_peered":"2024-05-09T04:51:05.648790+0000","last_clean":"2024-05-09T04:51:05.648790+0000","last_became_active":"2024-05-09T04:47:45.613365+0000","last_became_peered":"2024-05-09T04:47:45.613365+0000","last_unstale":"2024-05-09T04:51:05.648790+0000","last_undegraded":"2024-05-09T04:51:05.648790+0000","last_fullsized":"2024-05-09T04:51:05.648790+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:57.138068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.662064+0000","last_change":"2024-05-09T04:47:48.413343+0000","last_active":"2024-05-09T04:51:05.662064+0000","last_peered":"2024-05-09T04:51:05.662064+0000","last_clean":"2024-05-09T04:51:05.662064+0000","last_became_active":"2024-05-09T04:47:48.412941+0000","last_became_peered":"2024-05-09T04:47:48.412941+0000","last_unstale":"2024-05-09T04:51:05.662064+0000","last_undegraded":"2024-05-09T04:51:05.662064+0000","last_fullsized":"2024-05-09T04:51:05.662064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:28.764553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810369+0000","last_change":"2024-05-09T04:47:45.386487+0000","last_active":"2024-05-09T04:51:05.810369+0000","last_peered":"2024-05-09T04:51:05.810369+0000","last_clean":"2024-05-09T04:51:05.810369+0000","last_became_active":"2024-05-09T04:47:45.386078+0000","last_became_peered":"2024-05-09T04:47:45.386078+0000","last_unstale":"2024-05-09T04:51:05.810369+0000","last_undegraded":"2024-05-09T04:51:05.810369+0000","last_fullsized":"2024-05-09T04:51:05.810369+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:19.998977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.661951+0000","last_change":"2024-05-09T04:47:48.418955+0000","last_active":"2024-05-09T04:51:05.661951+0000","last_peered":"2024-05-09T04:51:05.661951+0000","last_clean":"2024-05-09T04:51:05.661951+0000","last_became_active":"2024-05-09T04:47:48.418528+0000","last_became_peered":"2024-05-09T04:47:48.418528+0000","last_unstale":"2024-05-09T04:51:05.661951+0000","last_undegraded":"2024-05-09T04:51:05.661951+0000","last_fullsized":"2024-05-09T04:51:05.661951+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:51:02.386342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.659089+0000","last_change":"2024-05-09T04:47:48.413304+0000","last_active":"2024-05-09T04:51:05.659089+0000","last_peered":"2024-05-09T04:51:05.659089+0000","last_clean":"2024-05-09T04:51:05.659089+0000","last_became_active":"2024-05-09T04:47:48.412582+0000","last_became_peered":"2024-05-09T04:47:48.412582+0000","last_unstale":"2024-05-09T04:51:05.659089+0000","last_undegraded":"2024-05-09T04:51:05.659089+0000","last_fullsized":"2024-05-09T04:51:05.659089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:40.027082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":217,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.803387+0000","last_change":"2024-05-09T04:47:45.612742+0000","last_active":"2024-05-09T04:51:05.803387+0000","last_peered":"2024-05-09T04:51:05.803387+0000","last_clean":"2024-05-09T04:51:05.803387+0000","last_became_active":"2024-05-09T04:47:45.612325+0000","last_became_peered":"2024-05-09T04:47:45.612325+0000","last_unstale":"2024-05-09T04:51:05.803387+0000","last_undegraded":"2024-05-09T04:51:05.803387+0000","last_fullsized":"2024-05-09T04:51:05.803387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:00.573996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":178,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651673+0000","last_change":"2024-05-09T04:47:48.412894+0000","last_active":"2024-05-09T04:51:05.651673+0000","last_peered":"2024-05-09T04:51:05.651673+0000","last_clean":"2024-05-09T04:51:05.651673+0000","last_became_active":"2024-05-09T04:47:48.412265+0000","last_became_peered":"2024-05-09T04:47:48.412265+0000","last_unstale":"2024-05-09T04:51:05.651673+0000","last_undegraded":"2024-05-09T04:51:05.651673+0000","last_fullsized":"2024-05-09T04:51:05.651673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:23.517566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.805325+0000","last_change":"2024-05-09T04:47:45.387715+0000","last_active":"2024-05-09T04:51:05.805325+0000","last_peered":"2024-05-09T04:51:05.805325+0000","last_clean":"2024-05-09T04:51:05.805325+0000","last_became_active":"2024-05-09T04:47:45.387229+0000","last_became_peered":"2024-05-09T04:47:45.387229+0000","last_unstale":"2024-05-09T04:51:05.805325+0000","last_undegraded":"2024-05-09T04:51:05.805325+0000","last_fullsized":"2024-05-09T04:51:05.805325+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:22.540214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":107,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810292+0000","last_change":"2024-05-09T04:46:21.632579+0000","last_active":"2024-05-09T04:51:05.810292+0000","last_peered":"2024-05-09T04:51:05.810292+0000","last_clean":"2024-05-09T04:51:05.810292+0000","last_became_active":"2024-05-09T04:46:21.627483+0000","last_became_peered":"2024-05-09T04:46:21.627483+0000","last_unstale":"2024-05-09T04:51:05.810292+0000","last_undegraded":"2024-05-09T04:51:05.810292+0000","last_fullsized":"2024-05-09T04:51:05.810292+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:42:54.179739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:42:54.179739+0000","last_clean_scrub_stamp":"2024-05-09T04:42:54.179739+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:44.331526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544198+0000","last_change":"2024-05-09T04:47:48.418570+0000","last_active":"2024-05-09T04:51:05.544198+0000","last_peered":"2024-05-09T04:51:05.544198+0000","last_clean":"2024-05-09T04:51:05.544198+0000","last_became_active":"2024-05-09T04:47:48.418270+0000","last_became_peered":"2024-05-09T04:47:48.418270+0000","last_unstale":"2024-05-09T04:51:05.544198+0000","last_undegraded":"2024-05-09T04:51:05.544198+0000","last_fullsized":"2024-05-09T04:51:05.544198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:28.406846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810387+0000","last_change":"2024-05-09T04:47:45.387161+0000","last_active":"2024-05-09T04:51:05.810387+0000","last_peered":"2024-05-09T04:51:05.810387+0000","last_clean":"2024-05-09T04:51:05.810387+0000","last_became_active":"2024-05-09T04:47:45.386753+0000","last_became_peered":"2024-05-09T04:47:45.386753+0000","last_unstale":"2024-05-09T04:51:05.810387+0000","last_undegraded":"2024-05-09T04:51:05.810387+0000","last_fullsized":"2024-05-09T04:51:05.810387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:15.402418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651727+0000","last_change":"2024-05-09T04:47:45.613551+0000","last_active":"2024-05-09T04:51:05.651727+0000","last_peered":"2024-05-09T04:51:05.651727+0000","last_clean":"2024-05-09T04:51:05.651727+0000","last_became_active":"2024-05-09T04:47:45.613086+0000","last_became_peered":"2024-05-09T04:47:45.613086+0000","last_unstale":"2024-05-09T04:51:05.651727+0000","last_undegraded":"2024-05-09T04:51:05.651727+0000","last_fullsized":"2024-05-09T04:51:05.651727+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:04.350960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544530+0000","last_change":"2024-05-09T04:47:48.403701+0000","last_active":"2024-05-09T04:51:05.544530+0000","last_peered":"2024-05-09T04:51:05.544530+0000","last_clean":"2024-05-09T04:51:05.544530+0000","last_became_active":"2024-05-09T04:47:48.403337+0000","last_became_peered":"2024-05-09T04:47:48.403337+0000","last_unstale":"2024-05-09T04:51:05.544530+0000","last_undegraded":"2024-05-09T04:51:05.544530+0000","last_fullsized":"2024-05-09T04:51:05.544530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:53.032348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'47","reported_seq":102,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.771184+0000","last_change":"2024-05-09T04:47:45.385935+0000","last_active":"2024-05-09T04:51:05.771184+0000","last_peered":"2024-05-09T04:51:05.771184+0000","last_clean":"2024-05-09T04:51:05.771184+0000","last_became_active":"2024-05-09T04:47:45.385533+0000","last_became_peered":"2024-05-09T04:47:45.385533+0000","last_unstale":"2024-05-09T04:51:05.771184+0000","last_undegraded":"2024-05-09T04:51:05.771184+0000","last_fullsized":"2024-05-09T04:51:05.771184+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:34:38.014201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":85,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.652135+0000","last_change":"2024-05-09T04:47:48.409590+0000","last_active":"2024-05-09T04:51:05.652135+0000","last_peered":"2024-05-09T04:51:05.652135+0000","last_clean":"2024-05-09T04:51:05.652135+0000","last_became_active":"2024-05-09T04:47:48.409191+0000","last_became_peered":"2024-05-09T04:47:48.409191+0000","last_unstale":"2024-05-09T04:51:05.652135+0000","last_undegraded":"2024-05-09T04:51:05.652135+0000","last_fullsized":"2024-05-09T04:51:05.652135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:50.440609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.805557+0000","last_change":"2024-05-09T04:47:45.389147+0000","last_active":"2024-05-09T04:51:05.805557+0000","last_peered":"2024-05-09T04:51:05.805557+0000","last_clean":"2024-05-09T04:51:05.805557+0000","last_became_active":"2024-05-09T04:47:45.388797+0000","last_became_peered":"2024-05-09T04:47:45.388797+0000","last_unstale":"2024-05-09T04:51:05.805557+0000","last_undegraded":"2024-05-09T04:51:05.805557+0000","last_fullsized":"2024-05-09T04:51:05.805557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:19.265278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814554+0000","last_change":"2024-05-09T04:47:48.419100+0000","last_active":"2024-05-09T04:51:05.814554+0000","last_peered":"2024-05-09T04:51:05.814554+0000","last_clean":"2024-05-09T04:51:05.814554+0000","last_became_active":"2024-05-09T04:47:48.418499+0000","last_became_peered":"2024-05-09T04:47:48.418499+0000","last_unstale":"2024-05-09T04:51:05.814554+0000","last_undegraded":"2024-05-09T04:51:05.814554+0000","last_fullsized":"2024-05-09T04:51:05.814554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:23.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548394+0000","last_change":"2024-05-09T04:47:45.613269+0000","last_active":"2024-05-09T04:51:05.548394+0000","last_peered":"2024-05-09T04:51:05.548394+0000","last_clean":"2024-05-09T04:51:05.548394+0000","last_became_active":"2024-05-09T04:47:45.612895+0000","last_became_peered":"2024-05-09T04:47:45.612895+0000","last_unstale":"2024-05-09T04:51:05.548394+0000","last_undegraded":"2024-05-09T04:51:05.548394+0000","last_fullsized":"2024-05-09T04:51:05.548394+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:52.966974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651326+0000","last_change":"2024-05-09T04:47:48.416658+0000","last_active":"2024-05-09T04:51:05.651326+0000","last_peered":"2024-05-09T04:51:05.651326+0000","last_clean":"2024-05-09T04:51:05.651326+0000","last_became_active":"2024-05-09T04:47:48.416238+0000","last_became_peered":"2024-05-09T04:47:48.416238+0000","last_unstale":"2024-05-09T04:51:05.651326+0000","last_undegraded":"2024-05-09T04:51:05.651326+0000","last_fullsized":"2024-05-09T04:51:05.651326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:35:50.126723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651564+0000","last_change":"2024-05-09T04:47:45.384018+0000","last_active":"2024-05-09T04:51:05.651564+0000","last_peered":"2024-05-09T04:51:05.651564+0000","last_clean":"2024-05-09T04:51:05.651564+0000","last_became_active":"2024-05-09T04:47:45.383647+0000","last_became_peered":"2024-05-09T04:47:45.383647+0000","last_unstale":"2024-05-09T04:51:05.651564+0000","last_undegraded":"2024-05-09T04:51:05.651564+0000","last_fullsized":"2024-05-09T04:51:05.651564+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:06.431233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.891798+0000","last_change":"2024-05-09T04:47:48.409089+0000","last_active":"2024-05-09T04:51:05.891798+0000","last_peered":"2024-05-09T04:51:05.891798+0000","last_clean":"2024-05-09T04:51:05.891798+0000","last_became_active":"2024-05-09T04:47:48.408548+0000","last_became_peered":"2024-05-09T04:47:48.408548+0000","last_unstale":"2024-05-09T04:51:05.891798+0000","last_undegraded":"2024-05-09T04:51:05.891798+0000","last_fullsized":"2024-05-09T04:51:05.891798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:13.633008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545683+0000","last_change":"2024-05-09T04:47:45.387580+0000","last_active":"2024-05-09T04:51:05.545683+0000","last_peered":"2024-05-09T04:51:05.545683+0000","last_clean":"2024-05-09T04:51:05.545683+0000","last_became_active":"2024-05-09T04:47:45.386929+0000","last_became_peered":"2024-05-09T04:47:45.386929+0000","last_unstale":"2024-05-09T04:51:05.545683+0000","last_undegraded":"2024-05-09T04:51:05.545683+0000","last_fullsized":"2024-05-09T04:51:05.545683+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:19.661763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651461+0000","last_change":"2024-05-09T04:47:48.416311+0000","last_active":"2024-05-09T04:51:05.651461+0000","last_peered":"2024-05-09T04:51:05.651461+0000","last_clean":"2024-05-09T04:51:05.651461+0000","last_became_active":"2024-05-09T04:47:48.415969+0000","last_became_peered":"2024-05-09T04:47:48.415969+0000","last_unstale":"2024-05-09T04:51:05.651461+0000","last_undegraded":"2024-05-09T04:51:05.651461+0000","last_fullsized":"2024-05-09T04:51:05.651461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:55.751210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651448+0000","last_change":"2024-05-09T04:48:28.573916+0000","last_active":"2024-05-09T04:51:05.651448+0000","last_peered":"2024-05-09T04:51:05.651448+0000","last_clean":"2024-05-09T04:51:05.651448+0000","last_became_active":"2024-05-09T04:48:28.562872+0000","last_became_peered":"2024-05-09T04:48:28.562872+0000","last_unstale":"2024-05-09T04:51:05.651448+0000","last_undegraded":"2024-05-09T04:51:05.651448+0000","last_fullsized":"2024-05-09T04:51:05.651448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:02.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810462+0000","last_change":"2024-05-09T04:47:48.408364+0000","last_active":"2024-05-09T04:51:05.810462+0000","last_peered":"2024-05-09T04:51:05.810462+0000","last_clean":"2024-05-09T04:51:05.810462+0000","last_became_active":"2024-05-09T04:47:48.407982+0000","last_became_peered":"2024-05-09T04:47:48.407982+0000","last_unstale":"2024-05-09T04:51:05.810462+0000","last_undegraded":"2024-05-09T04:51:05.810462+0000","last_fullsized":"2024-05-09T04:51:05.810462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:14.103171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545878+0000","last_change":"2024-05-09T04:47:45.614637+0000","last_active":"2024-05-09T04:51:05.545878+0000","last_peered":"2024-05-09T04:51:05.545878+0000","last_clean":"2024-05-09T04:51:05.545878+0000","last_became_active":"2024-05-09T04:47:45.613290+0000","last_became_peered":"2024-05-09T04:47:45.613290+0000","last_unstale":"2024-05-09T04:51:05.545878+0000","last_undegraded":"2024-05-09T04:51:05.545878+0000","last_fullsized":"2024-05-09T04:51:05.545878+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:48.547382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810433+0000","last_change":"2024-05-09T04:47:48.405537+0000","last_active":"2024-05-09T04:51:05.810433+0000","last_peered":"2024-05-09T04:51:05.810433+0000","last_clean":"2024-05-09T04:51:05.810433+0000","last_became_active":"2024-05-09T04:47:48.405148+0000","last_became_peered":"2024-05-09T04:47:48.405148+0000","last_unstale":"2024-05-09T04:51:05.810433+0000","last_undegraded":"2024-05-09T04:51:05.810433+0000","last_fullsized":"2024-05-09T04:51:05.810433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:46.981401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'39","reported_seq":79,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.768865+0000","last_change":"2024-05-09T04:47:45.389151+0000","last_active":"2024-05-09T04:51:05.768865+0000","last_peered":"2024-05-09T04:51:05.768865+0000","last_clean":"2024-05-09T04:51:05.768865+0000","last_became_active":"2024-05-09T04:47:45.388735+0000","last_became_peered":"2024-05-09T04:47:45.388735+0000","last_unstale":"2024-05-09T04:51:05.768865+0000","last_undegraded":"2024-05-09T04:51:05.768865+0000","last_fullsized":"2024-05-09T04:51:05.768865+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:26.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810206+0000","last_change":"2024-05-09T04:47:48.413881+0000","last_active":"2024-05-09T04:51:05.810206+0000","last_peered":"2024-05-09T04:51:05.810206+0000","last_clean":"2024-05-09T04:51:05.810206+0000","last_became_active":"2024-05-09T04:47:48.413559+0000","last_became_peered":"2024-05-09T04:47:48.413559+0000","last_unstale":"2024-05-09T04:51:05.810206+0000","last_undegraded":"2024-05-09T04:51:05.810206+0000","last_fullsized":"2024-05-09T04:51:05.810206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:40:40.305319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544766+0000","last_change":"2024-05-09T04:47:45.383406+0000","last_active":"2024-05-09T04:51:05.544766+0000","last_peered":"2024-05-09T04:51:05.544766+0000","last_clean":"2024-05-09T04:51:05.544766+0000","last_became_active":"2024-05-09T04:47:45.383025+0000","last_became_peered":"2024-05-09T04:47:45.383025+0000","last_unstale":"2024-05-09T04:51:05.544766+0000","last_undegraded":"2024-05-09T04:51:05.544766+0000","last_fullsized":"2024-05-09T04:51:05.544766+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:46.007454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810679+0000","last_change":"2024-05-09T04:47:48.412481+0000","last_active":"2024-05-09T04:51:05.810679+0000","last_peered":"2024-05-09T04:51:05.810679+0000","last_clean":"2024-05-09T04:51:05.810679+0000","last_became_active":"2024-05-09T04:47:48.412034+0000","last_became_peered":"2024-05-09T04:47:48.412034+0000","last_unstale":"2024-05-09T04:51:05.810679+0000","last_undegraded":"2024-05-09T04:51:05.810679+0000","last_fullsized":"2024-05-09T04:51:05.810679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:14.030981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.891860+0000","last_change":"2024-05-09T04:47:48.406958+0000","last_active":"2024-05-09T04:51:05.891860+0000","last_peered":"2024-05-09T04:51:05.891860+0000","last_clean":"2024-05-09T04:51:05.891860+0000","last_became_active":"2024-05-09T04:47:48.406575+0000","last_became_peered":"2024-05-09T04:47:48.406575+0000","last_unstale":"2024-05-09T04:51:05.891860+0000","last_undegraded":"2024-05-09T04:51:05.891860+0000","last_fullsized":"2024-05-09T04:51:05.891860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:23.389676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545618+0000","last_change":"2024-05-09T04:48:28.562376+0000","last_active":"2024-05-09T04:51:05.545618+0000","last_peered":"2024-05-09T04:51:05.545618+0000","last_clean":"2024-05-09T04:51:05.545618+0000","last_became_active":"2024-05-09T04:48:28.561866+0000","last_became_peered":"2024-05-09T04:48:28.561866+0000","last_unstale":"2024-05-09T04:51:05.545618+0000","last_undegraded":"2024-05-09T04:51:05.545618+0000","last_fullsized":"2024-05-09T04:51:05.545618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:51.974146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545565+0000","last_change":"2024-05-09T04:47:48.413908+0000","last_active":"2024-05-09T04:51:05.545565+0000","last_peered":"2024-05-09T04:51:05.545565+0000","last_clean":"2024-05-09T04:51:05.545565+0000","last_became_active":"2024-05-09T04:47:48.413575+0000","last_became_peered":"2024-05-09T04:47:48.413575+0000","last_unstale":"2024-05-09T04:51:05.545565+0000","last_undegraded":"2024-05-09T04:51:05.545565+0000","last_fullsized":"2024-05-09T04:51:05.545565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:40.742510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814127+0000","last_change":"2024-05-09T04:47:48.406607+0000","last_active":"2024-05-09T04:51:05.814127+0000","last_peered":"2024-05-09T04:51:05.814127+0000","last_clean":"2024-05-09T04:51:05.814127+0000","last_became_active":"2024-05-09T04:47:48.406290+0000","last_became_peered":"2024-05-09T04:47:48.406290+0000","last_unstale":"2024-05-09T04:51:05.814127+0000","last_undegraded":"2024-05-09T04:51:05.814127+0000","last_fullsized":"2024-05-09T04:51:05.814127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:22.107953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544591+0000","last_change":"2024-05-09T04:47:48.415153+0000","last_active":"2024-05-09T04:51:05.544591+0000","last_peered":"2024-05-09T04:51:05.544591+0000","last_clean":"2024-05-09T04:51:05.544591+0000","last_became_active":"2024-05-09T04:47:48.414793+0000","last_became_peered":"2024-05-09T04:47:48.414793+0000","last_unstale":"2024-05-09T04:51:05.544591+0000","last_undegraded":"2024-05-09T04:51:05.544591+0000","last_fullsized":"2024-05-09T04:51:05.544591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:57.564091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.892197+0000","last_change":"2024-05-09T04:47:48.408646+0000","last_active":"2024-05-09T04:51:05.892197+0000","last_peered":"2024-05-09T04:51:05.892197+0000","last_clean":"2024-05-09T04:51:05.892197+0000","last_became_active":"2024-05-09T04:47:48.408250+0000","last_became_peered":"2024-05-09T04:47:48.408250+0000","last_unstale":"2024-05-09T04:51:05.892197+0000","last_undegraded":"2024-05-09T04:51:05.892197+0000","last_fullsized":"2024-05-09T04:51:05.892197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:42:50.622937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.546131+0000","last_change":"2024-05-09T04:47:48.408686+0000","last_active":"2024-05-09T04:51:05.546131+0000","last_peered":"2024-05-09T04:51:05.546131+0000","last_clean":"2024-05-09T04:51:05.546131+0000","last_became_active":"2024-05-09T04:47:48.408318+0000","last_became_peered":"2024-05-09T04:47:48.408318+0000","last_unstale":"2024-05-09T04:51:05.546131+0000","last_undegraded":"2024-05-09T04:51:05.546131+0000","last_fullsized":"2024-05-09T04:51:05.546131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:51.240485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.652134+0000","last_change":"2024-05-09T04:47:48.421562+0000","last_active":"2024-05-09T04:51:05.652134+0000","last_peered":"2024-05-09T04:51:05.652134+0000","last_clean":"2024-05-09T04:51:05.652134+0000","last_became_active":"2024-05-09T04:47:48.421068+0000","last_became_peered":"2024-05-09T04:47:48.421068+0000","last_unstale":"2024-05-09T04:51:05.652134+0000","last_undegraded":"2024-05-09T04:51:05.652134+0000","last_fullsized":"2024-05-09T04:51:05.652134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:18:41.455187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.658896+0000","last_change":"2024-05-09T04:47:48.413942+0000","last_active":"2024-05-09T04:51:05.658896+0000","last_peered":"2024-05-09T04:51:05.658896+0000","last_clean":"2024-05-09T04:51:05.658896+0000","last_became_active":"2024-05-09T04:47:48.413561+0000","last_became_peered":"2024-05-09T04:47:48.413561+0000","last_unstale":"2024-05-09T04:51:05.658896+0000","last_undegraded":"2024-05-09T04:51:05.658896+0000","last_fullsized":"2024-05-09T04:51:05.658896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:33:48.148834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651456+0000","last_change":"2024-05-09T04:47:48.408752+0000","last_active":"2024-05-09T04:51:05.651456+0000","last_peered":"2024-05-09T04:51:05.651456+0000","last_clean":"2024-05-09T04:51:05.651456+0000","last_became_active":"2024-05-09T04:47:48.408393+0000","last_became_peered":"2024-05-09T04:47:48.408393+0000","last_unstale":"2024-05-09T04:51:05.651456+0000","last_undegraded":"2024-05-09T04:51:05.651456+0000","last_fullsized":"2024-05-09T04:51:05.651456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:06.778002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548656+0000","last_change":"2024-05-09T04:47:48.409303+0000","last_active":"2024-05-09T04:51:05.548656+0000","last_peered":"2024-05-09T04:51:05.548656+0000","last_clean":"2024-05-09T04:51:05.548656+0000","last_became_active":"2024-05-09T04:47:48.408971+0000","last_became_peered":"2024-05-09T04:47:48.408971+0000","last_unstale":"2024-05-09T04:51:05.548656+0000","last_undegraded":"2024-05-09T04:51:05.548656+0000","last_fullsized":"2024-05-09T04:51:05.548656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:45.830647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544522+0000","last_change":"2024-05-09T04:47:48.404961+0000","last_active":"2024-05-09T04:51:05.544522+0000","last_peered":"2024-05-09T04:51:05.544522+0000","last_clean":"2024-05-09T04:51:05.544522+0000","last_became_active":"2024-05-09T04:47:48.404577+0000","last_became_peered":"2024-05-09T04:47:48.404577+0000","last_unstale":"2024-05-09T04:51:05.544522+0000","last_undegraded":"2024-05-09T04:51:05.544522+0000","last_fullsized":"2024-05-09T04:51:05.544522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:40.528066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.891879+0000","last_change":"2024-05-09T04:47:48.424364+0000","last_active":"2024-05-09T04:51:05.891879+0000","last_peered":"2024-05-09T04:51:05.891879+0000","last_clean":"2024-05-09T04:51:05.891879+0000","last_became_active":"2024-05-09T04:47:48.423991+0000","last_became_peered":"2024-05-09T04:47:48.423991+0000","last_unstale":"2024-05-09T04:51:05.891879+0000","last_undegraded":"2024-05-09T04:51:05.891879+0000","last_fullsized":"2024-05-09T04:51:05.891879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:14.040560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108459,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597498,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1187,"num_read_kb":688,"num_write":293,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":50695,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":1792494,"data_compressed":84389,"data_compressed_allocated":110592,"data_compressed_original":1597440,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645367,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":917504,"data_stored":458040,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841594,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1249103,"data_compressed":56798,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005120,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":851492,"data_compressed":33635,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201351,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":458196,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397580,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":1051748,"data_compressed":50642,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561105,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":457665,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":524517,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":458936,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":459898,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117089,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":982792,"data_compressed":39791,"data_compressed_allocated":53248,"data_compressed_original":557056,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":510524,"data_compressed":20057,"data_compressed_allocated":28672,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757334,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":1182109,"data_compressed":56798,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":68626,"data_compressed":8483,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134166,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201160,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:51:08.937 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-05-09T04:51:08.937 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-09T04:51:08.986 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:08 smithi110 ceph-mon[34904]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:51:08.986 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:08 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:08.986 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:08 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:08.986 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:08 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:08.987 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:08 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:08.987 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:08 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:08.987 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:08 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:08.987 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:08 smithi110 ceph-mon[34904]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-05-09T04:51:09.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.092+0000 7f093a7d5640 1 -- 172.21.15.46:0/2539886169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934102b00 msgr2=0x7f0934103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:09.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.092+0000 7f093a7d5640 1 --2- 172.21.15.46:0/2539886169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934102b00 0x7f0934103800 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0924009900 tx=0x7f092402a410 comp rx=0 tx=0).stop 2024-05-09T04:51:09.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.092+0000 7f093a7d5640 1 -- 172.21.15.46:0/2539886169 shutdown_connections 2024-05-09T04:51:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.092+0000 7f093a7d5640 1 --2- 172.21.15.46:0/2539886169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0934103d40 0x7f0934104150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.092+0000 7f093a7d5640 1 --2- 172.21.15.46:0/2539886169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0934108a40 0x7f093410ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.092+0000 7f093a7d5640 1 --2- 172.21.15.46:0/2539886169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934102b00 0x7f0934103800 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.092+0000 7f093a7d5640 1 -- 172.21.15.46:0/2539886169 >> 172.21.15.46:0/2539886169 conn(0x7f09340fdcd0 msgr2=0x7f09341001a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.092+0000 7f093a7d5640 1 -- 172.21.15.46:0/2539886169 shutdown_connections 2024-05-09T04:51:09.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.092+0000 7f093a7d5640 1 -- 172.21.15.46:0/2539886169 wait complete. 2024-05-09T04:51:09.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.093+0000 7f093a7d5640 1 Processor -- start 2024-05-09T04:51:09.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.093+0000 7f093a7d5640 1 -- start start 2024-05-09T04:51:09.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f093a7d5640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934102b00 0x7f0934199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:09.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f093a7d5640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0934103d40 0x7f09341997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f093a7d5640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0934108a40 0x7f09341a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f093a7d5640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f093410e600 con 0x7f0934102b00 2024-05-09T04:51:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f093a7d5640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f093410e480 con 0x7f0934103d40 2024-05-09T04:51:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f093a7d5640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f093410e780 con 0x7f0934108a40 2024-05-09T04:51:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f0933fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934102b00 0x7f0934199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f0933fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934102b00 0x7f0934199260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36454/0 (socket says 172.21.15.46:36454) 2024-05-09T04:51:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f0933fff640 1 -- 172.21.15.46:0/1046853412 learned_addr learned my addr 172.21.15.46:0/1046853412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f09337fe640 1 --2- 172.21.15.46:0/1046853412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0934103d40 0x7f09341997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f0938d72640 1 --2- 172.21.15.46:0/1046853412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0934108a40 0x7f09341a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f0933fff640 1 -- 172.21.15.46:0/1046853412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0934108a40 msgr2=0x7f09341a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f0933fff640 1 --2- 172.21.15.46:0/1046853412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0934108a40 0x7f09341a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f0933fff640 1 -- 172.21.15.46:0/1046853412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0934103d40 msgr2=0x7f09341997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f0933fff640 1 --2- 172.21.15.46:0/1046853412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0934103d40 0x7f09341997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:09.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f0933fff640 1 -- 172.21.15.46:0/1046853412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09341a0da0 con 0x7f0934102b00 2024-05-09T04:51:09.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f09337fe640 1 --2- 172.21.15.46:0/1046853412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0934103d40 0x7f09341997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:09.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.094+0000 7f0933fff640 1 --2- 172.21.15.46:0/1046853412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934102b00 0x7f0934199260 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0924006010 tx=0x7f0924004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:09.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.095+0000 7f09317fa640 1 -- 172.21.15.46:0/1046853412 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f092403f030 con 0x7f0934102b00 2024-05-09T04:51:09.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.095+0000 7f09317fa640 1 -- 172.21.15.46:0/1046853412 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0924031910 con 0x7f0934102b00 2024-05-09T04:51:09.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.095+0000 7f09317fa640 1 -- 172.21.15.46:0/1046853412 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0924035070 con 0x7f0934102b00 2024-05-09T04:51:09.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.095+0000 7f093a7d5640 1 -- 172.21.15.46:0/1046853412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09341a1030 con 0x7f0934102b00 2024-05-09T04:51:09.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.095+0000 7f093a7d5640 1 -- 172.21.15.46:0/1046853412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09341a1520 con 0x7f0934102b00 2024-05-09T04:51:09.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.096+0000 7f093a7d5640 1 -- 172.21.15.46:0/1046853412 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0934107850 con 0x7f0934102b00 2024-05-09T04:51:09.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.098+0000 7f09317fa640 1 -- 172.21.15.46:0/1046853412 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 23) ==== 149772+0+0 (secure 0 0 0) 0x7f0924042030 con 0x7f0934102b00 2024-05-09T04:51:09.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.098+0000 7f09317fa640 1 --2- 172.21.15.46:0/1046853412 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f091c0b2960 0x7f091c0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:09.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.098+0000 7f09317fa640 1 -- 172.21.15.46:0/1046853412 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f0924102d70 con 0x7f0934102b00 2024-05-09T04:51:09.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.099+0000 7f09337fe640 1 --2- 172.21.15.46:0/1046853412 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f091c0b2960 0x7f091c0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:09.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.099+0000 7f09337fe640 1 --2- 172.21.15.46:0/1046853412 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f091c0b2960 0x7f091c0b4e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f093419a300 tx=0x7f0928005f90 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:09.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.102+0000 7f09317fa640 1 -- 172.21.15.46:0/1046853412 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f09240ca0e0 con 0x7f0934102b00 2024-05-09T04:51:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:08 smithi072 ceph-mon[34088]: pgmap v4: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:51:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:08 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:08 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:08 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:08 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:08 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:08 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:08 smithi072 ceph-mon[34088]: mgrmap e23: x(active, since 3s), standbys: z, y 2024-05-09T04:51:09.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.301+0000 7f093a7d5640 1 -- 172.21.15.46:0/1046853412 --> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0934063110 con 0x7f091c0b2960 2024-05-09T04:51:09.316 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.316+0000 7f09317fa640 1 -- 172.21.15.46:0/1046853412 <== mgr.15150 v2:172.21.15.46:6800/2595796783 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258023 (secure 0 0 0) 0x7f0934063110 con 0x7f091c0b2960 2024-05-09T04:51:09.320 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:51:09.321 INFO:teuthology.orchestra.run.smithi046.stderr:dumped all 2024-05-09T04:51:09.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.320+0000 7f093a7d5640 1 -- 172.21.15.46:0/1046853412 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f091c0b2960 msgr2=0x7f091c0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:09.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.320+0000 7f093a7d5640 1 --2- 172.21.15.46:0/1046853412 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f091c0b2960 0x7f091c0b4e70 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f093419a300 tx=0x7f0928005f90 comp rx=0 tx=0).stop 2024-05-09T04:51:09.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.320+0000 7f093a7d5640 1 -- 172.21.15.46:0/1046853412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934102b00 msgr2=0x7f0934199260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:09.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.320+0000 7f093a7d5640 1 --2- 172.21.15.46:0/1046853412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934102b00 0x7f0934199260 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0924006010 tx=0x7f0924004270 comp rx=0 tx=0).stop 2024-05-09T04:51:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.320+0000 7f093a7d5640 1 -- 172.21.15.46:0/1046853412 shutdown_connections 2024-05-09T04:51:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.320+0000 7f093a7d5640 1 --2- 172.21.15.46:0/1046853412 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f091c0b2960 0x7f091c0b4e70 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.320+0000 7f093a7d5640 1 --2- 172.21.15.46:0/1046853412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0934108a40 0x7f09341a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.320+0000 7f093a7d5640 1 --2- 172.21.15.46:0/1046853412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0934103d40 0x7f09341997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.320+0000 7f093a7d5640 1 --2- 172.21.15.46:0/1046853412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0934102b00 0x7f0934199260 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.320+0000 7f093a7d5640 1 -- 172.21.15.46:0/1046853412 >> 172.21.15.46:0/1046853412 conn(0x7f09340fdcd0 msgr2=0x7f0934104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.320+0000 7f093a7d5640 1 -- 172.21.15.46:0/1046853412 shutdown_connections 2024-05-09T04:51:09.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:09.320+0000 7f093a7d5640 1 -- 172.21.15.46:0/1046853412 wait complete. 2024-05-09T04:51:09.332 INFO:teuthology.orchestra.run.smithi046.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-05-09T04:51:07.560145+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1188100,"num_objects":66,"num_object_clones":0,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":0,"num_read":1268,"num_read_kb":757,"num_write":418,"num_write_kb":3591,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":3002535,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":823,"ondisk_log_size":823,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":368860,"kb_used_data":11356,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124572964,"statfs":{"total":1151940427776,"available":1151562715136,"internally_reserved":0,"allocated":11628544,"data_stored":8645020,"data_compressed":344655,"data_compressed_allocated":462848,"data_compressed_original":3457024,"omap_allocated":19142,"internal_metadata":365933882},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.989646"},"pg_stats":[{"pgid":"3.3f","version":"84'8","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.892107+0000","last_change":"2024-05-09T04:47:48.408878+0000","last_active":"2024-05-09T04:51:05.892107+0000","last_peered":"2024-05-09T04:51:05.892107+0000","last_clean":"2024-05-09T04:51:05.892107+0000","last_became_active":"2024-05-09T04:47:48.408479+0000","last_became_peered":"2024-05-09T04:47:48.408479+0000","last_unstale":"2024-05-09T04:51:05.892107+0000","last_undegraded":"2024-05-09T04:51:05.892107+0000","last_fullsized":"2024-05-09T04:51:05.892107+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:16:02.494768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"84'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814499+0000","last_change":"2024-05-09T04:47:48.415650+0000","last_active":"2024-05-09T04:51:05.814499+0000","last_peered":"2024-05-09T04:51:05.814499+0000","last_clean":"2024-05-09T04:51:05.814499+0000","last_became_active":"2024-05-09T04:47:48.415322+0000","last_became_peered":"2024-05-09T04:47:48.415322+0000","last_unstale":"2024-05-09T04:51:05.814499+0000","last_undegraded":"2024-05-09T04:51:05.814499+0000","last_fullsized":"2024-05-09T04:51:05.814499+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:20:45.142513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814332+0000","last_change":"2024-05-09T04:47:48.417706+0000","last_active":"2024-05-09T04:51:05.814332+0000","last_peered":"2024-05-09T04:51:05.814332+0000","last_clean":"2024-05-09T04:51:05.814332+0000","last_became_active":"2024-05-09T04:47:48.412068+0000","last_became_peered":"2024-05-09T04:47:48.412068+0000","last_unstale":"2024-05-09T04:51:05.814332+0000","last_undegraded":"2024-05-09T04:51:05.814332+0000","last_fullsized":"2024-05-09T04:51:05.814332+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:04:44.602567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"84'8","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.805515+0000","last_change":"2024-05-09T04:47:48.415751+0000","last_active":"2024-05-09T04:51:05.805515+0000","last_peered":"2024-05-09T04:51:05.805515+0000","last_clean":"2024-05-09T04:51:05.805515+0000","last_became_active":"2024-05-09T04:47:48.414312+0000","last_became_peered":"2024-05-09T04:47:48.414312+0000","last_unstale":"2024-05-09T04:51:05.805515+0000","last_undegraded":"2024-05-09T04:51:05.805515+0000","last_fullsized":"2024-05-09T04:51:05.805515+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:13:12.202761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"84'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651437+0000","last_change":"2024-05-09T04:47:48.415919+0000","last_active":"2024-05-09T04:51:05.651437+0000","last_peered":"2024-05-09T04:51:05.651437+0000","last_clean":"2024-05-09T04:51:05.651437+0000","last_became_active":"2024-05-09T04:47:48.415533+0000","last_became_peered":"2024-05-09T04:47:48.415533+0000","last_unstale":"2024-05-09T04:51:05.651437+0000","last_undegraded":"2024-05-09T04:51:05.651437+0000","last_fullsized":"2024-05-09T04:51:05.651437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:32.262416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.662001+0000","last_change":"2024-05-09T04:47:48.419705+0000","last_active":"2024-05-09T04:51:05.662001+0000","last_peered":"2024-05-09T04:51:05.662001+0000","last_clean":"2024-05-09T04:51:05.662001+0000","last_became_active":"2024-05-09T04:47:48.419339+0000","last_became_peered":"2024-05-09T04:47:48.419339+0000","last_unstale":"2024-05-09T04:51:05.662001+0000","last_undegraded":"2024-05-09T04:51:05.662001+0000","last_fullsized":"2024-05-09T04:51:05.662001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:50:23.754071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"84'5","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810211+0000","last_change":"2024-05-09T04:47:48.404106+0000","last_active":"2024-05-09T04:51:05.810211+0000","last_peered":"2024-05-09T04:51:05.810211+0000","last_clean":"2024-05-09T04:51:05.810211+0000","last_became_active":"2024-05-09T04:47:48.403724+0000","last_became_peered":"2024-05-09T04:47:48.403724+0000","last_unstale":"2024-05-09T04:51:05.810211+0000","last_undegraded":"2024-05-09T04:51:05.810211+0000","last_fullsized":"2024-05-09T04:51:05.810211+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:41:41.134752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"84'9","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.648971+0000","last_change":"2024-05-09T04:48:28.562515+0000","last_active":"2024-05-09T04:51:05.648971+0000","last_peered":"2024-05-09T04:51:05.648971+0000","last_clean":"2024-05-09T04:51:05.648971+0000","last_became_active":"2024-05-09T04:48:28.562104+0000","last_became_peered":"2024-05-09T04:48:28.562104+0000","last_unstale":"2024-05-09T04:51:05.648971+0000","last_undegraded":"2024-05-09T04:51:05.648971+0000","last_fullsized":"2024-05-09T04:51:05.648971+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:38:33.342877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,5,10],"acting":[9,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.37","version":"84'11","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548850+0000","last_change":"2024-05-09T04:47:48.400467+0000","last_active":"2024-05-09T04:51:05.548850+0000","last_peered":"2024-05-09T04:51:05.548850+0000","last_clean":"2024-05-09T04:51:05.548850+0000","last_became_active":"2024-05-09T04:47:48.399960+0000","last_became_peered":"2024-05-09T04:47:48.399960+0000","last_unstale":"2024-05-09T04:51:05.548850+0000","last_undegraded":"2024-05-09T04:51:05.548850+0000","last_fullsized":"2024-05-09T04:51:05.548850+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:23:44.473053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"84'16","reported_seq":41,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.662377+0000","last_change":"2024-05-09T04:47:48.418970+0000","last_active":"2024-05-09T04:51:05.662377+0000","last_peered":"2024-05-09T04:51:05.662377+0000","last_clean":"2024-05-09T04:51:05.662377+0000","last_became_active":"2024-05-09T04:47:48.418574+0000","last_became_peered":"2024-05-09T04:47:48.418574+0000","last_unstale":"2024-05-09T04:51:05.662377+0000","last_undegraded":"2024-05-09T04:51:05.662377+0000","last_fullsized":"2024-05-09T04:51:05.662377+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:00:48.075893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810307+0000","last_change":"2024-05-09T04:47:48.409879+0000","last_active":"2024-05-09T04:51:05.810307+0000","last_peered":"2024-05-09T04:51:05.810307+0000","last_clean":"2024-05-09T04:51:05.810307+0000","last_became_active":"2024-05-09T04:47:48.409515+0000","last_became_peered":"2024-05-09T04:47:48.409515+0000","last_unstale":"2024-05-09T04:51:05.810307+0000","last_undegraded":"2024-05-09T04:51:05.810307+0000","last_fullsized":"2024-05-09T04:51:05.810307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:47:15.546214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.805520+0000","last_change":"2024-05-09T04:47:48.417556+0000","last_active":"2024-05-09T04:51:05.805520+0000","last_peered":"2024-05-09T04:51:05.805520+0000","last_clean":"2024-05-09T04:51:05.805520+0000","last_became_active":"2024-05-09T04:47:48.416787+0000","last_became_peered":"2024-05-09T04:47:48.416787+0000","last_unstale":"2024-05-09T04:51:05.805520+0000","last_undegraded":"2024-05-09T04:51:05.805520+0000","last_fullsized":"2024-05-09T04:51:05.805520+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:38:20.360232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"84'10","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544479+0000","last_change":"2024-05-09T04:47:48.415145+0000","last_active":"2024-05-09T04:51:05.544479+0000","last_peered":"2024-05-09T04:51:05.544479+0000","last_clean":"2024-05-09T04:51:05.544479+0000","last_became_active":"2024-05-09T04:47:48.414830+0000","last_became_peered":"2024-05-09T04:47:48.414830+0000","last_unstale":"2024-05-09T04:51:05.544479+0000","last_undegraded":"2024-05-09T04:51:05.544479+0000","last_fullsized":"2024-05-09T04:51:05.544479+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:15:55.936409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814035+0000","last_change":"2024-05-09T04:47:48.413977+0000","last_active":"2024-05-09T04:51:05.814035+0000","last_peered":"2024-05-09T04:51:05.814035+0000","last_clean":"2024-05-09T04:51:05.814035+0000","last_became_active":"2024-05-09T04:47:48.413604+0000","last_became_peered":"2024-05-09T04:47:48.413604+0000","last_unstale":"2024-05-09T04:51:05.814035+0000","last_undegraded":"2024-05-09T04:51:05.814035+0000","last_fullsized":"2024-05-09T04:51:05.814035+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:27:25.462958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.813924+0000","last_change":"2024-05-09T04:47:48.419000+0000","last_active":"2024-05-09T04:51:05.813924+0000","last_peered":"2024-05-09T04:51:05.813924+0000","last_clean":"2024-05-09T04:51:05.813924+0000","last_became_active":"2024-05-09T04:47:48.418457+0000","last_became_peered":"2024-05-09T04:47:48.418457+0000","last_unstale":"2024-05-09T04:51:05.813924+0000","last_undegraded":"2024-05-09T04:51:05.813924+0000","last_fullsized":"2024-05-09T04:51:05.813924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:36.325460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548387+0000","last_change":"2024-05-09T04:47:48.406754+0000","last_active":"2024-05-09T04:51:05.548387+0000","last_peered":"2024-05-09T04:51:05.548387+0000","last_clean":"2024-05-09T04:51:05.548387+0000","last_became_active":"2024-05-09T04:47:48.406396+0000","last_became_peered":"2024-05-09T04:47:48.406396+0000","last_unstale":"2024-05-09T04:51:05.548387+0000","last_undegraded":"2024-05-09T04:51:05.548387+0000","last_fullsized":"2024-05-09T04:51:05.548387+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:24:46.421061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"84'8","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.892267+0000","last_change":"2024-05-09T04:47:48.409556+0000","last_active":"2024-05-09T04:51:05.892267+0000","last_peered":"2024-05-09T04:51:05.892267+0000","last_clean":"2024-05-09T04:51:05.892267+0000","last_became_active":"2024-05-09T04:47:48.409145+0000","last_became_peered":"2024-05-09T04:47:48.409145+0000","last_unstale":"2024-05-09T04:51:05.892267+0000","last_undegraded":"2024-05-09T04:51:05.892267+0000","last_fullsized":"2024-05-09T04:51:05.892267+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:42:25.486257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"84'10","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.649670+0000","last_change":"2024-05-09T04:47:48.415546+0000","last_active":"2024-05-09T04:51:05.649670+0000","last_peered":"2024-05-09T04:51:05.649670+0000","last_clean":"2024-05-09T04:51:05.649670+0000","last_became_active":"2024-05-09T04:47:48.415221+0000","last_became_peered":"2024-05-09T04:47:48.415221+0000","last_unstale":"2024-05-09T04:51:05.649670+0000","last_undegraded":"2024-05-09T04:51:05.649670+0000","last_fullsized":"2024-05-09T04:51:05.649670+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:57:29.112409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545923+0000","last_change":"2024-05-09T04:47:48.413106+0000","last_active":"2024-05-09T04:51:05.545923+0000","last_peered":"2024-05-09T04:51:05.545923+0000","last_clean":"2024-05-09T04:51:05.545923+0000","last_became_active":"2024-05-09T04:47:48.412700+0000","last_became_peered":"2024-05-09T04:47:48.412700+0000","last_unstale":"2024-05-09T04:51:05.545923+0000","last_undegraded":"2024-05-09T04:51:05.545923+0000","last_fullsized":"2024-05-09T04:51:05.545923+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:37:31.682008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"80'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.649600+0000","last_change":"2024-05-09T04:47:45.386931+0000","last_active":"2024-05-09T04:51:05.649600+0000","last_peered":"2024-05-09T04:51:05.649600+0000","last_clean":"2024-05-09T04:51:05.649600+0000","last_became_active":"2024-05-09T04:47:45.386562+0000","last_became_peered":"2024-05-09T04:47:45.386562+0000","last_unstale":"2024-05-09T04:51:05.649600+0000","last_undegraded":"2024-05-09T04:51:05.649600+0000","last_fullsized":"2024-05-09T04:51:05.649600+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:19:59.335687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.652165+0000","last_change":"2024-05-09T04:47:48.411064+0000","last_active":"2024-05-09T04:51:05.652165+0000","last_peered":"2024-05-09T04:51:05.652165+0000","last_clean":"2024-05-09T04:51:05.652165+0000","last_became_active":"2024-05-09T04:47:48.410689+0000","last_became_peered":"2024-05-09T04:47:48.410689+0000","last_unstale":"2024-05-09T04:51:05.652165+0000","last_undegraded":"2024-05-09T04:51:05.652165+0000","last_fullsized":"2024-05-09T04:51:05.652165+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:31:21.178831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"80'2","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.652322+0000","last_change":"2024-05-09T04:47:45.612837+0000","last_active":"2024-05-09T04:51:05.652322+0000","last_peered":"2024-05-09T04:51:05.652322+0000","last_clean":"2024-05-09T04:51:05.652322+0000","last_became_active":"2024-05-09T04:47:45.612307+0000","last_became_peered":"2024-05-09T04:47:45.612307+0000","last_unstale":"2024-05-09T04:51:05.652322+0000","last_undegraded":"2024-05-09T04:51:05.652322+0000","last_fullsized":"2024-05-09T04:51:05.652322+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:44:59.686391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814704+0000","last_change":"2024-05-09T04:47:48.412781+0000","last_active":"2024-05-09T04:51:05.814704+0000","last_peered":"2024-05-09T04:51:05.814704+0000","last_clean":"2024-05-09T04:51:05.814704+0000","last_became_active":"2024-05-09T04:47:48.411396+0000","last_became_peered":"2024-05-09T04:47:48.411396+0000","last_unstale":"2024-05-09T04:51:05.814704+0000","last_undegraded":"2024-05-09T04:51:05.814704+0000","last_fullsized":"2024-05-09T04:51:05.814704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:41:22.775432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"80'4","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.892130+0000","last_change":"2024-05-09T04:48:28.568024+0000","last_active":"2024-05-09T04:51:05.892130+0000","last_peered":"2024-05-09T04:51:05.892130+0000","last_clean":"2024-05-09T04:51:05.892130+0000","last_became_active":"2024-05-09T04:48:28.559441+0000","last_became_peered":"2024-05-09T04:48:28.559441+0000","last_unstale":"2024-05-09T04:51:05.892130+0000","last_undegraded":"2024-05-09T04:51:05.892130+0000","last_fullsized":"2024-05-09T04:51:05.892130+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:27:56.257589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548481+0000","last_change":"2024-05-09T04:47:48.421361+0000","last_active":"2024-05-09T04:51:05.548481+0000","last_peered":"2024-05-09T04:51:05.548481+0000","last_clean":"2024-05-09T04:51:05.548481+0000","last_became_active":"2024-05-09T04:47:48.421018+0000","last_became_peered":"2024-05-09T04:47:48.421018+0000","last_unstale":"2024-05-09T04:51:05.548481+0000","last_undegraded":"2024-05-09T04:51:05.548481+0000","last_fullsized":"2024-05-09T04:51:05.548481+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:07:00.634983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"80'2","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.891836+0000","last_change":"2024-05-09T04:49:29.402935+0000","last_active":"2024-05-09T04:51:05.891836+0000","last_peered":"2024-05-09T04:51:05.891836+0000","last_clean":"2024-05-09T04:51:05.891836+0000","last_became_active":"2024-05-09T04:49:29.400084+0000","last_became_peered":"2024-05-09T04:49:29.400084+0000","last_unstale":"2024-05-09T04:51:05.891836+0000","last_undegraded":"2024-05-09T04:51:05.891836+0000","last_fullsized":"2024-05-09T04:51:05.891836+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:06:06.683122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,5],"acting":[11,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1d","version":"84'15","reported_seq":46,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.659117+0000","last_change":"2024-05-09T04:48:28.564243+0000","last_active":"2024-05-09T04:51:05.659117+0000","last_peered":"2024-05-09T04:51:05.659117+0000","last_clean":"2024-05-09T04:51:05.659117+0000","last_became_active":"2024-05-09T04:48:28.563732+0000","last_became_peered":"2024-05-09T04:48:28.563732+0000","last_unstale":"2024-05-09T04:51:05.659117+0000","last_undegraded":"2024-05-09T04:51:05.659117+0000","last_fullsized":"2024-05-09T04:51:05.659117+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:32:26.982339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'8","reported_seq":221,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810097+0000","last_change":"2024-05-09T04:47:45.389868+0000","last_active":"2024-05-09T04:51:05.810097+0000","last_peered":"2024-05-09T04:51:05.810097+0000","last_clean":"2024-05-09T04:51:05.810097+0000","last_became_active":"2024-05-09T04:47:45.389508+0000","last_became_peered":"2024-05-09T04:47:45.389508+0000","last_unstale":"2024-05-09T04:51:05.810097+0000","last_undegraded":"2024-05-09T04:51:05.810097+0000","last_fullsized":"2024-05-09T04:51:05.810097+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:32:45.050187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1712,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":181,"num_read_kb":3,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651364+0000","last_change":"2024-05-09T04:47:48.404457+0000","last_active":"2024-05-09T04:51:05.651364+0000","last_peered":"2024-05-09T04:51:05.651364+0000","last_clean":"2024-05-09T04:51:05.651364+0000","last_became_active":"2024-05-09T04:47:48.403968+0000","last_became_peered":"2024-05-09T04:47:48.403968+0000","last_unstale":"2024-05-09T04:51:05.651364+0000","last_undegraded":"2024-05-09T04:51:05.651364+0000","last_fullsized":"2024-05-09T04:51:05.651364+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:23:40.364916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"80'3","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.661879+0000","last_change":"2024-05-09T04:47:45.388183+0000","last_active":"2024-05-09T04:51:05.661879+0000","last_peered":"2024-05-09T04:51:05.661879+0000","last_clean":"2024-05-09T04:51:05.661879+0000","last_became_active":"2024-05-09T04:47:45.386603+0000","last_became_peered":"2024-05-09T04:47:45.386603+0000","last_unstale":"2024-05-09T04:51:05.661879+0000","last_undegraded":"2024-05-09T04:51:05.661879+0000","last_fullsized":"2024-05-09T04:51:05.661879+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:53.433619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"84'8","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.649200+0000","last_change":"2024-05-09T04:48:28.663120+0000","last_active":"2024-05-09T04:51:05.649200+0000","last_peered":"2024-05-09T04:51:05.649200+0000","last_clean":"2024-05-09T04:51:05.649200+0000","last_became_active":"2024-05-09T04:48:28.662748+0000","last_became_peered":"2024-05-09T04:48:28.662748+0000","last_unstale":"2024-05-09T04:51:05.649200+0000","last_undegraded":"2024-05-09T04:51:05.649200+0000","last_fullsized":"2024-05-09T04:51:05.649200+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:50:57.677876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,3],"acting":[9,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.19","version":"80'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544377+0000","last_change":"2024-05-09T04:47:45.384192+0000","last_active":"2024-05-09T04:51:05.544377+0000","last_peered":"2024-05-09T04:51:05.544377+0000","last_clean":"2024-05-09T04:51:05.544377+0000","last_became_active":"2024-05-09T04:47:45.383843+0000","last_became_peered":"2024-05-09T04:47:45.383843+0000","last_unstale":"2024-05-09T04:51:05.544377+0000","last_undegraded":"2024-05-09T04:51:05.544377+0000","last_fullsized":"2024-05-09T04:51:05.544377+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:13:28.815207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651282+0000","last_change":"2024-05-09T04:47:48.409203+0000","last_active":"2024-05-09T04:51:05.651282+0000","last_peered":"2024-05-09T04:51:05.651282+0000","last_clean":"2024-05-09T04:51:05.651282+0000","last_became_active":"2024-05-09T04:47:48.408861+0000","last_became_peered":"2024-05-09T04:47:48.408861+0000","last_unstale":"2024-05-09T04:51:05.651282+0000","last_undegraded":"2024-05-09T04:51:05.651282+0000","last_fullsized":"2024-05-09T04:51:05.651282+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:09:07.160397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"80'1","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.658840+0000","last_change":"2024-05-09T04:47:45.392723+0000","last_active":"2024-05-09T04:51:05.658840+0000","last_peered":"2024-05-09T04:51:05.658840+0000","last_clean":"2024-05-09T04:51:05.658840+0000","last_became_active":"2024-05-09T04:47:45.392313+0000","last_became_peered":"2024-05-09T04:47:45.392313+0000","last_unstale":"2024-05-09T04:51:05.658840+0000","last_undegraded":"2024-05-09T04:51:05.658840+0000","last_fullsized":"2024-05-09T04:51:05.658840+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:50:44.358702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"84'11","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.648930+0000","last_change":"2024-05-09T04:47:48.414000+0000","last_active":"2024-05-09T04:51:05.648930+0000","last_peered":"2024-05-09T04:51:05.648930+0000","last_clean":"2024-05-09T04:51:05.648930+0000","last_became_active":"2024-05-09T04:47:48.413532+0000","last_became_peered":"2024-05-09T04:47:48.413532+0000","last_unstale":"2024-05-09T04:51:05.648930+0000","last_undegraded":"2024-05-09T04:51:05.648930+0000","last_fullsized":"2024-05-09T04:51:05.648930+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:14:22.774814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"84'4","reported_seq":214,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.662399+0000","last_change":"2024-05-09T04:47:45.388661+0000","last_active":"2024-05-09T04:51:05.662399+0000","last_peered":"2024-05-09T04:51:05.662399+0000","last_clean":"2024-05-09T04:51:05.662399+0000","last_became_active":"2024-05-09T04:47:45.386010+0000","last_became_peered":"2024-05-09T04:47:45.386010+0000","last_unstale":"2024-05-09T04:51:05.662399+0000","last_undegraded":"2024-05-09T04:51:05.662399+0000","last_fullsized":"2024-05-09T04:51:05.662399+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:55:23.250463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":178,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"84'10","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548947+0000","last_change":"2024-05-09T04:47:48.421706+0000","last_active":"2024-05-09T04:51:05.548947+0000","last_peered":"2024-05-09T04:51:05.548947+0000","last_clean":"2024-05-09T04:51:05.548947+0000","last_became_active":"2024-05-09T04:47:48.420529+0000","last_became_peered":"2024-05-09T04:47:48.420529+0000","last_unstale":"2024-05-09T04:51:05.548947+0000","last_undegraded":"2024-05-09T04:51:05.548947+0000","last_fullsized":"2024-05-09T04:51:05.548947+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:26:10.110366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"80'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810291+0000","last_change":"2024-05-09T04:47:45.386625+0000","last_active":"2024-05-09T04:51:05.810291+0000","last_peered":"2024-05-09T04:51:05.810291+0000","last_clean":"2024-05-09T04:51:05.810291+0000","last_became_active":"2024-05-09T04:47:45.385087+0000","last_became_peered":"2024-05-09T04:47:45.385087+0000","last_unstale":"2024-05-09T04:51:05.810291+0000","last_undegraded":"2024-05-09T04:51:05.810291+0000","last_fullsized":"2024-05-09T04:51:05.810291+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:59:15.546894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"84'9","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814698+0000","last_change":"2024-05-09T04:48:28.562517+0000","last_active":"2024-05-09T04:51:05.814698+0000","last_peered":"2024-05-09T04:51:05.814698+0000","last_clean":"2024-05-09T04:51:05.814698+0000","last_became_active":"2024-05-09T04:48:28.562047+0000","last_became_peered":"2024-05-09T04:48:28.562047+0000","last_unstale":"2024-05-09T04:51:05.814698+0000","last_undegraded":"2024-05-09T04:51:05.814698+0000","last_fullsized":"2024-05-09T04:51:05.814698+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:12:22.769844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"80'4","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.659142+0000","last_change":"2024-05-09T04:47:45.613713+0000","last_active":"2024-05-09T04:51:05.659142+0000","last_peered":"2024-05-09T04:51:05.659142+0000","last_clean":"2024-05-09T04:51:05.659142+0000","last_became_active":"2024-05-09T04:47:45.613313+0000","last_became_peered":"2024-05-09T04:47:45.613313+0000","last_unstale":"2024-05-09T04:51:05.659142+0000","last_undegraded":"2024-05-09T04:51:05.659142+0000","last_fullsized":"2024-05-09T04:51:05.659142+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:32:44.825695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"84'11","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548722+0000","last_change":"2024-05-09T04:47:48.411234+0000","last_active":"2024-05-09T04:51:05.548722+0000","last_peered":"2024-05-09T04:51:05.548722+0000","last_clean":"2024-05-09T04:51:05.548722+0000","last_became_active":"2024-05-09T04:47:48.410881+0000","last_became_peered":"2024-05-09T04:47:48.410881+0000","last_unstale":"2024-05-09T04:51:05.548722+0000","last_undegraded":"2024-05-09T04:51:05.548722+0000","last_fullsized":"2024-05-09T04:51:05.548722+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:43:06.747871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'36","reported_seq":453,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.816697+0000","last_change":"2024-05-09T04:48:28.570426+0000","last_active":"2024-05-09T04:51:05.816697+0000","last_peered":"2024-05-09T04:51:05.816697+0000","last_clean":"2024-05-09T04:51:05.816697+0000","last_became_active":"2024-05-09T04:48:28.561786+0000","last_became_peered":"2024-05-09T04:48:28.561786+0000","last_unstale":"2024-05-09T04:51:05.816697+0000","last_undegraded":"2024-05-09T04:51:05.816697+0000","last_fullsized":"2024-05-09T04:51:05.816697+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:17:32.367142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67059,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":362,"num_read_kb":232,"num_write":37,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":50695,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,4],"acting":[10,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651672+0000","last_change":"2024-05-09T04:47:48.406510+0000","last_active":"2024-05-09T04:51:05.651672+0000","last_peered":"2024-05-09T04:51:05.651672+0000","last_clean":"2024-05-09T04:51:05.651672+0000","last_became_active":"2024-05-09T04:47:48.406166+0000","last_became_peered":"2024-05-09T04:47:48.406166+0000","last_unstale":"2024-05-09T04:51:05.651672+0000","last_undegraded":"2024-05-09T04:51:05.651672+0000","last_fullsized":"2024-05-09T04:51:05.651672+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:19:18.579620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"80'6","reported_seq":39,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545845+0000","last_change":"2024-05-09T04:47:45.387164+0000","last_active":"2024-05-09T04:51:05.545845+0000","last_peered":"2024-05-09T04:51:05.545845+0000","last_clean":"2024-05-09T04:51:05.545845+0000","last_became_active":"2024-05-09T04:47:45.386755+0000","last_became_peered":"2024-05-09T04:47:45.386755+0000","last_unstale":"2024-05-09T04:51:05.545845+0000","last_undegraded":"2024-05-09T04:51:05.545845+0000","last_fullsized":"2024-05-09T04:51:05.545845+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:12:07.838782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544274+0000","last_change":"2024-05-09T04:47:48.405436+0000","last_active":"2024-05-09T04:51:05.544274+0000","last_peered":"2024-05-09T04:51:05.544274+0000","last_clean":"2024-05-09T04:51:05.544274+0000","last_became_active":"2024-05-09T04:47:48.405032+0000","last_became_peered":"2024-05-09T04:47:48.405032+0000","last_unstale":"2024-05-09T04:51:05.544274+0000","last_undegraded":"2024-05-09T04:51:05.544274+0000","last_fullsized":"2024-05-09T04:51:05.544274+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:30:27.030397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"80'3","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.658921+0000","last_change":"2024-05-09T04:47:45.391970+0000","last_active":"2024-05-09T04:51:05.658921+0000","last_peered":"2024-05-09T04:51:05.658921+0000","last_clean":"2024-05-09T04:51:05.658921+0000","last_became_active":"2024-05-09T04:47:45.391597+0000","last_became_peered":"2024-05-09T04:47:45.391597+0000","last_unstale":"2024-05-09T04:51:05.658921+0000","last_undegraded":"2024-05-09T04:51:05.658921+0000","last_fullsized":"2024-05-09T04:51:05.658921+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:22:27.076631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"84'15","reported_seq":42,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810305+0000","last_change":"2024-05-09T04:47:48.419328+0000","last_active":"2024-05-09T04:51:05.810305+0000","last_peered":"2024-05-09T04:51:05.810305+0000","last_clean":"2024-05-09T04:51:05.810305+0000","last_became_active":"2024-05-09T04:47:48.418972+0000","last_became_peered":"2024-05-09T04:47:48.418972+0000","last_unstale":"2024-05-09T04:51:05.810305+0000","last_undegraded":"2024-05-09T04:51:05.810305+0000","last_fullsized":"2024-05-09T04:51:05.810305+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:26:15.164747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'6","reported_seq":221,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.805695+0000","last_change":"2024-05-09T04:47:45.386173+0000","last_active":"2024-05-09T04:51:05.805695+0000","last_peered":"2024-05-09T04:51:05.805695+0000","last_clean":"2024-05-09T04:51:05.805695+0000","last_became_active":"2024-05-09T04:47:45.384296+0000","last_became_peered":"2024-05-09T04:47:45.384296+0000","last_unstale":"2024-05-09T04:51:05.805695+0000","last_undegraded":"2024-05-09T04:51:05.805695+0000","last_fullsized":"2024-05-09T04:51:05.805695+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:17:03.246187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":183,"num_read_kb":5,"num_write":5,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810042+0000","last_change":"2024-05-09T04:47:48.415645+0000","last_active":"2024-05-09T04:51:05.810042+0000","last_peered":"2024-05-09T04:51:05.810042+0000","last_clean":"2024-05-09T04:51:05.810042+0000","last_became_active":"2024-05-09T04:47:48.415303+0000","last_became_peered":"2024-05-09T04:47:48.415303+0000","last_unstale":"2024-05-09T04:51:05.810042+0000","last_undegraded":"2024-05-09T04:51:05.810042+0000","last_fullsized":"2024-05-09T04:51:05.810042+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:24:39.703948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"80'15","reported_seq":49,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651268+0000","last_change":"2024-05-09T04:47:45.613320+0000","last_active":"2024-05-09T04:51:05.651268+0000","last_peered":"2024-05-09T04:51:05.651268+0000","last_clean":"2024-05-09T04:51:05.651268+0000","last_became_active":"2024-05-09T04:47:45.612871+0000","last_became_peered":"2024-05-09T04:47:45.612871+0000","last_unstale":"2024-05-09T04:51:05.651268+0000","last_undegraded":"2024-05-09T04:51:05.651268+0000","last_fullsized":"2024-05-09T04:51:05.651268+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:24:31.516652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"84'6","reported_seq":24,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.648939+0000","last_change":"2024-05-09T04:48:28.563376+0000","last_active":"2024-05-09T04:51:05.648939+0000","last_peered":"2024-05-09T04:51:05.648939+0000","last_clean":"2024-05-09T04:51:05.648939+0000","last_became_active":"2024-05-09T04:48:28.562973+0000","last_became_peered":"2024-05-09T04:48:28.562973+0000","last_unstale":"2024-05-09T04:51:05.648939+0000","last_undegraded":"2024-05-09T04:51:05.648939+0000","last_fullsized":"2024-05-09T04:51:05.648939+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:06:20.068517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,4,1],"acting":[9,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.f","version":"80'2","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651691+0000","last_change":"2024-05-09T04:47:45.389785+0000","last_active":"2024-05-09T04:51:05.651691+0000","last_peered":"2024-05-09T04:51:05.651691+0000","last_clean":"2024-05-09T04:51:05.651691+0000","last_became_active":"2024-05-09T04:47:45.389159+0000","last_became_peered":"2024-05-09T04:47:45.389159+0000","last_unstale":"2024-05-09T04:51:05.651691+0000","last_undegraded":"2024-05-09T04:51:05.651691+0000","last_fullsized":"2024-05-09T04:51:05.651691+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:28:51.804123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"84'5","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.892023+0000","last_change":"2024-05-09T04:48:28.563170+0000","last_active":"2024-05-09T04:51:05.892023+0000","last_peered":"2024-05-09T04:51:05.892023+0000","last_clean":"2024-05-09T04:51:05.892023+0000","last_became_active":"2024-05-09T04:48:28.562189+0000","last_became_peered":"2024-05-09T04:48:28.562189+0000","last_unstale":"2024-05-09T04:51:05.892023+0000","last_undegraded":"2024-05-09T04:51:05.892023+0000","last_fullsized":"2024-05-09T04:51:05.892023+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:46:00.643292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,6],"acting":[11,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.648790+0000","last_change":"2024-05-09T04:47:45.613739+0000","last_active":"2024-05-09T04:51:05.648790+0000","last_peered":"2024-05-09T04:51:05.648790+0000","last_clean":"2024-05-09T04:51:05.648790+0000","last_became_active":"2024-05-09T04:47:45.613365+0000","last_became_peered":"2024-05-09T04:47:45.613365+0000","last_unstale":"2024-05-09T04:51:05.648790+0000","last_undegraded":"2024-05-09T04:51:05.648790+0000","last_fullsized":"2024-05-09T04:51:05.648790+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:36:57.138068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.662064+0000","last_change":"2024-05-09T04:47:48.413343+0000","last_active":"2024-05-09T04:51:05.662064+0000","last_peered":"2024-05-09T04:51:05.662064+0000","last_clean":"2024-05-09T04:51:05.662064+0000","last_became_active":"2024-05-09T04:47:48.412941+0000","last_became_peered":"2024-05-09T04:47:48.412941+0000","last_unstale":"2024-05-09T04:51:05.662064+0000","last_undegraded":"2024-05-09T04:51:05.662064+0000","last_fullsized":"2024-05-09T04:51:05.662064+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:23:28.764553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"80'1","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810369+0000","last_change":"2024-05-09T04:47:45.386487+0000","last_active":"2024-05-09T04:51:05.810369+0000","last_peered":"2024-05-09T04:51:05.810369+0000","last_clean":"2024-05-09T04:51:05.810369+0000","last_became_active":"2024-05-09T04:47:45.386078+0000","last_became_peered":"2024-05-09T04:47:45.386078+0000","last_unstale":"2024-05-09T04:51:05.810369+0000","last_undegraded":"2024-05-09T04:51:05.810369+0000","last_fullsized":"2024-05-09T04:51:05.810369+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:33:19.998977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"84'7","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.661951+0000","last_change":"2024-05-09T04:47:48.418955+0000","last_active":"2024-05-09T04:51:05.661951+0000","last_peered":"2024-05-09T04:51:05.661951+0000","last_clean":"2024-05-09T04:51:05.661951+0000","last_became_active":"2024-05-09T04:47:48.418528+0000","last_became_peered":"2024-05-09T04:47:48.418528+0000","last_unstale":"2024-05-09T04:51:05.661951+0000","last_undegraded":"2024-05-09T04:51:05.661951+0000","last_fullsized":"2024-05-09T04:51:05.661951+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:51:02.386342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.659089+0000","last_change":"2024-05-09T04:47:48.413304+0000","last_active":"2024-05-09T04:51:05.659089+0000","last_peered":"2024-05-09T04:51:05.659089+0000","last_clean":"2024-05-09T04:51:05.659089+0000","last_became_active":"2024-05-09T04:47:48.412582+0000","last_became_peered":"2024-05-09T04:47:48.412582+0000","last_unstale":"2024-05-09T04:51:05.659089+0000","last_undegraded":"2024-05-09T04:51:05.659089+0000","last_fullsized":"2024-05-09T04:51:05.659089+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:38:40.027082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'6","reported_seq":217,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.803387+0000","last_change":"2024-05-09T04:47:45.612742+0000","last_active":"2024-05-09T04:51:05.803387+0000","last_peered":"2024-05-09T04:51:05.803387+0000","last_clean":"2024-05-09T04:51:05.803387+0000","last_became_active":"2024-05-09T04:47:45.612325+0000","last_became_peered":"2024-05-09T04:47:45.612325+0000","last_unstale":"2024-05-09T04:51:05.803387+0000","last_undegraded":"2024-05-09T04:51:05.803387+0000","last_fullsized":"2024-05-09T04:51:05.803387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:51:00.573996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":178,"num_read_kb":178,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651673+0000","last_change":"2024-05-09T04:47:48.412894+0000","last_active":"2024-05-09T04:51:05.651673+0000","last_peered":"2024-05-09T04:51:05.651673+0000","last_clean":"2024-05-09T04:51:05.651673+0000","last_became_active":"2024-05-09T04:47:48.412265+0000","last_became_peered":"2024-05-09T04:47:48.412265+0000","last_unstale":"2024-05-09T04:51:05.651673+0000","last_undegraded":"2024-05-09T04:51:05.651673+0000","last_fullsized":"2024-05-09T04:51:05.651673+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:03:23.517566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"80'3","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.805325+0000","last_change":"2024-05-09T04:47:45.387715+0000","last_active":"2024-05-09T04:51:05.805325+0000","last_peered":"2024-05-09T04:51:05.805325+0000","last_clean":"2024-05-09T04:51:05.805325+0000","last_became_active":"2024-05-09T04:47:45.387229+0000","last_became_peered":"2024-05-09T04:47:45.387229+0000","last_unstale":"2024-05-09T04:51:05.805325+0000","last_undegraded":"2024-05-09T04:51:05.805325+0000","last_fullsized":"2024-05-09T04:51:05.805325+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:07:22.540214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":107,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810292+0000","last_change":"2024-05-09T04:46:21.632579+0000","last_active":"2024-05-09T04:51:05.810292+0000","last_peered":"2024-05-09T04:51:05.810292+0000","last_clean":"2024-05-09T04:51:05.810292+0000","last_became_active":"2024-05-09T04:46:21.627483+0000","last_became_peered":"2024-05-09T04:46:21.627483+0000","last_unstale":"2024-05-09T04:51:05.810292+0000","last_undegraded":"2024-05-09T04:51:05.810292+0000","last_fullsized":"2024-05-09T04:51:05.810292+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:42:54.179739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:42:54.179739+0000","last_clean_scrub_stamp":"2024-05-09T04:42:54.179739+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T07:17:44.331526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"84'5","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544198+0000","last_change":"2024-05-09T04:47:48.418570+0000","last_active":"2024-05-09T04:51:05.544198+0000","last_peered":"2024-05-09T04:51:05.544198+0000","last_clean":"2024-05-09T04:51:05.544198+0000","last_became_active":"2024-05-09T04:47:48.418270+0000","last_became_peered":"2024-05-09T04:47:48.418270+0000","last_unstale":"2024-05-09T04:51:05.544198+0000","last_undegraded":"2024-05-09T04:51:05.544198+0000","last_fullsized":"2024-05-09T04:51:05.544198+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:02:28.406846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"80'3","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810387+0000","last_change":"2024-05-09T04:47:45.387161+0000","last_active":"2024-05-09T04:51:05.810387+0000","last_peered":"2024-05-09T04:51:05.810387+0000","last_clean":"2024-05-09T04:51:05.810387+0000","last_became_active":"2024-05-09T04:47:45.386753+0000","last_became_peered":"2024-05-09T04:47:45.386753+0000","last_unstale":"2024-05-09T04:51:05.810387+0000","last_undegraded":"2024-05-09T04:51:05.810387+0000","last_fullsized":"2024-05-09T04:51:05.810387+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:34:15.402418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"80'1","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651727+0000","last_change":"2024-05-09T04:47:45.613551+0000","last_active":"2024-05-09T04:51:05.651727+0000","last_peered":"2024-05-09T04:51:05.651727+0000","last_clean":"2024-05-09T04:51:05.651727+0000","last_became_active":"2024-05-09T04:47:45.613086+0000","last_became_peered":"2024-05-09T04:47:45.613086+0000","last_unstale":"2024-05-09T04:51:05.651727+0000","last_undegraded":"2024-05-09T04:51:05.651727+0000","last_fullsized":"2024-05-09T04:51:05.651727+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:46:04.350960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544530+0000","last_change":"2024-05-09T04:47:48.403701+0000","last_active":"2024-05-09T04:51:05.544530+0000","last_peered":"2024-05-09T04:51:05.544530+0000","last_clean":"2024-05-09T04:51:05.544530+0000","last_became_active":"2024-05-09T04:47:48.403337+0000","last_became_peered":"2024-05-09T04:47:48.403337+0000","last_unstale":"2024-05-09T04:51:05.544530+0000","last_undegraded":"2024-05-09T04:51:05.544530+0000","last_fullsized":"2024-05-09T04:51:05.544530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:09:53.032348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'47","reported_seq":102,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.771184+0000","last_change":"2024-05-09T04:47:45.385935+0000","last_active":"2024-05-09T04:51:05.771184+0000","last_peered":"2024-05-09T04:51:05.771184+0000","last_clean":"2024-05-09T04:51:05.771184+0000","last_became_active":"2024-05-09T04:47:45.385533+0000","last_became_peered":"2024-05-09T04:47:45.385533+0000","last_unstale":"2024-05-09T04:51:05.771184+0000","last_undegraded":"2024-05-09T04:51:05.771184+0000","last_fullsized":"2024-05-09T04:51:05.771184+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:34:38.014201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":85,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"84'5","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.652135+0000","last_change":"2024-05-09T04:47:48.409590+0000","last_active":"2024-05-09T04:51:05.652135+0000","last_peered":"2024-05-09T04:51:05.652135+0000","last_clean":"2024-05-09T04:51:05.652135+0000","last_became_active":"2024-05-09T04:47:48.409191+0000","last_became_peered":"2024-05-09T04:47:48.409191+0000","last_unstale":"2024-05-09T04:51:05.652135+0000","last_undegraded":"2024-05-09T04:51:05.652135+0000","last_fullsized":"2024-05-09T04:51:05.652135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:01:50.440609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"80'4","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.805557+0000","last_change":"2024-05-09T04:47:45.389147+0000","last_active":"2024-05-09T04:51:05.805557+0000","last_peered":"2024-05-09T04:51:05.805557+0000","last_clean":"2024-05-09T04:51:05.805557+0000","last_became_active":"2024-05-09T04:47:45.388797+0000","last_became_peered":"2024-05-09T04:47:45.388797+0000","last_unstale":"2024-05-09T04:51:05.805557+0000","last_undegraded":"2024-05-09T04:51:05.805557+0000","last_fullsized":"2024-05-09T04:51:05.805557+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:25:19.265278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814554+0000","last_change":"2024-05-09T04:47:48.419100+0000","last_active":"2024-05-09T04:51:05.814554+0000","last_peered":"2024-05-09T04:51:05.814554+0000","last_clean":"2024-05-09T04:51:05.814554+0000","last_became_active":"2024-05-09T04:47:48.418499+0000","last_became_peered":"2024-05-09T04:47:48.418499+0000","last_unstale":"2024-05-09T04:51:05.814554+0000","last_undegraded":"2024-05-09T04:51:05.814554+0000","last_fullsized":"2024-05-09T04:51:05.814554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:19:23.821513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"80'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548394+0000","last_change":"2024-05-09T04:47:45.613269+0000","last_active":"2024-05-09T04:51:05.548394+0000","last_peered":"2024-05-09T04:51:05.548394+0000","last_clean":"2024-05-09T04:51:05.548394+0000","last_became_active":"2024-05-09T04:47:45.612895+0000","last_became_peered":"2024-05-09T04:47:45.612895+0000","last_unstale":"2024-05-09T04:51:05.548394+0000","last_undegraded":"2024-05-09T04:51:05.548394+0000","last_fullsized":"2024-05-09T04:51:05.548394+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:06:52.966974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"84'2","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651326+0000","last_change":"2024-05-09T04:47:48.416658+0000","last_active":"2024-05-09T04:51:05.651326+0000","last_peered":"2024-05-09T04:51:05.651326+0000","last_clean":"2024-05-09T04:51:05.651326+0000","last_became_active":"2024-05-09T04:47:48.416238+0000","last_became_peered":"2024-05-09T04:47:48.416238+0000","last_unstale":"2024-05-09T04:51:05.651326+0000","last_undegraded":"2024-05-09T04:51:05.651326+0000","last_fullsized":"2024-05-09T04:51:05.651326+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:35:50.126723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"80'3","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651564+0000","last_change":"2024-05-09T04:47:45.384018+0000","last_active":"2024-05-09T04:51:05.651564+0000","last_peered":"2024-05-09T04:51:05.651564+0000","last_clean":"2024-05-09T04:51:05.651564+0000","last_became_active":"2024-05-09T04:47:45.383647+0000","last_became_peered":"2024-05-09T04:47:45.383647+0000","last_unstale":"2024-05-09T04:51:05.651564+0000","last_undegraded":"2024-05-09T04:51:05.651564+0000","last_fullsized":"2024-05-09T04:51:05.651564+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:43:06.431233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.891798+0000","last_change":"2024-05-09T04:47:48.409089+0000","last_active":"2024-05-09T04:51:05.891798+0000","last_peered":"2024-05-09T04:51:05.891798+0000","last_clean":"2024-05-09T04:51:05.891798+0000","last_became_active":"2024-05-09T04:47:48.408548+0000","last_became_peered":"2024-05-09T04:47:48.408548+0000","last_unstale":"2024-05-09T04:51:05.891798+0000","last_undegraded":"2024-05-09T04:51:05.891798+0000","last_fullsized":"2024-05-09T04:51:05.891798+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:51:13.633008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'2","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545683+0000","last_change":"2024-05-09T04:47:45.387580+0000","last_active":"2024-05-09T04:51:05.545683+0000","last_peered":"2024-05-09T04:51:05.545683+0000","last_clean":"2024-05-09T04:51:05.545683+0000","last_became_active":"2024-05-09T04:47:45.386929+0000","last_became_peered":"2024-05-09T04:47:45.386929+0000","last_unstale":"2024-05-09T04:51:05.545683+0000","last_undegraded":"2024-05-09T04:51:05.545683+0000","last_fullsized":"2024-05-09T04:51:05.545683+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:40:19.661763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651461+0000","last_change":"2024-05-09T04:47:48.416311+0000","last_active":"2024-05-09T04:51:05.651461+0000","last_peered":"2024-05-09T04:51:05.651461+0000","last_clean":"2024-05-09T04:51:05.651461+0000","last_became_active":"2024-05-09T04:47:48.415969+0000","last_became_peered":"2024-05-09T04:47:48.415969+0000","last_unstale":"2024-05-09T04:51:05.651461+0000","last_undegraded":"2024-05-09T04:51:05.651461+0000","last_fullsized":"2024-05-09T04:51:05.651461+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:53:55.751210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"80'4","reported_seq":50,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651448+0000","last_change":"2024-05-09T04:48:28.573916+0000","last_active":"2024-05-09T04:51:05.651448+0000","last_peered":"2024-05-09T04:51:05.651448+0000","last_clean":"2024-05-09T04:51:05.651448+0000","last_became_active":"2024-05-09T04:48:28.562872+0000","last_became_peered":"2024-05-09T04:48:28.562872+0000","last_unstale":"2024-05-09T04:51:05.651448+0000","last_undegraded":"2024-05-09T04:51:05.651448+0000","last_fullsized":"2024-05-09T04:51:05.651448+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:51:02.443694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"84'5","reported_seq":32,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810462+0000","last_change":"2024-05-09T04:47:48.408364+0000","last_active":"2024-05-09T04:51:05.810462+0000","last_peered":"2024-05-09T04:51:05.810462+0000","last_clean":"2024-05-09T04:51:05.810462+0000","last_became_active":"2024-05-09T04:47:48.407982+0000","last_became_peered":"2024-05-09T04:47:48.407982+0000","last_unstale":"2024-05-09T04:51:05.810462+0000","last_undegraded":"2024-05-09T04:51:05.810462+0000","last_fullsized":"2024-05-09T04:51:05.810462+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:36:14.103171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"80'3","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545878+0000","last_change":"2024-05-09T04:47:45.614637+0000","last_active":"2024-05-09T04:51:05.545878+0000","last_peered":"2024-05-09T04:51:05.545878+0000","last_clean":"2024-05-09T04:51:05.545878+0000","last_became_active":"2024-05-09T04:47:45.613290+0000","last_became_peered":"2024-05-09T04:47:45.613290+0000","last_unstale":"2024-05-09T04:51:05.545878+0000","last_undegraded":"2024-05-09T04:51:05.545878+0000","last_fullsized":"2024-05-09T04:51:05.545878+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:07:48.547382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810433+0000","last_change":"2024-05-09T04:47:48.405537+0000","last_active":"2024-05-09T04:51:05.810433+0000","last_peered":"2024-05-09T04:51:05.810433+0000","last_clean":"2024-05-09T04:51:05.810433+0000","last_became_active":"2024-05-09T04:47:48.405148+0000","last_became_peered":"2024-05-09T04:47:48.405148+0000","last_unstale":"2024-05-09T04:51:05.810433+0000","last_undegraded":"2024-05-09T04:51:05.810433+0000","last_fullsized":"2024-05-09T04:51:05.810433+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:52:46.981401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"85'39","reported_seq":79,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.768865+0000","last_change":"2024-05-09T04:47:45.389151+0000","last_active":"2024-05-09T04:51:05.768865+0000","last_peered":"2024-05-09T04:51:05.768865+0000","last_clean":"2024-05-09T04:51:05.768865+0000","last_became_active":"2024-05-09T04:47:45.388735+0000","last_became_peered":"2024-05-09T04:47:45.388735+0000","last_unstale":"2024-05-09T04:51:05.768865+0000","last_undegraded":"2024-05-09T04:51:05.768865+0000","last_fullsized":"2024-05-09T04:51:05.768865+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T15:03:26.560224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810206+0000","last_change":"2024-05-09T04:47:48.413881+0000","last_active":"2024-05-09T04:51:05.810206+0000","last_peered":"2024-05-09T04:51:05.810206+0000","last_clean":"2024-05-09T04:51:05.810206+0000","last_became_active":"2024-05-09T04:47:48.413559+0000","last_became_peered":"2024-05-09T04:47:48.413559+0000","last_unstale":"2024-05-09T04:51:05.810206+0000","last_undegraded":"2024-05-09T04:51:05.810206+0000","last_fullsized":"2024-05-09T04:51:05.810206+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:40:40.305319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"80'2","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544766+0000","last_change":"2024-05-09T04:47:45.383406+0000","last_active":"2024-05-09T04:51:05.544766+0000","last_peered":"2024-05-09T04:51:05.544766+0000","last_clean":"2024-05-09T04:51:05.544766+0000","last_became_active":"2024-05-09T04:47:45.383025+0000","last_became_peered":"2024-05-09T04:47:45.383025+0000","last_unstale":"2024-05-09T04:51:05.544766+0000","last_undegraded":"2024-05-09T04:51:05.544766+0000","last_fullsized":"2024-05-09T04:51:05.544766+0000","mapping_epoch":74,"log_start":"0'0","ondisk_log_start":"0'0","created":74,"last_epoch_clean":75,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:44.359316+0000","last_clean_scrub_stamp":"2024-05-09T04:47:44.359316+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:56:46.007454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.810679+0000","last_change":"2024-05-09T04:47:48.412481+0000","last_active":"2024-05-09T04:51:05.810679+0000","last_peered":"2024-05-09T04:51:05.810679+0000","last_clean":"2024-05-09T04:51:05.810679+0000","last_became_active":"2024-05-09T04:47:48.412034+0000","last_became_peered":"2024-05-09T04:47:48.412034+0000","last_unstale":"2024-05-09T04:51:05.810679+0000","last_undegraded":"2024-05-09T04:51:05.810679+0000","last_fullsized":"2024-05-09T04:51:05.810679+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:09:14.030981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"84'9","reported_seq":36,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.891860+0000","last_change":"2024-05-09T04:47:48.406958+0000","last_active":"2024-05-09T04:51:05.891860+0000","last_peered":"2024-05-09T04:51:05.891860+0000","last_clean":"2024-05-09T04:51:05.891860+0000","last_became_active":"2024-05-09T04:47:48.406575+0000","last_became_peered":"2024-05-09T04:47:48.406575+0000","last_unstale":"2024-05-09T04:51:05.891860+0000","last_undegraded":"2024-05-09T04:51:05.891860+0000","last_fullsized":"2024-05-09T04:51:05.891860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T09:19:23.389676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"84'12","reported_seq":43,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545618+0000","last_change":"2024-05-09T04:48:28.562376+0000","last_active":"2024-05-09T04:51:05.545618+0000","last_peered":"2024-05-09T04:51:05.545618+0000","last_clean":"2024-05-09T04:51:05.545618+0000","last_became_active":"2024-05-09T04:48:28.561866+0000","last_became_peered":"2024-05-09T04:48:28.561866+0000","last_unstale":"2024-05-09T04:51:05.545618+0000","last_undegraded":"2024-05-09T04:51:05.545618+0000","last_fullsized":"2024-05-09T04:51:05.545618+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:36:51.974146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.545565+0000","last_change":"2024-05-09T04:47:48.413908+0000","last_active":"2024-05-09T04:51:05.545565+0000","last_peered":"2024-05-09T04:51:05.545565+0000","last_clean":"2024-05-09T04:51:05.545565+0000","last_became_active":"2024-05-09T04:47:48.413575+0000","last_became_peered":"2024-05-09T04:47:48.413575+0000","last_unstale":"2024-05-09T04:51:05.545565+0000","last_undegraded":"2024-05-09T04:51:05.545565+0000","last_fullsized":"2024-05-09T04:51:05.545565+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:17:40.742510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"84'7","reported_seq":34,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.814127+0000","last_change":"2024-05-09T04:47:48.406607+0000","last_active":"2024-05-09T04:51:05.814127+0000","last_peered":"2024-05-09T04:51:05.814127+0000","last_clean":"2024-05-09T04:51:05.814127+0000","last_became_active":"2024-05-09T04:47:48.406290+0000","last_became_peered":"2024-05-09T04:47:48.406290+0000","last_unstale":"2024-05-09T04:51:05.814127+0000","last_undegraded":"2024-05-09T04:51:05.814127+0000","last_fullsized":"2024-05-09T04:51:05.814127+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T06:56:22.107953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"84'4","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544591+0000","last_change":"2024-05-09T04:47:48.415153+0000","last_active":"2024-05-09T04:51:05.544591+0000","last_peered":"2024-05-09T04:51:05.544591+0000","last_clean":"2024-05-09T04:51:05.544591+0000","last_became_active":"2024-05-09T04:47:48.414793+0000","last_became_peered":"2024-05-09T04:47:48.414793+0000","last_unstale":"2024-05-09T04:51:05.544591+0000","last_undegraded":"2024-05-09T04:51:05.544591+0000","last_fullsized":"2024-05-09T04:51:05.544591+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:22:57.564091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"84'6","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.892197+0000","last_change":"2024-05-09T04:47:48.408646+0000","last_active":"2024-05-09T04:51:05.892197+0000","last_peered":"2024-05-09T04:51:05.892197+0000","last_clean":"2024-05-09T04:51:05.892197+0000","last_became_active":"2024-05-09T04:47:48.408250+0000","last_became_peered":"2024-05-09T04:47:48.408250+0000","last_unstale":"2024-05-09T04:51:05.892197+0000","last_undegraded":"2024-05-09T04:51:05.892197+0000","last_fullsized":"2024-05-09T04:51:05.892197+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T11:42:50.622937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"84'15","reported_seq":40,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.546131+0000","last_change":"2024-05-09T04:47:48.408686+0000","last_active":"2024-05-09T04:51:05.546131+0000","last_peered":"2024-05-09T04:51:05.546131+0000","last_clean":"2024-05-09T04:51:05.546131+0000","last_became_active":"2024-05-09T04:47:48.408318+0000","last_became_peered":"2024-05-09T04:47:48.408318+0000","last_unstale":"2024-05-09T04:51:05.546131+0000","last_undegraded":"2024-05-09T04:51:05.546131+0000","last_fullsized":"2024-05-09T04:51:05.546131+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T14:52:51.240485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"84'2","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.652134+0000","last_change":"2024-05-09T04:47:48.421562+0000","last_active":"2024-05-09T04:51:05.652134+0000","last_peered":"2024-05-09T04:51:05.652134+0000","last_clean":"2024-05-09T04:51:05.652134+0000","last_became_active":"2024-05-09T04:47:48.421068+0000","last_became_peered":"2024-05-09T04:47:48.421068+0000","last_unstale":"2024-05-09T04:51:05.652134+0000","last_undegraded":"2024-05-09T04:51:05.652134+0000","last_fullsized":"2024-05-09T04:51:05.652134+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T12:18:41.455187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"84'6","reported_seq":31,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.658896+0000","last_change":"2024-05-09T04:47:48.413942+0000","last_active":"2024-05-09T04:51:05.658896+0000","last_peered":"2024-05-09T04:51:05.658896+0000","last_clean":"2024-05-09T04:51:05.658896+0000","last_became_active":"2024-05-09T04:47:48.413561+0000","last_became_peered":"2024-05-09T04:47:48.413561+0000","last_unstale":"2024-05-09T04:51:05.658896+0000","last_undegraded":"2024-05-09T04:51:05.658896+0000","last_fullsized":"2024-05-09T04:51:05.658896+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T16:33:48.148834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"84'8","reported_seq":33,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.651456+0000","last_change":"2024-05-09T04:47:48.408752+0000","last_active":"2024-05-09T04:51:05.651456+0000","last_peered":"2024-05-09T04:51:05.651456+0000","last_clean":"2024-05-09T04:51:05.651456+0000","last_became_active":"2024-05-09T04:47:48.408393+0000","last_became_peered":"2024-05-09T04:47:48.408393+0000","last_unstale":"2024-05-09T04:51:05.651456+0000","last_undegraded":"2024-05-09T04:51:05.651456+0000","last_fullsized":"2024-05-09T04:51:05.651456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T10:54:06.778002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"84'10","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.548656+0000","last_change":"2024-05-09T04:47:48.409303+0000","last_active":"2024-05-09T04:51:05.548656+0000","last_peered":"2024-05-09T04:51:05.548656+0000","last_clean":"2024-05-09T04:51:05.548656+0000","last_became_active":"2024-05-09T04:47:48.408971+0000","last_became_peered":"2024-05-09T04:47:48.408971+0000","last_unstale":"2024-05-09T04:51:05.548656+0000","last_undegraded":"2024-05-09T04:51:05.548656+0000","last_fullsized":"2024-05-09T04:51:05.548656+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:45:45.830647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"84'11","reported_seq":38,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.544522+0000","last_change":"2024-05-09T04:47:48.404961+0000","last_active":"2024-05-09T04:51:05.544522+0000","last_peered":"2024-05-09T04:51:05.544522+0000","last_clean":"2024-05-09T04:51:05.544522+0000","last_became_active":"2024-05-09T04:47:48.404577+0000","last_became_peered":"2024-05-09T04:47:48.404577+0000","last_unstale":"2024-05-09T04:51:05.544522+0000","last_undegraded":"2024-05-09T04:51:05.544522+0000","last_fullsized":"2024-05-09T04:51:05.544522+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T13:51:40.528066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"84'8","reported_seq":35,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-05-09T04:51:05.891879+0000","last_change":"2024-05-09T04:47:48.424364+0000","last_active":"2024-05-09T04:51:05.891879+0000","last_peered":"2024-05-09T04:51:05.891879+0000","last_clean":"2024-05-09T04:51:05.891879+0000","last_became_active":"2024-05-09T04:47:48.423991+0000","last_became_peered":"2024-05-09T04:47:48.423991+0000","last_unstale":"2024-05-09T04:51:05.891879+0000","last_undegraded":"2024-05-09T04:51:05.891879+0000","last_fullsized":"2024-05-09T04:51:05.891879+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-09T04:47:47.379869+0000","last_clean_scrub_stamp":"2024-05-09T04:47:47.379869+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T08:34:14.040560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":108459,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":597498,"num_objects":62,"num_object_clones":0,"num_object_copies":186,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":0,"num_read":1187,"num_read_kb":688,"num_write":293,"num_write_kb":1439,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":50695,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":540672,"data_stored":1792494,"data_compressed":84389,"data_compressed_allocated":110592,"data_compressed_original":1597440,"omap_allocated":0,"internal_metadata":0},"log_size":229,"ondisk_log_size":229,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":72,"seq":309237645367,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36236,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708916,"statfs":{"total":95995035648,"available":95957929984,"internally_reserved":0,"allocated":917504,"data_stored":458040,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":66,"seq":283467841594,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28088,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717064,"statfs":{"total":95995035648,"available":95966273536,"internally_reserved":0,"allocated":1097728,"data_stored":1249103,"data_compressed":56798,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":61,"seq":261993005120,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":936,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":958464,"data_stored":851492,"data_compressed":33635,"data_compressed_allocated":45056,"data_compressed_original":434176,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":55,"seq":236223201351,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27932,"kb_used_data":912,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717220,"statfs":{"total":95995035648,"available":95966433280,"internally_reserved":0,"allocated":933888,"data_stored":458196,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397580,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27980,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717172,"statfs":{"total":95995035648,"available":95966384128,"internally_reserved":0,"allocated":983040,"data_stored":1051748,"data_compressed":50642,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561105,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27920,"kb_used_data":900,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717232,"statfs":{"total":95995035648,"available":95966445568,"internally_reserved":0,"allocated":921600,"data_stored":457665,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574954,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":991232,"data_stored":524517,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738479,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":942080,"data_stored":458936,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411429,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":916,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":937984,"data_stored":459898,"data_compressed":14489,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":27,"seq":115964117089,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36296,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708856,"statfs":{"total":95995035648,"available":95957868544,"internally_reserved":0,"allocated":983040,"data_stored":982792,"data_compressed":39791,"data_compressed_allocated":53248,"data_compressed_original":557056,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920859,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":946176,"data_stored":510524,"data_compressed":20057,"data_compressed_allocated":28672,"data_compressed_original":81920,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757334,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36328,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708824,"statfs":{"total":95995035648,"available":95957835776,"internally_reserved":0,"allocated":1015808,"data_stored":1182109,"data_compressed":56798,"data_compressed_allocated":73728,"data_compressed_original":753664,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36153,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67059,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":525100,"data_compressed":25302,"data_compressed_allocated":32768,"data_compressed_original":516096,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":68626,"data_compressed":8483,"data_compressed_allocated":12288,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":134166,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":393954,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":394034,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":131072,"data_stored":201160,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-09T04:51:09.336 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:51:09.625 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:09.934 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:09 smithi110 ceph-mon[34904]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:51:09.935 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:09 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:09.935 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:09 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:09.935 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:09 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:09.935 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:09 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:09.935 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:09 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:09.935 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:09 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:09.935 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:09 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:09.935 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:09 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:09.935 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:09 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:09.935 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:09 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:09.935 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:09 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:09.935 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:09 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:09.949 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:09 smithi046 ceph-mon[30136]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:51:09.949 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:09 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:09.949 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:09 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:09.949 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:09 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:09.949 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:09 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:09.949 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:09 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:09.950 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:09 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:09.950 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:09 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:09.950 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:09 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:09.950 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:09 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:09.950 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:09 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:09.950 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:09 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:09.950 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:09 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:09 smithi072 ceph-mon[34088]: from='client.15243 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:51:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:09 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:09 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:09 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:09 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:09 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:09 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:09 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:09 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:09 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:10.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:09 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:10.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:09 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:10.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:09 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi072 to 176.1M 2024-05-09T04:51:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: Unable to set osd_memory_target on smithi072 to 184725811: error parsing value: Value '184725811' is below minimum 939524096 2024-05-09T04:51:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: from='client.25022 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:51:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:51:11.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi110 to 176.2M 2024-05-09T04:51:11.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: Unable to set osd_memory_target on smithi110 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-05-09T04:51:11.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:51:11.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:11.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:11.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:11.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:11.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:11.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:11.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi046 to 177.2M 2024-05-09T04:51:11.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: Unable to set osd_memory_target on smithi046 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-09T04:51:11.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:11.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:11.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-09T04:51:11.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-09T04:51:11.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: Updating smithi110:/etc/ceph/ceph.conf 2024-05-09T04:51:11.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:10 smithi110 ceph-mon[34904]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-09T04:51:11.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi072 to 176.1M 2024-05-09T04:51:11.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: Unable to set osd_memory_target on smithi072 to 184725811: error parsing value: Value '184725811' is below minimum 939524096 2024-05-09T04:51:11.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: from='client.25022 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:51:11.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:51:11.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi110 to 176.2M 2024-05-09T04:51:11.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: Unable to set osd_memory_target on smithi110 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-05-09T04:51:11.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:51:11.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:11.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:11.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:11.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:11.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:11.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:11.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi046 to 177.2M 2024-05-09T04:51:11.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: Unable to set osd_memory_target on smithi046 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-09T04:51:11.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:11.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:11.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-09T04:51:11.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-09T04:51:11.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: Updating smithi110:/etc/ceph/ceph.conf 2024-05-09T04:51:11.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:10 smithi046 ceph-mon[30136]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-09T04:51:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi072 to 176.1M 2024-05-09T04:51:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: Unable to set osd_memory_target on smithi072 to 184725811: error parsing value: Value '184725811' is below minimum 939524096 2024-05-09T04:51:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: from='client.25022 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:51:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: from='client.15255 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-09T04:51:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi110 to 176.2M 2024-05-09T04:51:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: Unable to set osd_memory_target on smithi110 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-05-09T04:51:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: pgmap v5: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:51:11.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:11.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:11.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:11.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:11.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:11.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T04:51:11.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi046 to 177.2M 2024-05-09T04:51:11.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: Unable to set osd_memory_target on smithi046 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-09T04:51:11.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:11.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:11.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: Updating smithi046:/etc/ceph/ceph.conf 2024-05-09T04:51:11.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: Updating smithi072:/etc/ceph/ceph.conf 2024-05-09T04:51:11.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: Updating smithi110:/etc/ceph/ceph.conf 2024-05-09T04:51:11.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:10 smithi072 ceph-mon[34088]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-05-09T04:51:11.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.484+0000 7fd9927cb700 1 -- 172.21.15.46:0/3479223864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98c102ae0 msgr2=0x7fd98c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:11.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.484+0000 7fd9927cb700 1 --2- 172.21.15.46:0/3479223864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98c102ae0 0x7fd98c102ef0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fd97c0099c0 tx=0x7fd97c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.484+0000 7fd9927cb700 1 -- 172.21.15.46:0/3479223864 shutdown_connections 2024-05-09T04:51:11.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.484+0000 7fd9927cb700 1 --2- 172.21.15.46:0/3479223864 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd98c109850 0x7fd98c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.484+0000 7fd9927cb700 1 --2- 172.21.15.46:0/3479223864 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98c102ae0 0x7fd98c102ef0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.484+0000 7fd9927cb700 1 --2- 172.21.15.46:0/3479223864 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd98c108f00 0x7fd98c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.484+0000 7fd9927cb700 1 -- 172.21.15.46:0/3479223864 >> 172.21.15.46:0/3479223864 conn(0x7fd98c0fe540 msgr2=0x7fd98c100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:11.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.484+0000 7fd9927cb700 1 -- 172.21.15.46:0/3479223864 shutdown_connections 2024-05-09T04:51:11.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.484+0000 7fd9927cb700 1 -- 172.21.15.46:0/3479223864 wait complete. 2024-05-09T04:51:11.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.485+0000 7fd9927cb700 1 Processor -- start 2024-05-09T04:51:11.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.485+0000 7fd9927cb700 1 -- start start 2024-05-09T04:51:11.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.485+0000 7fd9927cb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd98c102ae0 0x7fd98c103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:11.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.485+0000 7fd9927cb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd98c108f00 0x7fd98c103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:11.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.485+0000 7fd9927cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98c109850 0x7fd98c1044b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:11.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.485+0000 7fd9927cb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd98c114480 con 0x7fd98c109850 2024-05-09T04:51:11.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.485+0000 7fd9927cb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd98c114300 con 0x7fd98c108f00 2024-05-09T04:51:11.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.485+0000 7fd9927cb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd98c114600 con 0x7fd98c102ae0 2024-05-09T04:51:11.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.486+0000 7fd990d68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98c109850 0x7fd98c1044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:11.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.486+0000 7fd990d68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98c109850 0x7fd98c1044b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36474/0 (socket says 172.21.15.46:36474) 2024-05-09T04:51:11.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.486+0000 7fd990d68700 1 -- 172.21.15.46:0/3081016103 learned_addr learned my addr 172.21.15.46:0/3081016103 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:11.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.486+0000 7fd98b7fe700 1 --2- 172.21.15.46:0/3081016103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd98c108f00 0x7fd98c103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:11.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.486+0000 7fd98bfff700 1 --2- 172.21.15.46:0/3081016103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd98c102ae0 0x7fd98c103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:11.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.486+0000 7fd990d68700 1 -- 172.21.15.46:0/3081016103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd98c102ae0 msgr2=0x7fd98c103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:11.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.486+0000 7fd990d68700 1 --2- 172.21.15.46:0/3081016103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd98c102ae0 0x7fd98c103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.486+0000 7fd990d68700 1 -- 172.21.15.46:0/3081016103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd98c108f00 msgr2=0x7fd98c103f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:11.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.486+0000 7fd990d68700 1 --2- 172.21.15.46:0/3081016103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd98c108f00 0x7fd98c103f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.486+0000 7fd990d68700 1 -- 172.21.15.46:0/3081016103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd98c1a9100 con 0x7fd98c109850 2024-05-09T04:51:11.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.486+0000 7fd990d68700 1 --2- 172.21.15.46:0/3081016103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98c109850 0x7fd98c1044b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd984007d70 tx=0x7fd98400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:11.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.486+0000 7fd9897fa700 1 -- 172.21.15.46:0/3081016103 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd984016020 con 0x7fd98c109850 2024-05-09T04:51:11.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.486+0000 7fd9927cb700 1 -- 172.21.15.46:0/3081016103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd98c1a93f0 con 0x7fd98c109850 2024-05-09T04:51:11.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.486+0000 7fd9897fa700 1 -- 172.21.15.46:0/3081016103 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd98400ac90 con 0x7fd98c109850 2024-05-09T04:51:11.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.486+0000 7fd9897fa700 1 -- 172.21.15.46:0/3081016103 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9840115e0 con 0x7fd98c109850 2024-05-09T04:51:11.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.487+0000 7fd9927cb700 1 -- 172.21.15.46:0/3081016103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd98c1a9930 con 0x7fd98c109850 2024-05-09T04:51:11.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.488+0000 7fd9927cb700 1 -- 172.21.15.46:0/3081016103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd98c0512c0 con 0x7fd98c109850 2024-05-09T04:51:11.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.489+0000 7fd9897fa700 1 -- 172.21.15.46:0/3081016103 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fd984019020 con 0x7fd98c109850 2024-05-09T04:51:11.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.490+0000 7fd9897fa700 1 --2- 172.21.15.46:0/3081016103 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd9740b2a80 0x7fd9740b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:11.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.490+0000 7fd98bfff700 1 --2- 172.21.15.46:0/3081016103 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd9740b2a80 0x7fd9740b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:11.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.490+0000 7fd9897fa700 1 -- 172.21.15.46:0/3081016103 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fd9840e2ab0 con 0x7fd98c109850 2024-05-09T04:51:11.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.490+0000 7fd98bfff700 1 --2- 172.21.15.46:0/3081016103 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd9740b2a80 0x7fd9740b4f90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009990 tx=0x7fd97c000bc0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:11.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.493+0000 7fd9897fa700 1 -- 172.21.15.46:0/3081016103 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd984020020 con 0x7fd98c109850 2024-05-09T04:51:11.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.755+0000 7fd9927cb700 1 -- 172.21.15.46:0/3081016103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd98c02e5f0 con 0x7fd98c109850 2024-05-09T04:51:11.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.756+0000 7fd9897fa700 1 -- 172.21.15.46:0/3081016103 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7fd9840aa410 con 0x7fd98c109850 2024-05-09T04:51:11.756 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:51:11.756 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:51:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.758+0000 7fd9927cb700 1 -- 172.21.15.46:0/3081016103 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd9740b2a80 msgr2=0x7fd9740b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:11.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.758+0000 7fd9927cb700 1 --2- 172.21.15.46:0/3081016103 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd9740b2a80 0x7fd9740b4f90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fd97c009990 tx=0x7fd97c000bc0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.758+0000 7fd9927cb700 1 -- 172.21.15.46:0/3081016103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98c109850 msgr2=0x7fd98c1044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.758+0000 7fd9927cb700 1 --2- 172.21.15.46:0/3081016103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98c109850 0x7fd98c1044b0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd984007d70 tx=0x7fd98400a340 comp rx=0 tx=0).stop 2024-05-09T04:51:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.759+0000 7fd9927cb700 1 -- 172.21.15.46:0/3081016103 shutdown_connections 2024-05-09T04:51:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.759+0000 7fd9927cb700 1 --2- 172.21.15.46:0/3081016103 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd9740b2a80 0x7fd9740b4f90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.759+0000 7fd9927cb700 1 --2- 172.21.15.46:0/3081016103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd98c109850 0x7fd98c1044b0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.759+0000 7fd9927cb700 1 --2- 172.21.15.46:0/3081016103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd98c102ae0 0x7fd98c103a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.759+0000 7fd9927cb700 1 --2- 172.21.15.46:0/3081016103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd98c108f00 0x7fd98c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.759+0000 7fd9927cb700 1 -- 172.21.15.46:0/3081016103 >> 172.21.15.46:0/3081016103 conn(0x7fd98c0fe540 msgr2=0x7fd98c107170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.759+0000 7fd9927cb700 1 -- 172.21.15.46:0/3081016103 shutdown_connections 2024-05-09T04:51:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:11.759+0000 7fd9927cb700 1 -- 172.21.15.46:0/3081016103 wait complete. 2024-05-09T04:51:11.759 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:51:12.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: Updating smithi046:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:51:12.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: Updating smithi110:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:51:12.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: Updating smithi072:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:51:12.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:51:12.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:51:12.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:51:12.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: Updating smithi046:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:51:12.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: Updating smithi072:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:51:12.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: Updating smithi110:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:51:12.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:12.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:12.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:12.059 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:11 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: Updating smithi046:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:51:12.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: Updating smithi110:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:51:12.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: Updating smithi072:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:51:12.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:51:12.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:51:12.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:51:12.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: Updating smithi046:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:51:12.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: Updating smithi072:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:51:12.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: Updating smithi110:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:51:12.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:12.082 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:12.082 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:12.082 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:11 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: Updating smithi046:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:51:12.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: Updating smithi110:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:51:12.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: Updating smithi072:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-09T04:51:12.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: Updating smithi046:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:51:12.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: Updating smithi072:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:51:12.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: Updating smithi110:/etc/ceph/ceph.client.admin.keyring 2024-05-09T04:51:12.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: Updating smithi046:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:51:12.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: Updating smithi072:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:51:12.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: Updating smithi110:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-09T04:51:12.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:12.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:12.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:12.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:11 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:12.448 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T04:51:12.707 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:13.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:12 smithi110 ceph-mon[34904]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:51:13.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:12 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3081016103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:51:13.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:12 smithi046 ceph-mon[30136]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:51:13.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:12 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3081016103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:51:13.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:12 smithi072 ceph-mon[34088]: pgmap v6: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T04:51:13.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:12 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3081016103' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:51:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.444+0000 7f8fb1c4c700 1 -- 172.21.15.46:0/3326598037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fac109850 msgr2=0x7f8fac1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:14.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.444+0000 7f8fb1c4c700 1 --2- 172.21.15.46:0/3326598037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fac109850 0x7f8fac1111b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400b0d0 tx=0x7f8fa400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:51:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.444+0000 7f8fb1c4c700 1 -- 172.21.15.46:0/3326598037 shutdown_connections 2024-05-09T04:51:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.444+0000 7f8fb1c4c700 1 --2- 172.21.15.46:0/3326598037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fac109850 0x7f8fac1111b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.444+0000 7f8fb1c4c700 1 --2- 172.21.15.46:0/3326598037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8fac102ae0 0x7f8fac102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.444+0000 7f8fb1c4c700 1 --2- 172.21.15.46:0/3326598037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fac108f00 0x7f8fac109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.444+0000 7f8fb1c4c700 1 -- 172.21.15.46:0/3326598037 >> 172.21.15.46:0/3326598037 conn(0x7f8fac0fe4e0 msgr2=0x7f8fac100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.444+0000 7f8fb1c4c700 1 -- 172.21.15.46:0/3326598037 shutdown_connections 2024-05-09T04:51:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.444+0000 7f8fb1c4c700 1 -- 172.21.15.46:0/3326598037 wait complete. 2024-05-09T04:51:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.445+0000 7f8fb1c4c700 1 Processor -- start 2024-05-09T04:51:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.445+0000 7f8fb1c4c700 1 -- start start 2024-05-09T04:51:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fb1c4c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fac102ae0 0x7f8fac19ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fb1c4c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8fac108f00 0x7f8fac19b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fb1c4c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fac109850 0x7f8fac19f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:14.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fb1c4c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac114480 con 0x7f8fac102ae0 2024-05-09T04:51:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fb1c4c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac114300 con 0x7f8fac109850 2024-05-09T04:51:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fb1c4c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8fac114600 con 0x7f8fac108f00 2024-05-09T04:51:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fab7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fac102ae0 0x7f8fac19ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fab7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fac102ae0 0x7f8fac19ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44334/0 (socket says 172.21.15.46:44334) 2024-05-09T04:51:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fab7fe700 1 -- 172.21.15.46:0/1107902902 learned_addr learned my addr 172.21.15.46:0/1107902902 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8faaffd700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8fac108f00 0x7f8fac19b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fabfff700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fac109850 0x7f8fac19f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fab7fe700 1 -- 172.21.15.46:0/1107902902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8fac108f00 msgr2=0x7f8fac19b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fab7fe700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8fac108f00 0x7f8fac19b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fab7fe700 1 -- 172.21.15.46:0/1107902902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fac109850 msgr2=0x7f8fac19f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fab7fe700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fac109850 0x7f8fac19f5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fab7fe700 1 -- 172.21.15.46:0/1107902902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fac19fc60 con 0x7f8fac102ae0 2024-05-09T04:51:14.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8faaffd700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8fac108f00 0x7f8fac19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:51:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.446+0000 7f8fabfff700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fac109850 0x7f8fac19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:51:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.447+0000 7f8fab7fe700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fac102ae0 0x7f8fac19ad90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400e9a0 tx=0x7f8f9400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.447+0000 7f8fa8ff9700 1 -- 172.21.15.46:0/1107902902 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f9400cbd0 con 0x7f8fac102ae0 2024-05-09T04:51:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.447+0000 7f8fa8ff9700 1 -- 172.21.15.46:0/1107902902 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f9400cd70 con 0x7f8fac102ae0 2024-05-09T04:51:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.447+0000 7f8fb1c4c700 1 -- 172.21.15.46:0/1107902902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fac19ff50 con 0x7f8fac102ae0 2024-05-09T04:51:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.447+0000 7f8fa8ff9700 1 -- 172.21.15.46:0/1107902902 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f94018680 con 0x7f8fac102ae0 2024-05-09T04:51:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.447+0000 7f8fb1c4c700 1 -- 172.21.15.46:0/1107902902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fac1a7870 con 0x7f8fac102ae0 2024-05-09T04:51:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.448+0000 7f8fb1c4c700 1 -- 172.21.15.46:0/1107902902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fac0512c0 con 0x7f8fac102ae0 2024-05-09T04:51:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.450+0000 7f8fa8ff9700 1 -- 172.21.15.46:0/1107902902 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f8f94010970 con 0x7f8fac102ae0 2024-05-09T04:51:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.451+0000 7f8fa8ff9700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8f980b2b00 0x7f8f980b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.451+0000 7f8faaffd700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8f980b2b00 0x7f8f980b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.451+0000 7f8fa8ff9700 1 -- 172.21.15.46:0/1107902902 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f8f940e2f80 con 0x7f8fac102ae0 2024-05-09T04:51:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.452+0000 7f8faaffd700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8f980b2b00 0x7f8f980b5010 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8fac101dc0 tx=0x7f8f9c009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:14.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.455+0000 7f8fa8ff9700 1 -- 172.21.15.46:0/1107902902 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8f940aa7d0 con 0x7f8fac102ae0 2024-05-09T04:51:14.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.669+0000 7f8fb1c4c700 1 -- 172.21.15.46:0/1107902902 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8fac02e5f0 con 0x7f8fac102ae0 2024-05-09T04:51:14.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.670+0000 7f8fa8ff9700 1 -- 172.21.15.46:0/1107902902 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+23062 (secure 0 0 0) 0x7f8f940aa0d0 con 0x7f8fac102ae0 2024-05-09T04:51:14.671 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:51:14.671 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":85,"fsid":"ecf9ae2c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:25.633876+0000","modified":"2024-05-09T04:51:05.539064+0000","last_up_change":"2024-05-09T04:46:46.418597+0000","last_in_change":"2024-05-09T04:46:25.903582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:42:53.301400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:47:43.413569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:47:47.264915+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d79cd3e7-6f82-4d60-96d9-26d8146c962f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6803","nonce":1576436508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6805","nonce":1576436508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6809","nonce":1576436508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6807","nonce":1576436508}]},"public_addr":"172.21.15.46:6803/1576436508","cluster_addr":"172.21.15.46:6805/1576436508","heartbeat_back_addr":"172.21.15.46:6809/1576436508","heartbeat_front_addr":"172.21.15.46:6807/1576436508","state":["exists","up"]},{"osd":1,"uuid":"698eba80-1ebd-4076-b6f9-953d2e7daf76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6801","nonce":379039775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6803","nonce":379039775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6807","nonce":379039775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6805","nonce":379039775}]},"public_addr":"172.21.15.72:6801/379039775","cluster_addr":"172.21.15.72:6803/379039775","heartbeat_back_addr":"172.21.15.72:6807/379039775","heartbeat_front_addr":"172.21.15.72:6805/379039775","state":["exists","up"]},{"osd":2,"uuid":"c549b131-cfa7-4724-926f-83e91a752d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6801","nonce":293447227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6803","nonce":293447227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6807","nonce":293447227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6805","nonce":293447227}]},"public_addr":"172.21.15.110:6801/293447227","cluster_addr":"172.21.15.110:6803/293447227","heartbeat_back_addr":"172.21.15.110:6807/293447227","heartbeat_front_addr":"172.21.15.110:6805/293447227","state":["exists","up"]},{"osd":3,"uuid":"549bb4f7-54d9-4cbc-ad40-7f3e717964f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6811","nonce":1115835653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6813","nonce":1115835653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6817","nonce":1115835653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6815","nonce":1115835653}]},"public_addr":"172.21.15.46:6811/1115835653","cluster_addr":"172.21.15.46:6813/1115835653","heartbeat_back_addr":"172.21.15.46:6817/1115835653","heartbeat_front_addr":"172.21.15.46:6815/1115835653","state":["exists","up"]},{"osd":4,"uuid":"444cd36b-6034-46cd-9a6b-29dce4ba6405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6809","nonce":59186693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6811","nonce":59186693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6815","nonce":59186693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6813","nonce":59186693}]},"public_addr":"172.21.15.72:6809/59186693","cluster_addr":"172.21.15.72:6811/59186693","heartbeat_back_addr":"172.21.15.72:6815/59186693","heartbeat_front_addr":"172.21.15.72:6813/59186693","state":["exists","up"]},{"osd":5,"uuid":"2c2bdd2f-43cf-4208-9ae1-f912807c84e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6809","nonce":1712162502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6811","nonce":1712162502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6815","nonce":1712162502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6813","nonce":1712162502}]},"public_addr":"172.21.15.110:6809/1712162502","cluster_addr":"172.21.15.110:6811/1712162502","heartbeat_back_addr":"172.21.15.110:6815/1712162502","heartbeat_front_addr":"172.21.15.110:6813/1712162502","state":["exists","up"]},{"osd":6,"uuid":"beb0b87f-c3a4-40dd-b224-1843e9301ca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6819","nonce":2429650065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6821","nonce":2429650065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6825","nonce":2429650065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6823","nonce":2429650065}]},"public_addr":"172.21.15.46:6819/2429650065","cluster_addr":"172.21.15.46:6821/2429650065","heartbeat_back_addr":"172.21.15.46:6825/2429650065","heartbeat_front_addr":"172.21.15.46:6823/2429650065","state":["exists","up"]},{"osd":7,"uuid":"4a634675-fad7-4a43-840c-8dec2631dec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6817","nonce":337796672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6819","nonce":337796672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6823","nonce":337796672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6821","nonce":337796672}]},"public_addr":"172.21.15.72:6817/337796672","cluster_addr":"172.21.15.72:6819/337796672","heartbeat_back_addr":"172.21.15.72:6823/337796672","heartbeat_front_addr":"172.21.15.72:6821/337796672","state":["exists","up"]},{"osd":8,"uuid":"01b5f638-90f0-4cf4-8042-20d20a725669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6817","nonce":189513203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6819","nonce":189513203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6823","nonce":189513203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6821","nonce":189513203}]},"public_addr":"172.21.15.110:6817/189513203","cluster_addr":"172.21.15.110:6819/189513203","heartbeat_back_addr":"172.21.15.110:6823/189513203","heartbeat_front_addr":"172.21.15.110:6821/189513203","state":["exists","up"]},{"osd":9,"uuid":"e423de41-50ec-48f4-a4ed-216de33c4fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6827","nonce":1762262822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6829","nonce":1762262822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6833","nonce":1762262822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6831","nonce":1762262822}]},"public_addr":"172.21.15.46:6827/1762262822","cluster_addr":"172.21.15.46:6829/1762262822","heartbeat_back_addr":"172.21.15.46:6833/1762262822","heartbeat_front_addr":"172.21.15.46:6831/1762262822","state":["exists","up"]},{"osd":10,"uuid":"f594d004-9f80-4508-b127-fb693d355f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6825","nonce":38453898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6827","nonce":38453898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6831","nonce":38453898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6829","nonce":38453898}]},"public_addr":"172.21.15.72:6825/38453898","cluster_addr":"172.21.15.72:6827/38453898","heartbeat_back_addr":"172.21.15.72:6831/38453898","heartbeat_front_addr":"172.21.15.72:6829/38453898","state":["exists","up"]},{"osd":11,"uuid":"f771d29f-89e3-4f96-9c29-00e424230c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6825","nonce":4153529632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6827","nonce":4153529632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6831","nonce":4153529632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6829","nonce":4153529632}]},"public_addr":"172.21.15.110:6825/4153529632","cluster_addr":"172.21.15.110:6827/4153529632","heartbeat_back_addr":"172.21.15.110:6831/4153529632","heartbeat_front_addr":"172.21.15.110:6829/4153529632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:02.615801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:26.453477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:50.334627+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:15.357320+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:40.403549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:06.030093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:31.904091+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:57.616372+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:24.045975+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:51.160898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:17.597845+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:44.636629+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/181034377":"2024-05-10T04:51:05.539029+0000","172.21.15.46:0/1963687293":"2024-05-10T04:51:05.539029+0000","172.21.15.46:0/2215389074":"2024-05-10T04:51:05.539029+0000","172.21.15.46:6800/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1596672353":"2024-05-10T04:40:27.201119+0000","172.21.15.46:6801/3691531627":"2024-05-10T04:51:05.539029+0000","172.21.15.46:0/1833069872":"2024-05-10T04:51:05.539029+0000","172.21.15.46:0/1846482551":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/1069571878":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6800/2738158981":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/3625745146":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6800/3691531627":"2024-05-10T04:51:05.539029+0000","172.21.15.46:6801/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/564539954":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1797199863":"2024-05-10T04:40:27.201119+0000","172.21.15.46:6801/2738158981":"2024-05-10T04:39:59.515628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T04:51:14.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.672+0000 7f8fb1c4c700 1 -- 172.21.15.46:0/1107902902 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8f980b2b00 msgr2=0x7f8f980b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.672+0000 7f8fb1c4c700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8f980b2b00 0x7f8f980b5010 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8fac101dc0 tx=0x7f8f9c009260 comp rx=0 tx=0).stop 2024-05-09T04:51:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.672+0000 7f8fb1c4c700 1 -- 172.21.15.46:0/1107902902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fac102ae0 msgr2=0x7f8fac19ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.672+0000 7f8fb1c4c700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fac102ae0 0x7f8fac19ad90 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f8f9400e9a0 tx=0x7f8f9400ed60 comp rx=0 tx=0).stop 2024-05-09T04:51:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.673+0000 7f8fb1c4c700 1 -- 172.21.15.46:0/1107902902 shutdown_connections 2024-05-09T04:51:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.673+0000 7f8fb1c4c700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8f980b2b00 0x7f8f980b5010 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.673+0000 7f8fb1c4c700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8fac109850 0x7f8fac19f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.673+0000 7f8fb1c4c700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8fac102ae0 0x7f8fac19ad90 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.673+0000 7f8fb1c4c700 1 --2- 172.21.15.46:0/1107902902 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8fac108f00 0x7f8fac19b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.673+0000 7f8fb1c4c700 1 -- 172.21.15.46:0/1107902902 >> 172.21.15.46:0/1107902902 conn(0x7f8fac0fe4e0 msgr2=0x7f8fac108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.673+0000 7f8fb1c4c700 1 -- 172.21.15.46:0/1107902902 shutdown_connections 2024-05-09T04:51:14.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:14.673+0000 7f8fb1c4c700 1 -- 172.21.15.46:0/1107902902 wait complete. 2024-05-09T04:51:15.047 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:14 smithi046 ceph-mon[30136]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-09T04:51:15.048 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:14 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1107902902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:51:15.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:14 smithi110 ceph-mon[34904]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-09T04:51:15.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:14 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1107902902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:51:15.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:14 smithi072 ceph-mon[34088]: pgmap v7: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 0 B/s wr, 15 op/s 2024-05-09T04:51:15.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:14 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1107902902' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T04:51:15.366 DEBUG:tasks.fwd_scrub:Yielding 2024-05-09T04:51:15.367 INFO:teuthology.run_tasks:Running task exec... 2024-05-09T04:51:15.377 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T04:51:15.377 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T04:51:15.377 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-09T04:51:15.378 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-05-09T04:51:15.378 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:51:15.378 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:51:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.507+0000 7fd297d94640 1 -- 172.21.15.46:0/3462767076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd290109060 msgr2=0x7fd290102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.507+0000 7fd297d94640 1 --2- 172.21.15.46:0/3462767076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd290109060 0x7fd290102ca0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fd28c009900 tx=0x7fd28c02a410 comp rx=0 tx=0).stop 2024-05-09T04:51:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.507+0000 7fd297d94640 1 -- 172.21.15.46:0/3462767076 shutdown_connections 2024-05-09T04:51:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.507+0000 7fd297d94640 1 --2- 172.21.15.46:0/3462767076 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd290103310 0x7fd29010f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.507+0000 7fd297d94640 1 --2- 172.21.15.46:0/3462767076 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd290109060 0x7fd290102ca0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.507+0000 7fd297d94640 1 --2- 172.21.15.46:0/3462767076 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd290108bc0 0x7fd290102760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.507+0000 7fd297d94640 1 -- 172.21.15.46:0/3462767076 >> 172.21.15.46:0/3462767076 conn(0x7fd2900781b0 msgr2=0x7fd290078610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.507+0000 7fd297d94640 1 -- 172.21.15.46:0/3462767076 shutdown_connections 2024-05-09T04:51:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.507+0000 7fd297d94640 1 -- 172.21.15.46:0/3462767076 wait complete. 2024-05-09T04:51:15.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.508+0000 7fd297d94640 1 Processor -- start 2024-05-09T04:51:15.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.508+0000 7fd297d94640 1 -- start start 2024-05-09T04:51:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.508+0000 7fd297d94640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd290103310 0x7fd29019b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.508+0000 7fd297d94640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd290108bc0 0x7fd29019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.508+0000 7fd297d94640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd290109060 0x7fd29019c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.508+0000 7fd297d94640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd290112db0 con 0x7fd290109060 2024-05-09T04:51:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.508+0000 7fd297d94640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd290112c30 con 0x7fd290103310 2024-05-09T04:51:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.508+0000 7fd297d94640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd290112f30 con 0x7fd290108bc0 2024-05-09T04:51:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd296331640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd290109060 0x7fd29019c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd296331640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd290109060 0x7fd29019c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44358/0 (socket says 172.21.15.46:44358) 2024-05-09T04:51:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd296331640 1 -- 172.21.15.46:0/3369098194 learned_addr learned my addr 172.21.15.46:0/3369098194 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd295b30640 1 --2- 172.21.15.46:0/3369098194 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd290103310 0x7fd29019b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd29532f640 1 --2- 172.21.15.46:0/3369098194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd290108bc0 0x7fd29019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd296331640 1 -- 172.21.15.46:0/3369098194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd290108bc0 msgr2=0x7fd29019bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd296331640 1 --2- 172.21.15.46:0/3369098194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd290108bc0 0x7fd29019bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd296331640 1 -- 172.21.15.46:0/3369098194 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd290103310 msgr2=0x7fd29019b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:15.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd296331640 1 --2- 172.21.15.46:0/3369098194 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd290103310 0x7fd29019b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd296331640 1 -- 172.21.15.46:0/3369098194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2901a6a50 con 0x7fd290109060 2024-05-09T04:51:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd296331640 1 --2- 172.21.15.46:0/3369098194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd290109060 0x7fd29019c1b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd28800e970 tx=0x7fd28800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd282ffd640 1 -- 172.21.15.46:0/3369098194 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd28800ccc0 con 0x7fd290109060 2024-05-09T04:51:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd297d94640 1 -- 172.21.15.46:0/3369098194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2901a6d40 con 0x7fd290109060 2024-05-09T04:51:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd282ffd640 1 -- 172.21.15.46:0/3369098194 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd28800ce60 con 0x7fd290109060 2024-05-09T04:51:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd282ffd640 1 -- 172.21.15.46:0/3369098194 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd288010430 con 0x7fd290109060 2024-05-09T04:51:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.509+0000 7fd297d94640 1 -- 172.21.15.46:0/3369098194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2901a71b0 con 0x7fd290109060 2024-05-09T04:51:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.510+0000 7fd297d94640 1 -- 172.21.15.46:0/3369098194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd290050870 con 0x7fd290109060 2024-05-09T04:51:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.512+0000 7fd282ffd640 1 -- 172.21.15.46:0/3369098194 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fd288004fb0 con 0x7fd290109060 2024-05-09T04:51:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.513+0000 7fd282ffd640 1 --2- 172.21.15.46:0/3369098194 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd27c0b29e0 0x7fd27c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.513+0000 7fd295b30640 1 --2- 172.21.15.46:0/3369098194 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd27c0b29e0 0x7fd27c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.513+0000 7fd282ffd640 1 -- 172.21.15.46:0/3369098194 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fd2880e2e60 con 0x7fd290109060 2024-05-09T04:51:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.514+0000 7fd295b30640 1 --2- 172.21.15.46:0/3369098194 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd27c0b29e0 0x7fd27c0b4ef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd2901038a0 tx=0x7fd284009500 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.517+0000 7fd282ffd640 1 -- 172.21.15.46:0/3369098194 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd288054df0 con 0x7fd290109060 2024-05-09T04:51:15.645 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:15.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.721+0000 7fd297d94640 1 -- 172.21.15.46:0/3369098194 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7fd29006b990 con 0x7fd290109060 2024-05-09T04:51:15.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.725+0000 7fd282ffd640 1 -- 172.21.15.46:0/3369098194 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v27)=0 v27) ==== 129+0+0 (secure 0 0 0) 0x7fd2880aa650 con 0x7fd290109060 2024-05-09T04:51:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.728+0000 7fd297d94640 1 -- 172.21.15.46:0/3369098194 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd27c0b29e0 msgr2=0x7fd27c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.728+0000 7fd297d94640 1 --2- 172.21.15.46:0/3369098194 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd27c0b29e0 0x7fd27c0b4ef0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd2901038a0 tx=0x7fd284009500 comp rx=0 tx=0).stop 2024-05-09T04:51:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.728+0000 7fd297d94640 1 -- 172.21.15.46:0/3369098194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd290109060 msgr2=0x7fd29019c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.728+0000 7fd297d94640 1 --2- 172.21.15.46:0/3369098194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd290109060 0x7fd29019c1b0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fd28800e970 tx=0x7fd28800ed70 comp rx=0 tx=0).stop 2024-05-09T04:51:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.728+0000 7fd297d94640 1 -- 172.21.15.46:0/3369098194 shutdown_connections 2024-05-09T04:51:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.728+0000 7fd297d94640 1 --2- 172.21.15.46:0/3369098194 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd27c0b29e0 0x7fd27c0b4ef0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.728+0000 7fd297d94640 1 --2- 172.21.15.46:0/3369098194 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd290108bc0 0x7fd29019bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.728+0000 7fd297d94640 1 --2- 172.21.15.46:0/3369098194 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd290109060 0x7fd29019c1b0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.728+0000 7fd297d94640 1 --2- 172.21.15.46:0/3369098194 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd290103310 0x7fd29019b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.728+0000 7fd297d94640 1 -- 172.21.15.46:0/3369098194 >> 172.21.15.46:0/3369098194 conn(0x7fd2900781b0 msgr2=0x7fd2901012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.728+0000 7fd297d94640 1 -- 172.21.15.46:0/3369098194 shutdown_connections 2024-05-09T04:51:15.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.728+0000 7fd297d94640 1 -- 172.21.15.46:0/3369098194 wait complete. 2024-05-09T04:51:15.740 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-09T04:51:15.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.864+0000 7fe9d9338640 1 -- 172.21.15.46:0/2141821083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d410e210 msgr2=0x7fe9d4110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:15.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.864+0000 7fe9d9338640 1 --2- 172.21.15.46:0/2141821083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d410e210 0x7fe9d4110700 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe9c8009900 tx=0x7fe9c802a510 comp rx=0 tx=0).stop 2024-05-09T04:51:15.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.864+0000 7fe9d9338640 1 -- 172.21.15.46:0/2141821083 shutdown_connections 2024-05-09T04:51:15.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.864+0000 7fe9d9338640 1 --2- 172.21.15.46:0/2141821083 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d410e210 0x7fe9d4110700 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.864+0000 7fe9d9338640 1 --2- 172.21.15.46:0/2141821083 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d410b860 0x7fe9d410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.864+0000 7fe9d9338640 1 --2- 172.21.15.46:0/2141821083 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9d4104e00 0x7fe9d4105210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.864+0000 7fe9d9338640 1 -- 172.21.15.46:0/2141821083 >> 172.21.15.46:0/2141821083 conn(0x7fe9d40782e0 msgr2=0x7fe9d4078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:15.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.864+0000 7fe9d9338640 1 -- 172.21.15.46:0/2141821083 shutdown_connections 2024-05-09T04:51:15.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.864+0000 7fe9d9338640 1 -- 172.21.15.46:0/2141821083 wait complete. 2024-05-09T04:51:15.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.865+0000 7fe9d9338640 1 Processor -- start 2024-05-09T04:51:15.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.865+0000 7fe9d9338640 1 -- start start 2024-05-09T04:51:15.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.865+0000 7fe9d9338640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d4104e00 0x7fe9d41026c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:15.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.865+0000 7fe9d9338640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9d410b860 0x7fe9d4102c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:15.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.865+0000 7fe9d9338640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d410e210 0x7fe9d4103140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.865+0000 7fe9d9338640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d4113e70 con 0x7fe9d4104e00 2024-05-09T04:51:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.865+0000 7fe9d9338640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d4113cf0 con 0x7fe9d410e210 2024-05-09T04:51:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.865+0000 7fe9d9338640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe9d4113ff0 con 0x7fe9d410b860 2024-05-09T04:51:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.865+0000 7fe9d2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d4104e00 0x7fe9d41026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.865+0000 7fe9d2ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d4104e00 0x7fe9d41026c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44384/0 (socket says 172.21.15.46:44384) 2024-05-09T04:51:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.865+0000 7fe9d2ffd640 1 -- 172.21.15.46:0/4280191235 learned_addr learned my addr 172.21.15.46:0/4280191235 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.866+0000 7fe9d37fe640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d410e210 0x7fe9d4103140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.866+0000 7fe9d27fc640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9d410b860 0x7fe9d4102c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.866+0000 7fe9d2ffd640 1 -- 172.21.15.46:0/4280191235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9d410b860 msgr2=0x7fe9d4102c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.866+0000 7fe9d2ffd640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9d410b860 0x7fe9d4102c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.866+0000 7fe9d2ffd640 1 -- 172.21.15.46:0/4280191235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d410e210 msgr2=0x7fe9d4103140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.866+0000 7fe9d2ffd640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d410e210 0x7fe9d4103140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.866+0000 7fe9d2ffd640 1 -- 172.21.15.46:0/4280191235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9d41a8bb0 con 0x7fe9d4104e00 2024-05-09T04:51:15.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.866+0000 7fe9d27fc640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9d410b860 0x7fe9d4102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:51:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.866+0000 7fe9d37fe640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d410e210 0x7fe9d4103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:51:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.866+0000 7fe9d2ffd640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d4104e00 0x7fe9d41026c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00eb90 tx=0x7fe9cc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.866+0000 7fe9bbfff640 1 -- 172.21.15.46:0/4280191235 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc004d20 con 0x7fe9d4104e00 2024-05-09T04:51:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.866+0000 7fe9bbfff640 1 -- 172.21.15.46:0/4280191235 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9cc00ee90 con 0x7fe9d4104e00 2024-05-09T04:51:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.866+0000 7fe9d9338640 1 -- 172.21.15.46:0/4280191235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9d41a8ea0 con 0x7fe9d4104e00 2024-05-09T04:51:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.866+0000 7fe9bbfff640 1 -- 172.21.15.46:0/4280191235 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9cc010670 con 0x7fe9d4104e00 2024-05-09T04:51:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.866+0000 7fe9d9338640 1 -- 172.21.15.46:0/4280191235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9d41a9310 con 0x7fe9d4104e00 2024-05-09T04:51:15.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.867+0000 7fe9d9338640 1 -- 172.21.15.46:0/4280191235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9d406a240 con 0x7fe9d4104e00 2024-05-09T04:51:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.869+0000 7fe9bbfff640 1 -- 172.21.15.46:0/4280191235 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fe9cc005020 con 0x7fe9d4104e00 2024-05-09T04:51:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.869+0000 7fe9bbfff640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe9bc0b29e0 0x7fe9bc0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.870+0000 7fe9d27fc640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe9bc0b29e0 0x7fe9bc0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.870+0000 7fe9bbfff640 1 -- 172.21.15.46:0/4280191235 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fe9cc0e2f80 con 0x7fe9d4104e00 2024-05-09T04:51:15.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.870+0000 7fe9d27fc640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe9bc0b29e0 0x7fe9bc0b4ef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe9d410ebc0 tx=0x7fe9c4005f90 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:15.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:15.874+0000 7fe9bbfff640 1 -- 172.21.15.46:0/4280191235 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe9cc0aa7d0 con 0x7fe9d4104e00 2024-05-09T04:51:16.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.082+0000 7fe9d9338640 1 -- 172.21.15.46:0/4280191235 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fe9d41a95a0 con 0x7fe9d4104e00 2024-05-09T04:51:16.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.087+0000 7fe9bbfff640 1 -- 172.21.15.46:0/4280191235 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v28)=0 v28) ==== 138+0+0 (secure 0 0 0) 0x7fe9cc0aa0d0 con 0x7fe9d4104e00 2024-05-09T04:51:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.089+0000 7fe9d9338640 1 -- 172.21.15.46:0/4280191235 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe9bc0b29e0 msgr2=0x7fe9bc0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:16.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.089+0000 7fe9d9338640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe9bc0b29e0 0x7fe9bc0b4ef0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe9d410ebc0 tx=0x7fe9c4005f90 comp rx=0 tx=0).stop 2024-05-09T04:51:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.089+0000 7fe9d9338640 1 -- 172.21.15.46:0/4280191235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d4104e00 msgr2=0x7fe9d41026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.089+0000 7fe9d9338640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d4104e00 0x7fe9d41026c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe9cc00eb90 tx=0x7fe9cc00c6a0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.089+0000 7fe9d9338640 1 -- 172.21.15.46:0/4280191235 shutdown_connections 2024-05-09T04:51:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.089+0000 7fe9d9338640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe9bc0b29e0 0x7fe9bc0b4ef0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.089+0000 7fe9d9338640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe9d410b860 0x7fe9d4102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.089+0000 7fe9d9338640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe9d410e210 0x7fe9d4103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.089+0000 7fe9d9338640 1 --2- 172.21.15.46:0/4280191235 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe9d4104e00 0x7fe9d41026c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.089+0000 7fe9d9338640 1 -- 172.21.15.46:0/4280191235 >> 172.21.15.46:0/4280191235 conn(0x7fe9d40782e0 msgr2=0x7fe9d41012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.089+0000 7fe9d9338640 1 -- 172.21.15.46:0/4280191235 shutdown_connections 2024-05-09T04:51:16.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.089+0000 7fe9d9338640 1 -- 172.21.15.46:0/4280191235 wait complete. 2024-05-09T04:51:16.102 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-09T04:51:16.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.219+0000 7f4578727640 1 -- 172.21.15.46:0/531561727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45701043f0 msgr2=0x7f45701107c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:16.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.219+0000 7f4578727640 1 --2- 172.21.15.46:0/531561727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45701043f0 0x7f45701107c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f4568009900 tx=0x7f456802a410 comp rx=0 tx=0).stop 2024-05-09T04:51:16.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.219+0000 7f4578727640 1 -- 172.21.15.46:0/531561727 shutdown_connections 2024-05-09T04:51:16.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.219+0000 7f4578727640 1 --2- 172.21.15.46:0/531561727 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f457010a140 0x7f4570103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.219+0000 7f4578727640 1 --2- 172.21.15.46:0/531561727 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f45701043f0 0x7f45701107c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.219+0000 7f4578727640 1 --2- 172.21.15.46:0/531561727 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4570109ca0 0x7f4570103840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.219+0000 7f4578727640 1 -- 172.21.15.46:0/531561727 >> 172.21.15.46:0/531561727 conn(0x7f45700ff340 msgr2=0x7f4570101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:16.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.219+0000 7f4578727640 1 -- 172.21.15.46:0/531561727 shutdown_connections 2024-05-09T04:51:16.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.219+0000 7f4578727640 1 -- 172.21.15.46:0/531561727 wait complete. 2024-05-09T04:51:16.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.220+0000 7f4578727640 1 Processor -- start 2024-05-09T04:51:16.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.220+0000 7f4578727640 1 -- start start 2024-05-09T04:51:16.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.220+0000 7f4578727640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45701043f0 0x7f45701a0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:16.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.220+0000 7f4578727640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4570109ca0 0x7f45701a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:16.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.220+0000 7f4578727640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457010a140 0x7f457019a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:16.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.220+0000 7f4578727640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4570113eb0 con 0x7f457010a140 2024-05-09T04:51:16.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.220+0000 7f4578727640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4570113d30 con 0x7f45701043f0 2024-05-09T04:51:16.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.220+0000 7f4578727640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4570114030 con 0x7f4570109ca0 2024-05-09T04:51:16.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.220+0000 7f4576cc4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457010a140 0x7f457019a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:16.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.220+0000 7f45764c3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45701043f0 0x7f45701a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.220+0000 7f4576cc4640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457010a140 0x7f457019a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44406/0 (socket says 172.21.15.46:44406) 2024-05-09T04:51:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.220+0000 7f45764c3640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45701043f0 0x7f45701a0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:40910/0 (socket says 172.21.15.46:40910) 2024-05-09T04:51:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.220+0000 7f4576cc4640 1 -- 172.21.15.46:0/945601634 learned_addr learned my addr 172.21.15.46:0/945601634 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.220+0000 7f4575cc2640 1 --2- 172.21.15.46:0/945601634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4570109ca0 0x7f45701a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.221+0000 7f45764c3640 1 -- 172.21.15.46:0/945601634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4570109ca0 msgr2=0x7f45701a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.221+0000 7f45764c3640 1 --2- 172.21.15.46:0/945601634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4570109ca0 0x7f45701a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.221+0000 7f45764c3640 1 -- 172.21.15.46:0/945601634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457010a140 msgr2=0x7f457019a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.221+0000 7f45764c3640 1 --2- 172.21.15.46:0/945601634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457010a140 0x7f457019a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.221+0000 7f45764c3640 1 -- 172.21.15.46:0/945601634 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f457019ae40 con 0x7f45701043f0 2024-05-09T04:51:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.221+0000 7f4576cc4640 1 --2- 172.21.15.46:0/945601634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457010a140 0x7f457019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.221+0000 7f45764c3640 1 --2- 172.21.15.46:0/945601634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45701043f0 0x7f45701a0540 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f456c005fd0 tx=0x7f456c00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.221+0000 7f45677fe640 1 -- 172.21.15.46:0/945601634 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c010430 con 0x7f45701043f0 2024-05-09T04:51:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.221+0000 7f4578727640 1 -- 172.21.15.46:0/945601634 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f457019b0d0 con 0x7f45701043f0 2024-05-09T04:51:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.221+0000 7f45677fe640 1 -- 172.21.15.46:0/945601634 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f456c010a80 con 0x7f45701043f0 2024-05-09T04:51:16.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.221+0000 7f45677fe640 1 -- 172.21.15.46:0/945601634 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456c017a50 con 0x7f45701043f0 2024-05-09T04:51:16.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.221+0000 7f4578727640 1 -- 172.21.15.46:0/945601634 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45701a72e0 con 0x7f45701043f0 2024-05-09T04:51:16.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.222+0000 7f4578727640 1 -- 172.21.15.46:0/945601634 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4570104f80 con 0x7f45701043f0 2024-05-09T04:51:16.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.224+0000 7f45677fe640 1 -- 172.21.15.46:0/945601634 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f456c017c10 con 0x7f45701043f0 2024-05-09T04:51:16.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.225+0000 7f45677fe640 1 --2- 172.21.15.46:0/945601634 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f455c0b29e0 0x7f455c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:16.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.225+0000 7f4575cc2640 1 --2- 172.21.15.46:0/945601634 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f455c0b29e0 0x7f455c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:16.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.225+0000 7f45677fe640 1 -- 172.21.15.46:0/945601634 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f456c0e3290 con 0x7f45701043f0 2024-05-09T04:51:16.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.225+0000 7f4575cc2640 1 --2- 172.21.15.46:0/945601634 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f455c0b29e0 0x7f455c0b4ef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f457019be60 tx=0x7f456000b390 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:16.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.229+0000 7f45677fe640 1 -- 172.21.15.46:0/945601634 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f456c0aab10 con 0x7f45701043f0 2024-05-09T04:51:16.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.432+0000 7f4578727640 1 -- 172.21.15.46:0/945601634 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f457010b820 con 0x7f45701043f0 2024-05-09T04:51:16.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.442+0000 7f45677fe640 1 -- 172.21.15.46:0/945601634 <== mon.2 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v29)=0 v29) ==== 133+0+0 (secure 0 0 0) 0x7f456c0aa410 con 0x7f45701043f0 2024-05-09T04:51:16.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.444+0000 7f4578727640 1 -- 172.21.15.46:0/945601634 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f455c0b29e0 msgr2=0x7f455c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:16.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.444+0000 7f4578727640 1 --2- 172.21.15.46:0/945601634 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f455c0b29e0 0x7f455c0b4ef0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f457019be60 tx=0x7f456000b390 comp rx=0 tx=0).stop 2024-05-09T04:51:16.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.444+0000 7f4578727640 1 -- 172.21.15.46:0/945601634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45701043f0 msgr2=0x7f45701a0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:16.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.444+0000 7f4578727640 1 --2- 172.21.15.46:0/945601634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45701043f0 0x7f45701a0540 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f456c005fd0 tx=0x7f456c00ef90 comp rx=0 tx=0).stop 2024-05-09T04:51:16.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.444+0000 7f4578727640 1 -- 172.21.15.46:0/945601634 shutdown_connections 2024-05-09T04:51:16.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.444+0000 7f4578727640 1 --2- 172.21.15.46:0/945601634 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f455c0b29e0 0x7f455c0b4ef0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.444+0000 7f4578727640 1 --2- 172.21.15.46:0/945601634 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4570109ca0 0x7f45701a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.444+0000 7f4578727640 1 --2- 172.21.15.46:0/945601634 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f457010a140 0x7f457019a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.444+0000 7f4578727640 1 --2- 172.21.15.46:0/945601634 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f45701043f0 0x7f45701a0540 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.444+0000 7f4578727640 1 -- 172.21.15.46:0/945601634 >> 172.21.15.46:0/945601634 conn(0x7f45700ff340 msgr2=0x7f45701066f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:16.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.444+0000 7f4578727640 1 -- 172.21.15.46:0/945601634 shutdown_connections 2024-05-09T04:51:16.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.444+0000 7f4578727640 1 -- 172.21.15.46:0/945601634 wait complete. 2024-05-09T04:51:16.457 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-09T04:51:16.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.579+0000 7fa89d6e0640 1 -- 172.21.15.46:0/2948382108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa898104420 msgr2=0x7fa8981107f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:16.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.579+0000 7fa89d6e0640 1 --2- 172.21.15.46:0/2948382108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa898104420 0x7fa8981107f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa890009ca0 tx=0x7fa89002a8c0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.579+0000 7fa89d6e0640 1 -- 172.21.15.46:0/2948382108 shutdown_connections 2024-05-09T04:51:16.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.579+0000 7fa89d6e0640 1 --2- 172.21.15.46:0/2948382108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa898104420 0x7fa8981107f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.579+0000 7fa89d6e0640 1 --2- 172.21.15.46:0/2948382108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa89810a170 0x7fa898103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.579+0000 7fa89d6e0640 1 --2- 172.21.15.46:0/2948382108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa898109cd0 0x7fa898103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.579+0000 7fa89d6e0640 1 -- 172.21.15.46:0/2948382108 >> 172.21.15.46:0/2948382108 conn(0x7fa8980793d0 msgr2=0x7fa898079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:16.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.579+0000 7fa89d6e0640 1 -- 172.21.15.46:0/2948382108 shutdown_connections 2024-05-09T04:51:16.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.579+0000 7fa89d6e0640 1 -- 172.21.15.46:0/2948382108 wait complete. 2024-05-09T04:51:16.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.580+0000 7fa89d6e0640 1 Processor -- start 2024-05-09T04:51:16.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.580+0000 7fa89d6e0640 1 -- start start 2024-05-09T04:51:16.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.580+0000 7fa89d6e0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa898104420 0x7fa8981a0500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:16.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.580+0000 7fa89d6e0640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa898109cd0 0x7fa8981a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:16.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.580+0000 7fa89d6e0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa89810a170 0x7fa89819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.580+0000 7fa89d6e0640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa898113ec0 con 0x7fa898104420 2024-05-09T04:51:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.580+0000 7fa89d6e0640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa898113d40 con 0x7fa898109cd0 2024-05-09T04:51:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.580+0000 7fa89d6e0640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa898114040 con 0x7fa89810a170 2024-05-09T04:51:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.580+0000 7fa896ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa898104420 0x7fa8981a0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.580+0000 7fa896ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa898104420 0x7fa8981a0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44438/0 (socket says 172.21.15.46:44438) 2024-05-09T04:51:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.580+0000 7fa896ffd640 1 -- 172.21.15.46:0/2487216882 learned_addr learned my addr 172.21.15.46:0/2487216882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.580+0000 7fa8967fc640 1 --2- 172.21.15.46:0/2487216882 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa898109cd0 0x7fa8981a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.580+0000 7fa8977fe640 1 --2- 172.21.15.46:0/2487216882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa89810a170 0x7fa89819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.581+0000 7fa896ffd640 1 -- 172.21.15.46:0/2487216882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa89810a170 msgr2=0x7fa89819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.581+0000 7fa896ffd640 1 --2- 172.21.15.46:0/2487216882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa89810a170 0x7fa89819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.581+0000 7fa896ffd640 1 -- 172.21.15.46:0/2487216882 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa898109cd0 msgr2=0x7fa8981a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.581+0000 7fa896ffd640 1 --2- 172.21.15.46:0/2487216882 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa898109cd0 0x7fa8981a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.581+0000 7fa896ffd640 1 -- 172.21.15.46:0/2487216882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa89819adb0 con 0x7fa898104420 2024-05-09T04:51:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.581+0000 7fa896ffd640 1 --2- 172.21.15.46:0/2487216882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa898104420 0x7fa8981a0500 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa88800d770 tx=0x7fa88800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.581+0000 7fa87ffff640 1 -- 172.21.15.46:0/2487216882 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa888017070 con 0x7fa898104420 2024-05-09T04:51:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.581+0000 7fa89d6e0640 1 -- 172.21.15.46:0/2487216882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa89819b040 con 0x7fa898104420 2024-05-09T04:51:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.581+0000 7fa87ffff640 1 -- 172.21.15.46:0/2487216882 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa888009c30 con 0x7fa898104420 2024-05-09T04:51:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.581+0000 7fa87ffff640 1 -- 172.21.15.46:0/2487216882 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa88800f620 con 0x7fa898104420 2024-05-09T04:51:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.581+0000 7fa89d6e0640 1 -- 172.21.15.46:0/2487216882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8981a72a0 con 0x7fa898104420 2024-05-09T04:51:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.582+0000 7fa89d6e0640 1 -- 172.21.15.46:0/2487216882 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa898104fb0 con 0x7fa898104420 2024-05-09T04:51:16.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.584+0000 7fa87ffff640 1 -- 172.21.15.46:0/2487216882 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fa88801a070 con 0x7fa898104420 2024-05-09T04:51:16.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.585+0000 7fa87ffff640 1 --2- 172.21.15.46:0/2487216882 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa8800b29e0 0x7fa8800b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:16.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.585+0000 7fa8967fc640 1 --2- 172.21.15.46:0/2487216882 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa8800b29e0 0x7fa8800b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:16.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.585+0000 7fa87ffff640 1 -- 172.21.15.46:0/2487216882 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa8880e2200 con 0x7fa898104420 2024-05-09T04:51:16.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.585+0000 7fa8967fc640 1 --2- 172.21.15.46:0/2487216882 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa8800b29e0 0x7fa8800b4ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa89819be20 tx=0x7fa88c005f90 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:16.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.589+0000 7fa87ffff640 1 -- 172.21.15.46:0/2487216882 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa888009de0 con 0x7fa898104420 2024-05-09T04:51:16.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.807+0000 7fa89d6e0640 1 -- 172.21.15.46:0/2487216882 --> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fa898063110 con 0x7fa8800b29e0 2024-05-09T04:51:16.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.881+0000 7fa87ffff640 1 -- 172.21.15.46:0/2487216882 <== mgr.15150 v2:172.21.15.46:6800/2595796783 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fa898063110 con 0x7fa8800b29e0 2024-05-09T04:51:16.882 INFO:teuthology.orchestra.run.smithi046.stdout:Schedule set for path / 2024-05-09T04:51:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.884+0000 7fa89d6e0640 1 -- 172.21.15.46:0/2487216882 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa8800b29e0 msgr2=0x7fa8800b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.884+0000 7fa89d6e0640 1 --2- 172.21.15.46:0/2487216882 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa8800b29e0 0x7fa8800b4ef0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa89819be20 tx=0x7fa88c005f90 comp rx=0 tx=0).stop 2024-05-09T04:51:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.884+0000 7fa89d6e0640 1 -- 172.21.15.46:0/2487216882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa898104420 msgr2=0x7fa8981a0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.884+0000 7fa89d6e0640 1 --2- 172.21.15.46:0/2487216882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa898104420 0x7fa8981a0500 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fa88800d770 tx=0x7fa88800dc30 comp rx=0 tx=0).stop 2024-05-09T04:51:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.884+0000 7fa89d6e0640 1 -- 172.21.15.46:0/2487216882 shutdown_connections 2024-05-09T04:51:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.884+0000 7fa89d6e0640 1 --2- 172.21.15.46:0/2487216882 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa8800b29e0 0x7fa8800b4ef0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.884+0000 7fa89d6e0640 1 --2- 172.21.15.46:0/2487216882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa89810a170 0x7fa89819a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.884+0000 7fa89d6e0640 1 --2- 172.21.15.46:0/2487216882 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa898109cd0 0x7fa8981a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.884+0000 7fa89d6e0640 1 --2- 172.21.15.46:0/2487216882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa898104420 0x7fa8981a0500 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.884+0000 7fa89d6e0640 1 -- 172.21.15.46:0/2487216882 >> 172.21.15.46:0/2487216882 conn(0x7fa8980793d0 msgr2=0x7fa898101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:16.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.884+0000 7fa89d6e0640 1 -- 172.21.15.46:0/2487216882 shutdown_connections 2024-05-09T04:51:16.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:16.884+0000 7fa89d6e0640 1 -- 172.21.15.46:0/2487216882 wait complete. 2024-05-09T04:51:16.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-09T04:51:17.008 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-09T04:51:17.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3369098194' entity='client.admin' 2024-05-09T04:51:17.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:17.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:17.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:17.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:17.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/4280191235' entity='client.admin' 2024-05-09T04:51:17.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:17.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:17.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:17.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:17.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: from='client.? ' entity='client.admin' 2024-05-09T04:51:17.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:17.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:17.010 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:17.010 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:16 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:17.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.017+0000 7f815456c640 1 -- 172.21.15.46:0/3150167714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c109cd0 msgr2=0x7f814c103870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.017+0000 7f815456c640 1 --2- 172.21.15.46:0/3150167714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c109cd0 0x7f814c103870 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8148009900 tx=0x7f814802a410 comp rx=0 tx=0).stop 2024-05-09T04:51:17.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.017+0000 7f815456c640 1 -- 172.21.15.46:0/3150167714 shutdown_connections 2024-05-09T04:51:17.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.017+0000 7f815456c640 1 --2- 172.21.15.46:0/3150167714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f814c104420 0x7f814c1107f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.017+0000 7f815456c640 1 --2- 172.21.15.46:0/3150167714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f814c10a170 0x7f814c103db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.017+0000 7f815456c640 1 --2- 172.21.15.46:0/3150167714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c109cd0 0x7f814c103870 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.017+0000 7f815456c640 1 -- 172.21.15.46:0/3150167714 >> 172.21.15.46:0/3150167714 conn(0x7f814c0793d0 msgr2=0x7f814c079810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:17.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.017+0000 7f815456c640 1 -- 172.21.15.46:0/3150167714 shutdown_connections 2024-05-09T04:51:17.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.017+0000 7f815456c640 1 -- 172.21.15.46:0/3150167714 wait complete. 2024-05-09T04:51:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.018+0000 7f815456c640 1 Processor -- start 2024-05-09T04:51:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.018+0000 7f815456c640 1 -- start start 2024-05-09T04:51:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.018+0000 7f815456c640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f814c104420 0x7f814c1a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.018+0000 7f815456c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c109cd0 0x7f814c1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.018+0000 7f815456c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f814c10a170 0x7f814c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.018+0000 7f815456c640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f814c113f10 con 0x7f814c109cd0 2024-05-09T04:51:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.018+0000 7f815456c640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f814c113d90 con 0x7f814c10a170 2024-05-09T04:51:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.018+0000 7f815456c640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f814c114090 con 0x7f814c104420 2024-05-09T04:51:17.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.018+0000 7f8151b07640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c109cd0 0x7f814c1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.018+0000 7f8151b07640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c109cd0 0x7f814c1a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44458/0 (socket says 172.21.15.46:44458) 2024-05-09T04:51:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.018+0000 7f8151b07640 1 -- 172.21.15.46:0/3040141345 learned_addr learned my addr 172.21.15.46:0/3040141345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.019+0000 7f8152308640 1 --2- 172.21.15.46:0/3040141345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f814c104420 0x7f814c1a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.019+0000 7f8152b09640 1 --2- 172.21.15.46:0/3040141345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f814c10a170 0x7f814c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.019+0000 7f8151b07640 1 -- 172.21.15.46:0/3040141345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f814c104420 msgr2=0x7f814c1a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.019+0000 7f8151b07640 1 --2- 172.21.15.46:0/3040141345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f814c104420 0x7f814c1a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.019+0000 7f8151b07640 1 -- 172.21.15.46:0/3040141345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f814c10a170 msgr2=0x7f814c19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.019+0000 7f8151b07640 1 --2- 172.21.15.46:0/3040141345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f814c10a170 0x7f814c19a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.019+0000 7f8151b07640 1 -- 172.21.15.46:0/3040141345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f814c19ae00 con 0x7f814c109cd0 2024-05-09T04:51:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.019+0000 7f8151b07640 1 --2- 172.21.15.46:0/3040141345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c109cd0 0x7f814c1a0a90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f813c00e9f0 tx=0x7f813c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.019+0000 7f81437fe640 1 -- 172.21.15.46:0/3040141345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c00ccc0 con 0x7f814c109cd0 2024-05-09T04:51:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.019+0000 7f815456c640 1 -- 172.21.15.46:0/3040141345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f814c19b090 con 0x7f814c109cd0 2024-05-09T04:51:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.019+0000 7f81437fe640 1 -- 172.21.15.46:0/3040141345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f813c0035f0 con 0x7f814c109cd0 2024-05-09T04:51:17.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.019+0000 7f81437fe640 1 -- 172.21.15.46:0/3040141345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c00ce60 con 0x7f814c109cd0 2024-05-09T04:51:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.019+0000 7f815456c640 1 -- 172.21.15.46:0/3040141345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f814c1a72f0 con 0x7f814c109cd0 2024-05-09T04:51:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.020+0000 7f815456c640 1 -- 172.21.15.46:0/3040141345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f814c104fb0 con 0x7f814c109cd0 2024-05-09T04:51:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.022+0000 7f81437fe640 1 -- 172.21.15.46:0/3040141345 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f813c003aa0 con 0x7f814c109cd0 2024-05-09T04:51:17.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.022+0000 7f81437fe640 1 --2- 172.21.15.46:0/3040141345 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f81380b29e0 0x7f81380b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.023+0000 7f8152308640 1 --2- 172.21.15.46:0/3040141345 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f81380b29e0 0x7f81380b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.023+0000 7f81437fe640 1 -- 172.21.15.46:0/3040141345 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f813c0e2c90 con 0x7f814c109cd0 2024-05-09T04:51:17.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.023+0000 7f8152308640 1 --2- 172.21.15.46:0/3040141345 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f81380b29e0 0x7f81380b4ef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8148006010 tx=0x7f8148031400 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:17.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.026+0000 7f81437fe640 1 -- 172.21.15.46:0/3040141345 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f813c0aa4e0 con 0x7f814c109cd0 2024-05-09T04:51:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-09T04:51:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3369098194' entity='client.admin' 2024-05-09T04:51:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/4280191235' entity='client.admin' 2024-05-09T04:51:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: from='client.? ' entity='client.admin' 2024-05-09T04:51:17.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:17.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:17.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:17.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:16 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: pgmap v8: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-09T04:51:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3369098194' entity='client.admin' 2024-05-09T04:51:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:17.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:17.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/4280191235' entity='client.admin' 2024-05-09T04:51:17.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:17.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:17.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:17.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:17.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: from='client.? ' entity='client.admin' 2024-05-09T04:51:17.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:51:17.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:51:17.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:51:17.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:16 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:51:17.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.243+0000 7f815456c640 1 -- 172.21.15.46:0/3040141345 --> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f814c108210 con 0x7f81380b29e0 2024-05-09T04:51:17.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.306+0000 7f81437fe640 1 -- 172.21.15.46:0/3040141345 <== mgr.15150 v2:172.21.15.46:6800/2595796783 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f8148004390 con 0x7f81380b29e0 2024-05-09T04:51:17.306 INFO:teuthology.orchestra.run.smithi046.stdout:Retention added to path / 2024-05-09T04:51:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.309+0000 7f815456c640 1 -- 172.21.15.46:0/3040141345 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f81380b29e0 msgr2=0x7f81380b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.309+0000 7f815456c640 1 --2- 172.21.15.46:0/3040141345 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f81380b29e0 0x7f81380b4ef0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8148006010 tx=0x7f8148031400 comp rx=0 tx=0).stop 2024-05-09T04:51:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.309+0000 7f815456c640 1 -- 172.21.15.46:0/3040141345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c109cd0 msgr2=0x7f814c1a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.309+0000 7f815456c640 1 --2- 172.21.15.46:0/3040141345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c109cd0 0x7f814c1a0a90 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f813c00e9f0 tx=0x7f813c00eeb0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.309+0000 7f815456c640 1 -- 172.21.15.46:0/3040141345 shutdown_connections 2024-05-09T04:51:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.309+0000 7f815456c640 1 --2- 172.21.15.46:0/3040141345 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f81380b29e0 0x7f81380b4ef0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.309+0000 7f815456c640 1 --2- 172.21.15.46:0/3040141345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f814c10a170 0x7f814c19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.309+0000 7f815456c640 1 --2- 172.21.15.46:0/3040141345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f814c109cd0 0x7f814c1a0a90 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.309+0000 7f815456c640 1 --2- 172.21.15.46:0/3040141345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f814c104420 0x7f814c1a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.309+0000 7f815456c640 1 -- 172.21.15.46:0/3040141345 >> 172.21.15.46:0/3040141345 conn(0x7f814c0793d0 msgr2=0x7f814c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:17.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.309+0000 7f815456c640 1 -- 172.21.15.46:0/3040141345 shutdown_connections 2024-05-09T04:51:17.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.309+0000 7f815456c640 1 -- 172.21.15.46:0/3040141345 wait complete. 2024-05-09T04:51:17.320 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-09T04:51:17.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.410+0000 7f03bcfa5700 1 -- 172.21.15.46:0/1102869025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8107df0 msgr2=0x7f03b8108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.410+0000 7f03bcfa5700 1 --2- 172.21.15.46:0/1102869025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8107df0 0x7f03b8108200 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0099c0 tx=0x7f03ac009cd0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.411+0000 7f03bcfa5700 1 -- 172.21.15.46:0/1102869025 shutdown_connections 2024-05-09T04:51:17.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.411+0000 7f03bcfa5700 1 --2- 172.21.15.46:0/1102869025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b8108740 0x7f03b81100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.411+0000 7f03bcfa5700 1 --2- 172.21.15.46:0/1102869025 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03b81034c0 0x7f03b81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.411+0000 7f03bcfa5700 1 --2- 172.21.15.46:0/1102869025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8107df0 0x7f03b8108200 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.411+0000 7f03bcfa5700 1 -- 172.21.15.46:0/1102869025 >> 172.21.15.46:0/1102869025 conn(0x7f03b80fedf0 msgr2=0x7f03b8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.411+0000 7f03bcfa5700 1 -- 172.21.15.46:0/1102869025 shutdown_connections 2024-05-09T04:51:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.411+0000 7f03bcfa5700 1 -- 172.21.15.46:0/1102869025 wait complete. 2024-05-09T04:51:17.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03bcfa5700 1 Processor -- start 2024-05-09T04:51:17.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03bcfa5700 1 -- start start 2024-05-09T04:51:17.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03bcfa5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b81034c0 0x7f03b819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03bcfa5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8107df0 0x7f03b819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03bcfa5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03b8108740 0x7f03b819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03bcfa5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03b8113350 con 0x7f03b8107df0 2024-05-09T04:51:17.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03bcfa5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f03b81131d0 con 0x7f03b8108740 2024-05-09T04:51:17.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03bcfa5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03b81134d0 con 0x7f03b81034c0 2024-05-09T04:51:17.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03b5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8107df0 0x7f03b819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03b659c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b81034c0 0x7f03b819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03b5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8107df0 0x7f03b819b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44478/0 (socket says 172.21.15.46:44478) 2024-05-09T04:51:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03b5d9b700 1 -- 172.21.15.46:0/2996969448 learned_addr learned my addr 172.21.15.46:0/2996969448 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03b6d9d700 1 --2- 172.21.15.46:0/2996969448 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03b8108740 0x7f03b819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03b5d9b700 1 -- 172.21.15.46:0/2996969448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b81034c0 msgr2=0x7f03b819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03b5d9b700 1 --2- 172.21.15.46:0/2996969448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b81034c0 0x7f03b819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03b5d9b700 1 -- 172.21.15.46:0/2996969448 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03b8108740 msgr2=0x7f03b819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03b5d9b700 1 --2- 172.21.15.46:0/2996969448 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03b8108740 0x7f03b819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.412+0000 7f03b5d9b700 1 -- 172.21.15.46:0/2996969448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03b819fce0 con 0x7f03b8107df0 2024-05-09T04:51:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.413+0000 7f03b5d9b700 1 --2- 172.21.15.46:0/2996969448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8107df0 0x7f03b819b3b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f03ac005f50 tx=0x7f03ac004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.413+0000 7f03a77fe700 1 -- 172.21.15.46:0/2996969448 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac026070 con 0x7f03b8107df0 2024-05-09T04:51:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.413+0000 7f03a77fe700 1 -- 172.21.15.46:0/2996969448 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03ac003c70 con 0x7f03b8107df0 2024-05-09T04:51:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.413+0000 7f03bcfa5700 1 -- 172.21.15.46:0/2996969448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03b819ff70 con 0x7f03b8107df0 2024-05-09T04:51:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.413+0000 7f03a77fe700 1 -- 172.21.15.46:0/2996969448 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac00fad0 con 0x7f03b8107df0 2024-05-09T04:51:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.413+0000 7f03bcfa5700 1 -- 172.21.15.46:0/2996969448 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03b81a0450 con 0x7f03b8107df0 2024-05-09T04:51:17.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.417+0000 7f03a77fe700 1 -- 172.21.15.46:0/2996969448 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f03ac029070 con 0x7f03b8107df0 2024-05-09T04:51:17.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.418+0000 7f03a77fe700 1 --2- 172.21.15.46:0/2996969448 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f03a00b2a80 0x7f03a00b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.418+0000 7f03cb9bbb80 1 Processor -- start 2024-05-09T04:51:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.418+0000 7f03b659c700 1 --2- 172.21.15.46:0/2996969448 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f03a00b2a80 0x7f03a00b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.418+0000 7f03a77fe700 1 -- 172.21.15.46:0/2996969448 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f03ac0e30c0 con 0x7f03b8107df0 2024-05-09T04:51:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.418+0000 7f03cb9bbb80 1 -- start start 2024-05-09T04:51:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.418+0000 7f03cb9bbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 0x5606fd6fb570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.418+0000 7f03cb9bbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 0x5606fd6fbab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.418+0000 7f03cb9bbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd6f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.418+0000 7f03cb9bbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5606fd618070 con 0x5606fd6f5720 2024-05-09T04:51:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.418+0000 7f03cb9bbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5606fd5b6870 con 0x5606fd578520 2024-05-09T04:51:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.418+0000 7f03cb9bbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5606fd614030 con 0x5606fd6621e0 2024-05-09T04:51:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.418+0000 7f03b6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd6f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.419+0000 7f03b6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd6f5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44494/0 (socket says 172.21.15.46:44494) 2024-05-09T04:51:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.419+0000 7f03b6d9d700 1 -- 172.21.15.46:0/3537883675 learned_addr learned my addr 172.21.15.46:0/3537883675 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.419+0000 7f03b659c700 1 --2- 172.21.15.46:0/2996969448 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f03a00b2a80 0x7f03a00b4f90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009c60 tx=0x7f03a8009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:17.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.419+0000 7f03b5d9b700 1 --2- 172.21.15.46:0/3537883675 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 0x5606fd6fbab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.419+0000 7f03b659c700 1 --2- 172.21.15.46:0/3537883675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 0x5606fd6fb570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.419+0000 7f03b6d9d700 1 -- 172.21.15.46:0/3537883675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 msgr2=0x5606fd6fb570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:17.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.419+0000 7f03b6d9d700 1 --2- 172.21.15.46:0/3537883675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 0x5606fd6fb570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.419+0000 7f03b6d9d700 1 -- 172.21.15.46:0/3537883675 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 msgr2=0x5606fd6fbab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:17.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.419+0000 7f03b6d9d700 1 --2- 172.21.15.46:0/3537883675 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 0x5606fd6fbab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.419+0000 7f03b6d9d700 1 -- 172.21.15.46:0/3537883675 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606fd5ae3f0 con 0x5606fd6f5720 2024-05-09T04:51:17.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.419+0000 7f03b5d9b700 1 --2- 172.21.15.46:0/3537883675 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 0x5606fd6fbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:51:17.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.419+0000 7f03b6d9d700 1 --2- 172.21.15.46:0/3537883675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd6f5b50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009440 tx=0x7f03b0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3cc803d7ec04614 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:17.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.419+0000 7f03a4ff9700 1 -- 172.21.15.46:0/3537883675 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0013750 con 0x5606fd6f5720 2024-05-09T04:51:17.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.419+0000 7f03a4ff9700 1 -- 172.21.15.46:0/3537883675 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b0013da0 con 0x5606fd6f5720 2024-05-09T04:51:17.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.419+0000 7f03a4ff9700 1 -- 172.21.15.46:0/3537883675 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0015e40 con 0x5606fd6f5720 2024-05-09T04:51:17.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.420+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/3537883675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 msgr2=0x5606fd6f5b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.420+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/3537883675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd6f5b50 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009440 tx=0x7f03b0009920 comp rx=0 tx=0).stop 2024-05-09T04:51:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.420+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/3537883675 shutdown_connections 2024-05-09T04:51:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.420+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/3537883675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd6f5b50 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.420+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/3537883675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 0x5606fd6fb570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.420+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/3537883675 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 0x5606fd6fbab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.420+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/3537883675 >> 172.21.15.46:0/3537883675 conn(0x5606fd5d8720 msgr2=0x5606fd6f0480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.421+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/3537883675 shutdown_connections 2024-05-09T04:51:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.421+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/3537883675 wait complete. 2024-05-09T04:51:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.421+0000 7f03cb9bbb80 1 -- start start 2024-05-09T04:51:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.421+0000 7f03cb9bbb80 1 Processor -- start 2024-05-09T04:51:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.421+0000 7f03cb9bbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd578520 0x5606fd6ff250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.421+0000 7f03cb9bbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd6621e0 0x5606fd6ff790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.421+0000 7f03cb9bbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6f5720 0x5606fd6ffcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.421+0000 7f03cb9bbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5606fd493c20 con 0x5606fd578520 2024-05-09T04:51:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.421+0000 7f03cb9bbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5606fd5d21d0 con 0x5606fd6621e0 2024-05-09T04:51:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.421+0000 7f03cb9bbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5606fd702250 con 0x5606fd6f5720 2024-05-09T04:51:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.421+0000 7f03cb9bbb80 10 client.0 _ll_register_callbacks cb 0x5606fd489740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:51:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.422+0000 7f03b659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd578520 0x5606fd6ff250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.422+0000 7f03b659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd578520 0x5606fd6ff250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44502/0 (socket says 172.21.15.46:44502) 2024-05-09T04:51:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.422+0000 7f03b659c700 1 -- 172.21.15.46:0/3394892477 learned_addr learned my addr 172.21.15.46:0/3394892477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.422+0000 7f03b5d9b700 1 --2- 172.21.15.46:0/3394892477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd6621e0 0x5606fd6ff790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.422+0000 7f03b6d9d700 1 --2- 172.21.15.46:0/3394892477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6f5720 0x5606fd6ffcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.422+0000 7f03b5d9b700 1 -- 172.21.15.46:0/3394892477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6f5720 msgr2=0x5606fd6ffcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.422+0000 7f03b5d9b700 1 --2- 172.21.15.46:0/3394892477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6f5720 0x5606fd6ffcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.422+0000 7f03b5d9b700 1 -- 172.21.15.46:0/3394892477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd578520 msgr2=0x5606fd6ff250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.422+0000 7f03b5d9b700 1 --2- 172.21.15.46:0/3394892477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd578520 0x5606fd6ff250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.422+0000 7f03b5d9b700 1 -- 172.21.15.46:0/3394892477 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606fd493510 con 0x5606fd6621e0 2024-05-09T04:51:17.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.422+0000 7f03b5d9b700 1 --2- 172.21.15.46:0/3394892477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd6621e0 0x5606fd6ff790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0e06d0 tx=0x7f03ac0e0c60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6c8961be2cbc327f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:17.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.422+0000 7f039dffb700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:51:17.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.422+0000 7f039dffb700 1 -- 172.21.15.46:0/3394892477 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac003e20 con 0x5606fd6621e0 2024-05-09T04:51:17.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.422+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/3394892477 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03b0013da0 con 0x5606fd6621e0 2024-05-09T04:51:17.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.422+0000 7f039dffb700 1 -- 172.21.15.46:0/3394892477 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f03ac0b2d90 con 0x5606fd6621e0 2024-05-09T04:51:17.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.423+0000 7f039dffb700 1 -- 172.21.15.46:0/3394892477 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f03ac0a9d50 con 0x5606fd6621e0 2024-05-09T04:51:17.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.423+0000 7f03cb9bbb80 10 client.25079 fetch_fsmap learned FSMap version 25 2024-05-09T04:51:17.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.423+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/3394892477 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5606fd70cc20 con 0x5606fd6621e0 2024-05-09T04:51:17.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.423+0000 7f039dffb700 1 -- 172.21.15.46:0/3394892477 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b0013da0 con 0x5606fd6621e0 2024-05-09T04:51:17.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.423+0000 7f039dffb700 1 -- 172.21.15.46:0/3394892477 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f03ac0ae120 con 0x5606fd6621e0 2024-05-09T04:51:17.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.423+0000 7f03cb9bbb80 10 client.25079 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:51:17.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.423+0000 7f03cb9bbb80 10 client.25079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:51:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.423+0000 7f03cb9bbb80 20 client.25079 populate_metadata read hostname 'smithi046' 2024-05-09T04:51:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.423+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/3394892477 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5606fd70f280 0x5606fd711730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.423+0000 7f03cb9bbb80 4 client.25079 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:51:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.423+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/3394892477 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606fd625a50 con 0x5606fd70f280 2024-05-09T04:51:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.424+0000 7f03b659c700 1 --2- 172.21.15.46:0/3394892477 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5606fd70f280 0x5606fd711730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.424+0000 7f039dffb700 10 client.25079 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.424+0000 7f03b659c700 1 --2- 172.21.15.46:0/3394892477 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5606fd70f280 0x5606fd711730 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f908033118e777 server_cookie=be490e61e37d1eb4 in_seq=0 out_seq=0 2024-05-09T04:51:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.425+0000 7f039dffb700 1 -- 172.21.15.46:0/3394892477 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5606fd625a50 con 0x5606fd70f280 2024-05-09T04:51:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.425+0000 7f039dffb700 10 client.25079 handle_command_reply: tid=0 2024-05-09T04:51:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.425+0000 7f03cb9bbb80 1 client.25079 shutdown 2024-05-09T04:51:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.425+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/3394892477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd6621e0 msgr2=0x5606fd6ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.425+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/3394892477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd6621e0 0x5606fd6ff790 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0e06d0 tx=0x7f03ac0e0c60 comp rx=0 tx=0).stop 2024-05-09T04:51:17.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.425+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/3394892477 shutdown_connections 2024-05-09T04:51:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.425+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/3394892477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6f5720 0x5606fd6ffcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.425+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/3394892477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd578520 0x5606fd6ff250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.425+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/3394892477 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5606fd70f280 0x5606fd711730 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.425+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/3394892477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd6621e0 0x5606fd6ff790 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.425+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/3394892477 >> 172.21.15.46:0/3394892477 conn(0x5606fd5d8720 msgr2=0x5606fd6f0240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.425+0000 7f039dffb700 0 client.25079 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.425+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/3394892477 shutdown_connections 2024-05-09T04:51:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.425+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/3394892477 wait complete. 2024-05-09T04:51:17.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.425+0000 7f03cb9bbb80 20 client.25079 trim_cache size 0 max 16384 2024-05-09T04:51:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.445+0000 7fb1b4e92640 1 -- 172.21.15.46:0/3821287202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1b01032f0 msgr2=0x7fb1b010f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.445+0000 7fb1b4e92640 1 --2- 172.21.15.46:0/3821287202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1b01032f0 0x7fb1b010f6c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb1a8009ca0 tx=0x7fb1a802a8c0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.445+0000 7fb1b4e92640 1 -- 172.21.15.46:0/3821287202 shutdown_connections 2024-05-09T04:51:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.445+0000 7fb1b4e92640 1 --2- 172.21.15.46:0/3821287202 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1b01032f0 0x7fb1b010f6c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.445+0000 7fb1b4e92640 1 --2- 172.21.15.46:0/3821287202 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1b0109040 0x7fb1b0102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.445+0000 7fb1b4e92640 1 --2- 172.21.15.46:0/3821287202 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1b0108ba0 0x7fb1b0102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.445+0000 7fb1b4e92640 1 -- 172.21.15.46:0/3821287202 >> 172.21.15.46:0/3821287202 conn(0x7fb1b00782e0 msgr2=0x7fb1b0078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.445+0000 7fb1b4e92640 1 -- 172.21.15.46:0/3821287202 shutdown_connections 2024-05-09T04:51:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.445+0000 7fb1b4e92640 1 -- 172.21.15.46:0/3821287202 wait complete. 2024-05-09T04:51:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.446+0000 7fb1b4e92640 1 Processor -- start 2024-05-09T04:51:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.446+0000 7fb1b4e92640 1 -- start start 2024-05-09T04:51:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.446+0000 7fb1b4e92640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1b01032f0 0x7fb1b019b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.446+0000 7fb1b4e92640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1b0108ba0 0x7fb1b019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.446+0000 7fb1b4e92640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1b0109040 0x7fb1b019c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.446+0000 7fb1b4e92640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b0112db0 con 0x7fb1b0108ba0 2024-05-09T04:51:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.446+0000 7fb1b4e92640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b0112c30 con 0x7fb1b0109040 2024-05-09T04:51:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.446+0000 7fb1b4e92640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb1b0112f30 con 0x7fb1b01032f0 2024-05-09T04:51:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.446+0000 7fb1add9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1b0108ba0 0x7fb1b019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.446+0000 7fb1add9b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1b0108ba0 0x7fb1b019bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44522/0 (socket says 172.21.15.46:44522) 2024-05-09T04:51:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.446+0000 7fb1add9b640 1 -- 172.21.15.46:0/381671766 learned_addr learned my addr 172.21.15.46:0/381671766 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.446+0000 7fb1ae59c640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1b01032f0 0x7fb1b019b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.446+0000 7fb1aed9d640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1b0109040 0x7fb1b019c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.447+0000 7fb1add9b640 1 -- 172.21.15.46:0/381671766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1b01032f0 msgr2=0x7fb1b019b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.447+0000 7fb1add9b640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1b01032f0 0x7fb1b019b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.447+0000 7fb1add9b640 1 -- 172.21.15.46:0/381671766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1b0109040 msgr2=0x7fb1b019c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.447+0000 7fb1add9b640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1b0109040 0x7fb1b019c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.447+0000 7fb1add9b640 1 -- 172.21.15.46:0/381671766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1b01a69d0 con 0x7fb1b0108ba0 2024-05-09T04:51:17.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.447+0000 7fb1ae59c640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1b01032f0 0x7fb1b019b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:17.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.447+0000 7fb1aed9d640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1b0109040 0x7fb1b019c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:17.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.447+0000 7fb1add9b640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1b0108ba0 0x7fb1b019bbd0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb198005fd0 tx=0x7fb19800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:17.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.447+0000 7fb1a77fe640 1 -- 172.21.15.46:0/381671766 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb198017070 con 0x7fb1b0108ba0 2024-05-09T04:51:17.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.447+0000 7fb1b4e92640 1 -- 172.21.15.46:0/381671766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1b01a6cc0 con 0x7fb1b0108ba0 2024-05-09T04:51:17.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.447+0000 7fb1a77fe640 1 -- 172.21.15.46:0/381671766 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb198009c30 con 0x7fb1b0108ba0 2024-05-09T04:51:17.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.447+0000 7fb1a77fe640 1 -- 172.21.15.46:0/381671766 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb198003d10 con 0x7fb1b0108ba0 2024-05-09T04:51:17.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.447+0000 7fb1b4e92640 1 -- 172.21.15.46:0/381671766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1b01a71c0 con 0x7fb1b0108ba0 2024-05-09T04:51:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.448+0000 7f03cb9bbb80 1 Processor -- start 2024-05-09T04:51:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.448+0000 7f03cb9bbb80 1 -- start start 2024-05-09T04:51:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.448+0000 7f03cb9bbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 0x5606fd6f9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.448+0000 7fb1b4e92640 1 -- 172.21.15.46:0/381671766 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1b006a240 con 0x7fb1b0108ba0 2024-05-09T04:51:17.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.448+0000 7f03cb9bbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 0x5606fd6fa250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.448+0000 7f03cb9bbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd6fa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.448+0000 7f03cb9bbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03b81af5a0 con 0x5606fd6f5720 2024-05-09T04:51:17.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.449+0000 7f03cb9bbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f03b81ace30 con 0x5606fd578520 2024-05-09T04:51:17.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.449+0000 7f03cb9bbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03b81aac80 con 0x5606fd6621e0 2024-05-09T04:51:17.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.449+0000 7f03b6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd6fa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.449+0000 7f03b6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd6fa790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44534/0 (socket says 172.21.15.46:44534) 2024-05-09T04:51:17.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.449+0000 7f03b6d9d700 1 -- 172.21.15.46:0/214062001 learned_addr learned my addr 172.21.15.46:0/214062001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:17.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.449+0000 7f03b5d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 0x5606fd6fa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.449+0000 7f03b659c700 1 --2- 172.21.15.46:0/214062001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 0x5606fd6f9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.449+0000 7f03b6d9d700 1 -- 172.21.15.46:0/214062001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 msgr2=0x5606fd6fa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:17.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.449+0000 7f03b6d9d700 1 --2- 172.21.15.46:0/214062001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 0x5606fd6fa250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.449+0000 7f03b6d9d700 1 -- 172.21.15.46:0/214062001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 msgr2=0x5606fd6f9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:17.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.449+0000 7f03b6d9d700 1 --2- 172.21.15.46:0/214062001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 0x5606fd6f9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.449+0000 7f03b6d9d700 1 -- 172.21.15.46:0/214062001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606fd6fb0d0 con 0x5606fd6f5720 2024-05-09T04:51:17.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.449+0000 7f03b6d9d700 1 --2- 172.21.15.46:0/214062001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd6fa790 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x5606fd6fc140 tx=0x7f03b0016d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46690f0c9d130def server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:17.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.450+0000 7fb1a77fe640 1 -- 172.21.15.46:0/381671766 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fb198003eb0 con 0x7fb1b0108ba0 2024-05-09T04:51:17.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.451+0000 7fb1a77fe640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb19c0b29e0 0x7fb19c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.452+0000 7fb1ae59c640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb19c0b29e0 0x7fb19c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.452+0000 7fb1a77fe640 1 -- 172.21.15.46:0/381671766 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fb1980e2ae0 con 0x7fb1b0108ba0 2024-05-09T04:51:17.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.452+0000 7fb1ae59c640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb19c0b29e0 0x7fb19c0b4ef0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0103880 tx=0x7fb1a0005f90 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:17.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.455+0000 7f039ffff700 1 -- 172.21.15.46:0/214062001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0004c90 con 0x5606fd6f5720 2024-05-09T04:51:17.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.455+0000 7f039ffff700 1 -- 172.21.15.46:0/214062001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03b0016e60 con 0x5606fd6f5720 2024-05-09T04:51:17.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.455+0000 7f039ffff700 1 -- 172.21.15.46:0/214062001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03b0004280 con 0x5606fd6f5720 2024-05-09T04:51:17.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.455+0000 7fb1a77fe640 1 -- 172.21.15.46:0/381671766 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb198020030 con 0x7fb1b0108ba0 2024-05-09T04:51:17.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.456+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/214062001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 msgr2=0x5606fd6fa790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.456+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/214062001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd6fa790 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x5606fd6fc140 tx=0x7f03b0016d70 comp rx=0 tx=0).stop 2024-05-09T04:51:17.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.457+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/214062001 shutdown_connections 2024-05-09T04:51:17.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.457+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/214062001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd6fa790 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.457+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/214062001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 0x5606fd6f9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.457+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/214062001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 0x5606fd6fa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.457+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/214062001 >> 172.21.15.46:0/214062001 conn(0x5606fd5d8720 msgr2=0x5606fd6fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.457+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/214062001 shutdown_connections 2024-05-09T04:51:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.457+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/214062001 wait complete. 2024-05-09T04:51:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.457+0000 7f03cb9bbb80 1 -- start start 2024-05-09T04:51:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.457+0000 7f03cb9bbb80 1 Processor -- start 2024-05-09T04:51:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.458+0000 7f03cb9bbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 0x5606fd7072f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.458+0000 7f03cb9bbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 0x5606fd707830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.458+0000 7f03cb9bbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd7022a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.458+0000 7f03cb9bbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03b81ab2f0 con 0x5606fd6f5720 2024-05-09T04:51:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.458+0000 7f03cb9bbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f03b81ac760 con 0x5606fd578520 2024-05-09T04:51:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.458+0000 7f03b6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd7022a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.458+0000 7f03b659c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 0x5606fd7072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.458+0000 7f03cb9bbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03b81ab9c0 con 0x5606fd6621e0 2024-05-09T04:51:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.458+0000 7f03b6d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd7022a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44548/0 (socket says 172.21.15.46:44548) 2024-05-09T04:51:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.458+0000 7f03b6d9d700 1 -- 172.21.15.46:0/1506860486 learned_addr learned my addr 172.21.15.46:0/1506860486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.458+0000 7f03b5d9b700 1 --2- 172.21.15.46:0/1506860486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 0x5606fd707830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.459+0000 7f03cb9bbb80 10 client.0 _ll_register_callbacks cb 0x5606fd489740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:51:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.459+0000 7f03b5d9b700 1 -- 172.21.15.46:0/1506860486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 msgr2=0x5606fd7072f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.459+0000 7f03b5d9b700 1 --2- 172.21.15.46:0/1506860486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 0x5606fd7072f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.459+0000 7f03b5d9b700 1 -- 172.21.15.46:0/1506860486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 msgr2=0x5606fd7022a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.459+0000 7f03b5d9b700 1 --2- 172.21.15.46:0/1506860486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd7022a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.459+0000 7f03b5d9b700 1 -- 172.21.15.46:0/1506860486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606fd7027e0 con 0x5606fd6621e0 2024-05-09T04:51:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.459+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/1506860486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5606fd61faf0 con 0x5606fd6621e0 2024-05-09T04:51:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.459+0000 7f03b5d9b700 1 --2- 172.21.15.46:0/1506860486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 0x5606fd707830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0a9f40 tx=0x7f03ac0a9a70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df5aef6b18d0ad2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.459+0000 7f039dffb700 10 client.25048 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:51:17.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.459+0000 7f039dffb700 1 -- 172.21.15.46:0/1506860486 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03ac0aa010 con 0x5606fd6621e0 2024-05-09T04:51:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.459+0000 7f039dffb700 1 -- 172.21.15.46:0/1506860486 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f03ac0039d0 con 0x5606fd6621e0 2024-05-09T04:51:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.459+0000 7f039dffb700 1 -- 172.21.15.46:0/1506860486 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f03ac003e20 con 0x5606fd6621e0 2024-05-09T04:51:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.460+0000 7f03cb9bbb80 10 client.25048 fetch_fsmap learned FSMap version 25 2024-05-09T04:51:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.460+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/1506860486 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5606fd77a130 con 0x5606fd6621e0 2024-05-09T04:51:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.460+0000 7f039dffb700 1 -- 172.21.15.46:0/1506860486 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f03ac0a94a0 con 0x5606fd6621e0 2024-05-09T04:51:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.460+0000 7f03cb9bbb80 10 client.25048 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:51:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.460+0000 7f03cb9bbb80 10 client.25048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:51:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.460+0000 7f03cb9bbb80 20 client.25048 populate_metadata read hostname 'smithi046' 2024-05-09T04:51:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.460+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/1506860486 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5606fd77a540 0x5606fd77c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.460+0000 7f03cb9bbb80 4 client.25048 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:51:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.460+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/1506860486 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5606fd4939f0 con 0x5606fd77a540 2024-05-09T04:51:17.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.461+0000 7f03b659c700 1 --2- 172.21.15.46:0/1506860486 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5606fd77a540 0x5606fd77c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.461+0000 7f03b659c700 1 --2- 172.21.15.46:0/1506860486 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5606fd77a540 0x5606fd77c980 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=864a1824bbc7fd57 server_cookie=fc4b94a31aec76b1 in_seq=0 out_seq=0 2024-05-09T04:51:17.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.461+0000 7f039dffb700 10 client.25048 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:17.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.462+0000 7f039dffb700 1 -- 172.21.15.46:0/1506860486 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5606fd4939f0 con 0x5606fd77a540 2024-05-09T04:51:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.462+0000 7f039dffb700 10 client.25048 handle_command_reply: tid=0 2024-05-09T04:51:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.462+0000 7f03cb9bbb80 1 client.25048 shutdown 2024-05-09T04:51:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.462+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/1506860486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 msgr2=0x5606fd707830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.463+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/1506860486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 0x5606fd707830 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0a9f40 tx=0x7f03ac0a9a70 comp rx=0 tx=0).stop 2024-05-09T04:51:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.463+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/1506860486 shutdown_connections 2024-05-09T04:51:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.463+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/1506860486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5606fd6f5720 0x5606fd7022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.463+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/1506860486 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5606fd77a540 0x5606fd77c980 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.463+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/1506860486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5606fd578520 0x5606fd7072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.463+0000 7f03cb9bbb80 1 --2- 172.21.15.46:0/1506860486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5606fd6621e0 0x5606fd707830 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.463+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/1506860486 >> 172.21.15.46:0/1506860486 conn(0x5606fd5d8720 msgr2=0x5606fd6f8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.463+0000 7f039dffb700 0 client.25048 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.463+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/1506860486 shutdown_connections 2024-05-09T04:51:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.463+0000 7f03cb9bbb80 1 -- 172.21.15.46:0/1506860486 wait complete. 2024-05-09T04:51:17.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.463+0000 7f03cb9bbb80 20 client.25048 trim_cache size 0 max 16384 2024-05-09T04:51:17.468 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:51:17.468 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:51:17.468 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "07fd9527-57a3-4e46-83d4-5599ec054bdf", 2024-05-09T04:51:17.468 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:51:17.468 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:51:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.464+0000 7f03bcfa5700 1 -- 172.21.15.46:0/2996969448 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f03a00b2a80 msgr2=0x7f03a00b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.464+0000 7f03bcfa5700 1 --2- 172.21.15.46:0/2996969448 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f03a00b2a80 0x7f03a00b4f90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009c60 tx=0x7f03a8009260 comp rx=0 tx=0).stop 2024-05-09T04:51:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.464+0000 7f03bcfa5700 1 -- 172.21.15.46:0/2996969448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8107df0 msgr2=0x7f03b819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.464+0000 7f03bcfa5700 1 --2- 172.21.15.46:0/2996969448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8107df0 0x7f03b819b3b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f03ac005f50 tx=0x7f03ac004270 comp rx=0 tx=0).stop 2024-05-09T04:51:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.464+0000 7f03bcfa5700 1 -- 172.21.15.46:0/2996969448 shutdown_connections 2024-05-09T04:51:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.464+0000 7f03bcfa5700 1 --2- 172.21.15.46:0/2996969448 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f03a00b2a80 0x7f03a00b4f90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.464+0000 7f03bcfa5700 1 --2- 172.21.15.46:0/2996969448 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03b8108740 0x7f03b819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.464+0000 7f03bcfa5700 1 --2- 172.21.15.46:0/2996969448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03b81034c0 0x7f03b819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.464+0000 7f03bcfa5700 1 --2- 172.21.15.46:0/2996969448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03b8107df0 0x7f03b819b3b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.464+0000 7f03bcfa5700 1 -- 172.21.15.46:0/2996969448 >> 172.21.15.46:0/2996969448 conn(0x7f03b80fedf0 msgr2=0x7f03b8075d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.464+0000 7f03bcfa5700 1 -- 172.21.15.46:0/2996969448 shutdown_connections 2024-05-09T04:51:17.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.464+0000 7f03bcfa5700 1 -- 172.21.15.46:0/2996969448 wait complete. 2024-05-09T04:51:17.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.683+0000 7fb1b4e92640 1 -- 172.21.15.46:0/381671766 --> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fb1b010ad20 con 0x7fb19c0b29e0 2024-05-09T04:51:17.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.692+0000 7fb1a77fe640 1 -- 172.21.15.46:0/381671766 <== mgr.15150 v2:172.21.15.46:6800/2595796783 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fb1a0004030 con 0x7fb19c0b29e0 2024-05-09T04:51:17.692 INFO:teuthology.orchestra.run.smithi046.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-09T00:00:00", "created": "2024-05-09T04:51:16", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-09T04:51:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.694+0000 7fb1b4e92640 1 -- 172.21.15.46:0/381671766 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb19c0b29e0 msgr2=0x7fb19c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.694+0000 7fb1b4e92640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb19c0b29e0 0x7fb19c0b4ef0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0103880 tx=0x7fb1a0005f90 comp rx=0 tx=0).stop 2024-05-09T04:51:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.694+0000 7fb1b4e92640 1 -- 172.21.15.46:0/381671766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1b0108ba0 msgr2=0x7fb1b019bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.694+0000 7fb1b4e92640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1b0108ba0 0x7fb1b019bbd0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb198005fd0 tx=0x7fb19800dde0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.694+0000 7fb1b4e92640 1 -- 172.21.15.46:0/381671766 shutdown_connections 2024-05-09T04:51:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.694+0000 7fb1b4e92640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb19c0b29e0 0x7fb19c0b4ef0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.694+0000 7fb1b4e92640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb1b0109040 0x7fb1b019c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.694+0000 7fb1b4e92640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb1b0108ba0 0x7fb1b019bbd0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.694+0000 7fb1b4e92640 1 --2- 172.21.15.46:0/381671766 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb1b01032f0 0x7fb1b019b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.694+0000 7fb1b4e92640 1 -- 172.21.15.46:0/381671766 >> 172.21.15.46:0/381671766 conn(0x7fb1b00782e0 msgr2=0x7fb1b00ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.694+0000 7fb1b4e92640 1 -- 172.21.15.46:0/381671766 shutdown_connections 2024-05-09T04:51:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.694+0000 7fb1b4e92640 1 -- 172.21.15.46:0/381671766 wait complete. 2024-05-09T04:51:17.705 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-09T04:51:17.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.825+0000 7fa545bce640 1 -- 172.21.15.46:0/4120259772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa54010b7f0 msgr2=0x7fa54010dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.825+0000 7fa545bce640 1 --2- 172.21.15.46:0/4120259772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa54010b7f0 0x7fa54010dc10 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fa534009900 tx=0x7fa53402a410 comp rx=0 tx=0).stop 2024-05-09T04:51:17.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.825+0000 7fa545bce640 1 -- 172.21.15.46:0/4120259772 shutdown_connections 2024-05-09T04:51:17.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.825+0000 7fa545bce640 1 --2- 172.21.15.46:0/4120259772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa54010e1e0 0x7fa5401106d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.825+0000 7fa545bce640 1 --2- 172.21.15.46:0/4120259772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa54010b7f0 0x7fa54010dc10 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.825+0000 7fa545bce640 1 --2- 172.21.15.46:0/4120259772 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa540104d90 0x7fa5401051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.825+0000 7fa545bce640 1 -- 172.21.15.46:0/4120259772 >> 172.21.15.46:0/4120259772 conn(0x7fa5400fe390 msgr2=0x7fa5401007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:17.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.825+0000 7fa545bce640 1 -- 172.21.15.46:0/4120259772 shutdown_connections 2024-05-09T04:51:17.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.825+0000 7fa545bce640 1 -- 172.21.15.46:0/4120259772 wait complete. 2024-05-09T04:51:17.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.825+0000 7fa545bce640 1 Processor -- start 2024-05-09T04:51:17.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa545bce640 1 -- start start 2024-05-09T04:51:17.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa545bce640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa540104d90 0x7fa540102650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa545bce640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa54010b7f0 0x7fa540102b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa545bce640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa54010e1e0 0x7fa5401030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa545bce640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa540113e40 con 0x7fa54010b7f0 2024-05-09T04:51:17.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa545bce640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa540113cc0 con 0x7fa54010e1e0 2024-05-09T04:51:17.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa545bce640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa540113fc0 con 0x7fa540104d90 2024-05-09T04:51:17.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa53effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa54010b7f0 0x7fa540102b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa53effd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa54010b7f0 0x7fa540102b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44560/0 (socket says 172.21.15.46:44560) 2024-05-09T04:51:17.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa53effd640 1 -- 172.21.15.46:0/462169220 learned_addr learned my addr 172.21.15.46:0/462169220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:17.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa53f7fe640 1 --2- 172.21.15.46:0/462169220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa540104d90 0x7fa540102650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa53ffff640 1 --2- 172.21.15.46:0/462169220 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa54010e1e0 0x7fa5401030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa53effd640 1 -- 172.21.15.46:0/462169220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa540104d90 msgr2=0x7fa540102650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa53effd640 1 --2- 172.21.15.46:0/462169220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa540104d90 0x7fa540102650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa53effd640 1 -- 172.21.15.46:0/462169220 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa54010e1e0 msgr2=0x7fa5401030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:17.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa53effd640 1 --2- 172.21.15.46:0/462169220 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa54010e1e0 0x7fa5401030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:17.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.826+0000 7fa53effd640 1 -- 172.21.15.46:0/462169220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5401a8be0 con 0x7fa54010b7f0 2024-05-09T04:51:17.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.827+0000 7fa53effd640 1 --2- 172.21.15.46:0/462169220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa54010b7f0 0x7fa540102b90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa534005940 tx=0x7fa5340040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:17.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.827+0000 7fa53cff9640 1 -- 172.21.15.46:0/462169220 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa53403f030 con 0x7fa54010b7f0 2024-05-09T04:51:17.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.827+0000 7fa53cff9640 1 -- 172.21.15.46:0/462169220 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5340044c0 con 0x7fa54010b7f0 2024-05-09T04:51:17.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.827+0000 7fa545bce640 1 -- 172.21.15.46:0/462169220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5401a8e70 con 0x7fa54010b7f0 2024-05-09T04:51:17.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.827+0000 7fa53cff9640 1 -- 172.21.15.46:0/462169220 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa534035070 con 0x7fa54010b7f0 2024-05-09T04:51:17.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.827+0000 7fa545bce640 1 -- 172.21.15.46:0/462169220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5401a9270 con 0x7fa54010b7f0 2024-05-09T04:51:17.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.828+0000 7fa545bce640 1 -- 172.21.15.46:0/462169220 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa540050870 con 0x7fa54010b7f0 2024-05-09T04:51:17.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.830+0000 7fa53cff9640 1 -- 172.21.15.46:0/462169220 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fa534042030 con 0x7fa54010b7f0 2024-05-09T04:51:17.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.830+0000 7fa53cff9640 1 --2- 172.21.15.46:0/462169220 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa5280b29e0 0x7fa5280b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:17.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.831+0000 7fa53f7fe640 1 --2- 172.21.15.46:0/462169220 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa5280b29e0 0x7fa5280b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:17.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.831+0000 7fa53cff9640 1 -- 172.21.15.46:0/462169220 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa534102de0 con 0x7fa54010b7f0 2024-05-09T04:51:17.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.831+0000 7fa53f7fe640 1 --2- 172.21.15.46:0/462169220 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa5280b29e0 0x7fa5280b4ef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa530009bd0 tx=0x7fa530005f90 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:17.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:17.834+0000 7fa53cff9640 1 -- 172.21.15.46:0/462169220 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa5340ca5d0 con 0x7fa54010b7f0 2024-05-09T04:51:18.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:18.050+0000 7fa545bce640 1 -- 172.21.15.46:0/462169220 --> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fa54010edd0 con 0x7fa5280b29e0 2024-05-09T04:51:18.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:18.057+0000 7fa53cff9640 1 -- 172.21.15.46:0/462169220 <== mgr.15150 v2:172.21.15.46:6800/2595796783 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fa54010edd0 con 0x7fa5280b29e0 2024-05-09T04:51:18.057 INFO:teuthology.orchestra.run.smithi046.stdout:/ 1m 6m3h 2024-05-09T04:51:18.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:18.059+0000 7fa545bce640 1 -- 172.21.15.46:0/462169220 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa5280b29e0 msgr2=0x7fa5280b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:18.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:18.059+0000 7fa545bce640 1 --2- 172.21.15.46:0/462169220 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa5280b29e0 0x7fa5280b4ef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa530009bd0 tx=0x7fa530005f90 comp rx=0 tx=0).stop 2024-05-09T04:51:18.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:18.059+0000 7fa545bce640 1 -- 172.21.15.46:0/462169220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa54010b7f0 msgr2=0x7fa540102b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:18.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:18.059+0000 7fa545bce640 1 --2- 172.21.15.46:0/462169220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa54010b7f0 0x7fa540102b90 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fa534005940 tx=0x7fa5340040c0 comp rx=0 tx=0).stop 2024-05-09T04:51:18.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:18.059+0000 7fa545bce640 1 -- 172.21.15.46:0/462169220 shutdown_connections 2024-05-09T04:51:18.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:18.059+0000 7fa545bce640 1 --2- 172.21.15.46:0/462169220 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa5280b29e0 0x7fa5280b4ef0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:18.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:18.059+0000 7fa545bce640 1 --2- 172.21.15.46:0/462169220 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa54010e1e0 0x7fa5401030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:18.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:18.059+0000 7fa545bce640 1 --2- 172.21.15.46:0/462169220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa54010b7f0 0x7fa540102b90 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:18.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:18.060+0000 7fa545bce640 1 --2- 172.21.15.46:0/462169220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa540104d90 0x7fa540102650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:18.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:18.060+0000 7fa545bce640 1 -- 172.21.15.46:0/462169220 >> 172.21.15.46:0/462169220 conn(0x7fa5400fe390 msgr2=0x7fa54010e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:18.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:18.060+0000 7fa545bce640 1 -- 172.21.15.46:0/462169220 shutdown_connections 2024-05-09T04:51:18.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:18.060+0000 7fa545bce640 1 -- 172.21.15.46:0/462169220 wait complete. 2024-05-09T04:51:18.071 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-09T04:51:18.099 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-09T04:51:18.109 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-09T04:51:18.188 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T04:51:18.188 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-09T04:51:18.190 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:51:18.233 INFO:teuthology.run_tasks:Running task workunit... 2024-05-09T04:51:18.245 INFO:tasks.workunit:Pulling workunits from ref c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:51:18.245 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-09T04:51:18.245 INFO:tasks.workunit:timeout=3h 2024-05-09T04:51:18.246 INFO:tasks.workunit:cleanup=True 2024-05-09T04:51:18.246 DEBUG:teuthology.orchestra.run.smithi046:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-09T04:51:18.262 INFO:teuthology.orchestra.run.smithi046.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-09T04:51:18.262 INFO:teuthology.orchestra.run.smithi046.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-09T04:51:18.262 INFO:teuthology.orchestra.run.smithi046.stdout:Device: 72h/114d Inode: 1099511628283 Links: 2 2024-05-09T04:51:18.263 INFO:teuthology.orchestra.run.smithi046.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-09T04:51:18.263 INFO:teuthology.orchestra.run.smithi046.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-09T04:51:18.263 INFO:teuthology.orchestra.run.smithi046.stdout:Access: 2024-05-09 04:48:24.630016865 +0000 2024-05-09T04:51:18.263 INFO:teuthology.orchestra.run.smithi046.stdout:Modify: 2024-05-09 04:48:24.630016865 +0000 2024-05-09T04:51:18.263 INFO:teuthology.orchestra.run.smithi046.stdout:Change: 2024-05-09 04:51:02.354447852 +0000 2024-05-09T04:51:18.263 INFO:teuthology.orchestra.run.smithi046.stdout: Birth: 2024-05-09 04:48:24.630016865 +0000 2024-05-09T04:51:18.263 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-09T04:51:18.263 DEBUG:teuthology.orchestra.run.smithi046:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-09T04:51:18.338 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout c9da4049f638c0f84bdd9b4e33c42e86046ac554 2024-05-09T04:51:18.396 INFO:tasks.workunit.client.0.smithi046.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-09T04:51:18.502 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:18.956 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:18 smithi046 ceph-mon[30136]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:18.956 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:18 smithi046 ceph-mon[30136]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:18.956 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:18 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:51:18.956 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:18 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:51:18.956 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:18 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:51:18.956 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:18 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:51:18.956 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:18 smithi046 ceph-mon[30136]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-09T04:51:18.956 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:18 smithi046 ceph-mon[30136]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:18 smithi110 ceph-mon[34904]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:18 smithi110 ceph-mon[34904]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:18 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:51:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:18 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:51:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:18 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:51:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:18 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:51:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:18 smithi110 ceph-mon[34904]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-09T04:51:19.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:18 smithi110 ceph-mon[34904]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:18 smithi072 ceph-mon[34088]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:18 smithi072 ceph-mon[34088]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:18 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:51:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:18 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:51:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:18 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:51:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:18 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:51:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:18 smithi072 ceph-mon[34088]: pgmap v9: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 0 B/s wr, 12 op/s 2024-05-09T04:51:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:18 smithi072 ceph-mon[34088]: from='client.15315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:19.891 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:19 smithi046 ceph-mon[30136]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:20.009 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:19 smithi110 ceph-mon[34904]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:20.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:19 smithi072 ceph-mon[34088]: from='client.15327 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T04:51:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.132+0000 7f27b6fc3700 1 -- 172.21.15.46:0/3939049230 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27b0107e30 msgr2=0x7f27b0108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:20.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.132+0000 7f27b6fc3700 1 --2- 172.21.15.46:0/3939049230 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27b0107e30 0x7f27b0108240 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f27a4009990 tx=0x7f27a4009ca0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.132+0000 7f27b6fc3700 1 -- 172.21.15.46:0/3939049230 shutdown_connections 2024-05-09T04:51:20.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.132+0000 7f27b6fc3700 1 --2- 172.21.15.46:0/3939049230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27b0108780 0x7f27b01100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.132+0000 7f27b6fc3700 1 --2- 172.21.15.46:0/3939049230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b0103550 0x7f27b0103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.132+0000 7f27b6fc3700 1 --2- 172.21.15.46:0/3939049230 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27b0107e30 0x7f27b0108240 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.132+0000 7f27b6fc3700 1 -- 172.21.15.46:0/3939049230 >> 172.21.15.46:0/3939049230 conn(0x7f27b0078e00 msgr2=0x7f27b0079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:20.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.133+0000 7f27b6fc3700 1 -- 172.21.15.46:0/3939049230 shutdown_connections 2024-05-09T04:51:20.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.133+0000 7f27b6fc3700 1 -- 172.21.15.46:0/3939049230 wait complete. 2024-05-09T04:51:20.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b6fc3700 1 Processor -- start 2024-05-09T04:51:20.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b6fc3700 1 -- start start 2024-05-09T04:51:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b6fc3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27b0103550 0x7f27b019bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b6fc3700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27b0107e30 0x7f27b019c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b6fc3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b0108780 0x7f27b019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b6fc3700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27b0113390 con 0x7f27b0108780 2024-05-09T04:51:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b6fc3700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f27b0113210 con 0x7f27b0103550 2024-05-09T04:51:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b6fc3700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27b0113510 con 0x7f27b0107e30 2024-05-09T04:51:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b5560700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b0108780 0x7f27b019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b5560700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b0108780 0x7f27b019c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44576/0 (socket says 172.21.15.46:44576) 2024-05-09T04:51:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b5560700 1 -- 172.21.15.46:0/2667534860 learned_addr learned my addr 172.21.15.46:0/2667534860 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b4d5f700 1 --2- 172.21.15.46:0/2667534860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27b0103550 0x7f27b019bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27affff700 1 --2- 172.21.15.46:0/2667534860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27b0107e30 0x7f27b019c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b5560700 1 -- 172.21.15.46:0/2667534860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27b0107e30 msgr2=0x7f27b019c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b5560700 1 --2- 172.21.15.46:0/2667534860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27b0107e30 0x7f27b019c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b5560700 1 -- 172.21.15.46:0/2667534860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27b0103550 msgr2=0x7f27b019bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.134+0000 7f27b5560700 1 --2- 172.21.15.46:0/2667534860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27b0103550 0x7f27b019bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.135+0000 7f27b5560700 1 -- 172.21.15.46:0/2667534860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b01a6fc0 con 0x7f27b0108780 2024-05-09T04:51:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.135+0000 7f27b4d5f700 1 --2- 172.21.15.46:0/2667534860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27b0103550 0x7f27b019bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.135+0000 7f27b5560700 1 --2- 172.21.15.46:0/2667534860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b0108780 0x7f27b019c710 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f27a800ef30 tx=0x7f27a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.135+0000 7f27adffb700 1 -- 172.21.15.46:0/2667534860 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a80092e0 con 0x7f27b0108780 2024-05-09T04:51:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.135+0000 7f27b6fc3700 1 -- 172.21.15.46:0/2667534860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b01a7250 con 0x7f27b0108780 2024-05-09T04:51:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.135+0000 7f27adffb700 1 -- 172.21.15.46:0/2667534860 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a8004bb0 con 0x7f27b0108780 2024-05-09T04:51:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.135+0000 7f27adffb700 1 -- 172.21.15.46:0/2667534860 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a8004020 con 0x7f27b0108780 2024-05-09T04:51:20.137 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.135+0000 7f27b6fc3700 1 -- 172.21.15.46:0/2667534860 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b01a7760 con 0x7f27b0108780 2024-05-09T04:51:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.138+0000 7f27adffb700 1 -- 172.21.15.46:0/2667534860 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f27a801b030 con 0x7f27b0108780 2024-05-09T04:51:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.138+0000 7f27adffb700 1 --2- 172.21.15.46:0/2667534860 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f27980b2a80 0x7f27980b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.139+0000 7f27b4d5f700 1 --2- 172.21.15.46:0/2667534860 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f27980b2a80 0x7f27980b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.139+0000 7f27adffb700 1 -- 172.21.15.46:0/2667534860 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f27a80e2210 con 0x7f27b0108780 2024-05-09T04:51:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.139+0000 7f27b4d5f700 1 --2- 172.21.15.46:0/2667534860 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f27980b2a80 0x7f27980b4f90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f27a0011490 tx=0x7f27a0009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27c59d9b80 1 Processor -- start 2024-05-09T04:51:20.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27c59d9b80 1 -- start start 2024-05-09T04:51:20.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27c59d9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86766d00 0x55ae8686f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27c59d9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae8686f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27c59d9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86869400 0x55ae86869810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27c59d9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ae8678c0a0 con 0x55ae866ec4a0 2024-05-09T04:51:20.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27c59d9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ae867319a0 con 0x55ae86869400 2024-05-09T04:51:20.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27c59d9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ae86788060 con 0x55ae86766d00 2024-05-09T04:51:20.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae8686f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae8686f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44588/0 (socket says 172.21.15.46:44588) 2024-05-09T04:51:20.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27affff700 1 -- 172.21.15.46:0/2513672073 learned_addr learned my addr 172.21.15.46:0/2513672073 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27b5560700 1 --2- 172.21.15.46:0/2513672073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86869400 0x55ae86869810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27b4d5f700 1 --2- 172.21.15.46:0/2513672073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86766d00 0x55ae8686f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27affff700 1 -- 172.21.15.46:0/2513672073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86766d00 msgr2=0x55ae8686f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27affff700 1 --2- 172.21.15.46:0/2513672073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86766d00 0x55ae8686f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27affff700 1 -- 172.21.15.46:0/2513672073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86869400 msgr2=0x55ae86869810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27affff700 1 --2- 172.21.15.46:0/2513672073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86869400 0x55ae86869810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27affff700 1 -- 172.21.15.46:0/2513672073 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae86722500 con 0x55ae866ec4a0 2024-05-09T04:51:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.140+0000 7f27b5560700 1 --2- 172.21.15.46:0/2513672073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86869400 0x55ae86869810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.141+0000 7f27affff700 1 --2- 172.21.15.46:0/2513672073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae8686f780 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f27b006d180 tx=0x7f27a4009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e301d1898ccf751 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.141+0000 7f2796ffd700 1 -- 172.21.15.46:0/2513672073 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4027070 con 0x55ae866ec4a0 2024-05-09T04:51:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.141+0000 7f2796ffd700 1 -- 172.21.15.46:0/2513672073 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a40056b0 con 0x55ae866ec4a0 2024-05-09T04:51:20.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.141+0000 7f2796ffd700 1 -- 172.21.15.46:0/2513672073 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4003b40 con 0x55ae866ec4a0 2024-05-09T04:51:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.142+0000 7f27c59d9b80 1 -- 172.21.15.46:0/2513672073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 msgr2=0x55ae8686f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.142+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/2513672073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae8686f780 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f27b006d180 tx=0x7f27a4009570 comp rx=0 tx=0).stop 2024-05-09T04:51:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.142+0000 7f27c59d9b80 1 -- 172.21.15.46:0/2513672073 shutdown_connections 2024-05-09T04:51:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.142+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/2513672073 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86869400 0x55ae86869810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.142+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/2513672073 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86766d00 0x55ae8686f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.142+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/2513672073 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae8686f780 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.142+0000 7f27c59d9b80 1 -- 172.21.15.46:0/2513672073 >> 172.21.15.46:0/2513672073 conn(0x55ae866eff70 msgr2=0x55ae86864220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.142+0000 7f27c59d9b80 1 -- 172.21.15.46:0/2513672073 shutdown_connections 2024-05-09T04:51:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.142+0000 7f27c59d9b80 1 -- 172.21.15.46:0/2513672073 wait complete. 2024-05-09T04:51:20.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.142+0000 7f27c59d9b80 1 -- start start 2024-05-09T04:51:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.143+0000 7f27c59d9b80 1 Processor -- start 2024-05-09T04:51:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.143+0000 7f27c59d9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae86872fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.143+0000 7f27c59d9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86766d00 0x55ae86873500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.143+0000 7f27c59d9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 0x55ae86873a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.143+0000 7f27c59d9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ae8686a100 con 0x55ae866ec4a0 2024-05-09T04:51:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.143+0000 7f27c59d9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ae86726b20 con 0x55ae86766d00 2024-05-09T04:51:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.143+0000 7f27c59d9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ae8687f1d0 con 0x55ae86869400 2024-05-09T04:51:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.143+0000 7f27b4d5f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae86872fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.143+0000 7f27affff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86766d00 0x55ae86873500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.143+0000 7f27b5560700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 0x55ae86873a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.143+0000 7f27affff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86766d00 0x55ae86873500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:41076/0 (socket says 172.21.15.46:41076) 2024-05-09T04:51:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.143+0000 7f27b4d5f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae86872fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44600/0 (socket says 172.21.15.46:44600) 2024-05-09T04:51:20.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.143+0000 7f27affff700 1 -- 172.21.15.46:0/1642090957 learned_addr learned my addr 172.21.15.46:0/1642090957 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.143+0000 7f27c59d9b80 10 client.0 _ll_register_callbacks cb 0x55ae865fd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:51:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.144+0000 7f27b5560700 1 -- 172.21.15.46:0/1642090957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86766d00 msgr2=0x55ae86873500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.144+0000 7f27b5560700 1 --2- 172.21.15.46:0/1642090957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86766d00 0x55ae86873500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.144+0000 7f27b5560700 1 -- 172.21.15.46:0/1642090957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 msgr2=0x55ae86872fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.144+0000 7f27b5560700 1 --2- 172.21.15.46:0/1642090957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae86872fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.144+0000 7f27b5560700 1 -- 172.21.15.46:0/1642090957 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae86607490 con 0x55ae86869400 2024-05-09T04:51:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.144+0000 7f27b5560700 1 --2- 172.21.15.46:0/1642090957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 0x55ae86873a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27a80df880 tx=0x7f27a8004f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79d101412a12b3e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.144+0000 7f278ffff700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:51:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.144+0000 7f278ffff700 1 -- 172.21.15.46:0/1642090957 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a80c2070 con 0x55ae86869400 2024-05-09T04:51:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.144+0000 7f278ffff700 1 -- 172.21.15.46:0/1642090957 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f27a80b1f40 con 0x55ae86869400 2024-05-09T04:51:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.144+0000 7f27c59d9b80 1 -- 172.21.15.46:0/1642090957 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27a40056b0 con 0x55ae86869400 2024-05-09T04:51:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.144+0000 7f278ffff700 1 -- 172.21.15.46:0/1642090957 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f27a80a9af0 con 0x55ae86869400 2024-05-09T04:51:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.144+0000 7f27c59d9b80 10 client.25066 fetch_fsmap learned FSMap version 25 2024-05-09T04:51:20.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.144+0000 7f27c59d9b80 1 -- 172.21.15.46:0/1642090957 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae86880ae0 con 0x55ae86869400 2024-05-09T04:51:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.145+0000 7f278ffff700 1 -- 172.21.15.46:0/1642090957 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a40056b0 con 0x55ae86869400 2024-05-09T04:51:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.145+0000 7f278ffff700 1 -- 172.21.15.46:0/1642090957 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f27a80a3cd0 con 0x55ae86869400 2024-05-09T04:51:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.145+0000 7f27c59d9b80 10 client.25066 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:51:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.145+0000 7f27c59d9b80 10 client.25066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:51:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.145+0000 7f27c59d9b80 20 client.25066 populate_metadata read hostname 'smithi046' 2024-05-09T04:51:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.145+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/1642090957 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae86883120 0x55ae868855c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.145+0000 7f27c59d9b80 4 client.25066 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:51:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.145+0000 7f27c59d9b80 1 -- 172.21.15.46:0/1642090957 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae86799a80 con 0x55ae86883120 2024-05-09T04:51:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.145+0000 7f27b4d5f700 1 --2- 172.21.15.46:0/1642090957 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae86883120 0x55ae868855c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.146+0000 7f278ffff700 10 client.25066 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.146+0000 7f27b4d5f700 1 --2- 172.21.15.46:0/1642090957 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae86883120 0x55ae868855c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af65cb16ca5c3ac1 server_cookie=4764477e7393b5e5 in_seq=0 out_seq=0 2024-05-09T04:51:20.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.146+0000 7f278ffff700 1 -- 172.21.15.46:0/1642090957 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ae86799a80 con 0x55ae86883120 2024-05-09T04:51:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.146+0000 7f278ffff700 10 client.25066 handle_command_reply: tid=0 2024-05-09T04:51:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.146+0000 7f27c59d9b80 1 client.25066 shutdown 2024-05-09T04:51:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.146+0000 7f27c59d9b80 1 -- 172.21.15.46:0/1642090957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 msgr2=0x55ae86873a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.146+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/1642090957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 0x55ae86873a40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27a80df880 tx=0x7f27a8004f90 comp rx=0 tx=0).stop 2024-05-09T04:51:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.147+0000 7f27c59d9b80 1 -- 172.21.15.46:0/1642090957 shutdown_connections 2024-05-09T04:51:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.147+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/1642090957 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 0x55ae86873a40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.147+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/1642090957 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae86872fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.147+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/1642090957 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae86883120 0x55ae868855c0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.147+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/1642090957 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86766d00 0x55ae86873500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.147+0000 7f27c59d9b80 1 -- 172.21.15.46:0/1642090957 >> 172.21.15.46:0/1642090957 conn(0x55ae866eff70 msgr2=0x55ae8685dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.147+0000 7f278ffff700 0 client.25066 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.147+0000 7f27c59d9b80 1 -- 172.21.15.46:0/1642090957 shutdown_connections 2024-05-09T04:51:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.147+0000 7f27c59d9b80 1 -- 172.21.15.46:0/1642090957 wait complete. 2024-05-09T04:51:20.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.147+0000 7f27c59d9b80 20 client.25066 trim_cache size 0 max 16384 2024-05-09T04:51:20.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.167+0000 7f27c59d9b80 1 Processor -- start 2024-05-09T04:51:20.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.167+0000 7f27c59d9b80 1 -- start start 2024-05-09T04:51:20.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27c59d9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae86872790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27c59d9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86766d00 0x55ae8685e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27c59d9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 0x55ae86872f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27c59d9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01af570 con 0x55ae866ec4a0 2024-05-09T04:51:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27c59d9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01ace00 con 0x55ae86766d00 2024-05-09T04:51:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27c59d9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01aac50 con 0x55ae86869400 2024-05-09T04:51:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27b4d5f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae86872790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27b4d5f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae86872790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44608/0 (socket says 172.21.15.46:44608) 2024-05-09T04:51:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27b4d5f700 1 -- 172.21.15.46:0/2895448333 learned_addr learned my addr 172.21.15.46:0/2895448333 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27b5560700 1 --2- 172.21.15.46:0/2895448333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 0x55ae86872f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27affff700 1 --2- 172.21.15.46:0/2895448333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86766d00 0x55ae8685e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27b4d5f700 1 -- 172.21.15.46:0/2895448333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 msgr2=0x55ae86872f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27b4d5f700 1 --2- 172.21.15.46:0/2895448333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 0x55ae86872f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27b4d5f700 1 -- 172.21.15.46:0/2895448333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86766d00 msgr2=0x55ae8685e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:20.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27b4d5f700 1 --2- 172.21.15.46:0/2895448333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86766d00 0x55ae8685e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27b4d5f700 1 -- 172.21.15.46:0/2895448333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae86872cd0 con 0x55ae866ec4a0 2024-05-09T04:51:20.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27b5560700 1 --2- 172.21.15.46:0/2895448333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 0x55ae86872f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:51:20.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27b4d5f700 1 --2- 172.21.15.46:0/2895448333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae86872790 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f27a00043b0 tx=0x7f27a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=535cb60d09f6a2dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:20.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.168+0000 7f27967fc700 1 -- 172.21.15.46:0/2895448333 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0003880 con 0x55ae866ec4a0 2024-05-09T04:51:20.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.169+0000 7f27967fc700 1 -- 172.21.15.46:0/2895448333 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a0011e50 con 0x55ae866ec4a0 2024-05-09T04:51:20.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.169+0000 7f27967fc700 1 -- 172.21.15.46:0/2895448333 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a002f9c0 con 0x55ae866ec4a0 2024-05-09T04:51:20.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.169+0000 7f27c59d9b80 1 -- 172.21.15.46:0/2895448333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 msgr2=0x55ae86872790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:20.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.169+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/2895448333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae86872790 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f27a00043b0 tx=0x7f27a0014000 comp rx=0 tx=0).stop 2024-05-09T04:51:20.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.169+0000 7f27c59d9b80 1 -- 172.21.15.46:0/2895448333 shutdown_connections 2024-05-09T04:51:20.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.169+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/2895448333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 0x55ae86872f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.169+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/2895448333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae866ec4a0 0x55ae86872790 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.169+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/2895448333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae86766d00 0x55ae8685e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.169+0000 7f27c59d9b80 1 -- 172.21.15.46:0/2895448333 >> 172.21.15.46:0/2895448333 conn(0x55ae866eff70 msgr2=0x55ae8685dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:20.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.170+0000 7f27c59d9b80 1 -- 172.21.15.46:0/2895448333 shutdown_connections 2024-05-09T04:51:20.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.170+0000 7f27c59d9b80 1 -- 172.21.15.46:0/2895448333 wait complete. 2024-05-09T04:51:20.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.170+0000 7f27c59d9b80 1 -- start start 2024-05-09T04:51:20.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.170+0000 7f27c59d9b80 1 Processor -- start 2024-05-09T04:51:20.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.170+0000 7f27c59d9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae866ec4a0 0x55ae8687b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.170+0000 7f27c59d9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae86766d00 0x55ae86871780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.170+0000 7f27c59d9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 0x55ae86871cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.170+0000 7f27c59d9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01ab2c0 con 0x55ae86766d00 2024-05-09T04:51:20.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.170+0000 7f27c59d9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01ac730 con 0x55ae866ec4a0 2024-05-09T04:51:20.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.170+0000 7f27c59d9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01ab990 con 0x55ae86869400 2024-05-09T04:51:20.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.170+0000 7f27c59d9b80 10 client.0 _ll_register_callbacks cb 0x55ae865fd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:51:20.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f27affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae86766d00 0x55ae86871780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f27affff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae86766d00 0x55ae86871780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44624/0 (socket says 172.21.15.46:44624) 2024-05-09T04:51:20.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f27affff700 1 -- 172.21.15.46:0/620023408 learned_addr learned my addr 172.21.15.46:0/620023408 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:20.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f27b4d5f700 1 --2- 172.21.15.46:0/620023408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae866ec4a0 0x55ae8687b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f27b5560700 1 --2- 172.21.15.46:0/620023408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 0x55ae86871cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f27affff700 1 -- 172.21.15.46:0/620023408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 msgr2=0x55ae86871cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:20.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f27affff700 1 --2- 172.21.15.46:0/620023408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 0x55ae86871cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f27affff700 1 -- 172.21.15.46:0/620023408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae866ec4a0 msgr2=0x55ae8687b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:20.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f27affff700 1 --2- 172.21.15.46:0/620023408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae866ec4a0 0x55ae8687b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f27affff700 1 -- 172.21.15.46:0/620023408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae868723d0 con 0x55ae86766d00 2024-05-09T04:51:20.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f27b4d5f700 1 --2- 172.21.15.46:0/620023408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae866ec4a0 0x55ae8687b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:20.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f27affff700 1 --2- 172.21.15.46:0/620023408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae86766d00 0x55ae86871780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f27a40098e0 tx=0x7f27a4009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61dbdda64bfbb09b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:20.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f278ffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:51:20.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f278ffff700 1 -- 172.21.15.46:0/620023408 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a4027040 con 0x55ae86766d00 2024-05-09T04:51:20.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f278ffff700 1 -- 172.21.15.46:0/620023408 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f27a400b9f0 con 0x55ae86766d00 2024-05-09T04:51:20.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.171+0000 7f27c59d9b80 1 -- 172.21.15.46:0/620023408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27a0011e50 con 0x55ae86766d00 2024-05-09T04:51:20.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.172+0000 7f278ffff700 1 -- 172.21.15.46:0/620023408 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f27a403a050 con 0x55ae86766d00 2024-05-09T04:51:20.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.172+0000 7f27c59d9b80 10 client.15345 fetch_fsmap learned FSMap version 25 2024-05-09T04:51:20.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.172+0000 7f27c59d9b80 1 -- 172.21.15.46:0/620023408 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae868ed820 con 0x55ae86766d00 2024-05-09T04:51:20.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.172+0000 7f278ffff700 1 -- 172.21.15.46:0/620023408 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27a0011e50 con 0x55ae86766d00 2024-05-09T04:51:20.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.172+0000 7f278ffff700 1 -- 172.21.15.46:0/620023408 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f27a4035030 con 0x55ae86766d00 2024-05-09T04:51:20.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.172+0000 7f27c59d9b80 10 client.15345 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:51:20.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.172+0000 7f27c59d9b80 10 client.15345 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:51:20.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.172+0000 7f27c59d9b80 20 client.15345 populate_metadata read hostname 'smithi046' 2024-05-09T04:51:20.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.172+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/620023408 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae868ed9f0 0x55ae868efe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:20.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.172+0000 7f27c59d9b80 4 client.15345 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:51:20.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.172+0000 7f27c59d9b80 1 -- 172.21.15.46:0/620023408 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ae86793b20 con 0x55ae868ed9f0 2024-05-09T04:51:20.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.172+0000 7f27b4d5f700 1 --2- 172.21.15.46:0/620023408 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae868ed9f0 0x55ae868efe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:20.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f278ffff700 10 client.15345 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:20.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f27b4d5f700 1 --2- 172.21.15.46:0/620023408 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae868ed9f0 0x55ae868efe10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e11c05ba692ca38 server_cookie=26d819524f6cf2d8 in_seq=0 out_seq=0 2024-05-09T04:51:20.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f278ffff700 1 -- 172.21.15.46:0/620023408 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ae86793b20 con 0x55ae868ed9f0 2024-05-09T04:51:20.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f278ffff700 10 client.15345 handle_command_reply: tid=0 2024-05-09T04:51:20.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f27c59d9b80 1 client.15345 shutdown 2024-05-09T04:51:20.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f27c59d9b80 1 -- 172.21.15.46:0/620023408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae86766d00 msgr2=0x55ae86871780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:20.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/620023408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae86766d00 0x55ae86871780 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f27a40098e0 tx=0x7f27a4009670 comp rx=0 tx=0).stop 2024-05-09T04:51:20.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f27c59d9b80 1 -- 172.21.15.46:0/620023408 shutdown_connections 2024-05-09T04:51:20.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/620023408 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae86869400 0x55ae86871cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/620023408 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae868ed9f0 0x55ae868efe10 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f278ffff700 0 client.15345 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:20.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/620023408 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae866ec4a0 0x55ae8687b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f27c59d9b80 1 --2- 172.21.15.46:0/620023408 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae86766d00 0x55ae86871780 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f27c59d9b80 1 -- 172.21.15.46:0/620023408 >> 172.21.15.46:0/620023408 conn(0x55ae866eff70 msgr2=0x55ae8686fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:20.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f27c59d9b80 1 -- 172.21.15.46:0/620023408 shutdown_connections 2024-05-09T04:51:20.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.173+0000 7f27c59d9b80 1 -- 172.21.15.46:0/620023408 wait complete. 2024-05-09T04:51:20.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.174+0000 7f27c59d9b80 20 client.15345 trim_cache size 0 max 16384 2024-05-09T04:51:20.179 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:51:20.180 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T04:51:20.180 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-09T04:51:20.180 INFO:teuthology.orchestra.run.smithi046.stdout: "07fd9527-57a3-4e46-83d4-5599ec054bdf": { 2024-05-09T04:51:20.180 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-09T04:51:20.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "07fd9527-57a3-4e46-83d4-5599ec054bdf", 2024-05-09T04:51:20.180 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-09T04:51:20.180 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T04:51:20.180 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T04:51:20.180 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:51:20.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.174+0000 7f27b6fc3700 1 -- 172.21.15.46:0/2667534860 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f27980b2a80 msgr2=0x7f27980b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:20.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.174+0000 7f27b6fc3700 1 --2- 172.21.15.46:0/2667534860 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f27980b2a80 0x7f27980b4f90 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f27a0011490 tx=0x7f27a0009260 comp rx=0 tx=0).stop 2024-05-09T04:51:20.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.174+0000 7f27b6fc3700 1 -- 172.21.15.46:0/2667534860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b0108780 msgr2=0x7f27b019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:20.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.174+0000 7f27b6fc3700 1 --2- 172.21.15.46:0/2667534860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b0108780 0x7f27b019c710 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f27a800ef30 tx=0x7f27a800c3a0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.174+0000 7f27b6fc3700 1 -- 172.21.15.46:0/2667534860 shutdown_connections 2024-05-09T04:51:20.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.174+0000 7f27b6fc3700 1 --2- 172.21.15.46:0/2667534860 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f27980b2a80 0x7f27980b4f90 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.174+0000 7f27b6fc3700 1 --2- 172.21.15.46:0/2667534860 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f27b0108780 0x7f27b019c710 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.174+0000 7f27b6fc3700 1 --2- 172.21.15.46:0/2667534860 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f27b0103550 0x7f27b019bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.174+0000 7f27b6fc3700 1 --2- 172.21.15.46:0/2667534860 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f27b0107e30 0x7f27b019c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.174+0000 7f27b6fc3700 1 -- 172.21.15.46:0/2667534860 >> 172.21.15.46:0/2667534860 conn(0x7f27b0078e00 msgr2=0x7f27b0100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.174+0000 7f27b6fc3700 1 -- 172.21.15.46:0/2667534860 shutdown_connections 2024-05-09T04:51:20.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:20.174+0000 7f27b6fc3700 1 -- 172.21.15.46:0/2667534860 wait complete. 2024-05-09T04:51:20.835 INFO:tasks.cephfs.filesystem:scrub status for tag:07fd9527-57a3-4e46-83d4-5599ec054bdf - {'path': '/', 'tag': '07fd9527-57a3-4e46-83d4-5599ec054bdf', 'options': 'recursive,force'} 2024-05-09T04:51:21.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:20 smithi110 ceph-mon[34904]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-09T04:51:21.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:20 smithi046 ceph-mon[30136]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-09T04:51:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:20 smithi072 ceph-mon[34088]: pgmap v10: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 0 B/s wr, 11 op/s 2024-05-09T04:51:23.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:22 smithi110 ceph-mon[34904]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 41 op/s 2024-05-09T04:51:23.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:22 smithi046 ceph-mon[30136]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 41 op/s 2024-05-09T04:51:23.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:22 smithi072 ceph-mon[34088]: pgmap v11: 97 pgs: 97 active+clean; 1.1 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 38 KiB/s wr, 41 op/s 2024-05-09T04:51:25.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:24 smithi110 ceph-mon[34904]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-05-09T04:51:25.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:24 smithi046 ceph-mon[30136]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-05-09T04:51:25.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:24 smithi072 ceph-mon[34088]: pgmap v12: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 46 op/s 2024-05-09T04:51:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:26 smithi110 ceph-mon[34904]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-09T04:51:27.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:26 smithi046 ceph-mon[30136]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-09T04:51:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:26 smithi072 ceph-mon[34088]: pgmap v13: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-09T04:51:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:28 smithi110 ceph-mon[34904]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-05-09T04:51:29.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:28 smithi046 ceph-mon[30136]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-05-09T04:51:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:28 smithi072 ceph-mon[34088]: pgmap v14: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 77 KiB/s wr, 39 op/s 2024-05-09T04:51:31.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:30 smithi110 ceph-mon[34904]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-09T04:51:31.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:30 smithi046 ceph-mon[30136]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-09T04:51:31.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:30 smithi072 ceph-mon[34088]: pgmap v15: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 77 KiB/s wr, 37 op/s 2024-05-09T04:51:31.410 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.392+0000 7f9efb5d6640 1 -- 172.21.15.46:0/496702817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef4104400 msgr2=0x7f9ef41107d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.392+0000 7f9efb5d6640 1 --2- 172.21.15.46:0/496702817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef4104400 0x7f9ef41107d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f9eec009ca0 tx=0x7f9eec02a8c0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.411 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.392+0000 7f9efb5d6640 1 -- 172.21.15.46:0/496702817 shutdown_connections 2024-05-09T04:51:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.392+0000 7f9efb5d6640 1 --2- 172.21.15.46:0/496702817 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef4104400 0x7f9ef41107d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.392+0000 7f9efb5d6640 1 --2- 172.21.15.46:0/496702817 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ef410a150 0x7f9ef4103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.392+0000 7f9efb5d6640 1 --2- 172.21.15.46:0/496702817 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ef4109cb0 0x7f9ef4103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.392+0000 7f9efb5d6640 1 -- 172.21.15.46:0/496702817 >> 172.21.15.46:0/496702817 conn(0x7f9ef40793a0 msgr2=0x7f9ef40797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.392+0000 7f9efb5d6640 1 -- 172.21.15.46:0/496702817 shutdown_connections 2024-05-09T04:51:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.392+0000 7f9efb5d6640 1 -- 172.21.15.46:0/496702817 wait complete. 2024-05-09T04:51:31.412 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9efb5d6640 1 Processor -- start 2024-05-09T04:51:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9efb5d6640 1 -- start start 2024-05-09T04:51:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9efb5d6640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef4104400 0x7f9ef41a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9efb5d6640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ef4109cb0 0x7f9ef41a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9efb5d6640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ef410a150 0x7f9ef419a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9efb5d6640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4113ea0 con 0x7f9ef4104400 2024-05-09T04:51:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9efb5d6640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4113d20 con 0x7f9ef410a150 2024-05-09T04:51:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9efb5d6640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef4114020 con 0x7f9ef4109cb0 2024-05-09T04:51:31.413 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9ef9372640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef4104400 0x7f9ef41a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9ef9372640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef4104400 0x7f9ef41a0580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39832/0 (socket says 172.21.15.46:39832) 2024-05-09T04:51:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9ef9372640 1 -- 172.21.15.46:0/540299247 learned_addr learned my addr 172.21.15.46:0/540299247 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9ef9b73640 1 --2- 172.21.15.46:0/540299247 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ef410a150 0x7f9ef419a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9ef8b71640 1 --2- 172.21.15.46:0/540299247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ef4109cb0 0x7f9ef41a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9ef9372640 1 -- 172.21.15.46:0/540299247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ef4109cb0 msgr2=0x7f9ef41a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9ef9372640 1 --2- 172.21.15.46:0/540299247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ef4109cb0 0x7f9ef41a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9ef9372640 1 -- 172.21.15.46:0/540299247 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ef410a150 msgr2=0x7f9ef419a700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:31.414 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9ef9372640 1 --2- 172.21.15.46:0/540299247 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ef410a150 0x7f9ef419a700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9ef9372640 1 -- 172.21.15.46:0/540299247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef419ae30 con 0x7f9ef4104400 2024-05-09T04:51:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.393+0000 7f9ef9b73640 1 --2- 172.21.15.46:0/540299247 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ef410a150 0x7f9ef419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:51:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.394+0000 7f9ef9372640 1 --2- 172.21.15.46:0/540299247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef4104400 0x7f9ef41a0580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000d770 tx=0x7f9ef000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.394+0000 7f9eea7fc640 1 -- 172.21.15.46:0/540299247 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0017070 con 0x7f9ef4104400 2024-05-09T04:51:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.394+0000 7f9efb5d6640 1 -- 172.21.15.46:0/540299247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef419b0c0 con 0x7f9ef4104400 2024-05-09T04:51:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.394+0000 7f9eea7fc640 1 -- 172.21.15.46:0/540299247 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef0009c30 con 0x7f9ef4104400 2024-05-09T04:51:31.415 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.394+0000 7f9efb5d6640 1 -- 172.21.15.46:0/540299247 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef41a7320 con 0x7f9ef4104400 2024-05-09T04:51:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.394+0000 7f9eea7fc640 1 -- 172.21.15.46:0/540299247 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0021060 con 0x7f9ef4104400 2024-05-09T04:51:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.397+0000 7f9eea7fc640 1 -- 172.21.15.46:0/540299247 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f9ef001a070 con 0x7f9ef4104400 2024-05-09T04:51:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.397+0000 7f9f02485740 1 Processor -- start 2024-05-09T04:51:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.397+0000 7f9f02485740 1 -- start start 2024-05-09T04:51:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.397+0000 7f9f02485740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 0x559dabe68680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.416 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.397+0000 7f9f02485740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559daba9c2f0 0x559dabe68bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.397+0000 7f9f02485740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 0x559dabe69100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.397+0000 7f9f02485740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559dabdcd7c0 con 0x559daba9c2f0 2024-05-09T04:51:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.397+0000 7f9f02485740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559dabd690e0 con 0x559dabac51b0 2024-05-09T04:51:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.397+0000 7f9f02485740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559dabdc9780 con 0x559dabaa44d0 2024-05-09T04:51:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.397+0000 7f9ef9372640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559daba9c2f0 0x559dabe68bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.417 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.398+0000 7f9ef9372640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559daba9c2f0 0x559dabe68bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39840/0 (socket says 172.21.15.46:39840) 2024-05-09T04:51:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.398+0000 7f9ef9372640 1 -- 172.21.15.46:0/2829595185 learned_addr learned my addr 172.21.15.46:0/2829595185 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.398+0000 7f9ef9b73640 1 --2- 172.21.15.46:0/2829595185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 0x559dabe68680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.398+0000 7f9ef8b71640 1 --2- 172.21.15.46:0/2829595185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 0x559dabe69100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.398+0000 7f9eea7fc640 1 --2- 172.21.15.46:0/540299247 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9ee00b29e0 0x7f9ee00b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.418 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.398+0000 7f9eea7fc640 1 -- 172.21.15.46:0/540299247 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f9ef00e2730 con 0x7f9ef4104400 2024-05-09T04:51:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.398+0000 7f9ef9b73640 1 --2- 172.21.15.46:0/540299247 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9ee00b29e0 0x7f9ee00b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.398+0000 7f9ef9372640 1 -- 172.21.15.46:0/2829595185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 msgr2=0x559dabe69100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.398+0000 7f9ef9372640 1 --2- 172.21.15.46:0/2829595185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 0x559dabe69100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.398+0000 7f9ef9372640 1 -- 172.21.15.46:0/2829595185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 msgr2=0x559dabe68680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.398+0000 7f9ef9372640 1 --2- 172.21.15.46:0/2829595185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 0x559dabe68680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.419 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.398+0000 7f9ef9372640 1 -- 172.21.15.46:0/2829595185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dabd927c0 con 0x559daba9c2f0 2024-05-09T04:51:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.398+0000 7f9ef9372640 1 --2- 172.21.15.46:0/2829595185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559daba9c2f0 0x559dabe68bc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00dfd40 tx=0x7f9ef000de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d456291377f0446e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.398+0000 7f9edf7fe640 1 -- 172.21.15.46:0/2829595185 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00a9390 con 0x559daba9c2f0 2024-05-09T04:51:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.398+0000 7f9ef9b73640 1 --2- 172.21.15.46:0/540299247 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9ee00b29e0 0x7f9ee00b4ef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x559dabe657f0 tx=0x7f9eec02a8c0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.399+0000 7f9edf7fe640 1 -- 172.21.15.46:0/2829595185 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef0009de0 con 0x559daba9c2f0 2024-05-09T04:51:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.399+0000 7f9edf7fe640 1 -- 172.21.15.46:0/2829595185 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00e0050 con 0x559daba9c2f0 2024-05-09T04:51:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.399+0000 7f9f02485740 1 -- 172.21.15.46:0/2829595185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559daba9c2f0 msgr2=0x559dabe68bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.399+0000 7f9f02485740 1 --2- 172.21.15.46:0/2829595185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559daba9c2f0 0x559dabe68bc0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f9ef00dfd40 tx=0x7f9ef000de80 comp rx=0 tx=0).stop 2024-05-09T04:51:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.399+0000 7f9f02485740 1 -- 172.21.15.46:0/2829595185 shutdown_connections 2024-05-09T04:51:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.399+0000 7f9f02485740 1 --2- 172.21.15.46:0/2829595185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 0x559dabe69100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.420 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.399+0000 7f9f02485740 1 --2- 172.21.15.46:0/2829595185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559daba9c2f0 0x559dabe68bc0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.399+0000 7f9f02485740 1 --2- 172.21.15.46:0/2829595185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 0x559dabe68680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.399+0000 7f9f02485740 1 -- 172.21.15.46:0/2829595185 >> 172.21.15.46:0/2829595185 conn(0x559dabab9300 msgr2=0x559dabe64e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.400+0000 7f9f02485740 1 -- 172.21.15.46:0/2829595185 shutdown_connections 2024-05-09T04:51:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.400+0000 7f9f02485740 1 -- 172.21.15.46:0/2829595185 wait complete. 2024-05-09T04:51:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.400+0000 7f9f02485740 1 -- start start 2024-05-09T04:51:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.400+0000 7f9f02485740 1 Processor -- start 2024-05-09T04:51:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9f02485740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559daba9c2f0 0x559dabe72d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9f02485740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 0x559dabe732a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9f02485740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 0x559dabe78c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9f02485740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559dabe75750 con 0x559daba9c2f0 2024-05-09T04:51:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9f02485740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559dabd549c0 con 0x559dabac51b0 2024-05-09T04:51:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9f02485740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559dabe758d0 con 0x559dabaa44d0 2024-05-09T04:51:31.421 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9f02485740 10 client.0 _ll_register_callbacks cb 0x559dabd81020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:51:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9ef9372640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559daba9c2f0 0x559dabe72d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9ef9372640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559daba9c2f0 0x559dabe72d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39848/0 (socket says 172.21.15.46:39848) 2024-05-09T04:51:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9ef9372640 1 -- 172.21.15.46:0/234848426 learned_addr learned my addr 172.21.15.46:0/234848426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9ef9b73640 1 --2- 172.21.15.46:0/234848426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 0x559dabe78c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9ef8b71640 1 --2- 172.21.15.46:0/234848426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 0x559dabe732a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9ef9372640 1 -- 172.21.15.46:0/234848426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 msgr2=0x559dabe732a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9ef9372640 1 --2- 172.21.15.46:0/234848426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 0x559dabe732a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.422 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9ef9372640 1 -- 172.21.15.46:0/234848426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 msgr2=0x559dabe78c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9ef9372640 1 --2- 172.21.15.46:0/234848426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 0x559dabe78c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9ef9372640 1 -- 172.21.15.46:0/234848426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dabd7b1f0 con 0x559daba9c2f0 2024-05-09T04:51:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9ef8b71640 1 --2- 172.21.15.46:0/234848426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 0x559dabe732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:51:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9ef9b73640 1 --2- 172.21.15.46:0/234848426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 0x559dabe78c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:51:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9ef9372640 1 --2- 172.21.15.46:0/234848426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559daba9c2f0 0x559dabe72d40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0000c00 tx=0x7f9ef00a5d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82fef92157323873 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9edcff9640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:51:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.401+0000 7f9edcff9640 1 -- 172.21.15.46:0/234848426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00a9390 con 0x559daba9c2f0 2024-05-09T04:51:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.402+0000 7f9edcff9640 1 -- 172.21.15.46:0/234848426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f9ef00e2be0 con 0x559daba9c2f0 2024-05-09T04:51:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.402+0000 7f9f02485740 1 -- 172.21.15.46:0/234848426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ef0009de0 con 0x559daba9c2f0 2024-05-09T04:51:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.402+0000 7f9edcff9640 1 -- 172.21.15.46:0/234848426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9ef00e2e60 con 0x559daba9c2f0 2024-05-09T04:51:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.402+0000 7f9f02485740 10 client.15357 fetch_fsmap learned FSMap version 25 2024-05-09T04:51:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.402+0000 7f9f02485740 1 -- 172.21.15.46:0/234848426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559dabb30790 con 0x559daba9c2f0 2024-05-09T04:51:31.423 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.402+0000 7f9edcff9640 1 -- 172.21.15.46:0/234848426 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ef0009de0 con 0x559daba9c2f0 2024-05-09T04:51:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.402+0000 7f9edcff9640 1 -- 172.21.15.46:0/234848426 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9ef006f0d0 con 0x559daba9c2f0 2024-05-09T04:51:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.402+0000 7f9f02485740 10 client.15357 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:51:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.402+0000 7f9f02485740 10 client.15357 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:51:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.402+0000 7f9f02485740 20 client.15357 populate_metadata read hostname 'smithi046' 2024-05-09T04:51:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.402+0000 7f9f02485740 1 --2- 172.21.15.46:0/234848426 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559daba83d80 0x559dabe84db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.402+0000 7f9f02485740 4 client.15357 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:51:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.402+0000 7f9f02485740 1 -- 172.21.15.46:0/234848426 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dabddb3d0 con 0x559daba83d80 2024-05-09T04:51:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.403+0000 7f9ef8b71640 1 --2- 172.21.15.46:0/234848426 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559daba83d80 0x559dabe84db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.403+0000 7f9ef8b71640 1 --2- 172.21.15.46:0/234848426 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559daba83d80 0x559dabe84db0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f143bf9d68bac64 server_cookie=f5e00bba38c7a1c1 in_seq=0 out_seq=0 2024-05-09T04:51:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.403+0000 7f9edcff9640 10 client.15357 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.404+0000 7f9edcff9640 1 -- 172.21.15.46:0/234848426 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559dabddb3d0 con 0x559daba83d80 2024-05-09T04:51:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.404+0000 7f9edcff9640 10 client.15357 handle_command_reply: tid=0 2024-05-09T04:51:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.404+0000 7f9f02485740 1 client.15357 shutdown 2024-05-09T04:51:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.404+0000 7f9f02485740 1 -- 172.21.15.46:0/234848426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559daba9c2f0 msgr2=0x559dabe72d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:31.424 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.404+0000 7f9f02485740 1 --2- 172.21.15.46:0/234848426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559daba9c2f0 0x559dabe72d40 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f9ef0000c00 tx=0x7f9ef00a5d40 comp rx=0 tx=0).stop 2024-05-09T04:51:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.404+0000 7f9f02485740 1 -- 172.21.15.46:0/234848426 shutdown_connections 2024-05-09T04:51:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.404+0000 7f9f02485740 1 --2- 172.21.15.46:0/234848426 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559daba83d80 0x559dabe84db0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.404+0000 7f9f02485740 1 --2- 172.21.15.46:0/234848426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 0x559dabe732a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.404+0000 7f9edcff9640 0 client.15357 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.404+0000 7f9f02485740 1 --2- 172.21.15.46:0/234848426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 0x559dabe78c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.404+0000 7f9f02485740 1 --2- 172.21.15.46:0/234848426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559daba9c2f0 0x559dabe72d40 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.404+0000 7f9f02485740 1 -- 172.21.15.46:0/234848426 >> 172.21.15.46:0/234848426 conn(0x559dabab9300 msgr2=0x559dabe63700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.404+0000 7f9f02485740 1 -- 172.21.15.46:0/234848426 shutdown_connections 2024-05-09T04:51:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.404+0000 7f9f02485740 1 -- 172.21.15.46:0/234848426 wait complete. 2024-05-09T04:51:31.425 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.404+0000 7f9f02485740 20 client.15357 trim_cache size 0 max 16384 2024-05-09T04:51:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9f02485740 1 Processor -- start 2024-05-09T04:51:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9f02485740 1 -- start start 2024-05-09T04:51:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9f02485740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559daba9c2f0 0x559dabe75020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9f02485740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dabaa44d0 0x559dabe73220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9f02485740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 0x559dabe69420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9f02485740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41b26b0 con 0x559dabaa44d0 2024-05-09T04:51:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9f02485740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41aff80 con 0x559dabac51b0 2024-05-09T04:51:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9f02485740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41addd0 con 0x559daba9c2f0 2024-05-09T04:51:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9ef8b71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dabaa44d0 0x559dabe73220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9ef8b71640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dabaa44d0 0x559dabe73220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39852/0 (socket says 172.21.15.46:39852) 2024-05-09T04:51:31.432 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9ef8b71640 1 -- 172.21.15.46:0/131199418 learned_addr learned my addr 172.21.15.46:0/131199418 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9ef9372640 1 --2- 172.21.15.46:0/131199418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559daba9c2f0 0x559dabe75020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9ef9b73640 1 --2- 172.21.15.46:0/131199418 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 0x559dabe69420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9ef8b71640 1 -- 172.21.15.46:0/131199418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559daba9c2f0 msgr2=0x559dabe75020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9ef8b71640 1 --2- 172.21.15.46:0/131199418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559daba9c2f0 0x559dabe75020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9ef8b71640 1 -- 172.21.15.46:0/131199418 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 msgr2=0x559dabe69420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9ef8b71640 1 --2- 172.21.15.46:0/131199418 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 0x559dabe69420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9ef8b71640 1 -- 172.21.15.46:0/131199418 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dabba75f0 con 0x559dabaa44d0 2024-05-09T04:51:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.420+0000 7f9ef9372640 1 --2- 172.21.15.46:0/131199418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559daba9c2f0 0x559dabe75020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:51:31.433 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.421+0000 7f9ef8b71640 1 --2- 172.21.15.46:0/131199418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dabaa44d0 0x559dabe73220 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9ef410a580 tx=0x7f9ee4005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f9cf97057070ae0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.421+0000 7f9edeffd640 1 -- 172.21.15.46:0/131199418 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4026070 con 0x559dabaa44d0 2024-05-09T04:51:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.421+0000 7f9edeffd640 1 -- 172.21.15.46:0/131199418 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ee4004250 con 0x559dabaa44d0 2024-05-09T04:51:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.421+0000 7f9edeffd640 1 -- 172.21.15.46:0/131199418 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ee4020370 con 0x559dabaa44d0 2024-05-09T04:51:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.421+0000 7f9f02485740 1 -- 172.21.15.46:0/131199418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dabaa44d0 msgr2=0x559dabe73220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.421+0000 7f9f02485740 1 --2- 172.21.15.46:0/131199418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dabaa44d0 0x559dabe73220 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f9ef410a580 tx=0x7f9ee4005f90 comp rx=0 tx=0).stop 2024-05-09T04:51:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.421+0000 7f9f02485740 1 -- 172.21.15.46:0/131199418 shutdown_connections 2024-05-09T04:51:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.422+0000 7f9f02485740 1 --2- 172.21.15.46:0/131199418 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559dabac51b0 0x559dabe69420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.422+0000 7f9f02485740 1 --2- 172.21.15.46:0/131199418 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dabaa44d0 0x559dabe73220 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.422+0000 7f9f02485740 1 --2- 172.21.15.46:0/131199418 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559daba9c2f0 0x559dabe75020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.422+0000 7f9f02485740 1 -- 172.21.15.46:0/131199418 >> 172.21.15.46:0/131199418 conn(0x559dabab9300 msgr2=0x559dabe72b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.422+0000 7f9f02485740 1 -- 172.21.15.46:0/131199418 shutdown_connections 2024-05-09T04:51:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.422+0000 7f9f02485740 1 -- 172.21.15.46:0/131199418 wait complete. 2024-05-09T04:51:31.434 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.422+0000 7f9f02485740 1 -- start start 2024-05-09T04:51:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.422+0000 7f9f02485740 1 Processor -- start 2024-05-09T04:51:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.422+0000 7f9f02485740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559daba9c2f0 0x559dabe7f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.422+0000 7f9f02485740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 0x559dabe7ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9f02485740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dabac51b0 0x559dabe7cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9f02485740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41ae440 con 0x559dabac51b0 2024-05-09T04:51:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9f02485740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41af8b0 con 0x559daba9c2f0 2024-05-09T04:51:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9f02485740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef41aeb10 con 0x559dabaa44d0 2024-05-09T04:51:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9f02485740 10 client.0 _ll_register_callbacks cb 0x7f9ee4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:51:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9ef9b73640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dabac51b0 0x559dabe7cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9ef9b73640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dabac51b0 0x559dabe7cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39854/0 (socket says 172.21.15.46:39854) 2024-05-09T04:51:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9ef9b73640 1 -- 172.21.15.46:0/2047921196 learned_addr learned my addr 172.21.15.46:0/2047921196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9ef9372640 1 --2- 172.21.15.46:0/2047921196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559daba9c2f0 0x559dabe7f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9ef8b71640 1 --2- 172.21.15.46:0/2047921196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 0x559dabe7ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9ef9b73640 1 -- 172.21.15.46:0/2047921196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 msgr2=0x559dabe7ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:31.435 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9ef9b73640 1 --2- 172.21.15.46:0/2047921196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 0x559dabe7ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9ef9b73640 1 -- 172.21.15.46:0/2047921196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559daba9c2f0 msgr2=0x559dabe7f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9ef9b73640 1 --2- 172.21.15.46:0/2047921196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559daba9c2f0 0x559dabe7f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9ef9b73640 1 -- 172.21.15.46:0/2047921196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dabd91ed0 con 0x559dabac51b0 2024-05-09T04:51:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9ef9372640 1 --2- 172.21.15.46:0/2047921196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559daba9c2f0 0x559dabe7f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:51:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9ef9b73640 1 --2- 172.21.15.46:0/2047921196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dabac51b0 0x559dabe7cf70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9eec035000 tx=0x7f9eec0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48aa753e3b45ef0e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9edcff9640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:51:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9edcff9640 1 -- 172.21.15.46:0/2047921196 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eec045070 con 0x559dabac51b0 2024-05-09T04:51:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9edcff9640 1 -- 172.21.15.46:0/2047921196 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f9eec004450 con 0x559dabac51b0 2024-05-09T04:51:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.423+0000 7f9f02485740 1 -- 172.21.15.46:0/2047921196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ee4004250 con 0x559dabac51b0 2024-05-09T04:51:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.424+0000 7f9edcff9640 1 -- 172.21.15.46:0/2047921196 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9eec004650 con 0x559dabac51b0 2024-05-09T04:51:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.424+0000 7f9f02485740 10 client.15363 fetch_fsmap learned FSMap version 25 2024-05-09T04:51:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.424+0000 7f9f02485740 1 -- 172.21.15.46:0/2047921196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559dabef0e80 con 0x559dabac51b0 2024-05-09T04:51:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.424+0000 7f9edcff9640 1 -- 172.21.15.46:0/2047921196 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ee4004250 con 0x559dabac51b0 2024-05-09T04:51:31.436 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.424+0000 7f9edcff9640 1 -- 172.21.15.46:0/2047921196 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9eec055020 con 0x559dabac51b0 2024-05-09T04:51:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.424+0000 7f9f02485740 10 client.15363 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:51:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.424+0000 7f9f02485740 10 client.15363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:51:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.424+0000 7f9f02485740 20 client.15363 populate_metadata read hostname 'smithi046' 2024-05-09T04:51:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.424+0000 7f9f02485740 1 --2- 172.21.15.46:0/2047921196 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559daba591c0 0x559dabef35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.424+0000 7f9f02485740 4 client.15363 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T04:51:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.424+0000 7f9f02485740 1 -- 172.21.15.46:0/2047921196 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559dabdd5240 con 0x559daba591c0 2024-05-09T04:51:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.424+0000 7f9ef9372640 1 --2- 172.21.15.46:0/2047921196 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559daba591c0 0x559dabef35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.425+0000 7f9ef9372640 1 --2- 172.21.15.46:0/2047921196 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559daba591c0 0x559dabef35e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=715a71ca8c61408d server_cookie=d2d7d0e162492e72 in_seq=0 out_seq=0 2024-05-09T04:51:31.437 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.425+0000 7f9edcff9640 10 client.15363 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:31.439 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.439+0000 7f9edcff9640 1 -- 172.21.15.46:0/2047921196 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x559dabdd5240 con 0x559daba591c0 2024-05-09T04:51:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.439+0000 7f9edcff9640 10 client.15363 handle_command_reply: tid=0 2024-05-09T04:51:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.439+0000 7f9f02485740 1 client.15363 shutdown 2024-05-09T04:51:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.439+0000 7f9f02485740 1 -- 172.21.15.46:0/2047921196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dabac51b0 msgr2=0x559dabe7cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.439+0000 7f9f02485740 1 --2- 172.21.15.46:0/2047921196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dabac51b0 0x559dabe7cf70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f9eec035000 tx=0x7f9eec0095c0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.439+0000 7f9f02485740 1 -- 172.21.15.46:0/2047921196 shutdown_connections 2024-05-09T04:51:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.439+0000 7f9f02485740 1 --2- 172.21.15.46:0/2047921196 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559daba591c0 0x559dabef35e0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.439+0000 7f9f02485740 1 --2- 172.21.15.46:0/2047921196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559dabaa44d0 0x559dabe7ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.439+0000 7f9edcff9640 0 client.15363 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.439+0000 7f9f02485740 1 --2- 172.21.15.46:0/2047921196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559dabac51b0 0x559dabe7cf70 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.439+0000 7f9f02485740 1 --2- 172.21.15.46:0/2047921196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559daba9c2f0 0x559dabe7f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.439+0000 7f9f02485740 1 -- 172.21.15.46:0/2047921196 >> 172.21.15.46:0/2047921196 conn(0x559dabab9300 msgr2=0x559dabe64750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.439+0000 7f9f02485740 1 -- 172.21.15.46:0/2047921196 shutdown_connections 2024-05-09T04:51:31.440 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.439+0000 7f9f02485740 1 -- 172.21.15.46:0/2047921196 wait complete. 2024-05-09T04:51:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.439+0000 7f9f02485740 20 client.15363 trim_cache size 0 max 16384 2024-05-09T04:51:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-05-09T04:51:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-05-09T04:51:31.441 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-05-09T04:51:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.440+0000 7f9efb5d6640 1 -- 172.21.15.46:0/540299247 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9ee00b29e0 msgr2=0x7f9ee00b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.440+0000 7f9efb5d6640 1 --2- 172.21.15.46:0/540299247 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9ee00b29e0 0x7f9ee00b4ef0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x559dabe657f0 tx=0x7f9eec02a8c0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.440+0000 7f9efb5d6640 1 -- 172.21.15.46:0/540299247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef4104400 msgr2=0x7f9ef41a0580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.440+0000 7f9efb5d6640 1 --2- 172.21.15.46:0/540299247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef4104400 0x7f9ef41a0580 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f9ef000d770 tx=0x7f9ef000dc30 comp rx=0 tx=0).stop 2024-05-09T04:51:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.440+0000 7f9efb5d6640 1 -- 172.21.15.46:0/540299247 shutdown_connections 2024-05-09T04:51:31.443 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.440+0000 7f9efb5d6640 1 --2- 172.21.15.46:0/540299247 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9ee00b29e0 0x7f9ee00b4ef0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.440+0000 7f9efb5d6640 1 --2- 172.21.15.46:0/540299247 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9ef4109cb0 0x7f9ef41a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.440+0000 7f9efb5d6640 1 --2- 172.21.15.46:0/540299247 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9ef410a150 0x7f9ef419a700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.440+0000 7f9efb5d6640 1 --2- 172.21.15.46:0/540299247 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9ef4104400 0x7f9ef41a0580 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.440+0000 7f9efb5d6640 1 -- 172.21.15.46:0/540299247 >> 172.21.15.46:0/540299247 conn(0x7f9ef40793a0 msgr2=0x7f9ef4100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.440+0000 7f9efb5d6640 1 -- 172.21.15.46:0/540299247 shutdown_connections 2024-05-09T04:51:31.444 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:31.440+0000 7f9efb5d6640 1 -- 172.21.15.46:0/540299247 wait complete. 2024-05-09T04:51:31.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:31 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:51:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:31 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:51:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:31 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:51:33.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:32 smithi110 ceph-mon[34904]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 78 KiB/s wr, 42 op/s 2024-05-09T04:51:33.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:32 smithi046 ceph-mon[30136]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 78 KiB/s wr, 42 op/s 2024-05-09T04:51:33.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:32 smithi072 ceph-mon[34088]: pgmap v16: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 78 KiB/s wr, 42 op/s 2024-05-09T04:51:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:34 smithi110 ceph-mon[34904]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-05-09T04:51:35.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:34 smithi046 ceph-mon[30136]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-05-09T04:51:35.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:34 smithi072 ceph-mon[34088]: pgmap v17: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 41 KiB/s wr, 14 op/s 2024-05-09T04:51:37.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:36 smithi110 ceph-mon[34904]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-09T04:51:37.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:36 smithi046 ceph-mon[30136]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-09T04:51:37.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:36 smithi072 ceph-mon[34088]: pgmap v18: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-09T04:51:38.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:37 smithi046 ceph-mon[30136]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-09T04:51:38.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:37 smithi072 ceph-mon[34088]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-09T04:51:38.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:37 smithi110 ceph-mon[34904]: pgmap v19: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-09T04:51:40.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:40 smithi072 ceph-mon[34088]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-09T04:51:41.010 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:40 smithi110 ceph-mon[34904]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-09T04:51:41.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:40 smithi046 ceph-mon[30136]: pgmap v20: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 9 op/s 2024-05-09T04:51:42.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:42 smithi072 ceph-mon[34088]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-09T04:51:43.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:42 smithi110 ceph-mon[34904]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-09T04:51:43.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:42 smithi046 ceph-mon[30136]: pgmap v21: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 11 op/s 2024-05-09T04:51:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.553+0000 7fc5e1976640 1 -- 172.21.15.46:0/145051180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc10b850 msgr2=0x7fc5dc10dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.553+0000 7fc5e1976640 1 --2- 172.21.15.46:0/145051180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc10b850 0x7fc5dc10dc70 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0009900 tx=0x7fc5d002a510 comp rx=0 tx=0).stop 2024-05-09T04:51:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.553+0000 7fc5e1976640 1 -- 172.21.15.46:0/145051180 shutdown_connections 2024-05-09T04:51:44.554 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.553+0000 7fc5e1976640 1 --2- 172.21.15.46:0/145051180 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5dc10e200 0x7fc5dc1106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.553+0000 7fc5e1976640 1 --2- 172.21.15.46:0/145051180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc10b850 0x7fc5dc10dc70 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.553+0000 7fc5e1976640 1 --2- 172.21.15.46:0/145051180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5dc104df0 0x7fc5dc105200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.553+0000 7fc5e1976640 1 -- 172.21.15.46:0/145051180 >> 172.21.15.46:0/145051180 conn(0x7fc5dc0fe390 msgr2=0x7fc5dc1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.553+0000 7fc5e1976640 1 -- 172.21.15.46:0/145051180 shutdown_connections 2024-05-09T04:51:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.553+0000 7fc5e1976640 1 -- 172.21.15.46:0/145051180 wait complete. 2024-05-09T04:51:44.555 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.553+0000 7fc5e1976640 1 Processor -- start 2024-05-09T04:51:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.554+0000 7fc5e1976640 1 -- start start 2024-05-09T04:51:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.554+0000 7fc5e1976640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5dc104df0 0x7fc5dc1a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.554+0000 7fc5e1976640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5dc10b850 0x7fc5dc1a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.554+0000 7fc5e1976640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc10e200 0x7fc5dc19a700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.554+0000 7fc5e1976640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc113e60 con 0x7fc5dc10e200 2024-05-09T04:51:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.554+0000 7fc5e1976640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc113ce0 con 0x7fc5dc10b850 2024-05-09T04:51:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.554+0000 7fc5e1976640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc113fe0 con 0x7fc5dc104df0 2024-05-09T04:51:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.554+0000 7fc5db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc10e200 0x7fc5dc19a700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.554+0000 7fc5db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc10e200 0x7fc5dc19a700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47016/0 (socket says 172.21.15.46:47016) 2024-05-09T04:51:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.554+0000 7fc5db7fe640 1 -- 172.21.15.46:0/1692515844 learned_addr learned my addr 172.21.15.46:0/1692515844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.554+0000 7fc5da7fc640 1 --2- 172.21.15.46:0/1692515844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5dc10b850 0x7fc5dc1a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.554+0000 7fc5daffd640 1 --2- 172.21.15.46:0/1692515844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5dc104df0 0x7fc5dc1a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.555+0000 7fc5db7fe640 1 -- 172.21.15.46:0/1692515844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5dc104df0 msgr2=0x7fc5dc1a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:44.557 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.555+0000 7fc5db7fe640 1 --2- 172.21.15.46:0/1692515844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5dc104df0 0x7fc5dc1a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.555+0000 7fc5db7fe640 1 -- 172.21.15.46:0/1692515844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5dc10b850 msgr2=0x7fc5dc1a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.555+0000 7fc5db7fe640 1 --2- 172.21.15.46:0/1692515844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5dc10b850 0x7fc5dc1a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.555+0000 7fc5db7fe640 1 -- 172.21.15.46:0/1692515844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc19ae30 con 0x7fc5dc10e200 2024-05-09T04:51:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.555+0000 7fc5db7fe640 1 --2- 172.21.15.46:0/1692515844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc10e200 0x7fc5dc19a700 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400eb10 tx=0x7fc5d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.555+0000 7fc5e0974640 1 -- 172.21.15.46:0/1692515844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4004b20 con 0x7fc5dc10e200 2024-05-09T04:51:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.555+0000 7fc5e0974640 1 -- 172.21.15.46:0/1692515844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5d4004cc0 con 0x7fc5dc10e200 2024-05-09T04:51:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.555+0000 7fc5e1976640 1 -- 172.21.15.46:0/1692515844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc19b0c0 con 0x7fc5dc10e200 2024-05-09T04:51:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.555+0000 7fc5e0974640 1 -- 172.21.15.46:0/1692515844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4010670 con 0x7fc5dc10e200 2024-05-09T04:51:44.558 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.555+0000 7fc5e1976640 1 -- 172.21.15.46:0/1692515844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc1a7320 con 0x7fc5dc10e200 2024-05-09T04:51:44.560 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.558+0000 7fc5e882d740 1 Processor -- start 2024-05-09T04:51:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.558+0000 7fc5e0974640 1 -- 172.21.15.46:0/1692515844 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fc5d4005020 con 0x7fc5dc10e200 2024-05-09T04:51:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.558+0000 7fc5e882d740 1 -- start start 2024-05-09T04:51:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.558+0000 7fc5e882d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9b31b0 0x556b8fd56510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.558+0000 7fc5e882d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f98a2f0 0x556b8fd56a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.558+0000 7fc5e882d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f9924d0 0x556b8fd56f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.558+0000 7fc5e882d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556b8fcbb7c0 con 0x556b8f98a2f0 2024-05-09T04:51:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.558+0000 7fc5e882d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556b8fc570e0 con 0x556b8f9b31b0 2024-05-09T04:51:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.558+0000 7fc5e882d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556b8fcb7780 con 0x556b8f9924d0 2024-05-09T04:51:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.558+0000 7fc5daffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f98a2f0 0x556b8fd56a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5daffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f98a2f0 0x556b8fd56a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47022/0 (socket says 172.21.15.46:47022) 2024-05-09T04:51:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5daffd640 1 -- 172.21.15.46:0/1314937493 learned_addr learned my addr 172.21.15.46:0/1314937493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5da7fc640 1 --2- 172.21.15.46:0/1314937493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f9924d0 0x556b8fd56f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5da7fc640 1 --2- 172.21.15.46:0/1314937493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9b31b0 0x556b8fd56510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5daffd640 1 -- 172.21.15.46:0/1314937493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f9924d0 msgr2=0x556b8fd56f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:44.561 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5daffd640 1 --2- 172.21.15.46:0/1314937493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f9924d0 0x556b8fd56f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5daffd640 1 -- 172.21.15.46:0/1314937493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9b31b0 msgr2=0x556b8fd56510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5daffd640 1 --2- 172.21.15.46:0/1314937493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9b31b0 0x556b8fd56510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5daffd640 1 -- 172.21.15.46:0/1314937493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8fc807c0 con 0x556b8f98a2f0 2024-05-09T04:51:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5da7fc640 1 --2- 172.21.15.46:0/1314937493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9b31b0 0x556b8fd56510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:51:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5e0974640 1 --2- 172.21.15.46:0/1692515844 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5c40b2a60 0x7fc5c40b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5daffd640 1 --2- 172.21.15.46:0/1314937493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f98a2f0 0x556b8fd56a50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00b760 tx=0x7fc5cc00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ddea976931541b1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5da7fc640 1 --2- 172.21.15.46:0/1692515844 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5c40b2a60 0x7fc5c40b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5e0974640 1 -- 172.21.15.46:0/1692515844 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fc5d40e2ca0 con 0x7fc5dc10e200 2024-05-09T04:51:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5c1ffb640 1 -- 172.21.15.46:0/1314937493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc003dd0 con 0x556b8f98a2f0 2024-05-09T04:51:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5c1ffb640 1 -- 172.21.15.46:0/1314937493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5cc015070 con 0x556b8f98a2f0 2024-05-09T04:51:44.562 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.559+0000 7fc5c1ffb640 1 -- 172.21.15.46:0/1314937493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc004da0 con 0x556b8f98a2f0 2024-05-09T04:51:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.560+0000 7fc5e882d740 1 -- 172.21.15.46:0/1314937493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f98a2f0 msgr2=0x556b8fd56a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.560+0000 7fc5e882d740 1 --2- 172.21.15.46:0/1314937493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f98a2f0 0x556b8fd56a50 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc00b760 tx=0x7fc5cc00bc20 comp rx=0 tx=0).stop 2024-05-09T04:51:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.560+0000 7fc5da7fc640 1 --2- 172.21.15.46:0/1692515844 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5c40b2a60 0x7fc5c40b4f70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0031440 tx=0x7fc5d0005780 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:44.568 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.560+0000 7fc5e882d740 1 -- 172.21.15.46:0/1314937493 shutdown_connections 2024-05-09T04:51:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.560+0000 7fc5e882d740 1 --2- 172.21.15.46:0/1314937493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f9924d0 0x556b8fd56f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.560+0000 7fc5e882d740 1 --2- 172.21.15.46:0/1314937493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f98a2f0 0x556b8fd56a50 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.560+0000 7fc5e882d740 1 --2- 172.21.15.46:0/1314937493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9b31b0 0x556b8fd56510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.560+0000 7fc5e882d740 1 -- 172.21.15.46:0/1314937493 >> 172.21.15.46:0/1314937493 conn(0x556b8f9a7300 msgr2=0x556b8fd52e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.560+0000 7fc5e882d740 1 -- 172.21.15.46:0/1314937493 shutdown_connections 2024-05-09T04:51:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.560+0000 7fc5e882d740 1 -- 172.21.15.46:0/1314937493 wait complete. 2024-05-09T04:51:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.561+0000 7fc5e882d740 1 -- start start 2024-05-09T04:51:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.561+0000 7fc5e882d740 1 Processor -- start 2024-05-09T04:51:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.561+0000 7fc5e882d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f98a2f0 0x556b8fd651d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.561+0000 7fc5e882d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f9924d0 0x556b8fd65710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.561+0000 7fc5e882d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f9b31b0 0x556b8fd6b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.561+0000 7fc5e882d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556b8fd67c20 con 0x556b8f9b31b0 2024-05-09T04:51:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.561+0000 7fc5e882d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556b8fc429c0 con 0x556b8f98a2f0 2024-05-09T04:51:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.561+0000 7fc5e882d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556b8fd67da0 con 0x556b8f9924d0 2024-05-09T04:51:44.569 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.561+0000 7fc5e882d740 10 client.0 _ll_register_callbacks cb 0x556b8fc6f020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:51:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.561+0000 7fc5db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f9b31b0 0x556b8fd6b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f9b31b0 0x556b8fd6b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47024/0 (socket says 172.21.15.46:47024) 2024-05-09T04:51:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5db7fe640 1 -- 172.21.15.46:0/3716251425 learned_addr learned my addr 172.21.15.46:0/3716251425 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5daffd640 1 --2- 172.21.15.46:0/3716251425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f98a2f0 0x556b8fd651d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5da7fc640 1 --2- 172.21.15.46:0/3716251425 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f9924d0 0x556b8fd65710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5db7fe640 1 -- 172.21.15.46:0/3716251425 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f9924d0 msgr2=0x556b8fd65710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5db7fe640 1 --2- 172.21.15.46:0/3716251425 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f9924d0 0x556b8fd65710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5db7fe640 1 -- 172.21.15.46:0/3716251425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f98a2f0 msgr2=0x556b8fd651d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5db7fe640 1 --2- 172.21.15.46:0/3716251425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f98a2f0 0x556b8fd651d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5db7fe640 1 -- 172.21.15.46:0/3716251425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8fc691f0 con 0x556b8f9b31b0 2024-05-09T04:51:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5db7fe640 1 --2- 172.21.15.46:0/3716251425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f9b31b0 0x556b8fd6b110 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40e0340 tx=0x7fc5d40e0840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1d0a2db09aec40 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5baffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:51:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5baffd640 1 -- 172.21.15.46:0/3716251425 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d40a98a0 con 0x556b8f9b31b0 2024-05-09T04:51:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5baffd640 1 -- 172.21.15.46:0/3716251425 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fc5d40a9ff0 con 0x556b8f9b31b0 2024-05-09T04:51:44.570 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5e882d740 1 -- 172.21.15.46:0/3716251425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5cc015070 con 0x556b8f9b31b0 2024-05-09T04:51:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5baffd640 1 -- 172.21.15.46:0/3716251425 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc5d40aa490 con 0x556b8f9b31b0 2024-05-09T04:51:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5e882d740 10 client.15375 fetch_fsmap learned FSMap version 25 2024-05-09T04:51:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5e882d740 1 -- 172.21.15.46:0/3716251425 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b8fa1e790 con 0x556b8f9b31b0 2024-05-09T04:51:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.562+0000 7fc5baffd640 1 -- 172.21.15.46:0/3716251425 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5cc015070 con 0x556b8f9b31b0 2024-05-09T04:51:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.563+0000 7fc5baffd640 1 -- 172.21.15.46:0/3716251425 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc5d40b2760 con 0x556b8f9b31b0 2024-05-09T04:51:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.563+0000 7fc5e882d740 10 client.15375 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:51:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.563+0000 7fc5e882d740 10 client.15375 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:51:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.563+0000 7fc5e882d740 20 client.15375 populate_metadata read hostname 'smithi046' 2024-05-09T04:51:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.563+0000 7fc5e882d740 1 --2- 172.21.15.46:0/3716251425 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8f971d80 0x556b8fd773c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.563+0000 7fc5e882d740 4 client.15375 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:51:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.563+0000 7fc5e882d740 1 -- 172.21.15.46:0/3716251425 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b8fcc93d0 con 0x556b8f971d80 2024-05-09T04:51:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.563+0000 7fc5daffd640 1 --2- 172.21.15.46:0/3716251425 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8f971d80 0x556b8fd773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.564+0000 7fc5baffd640 10 client.15375 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.564+0000 7fc5daffd640 1 --2- 172.21.15.46:0/3716251425 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8f971d80 0x556b8fd773c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=480de49af000b650 server_cookie=1d60db538e3bc212 in_seq=0 out_seq=0 2024-05-09T04:51:44.571 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.564+0000 7fc5baffd640 1 -- 172.21.15.46:0/3716251425 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556b8fcc93d0 con 0x556b8f971d80 2024-05-09T04:51:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.564+0000 7fc5baffd640 10 client.15375 handle_command_reply: tid=0 2024-05-09T04:51:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.564+0000 7fc5e882d740 1 client.15375 shutdown 2024-05-09T04:51:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.564+0000 7fc5e882d740 1 -- 172.21.15.46:0/3716251425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f9b31b0 msgr2=0x556b8fd6b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.565+0000 7fc5e882d740 1 --2- 172.21.15.46:0/3716251425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f9b31b0 0x556b8fd6b110 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40e0340 tx=0x7fc5d40e0840 comp rx=0 tx=0).stop 2024-05-09T04:51:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.565+0000 7fc5e882d740 1 -- 172.21.15.46:0/3716251425 shutdown_connections 2024-05-09T04:51:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.565+0000 7fc5e882d740 1 --2- 172.21.15.46:0/3716251425 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8f971d80 0x556b8fd773c0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.565+0000 7fc5e882d740 1 --2- 172.21.15.46:0/3716251425 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f9924d0 0x556b8fd65710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.565+0000 7fc5e882d740 1 --2- 172.21.15.46:0/3716251425 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f9b31b0 0x556b8fd6b110 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.565+0000 7fc5baffd640 0 client.15375 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.565+0000 7fc5e882d740 1 --2- 172.21.15.46:0/3716251425 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f98a2f0 0x556b8fd651d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.565+0000 7fc5e882d740 1 -- 172.21.15.46:0/3716251425 >> 172.21.15.46:0/3716251425 conn(0x556b8f9a7300 msgr2=0x556b8fd51700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.565+0000 7fc5e882d740 1 -- 172.21.15.46:0/3716251425 shutdown_connections 2024-05-09T04:51:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.565+0000 7fc5e882d740 1 -- 172.21.15.46:0/3716251425 wait complete. 2024-05-09T04:51:44.572 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.565+0000 7fc5e882d740 20 client.15375 trim_cache size 0 max 16384 2024-05-09T04:51:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.580+0000 7fc5e882d740 1 Processor -- start 2024-05-09T04:51:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.580+0000 7fc5e882d740 1 -- start start 2024-05-09T04:51:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.580+0000 7fc5e882d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f98a2f0 0x556b8fd51e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.580+0000 7fc5e882d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9924d0 0x556b8fd5e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.580+0000 7fc5e882d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f9b31b0 0x556b8fd5ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.580+0000 7fc5e882d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc1b0350 con 0x556b8f98a2f0 2024-05-09T04:51:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.580+0000 7fc5e882d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc1adc20 con 0x556b8f9924d0 2024-05-09T04:51:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.580+0000 7fc5e882d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc1aba90 con 0x556b8f9b31b0 2024-05-09T04:51:44.582 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.580+0000 7fc5daffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f98a2f0 0x556b8fd51e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.580+0000 7fc5daffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f98a2f0 0x556b8fd51e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47028/0 (socket says 172.21.15.46:47028) 2024-05-09T04:51:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.580+0000 7fc5daffd640 1 -- 172.21.15.46:0/2695502586 learned_addr learned my addr 172.21.15.46:0/2695502586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.580+0000 7fc5db7fe640 1 --2- 172.21.15.46:0/2695502586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f9b31b0 0x556b8fd5ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.580+0000 7fc5da7fc640 1 --2- 172.21.15.46:0/2695502586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9924d0 0x556b8fd5e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.581+0000 7fc5daffd640 1 -- 172.21.15.46:0/2695502586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f9b31b0 msgr2=0x556b8fd5ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.581+0000 7fc5daffd640 1 --2- 172.21.15.46:0/2695502586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f9b31b0 0x556b8fd5ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.581+0000 7fc5daffd640 1 -- 172.21.15.46:0/2695502586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9924d0 msgr2=0x556b8fd5e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.581+0000 7fc5daffd640 1 --2- 172.21.15.46:0/2695502586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9924d0 0x556b8fd5e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.581+0000 7fc5daffd640 1 -- 172.21.15.46:0/2695502586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b8fa955f0 con 0x556b8f98a2f0 2024-05-09T04:51:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.581+0000 7fc5da7fc640 1 --2- 172.21.15.46:0/2695502586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9924d0 0x556b8fd5e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:51:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.581+0000 7fc5daffd640 1 --2- 172.21.15.46:0/2695502586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f98a2f0 0x556b8fd51e10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc004a30 tx=0x7fc5cc0049c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6fd3273f604ba6c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.581+0000 7fc5c17fa640 1 -- 172.21.15.46:0/2695502586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc003cf0 con 0x556b8f98a2f0 2024-05-09T04:51:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.581+0000 7fc5c17fa640 1 -- 172.21.15.46:0/2695502586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5cc02d070 con 0x556b8f98a2f0 2024-05-09T04:51:44.583 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.581+0000 7fc5c17fa640 1 -- 172.21.15.46:0/2695502586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5cc011800 con 0x556b8f98a2f0 2024-05-09T04:51:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.582+0000 7fc5e882d740 1 -- 172.21.15.46:0/2695502586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f98a2f0 msgr2=0x556b8fd51e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.582+0000 7fc5e882d740 1 --2- 172.21.15.46:0/2695502586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f98a2f0 0x556b8fd51e10 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc004a30 tx=0x7fc5cc0049c0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.582+0000 7fc5e882d740 1 -- 172.21.15.46:0/2695502586 shutdown_connections 2024-05-09T04:51:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.582+0000 7fc5e882d740 1 --2- 172.21.15.46:0/2695502586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f9b31b0 0x556b8fd5ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.582+0000 7fc5e882d740 1 --2- 172.21.15.46:0/2695502586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9924d0 0x556b8fd5e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.582+0000 7fc5e882d740 1 --2- 172.21.15.46:0/2695502586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f98a2f0 0x556b8fd51e10 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.582+0000 7fc5e882d740 1 -- 172.21.15.46:0/2695502586 >> 172.21.15.46:0/2695502586 conn(0x556b8f9a7300 msgr2=0x556b8fd51700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.582+0000 7fc5e882d740 1 -- 172.21.15.46:0/2695502586 shutdown_connections 2024-05-09T04:51:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.582+0000 7fc5e882d740 1 -- 172.21.15.46:0/2695502586 wait complete. 2024-05-09T04:51:44.587 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.582+0000 7fc5e882d740 1 -- start start 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.582+0000 7fc5e882d740 1 Processor -- start 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5e882d740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f98a2f0 0x556b8fd716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5e882d740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9924d0 0x556b8fd71c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5e882d740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f9b31b0 0x556b8fd6aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5e882d740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc1ac0e0 con 0x556b8f9b31b0 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5e882d740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc1ad550 con 0x556b8f9924d0 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5e882d740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc1ac7b0 con 0x556b8f98a2f0 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5e882d740 10 client.0 _ll_register_callbacks cb 0x7fc5d40e0140 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f9b31b0 0x556b8fd6aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5db7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f9b31b0 0x556b8fd6aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47034/0 (socket says 172.21.15.46:47034) 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5da7fc640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9924d0 0x556b8fd71c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5db7fe640 1 -- 172.21.15.46:0/1514532811 learned_addr learned my addr 172.21.15.46:0/1514532811 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5daffd640 1 --2- 172.21.15.46:0/1514532811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f98a2f0 0x556b8fd716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5db7fe640 1 -- 172.21.15.46:0/1514532811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f98a2f0 msgr2=0x556b8fd716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5db7fe640 1 --2- 172.21.15.46:0/1514532811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f98a2f0 0x556b8fd716f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5db7fe640 1 -- 172.21.15.46:0/1514532811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9924d0 msgr2=0x556b8fd71c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:44.588 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5db7fe640 1 --2- 172.21.15.46:0/1514532811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9924d0 0x556b8fd71c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5db7fe640 1 -- 172.21.15.46:0/1514532811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b8fc7fed0 con 0x556b8f9b31b0 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5db7fe640 1 --2- 172.21.15.46:0/1514532811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f9b31b0 0x556b8fd6aaa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40a9d20 tx=0x7fc5d400c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8b5fb92646ffb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.583+0000 7fc5baffd640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.584+0000 7fc5baffd640 1 -- 172.21.15.46:0/1514532811 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d40a5c50 con 0x556b8f9b31b0 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.584+0000 7fc5baffd640 1 -- 172.21.15.46:0/1514532811 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fc5d40a57d0 con 0x556b8f9b31b0 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.584+0000 7fc5e882d740 1 -- 172.21.15.46:0/1514532811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5cc02d070 con 0x556b8f9b31b0 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.584+0000 7fc5baffd640 1 -- 172.21.15.46:0/1514532811 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc5d40a96e0 con 0x556b8f9b31b0 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.584+0000 7fc5e882d740 10 client.15381 fetch_fsmap learned FSMap version 25 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.584+0000 7fc5e882d740 1 -- 172.21.15.46:0/1514532811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b8fd6fa00 con 0x556b8f9b31b0 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.584+0000 7fc5baffd640 1 -- 172.21.15.46:0/1514532811 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5cc02d070 con 0x556b8f9b31b0 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.584+0000 7fc5baffd640 1 -- 172.21.15.46:0/1514532811 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc5d40e0ac0 con 0x556b8f9b31b0 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.584+0000 7fc5e882d740 10 client.15381 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.584+0000 7fc5e882d740 10 client.15381 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.584+0000 7fc5e882d740 20 client.15381 populate_metadata read hostname 'smithi046' 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.584+0000 7fc5e882d740 1 --2- 172.21.15.46:0/1514532811 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8f9471c0 0x556b8fde1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:44.589 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.584+0000 7fc5e882d740 4 client.15381 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T04:51:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.584+0000 7fc5e882d740 1 -- 172.21.15.46:0/1514532811 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556b8fcc3240 con 0x556b8f9471c0 2024-05-09T04:51:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.585+0000 7fc5daffd640 1 --2- 172.21.15.46:0/1514532811 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8f9471c0 0x556b8fde1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.585+0000 7fc5daffd640 1 --2- 172.21.15.46:0/1514532811 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8f9471c0 0x556b8fde1780 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49aacd6bb27eb5bc server_cookie=a4cc261b1a858b09 in_seq=0 out_seq=0 2024-05-09T04:51:44.590 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.585+0000 7fc5baffd640 10 client.15381 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.593+0000 7fc5baffd640 1 -- 172.21.15.46:0/1514532811 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556b8fcc3240 con 0x556b8f9471c0 2024-05-09T04:51:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.593+0000 7fc5baffd640 10 client.15381 handle_command_reply: tid=0 2024-05-09T04:51:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.593+0000 7fc5e882d740 1 client.15381 shutdown 2024-05-09T04:51:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.594+0000 7fc5e882d740 1 -- 172.21.15.46:0/1514532811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f9b31b0 msgr2=0x556b8fd6aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.594+0000 7fc5e882d740 1 --2- 172.21.15.46:0/1514532811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f9b31b0 0x556b8fd6aaa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc5d40a9d20 tx=0x7fc5d400c9a0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.594+0000 7fc5e882d740 1 -- 172.21.15.46:0/1514532811 shutdown_connections 2024-05-09T04:51:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.594+0000 7fc5e882d740 1 --2- 172.21.15.46:0/1514532811 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8f9471c0 0x556b8fde1780 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.595 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.594+0000 7fc5e882d740 1 --2- 172.21.15.46:0/1514532811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b8f9b31b0 0x556b8fd6aaa0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.594+0000 7fc5e882d740 1 --2- 172.21.15.46:0/1514532811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b8f9924d0 0x556b8fd71c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.594+0000 7fc5baffd640 0 client.15381 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.594+0000 7fc5e882d740 1 --2- 172.21.15.46:0/1514532811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b8f98a2f0 0x556b8fd716f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.594+0000 7fc5e882d740 1 -- 172.21.15.46:0/1514532811 >> 172.21.15.46:0/1514532811 conn(0x556b8f9a7300 msgr2=0x556b8fd527b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.594+0000 7fc5e882d740 1 -- 172.21.15.46:0/1514532811 shutdown_connections 2024-05-09T04:51:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.594+0000 7fc5e882d740 1 -- 172.21.15.46:0/1514532811 wait complete. 2024-05-09T04:51:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.594+0000 7fc5e882d740 20 client.15381 trim_cache size 0 max 16384 2024-05-09T04:51:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-05-09T04:51:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-05-09T04:51:44.596 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-05-09T04:51:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.595+0000 7fc5e1976640 1 -- 172.21.15.46:0/1692515844 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5c40b2a60 msgr2=0x7fc5c40b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.595+0000 7fc5e1976640 1 --2- 172.21.15.46:0/1692515844 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5c40b2a60 0x7fc5c40b4f70 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc5d0031440 tx=0x7fc5d0005780 comp rx=0 tx=0).stop 2024-05-09T04:51:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.595+0000 7fc5e1976640 1 -- 172.21.15.46:0/1692515844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc10e200 msgr2=0x7fc5dc19a700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.595+0000 7fc5e1976640 1 --2- 172.21.15.46:0/1692515844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc10e200 0x7fc5dc19a700 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400eb10 tx=0x7fc5d400c2d0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.595+0000 7fc5e1976640 1 -- 172.21.15.46:0/1692515844 shutdown_connections 2024-05-09T04:51:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.595+0000 7fc5e1976640 1 --2- 172.21.15.46:0/1692515844 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5c40b2a60 0x7fc5c40b4f70 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.595+0000 7fc5e1976640 1 --2- 172.21.15.46:0/1692515844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc5dc10e200 0x7fc5dc19a700 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.595+0000 7fc5e1976640 1 --2- 172.21.15.46:0/1692515844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc5dc10b850 0x7fc5dc1a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.597 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.595+0000 7fc5e1976640 1 --2- 172.21.15.46:0/1692515844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc5dc104df0 0x7fc5dc1a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.595+0000 7fc5e1976640 1 -- 172.21.15.46:0/1692515844 >> 172.21.15.46:0/1692515844 conn(0x7fc5dc0fe390 msgr2=0x7fc5dc10e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.595+0000 7fc5e1976640 1 -- 172.21.15.46:0/1692515844 shutdown_connections 2024-05-09T04:51:44.598 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:44.595+0000 7fc5e1976640 1 -- 172.21.15.46:0/1692515844 wait complete. 2024-05-09T04:51:44.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:44 smithi072 ceph-mon[34088]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-09T04:51:45.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:44 smithi110 ceph-mon[34904]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-09T04:51:45.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:44 smithi046 ceph-mon[30136]: pgmap v22: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-09T04:51:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:46 smithi110 ceph-mon[34904]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:51:47.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:46 smithi046 ceph-mon[30136]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:51:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:46 smithi072 ceph-mon[34088]: pgmap v23: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:51:48.931 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:48 smithi072 ceph-mon[34088]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-09T04:51:49.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:48 smithi110 ceph-mon[34904]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-09T04:51:49.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:48 smithi046 ceph-mon[30136]: pgmap v24: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-09T04:51:50.836 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:51:51.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:50 smithi110 ceph-mon[34904]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T04:51:51.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:50 smithi046 ceph-mon[30136]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T04:51:51.105 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:51.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:50 smithi072 ceph-mon[34088]: pgmap v25: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T04:51:52.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.876+0000 7fa82e63a700 1 -- 172.21.15.46:0/54569314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa828109860 msgr2=0x7fa8281111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.876+0000 7fa82e63a700 1 --2- 172.21.15.46:0/54569314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa828109860 0x7fa8281111c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fa82000b0d0 tx=0x7fa82000b3e0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.876+0000 7fa82e63a700 1 -- 172.21.15.46:0/54569314 shutdown_connections 2024-05-09T04:51:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.876+0000 7fa82e63a700 1 --2- 172.21.15.46:0/54569314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa828109860 0x7fa8281111c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.876+0000 7fa82e63a700 1 --2- 172.21.15.46:0/54569314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa8281045e0 0x7fa8281049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.876+0000 7fa82e63a700 1 --2- 172.21.15.46:0/54569314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa828108f10 0x7fa828109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.876+0000 7fa82e63a700 1 -- 172.21.15.46:0/54569314 >> 172.21.15.46:0/54569314 conn(0x7fa8280ffed0 msgr2=0x7fa828102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.876+0000 7fa82e63a700 1 -- 172.21.15.46:0/54569314 shutdown_connections 2024-05-09T04:51:52.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.876+0000 7fa82e63a700 1 -- 172.21.15.46:0/54569314 wait complete. 2024-05-09T04:51:52.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.877+0000 7fa82e63a700 1 Processor -- start 2024-05-09T04:51:52.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.877+0000 7fa82e63a700 1 -- start start 2024-05-09T04:51:52.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.877+0000 7fa82e63a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8281045e0 0x7fa82819ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.877+0000 7fa82e63a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa828108f10 0x7fa82819b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.877+0000 7fa82e63a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa828109860 0x7fa82819f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.877+0000 7fa82e63a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281144b0 con 0x7fa8281045e0 2024-05-09T04:51:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.877+0000 7fa82e63a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa828114330 con 0x7fa828109860 2024-05-09T04:51:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.877+0000 7fa82e63a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa828114630 con 0x7fa828108f10 2024-05-09T04:51:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa827fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8281045e0 0x7fa82819ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa827fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8281045e0 0x7fa82819ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47058/0 (socket says 172.21.15.46:47058) 2024-05-09T04:51:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa827fff700 1 -- 172.21.15.46:0/3287537010 learned_addr learned my addr 172.21.15.46:0/3287537010 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa81ffff700 1 --2- 172.21.15.46:0/3287537010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa828108f10 0x7fa82819b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa82cbd7700 1 --2- 172.21.15.46:0/3287537010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa828109860 0x7fa82819f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa827fff700 1 -- 172.21.15.46:0/3287537010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa828108f10 msgr2=0x7fa82819b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa827fff700 1 --2- 172.21.15.46:0/3287537010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa828108f10 0x7fa82819b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa827fff700 1 -- 172.21.15.46:0/3287537010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa828109860 msgr2=0x7fa82819f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa827fff700 1 --2- 172.21.15.46:0/3287537010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa828109860 0x7fa82819f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa827fff700 1 -- 172.21.15.46:0/3287537010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa82819fc20 con 0x7fa8281045e0 2024-05-09T04:51:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa827fff700 1 --2- 172.21.15.46:0/3287537010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8281045e0 0x7fa82819ae00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa81800c9b0 tx=0x7fa81800ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:52.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa825ffb700 1 -- 172.21.15.46:0/3287537010 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818007b00 con 0x7fa8281045e0 2024-05-09T04:51:52.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa825ffb700 1 -- 172.21.15.46:0/3287537010 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa818007ca0 con 0x7fa8281045e0 2024-05-09T04:51:52.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa82e63a700 1 -- 172.21.15.46:0/3287537010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa82819ff10 con 0x7fa8281045e0 2024-05-09T04:51:52.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa825ffb700 1 -- 172.21.15.46:0/3287537010 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa818018660 con 0x7fa8281045e0 2024-05-09T04:51:52.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.878+0000 7fa82e63a700 1 -- 172.21.15.46:0/3287537010 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8281a0450 con 0x7fa8281045e0 2024-05-09T04:51:52.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.881+0000 7fa825ffb700 1 -- 172.21.15.46:0/3287537010 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fa818004830 con 0x7fa8281045e0 2024-05-09T04:51:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.882+0000 7fa825ffb700 1 --2- 172.21.15.46:0/3287537010 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa8140b2a80 0x7fa8140b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.882+0000 7fa81ffff700 1 --2- 172.21.15.46:0/3287537010 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa8140b2a80 0x7fa8140b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.882+0000 7fa825ffb700 1 -- 172.21.15.46:0/3287537010 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa8180e2810 con 0x7fa8281045e0 2024-05-09T04:51:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.882+0000 7fa83d050b80 1 Processor -- start 2024-05-09T04:51:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.882+0000 7fa83d050b80 1 -- start start 2024-05-09T04:51:52.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.882+0000 7fa81ffff700 1 --2- 172.21.15.46:0/3287537010 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa8140b2a80 0x7fa8140b4f90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa828107ce0 tx=0x7fa810009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa83d050b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd329160 0x559fcd3c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa83d050b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd23f4a0 0x559fcd3c2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa83d050b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd3bc6a0 0x559fcd3bcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa83d050b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559fcd2deff0 con 0x559fcd329160 2024-05-09T04:51:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa83d050b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559fcd27d7f0 con 0x559fcd23f4a0 2024-05-09T04:51:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa83d050b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559fcd2dafb0 con 0x559fcd3bc6a0 2024-05-09T04:51:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa827fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd329160 0x559fcd3c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa827fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd329160 0x559fcd3c24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47062/0 (socket says 172.21.15.46:47062) 2024-05-09T04:51:52.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa827fff700 1 -- 172.21.15.46:0/2539800680 learned_addr learned my addr 172.21.15.46:0/2539800680 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:52.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa81ffff700 1 --2- 172.21.15.46:0/2539800680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd23f4a0 0x559fcd3c2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa82cbd7700 1 --2- 172.21.15.46:0/2539800680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd3bc6a0 0x559fcd3bcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa827fff700 1 -- 172.21.15.46:0/2539800680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd3bc6a0 msgr2=0x559fcd3bcad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:52.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa827fff700 1 --2- 172.21.15.46:0/2539800680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd3bc6a0 0x559fcd3bcad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa827fff700 1 -- 172.21.15.46:0/2539800680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd23f4a0 msgr2=0x559fcd3c2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:52.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa827fff700 1 --2- 172.21.15.46:0/2539800680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd23f4a0 0x559fcd3c2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.883+0000 7fa827fff700 1 -- 172.21.15.46:0/2539800680 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcd275370 con 0x559fcd329160 2024-05-09T04:51:52.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.884+0000 7fa827fff700 1 --2- 172.21.15.46:0/2539800680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd329160 0x559fcd3c24f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa81800afd0 tx=0x7fa8180a92b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71650d3afcfcc09d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:52.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.884+0000 7fa81e7fc700 1 -- 172.21.15.46:0/2539800680 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8180b2110 con 0x559fcd329160 2024-05-09T04:51:52.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.884+0000 7fa81e7fc700 1 -- 172.21.15.46:0/2539800680 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8180a9470 con 0x559fcd329160 2024-05-09T04:51:52.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.884+0000 7fa81e7fc700 1 -- 172.21.15.46:0/2539800680 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8180b27c0 con 0x559fcd329160 2024-05-09T04:51:52.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.885+0000 7fa83d050b80 1 -- 172.21.15.46:0/2539800680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd329160 msgr2=0x559fcd3c24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:52.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.885+0000 7fa83d050b80 1 --2- 172.21.15.46:0/2539800680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd329160 0x559fcd3c24f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa81800afd0 tx=0x7fa8180a92b0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.885+0000 7fa83d050b80 1 -- 172.21.15.46:0/2539800680 shutdown_connections 2024-05-09T04:51:52.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.885+0000 7fa83d050b80 1 --2- 172.21.15.46:0/2539800680 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd3bc6a0 0x559fcd3bcad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.885+0000 7fa83d050b80 1 --2- 172.21.15.46:0/2539800680 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd329160 0x559fcd3c24f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.885+0000 7fa83d050b80 1 --2- 172.21.15.46:0/2539800680 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd23f4a0 0x559fcd3c2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.885+0000 7fa83d050b80 1 -- 172.21.15.46:0/2539800680 >> 172.21.15.46:0/2539800680 conn(0x559fcd29f6a0 msgr2=0x559fcd3b7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:52.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.885+0000 7fa83d050b80 1 -- 172.21.15.46:0/2539800680 shutdown_connections 2024-05-09T04:51:52.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.885+0000 7fa83d050b80 1 -- 172.21.15.46:0/2539800680 wait complete. 2024-05-09T04:51:52.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.885+0000 7fa83d050b80 1 -- start start 2024-05-09T04:51:52.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.885+0000 7fa83d050b80 1 Processor -- start 2024-05-09T04:51:52.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.886+0000 7fa83d050b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd23f4a0 0x559fcd3c62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.886+0000 7fa83d050b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd329160 0x559fcd3c6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.886+0000 7fa83d050b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd3bc6a0 0x559fcd3c6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:52 smithi046 ceph-mon[30136]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-09T04:51:52.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.886+0000 7fa83d050b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559fcd15aba0 con 0x559fcd23f4a0 2024-05-09T04:51:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.886+0000 7fa83d050b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559fcd299150 con 0x559fcd329160 2024-05-09T04:51:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.886+0000 7fa83d050b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559fcd3c91d0 con 0x559fcd3bc6a0 2024-05-09T04:51:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.886+0000 7fa83d050b80 10 client.0 _ll_register_callbacks cb 0x559fcd1506c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:51:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.886+0000 7fa827fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd23f4a0 0x559fcd3c62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.886+0000 7fa827fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd23f4a0 0x559fcd3c62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47070/0 (socket says 172.21.15.46:47070) 2024-05-09T04:51:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.886+0000 7fa827fff700 1 -- 172.21.15.46:0/1678139773 learned_addr learned my addr 172.21.15.46:0/1678139773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.886+0000 7fa81ffff700 1 --2- 172.21.15.46:0/1678139773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd329160 0x559fcd3c6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.886+0000 7fa82cbd7700 1 --2- 172.21.15.46:0/1678139773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd3bc6a0 0x559fcd3c6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.887+0000 7fa827fff700 1 -- 172.21.15.46:0/1678139773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd3bc6a0 msgr2=0x559fcd3c6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:52.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.887+0000 7fa827fff700 1 --2- 172.21.15.46:0/1678139773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd3bc6a0 0x559fcd3c6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.887+0000 7fa827fff700 1 -- 172.21.15.46:0/1678139773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd329160 msgr2=0x559fcd3c6830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.887+0000 7fa827fff700 1 --2- 172.21.15.46:0/1678139773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd329160 0x559fcd3c6830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.887+0000 7fa827fff700 1 -- 172.21.15.46:0/1678139773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fcd15a490 con 0x559fcd23f4a0 2024-05-09T04:51:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.887+0000 7fa81ffff700 1 --2- 172.21.15.46:0/1678139773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd329160 0x559fcd3c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.887+0000 7fa827fff700 1 --2- 172.21.15.46:0/1678139773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd23f4a0 0x559fcd3c62f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa818004e20 tx=0x7fa81800fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67de4794c515211 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.887+0000 7fa8077fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:51:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.887+0000 7fa8077fe700 1 -- 172.21.15.46:0/1678139773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8180b2110 con 0x559fcd23f4a0 2024-05-09T04:51:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.887+0000 7fa8077fe700 1 -- 172.21.15.46:0/1678139773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa8180a9dd0 con 0x559fcd23f4a0 2024-05-09T04:51:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.887+0000 7fa83d050b80 1 -- 172.21.15.46:0/1678139773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8180a9470 con 0x559fcd23f4a0 2024-05-09T04:51:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.887+0000 7fa8077fe700 1 -- 172.21.15.46:0/1678139773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa8180a9960 con 0x559fcd23f4a0 2024-05-09T04:51:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.887+0000 7fa83d050b80 10 client.15393 fetch_fsmap learned FSMap version 25 2024-05-09T04:51:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.887+0000 7fa83d050b80 1 -- 172.21.15.46:0/1678139773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559fcd3d3cc0 con 0x559fcd23f4a0 2024-05-09T04:51:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.888+0000 7fa8077fe700 1 -- 172.21.15.46:0/1678139773 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa8180a9470 con 0x559fcd23f4a0 2024-05-09T04:51:52.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.888+0000 7fa8077fe700 1 -- 172.21.15.46:0/1678139773 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa818070120 con 0x559fcd23f4a0 2024-05-09T04:51:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.888+0000 7fa83d050b80 10 client.15393 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:51:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.888+0000 7fa83d050b80 10 client.15393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:51:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.888+0000 7fa83d050b80 20 client.15393 populate_metadata read hostname 'smithi046' 2024-05-09T04:51:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.888+0000 7fa83d050b80 1 --2- 172.21.15.46:0/1678139773 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559fcd3d6190 0x559fcd3d85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.888+0000 7fa83d050b80 4 client.15393 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:51:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.888+0000 7fa83d050b80 1 -- 172.21.15.46:0/1678139773 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559fcd2ec9d0 con 0x559fcd3d6190 2024-05-09T04:51:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.888+0000 7fa81ffff700 1 --2- 172.21.15.46:0/1678139773 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559fcd3d6190 0x559fcd3d85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.889+0000 7fa8077fe700 10 client.15393 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.889+0000 7fa81ffff700 1 --2- 172.21.15.46:0/1678139773 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559fcd3d6190 0x559fcd3d85b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f979f37984dbe30 server_cookie=3757bb45e431b04d in_seq=0 out_seq=0 2024-05-09T04:51:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.889+0000 7fa8077fe700 1 -- 172.21.15.46:0/1678139773 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559fcd2ec9d0 con 0x559fcd3d6190 2024-05-09T04:51:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.889+0000 7fa8077fe700 10 client.15393 handle_command_reply: tid=0 2024-05-09T04:51:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.890+0000 7fa83d050b80 1 client.15393 shutdown 2024-05-09T04:51:52.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.890+0000 7fa83d050b80 1 -- 172.21.15.46:0/1678139773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd23f4a0 msgr2=0x559fcd3c62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.890+0000 7fa83d050b80 1 --2- 172.21.15.46:0/1678139773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd23f4a0 0x559fcd3c62f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa818004e20 tx=0x7fa81800fc60 comp rx=0 tx=0).stop 2024-05-09T04:51:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.890+0000 7fa83d050b80 1 -- 172.21.15.46:0/1678139773 shutdown_connections 2024-05-09T04:51:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.890+0000 7fa83d050b80 1 --2- 172.21.15.46:0/1678139773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd3bc6a0 0x559fcd3c6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.890+0000 7fa83d050b80 1 --2- 172.21.15.46:0/1678139773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd23f4a0 0x559fcd3c62f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.890+0000 7fa83d050b80 1 --2- 172.21.15.46:0/1678139773 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559fcd3d6190 0x559fcd3d85b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.890+0000 7fa83d050b80 1 --2- 172.21.15.46:0/1678139773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd329160 0x559fcd3c6830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.890+0000 7fa83d050b80 1 -- 172.21.15.46:0/1678139773 >> 172.21.15.46:0/1678139773 conn(0x559fcd29f6a0 msgr2=0x559fcd3b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.890+0000 7fa8077fe700 0 client.15393 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.890+0000 7fa83d050b80 1 -- 172.21.15.46:0/1678139773 shutdown_connections 2024-05-09T04:51:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.890+0000 7fa83d050b80 1 -- 172.21.15.46:0/1678139773 wait complete. 2024-05-09T04:51:52.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.890+0000 7fa83d050b80 20 client.15393 trim_cache size 0 max 16384 2024-05-09T04:51:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa83d050b80 1 Processor -- start 2024-05-09T04:51:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa83d050b80 1 -- start start 2024-05-09T04:51:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa83d050b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd23f4a0 0x559fcd3c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa83d050b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd329160 0x559fcd3d2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa83d050b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd3bc6a0 0x559fcd3bbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa83d050b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281af580 con 0x559fcd3bc6a0 2024-05-09T04:51:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa83d050b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281ace10 con 0x559fcd329160 2024-05-09T04:51:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa83d050b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281aac60 con 0x559fcd23f4a0 2024-05-09T04:51:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa82cbd7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd3bc6a0 0x559fcd3bbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa82cbd7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd3bc6a0 0x559fcd3bbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47086/0 (socket says 172.21.15.46:47086) 2024-05-09T04:51:52.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa82cbd7700 1 -- 172.21.15.46:0/761829014 learned_addr learned my addr 172.21.15.46:0/761829014 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa81ffff700 1 --2- 172.21.15.46:0/761829014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd329160 0x559fcd3d2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa827fff700 1 --2- 172.21.15.46:0/761829014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd23f4a0 0x559fcd3c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa82cbd7700 1 -- 172.21.15.46:0/761829014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd23f4a0 msgr2=0x559fcd3c6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa82cbd7700 1 --2- 172.21.15.46:0/761829014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd23f4a0 0x559fcd3c6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa82cbd7700 1 -- 172.21.15.46:0/761829014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd329160 msgr2=0x559fcd3d2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa82cbd7700 1 --2- 172.21.15.46:0/761829014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd329160 0x559fcd3d2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa82cbd7700 1 -- 172.21.15.46:0/761829014 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559fcd3b8d00 con 0x559fcd3bc6a0 2024-05-09T04:51:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.911+0000 7fa81ffff700 1 --2- 172.21.15.46:0/761829014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd329160 0x559fcd3d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:51:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.912+0000 7fa82cbd7700 1 --2- 172.21.15.46:0/761829014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd3bc6a0 0x559fcd3bbda0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa820015000 tx=0x7fa820009520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a72bed4dcd2855a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.912+0000 7fa81dffb700 1 -- 172.21.15.46:0/761829014 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82000bca0 con 0x559fcd3bc6a0 2024-05-09T04:51:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.912+0000 7fa81dffb700 1 -- 172.21.15.46:0/761829014 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa82000be40 con 0x559fcd3bc6a0 2024-05-09T04:51:52.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.912+0000 7fa81dffb700 1 -- 172.21.15.46:0/761829014 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820005050 con 0x559fcd3bc6a0 2024-05-09T04:51:52.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.913+0000 7fa83d050b80 1 -- 172.21.15.46:0/761829014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd3bc6a0 msgr2=0x559fcd3bbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:52.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.913+0000 7fa83d050b80 1 --2- 172.21.15.46:0/761829014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd3bc6a0 0x559fcd3bbda0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fa820015000 tx=0x7fa820009520 comp rx=0 tx=0).stop 2024-05-09T04:51:52.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.913+0000 7fa83d050b80 1 -- 172.21.15.46:0/761829014 shutdown_connections 2024-05-09T04:51:52.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.913+0000 7fa83d050b80 1 --2- 172.21.15.46:0/761829014 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd3bc6a0 0x559fcd3bbda0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.913+0000 7fa83d050b80 1 --2- 172.21.15.46:0/761829014 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd23f4a0 0x559fcd3c6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.913+0000 7fa83d050b80 1 --2- 172.21.15.46:0/761829014 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd329160 0x559fcd3d2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.913+0000 7fa83d050b80 1 -- 172.21.15.46:0/761829014 >> 172.21.15.46:0/761829014 conn(0x559fcd29f6a0 msgr2=0x559fcd3bb690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.913+0000 7fa83d050b80 1 -- 172.21.15.46:0/761829014 shutdown_connections 2024-05-09T04:51:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.913+0000 7fa83d050b80 1 -- 172.21.15.46:0/761829014 wait complete. 2024-05-09T04:51:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.913+0000 7fa83d050b80 1 -- start start 2024-05-09T04:51:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.913+0000 7fa83d050b80 1 Processor -- start 2024-05-09T04:51:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa83d050b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd23f4a0 0x559fcd3d98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa83d050b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd329160 0x559fcd3d9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa83d050b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd3bc6a0 0x559fcd43f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa83d050b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281ab2d0 con 0x559fcd3bc6a0 2024-05-09T04:51:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa83d050b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281ac740 con 0x559fcd23f4a0 2024-05-09T04:51:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa83d050b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa8281ab9a0 con 0x559fcd329160 2024-05-09T04:51:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa83d050b80 10 client.0 _ll_register_callbacks cb 0x559fcd1506c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:51:52.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa82cbd7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd3bc6a0 0x559fcd43f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa82cbd7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd3bc6a0 0x559fcd43f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47092/0 (socket says 172.21.15.46:47092) 2024-05-09T04:51:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa82cbd7700 1 -- 172.21.15.46:0/1724360372 learned_addr learned my addr 172.21.15.46:0/1724360372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa827fff700 1 --2- 172.21.15.46:0/1724360372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd23f4a0 0x559fcd3d98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa81ffff700 1 --2- 172.21.15.46:0/1724360372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd329160 0x559fcd3d9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa82cbd7700 1 -- 172.21.15.46:0/1724360372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd329160 msgr2=0x559fcd3d9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa82cbd7700 1 --2- 172.21.15.46:0/1724360372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd329160 0x559fcd3d9e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa82cbd7700 1 -- 172.21.15.46:0/1724360372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd23f4a0 msgr2=0x559fcd3d98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa82cbd7700 1 --2- 172.21.15.46:0/1724360372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd23f4a0 0x559fcd3d98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa82cbd7700 1 -- 172.21.15.46:0/1724360372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559fcd3c1ac0 con 0x559fcd3bc6a0 2024-05-09T04:51:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa81ffff700 1 --2- 172.21.15.46:0/1724360372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd329160 0x559fcd3d9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:51:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa82cbd7700 1 --2- 172.21.15.46:0/1724360372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd3bc6a0 0x559fcd43f6e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa82000b420 tx=0x7fa820005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c67ec4e03f66e4d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.914+0000 7fa8077fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:51:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.915+0000 7fa8077fe700 1 -- 172.21.15.46:0/1724360372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa820007d90 con 0x559fcd3bc6a0 2024-05-09T04:51:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.915+0000 7fa8077fe700 1 -- 172.21.15.46:0/1724360372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa820027470 con 0x559fcd3bc6a0 2024-05-09T04:51:52.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.915+0000 7fa83d050b80 1 -- 172.21.15.46:0/1724360372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559fcd2e6a70 con 0x559fcd3bc6a0 2024-05-09T04:51:52.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.915+0000 7fa8077fe700 1 -- 172.21.15.46:0/1724360372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa820027670 con 0x559fcd3bc6a0 2024-05-09T04:51:52.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.915+0000 7fa83d050b80 10 client.15399 fetch_fsmap learned FSMap version 25 2024-05-09T04:51:52.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.915+0000 7fa83d050b80 1 -- 172.21.15.46:0/1724360372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559fcd440c10 con 0x559fcd3bc6a0 2024-05-09T04:51:52.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.915+0000 7fa8077fe700 1 -- 172.21.15.46:0/1724360372 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559fcd2e6a70 con 0x559fcd3bc6a0 2024-05-09T04:51:52.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.915+0000 7fa8077fe700 1 -- 172.21.15.46:0/1724360372 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa820011070 con 0x559fcd3bc6a0 2024-05-09T04:51:52.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.915+0000 7fa83d050b80 10 client.15399 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:51:52.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.915+0000 7fa83d050b80 10 client.15399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:51:52.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.915+0000 7fa83d050b80 20 client.15399 populate_metadata read hostname 'smithi046' 2024-05-09T04:51:52.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.915+0000 7fa83d050b80 1 --2- 172.21.15.46:0/1724360372 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559fcd4411b0 0x559fcd4435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:52.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.915+0000 7fa83d050b80 4 client.15399 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:51:52.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.915+0000 7fa83d050b80 1 -- 172.21.15.46:0/1724360372 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559fcd15a970 con 0x559fcd4411b0 2024-05-09T04:51:52.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.916+0000 7fa827fff700 1 --2- 172.21.15.46:0/1724360372 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559fcd4411b0 0x559fcd4435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:52.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.916+0000 7fa8077fe700 10 client.15399 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:52.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.916+0000 7fa827fff700 1 --2- 172.21.15.46:0/1724360372 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559fcd4411b0 0x559fcd4435d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d557f99030268099 server_cookie=237ff5783d9a0466 in_seq=0 out_seq=0 2024-05-09T04:51:52.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.916+0000 7fa8077fe700 1 -- 172.21.15.46:0/1724360372 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559fcd15a970 con 0x559fcd4411b0 2024-05-09T04:51:52.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.916+0000 7fa8077fe700 10 client.15399 handle_command_reply: tid=0 2024-05-09T04:51:52.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.916+0000 7fa83d050b80 1 client.15399 shutdown 2024-05-09T04:51:52.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.916+0000 7fa83d050b80 1 -- 172.21.15.46:0/1724360372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd3bc6a0 msgr2=0x559fcd43f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:52.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.916+0000 7fa83d050b80 1 --2- 172.21.15.46:0/1724360372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd3bc6a0 0x559fcd43f6e0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fa82000b420 tx=0x7fa820005a60 comp rx=0 tx=0).stop 2024-05-09T04:51:52.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.916+0000 7fa83d050b80 1 -- 172.21.15.46:0/1724360372 shutdown_connections 2024-05-09T04:51:52.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa83d050b80 1 --2- 172.21.15.46:0/1724360372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559fcd3bc6a0 0x559fcd43f6e0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa83d050b80 1 --2- 172.21.15.46:0/1724360372 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559fcd4411b0 0x559fcd4435d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa83d050b80 1 --2- 172.21.15.46:0/1724360372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559fcd23f4a0 0x559fcd3d98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa83d050b80 1 --2- 172.21.15.46:0/1724360372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559fcd329160 0x559fcd3d9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa8077fe700 0 client.15399 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:52.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa83d050b80 1 -- 172.21.15.46:0/1724360372 >> 172.21.15.46:0/1724360372 conn(0x559fcd29f6a0 msgr2=0x559fcd3bbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:52.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa83d050b80 1 -- 172.21.15.46:0/1724360372 shutdown_connections 2024-05-09T04:51:52.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa83d050b80 1 -- 172.21.15.46:0/1724360372 wait complete. 2024-05-09T04:51:52.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa83d050b80 20 client.15399 trim_cache size 0 max 16384 2024-05-09T04:51:52.923 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:51:52.923 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:51:52.923 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:51:52.923 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:51:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa82e63a700 1 -- 172.21.15.46:0/3287537010 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa8140b2a80 msgr2=0x7fa8140b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa82e63a700 1 --2- 172.21.15.46:0/3287537010 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa8140b2a80 0x7fa8140b4f90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa828107ce0 tx=0x7fa810009260 comp rx=0 tx=0).stop 2024-05-09T04:51:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa82e63a700 1 -- 172.21.15.46:0/3287537010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8281045e0 msgr2=0x7fa82819ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa82e63a700 1 --2- 172.21.15.46:0/3287537010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8281045e0 0x7fa82819ae00 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa81800c9b0 tx=0x7fa81800ccc0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa82e63a700 1 -- 172.21.15.46:0/3287537010 shutdown_connections 2024-05-09T04:51:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa82e63a700 1 --2- 172.21.15.46:0/3287537010 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa8140b2a80 0x7fa8140b4f90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa82e63a700 1 --2- 172.21.15.46:0/3287537010 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa828109860 0x7fa82819f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.917+0000 7fa82e63a700 1 --2- 172.21.15.46:0/3287537010 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa8281045e0 0x7fa82819ae00 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.918+0000 7fa82e63a700 1 --2- 172.21.15.46:0/3287537010 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa828108f10 0x7fa82819b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:52.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.918+0000 7fa82e63a700 1 -- 172.21.15.46:0/3287537010 >> 172.21.15.46:0/3287537010 conn(0x7fa8280ffed0 msgr2=0x7fa828108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:52.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.918+0000 7fa82e63a700 1 -- 172.21.15.46:0/3287537010 shutdown_connections 2024-05-09T04:51:52.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:52.918+0000 7fa82e63a700 1 -- 172.21.15.46:0/3287537010 wait complete. 2024-05-09T04:51:53.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:52 smithi110 ceph-mon[34904]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-09T04:51:53.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:52 smithi072 ceph-mon[34088]: pgmap v26: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 682 B/s wr, 8 op/s 2024-05-09T04:51:53.544 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:51:53.544 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:51:53.804 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:54.964 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:54 smithi046 ceph-mon[30136]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T04:51:55.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:54 smithi110 ceph-mon[34904]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T04:51:55.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:54 smithi072 ceph-mon[34088]: pgmap v27: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T04:51:55.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.534+0000 7febb6509700 1 -- 172.21.15.46:0/2087860954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb0108740 msgr2=0x7febb01100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:55.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.534+0000 7febb6509700 1 --2- 172.21.15.46:0/2087860954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb0108740 0x7febb01100a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7feba800b0d0 tx=0x7feba800b3e0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.534+0000 7febb6509700 1 -- 172.21.15.46:0/2087860954 shutdown_connections 2024-05-09T04:51:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.534+0000 7febb6509700 1 --2- 172.21.15.46:0/2087860954 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb0108740 0x7febb01100a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.534+0000 7febb6509700 1 --2- 172.21.15.46:0/2087860954 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb01034c0 0x7febb01038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.534+0000 7febb6509700 1 --2- 172.21.15.46:0/2087860954 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febb0107df0 0x7febb0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.534+0000 7febb6509700 1 -- 172.21.15.46:0/2087860954 >> 172.21.15.46:0/2087860954 conn(0x7febb00fedf0 msgr2=0x7febb0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.534+0000 7febb6509700 1 -- 172.21.15.46:0/2087860954 shutdown_connections 2024-05-09T04:51:55.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.534+0000 7febb6509700 1 -- 172.21.15.46:0/2087860954 wait complete. 2024-05-09T04:51:55.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.535+0000 7febb6509700 1 Processor -- start 2024-05-09T04:51:55.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.535+0000 7febb6509700 1 -- start start 2024-05-09T04:51:55.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.535+0000 7febb6509700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb01034c0 0x7febb019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:55.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.535+0000 7febb6509700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febb0107df0 0x7febb019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:55.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.535+0000 7febb6509700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb0108740 0x7febb019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:55.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.536+0000 7febb6509700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7febb0113370 con 0x7febb0108740 2024-05-09T04:51:55.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.536+0000 7febb6509700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7febb01131f0 con 0x7febb01034c0 2024-05-09T04:51:55.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.536+0000 7febb6509700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7febb01134f0 con 0x7febb0107df0 2024-05-09T04:51:55.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.536+0000 7febb4aa6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb0108740 0x7febb019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:55.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.536+0000 7febb4aa6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb0108740 0x7febb019f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58386/0 (socket says 172.21.15.46:58386) 2024-05-09T04:51:55.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.536+0000 7febaffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb01034c0 0x7febb019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:55.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.536+0000 7febb4aa6700 1 -- 172.21.15.46:0/3342262369 learned_addr learned my addr 172.21.15.46:0/3342262369 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:55.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.536+0000 7feba7fff700 1 --2- 172.21.15.46:0/3342262369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febb0107df0 0x7febb019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:55.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.536+0000 7febb4aa6700 1 -- 172.21.15.46:0/3342262369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febb0107df0 msgr2=0x7febb019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:55.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.536+0000 7febb4aa6700 1 --2- 172.21.15.46:0/3342262369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febb0107df0 0x7febb019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.536+0000 7febb4aa6700 1 -- 172.21.15.46:0/3342262369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb01034c0 msgr2=0x7febb019adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:55.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.536+0000 7febb4aa6700 1 --2- 172.21.15.46:0/3342262369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb01034c0 0x7febb019adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.536+0000 7febb4aa6700 1 -- 172.21.15.46:0/3342262369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febb019fc80 con 0x7febb0108740 2024-05-09T04:51:55.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.536+0000 7febb4aa6700 1 --2- 172.21.15.46:0/3342262369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb0108740 0x7febb019f5e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7feba8009850 tx=0x7feba8007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:55.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.536+0000 7febadffb700 1 -- 172.21.15.46:0/3342262369 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba801f070 con 0x7febb0108740 2024-05-09T04:51:55.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.537+0000 7febadffb700 1 -- 172.21.15.46:0/3342262369 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feba8004c50 con 0x7febb0108740 2024-05-09T04:51:55.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.537+0000 7febb6509700 1 -- 172.21.15.46:0/3342262369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febb019ff10 con 0x7febb0108740 2024-05-09T04:51:55.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.537+0000 7febadffb700 1 -- 172.21.15.46:0/3342262369 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feba801b690 con 0x7febb0108740 2024-05-09T04:51:55.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.537+0000 7febb6509700 1 -- 172.21.15.46:0/3342262369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febb01a03f0 con 0x7febb0108740 2024-05-09T04:51:55.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.538+0000 7febb6509700 1 -- 172.21.15.46:0/3342262369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7febb00512c0 con 0x7febb0108740 2024-05-09T04:51:55.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.539+0000 7febadffb700 1 -- 172.21.15.46:0/3342262369 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7feba8022070 con 0x7febb0108740 2024-05-09T04:51:55.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.540+0000 7febadffb700 1 --2- 172.21.15.46:0/3342262369 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7feb9c0b2a80 0x7feb9c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:55.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.540+0000 7febaffff700 1 --2- 172.21.15.46:0/3342262369 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7feb9c0b2a80 0x7feb9c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:55.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.540+0000 7febadffb700 1 -- 172.21.15.46:0/3342262369 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7feba800f020 con 0x7febb0108740 2024-05-09T04:51:55.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.541+0000 7febaffff700 1 --2- 172.21.15.46:0/3342262369 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7feb9c0b2a80 0x7feb9c0b4f90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feba0007be0 tx=0x7feba00073d0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:55.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.545+0000 7febadffb700 1 -- 172.21.15.46:0/3342262369 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feba8029020 con 0x7febb0108740 2024-05-09T04:51:55.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.810+0000 7febb6509700 1 -- 172.21.15.46:0/3342262369 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7febb01a7890 con 0x7febb0108740 2024-05-09T04:51:55.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.811+0000 7febadffb700 1 -- 172.21.15.46:0/3342262369 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7feba80a6030 con 0x7febb0108740 2024-05-09T04:51:55.812 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:51:55.812 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:51:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.813+0000 7febb6509700 1 -- 172.21.15.46:0/3342262369 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7feb9c0b2a80 msgr2=0x7feb9c0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.813+0000 7febb6509700 1 --2- 172.21.15.46:0/3342262369 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7feb9c0b2a80 0x7feb9c0b4f90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feba0007be0 tx=0x7feba00073d0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.813+0000 7febb6509700 1 -- 172.21.15.46:0/3342262369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb0108740 msgr2=0x7febb019f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.813+0000 7febb6509700 1 --2- 172.21.15.46:0/3342262369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb0108740 0x7febb019f5e0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7feba8009850 tx=0x7feba8007a20 comp rx=0 tx=0).stop 2024-05-09T04:51:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.813+0000 7febb6509700 1 -- 172.21.15.46:0/3342262369 shutdown_connections 2024-05-09T04:51:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.813+0000 7febb6509700 1 --2- 172.21.15.46:0/3342262369 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7feb9c0b2a80 0x7feb9c0b4f90 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.813+0000 7febb6509700 1 --2- 172.21.15.46:0/3342262369 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7febb0108740 0x7febb019f5e0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.814 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.813+0000 7febb6509700 1 --2- 172.21.15.46:0/3342262369 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7febb01034c0 0x7febb019adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.813+0000 7febb6509700 1 --2- 172.21.15.46:0/3342262369 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7febb0107df0 0x7febb019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:55.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.813+0000 7febb6509700 1 -- 172.21.15.46:0/3342262369 >> 172.21.15.46:0/3342262369 conn(0x7febb00fedf0 msgr2=0x7febb0107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:55.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.813+0000 7febb6509700 1 -- 172.21.15.46:0/3342262369 shutdown_connections 2024-05-09T04:51:55.815 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:55.813+0000 7febb6509700 1 -- 172.21.15.46:0/3342262369 wait complete. 2024-05-09T04:51:55.815 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:51:56.425 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:51:56.684 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:51:56.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:56 smithi046 ceph-mon[30136]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T04:51:56.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:56 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3342262369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:51:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:56 smithi110 ceph-mon[34904]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T04:51:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:56 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3342262369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:51:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:56 smithi072 ceph-mon[34088]: pgmap v28: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 682 B/s wr, 6 op/s 2024-05-09T04:51:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:56 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3342262369' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:51:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.705+0000 7fed52d5e640 1 -- 172.21.15.46:0/1644779668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed4c108bc0 msgr2=0x7fed4c102760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.705+0000 7fed52d5e640 1 --2- 172.21.15.46:0/1644779668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed4c108bc0 0x7fed4c102760 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fed44009900 tx=0x7fed4402a410 comp rx=0 tx=0).stop 2024-05-09T04:51:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.705+0000 7fed52d5e640 1 -- 172.21.15.46:0/1644779668 shutdown_connections 2024-05-09T04:51:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.705+0000 7fed52d5e640 1 --2- 172.21.15.46:0/1644779668 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed4c103310 0x7fed4c10f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.705+0000 7fed52d5e640 1 --2- 172.21.15.46:0/1644779668 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed4c109060 0x7fed4c102ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.705+0000 7fed52d5e640 1 --2- 172.21.15.46:0/1644779668 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed4c108bc0 0x7fed4c102760 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.705+0000 7fed52d5e640 1 -- 172.21.15.46:0/1644779668 >> 172.21.15.46:0/1644779668 conn(0x7fed4c0fe390 msgr2=0x7fed4c1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.705+0000 7fed52d5e640 1 -- 172.21.15.46:0/1644779668 shutdown_connections 2024-05-09T04:51:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.705+0000 7fed52d5e640 1 -- 172.21.15.46:0/1644779668 wait complete. 2024-05-09T04:51:57.707 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.706+0000 7fed52d5e640 1 Processor -- start 2024-05-09T04:51:57.708 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.706+0000 7fed52d5e640 1 -- start start 2024-05-09T04:51:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.706+0000 7fed52d5e640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed4c103310 0x7fed4c1a05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.709 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.706+0000 7fed52d5e640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed4c108bc0 0x7fed4c1a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.706+0000 7fed52d5e640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed4c109060 0x7fed4c19a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.706+0000 7fed52d5e640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c112db0 con 0x7fed4c103310 2024-05-09T04:51:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.706+0000 7fed52d5e640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c112c30 con 0x7fed4c108bc0 2024-05-09T04:51:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.706+0000 7fed52d5e640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c112f30 con 0x7fed4c109060 2024-05-09T04:51:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.707+0000 7fed50afa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed4c103310 0x7fed4c1a05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.707+0000 7fed50afa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed4c103310 0x7fed4c1a05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58404/0 (socket says 172.21.15.46:58404) 2024-05-09T04:51:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.707+0000 7fed50afa640 1 -- 172.21.15.46:0/1004526585 learned_addr learned my addr 172.21.15.46:0/1004526585 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.707+0000 7fed4bfff640 1 --2- 172.21.15.46:0/1004526585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed4c108bc0 0x7fed4c1a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.707+0000 7fed512fb640 1 --2- 172.21.15.46:0/1004526585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed4c109060 0x7fed4c19a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.707+0000 7fed50afa640 1 -- 172.21.15.46:0/1004526585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed4c109060 msgr2=0x7fed4c19a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.707+0000 7fed50afa640 1 --2- 172.21.15.46:0/1004526585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed4c109060 0x7fed4c19a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.707+0000 7fed50afa640 1 -- 172.21.15.46:0/1004526585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed4c108bc0 msgr2=0x7fed4c1a0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.707+0000 7fed50afa640 1 --2- 172.21.15.46:0/1004526585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed4c108bc0 0x7fed4c1a0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.710 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.707+0000 7fed50afa640 1 -- 172.21.15.46:0/1004526585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed4c19ae50 con 0x7fed4c103310 2024-05-09T04:51:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.707+0000 7fed50afa640 1 --2- 172.21.15.46:0/1004526585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed4c103310 0x7fed4c1a05a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fed44006010 tx=0x7fed44004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.707+0000 7fed49ffb640 1 -- 172.21.15.46:0/1004526585 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed4403f030 con 0x7fed4c103310 2024-05-09T04:51:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.707+0000 7fed49ffb640 1 -- 172.21.15.46:0/1004526585 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed44003aa0 con 0x7fed4c103310 2024-05-09T04:51:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.707+0000 7fed52d5e640 1 -- 172.21.15.46:0/1004526585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed4c19b080 con 0x7fed4c103310 2024-05-09T04:51:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.707+0000 7fed49ffb640 1 -- 172.21.15.46:0/1004526585 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44035070 con 0x7fed4c103310 2024-05-09T04:51:57.711 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.708+0000 7fed52d5e640 1 -- 172.21.15.46:0/1004526585 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed4c19b570 con 0x7fed4c103310 2024-05-09T04:51:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.711+0000 7fed59c15740 1 Processor -- start 2024-05-09T04:51:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.711+0000 7fed59c15740 1 -- start start 2024-05-09T04:51:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.711+0000 7fed49ffb640 1 -- 172.21.15.46:0/1004526585 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7fed44042030 con 0x7fed4c103310 2024-05-09T04:51:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.711+0000 7fed59c15740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe461b0 0x55dedc1e9680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.711+0000 7fed59c15740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe1d2f0 0x55dedc1e9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.711+0000 7fed59c15740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe254d0 0x55dedc1ea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.711+0000 7fed59c15740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dedc14e7c0 con 0x55dedbe1d2f0 2024-05-09T04:51:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.711+0000 7fed59c15740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55dedc0ea0e0 con 0x55dedbe461b0 2024-05-09T04:51:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.711+0000 7fed59c15740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dedc14a780 con 0x55dedbe254d0 2024-05-09T04:51:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed50afa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe1d2f0 0x55dedc1e9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.714 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed50afa640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe1d2f0 0x55dedc1e9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58418/0 (socket says 172.21.15.46:58418) 2024-05-09T04:51:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed50afa640 1 -- 172.21.15.46:0/3193677041 learned_addr learned my addr 172.21.15.46:0/3193677041 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed512fb640 1 --2- 172.21.15.46:0/3193677041 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe461b0 0x55dedc1e9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed4bfff640 1 --2- 172.21.15.46:0/3193677041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe254d0 0x55dedc1ea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed50afa640 1 -- 172.21.15.46:0/3193677041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe254d0 msgr2=0x55dedc1ea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed50afa640 1 --2- 172.21.15.46:0/3193677041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe254d0 0x55dedc1ea100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed50afa640 1 -- 172.21.15.46:0/3193677041 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe461b0 msgr2=0x55dedc1e9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed50afa640 1 --2- 172.21.15.46:0/3193677041 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe461b0 0x55dedc1e9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed50afa640 1 -- 172.21.15.46:0/3193677041 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dedc1137c0 con 0x55dedbe1d2f0 2024-05-09T04:51:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed4bfff640 1 --2- 172.21.15.46:0/3193677041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe254d0 0x55dedc1ea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:51:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed49ffb640 1 --2- 172.21.15.46:0/1004526585 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fed380b29e0 0x7fed380b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed50afa640 1 --2- 172.21.15.46:0/3193677041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe1d2f0 0x55dedc1e9bc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fed44100320 tx=0x7fed44106ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3a36e8b2f0ef36 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed512fb640 1 --2- 172.21.15.46:0/1004526585 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fed380b29e0 0x7fed380b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed49ffb640 1 -- 172.21.15.46:0/1004526585 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fed44102da0 con 0x7fed4c103310 2024-05-09T04:51:57.715 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.712+0000 7fed42ffd640 1 -- 172.21.15.46:0/3193677041 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed440c5d50 con 0x55dedbe1d2f0 2024-05-09T04:51:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.713+0000 7fed42ffd640 1 -- 172.21.15.46:0/3193677041 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed4403b460 con 0x55dedbe1d2f0 2024-05-09T04:51:57.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.713+0000 7fed42ffd640 1 -- 172.21.15.46:0/3193677041 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed440c9600 con 0x55dedbe1d2f0 2024-05-09T04:51:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.713+0000 7fed59c15740 1 -- 172.21.15.46:0/3193677041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe1d2f0 msgr2=0x55dedc1e9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.713+0000 7fed512fb640 1 --2- 172.21.15.46:0/1004526585 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fed380b29e0 0x7fed380b4ef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55dedc1e67f0 tx=0x7fed3400b450 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.713+0000 7fed59c15740 1 --2- 172.21.15.46:0/3193677041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe1d2f0 0x55dedc1e9bc0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fed44100320 tx=0x7fed44106ea0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.714+0000 7fed59c15740 1 -- 172.21.15.46:0/3193677041 shutdown_connections 2024-05-09T04:51:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.714+0000 7fed59c15740 1 --2- 172.21.15.46:0/3193677041 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe254d0 0x55dedc1ea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.714+0000 7fed59c15740 1 --2- 172.21.15.46:0/3193677041 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe1d2f0 0x55dedc1e9bc0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.714+0000 7fed59c15740 1 --2- 172.21.15.46:0/3193677041 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe461b0 0x55dedc1e9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.714+0000 7fed59c15740 1 -- 172.21.15.46:0/3193677041 >> 172.21.15.46:0/3193677041 conn(0x55dedbe3a300 msgr2=0x55dedc1e5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.714+0000 7fed59c15740 1 -- 172.21.15.46:0/3193677041 shutdown_connections 2024-05-09T04:51:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.714+0000 7fed59c15740 1 -- 172.21.15.46:0/3193677041 wait complete. 2024-05-09T04:51:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.714+0000 7fed59c15740 1 -- start start 2024-05-09T04:51:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.714+0000 7fed59c15740 1 Processor -- start 2024-05-09T04:51:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed59c15740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe1d2f0 0x55dedc1f3d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed59c15740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe254d0 0x55dedc1f42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed59c15740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc1f99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed59c15740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dedc1f6570 con 0x55dedbe461b0 2024-05-09T04:51:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed59c15740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55dedc0d59c0 con 0x55dedbe1d2f0 2024-05-09T04:51:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed59c15740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dedc1f66f0 con 0x55dedbe254d0 2024-05-09T04:51:57.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed59c15740 10 client.0 _ll_register_callbacks cb 0x55dedc102020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:51:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed512fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc1f99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed512fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc1f99a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58424/0 (socket says 172.21.15.46:58424) 2024-05-09T04:51:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed512fb640 1 -- 172.21.15.46:0/4285587358 learned_addr learned my addr 172.21.15.46:0/4285587358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed4bfff640 1 --2- 172.21.15.46:0/4285587358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe254d0 0x55dedc1f42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed50afa640 1 --2- 172.21.15.46:0/4285587358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe1d2f0 0x55dedc1f3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed512fb640 1 -- 172.21.15.46:0/4285587358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe254d0 msgr2=0x55dedc1f42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed512fb640 1 --2- 172.21.15.46:0/4285587358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe254d0 0x55dedc1f42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed512fb640 1 -- 172.21.15.46:0/4285587358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe1d2f0 msgr2=0x55dedc1f3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed512fb640 1 --2- 172.21.15.46:0/4285587358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe1d2f0 0x55dedc1f3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed512fb640 1 -- 172.21.15.46:0/4285587358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dedc0fc1f0 con 0x55dedbe461b0 2024-05-09T04:51:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed512fb640 1 --2- 172.21.15.46:0/4285587358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc1f99a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fed3401d670 tx=0x7fed340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5eb727d11a7d6ed server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:51:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.715+0000 7fed2bfff640 1 -- 172.21.15.46:0/4285587358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed34029680 con 0x55dedbe461b0 2024-05-09T04:51:57.722 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.716+0000 7fed2bfff640 1 -- 172.21.15.46:0/4285587358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fed34029820 con 0x55dedbe461b0 2024-05-09T04:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.716+0000 7fed59c15740 1 -- 172.21.15.46:0/4285587358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dedc15c3d0 con 0x55dedbe461b0 2024-05-09T04:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.716+0000 7fed2bfff640 1 -- 172.21.15.46:0/4285587358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fed34029a60 con 0x55dedbe461b0 2024-05-09T04:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.716+0000 7fed59c15740 10 client.15417 fetch_fsmap learned FSMap version 25 2024-05-09T04:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.716+0000 7fed59c15740 1 -- 172.21.15.46:0/4285587358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dedbeb1790 con 0x55dedbe461b0 2024-05-09T04:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.716+0000 7fed2bfff640 1 -- 172.21.15.46:0/4285587358 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dedc15c3d0 con 0x55dedbe461b0 2024-05-09T04:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.716+0000 7fed2bfff640 1 -- 172.21.15.46:0/4285587358 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fed3401e580 con 0x55dedbe461b0 2024-05-09T04:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.716+0000 7fed59c15740 10 client.15417 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.716+0000 7fed59c15740 10 client.15417 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.716+0000 7fed59c15740 20 client.15417 populate_metadata read hostname 'smithi046' 2024-05-09T04:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.716+0000 7fed59c15740 1 --2- 172.21.15.46:0/4285587358 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dedbe04d80 0x55dedc205c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.716+0000 7fed59c15740 4 client.15417 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.716+0000 7fed59c15740 1 -- 172.21.15.46:0/4285587358 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dedc156240 con 0x55dedbe04d80 2024-05-09T04:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.717+0000 7fed50afa640 1 --2- 172.21.15.46:0/4285587358 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dedbe04d80 0x55dedc205c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.723 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.717+0000 7fed50afa640 1 --2- 172.21.15.46:0/4285587358 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dedbe04d80 0x55dedc205c80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d53e73069900bd9 server_cookie=c4b64886a3d124a5 in_seq=0 out_seq=0 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.717+0000 7fed2bfff640 10 client.15417 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.718+0000 7fed2bfff640 1 -- 172.21.15.46:0/4285587358 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dedc156240 con 0x55dedbe04d80 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.718+0000 7fed2bfff640 10 client.15417 handle_command_reply: tid=0 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.718+0000 7fed59c15740 1 client.15417 shutdown 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.718+0000 7fed59c15740 1 -- 172.21.15.46:0/4285587358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 msgr2=0x55dedc1f99a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.718+0000 7fed59c15740 1 --2- 172.21.15.46:0/4285587358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc1f99a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fed3401d670 tx=0x7fed340046c0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.718+0000 7fed59c15740 1 -- 172.21.15.46:0/4285587358 shutdown_connections 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.718+0000 7fed59c15740 1 --2- 172.21.15.46:0/4285587358 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dedbe04d80 0x55dedc205c80 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.718+0000 7fed59c15740 1 --2- 172.21.15.46:0/4285587358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe254d0 0x55dedc1f42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.718+0000 7fed59c15740 1 --2- 172.21.15.46:0/4285587358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc1f99a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.718+0000 7fed2bfff640 0 client.15417 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.718+0000 7fed59c15740 1 --2- 172.21.15.46:0/4285587358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe1d2f0 0x55dedc1f3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.718+0000 7fed59c15740 1 -- 172.21.15.46:0/4285587358 >> 172.21.15.46:0/4285587358 conn(0x55dedbe3a300 msgr2=0x55dedc1e4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.718+0000 7fed59c15740 1 -- 172.21.15.46:0/4285587358 shutdown_connections 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.718+0000 7fed59c15740 1 -- 172.21.15.46:0/4285587358 wait complete. 2024-05-09T04:51:57.724 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.718+0000 7fed59c15740 20 client.15417 trim_cache size 0 max 16384 2024-05-09T04:51:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.734+0000 7fed59c15740 1 Processor -- start 2024-05-09T04:51:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.734+0000 7fed59c15740 1 -- start start 2024-05-09T04:51:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.734+0000 7fed59c15740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe1d2f0 0x55dedc1ed9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.734+0000 7fed59c15740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe254d0 0x55dedc1f9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.734+0000 7fed59c15740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc1f9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.734+0000 7fed59c15740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c1b25d0 con 0x55dedbe461b0 2024-05-09T04:51:57.736 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.734+0000 7fed59c15740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c1afea0 con 0x55dedbe254d0 2024-05-09T04:51:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.734+0000 7fed59c15740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c1adcf0 con 0x55dedbe1d2f0 2024-05-09T04:51:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.734+0000 7fed512fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc1f9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.734+0000 7fed512fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc1f9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58426/0 (socket says 172.21.15.46:58426) 2024-05-09T04:51:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.734+0000 7fed512fb640 1 -- 172.21.15.46:0/2367539258 learned_addr learned my addr 172.21.15.46:0/2367539258 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.734+0000 7fed50afa640 1 --2- 172.21.15.46:0/2367539258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe1d2f0 0x55dedc1ed9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.734+0000 7fed4bfff640 1 --2- 172.21.15.46:0/2367539258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe254d0 0x55dedc1f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.735+0000 7fed512fb640 1 -- 172.21.15.46:0/2367539258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe1d2f0 msgr2=0x55dedc1ed9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.735+0000 7fed512fb640 1 --2- 172.21.15.46:0/2367539258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe1d2f0 0x55dedc1ed9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.735+0000 7fed512fb640 1 -- 172.21.15.46:0/2367539258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe254d0 msgr2=0x55dedc1f9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.735+0000 7fed512fb640 1 --2- 172.21.15.46:0/2367539258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe254d0 0x55dedc1f9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.735+0000 7fed512fb640 1 -- 172.21.15.46:0/2367539258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dedc1f40f0 con 0x55dedbe461b0 2024-05-09T04:51:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.735+0000 7fed512fb640 1 --2- 172.21.15.46:0/2367539258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc1f9e60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fed34017040 tx=0x7fed34005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8586db579b3b824d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:57.737 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.735+0000 7fed427fc640 1 -- 172.21.15.46:0/2367539258 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3401edc0 con 0x55dedbe461b0 2024-05-09T04:51:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.735+0000 7fed427fc640 1 -- 172.21.15.46:0/2367539258 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed34029480 con 0x55dedbe461b0 2024-05-09T04:51:57.738 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.735+0000 7fed427fc640 1 -- 172.21.15.46:0/2367539258 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed340055f0 con 0x55dedbe461b0 2024-05-09T04:51:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.736+0000 7fed59c15740 1 -- 172.21.15.46:0/2367539258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 msgr2=0x55dedc1f9e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.736+0000 7fed59c15740 1 --2- 172.21.15.46:0/2367539258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc1f9e60 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fed34017040 tx=0x7fed34005c10 comp rx=0 tx=0).stop 2024-05-09T04:51:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.736+0000 7fed59c15740 1 -- 172.21.15.46:0/2367539258 shutdown_connections 2024-05-09T04:51:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.736+0000 7fed59c15740 1 --2- 172.21.15.46:0/2367539258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc1f9e60 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.736+0000 7fed59c15740 1 --2- 172.21.15.46:0/2367539258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe254d0 0x55dedc1f9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.736+0000 7fed59c15740 1 --2- 172.21.15.46:0/2367539258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe1d2f0 0x55dedc1ed9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.736+0000 7fed59c15740 1 -- 172.21.15.46:0/2367539258 >> 172.21.15.46:0/2367539258 conn(0x55dedbe3a300 msgr2=0x55dedc1f2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.736+0000 7fed59c15740 1 -- 172.21.15.46:0/2367539258 shutdown_connections 2024-05-09T04:51:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.736+0000 7fed59c15740 1 -- 172.21.15.46:0/2367539258 wait complete. 2024-05-09T04:51:57.741 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.736+0000 7fed59c15740 1 -- start start 2024-05-09T04:51:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.736+0000 7fed59c15740 1 Processor -- start 2024-05-09T04:51:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed59c15740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe1d2f0 0x55dedc1faa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed59c15740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe254d0 0x55dedc1fafb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed59c15740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc26bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed59c15740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c1ae360 con 0x55dedbe461b0 2024-05-09T04:51:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed59c15740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c1af7d0 con 0x55dedbe254d0 2024-05-09T04:51:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed59c15740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fed4c1aea30 con 0x55dedbe1d2f0 2024-05-09T04:51:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed59c15740 10 client.0 _ll_register_callbacks cb 0x7fed3400b670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:51:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed512fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc26bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed512fb640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc26bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58442/0 (socket says 172.21.15.46:58442) 2024-05-09T04:51:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed512fb640 1 -- 172.21.15.46:0/3352743060 learned_addr learned my addr 172.21.15.46:0/3352743060 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed50afa640 1 --2- 172.21.15.46:0/3352743060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe1d2f0 0x55dedc1faa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.742 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed4bfff640 1 --2- 172.21.15.46:0/3352743060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe254d0 0x55dedc1fafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed512fb640 1 -- 172.21.15.46:0/3352743060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe1d2f0 msgr2=0x55dedc1faa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed512fb640 1 --2- 172.21.15.46:0/3352743060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe1d2f0 0x55dedc1faa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed512fb640 1 -- 172.21.15.46:0/3352743060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe254d0 msgr2=0x55dedc1fafb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed512fb640 1 --2- 172.21.15.46:0/3352743060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe254d0 0x55dedc1fafb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed512fb640 1 -- 172.21.15.46:0/3352743060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dedbf285f0 con 0x55dedbe461b0 2024-05-09T04:51:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed512fb640 1 --2- 172.21.15.46:0/3352743060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc26bfe0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fed3403f040 tx=0x7fed34029630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d85839bdaf34c37 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.737+0000 7fed2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:51:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.738+0000 7fed2bfff640 1 -- 172.21.15.46:0/3352743060 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed3404b060 con 0x55dedbe461b0 2024-05-09T04:51:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.738+0000 7fed2bfff640 1 -- 172.21.15.46:0/3352743060 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fed34005370 con 0x55dedbe461b0 2024-05-09T04:51:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.738+0000 7fed59c15740 1 -- 172.21.15.46:0/3352743060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed34029480 con 0x55dedbe461b0 2024-05-09T04:51:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.738+0000 7fed2bfff640 1 -- 172.21.15.46:0/3352743060 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fed34005570 con 0x55dedbe461b0 2024-05-09T04:51:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.738+0000 7fed59c15740 10 client.15423 fetch_fsmap learned FSMap version 25 2024-05-09T04:51:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.738+0000 7fed59c15740 1 -- 172.21.15.46:0/3352743060 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dedc112ed0 con 0x55dedbe461b0 2024-05-09T04:51:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.738+0000 7fed2bfff640 1 -- 172.21.15.46:0/3352743060 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed34029480 con 0x55dedbe461b0 2024-05-09T04:51:57.743 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.738+0000 7fed2bfff640 1 -- 172.21.15.46:0/3352743060 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fed34032450 con 0x55dedbe461b0 2024-05-09T04:51:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.738+0000 7fed59c15740 10 client.15423 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:51:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.738+0000 7fed59c15740 10 client.15423 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:51:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.738+0000 7fed59c15740 20 client.15423 populate_metadata read hostname 'smithi046' 2024-05-09T04:51:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.738+0000 7fed59c15740 1 --2- 172.21.15.46:0/3352743060 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dedbdda1c0 0x55dedc26fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.738+0000 7fed59c15740 4 client.15423 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T04:51:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.738+0000 7fed59c15740 1 -- 172.21.15.46:0/3352743060 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dedc0d1350 con 0x55dedbdda1c0 2024-05-09T04:51:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.739+0000 7fed50afa640 1 --2- 172.21.15.46:0/3352743060 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dedbdda1c0 0x55dedc26fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.739+0000 7fed50afa640 1 --2- 172.21.15.46:0/3352743060 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dedbdda1c0 0x55dedc26fa00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67b8da2893137bd8 server_cookie=f79cb4dfc3deb8e0 in_seq=0 out_seq=0 2024-05-09T04:51:57.744 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.739+0000 7fed2bfff640 10 client.15423 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.747+0000 7fed2bfff640 1 -- 172.21.15.46:0/3352743060 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dedc0d1350 con 0x55dedbdda1c0 2024-05-09T04:51:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.747+0000 7fed2bfff640 10 client.15423 handle_command_reply: tid=0 2024-05-09T04:51:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.747+0000 7fed59c15740 1 client.15423 shutdown 2024-05-09T04:51:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.747+0000 7fed59c15740 1 -- 172.21.15.46:0/3352743060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 msgr2=0x55dedc26bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.747+0000 7fed59c15740 1 --2- 172.21.15.46:0/3352743060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc26bfe0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fed3403f040 tx=0x7fed34029630 comp rx=0 tx=0).stop 2024-05-09T04:51:57.748 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.747+0000 7fed59c15740 1 -- 172.21.15.46:0/3352743060 shutdown_connections 2024-05-09T04:51:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.747+0000 7fed59c15740 1 --2- 172.21.15.46:0/3352743060 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dedbdda1c0 0x55dedc26fa00 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.747+0000 7fed59c15740 1 --2- 172.21.15.46:0/3352743060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dedbe461b0 0x55dedc26bfe0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.747+0000 7fed59c15740 1 --2- 172.21.15.46:0/3352743060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dedbe254d0 0x55dedc1fafb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.747+0000 7fed2bfff640 0 client.15423 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:51:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.747+0000 7fed59c15740 1 --2- 172.21.15.46:0/3352743060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dedbe1d2f0 0x55dedc1faa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.747+0000 7fed59c15740 1 -- 172.21.15.46:0/3352743060 >> 172.21.15.46:0/3352743060 conn(0x55dedbe3a300 msgr2=0x55dedc1e4700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.747+0000 7fed59c15740 1 -- 172.21.15.46:0/3352743060 shutdown_connections 2024-05-09T04:51:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.747+0000 7fed59c15740 1 -- 172.21.15.46:0/3352743060 wait complete. 2024-05-09T04:51:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.747+0000 7fed59c15740 20 client.15423 trim_cache size 0 max 16384 2024-05-09T04:51:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-05-09T04:51:57.749 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-05-09T04:51:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-05-09T04:51:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.748+0000 7fed52d5e640 1 -- 172.21.15.46:0/1004526585 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fed380b29e0 msgr2=0x7fed380b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:57.750 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.748+0000 7fed52d5e640 1 --2- 172.21.15.46:0/1004526585 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fed380b29e0 0x7fed380b4ef0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x55dedc1e67f0 tx=0x7fed3400b450 comp rx=0 tx=0).stop 2024-05-09T04:51:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.748+0000 7fed52d5e640 1 -- 172.21.15.46:0/1004526585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed4c103310 msgr2=0x7fed4c1a05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.748+0000 7fed52d5e640 1 --2- 172.21.15.46:0/1004526585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed4c103310 0x7fed4c1a05a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fed44006010 tx=0x7fed44004060 comp rx=0 tx=0).stop 2024-05-09T04:51:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.748+0000 7fed52d5e640 1 -- 172.21.15.46:0/1004526585 shutdown_connections 2024-05-09T04:51:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.748+0000 7fed52d5e640 1 --2- 172.21.15.46:0/1004526585 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fed380b29e0 0x7fed380b4ef0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.748+0000 7fed52d5e640 1 --2- 172.21.15.46:0/1004526585 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fed4c109060 0x7fed4c19a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.748+0000 7fed52d5e640 1 --2- 172.21.15.46:0/1004526585 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fed4c108bc0 0x7fed4c1a0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.748+0000 7fed52d5e640 1 --2- 172.21.15.46:0/1004526585 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fed4c103310 0x7fed4c1a05a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.748+0000 7fed52d5e640 1 -- 172.21.15.46:0/1004526585 >> 172.21.15.46:0/1004526585 conn(0x7fed4c0fe390 msgr2=0x7fed4c100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.748+0000 7fed52d5e640 1 -- 172.21.15.46:0/1004526585 shutdown_connections 2024-05-09T04:51:57.751 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:51:57.748+0000 7fed52d5e640 1 -- 172.21.15.46:0/1004526585 wait complete. 2024-05-09T04:51:58.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.467+0000 7f91e3e09700 1 -- 172.21.15.46:0/795348638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91dc107df0 msgr2=0x7f91dc108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:58.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.467+0000 7f91e3e09700 1 --2- 172.21.15.46:0/795348638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91dc107df0 0x7f91dc108200 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f91d00099c0 tx=0x7f91d0009cd0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.467+0000 7f91e3e09700 1 -- 172.21.15.46:0/795348638 shutdown_connections 2024-05-09T04:51:58.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.467+0000 7f91e3e09700 1 --2- 172.21.15.46:0/795348638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91dc108740 0x7f91dc1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.467+0000 7f91e3e09700 1 --2- 172.21.15.46:0/795348638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91dc1034c0 0x7f91dc1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.467+0000 7f91e3e09700 1 --2- 172.21.15.46:0/795348638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91dc107df0 0x7f91dc108200 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.467+0000 7f91e3e09700 1 -- 172.21.15.46:0/795348638 >> 172.21.15.46:0/795348638 conn(0x7f91dc0fedf0 msgr2=0x7f91dc101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:58.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.467+0000 7f91e3e09700 1 -- 172.21.15.46:0/795348638 shutdown_connections 2024-05-09T04:51:58.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.467+0000 7f91e3e09700 1 -- 172.21.15.46:0/795348638 wait complete. 2024-05-09T04:51:58.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.468+0000 7f91e3e09700 1 Processor -- start 2024-05-09T04:51:58.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.468+0000 7f91e3e09700 1 -- start start 2024-05-09T04:51:58.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.468+0000 7f91e3e09700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91dc1034c0 0x7f91dc0ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.468+0000 7f91e3e09700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91dc107df0 0x7f91dc0ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.468+0000 7f91e3e09700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91dc108740 0x7f91dc0ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.469+0000 7f91e3e09700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc1133c0 con 0x7f91dc107df0 2024-05-09T04:51:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.469+0000 7f91e3e09700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc113240 con 0x7f91dc1034c0 2024-05-09T04:51:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.469+0000 7f91e3e09700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc113540 con 0x7f91dc108740 2024-05-09T04:51:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.469+0000 7f91e13a4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91dc107df0 0x7f91dc0ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.469+0000 7f91e13a4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91dc107df0 0x7f91dc0ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58466/0 (socket says 172.21.15.46:58466) 2024-05-09T04:51:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.469+0000 7f91e13a4700 1 -- 172.21.15.46:0/2780768722 learned_addr learned my addr 172.21.15.46:0/2780768722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.469+0000 7f91e1ba5700 1 --2- 172.21.15.46:0/2780768722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91dc1034c0 0x7f91dc0ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.469+0000 7f91e23a6700 1 --2- 172.21.15.46:0/2780768722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91dc108740 0x7f91dc0ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.469+0000 7f91e13a4700 1 -- 172.21.15.46:0/2780768722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91dc108740 msgr2=0x7f91dc0ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.469+0000 7f91e13a4700 1 --2- 172.21.15.46:0/2780768722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91dc108740 0x7f91dc0ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.469+0000 7f91e13a4700 1 -- 172.21.15.46:0/2780768722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91dc1034c0 msgr2=0x7f91dc0ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.469+0000 7f91e13a4700 1 --2- 172.21.15.46:0/2780768722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91dc1034c0 0x7f91dc0ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.469+0000 7f91e13a4700 1 -- 172.21.15.46:0/2780768722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91dc1a9260 con 0x7f91dc107df0 2024-05-09T04:51:58.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.469+0000 7f91e13a4700 1 --2- 172.21.15.46:0/2780768722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91dc107df0 0x7f91dc0ff810 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f91d00098e0 tx=0x7f91d00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:58.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.469+0000 7f91daffd700 1 -- 172.21.15.46:0/2780768722 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0027070 con 0x7f91dc107df0 2024-05-09T04:51:58.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.470+0000 7f91daffd700 1 -- 172.21.15.46:0/2780768722 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d0003f20 con 0x7f91dc107df0 2024-05-09T04:51:58.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.470+0000 7f91e3e09700 1 -- 172.21.15.46:0/2780768722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91dc1a94f0 con 0x7f91dc107df0 2024-05-09T04:51:58.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.470+0000 7f91daffd700 1 -- 172.21.15.46:0/2780768722 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0022c70 con 0x7f91dc107df0 2024-05-09T04:51:58.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.470+0000 7f91e3e09700 1 -- 172.21.15.46:0/2780768722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91dc1a99a0 con 0x7f91dc107df0 2024-05-09T04:51:58.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.472+0000 7f91daffd700 1 -- 172.21.15.46:0/2780768722 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f91d002a070 con 0x7f91dc107df0 2024-05-09T04:51:58.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.473+0000 7f91daffd700 1 --2- 172.21.15.46:0/2780768722 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f91c40b2b00 0x7f91c40b5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.473+0000 7f91e1ba5700 1 --2- 172.21.15.46:0/2780768722 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f91c40b2b00 0x7f91c40b5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.473+0000 7f91f281fb80 1 Processor -- start 2024-05-09T04:51:58.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.473+0000 7f91daffd700 1 -- 172.21.15.46:0/2780768722 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f91d00e35e0 con 0x7f91dc107df0 2024-05-09T04:51:58.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91f281fb80 1 -- start start 2024-05-09T04:51:58.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91e1ba5700 1 --2- 172.21.15.46:0/2780768722 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f91c40b2b00 0x7f91c40b5010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009550 tx=0x7f91cc0091b0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:58.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91f281fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559104311160 0x5591043aa4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91f281fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591042274a0 0x5591043aaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91f281fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591043a46a0 0x5591043a4ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91f281fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5591042c6ff0 con 0x5591043a46a0 2024-05-09T04:51:58.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91f281fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5591042657f0 con 0x559104311160 2024-05-09T04:51:58.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91f281fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5591042c2fb0 con 0x5591042274a0 2024-05-09T04:51:58.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91e23a6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591043a46a0 0x5591043a4ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91e23a6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591043a46a0 0x5591043a4ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58480/0 (socket says 172.21.15.46:58480) 2024-05-09T04:51:58.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91e23a6700 1 -- 172.21.15.46:0/1378255134 learned_addr learned my addr 172.21.15.46:0/1378255134 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:58.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91e1ba5700 1 --2- 172.21.15.46:0/1378255134 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559104311160 0x5591043aa4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91e13a4700 1 --2- 172.21.15.46:0/1378255134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591042274a0 0x5591043aaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91e23a6700 1 -- 172.21.15.46:0/1378255134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591042274a0 msgr2=0x5591043aaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:58.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91e23a6700 1 --2- 172.21.15.46:0/1378255134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591042274a0 0x5591043aaa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91e23a6700 1 -- 172.21.15.46:0/1378255134 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559104311160 msgr2=0x5591043aa4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:58.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91e23a6700 1 --2- 172.21.15.46:0/1378255134 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559104311160 0x5591043aa4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.474+0000 7f91e23a6700 1 -- 172.21.15.46:0/1378255134 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55910425d370 con 0x5591043a46a0 2024-05-09T04:51:58.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.475+0000 7f91e23a6700 1 --2- 172.21.15.46:0/1378255134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591043a46a0 0x5591043a4ad0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009a70 tx=0x7f91d4009d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6cd957acaee9361 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:58.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.475+0000 7f91c3fff700 1 -- 172.21.15.46:0/1378255134 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4017070 con 0x5591043a46a0 2024-05-09T04:51:58.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.475+0000 7f91c3fff700 1 -- 172.21.15.46:0/1378255134 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d400ed80 con 0x5591043a46a0 2024-05-09T04:51:58.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.475+0000 7f91c3fff700 1 -- 172.21.15.46:0/1378255134 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4013db0 con 0x5591043a46a0 2024-05-09T04:51:58.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.476+0000 7f91f281fb80 1 -- 172.21.15.46:0/1378255134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591043a46a0 msgr2=0x5591043a4ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:58.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.476+0000 7f91f281fb80 1 --2- 172.21.15.46:0/1378255134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591043a46a0 0x5591043a4ad0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009a70 tx=0x7f91d4009d80 comp rx=0 tx=0).stop 2024-05-09T04:51:58.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.476+0000 7f91f281fb80 1 -- 172.21.15.46:0/1378255134 shutdown_connections 2024-05-09T04:51:58.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.476+0000 7f91f281fb80 1 --2- 172.21.15.46:0/1378255134 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591043a46a0 0x5591043a4ad0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.476+0000 7f91f281fb80 1 --2- 172.21.15.46:0/1378255134 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559104311160 0x5591043aa4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.476+0000 7f91f281fb80 1 --2- 172.21.15.46:0/1378255134 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591042274a0 0x5591043aaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.476+0000 7f91f281fb80 1 -- 172.21.15.46:0/1378255134 >> 172.21.15.46:0/1378255134 conn(0x5591042876a0 msgr2=0x55910439f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:58.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.476+0000 7f91f281fb80 1 -- 172.21.15.46:0/1378255134 shutdown_connections 2024-05-09T04:51:58.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.476+0000 7f91f281fb80 1 -- 172.21.15.46:0/1378255134 wait complete. 2024-05-09T04:51:58.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.476+0000 7f91f281fb80 1 -- start start 2024-05-09T04:51:58.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.476+0000 7f91f281fb80 1 Processor -- start 2024-05-09T04:51:58.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91f281fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591042274a0 0x5591043ae140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91f281fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559104311160 0x5591043ae680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91f281fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591043a46a0 0x5591043aebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91f281fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559104142ba0 con 0x5591042274a0 2024-05-09T04:51:58.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91f281fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559104281150 con 0x559104311160 2024-05-09T04:51:58.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91f281fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5591043b11d0 con 0x5591043a46a0 2024-05-09T04:51:58.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91f281fb80 10 client.0 _ll_register_callbacks cb 0x5591041386c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:51:58.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91e1ba5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591042274a0 0x5591043ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91e1ba5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591042274a0 0x5591043ae140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58496/0 (socket says 172.21.15.46:58496) 2024-05-09T04:51:58.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91e1ba5700 1 -- 172.21.15.46:0/3207192706 learned_addr learned my addr 172.21.15.46:0/3207192706 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:58.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91e23a6700 1 --2- 172.21.15.46:0/3207192706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591043a46a0 0x5591043aebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91e13a4700 1 --2- 172.21.15.46:0/3207192706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559104311160 0x5591043ae680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91e1ba5700 1 -- 172.21.15.46:0/3207192706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591043a46a0 msgr2=0x5591043aebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:58.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91e1ba5700 1 --2- 172.21.15.46:0/3207192706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591043a46a0 0x5591043aebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91e1ba5700 1 -- 172.21.15.46:0/3207192706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559104311160 msgr2=0x5591043ae680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:58.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91e1ba5700 1 --2- 172.21.15.46:0/3207192706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559104311160 0x5591043ae680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91e1ba5700 1 -- 172.21.15.46:0/3207192706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559104142490 con 0x5591042274a0 2024-05-09T04:51:58.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.477+0000 7f91e13a4700 1 --2- 172.21.15.46:0/3207192706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559104311160 0x5591043ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:51:58.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.478+0000 7f91e1ba5700 1 --2- 172.21.15.46:0/3207192706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591042274a0 0x5591043ae140 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f91cc007ab0 tx=0x7f91cc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d718aab5ce659a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:58.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.478+0000 7f91c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:51:58.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.478+0000 7f91c17fa700 1 -- 172.21.15.46:0/3207192706 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91cc026070 con 0x5591042274a0 2024-05-09T04:51:58.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.478+0000 7f91c17fa700 1 -- 172.21.15.46:0/3207192706 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f91cc015ce0 con 0x5591042274a0 2024-05-09T04:51:58.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.478+0000 7f91f281fb80 1 -- 172.21.15.46:0/3207192706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91d400ed80 con 0x5591042274a0 2024-05-09T04:51:58.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.478+0000 7f91c17fa700 1 -- 172.21.15.46:0/3207192706 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f91cc032050 con 0x5591042274a0 2024-05-09T04:51:58.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.478+0000 7f91f281fb80 10 client.15435 fetch_fsmap learned FSMap version 25 2024-05-09T04:51:58.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.478+0000 7f91f281fb80 1 -- 172.21.15.46:0/3207192706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5591043bbb70 con 0x5591042274a0 2024-05-09T04:51:58.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.478+0000 7f91c17fa700 1 -- 172.21.15.46:0/3207192706 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d400ed80 con 0x5591042274a0 2024-05-09T04:51:58.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.479+0000 7f91c17fa700 1 -- 172.21.15.46:0/3207192706 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f91cc0163f0 con 0x5591042274a0 2024-05-09T04:51:58.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.479+0000 7f91f281fb80 10 client.15435 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:51:58.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.479+0000 7f91f281fb80 10 client.15435 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:51:58.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.479+0000 7f91f281fb80 20 client.15435 populate_metadata read hostname 'smithi046' 2024-05-09T04:51:58.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.479+0000 7f91f281fb80 1 --2- 172.21.15.46:0/3207192706 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5591043be150 0x5591043c05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.479+0000 7f91f281fb80 4 client.15435 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:51:58.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.479+0000 7f91f281fb80 1 -- 172.21.15.46:0/3207192706 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591042d49d0 con 0x5591043be150 2024-05-09T04:51:58.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.479+0000 7f91e13a4700 1 --2- 172.21.15.46:0/3207192706 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5591043be150 0x5591043c05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.480+0000 7f91e13a4700 1 --2- 172.21.15.46:0/3207192706 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5591043be150 0x5591043c05b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2faef0544d8d6a23 server_cookie=d4d6c5d6ed559ff5 in_seq=0 out_seq=0 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.480+0000 7f91c17fa700 10 client.15435 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.480+0000 7f91c17fa700 1 -- 172.21.15.46:0/3207192706 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5591042d49d0 con 0x5591043be150 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.480+0000 7f91c17fa700 10 client.15435 handle_command_reply: tid=0 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.480+0000 7f91f281fb80 1 client.15435 shutdown 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.481+0000 7f91f281fb80 1 -- 172.21.15.46:0/3207192706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591042274a0 msgr2=0x5591043ae140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.481+0000 7f91f281fb80 1 --2- 172.21.15.46:0/3207192706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591042274a0 0x5591043ae140 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f91cc007ab0 tx=0x7f91cc004b40 comp rx=0 tx=0).stop 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.481+0000 7f91f281fb80 1 -- 172.21.15.46:0/3207192706 shutdown_connections 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.481+0000 7f91f281fb80 1 --2- 172.21.15.46:0/3207192706 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5591043be150 0x5591043c05b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.481+0000 7f91f281fb80 1 --2- 172.21.15.46:0/3207192706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591043a46a0 0x5591043aebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.481+0000 7f91f281fb80 1 --2- 172.21.15.46:0/3207192706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591042274a0 0x5591043ae140 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.481+0000 7f91f281fb80 1 --2- 172.21.15.46:0/3207192706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559104311160 0x5591043ae680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.481+0000 7f91c17fa700 0 client.15435 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.481+0000 7f91f281fb80 1 -- 172.21.15.46:0/3207192706 >> 172.21.15.46:0/3207192706 conn(0x5591042876a0 msgr2=0x55910439f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.481+0000 7f91f281fb80 1 -- 172.21.15.46:0/3207192706 shutdown_connections 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.481+0000 7f91f281fb80 1 -- 172.21.15.46:0/3207192706 wait complete. 2024-05-09T04:51:58.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.481+0000 7f91f281fb80 20 client.15435 trim_cache size 0 max 16384 2024-05-09T04:51:58.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.496+0000 7f91f281fb80 1 Processor -- start 2024-05-09T04:51:58.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.496+0000 7f91f281fb80 1 -- start start 2024-05-09T04:51:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.496+0000 7f91f281fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5591042274a0 0x5591043a8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.496+0000 7f91f281fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559104311160 0x5591043a9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.496+0000 7f91f281fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591043a46a0 0x5591043aea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.496+0000 7f91f281fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc1b3980 con 0x5591043a46a0 2024-05-09T04:51:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.496+0000 7f91f281fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc1b1210 con 0x5591042274a0 2024-05-09T04:51:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.496+0000 7f91f281fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc1af060 con 0x559104311160 2024-05-09T04:51:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.497+0000 7f91e23a6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591043a46a0 0x5591043aea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.497+0000 7f91e23a6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591043a46a0 0x5591043aea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58512/0 (socket says 172.21.15.46:58512) 2024-05-09T04:51:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.497+0000 7f91e13a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559104311160 0x5591043a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.497+0000 7f91e1ba5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5591042274a0 0x5591043a8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.497+0000 7f91e13a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559104311160 0x5591043a9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:56914/0 (socket says 172.21.15.46:56914) 2024-05-09T04:51:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.497+0000 7f91e1ba5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5591042274a0 0x5591043a8720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:46884/0 (socket says 172.21.15.46:46884) 2024-05-09T04:51:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.497+0000 7f91e23a6700 1 -- 172.21.15.46:0/960955629 learned_addr learned my addr 172.21.15.46:0/960955629 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:58.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.497+0000 7f91e23a6700 1 -- 172.21.15.46:0/960955629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559104311160 msgr2=0x5591043a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:58.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.497+0000 7f91e23a6700 1 --2- 172.21.15.46:0/960955629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559104311160 0x5591043a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.497+0000 7f91e23a6700 1 -- 172.21.15.46:0/960955629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5591042274a0 msgr2=0x5591043a8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:58.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.497+0000 7f91e23a6700 1 --2- 172.21.15.46:0/960955629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5591042274a0 0x5591043a8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.497+0000 7f91e23a6700 1 -- 172.21.15.46:0/960955629 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591043aef40 con 0x5591043a46a0 2024-05-09T04:51:58.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.497+0000 7f91e23a6700 1 --2- 172.21.15.46:0/960955629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591043a46a0 0x5591043aea00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009a70 tx=0x7f91d40138b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0aada857f394417 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:58.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.498+0000 7f91c37fe700 1 -- 172.21.15.46:0/960955629 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d402a070 con 0x5591043a46a0 2024-05-09T04:51:58.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.498+0000 7f91c37fe700 1 -- 172.21.15.46:0/960955629 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d400e8d0 con 0x5591043a46a0 2024-05-09T04:51:58.500 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.498+0000 7f91c37fe700 1 -- 172.21.15.46:0/960955629 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d4017040 con 0x5591043a46a0 2024-05-09T04:51:58.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.498+0000 7f91f281fb80 1 -- 172.21.15.46:0/960955629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591043a46a0 msgr2=0x5591043aea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:58.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.498+0000 7f91f281fb80 1 --2- 172.21.15.46:0/960955629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591043a46a0 0x5591043aea00 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f91d4009a70 tx=0x7f91d40138b0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.498+0000 7f91f281fb80 1 -- 172.21.15.46:0/960955629 shutdown_connections 2024-05-09T04:51:58.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.498+0000 7f91f281fb80 1 --2- 172.21.15.46:0/960955629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5591043a46a0 0x5591043aea00 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.498+0000 7f91f281fb80 1 --2- 172.21.15.46:0/960955629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5591042274a0 0x5591043a8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.498+0000 7f91f281fb80 1 --2- 172.21.15.46:0/960955629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559104311160 0x5591043a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.498+0000 7f91f281fb80 1 -- 172.21.15.46:0/960955629 >> 172.21.15.46:0/960955629 conn(0x5591042876a0 msgr2=0x5591043a9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:58.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.499+0000 7f91f281fb80 1 -- 172.21.15.46:0/960955629 shutdown_connections 2024-05-09T04:51:58.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.499+0000 7f91f281fb80 1 -- 172.21.15.46:0/960955629 wait complete. 2024-05-09T04:51:58.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.499+0000 7f91f281fb80 1 -- start start 2024-05-09T04:51:58.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.499+0000 7f91f281fb80 1 Processor -- start 2024-05-09T04:51:58.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91f281fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5591042274a0 0x5591043c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91f281fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559104311160 0x5591043c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91f281fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591043a46a0 0x5591043c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91f281fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc1af6d0 con 0x559104311160 2024-05-09T04:51:58.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91f281fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc1b0b40 con 0x5591042274a0 2024-05-09T04:51:58.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91f281fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f91dc1afda0 con 0x5591043a46a0 2024-05-09T04:51:58.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91f281fb80 10 client.0 _ll_register_callbacks cb 0x5591041386c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:51:58.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91e13a4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559104311160 0x5591043c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91e13a4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559104311160 0x5591043c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58514/0 (socket says 172.21.15.46:58514) 2024-05-09T04:51:58.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91e13a4700 1 -- 172.21.15.46:0/1858062814 learned_addr learned my addr 172.21.15.46:0/1858062814 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:51:58.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91e1ba5700 1 --2- 172.21.15.46:0/1858062814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5591042274a0 0x5591043c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91e23a6700 1 --2- 172.21.15.46:0/1858062814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591043a46a0 0x5591043c57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91e13a4700 1 -- 172.21.15.46:0/1858062814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591043a46a0 msgr2=0x5591043c57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:58.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91e13a4700 1 --2- 172.21.15.46:0/1858062814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591043a46a0 0x5591043c57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91e13a4700 1 -- 172.21.15.46:0/1858062814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5591042274a0 msgr2=0x5591043c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:51:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91e13a4700 1 --2- 172.21.15.46:0/1858062814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5591042274a0 0x5591043c4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91e13a4700 1 -- 172.21.15.46:0/1858062814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591043b5440 con 0x559104311160 2024-05-09T04:51:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91e23a6700 1 --2- 172.21.15.46:0/1858062814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591043a46a0 0x5591043c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:51:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91e13a4700 1 --2- 172.21.15.46:0/1858062814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559104311160 0x5591043c52a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009990 tx=0x7f91d0003790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13753a111c99d186 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:51:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:51:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91c17fa700 1 -- 172.21.15.46:0/1858062814 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d00aa760 con 0x559104311160 2024-05-09T04:51:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91c17fa700 1 -- 172.21.15.46:0/1858062814 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f91d00a6d50 con 0x559104311160 2024-05-09T04:51:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.500+0000 7f91f281fb80 1 -- 172.21.15.46:0/1858062814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f91d400e8d0 con 0x559104311160 2024-05-09T04:51:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.501+0000 7f91c17fa700 1 -- 172.21.15.46:0/1858062814 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f91d00aeb10 con 0x559104311160 2024-05-09T04:51:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.501+0000 7f91f281fb80 10 client.15441 fetch_fsmap learned FSMap version 25 2024-05-09T04:51:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.501+0000 7f91f281fb80 1 -- 172.21.15.46:0/1858062814 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5591043b6720 con 0x559104311160 2024-05-09T04:51:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.501+0000 7f91c17fa700 1 -- 172.21.15.46:0/1858062814 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d400e8d0 con 0x559104311160 2024-05-09T04:51:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.501+0000 7f91c17fa700 1 -- 172.21.15.46:0/1858062814 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f91d00b30a0 con 0x559104311160 2024-05-09T04:51:58.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.501+0000 7f91f281fb80 10 client.15441 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:51:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.501+0000 7f91f281fb80 10 client.15441 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:51:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.501+0000 7f91f281fb80 20 client.15441 populate_metadata read hostname 'smithi046' 2024-05-09T04:51:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.501+0000 7f91f281fb80 1 --2- 172.21.15.46:0/1858062814 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x559104428920 0x55910442ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:51:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.501+0000 7f91f281fb80 4 client.15441 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:51:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.501+0000 7f91f281fb80 1 -- 172.21.15.46:0/1858062814 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591042cea70 con 0x559104428920 2024-05-09T04:51:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.501+0000 7f91e1ba5700 1 --2- 172.21.15.46:0/1858062814 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x559104428920 0x55910442ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:51:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91c17fa700 10 client.15441 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:51:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91e1ba5700 1 --2- 172.21.15.46:0/1858062814 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x559104428920 0x55910442ad40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a1ec031113cd4932 server_cookie=814065a150a2df4 in_seq=0 out_seq=0 2024-05-09T04:51:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91c17fa700 1 -- 172.21.15.46:0/1858062814 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591042cea70 con 0x559104428920 2024-05-09T04:51:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91c17fa700 10 client.15441 handle_command_reply: tid=0 2024-05-09T04:51:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91f281fb80 1 client.15441 shutdown 2024-05-09T04:51:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91f281fb80 1 -- 172.21.15.46:0/1858062814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559104311160 msgr2=0x5591043c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91f281fb80 1 --2- 172.21.15.46:0/1858062814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559104311160 0x5591043c52a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009990 tx=0x7f91d0003790 comp rx=0 tx=0).stop 2024-05-09T04:51:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91f281fb80 1 -- 172.21.15.46:0/1858062814 shutdown_connections 2024-05-09T04:51:58.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91f281fb80 1 --2- 172.21.15.46:0/1858062814 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x559104428920 0x55910442ad40 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91f281fb80 1 --2- 172.21.15.46:0/1858062814 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5591043a46a0 0x5591043c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91f281fb80 1 --2- 172.21.15.46:0/1858062814 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5591042274a0 0x5591043c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91f281fb80 1 --2- 172.21.15.46:0/1858062814 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559104311160 0x5591043c52a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91c17fa700 0 client.15441 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:51:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91f281fb80 1 -- 172.21.15.46:0/1858062814 >> 172.21.15.46:0/1858062814 conn(0x5591042876a0 msgr2=0x5591043a7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91f281fb80 1 -- 172.21.15.46:0/1858062814 shutdown_connections 2024-05-09T04:51:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91f281fb80 1 -- 172.21.15.46:0/1858062814 wait complete. 2024-05-09T04:51:58.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.502+0000 7f91f281fb80 20 client.15441 trim_cache size 0 max 16384 2024-05-09T04:51:58.508 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:51:58.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.503+0000 7f91e3e09700 1 -- 172.21.15.46:0/2780768722 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f91c40b2b00 msgr2=0x7f91c40b5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:58.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.503+0000 7f91e3e09700 1 --2- 172.21.15.46:0/2780768722 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f91c40b2b00 0x7f91c40b5010 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f91cc009550 tx=0x7f91cc0091b0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.503+0000 7f91e3e09700 1 -- 172.21.15.46:0/2780768722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91dc107df0 msgr2=0x7f91dc0ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:51:58.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.503+0000 7f91e3e09700 1 --2- 172.21.15.46:0/2780768722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91dc107df0 0x7f91dc0ff810 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f91d00098e0 tx=0x7f91d00057c0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.503+0000 7f91e3e09700 1 -- 172.21.15.46:0/2780768722 shutdown_connections 2024-05-09T04:51:58.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.503+0000 7f91e3e09700 1 --2- 172.21.15.46:0/2780768722 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f91c40b2b00 0x7f91c40b5010 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.503+0000 7f91e3e09700 1 --2- 172.21.15.46:0/2780768722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f91dc108740 0x7f91dc0ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.503+0000 7f91e3e09700 1 --2- 172.21.15.46:0/2780768722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f91dc1034c0 0x7f91dc0ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.503+0000 7f91e3e09700 1 --2- 172.21.15.46:0/2780768722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f91dc107df0 0x7f91dc0ff810 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:51:58.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.503+0000 7f91e3e09700 1 -- 172.21.15.46:0/2780768722 >> 172.21.15.46:0/2780768722 conn(0x7f91dc0fedf0 msgr2=0x7f91dc075df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:51:58.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.503+0000 7f91e3e09700 1 -- 172.21.15.46:0/2780768722 shutdown_connections 2024-05-09T04:51:58.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:51:58.503+0000 7f91e3e09700 1 -- 172.21.15.46:0/2780768722 wait complete. 2024-05-09T04:51:59.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:51:58 smithi110 ceph-mon[34904]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-09T04:51:59.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:51:58 smithi046 ceph-mon[30136]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-09T04:51:59.126 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:51:59.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:51:58 smithi072 ceph-mon[34088]: pgmap v29: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-05-09T04:51:59.378 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:00.580 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:52:00 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:52:00.316+0000 7f884664d700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:52:00.580 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:52:00 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:52:00.316+0000 7f884664d700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:52:00.580 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:52:00 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:52:00.316+0000 7f884664d700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:52:00.580 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:52:00 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:52:00.316+0000 7f884664d700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:52:00.580 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 04:52:00 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T04:52:00.316+0000 7f884664d700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-09T04:52:00.918 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:00 smithi046 ceph-mon[30136]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-09T04:52:01.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:00 smithi110 ceph-mon[34904]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-09T04:52:01.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:00 smithi072 ceph-mon[34088]: pgmap v30: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 5 op/s 2024-05-09T04:52:01.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.219+0000 7f95f4c93700 1 -- 172.21.15.46:0/2203614745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f01034c0 msgr2=0x7f95f01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:01.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.219+0000 7f95f4c93700 1 --2- 172.21.15.46:0/2203614745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f01034c0 0x7f95f01038d0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f95e00099c0 tx=0x7f95e0009cd0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.219+0000 7f95f4c93700 1 -- 172.21.15.46:0/2203614745 shutdown_connections 2024-05-09T04:52:01.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.219+0000 7f95f4c93700 1 --2- 172.21.15.46:0/2203614745 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95f0108740 0x7f95f01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.219+0000 7f95f4c93700 1 --2- 172.21.15.46:0/2203614745 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f01034c0 0x7f95f01038d0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.219+0000 7f95f4c93700 1 --2- 172.21.15.46:0/2203614745 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95f0107df0 0x7f95f0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.219+0000 7f95f4c93700 1 -- 172.21.15.46:0/2203614745 >> 172.21.15.46:0/2203614745 conn(0x7f95f00fedc0 msgr2=0x7f95f01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:01.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.220+0000 7f95f4c93700 1 -- 172.21.15.46:0/2203614745 shutdown_connections 2024-05-09T04:52:01.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.220+0000 7f95f4c93700 1 -- 172.21.15.46:0/2203614745 wait complete. 2024-05-09T04:52:01.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.220+0000 7f95f4c93700 1 Processor -- start 2024-05-09T04:52:01.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.221+0000 7f95f4c93700 1 -- start start 2024-05-09T04:52:01.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.221+0000 7f95f4c93700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95f01034c0 0x7f95f019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.221+0000 7f95f4c93700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95f0107df0 0x7f95f019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.221+0000 7f95f4c93700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f0108740 0x7f95f019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.221+0000 7f95f4c93700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01144c0 con 0x7f95f0108740 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.221+0000 7f95f4c93700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0114340 con 0x7f95f01034c0 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.221+0000 7f95f4c93700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0114640 con 0x7f95f0107df0 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.221+0000 7f95eed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f0108740 0x7f95f019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.221+0000 7f95e5d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95f0107df0 0x7f95f019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.221+0000 7f95eed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f0108740 0x7f95f019f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58570/0 (socket says 172.21.15.46:58570) 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.221+0000 7f95eed9d700 1 -- 172.21.15.46:0/4033490432 learned_addr learned my addr 172.21.15.46:0/4033490432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.221+0000 7f95ee59c700 1 --2- 172.21.15.46:0/4033490432 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95f01034c0 0x7f95f019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.222+0000 7f95eed9d700 1 -- 172.21.15.46:0/4033490432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95f0107df0 msgr2=0x7f95f019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.222+0000 7f95eed9d700 1 --2- 172.21.15.46:0/4033490432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95f0107df0 0x7f95f019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.222+0000 7f95eed9d700 1 -- 172.21.15.46:0/4033490432 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95f01034c0 msgr2=0x7f95f019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.222+0000 7f95eed9d700 1 --2- 172.21.15.46:0/4033490432 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95f01034c0 0x7f95f019adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.222+0000 7f95eed9d700 1 -- 172.21.15.46:0/4033490432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f019fc10 con 0x7f95f0108740 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.222+0000 7f95eed9d700 1 --2- 172.21.15.46:0/4033490432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f0108740 0x7f95f019f620 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f95e8007d70 tx=0x7f95e800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.222+0000 7f95e7fff700 1 -- 172.21.15.46:0/4033490432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8016020 con 0x7f95f0108740 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.222+0000 7f95e7fff700 1 -- 172.21.15.46:0/4033490432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e800ac90 con 0x7f95f0108740 2024-05-09T04:52:01.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.222+0000 7f95e7fff700 1 -- 172.21.15.46:0/4033490432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80115e0 con 0x7f95f0108740 2024-05-09T04:52:01.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.222+0000 7f95f4c93700 1 -- 172.21.15.46:0/4033490432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f019ff00 con 0x7f95f0108740 2024-05-09T04:52:01.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.222+0000 7f95f4c93700 1 -- 172.21.15.46:0/4033490432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f01a03f0 con 0x7f95f0108740 2024-05-09T04:52:01.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.226+0000 7f95e7fff700 1 -- 172.21.15.46:0/4033490432 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 24) ==== 149828+0+0 (secure 0 0 0) 0x7f95e8019020 con 0x7f95f0108740 2024-05-09T04:52:01.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.227+0000 7f96036a9b80 1 Processor -- start 2024-05-09T04:52:01.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.227+0000 7f96036a9b80 1 -- start start 2024-05-09T04:52:01.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.227+0000 7f95e7fff700 1 --2- 172.21.15.46:0/4033490432 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f95dc0b2a80 0x7f95dc0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.227+0000 7f95e5d9b700 1 --2- 172.21.15.46:0/4033490432 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f95dc0b2a80 0x7f95dc0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.227+0000 7f95e7fff700 1 -- 172.21.15.46:0/4033490432 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f95e80e2600 con 0x7f95f0108740 2024-05-09T04:52:01.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.227+0000 7f96036a9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af06b8d00 0x560af07c1240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.227+0000 7f96036a9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af063e4a0 0x560af07c1780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.227+0000 7f96036a9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af07bb400 0x560af07bb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.227+0000 7f96036a9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560af06de0a0 con 0x560af07bb400 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.227+0000 7f96036a9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x560af06839a0 con 0x560af06b8d00 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.227+0000 7f96036a9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560af06da060 con 0x560af063e4a0 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95eed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af07bb400 0x560af07bb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95ee59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af06b8d00 0x560af07c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95eed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af07bb400 0x560af07bb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58582/0 (socket says 172.21.15.46:58582) 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95ee59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af06b8d00 0x560af07c1240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:46940/0 (socket says 172.21.15.46:46940) 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95eed9d700 1 -- 172.21.15.46:0/121526811 learned_addr learned my addr 172.21.15.46:0/121526811 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95e5d9b700 1 --2- 172.21.15.46:0/121526811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af063e4a0 0x560af07c1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95e5d9b700 1 --2- 172.21.15.46:0/4033490432 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f95dc0b2a80 0x7f95dc0b4f90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f95f0107cf0 tx=0x7f95d8011440 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95eed9d700 1 -- 172.21.15.46:0/121526811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af063e4a0 msgr2=0x560af07c1780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95eed9d700 1 --2- 172.21.15.46:0/121526811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af063e4a0 0x560af07c1780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95eed9d700 1 -- 172.21.15.46:0/121526811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af06b8d00 msgr2=0x560af07c1240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95eed9d700 1 --2- 172.21.15.46:0/121526811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af06b8d00 0x560af07c1240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95eed9d700 1 -- 172.21.15.46:0/121526811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af0674500 con 0x560af07bb400 2024-05-09T04:52:01.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95ee59c700 1 --2- 172.21.15.46:0/121526811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af06b8d00 0x560af07c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:01.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95eed9d700 1 --2- 172.21.15.46:0/121526811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af07bb400 0x560af07bb810 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f95e80a9790 tx=0x7f95e80a9220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f682472aec17626c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:01.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95e4d99700 1 -- 172.21.15.46:0/121526811 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80a9310 con 0x560af07bb400 2024-05-09T04:52:01.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95e4d99700 1 -- 172.21.15.46:0/121526811 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e80a9f00 con 0x560af07bb400 2024-05-09T04:52:01.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.228+0000 7f95e4d99700 1 -- 172.21.15.46:0/121526811 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80b21d0 con 0x560af07bb400 2024-05-09T04:52:01.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.229+0000 7f96036a9b80 1 -- 172.21.15.46:0/121526811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af07bb400 msgr2=0x560af07bb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:01.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.229+0000 7f96036a9b80 1 --2- 172.21.15.46:0/121526811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af07bb400 0x560af07bb810 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f95e80a9790 tx=0x7f95e80a9220 comp rx=0 tx=0).stop 2024-05-09T04:52:01.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.229+0000 7f96036a9b80 1 -- 172.21.15.46:0/121526811 shutdown_connections 2024-05-09T04:52:01.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.229+0000 7f96036a9b80 1 --2- 172.21.15.46:0/121526811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af07bb400 0x560af07bb810 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.229+0000 7f96036a9b80 1 --2- 172.21.15.46:0/121526811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af06b8d00 0x560af07c1240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.229+0000 7f96036a9b80 1 --2- 172.21.15.46:0/121526811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af063e4a0 0x560af07c1780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.229+0000 7f96036a9b80 1 -- 172.21.15.46:0/121526811 >> 172.21.15.46:0/121526811 conn(0x560af0641f70 msgr2=0x560af07b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:01.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.230+0000 7f96036a9b80 1 -- 172.21.15.46:0/121526811 shutdown_connections 2024-05-09T04:52:01.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.230+0000 7f96036a9b80 1 -- 172.21.15.46:0/121526811 wait complete. 2024-05-09T04:52:01.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.230+0000 7f96036a9b80 1 -- start start 2024-05-09T04:52:01.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.230+0000 7f96036a9b80 1 Processor -- start 2024-05-09T04:52:01.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.230+0000 7f96036a9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af063e4a0 0x560af07c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.230+0000 7f96036a9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af06b8d00 0x560af07c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.230+0000 7f96036a9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af07bb400 0x560af07c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.230+0000 7f96036a9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560af07bc0a0 con 0x560af06b8d00 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.230+0000 7f96036a9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x560af0678b20 con 0x560af063e4a0 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.230+0000 7f96036a9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560af07bc220 con 0x560af07bb400 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.230+0000 7f96036a9b80 10 client.0 _ll_register_callbacks cb 0x560af054f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f95e5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af06b8d00 0x560af07c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f95e5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af06b8d00 0x560af07c55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58590/0 (socket says 172.21.15.46:58590) 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f95e5d9b700 1 -- 172.21.15.46:0/1438340017 learned_addr learned my addr 172.21.15.46:0/1438340017 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f95ee59c700 1 --2- 172.21.15.46:0/1438340017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af063e4a0 0x560af07c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f95eed9d700 1 --2- 172.21.15.46:0/1438340017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af07bb400 0x560af07c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f95e5d9b700 1 -- 172.21.15.46:0/1438340017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af07bb400 msgr2=0x560af07c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f95e5d9b700 1 --2- 172.21.15.46:0/1438340017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af07bb400 0x560af07c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f95e5d9b700 1 -- 172.21.15.46:0/1438340017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af063e4a0 msgr2=0x560af07c50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f95e5d9b700 1 --2- 172.21.15.46:0/1438340017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af063e4a0 0x560af07c50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f95e5d9b700 1 -- 172.21.15.46:0/1438340017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af0559490 con 0x560af06b8d00 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f95eed9d700 1 --2- 172.21.15.46:0/1438340017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af07bb400 0x560af07c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f95e5d9b700 1 --2- 172.21.15.46:0/1438340017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af06b8d00 0x560af07c55f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f95d801e880 tx=0x7f95d80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee6b380bc8b01e2a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f95d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f95d5ffb700 1 -- 172.21.15.46:0/1438340017 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8017030 con 0x560af06b8d00 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f95d5ffb700 1 -- 172.21.15.46:0/1438340017 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f95d800f650 con 0x560af06b8d00 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.231+0000 7f96036a9b80 1 -- 172.21.15.46:0/1438340017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e80a9f00 con 0x560af06b8d00 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.232+0000 7f95d5ffb700 1 -- 172.21.15.46:0/1438340017 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f95d800f850 con 0x560af06b8d00 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.232+0000 7f96036a9b80 10 client.15459 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.232+0000 7f96036a9b80 1 -- 172.21.15.46:0/1438340017 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560af07d29e0 con 0x560af06b8d00 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.232+0000 7f95d5ffb700 1 -- 172.21.15.46:0/1438340017 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e80a9f00 con 0x560af06b8d00 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.232+0000 7f95d5ffb700 1 -- 172.21.15.46:0/1438340017 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f95d8031bd0 con 0x560af06b8d00 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.232+0000 7f96036a9b80 10 client.15459 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.232+0000 7f96036a9b80 10 client.15459 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.232+0000 7f96036a9b80 20 client.15459 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.232+0000 7f96036a9b80 1 --2- 172.21.15.46:0/1438340017 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x560af07d5070 0x560af07d7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.232+0000 7f96036a9b80 4 client.15459 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:01.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.232+0000 7f96036a9b80 1 -- 172.21.15.46:0/1438340017 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560af06eba80 con 0x560af07d5070 2024-05-09T04:52:01.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.233+0000 7f95ee59c700 1 --2- 172.21.15.46:0/1438340017 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x560af07d5070 0x560af07d7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.233+0000 7f95ee59c700 1 --2- 172.21.15.46:0/1438340017 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x560af07d5070 0x560af07d7490 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6b63334ccdeeb002 server_cookie=576230998141f91a in_seq=0 out_seq=0 2024-05-09T04:52:01.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.233+0000 7f95d5ffb700 10 client.15459 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:52:01.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.234+0000 7f95d5ffb700 1 -- 172.21.15.46:0/1438340017 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560af06eba80 con 0x560af07d5070 2024-05-09T04:52:01.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.234+0000 7f95d5ffb700 10 client.15459 handle_command_reply: tid=0 2024-05-09T04:52:01.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.234+0000 7f96036a9b80 1 client.15459 shutdown 2024-05-09T04:52:01.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.234+0000 7f96036a9b80 1 -- 172.21.15.46:0/1438340017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af06b8d00 msgr2=0x560af07c55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:01.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.234+0000 7f96036a9b80 1 --2- 172.21.15.46:0/1438340017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af06b8d00 0x560af07c55f0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f95d801e880 tx=0x7f95d80047c0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.234+0000 7f96036a9b80 1 -- 172.21.15.46:0/1438340017 shutdown_connections 2024-05-09T04:52:01.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.234+0000 7f96036a9b80 1 --2- 172.21.15.46:0/1438340017 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x560af07d5070 0x560af07d7490 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.234+0000 7f96036a9b80 1 --2- 172.21.15.46:0/1438340017 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af07bb400 0x560af07c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.234+0000 7f96036a9b80 1 --2- 172.21.15.46:0/1438340017 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af063e4a0 0x560af07c50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.234+0000 7f96036a9b80 1 --2- 172.21.15.46:0/1438340017 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af06b8d00 0x560af07c55f0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.234+0000 7f95d5ffb700 0 client.15459 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:52:01.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.234+0000 7f96036a9b80 1 -- 172.21.15.46:0/1438340017 >> 172.21.15.46:0/1438340017 conn(0x560af0641f70 msgr2=0x560af07afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:01.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.234+0000 7f96036a9b80 1 -- 172.21.15.46:0/1438340017 shutdown_connections 2024-05-09T04:52:01.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.234+0000 7f96036a9b80 1 -- 172.21.15.46:0/1438340017 wait complete. 2024-05-09T04:52:01.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.234+0000 7f96036a9b80 20 client.15459 trim_cache size 0 max 16384 2024-05-09T04:52:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.249+0000 7f96036a9b80 1 Processor -- start 2024-05-09T04:52:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.249+0000 7f96036a9b80 1 -- start start 2024-05-09T04:52:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f96036a9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af063e4a0 0x560af07b0700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f96036a9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af06b8d00 0x560af07c5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f96036a9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af07bb400 0x560af07d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f96036a9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01b1770 con 0x560af06b8d00 2024-05-09T04:52:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f96036a9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01af000 con 0x560af063e4a0 2024-05-09T04:52:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f96036a9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01ace50 con 0x560af07bb400 2024-05-09T04:52:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f95e5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af06b8d00 0x560af07c5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f95e5d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af06b8d00 0x560af07c5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58596/0 (socket says 172.21.15.46:58596) 2024-05-09T04:52:01.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f95e5d9b700 1 -- 172.21.15.46:0/463189140 learned_addr learned my addr 172.21.15.46:0/463189140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:01.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f95ee59c700 1 --2- 172.21.15.46:0/463189140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af063e4a0 0x560af07b0700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f95eed9d700 1 --2- 172.21.15.46:0/463189140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af07bb400 0x560af07d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f95e5d9b700 1 -- 172.21.15.46:0/463189140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af07bb400 msgr2=0x560af07d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:01.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f95e5d9b700 1 --2- 172.21.15.46:0/463189140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af07bb400 0x560af07d0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f95e5d9b700 1 -- 172.21.15.46:0/463189140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af063e4a0 msgr2=0x560af07b0700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:01.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f95e5d9b700 1 --2- 172.21.15.46:0/463189140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af063e4a0 0x560af07b0700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f95e5d9b700 1 -- 172.21.15.46:0/463189140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560af07bc5b0 con 0x560af06b8d00 2024-05-09T04:52:01.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.250+0000 7f95e5d9b700 1 --2- 172.21.15.46:0/463189140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af06b8d00 0x560af07c5620 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f95d8005950 tx=0x7f95d800f6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401fed77dfc0136a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:01.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.251+0000 7f95d7fff700 1 -- 172.21.15.46:0/463189140 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d800fbf0 con 0x560af06b8d00 2024-05-09T04:52:01.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.251+0000 7f95d7fff700 1 -- 172.21.15.46:0/463189140 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95d8019050 con 0x560af06b8d00 2024-05-09T04:52:01.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.251+0000 7f95d7fff700 1 -- 172.21.15.46:0/463189140 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95d8028c80 con 0x560af06b8d00 2024-05-09T04:52:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.251+0000 7f96036a9b80 1 -- 172.21.15.46:0/463189140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af06b8d00 msgr2=0x560af07c5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.251+0000 7f96036a9b80 1 --2- 172.21.15.46:0/463189140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af06b8d00 0x560af07c5620 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f95d8005950 tx=0x7f95d800f6d0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.251+0000 7f96036a9b80 1 -- 172.21.15.46:0/463189140 shutdown_connections 2024-05-09T04:52:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.251+0000 7f96036a9b80 1 --2- 172.21.15.46:0/463189140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af07bb400 0x560af07d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.251+0000 7f96036a9b80 1 --2- 172.21.15.46:0/463189140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af063e4a0 0x560af07b0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.251+0000 7f96036a9b80 1 --2- 172.21.15.46:0/463189140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af06b8d00 0x560af07c5620 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.251+0000 7f96036a9b80 1 -- 172.21.15.46:0/463189140 >> 172.21.15.46:0/463189140 conn(0x560af0641f70 msgr2=0x560af07afff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.252+0000 7f96036a9b80 1 -- 172.21.15.46:0/463189140 shutdown_connections 2024-05-09T04:52:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.252+0000 7f96036a9b80 1 -- 172.21.15.46:0/463189140 wait complete. 2024-05-09T04:52:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.252+0000 7f96036a9b80 1 -- start start 2024-05-09T04:52:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.252+0000 7f96036a9b80 1 Processor -- start 2024-05-09T04:52:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.252+0000 7f96036a9b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af063e4a0 0x560af07d06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.252+0000 7f96036a9b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af06b8d00 0x560af07d0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.252+0000 7f96036a9b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af07bb400 0x560af07d1150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.252+0000 7f96036a9b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01ad4c0 con 0x560af07bb400 2024-05-09T04:52:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.252+0000 7f96036a9b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01ae930 con 0x560af06b8d00 2024-05-09T04:52:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.252+0000 7f96036a9b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f95f01adb90 con 0x560af063e4a0 2024-05-09T04:52:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.252+0000 7f96036a9b80 10 client.0 _ll_register_callbacks cb 0x560af054f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.252+0000 7f95eed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af07bb400 0x560af07d1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f95eed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af07bb400 0x560af07d1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58598/0 (socket says 172.21.15.46:58598) 2024-05-09T04:52:01.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f95eed9d700 1 -- 172.21.15.46:0/3364016930 learned_addr learned my addr 172.21.15.46:0/3364016930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f95ee59c700 1 --2- 172.21.15.46:0/3364016930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af063e4a0 0x560af07d06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f95e5d9b700 1 --2- 172.21.15.46:0/3364016930 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af06b8d00 0x560af07d0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f95eed9d700 1 -- 172.21.15.46:0/3364016930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af063e4a0 msgr2=0x560af07d06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f95eed9d700 1 --2- 172.21.15.46:0/3364016930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af063e4a0 0x560af07d06d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f95eed9d700 1 -- 172.21.15.46:0/3364016930 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af06b8d00 msgr2=0x560af07d0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f95eed9d700 1 --2- 172.21.15.46:0/3364016930 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af06b8d00 0x560af07d0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f95eed9d700 1 -- 172.21.15.46:0/3364016930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560af083e850 con 0x560af07bb400 2024-05-09T04:52:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f95eed9d700 1 --2- 172.21.15.46:0/3364016930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af07bb400 0x560af07d1150 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f95e80a6a70 tx=0x7f95e80a9c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a7e76c43f55f3aa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f95d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f95d5ffb700 1 -- 172.21.15.46:0/3364016930 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e80c2070 con 0x560af07bb400 2024-05-09T04:52:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f95d5ffb700 1 -- 172.21.15.46:0/3364016930 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f95e80b23e0 con 0x560af07bb400 2024-05-09T04:52:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f96036a9b80 1 -- 172.21.15.46:0/3364016930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95d8019050 con 0x560af07bb400 2024-05-09T04:52:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f95d5ffb700 1 -- 172.21.15.46:0/3364016930 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f95e80a9310 con 0x560af07bb400 2024-05-09T04:52:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f96036a9b80 10 client.15465 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:01.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.253+0000 7f96036a9b80 1 -- 172.21.15.46:0/3364016930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560af083fab0 con 0x560af07bb400 2024-05-09T04:52:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.254+0000 7f95d5ffb700 1 -- 172.21.15.46:0/3364016930 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95d8019050 con 0x560af07bb400 2024-05-09T04:52:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.254+0000 7f95d5ffb700 1 -- 172.21.15.46:0/3364016930 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f95e80e2110 con 0x560af07bb400 2024-05-09T04:52:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.254+0000 7f96036a9b80 10 client.15465 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.254+0000 7f96036a9b80 10 client.15465 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:52:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.254+0000 7f96036a9b80 20 client.15465 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.254+0000 7f96036a9b80 1 --2- 172.21.15.46:0/3364016930 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x560af083fdf0 0x560af0842210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.254+0000 7f96036a9b80 4 client.15465 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:52:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.254+0000 7f96036a9b80 1 -- 172.21.15.46:0/3364016930 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560af06e5b20 con 0x560af083fdf0 2024-05-09T04:52:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.254+0000 7f95ee59c700 1 --2- 172.21.15.46:0/3364016930 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x560af083fdf0 0x560af0842210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.254+0000 7f95ee59c700 1 --2- 172.21.15.46:0/3364016930 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x560af083fdf0 0x560af0842210 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5a564d9bc6aaa65b server_cookie=dc13369f7284dad6 in_seq=0 out_seq=0 2024-05-09T04:52:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.254+0000 7f95d5ffb700 10 client.15465 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:52:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.255+0000 7f95d5ffb700 1 -- 172.21.15.46:0/3364016930 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560af06e5b20 con 0x560af083fdf0 2024-05-09T04:52:01.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.255+0000 7f95d5ffb700 10 client.15465 handle_command_reply: tid=0 2024-05-09T04:52:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.255+0000 7f96036a9b80 1 client.15465 shutdown 2024-05-09T04:52:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.255+0000 7f96036a9b80 1 -- 172.21.15.46:0/3364016930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af07bb400 msgr2=0x560af07d1150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.255+0000 7f96036a9b80 1 --2- 172.21.15.46:0/3364016930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af07bb400 0x560af07d1150 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f95e80a6a70 tx=0x7f95e80a9c70 comp rx=0 tx=0).stop 2024-05-09T04:52:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.255+0000 7f96036a9b80 1 -- 172.21.15.46:0/3364016930 shutdown_connections 2024-05-09T04:52:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.255+0000 7f96036a9b80 1 --2- 172.21.15.46:0/3364016930 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x560af083fdf0 0x560af0842210 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.255+0000 7f96036a9b80 1 --2- 172.21.15.46:0/3364016930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560af07bb400 0x560af07d1150 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.255+0000 7f96036a9b80 1 --2- 172.21.15.46:0/3364016930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560af063e4a0 0x560af07d06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.255+0000 7f96036a9b80 1 --2- 172.21.15.46:0/3364016930 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560af06b8d00 0x560af07d0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.255+0000 7f95d5ffb700 0 client.15465 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:52:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.255+0000 7f96036a9b80 1 -- 172.21.15.46:0/3364016930 >> 172.21.15.46:0/3364016930 conn(0x560af0641f70 msgr2=0x560af07becd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.255+0000 7f96036a9b80 1 -- 172.21.15.46:0/3364016930 shutdown_connections 2024-05-09T04:52:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.255+0000 7f96036a9b80 1 -- 172.21.15.46:0/3364016930 wait complete. 2024-05-09T04:52:01.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.255+0000 7f96036a9b80 20 client.15465 trim_cache size 0 max 16384 2024-05-09T04:52:01.262 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:52:01.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.256+0000 7f95f4c93700 1 -- 172.21.15.46:0/4033490432 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f95dc0b2a80 msgr2=0x7f95dc0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:01.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.256+0000 7f95f4c93700 1 --2- 172.21.15.46:0/4033490432 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f95dc0b2a80 0x7f95dc0b4f90 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f95f0107cf0 tx=0x7f95d8011440 comp rx=0 tx=0).stop 2024-05-09T04:52:01.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.256+0000 7f95f4c93700 1 -- 172.21.15.46:0/4033490432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f0108740 msgr2=0x7f95f019f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:01.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.256+0000 7f95f4c93700 1 --2- 172.21.15.46:0/4033490432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f0108740 0x7f95f019f620 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f95e8007d70 tx=0x7f95e800a340 comp rx=0 tx=0).stop 2024-05-09T04:52:01.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.256+0000 7f95f4c93700 1 -- 172.21.15.46:0/4033490432 shutdown_connections 2024-05-09T04:52:01.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.256+0000 7f95f4c93700 1 --2- 172.21.15.46:0/4033490432 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f95dc0b2a80 0x7f95dc0b4f90 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.256+0000 7f95f4c93700 1 --2- 172.21.15.46:0/4033490432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f95f0108740 0x7f95f019f620 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.256+0000 7f95f4c93700 1 --2- 172.21.15.46:0/4033490432 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f95f01034c0 0x7f95f019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.256+0000 7f95f4c93700 1 --2- 172.21.15.46:0/4033490432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f95f0107df0 0x7f95f019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.256+0000 7f95f4c93700 1 -- 172.21.15.46:0/4033490432 >> 172.21.15.46:0/4033490432 conn(0x7f95f00fedc0 msgr2=0x7f95f01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.256+0000 7f95f4c93700 1 -- 172.21.15.46:0/4033490432 shutdown_connections 2024-05-09T04:52:01.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:01.256+0000 7f95f4c93700 1 -- 172.21.15.46:0/4033490432 wait complete. 2024-05-09T04:52:01.968 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:52:02.235 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:03.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:02 smithi110 ceph-mon[34904]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-09T04:52:03.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:02 smithi110 ceph-mon[34904]: mgrmap e25: x(active, since 56s), standbys: z, y 2024-05-09T04:52:03.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:02 smithi046 ceph-mon[30136]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-09T04:52:03.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:02 smithi046 ceph-mon[30136]: mgrmap e25: x(active, since 56s), standbys: z, y 2024-05-09T04:52:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:02 smithi072 ceph-mon[34088]: pgmap v31: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-09T04:52:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:02 smithi072 ceph-mon[34088]: mgrmap e25: x(active, since 56s), standbys: z, y 2024-05-09T04:52:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.156+0000 7f29e54f5700 1 -- 172.21.15.46:0/16436603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e01034c0 msgr2=0x7f29e01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.156+0000 7f29e54f5700 1 --2- 172.21.15.46:0/16436603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e01034c0 0x7f29e01038d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f29d00099c0 tx=0x7f29d0009cd0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.156+0000 7f29e54f5700 1 -- 172.21.15.46:0/16436603 shutdown_connections 2024-05-09T04:52:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.156+0000 7f29e54f5700 1 --2- 172.21.15.46:0/16436603 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e0108740 0x7f29e01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.156+0000 7f29e54f5700 1 --2- 172.21.15.46:0/16436603 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e01034c0 0x7f29e01038d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.156+0000 7f29e54f5700 1 --2- 172.21.15.46:0/16436603 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29e0107df0 0x7f29e0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.156+0000 7f29e54f5700 1 -- 172.21.15.46:0/16436603 >> 172.21.15.46:0/16436603 conn(0x7f29e00fedc0 msgr2=0x7f29e01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:04.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.156+0000 7f29e54f5700 1 -- 172.21.15.46:0/16436603 shutdown_connections 2024-05-09T04:52:04.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.156+0000 7f29e54f5700 1 -- 172.21.15.46:0/16436603 wait complete. 2024-05-09T04:52:04.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.157+0000 7f29e54f5700 1 Processor -- start 2024-05-09T04:52:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.157+0000 7f29e54f5700 1 -- start start 2024-05-09T04:52:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.157+0000 7f29e54f5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e01034c0 0x7f29e00ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.157+0000 7f29e54f5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e0107df0 0x7f29e00ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.157+0000 7f29e54f5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29e0108740 0x7f29e00ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.157+0000 7f29e54f5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29e0113370 con 0x7f29e01034c0 2024-05-09T04:52:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.157+0000 7f29e54f5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01131f0 con 0x7f29e0107df0 2024-05-09T04:52:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.157+0000 7f29e54f5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01134f0 con 0x7f29e0108740 2024-05-09T04:52:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.158+0000 7f29deffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e01034c0 0x7f29e00ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.158+0000 7f29de7fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e0107df0 0x7f29e00ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.158+0000 7f29de7fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e0107df0 0x7f29e00ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:39016/0 (socket says 172.21.15.46:39016) 2024-05-09T04:52:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.158+0000 7f29de7fc700 1 -- 172.21.15.46:0/321163976 learned_addr learned my addr 172.21.15.46:0/321163976 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.158+0000 7f29deffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e01034c0 0x7f29e00ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34386/0 (socket says 172.21.15.46:34386) 2024-05-09T04:52:04.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.158+0000 7f29df7fe700 1 --2- 172.21.15.46:0/321163976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29e0108740 0x7f29e00ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.158+0000 7f29deffd700 1 -- 172.21.15.46:0/321163976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29e0108740 msgr2=0x7f29e00ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:04.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.158+0000 7f29deffd700 1 --2- 172.21.15.46:0/321163976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29e0108740 0x7f29e00ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.158+0000 7f29deffd700 1 -- 172.21.15.46:0/321163976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e0107df0 msgr2=0x7f29e00ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:04.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.158+0000 7f29deffd700 1 --2- 172.21.15.46:0/321163976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e0107df0 0x7f29e00ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.158+0000 7f29deffd700 1 -- 172.21.15.46:0/321163976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e01a9150 con 0x7f29e01034c0 2024-05-09T04:52:04.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.158+0000 7f29de7fc700 1 --2- 172.21.15.46:0/321163976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e0107df0 0x7f29e00ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:04.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.158+0000 7f29deffd700 1 --2- 172.21.15.46:0/321163976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e01034c0 0x7f29e00ff2a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f29d00098e0 tx=0x7f29d00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:04.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.158+0000 7f29c7fff700 1 -- 172.21.15.46:0/321163976 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0026070 con 0x7f29e01034c0 2024-05-09T04:52:04.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.159+0000 7f29e54f5700 1 -- 172.21.15.46:0/321163976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e01a93e0 con 0x7f29e01034c0 2024-05-09T04:52:04.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.159+0000 7f29c7fff700 1 -- 172.21.15.46:0/321163976 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d000f460 con 0x7f29e01034c0 2024-05-09T04:52:04.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.159+0000 7f29c7fff700 1 -- 172.21.15.46:0/321163976 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d0009e60 con 0x7f29e01034c0 2024-05-09T04:52:04.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.159+0000 7f29e54f5700 1 -- 172.21.15.46:0/321163976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e01a9890 con 0x7f29e01034c0 2024-05-09T04:52:04.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.161+0000 7f29c7fff700 1 -- 172.21.15.46:0/321163976 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f29d0029070 con 0x7f29e01034c0 2024-05-09T04:52:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.162+0000 7f29c7fff700 1 --2- 172.21.15.46:0/321163976 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f29c80b2b90 0x7f29c80b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.162+0000 7f29de7fc700 1 --2- 172.21.15.46:0/321163976 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f29c80b2b90 0x7f29c80b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.162+0000 7f29c7fff700 1 -- 172.21.15.46:0/321163976 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f29d00e2fa0 con 0x7f29e01034c0 2024-05-09T04:52:04.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.163+0000 7f29de7fc700 1 --2- 172.21.15.46:0/321163976 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f29c80b2b90 0x7f29c80b50a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f29e01086b0 tx=0x7f29d400b480 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:04.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.164+0000 7f29f3f0bb80 1 Processor -- start 2024-05-09T04:52:04.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.164+0000 7f29f3f0bb80 1 -- start start 2024-05-09T04:52:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.164+0000 7f29f3f0bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 0x55be3e216240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.164+0000 7f29f3f0bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e0934a0 0x55be3e216780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.164+0000 7f29f3f0bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e210810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.164+0000 7f29f3f0bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55be3e1330a0 con 0x55be3e210400 2024-05-09T04:52:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.164+0000 7f29f3f0bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55be3e0d89a0 con 0x55be3e10dd00 2024-05-09T04:52:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.164+0000 7f29f3f0bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55be3e12f060 con 0x55be3e0934a0 2024-05-09T04:52:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.164+0000 7f29df7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e210810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.165+0000 7f29df7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e210810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34402/0 (socket says 172.21.15.46:34402) 2024-05-09T04:52:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.165+0000 7f29df7fe700 1 -- 172.21.15.46:0/1404462357 learned_addr learned my addr 172.21.15.46:0/1404462357 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:04.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.165+0000 7f29deffd700 1 --2- 172.21.15.46:0/1404462357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 0x55be3e216240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.165+0000 7f29de7fc700 1 --2- 172.21.15.46:0/1404462357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e0934a0 0x55be3e216780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.165+0000 7f29df7fe700 1 -- 172.21.15.46:0/1404462357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e0934a0 msgr2=0x55be3e216780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:04.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.165+0000 7f29df7fe700 1 --2- 172.21.15.46:0/1404462357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e0934a0 0x55be3e216780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.165+0000 7f29df7fe700 1 -- 172.21.15.46:0/1404462357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 msgr2=0x55be3e216240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:04.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.165+0000 7f29df7fe700 1 --2- 172.21.15.46:0/1404462357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 0x55be3e216240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.165+0000 7f29df7fe700 1 -- 172.21.15.46:0/1404462357 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3e0c9500 con 0x55be3e210400 2024-05-09T04:52:04.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.165+0000 7f29df7fe700 1 --2- 172.21.15.46:0/1404462357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e210810 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f29d8011680 tx=0x7f29d8009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17a882c211a23e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:04.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.165+0000 7f29c57fa700 1 -- 172.21.15.46:0/1404462357 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8019030 con 0x55be3e210400 2024-05-09T04:52:04.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.165+0000 7f29c57fa700 1 -- 172.21.15.46:0/1404462357 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d80092e0 con 0x55be3e210400 2024-05-09T04:52:04.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.165+0000 7f29c57fa700 1 -- 172.21.15.46:0/1404462357 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8007500 con 0x55be3e210400 2024-05-09T04:52:04.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.166+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/1404462357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 msgr2=0x55be3e210810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:04.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.166+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/1404462357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e210810 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f29d8011680 tx=0x7f29d8009920 comp rx=0 tx=0).stop 2024-05-09T04:52:04.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.166+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/1404462357 shutdown_connections 2024-05-09T04:52:04.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.166+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/1404462357 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e210810 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.166+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/1404462357 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 0x55be3e216240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.166+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/1404462357 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e0934a0 0x55be3e216780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.166+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/1404462357 >> 172.21.15.46:0/1404462357 conn(0x55be3e096f70 msgr2=0x55be3e20b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:04.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.167+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/1404462357 shutdown_connections 2024-05-09T04:52:04.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.167+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/1404462357 wait complete. 2024-05-09T04:52:04.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.167+0000 7f29f3f0bb80 1 -- start start 2024-05-09T04:52:04.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.167+0000 7f29f3f0bb80 1 Processor -- start 2024-05-09T04:52:04.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29f3f0bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e0934a0 0x55be3e21a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29f3f0bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 0x55be3e21a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29f3f0bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e210400 0x55be3e21ab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29f3f0bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55be3e211160 con 0x55be3e0934a0 2024-05-09T04:52:04.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29f3f0bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55be3e0cdb20 con 0x55be3e10dd00 2024-05-09T04:52:04.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29f3f0bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55be3e21d020 con 0x55be3e210400 2024-05-09T04:52:04.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29f3f0bb80 10 client.0 _ll_register_callbacks cb 0x55be3dfa46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29de7fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 0x55be3e21a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29de7fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 0x55be3e21a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:39036/0 (socket says 172.21.15.46:39036) 2024-05-09T04:52:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29df7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e210400 0x55be3e21ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29de7fc700 1 -- 172.21.15.46:0/4281554490 learned_addr learned my addr 172.21.15.46:0/4281554490 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29deffd700 1 --2- 172.21.15.46:0/4281554490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e0934a0 0x55be3e21a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29de7fc700 1 -- 172.21.15.46:0/4281554490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e210400 msgr2=0x55be3e21ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29de7fc700 1 --2- 172.21.15.46:0/4281554490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e210400 0x55be3e21ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29de7fc700 1 -- 172.21.15.46:0/4281554490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e0934a0 msgr2=0x55be3e21a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29de7fc700 1 --2- 172.21.15.46:0/4281554490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e0934a0 0x55be3e21a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29de7fc700 1 -- 172.21.15.46:0/4281554490 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be3dfae490 con 0x55be3e10dd00 2024-05-09T04:52:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29deffd700 1 --2- 172.21.15.46:0/4281554490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e0934a0 0x55be3e21a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:52:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29de7fc700 1 --2- 172.21.15.46:0/4281554490 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 0x55be3e21a610 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f29d4003930 tx=0x7f29d40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdaa5b0effdad5ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.168+0000 7f29c2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:52:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.169+0000 7f29c2ff5700 1 -- 172.21.15.46:0/4281554490 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d4024070 con 0x55be3e10dd00 2024-05-09T04:52:04.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.169+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/4281554490 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29d80092e0 con 0x55be3e10dd00 2024-05-09T04:52:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.169+0000 7f29c2ff5700 1 -- 172.21.15.46:0/4281554490 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f29d4011650 con 0x55be3e10dd00 2024-05-09T04:52:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.169+0000 7f29c2ff5700 1 -- 172.21.15.46:0/4281554490 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f29d4030050 con 0x55be3e10dd00 2024-05-09T04:52:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.169+0000 7f29f3f0bb80 10 client.25238 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.169+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/4281554490 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55be3e227b60 con 0x55be3e10dd00 2024-05-09T04:52:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.169+0000 7f29c2ff5700 1 -- 172.21.15.46:0/4281554490 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d80092e0 con 0x55be3e10dd00 2024-05-09T04:52:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.169+0000 7f29c2ff5700 1 -- 172.21.15.46:0/4281554490 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f29d4020850 con 0x55be3e10dd00 2024-05-09T04:52:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.169+0000 7f29f3f0bb80 10 client.25238 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.169+0000 7f29f3f0bb80 10 client.25238 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.169+0000 7f29f3f0bb80 20 client.25238 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.169+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/4281554490 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55be3e22a520 0x55be3e22c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.169+0000 7f29f3f0bb80 4 client.25238 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.169+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/4281554490 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55be3e140a80 con 0x55be3e22a520 2024-05-09T04:52:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29deffd700 1 --2- 172.21.15.46:0/4281554490 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55be3e22a520 0x55be3e22c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29deffd700 1 --2- 172.21.15.46:0/4281554490 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55be3e22a520 0x55be3e22c9d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bce14b8ce2cbccb1 server_cookie=2dd0df04a0807718 in_seq=0 out_seq=0 2024-05-09T04:52:04.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29c2ff5700 10 client.25238 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:04.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29c2ff5700 1 -- 172.21.15.46:0/4281554490 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55be3e140a80 con 0x55be3e22a520 2024-05-09T04:52:04.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29c2ff5700 10 client.25238 handle_command_reply: tid=0 2024-05-09T04:52:04.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29f3f0bb80 1 client.25238 shutdown 2024-05-09T04:52:04.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/4281554490 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 msgr2=0x55be3e21a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:04.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/4281554490 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 0x55be3e21a610 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f29d4003930 tx=0x7f29d40047c0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/4281554490 shutdown_connections 2024-05-09T04:52:04.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/4281554490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e210400 0x55be3e21ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/4281554490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e0934a0 0x55be3e21a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/4281554490 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55be3e22a520 0x55be3e22c9d0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/4281554490 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 0x55be3e21a610 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/4281554490 >> 172.21.15.46:0/4281554490 conn(0x55be3e096f70 msgr2=0x55be3e204ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:04.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29c2ff5700 0 client.25238 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:04.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/4281554490 shutdown_connections 2024-05-09T04:52:04.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.171+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/4281554490 wait complete. 2024-05-09T04:52:04.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.172+0000 7f29f3f0bb80 20 client.25238 trim_cache size 0 max 16384 2024-05-09T04:52:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.187+0000 7f29f3f0bb80 1 Processor -- start 2024-05-09T04:52:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.187+0000 7f29f3f0bb80 1 -- start start 2024-05-09T04:52:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.187+0000 7f29f3f0bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e0934a0 0x55be3e217dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.187+0000 7f29f3f0bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 0x55be3e225b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.187+0000 7f29f3f0bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e204ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.187+0000 7f29f3f0bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01b3930 con 0x55be3e210400 2024-05-09T04:52:04.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.187+0000 7f29f3f0bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01b11c0 con 0x55be3e10dd00 2024-05-09T04:52:04.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.187+0000 7f29f3f0bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01af010 con 0x55be3e0934a0 2024-05-09T04:52:04.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.188+0000 7f29df7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e204ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.188+0000 7f29df7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e204ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34416/0 (socket says 172.21.15.46:34416) 2024-05-09T04:52:04.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.188+0000 7f29df7fe700 1 -- 172.21.15.46:0/1765996103 learned_addr learned my addr 172.21.15.46:0/1765996103 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:04.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.188+0000 7f29deffd700 1 --2- 172.21.15.46:0/1765996103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e0934a0 0x55be3e217dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.188+0000 7f29de7fc700 1 --2- 172.21.15.46:0/1765996103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 0x55be3e225b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.188+0000 7f29df7fe700 1 -- 172.21.15.46:0/1765996103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e0934a0 msgr2=0x55be3e217dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:04.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.188+0000 7f29df7fe700 1 --2- 172.21.15.46:0/1765996103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e0934a0 0x55be3e217dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.188+0000 7f29df7fe700 1 -- 172.21.15.46:0/1765996103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 msgr2=0x55be3e225b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:04.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.188+0000 7f29df7fe700 1 --2- 172.21.15.46:0/1765996103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 0x55be3e225b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.188+0000 7f29df7fe700 1 -- 172.21.15.46:0/1765996103 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55be3e2079d0 con 0x55be3e210400 2024-05-09T04:52:04.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.188+0000 7f29df7fe700 1 --2- 172.21.15.46:0/1765996103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e204ff0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f29d8011680 tx=0x7f29d8009470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a7f715edfdabf4b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:04.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.188+0000 7f29c4ff9700 1 -- 172.21.15.46:0/1765996103 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8003c00 con 0x55be3e210400 2024-05-09T04:52:04.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.188+0000 7f29c4ff9700 1 -- 172.21.15.46:0/1765996103 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d8004250 con 0x55be3e210400 2024-05-09T04:52:04.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.188+0000 7f29c4ff9700 1 -- 172.21.15.46:0/1765996103 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d80044f0 con 0x55be3e210400 2024-05-09T04:52:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.189+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/1765996103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 msgr2=0x55be3e204ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.189+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/1765996103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e204ff0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f29d8011680 tx=0x7f29d8009470 comp rx=0 tx=0).stop 2024-05-09T04:52:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.189+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/1765996103 shutdown_connections 2024-05-09T04:52:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.189+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/1765996103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e204ff0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.189+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/1765996103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e0934a0 0x55be3e217dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.189+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/1765996103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e10dd00 0x55be3e225b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.189+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/1765996103 >> 172.21.15.46:0/1765996103 conn(0x55be3e096f70 msgr2=0x55be3e2189c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.189+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/1765996103 shutdown_connections 2024-05-09T04:52:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.190+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/1765996103 wait complete. 2024-05-09T04:52:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.190+0000 7f29f3f0bb80 1 -- start start 2024-05-09T04:52:04.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.190+0000 7f29f3f0bb80 1 Processor -- start 2024-05-09T04:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.190+0000 7f29f3f0bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e0934a0 0x55be3e222380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.190+0000 7f29f3f0bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e10dd00 0x55be3e216dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.190+0000 7f29f3f0bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e217300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29f3f0bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01af680 con 0x55be3e210400 2024-05-09T04:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29f3f0bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01b0af0 con 0x55be3e0934a0 2024-05-09T04:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29f3f0bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f29e01afd50 con 0x55be3e10dd00 2024-05-09T04:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29f3f0bb80 10 client.0 _ll_register_callbacks cb 0x55be3e210dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29df7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e217300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29deffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e0934a0 0x55be3e222380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29df7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e217300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34418/0 (socket says 172.21.15.46:34418) 2024-05-09T04:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29df7fe700 1 -- 172.21.15.46:0/3754433367 learned_addr learned my addr 172.21.15.46:0/3754433367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29de7fc700 1 --2- 172.21.15.46:0/3754433367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e10dd00 0x55be3e216dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29df7fe700 1 -- 172.21.15.46:0/3754433367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e10dd00 msgr2=0x55be3e216dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29df7fe700 1 --2- 172.21.15.46:0/3754433367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e10dd00 0x55be3e216dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29df7fe700 1 -- 172.21.15.46:0/3754433367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e0934a0 msgr2=0x55be3e222380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29df7fe700 1 --2- 172.21.15.46:0/3754433367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e0934a0 0x55be3e222380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29df7fe700 1 -- 172.21.15.46:0/3754433367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55be3e211600 con 0x55be3e210400 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29deffd700 1 --2- 172.21.15.46:0/3754433367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e0934a0 0x55be3e222380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29df7fe700 1 --2- 172.21.15.46:0/3754433367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e217300 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009940 tx=0x7f29d8008fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2631a59a75bc3e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29c2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29c2ff5700 1 -- 172.21.15.46:0/3754433367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29d8003e10 con 0x55be3e210400 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29c2ff5700 1 -- 172.21.15.46:0/3754433367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f29d8003fb0 con 0x55be3e210400 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.191+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/3754433367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29d8004250 con 0x55be3e210400 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.192+0000 7f29c2ff5700 1 -- 172.21.15.46:0/3754433367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f29d8039050 con 0x55be3e210400 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.192+0000 7f29f3f0bb80 10 client.15480 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.192+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/3754433367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55be3e294940 con 0x55be3e210400 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.192+0000 7f29c2ff5700 1 -- 172.21.15.46:0/3754433367 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29d8004250 con 0x55be3e210400 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.192+0000 7f29c2ff5700 1 -- 172.21.15.46:0/3754433367 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f29d8015450 con 0x55be3e210400 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.192+0000 7f29f3f0bb80 10 client.15480 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.192+0000 7f29f3f0bb80 10 client.15480 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:04.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.192+0000 7f29f3f0bb80 20 client.15480 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.192+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/3754433367 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55be3e294de0 0x55be3e297200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.192+0000 7f29f3f0bb80 4 client.15480 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.192+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/3754433367 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55be3e13ab20 con 0x55be3e294de0 2024-05-09T04:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.192+0000 7f29deffd700 1 --2- 172.21.15.46:0/3754433367 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55be3e294de0 0x55be3e297200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.193+0000 7f29deffd700 1 --2- 172.21.15.46:0/3754433367 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55be3e294de0 0x55be3e297200 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcc79e26b7a47b1b server_cookie=b71a36a757909ad in_seq=0 out_seq=0 2024-05-09T04:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.193+0000 7f29c2ff5700 10 client.15480 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.193+0000 7f29c2ff5700 1 -- 172.21.15.46:0/3754433367 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55be3e13ab20 con 0x55be3e294de0 2024-05-09T04:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.193+0000 7f29c2ff5700 10 client.15480 handle_command_reply: tid=0 2024-05-09T04:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.193+0000 7f29f3f0bb80 1 client.15480 shutdown 2024-05-09T04:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.193+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/3754433367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 msgr2=0x55be3e217300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.193+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/3754433367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e217300 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f29d8009940 tx=0x7f29d8008fd0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.193+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/3754433367 shutdown_connections 2024-05-09T04:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.193+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/3754433367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55be3e210400 0x55be3e217300 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.193+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/3754433367 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55be3e294de0 0x55be3e297200 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.193+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/3754433367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55be3e0934a0 0x55be3e222380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.193+0000 7f29c2ff5700 0 client.15480 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.193+0000 7f29f3f0bb80 1 --2- 172.21.15.46:0/3754433367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55be3e10dd00 0x55be3e216dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.193+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/3754433367 >> 172.21.15.46:0/3754433367 conn(0x55be3e096f70 msgr2=0x55be3e213cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.194+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/3754433367 shutdown_connections 2024-05-09T04:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.194+0000 7f29f3f0bb80 1 -- 172.21.15.46:0/3754433367 wait complete. 2024-05-09T04:52:04.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.194+0000 7f29f3f0bb80 20 client.15480 trim_cache size 0 max 16384 2024-05-09T04:52:04.199 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:52:04.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.194+0000 7f29e54f5700 1 -- 172.21.15.46:0/321163976 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f29c80b2b90 msgr2=0x7f29c80b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.194+0000 7f29e54f5700 1 --2- 172.21.15.46:0/321163976 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f29c80b2b90 0x7f29c80b50a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f29e01086b0 tx=0x7f29d400b480 comp rx=0 tx=0).stop 2024-05-09T04:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.194+0000 7f29e54f5700 1 -- 172.21.15.46:0/321163976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e01034c0 msgr2=0x7f29e00ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.194+0000 7f29e54f5700 1 --2- 172.21.15.46:0/321163976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e01034c0 0x7f29e00ff2a0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f29d00098e0 tx=0x7f29d00045e0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.194+0000 7f29e54f5700 1 -- 172.21.15.46:0/321163976 shutdown_connections 2024-05-09T04:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.194+0000 7f29e54f5700 1 --2- 172.21.15.46:0/321163976 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f29c80b2b90 0x7f29c80b50a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.194+0000 7f29e54f5700 1 --2- 172.21.15.46:0/321163976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f29e0108740 0x7f29e00ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.194+0000 7f29e54f5700 1 --2- 172.21.15.46:0/321163976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f29e01034c0 0x7f29e00ff2a0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.194+0000 7f29e54f5700 1 --2- 172.21.15.46:0/321163976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f29e0107df0 0x7f29e00ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.194+0000 7f29e54f5700 1 -- 172.21.15.46:0/321163976 >> 172.21.15.46:0/321163976 conn(0x7f29e00fedc0 msgr2=0x7f29e006b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.195+0000 7f29e54f5700 1 -- 172.21.15.46:0/321163976 shutdown_connections 2024-05-09T04:52:04.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:04.195+0000 7f29e54f5700 1 -- 172.21.15.46:0/321163976 wait complete. 2024-05-09T04:52:04.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:04 smithi046 ceph-mon[30136]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-09T04:52:05.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:04 smithi110 ceph-mon[34904]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-09T04:52:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:04 smithi072 ceph-mon[34088]: pgmap v32: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-09T04:52:06.008 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:52:06.008 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:52:06.261 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:07.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:06 smithi110 ceph-mon[34904]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-09T04:52:07.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:06 smithi046 ceph-mon[30136]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-09T04:52:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:06 smithi072 ceph-mon[34088]: pgmap v33: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 22 op/s 2024-05-09T04:52:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.060+0000 7fea97cb5700 1 -- 172.21.15.46:0/4276377180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea90105e30 msgr2=0x7fea9010e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.060+0000 7fea97cb5700 1 --2- 172.21.15.46:0/4276377180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea90105e30 0x7fea9010e380 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fea80009990 tx=0x7fea80009ca0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.060+0000 7fea97cb5700 1 -- 172.21.15.46:0/4276377180 shutdown_connections 2024-05-09T04:52:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.060+0000 7fea97cb5700 1 --2- 172.21.15.46:0/4276377180 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea9010e910 0x7fea90110df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.060+0000 7fea97cb5700 1 --2- 172.21.15.46:0/4276377180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea901054e0 0x7fea901058f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.060+0000 7fea97cb5700 1 --2- 172.21.15.46:0/4276377180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea90105e30 0x7fea9010e380 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.060+0000 7fea97cb5700 1 -- 172.21.15.46:0/4276377180 >> 172.21.15.46:0/4276377180 conn(0x7fea900fedc0 msgr2=0x7fea901011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.060+0000 7fea97cb5700 1 -- 172.21.15.46:0/4276377180 shutdown_connections 2024-05-09T04:52:08.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.060+0000 7fea97cb5700 1 -- 172.21.15.46:0/4276377180 wait complete. 2024-05-09T04:52:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.061+0000 7fea97cb5700 1 Processor -- start 2024-05-09T04:52:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.061+0000 7fea97cb5700 1 -- start start 2024-05-09T04:52:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.061+0000 7fea97cb5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea901054e0 0x7fea901031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.061+0000 7fea97cb5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea90105e30 0x7fea90103710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.061+0000 7fea97cb5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9010e910 0x7fea90103c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.061+0000 7fea97cb5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fea90114440 con 0x7fea9010e910 2024-05-09T04:52:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.061+0000 7fea97cb5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fea901142c0 con 0x7fea901054e0 2024-05-09T04:52:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.061+0000 7fea97cb5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea901145c0 con 0x7fea90105e30 2024-05-09T04:52:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.062+0000 7fea95250700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea90105e30 0x7fea90103710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.062+0000 7fea95250700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea90105e30 0x7fea90103710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:55626/0 (socket says 172.21.15.46:55626) 2024-05-09T04:52:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.062+0000 7fea96252700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9010e910 0x7fea90103c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.062+0000 7fea95250700 1 -- 172.21.15.46:0/4029849767 learned_addr learned my addr 172.21.15.46:0/4029849767 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:08.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.062+0000 7fea95a51700 1 --2- 172.21.15.46:0/4029849767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea901054e0 0x7fea901031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.062+0000 7fea95250700 1 -- 172.21.15.46:0/4029849767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea901054e0 msgr2=0x7fea901031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.062+0000 7fea95250700 1 --2- 172.21.15.46:0/4029849767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea901054e0 0x7fea901031d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.062+0000 7fea95250700 1 -- 172.21.15.46:0/4029849767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9010e910 msgr2=0x7fea90103c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.062+0000 7fea95250700 1 --2- 172.21.15.46:0/4029849767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9010e910 0x7fea90103c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.062+0000 7fea95250700 1 -- 172.21.15.46:0/4029849767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea901a9150 con 0x7fea90105e30 2024-05-09T04:52:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.062+0000 7fea95250700 1 --2- 172.21.15.46:0/4029849767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea90105e30 0x7fea90103710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fea80000c00 tx=0x7fea8000bfd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.063+0000 7fea86ffd700 1 -- 172.21.15.46:0/4029849767 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80027070 con 0x7fea90105e30 2024-05-09T04:52:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.063+0000 7fea97cb5700 1 -- 172.21.15.46:0/4029849767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea901a93e0 con 0x7fea90105e30 2024-05-09T04:52:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.063+0000 7fea86ffd700 1 -- 172.21.15.46:0/4029849767 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea80022470 con 0x7fea90105e30 2024-05-09T04:52:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.063+0000 7fea86ffd700 1 -- 172.21.15.46:0/4029849767 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea80017650 con 0x7fea90105e30 2024-05-09T04:52:08.064 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.063+0000 7fea97cb5700 1 -- 172.21.15.46:0/4029849767 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea901a9910 con 0x7fea90105e30 2024-05-09T04:52:08.066 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.066+0000 7fea86ffd700 1 -- 172.21.15.46:0/4029849767 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fea8002a070 con 0x7fea90105e30 2024-05-09T04:52:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.067+0000 7fea86ffd700 1 --2- 172.21.15.46:0/4029849767 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fea7c0b2b90 0x7fea7c0b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.067+0000 7fea95a51700 1 --2- 172.21.15.46:0/4029849767 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fea7c0b2b90 0x7fea7c0b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.067+0000 7fea86ffd700 1 -- 172.21.15.46:0/4029849767 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fea800e3510 con 0x7fea90105e30 2024-05-09T04:52:08.067 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.068+0000 7feaa66cbb80 1 Processor -- start 2024-05-09T04:52:08.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.068+0000 7feaa66cbb80 1 -- start start 2024-05-09T04:52:08.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.068+0000 7feaa66cbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603cea851e0 0x5603ceb1e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.068+0000 7fea95a51700 1 --2- 172.21.15.46:0/4029849767 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fea7c0b2b90 0x7fea7c0b50a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009c60 tx=0x7fea8c009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:08.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.068+0000 7feaa66cbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ce99b520 0x5603ceb1eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.068+0000 7feaa66cbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603ceb18720 0x5603ceb18b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.068+0000 7feaa66cbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5603cea3b070 con 0x5603cea851e0 2024-05-09T04:52:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.068+0000 7feaa66cbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5603ce9d9870 con 0x5603ce99b520 2024-05-09T04:52:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.068+0000 7feaa66cbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5603cea37030 con 0x5603ceb18720 2024-05-09T04:52:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.068+0000 7fea95a51700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603cea851e0 0x5603ceb1e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.068+0000 7fea95a51700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603cea851e0 0x5603ceb1e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34448/0 (socket says 172.21.15.46:34448) 2024-05-09T04:52:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.068+0000 7fea95a51700 1 -- 172.21.15.46:0/4119612022 learned_addr learned my addr 172.21.15.46:0/4119612022 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.068+0000 7fea95250700 1 --2- 172.21.15.46:0/4119612022 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ce99b520 0x5603ceb1eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.068+0000 7fea96252700 1 --2- 172.21.15.46:0/4119612022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603ceb18720 0x5603ceb18b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.069+0000 7fea95250700 1 -- 172.21.15.46:0/4119612022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603ceb18720 msgr2=0x5603ceb18b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.069+0000 7fea95250700 1 --2- 172.21.15.46:0/4119612022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603ceb18720 0x5603ceb18b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.069+0000 7fea95250700 1 -- 172.21.15.46:0/4119612022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603cea851e0 msgr2=0x5603ceb1e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.069+0000 7fea95250700 1 --2- 172.21.15.46:0/4119612022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603cea851e0 0x5603ceb1e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.069+0000 7fea95250700 1 -- 172.21.15.46:0/4119612022 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ce9d13f0 con 0x5603ce99b520 2024-05-09T04:52:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.069+0000 7fea95a51700 1 --2- 172.21.15.46:0/4119612022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603cea851e0 0x5603ceb1e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.069+0000 7fea95250700 1 --2- 172.21.15.46:0/4119612022 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ce99b520 0x5603ceb1eab0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fea800e0ac0 tx=0x7fea800a9e00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d969134f9d467e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.069+0000 7fea7bfff700 1 -- 172.21.15.46:0/4119612022 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800aa610 con 0x5603ce99b520 2024-05-09T04:52:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.069+0000 7fea7bfff700 1 -- 172.21.15.46:0/4119612022 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea800ae9c0 con 0x5603ce99b520 2024-05-09T04:52:08.071 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.069+0000 7fea7bfff700 1 -- 172.21.15.46:0/4119612022 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea800b31b0 con 0x5603ce99b520 2024-05-09T04:52:08.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.070+0000 7feaa66cbb80 1 -- 172.21.15.46:0/4119612022 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ce99b520 msgr2=0x5603ceb1eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:08.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.070+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/4119612022 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ce99b520 0x5603ceb1eab0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fea800e0ac0 tx=0x7fea800a9e00 comp rx=0 tx=0).stop 2024-05-09T04:52:08.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.070+0000 7feaa66cbb80 1 -- 172.21.15.46:0/4119612022 shutdown_connections 2024-05-09T04:52:08.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.070+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/4119612022 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603ceb18720 0x5603ceb18b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.070+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/4119612022 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603cea851e0 0x5603ceb1e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.070+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/4119612022 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ce99b520 0x5603ceb1eab0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.070+0000 7feaa66cbb80 1 -- 172.21.15.46:0/4119612022 >> 172.21.15.46:0/4119612022 conn(0x5603ce9fb720 msgr2=0x5603ceb13480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:08.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.070+0000 7feaa66cbb80 1 -- 172.21.15.46:0/4119612022 shutdown_connections 2024-05-09T04:52:08.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.070+0000 7feaa66cbb80 1 -- 172.21.15.46:0/4119612022 wait complete. 2024-05-09T04:52:08.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.071+0000 7feaa66cbb80 1 -- start start 2024-05-09T04:52:08.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.071+0000 7feaa66cbb80 1 Processor -- start 2024-05-09T04:52:08.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.071+0000 7feaa66cbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce99b520 0x5603ceb27ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.071+0000 7feaa66cbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603cea851e0 0x5603ceb28220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.071+0000 7feaa66cbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ceb18720 0x5603ceb220e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7feaa66cbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5603ce8b6c20 con 0x5603ce99b520 2024-05-09T04:52:08.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7feaa66cbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5603ce9f51d0 con 0x5603ceb18720 2024-05-09T04:52:08.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7feaa66cbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5603ceb193e0 con 0x5603cea851e0 2024-05-09T04:52:08.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7feaa66cbb80 10 client.0 _ll_register_callbacks cb 0x5603ce8ac740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:08.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7fea95a51700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce99b520 0x5603ceb27ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7fea95a51700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce99b520 0x5603ceb27ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34460/0 (socket says 172.21.15.46:34460) 2024-05-09T04:52:08.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7fea95a51700 1 -- 172.21.15.46:0/1434970875 learned_addr learned my addr 172.21.15.46:0/1434970875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:08.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7fea96252700 1 --2- 172.21.15.46:0/1434970875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ceb18720 0x5603ceb220e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7fea95250700 1 --2- 172.21.15.46:0/1434970875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603cea851e0 0x5603ceb28220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7fea95a51700 1 -- 172.21.15.46:0/1434970875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603cea851e0 msgr2=0x5603ceb28220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:08.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7fea95a51700 1 --2- 172.21.15.46:0/1434970875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603cea851e0 0x5603ceb28220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7fea95a51700 1 -- 172.21.15.46:0/1434970875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ceb18720 msgr2=0x5603ceb220e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:08.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7fea95a51700 1 --2- 172.21.15.46:0/1434970875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ceb18720 0x5603ceb220e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7fea95a51700 1 -- 172.21.15.46:0/1434970875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603ce8b6510 con 0x5603ce99b520 2024-05-09T04:52:08.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7fea96252700 1 --2- 172.21.15.46:0/1434970875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ceb18720 0x5603ceb220e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:08.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7fea95a51700 1 --2- 172.21.15.46:0/1434970875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce99b520 0x5603ceb27ce0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fea8c005c50 tx=0x7fea8c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69540ef0761ca26 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:08.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.072+0000 7fea797fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:08.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.073+0000 7fea797fa700 1 -- 172.21.15.46:0/1434970875 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c01d620 con 0x5603ce99b520 2024-05-09T04:52:08.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.073+0000 7fea797fa700 1 -- 172.21.15.46:0/1434970875 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fea8c01d7c0 con 0x5603ce99b520 2024-05-09T04:52:08.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.073+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1434970875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea800ae9c0 con 0x5603ce99b520 2024-05-09T04:52:08.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.073+0000 7fea797fa700 1 -- 172.21.15.46:0/1434970875 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea8c01d9c0 con 0x5603ce99b520 2024-05-09T04:52:08.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.073+0000 7feaa66cbb80 10 client.15492 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:08.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.073+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1434970875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5603ceb2f960 con 0x5603ce99b520 2024-05-09T04:52:08.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.073+0000 7fea797fa700 1 -- 172.21.15.46:0/1434970875 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea800ae9c0 con 0x5603ce99b520 2024-05-09T04:52:08.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.073+0000 7fea797fa700 1 -- 172.21.15.46:0/1434970875 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fea8c02fbd0 con 0x5603ce99b520 2024-05-09T04:52:08.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.073+0000 7feaa66cbb80 10 client.15492 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:08.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.073+0000 7feaa66cbb80 10 client.15492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:08.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.073+0000 7feaa66cbb80 20 client.15492 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:08.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.073+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/1434970875 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5603ceb31fa0 0x5603ceb34450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.074+0000 7feaa66cbb80 4 client.15492 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:08.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.074+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1434970875 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5603cea48a50 con 0x5603ceb31fa0 2024-05-09T04:52:08.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.074+0000 7fea95250700 1 --2- 172.21.15.46:0/1434970875 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5603ceb31fa0 0x5603ceb34450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.074+0000 7fea95250700 1 --2- 172.21.15.46:0/1434970875 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5603ceb31fa0 0x5603ceb34450 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5983aad5407c04a2 server_cookie=4742102c6760c20e in_seq=0 out_seq=0 2024-05-09T04:52:08.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.074+0000 7fea797fa700 10 client.15492 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:08.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.075+0000 7fea797fa700 1 -- 172.21.15.46:0/1434970875 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5603cea48a50 con 0x5603ceb31fa0 2024-05-09T04:52:08.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.075+0000 7fea797fa700 10 client.15492 handle_command_reply: tid=0 2024-05-09T04:52:08.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.075+0000 7feaa66cbb80 1 client.15492 shutdown 2024-05-09T04:52:08.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.075+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1434970875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce99b520 msgr2=0x5603ceb27ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:08.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.075+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/1434970875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce99b520 0x5603ceb27ce0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fea8c005c50 tx=0x7fea8c0047c0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.075+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1434970875 shutdown_connections 2024-05-09T04:52:08.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.075+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/1434970875 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5603ceb31fa0 0x5603ceb34450 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.075+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/1434970875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ceb18720 0x5603ceb220e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.075+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/1434970875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce99b520 0x5603ceb27ce0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.075+0000 7fea797fa700 0 client.15492 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:08.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.075+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/1434970875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603cea851e0 0x5603ceb28220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.075+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1434970875 >> 172.21.15.46:0/1434970875 conn(0x5603ce9fb720 msgr2=0x5603ceb13240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:08.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.075+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1434970875 shutdown_connections 2024-05-09T04:52:08.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.075+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1434970875 wait complete. 2024-05-09T04:52:08.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.076+0000 7feaa66cbb80 20 client.15492 trim_cache size 0 max 16384 2024-05-09T04:52:08.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.098+0000 7feaa66cbb80 1 Processor -- start 2024-05-09T04:52:08.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.098+0000 7feaa66cbb80 1 -- start start 2024-05-09T04:52:08.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7feaa66cbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce9fb720 0x5603ceb1c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7feaa66cbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603ce99b520 0x5603ceb1ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7feaa66cbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603cea851e0 0x5603ceb20fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7feaa66cbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b39d0 con 0x5603ce9fb720 2024-05-09T04:52:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7feaa66cbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b1260 con 0x5603cea851e0 2024-05-09T04:52:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7feaa66cbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea901af0b0 con 0x5603ce99b520 2024-05-09T04:52:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7fea95a51700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce9fb720 0x5603ceb1c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7fea95a51700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce9fb720 0x5603ceb1c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34466/0 (socket says 172.21.15.46:34466) 2024-05-09T04:52:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7fea95a51700 1 -- 172.21.15.46:0/23158288 learned_addr learned my addr 172.21.15.46:0/23158288 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7fea96252700 1 --2- 172.21.15.46:0/23158288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603cea851e0 0x5603ceb20fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7fea95250700 1 --2- 172.21.15.46:0/23158288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603ce99b520 0x5603ceb1ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7fea95a51700 1 -- 172.21.15.46:0/23158288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603ce99b520 msgr2=0x5603ceb1ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:08.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7fea95a51700 1 --2- 172.21.15.46:0/23158288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603ce99b520 0x5603ceb1ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7fea95a51700 1 -- 172.21.15.46:0/23158288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603cea851e0 msgr2=0x5603ceb20fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:08.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7fea95a51700 1 --2- 172.21.15.46:0/23158288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603cea851e0 0x5603ceb20fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7fea95a51700 1 -- 172.21.15.46:0/23158288 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603ceb0cfe0 con 0x5603ce9fb720 2024-05-09T04:52:08.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.099+0000 7fea95a51700 1 --2- 172.21.15.46:0/23158288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce9fb720 0x5603ceb1c7a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0275b0 tx=0x7fea8c027840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73d457c2244b6322 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:08.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.100+0000 7fea7b7fe700 1 -- 172.21.15.46:0/23158288 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c01d620 con 0x5603ce9fb720 2024-05-09T04:52:08.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.100+0000 7fea7b7fe700 1 -- 172.21.15.46:0/23158288 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea8c027e10 con 0x5603ce9fb720 2024-05-09T04:52:08.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.100+0000 7fea7b7fe700 1 -- 172.21.15.46:0/23158288 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c02f710 con 0x5603ce9fb720 2024-05-09T04:52:08.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.101+0000 7feaa66cbb80 1 -- 172.21.15.46:0/23158288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce9fb720 msgr2=0x5603ceb1c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:08.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.101+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/23158288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce9fb720 0x5603ceb1c7a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0275b0 tx=0x7fea8c027840 comp rx=0 tx=0).stop 2024-05-09T04:52:08.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.101+0000 7feaa66cbb80 1 -- 172.21.15.46:0/23158288 shutdown_connections 2024-05-09T04:52:08.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.101+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/23158288 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603cea851e0 0x5603ceb20fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.101+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/23158288 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce9fb720 0x5603ceb1c7a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.101+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/23158288 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603ce99b520 0x5603ceb1ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.101+0000 7feaa66cbb80 1 -- 172.21.15.46:0/23158288 >> 172.21.15.46:0/23158288 conn(0x5603ceb1ed30 msgr2=0x5603ceb208c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.101+0000 7feaa66cbb80 1 -- 172.21.15.46:0/23158288 shutdown_connections 2024-05-09T04:52:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.101+0000 7feaa66cbb80 1 -- 172.21.15.46:0/23158288 wait complete. 2024-05-09T04:52:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.102+0000 7feaa66cbb80 1 -- start start 2024-05-09T04:52:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.102+0000 7feaa66cbb80 1 Processor -- start 2024-05-09T04:52:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.102+0000 7feaa66cbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce99b520 0x5603ceb2e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.102+0000 7feaa66cbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ce9fb720 0x5603ceb29ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.102+0000 7feaa66cbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603cea851e0 0x5603ceb2a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.102+0000 7feaa66cbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fea901af720 con 0x5603ce99b520 2024-05-09T04:52:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.102+0000 7feaa66cbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fea901b0b90 con 0x5603ce9fb720 2024-05-09T04:52:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.102+0000 7feaa66cbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea901afdf0 con 0x5603cea851e0 2024-05-09T04:52:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.102+0000 7feaa66cbb80 10 client.0 _ll_register_callbacks cb 0x5603ce8ac740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:08.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.102+0000 7fea95a51700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce99b520 0x5603ceb2e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.102+0000 7fea95a51700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce99b520 0x5603ceb2e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34478/0 (socket says 172.21.15.46:34478) 2024-05-09T04:52:08.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.102+0000 7fea95a51700 1 -- 172.21.15.46:0/1954467930 learned_addr learned my addr 172.21.15.46:0/1954467930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:08.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.103+0000 7fea95250700 1 --2- 172.21.15.46:0/1954467930 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ce9fb720 0x5603ceb29ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.103+0000 7fea96252700 1 --2- 172.21.15.46:0/1954467930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603cea851e0 0x5603ceb2a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.103+0000 7fea95a51700 1 -- 172.21.15.46:0/1954467930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603cea851e0 msgr2=0x5603ceb2a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:08.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.103+0000 7fea95a51700 1 --2- 172.21.15.46:0/1954467930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603cea851e0 0x5603ceb2a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.103+0000 7fea95a51700 1 -- 172.21.15.46:0/1954467930 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ce9fb720 msgr2=0x5603ceb29ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:08.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.103+0000 7fea95a51700 1 --2- 172.21.15.46:0/1954467930 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ce9fb720 0x5603ceb29ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.103+0000 7fea95a51700 1 -- 172.21.15.46:0/1954467930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603ceb2a9c0 con 0x5603ce99b520 2024-05-09T04:52:08.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.103+0000 7fea95a51700 1 --2- 172.21.15.46:0/1954467930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce99b520 0x5603ceb2e840 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0274e0 tx=0x7fea8c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76998da25bdeda5f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:08.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.103+0000 7fea797fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:08.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.103+0000 7fea797fa700 1 -- 172.21.15.46:0/1954467930 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea8c01d720 con 0x5603ce99b520 2024-05-09T04:52:08.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.103+0000 7fea797fa700 1 -- 172.21.15.46:0/1954467930 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fea8c027960 con 0x5603ce99b520 2024-05-09T04:52:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.103+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1954467930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea8c027e10 con 0x5603ce99b520 2024-05-09T04:52:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.103+0000 7fea797fa700 1 -- 172.21.15.46:0/1954467930 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea8c027b60 con 0x5603ce99b520 2024-05-09T04:52:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.103+0000 7feaa66cbb80 10 client.15498 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.103+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1954467930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5603ceb9c8c0 con 0x5603ce99b520 2024-05-09T04:52:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.104+0000 7fea797fa700 1 -- 172.21.15.46:0/1954467930 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea8c027e10 con 0x5603ce99b520 2024-05-09T04:52:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.104+0000 7fea797fa700 1 -- 172.21.15.46:0/1954467930 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fea8c004a00 con 0x5603ce99b520 2024-05-09T04:52:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.104+0000 7feaa66cbb80 10 client.15498 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.104+0000 7feaa66cbb80 10 client.15498 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.104+0000 7feaa66cbb80 20 client.15498 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.104+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/1954467930 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5603ceb18720 0x5603ceb9ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.104+0000 7feaa66cbb80 4 client.15498 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:52:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.104+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1954467930 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5603cea42af0 con 0x5603ceb18720 2024-05-09T04:52:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.104+0000 7fea95250700 1 --2- 172.21.15.46:0/1954467930 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5603ceb18720 0x5603ceb9ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:08.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.105+0000 7fea797fa700 10 client.15498 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:08.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.105+0000 7fea95250700 1 --2- 172.21.15.46:0/1954467930 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5603ceb18720 0x5603ceb9ef30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ad7edd5611f1bc03 server_cookie=f4423a447f1565 in_seq=0 out_seq=0 2024-05-09T04:52:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.106+0000 7fea797fa700 1 -- 172.21.15.46:0/1954467930 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5603cea42af0 con 0x5603ceb18720 2024-05-09T04:52:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.106+0000 7fea797fa700 10 client.15498 handle_command_reply: tid=0 2024-05-09T04:52:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.106+0000 7feaa66cbb80 1 client.15498 shutdown 2024-05-09T04:52:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.106+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1954467930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce99b520 msgr2=0x5603ceb2e840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.106+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/1954467930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce99b520 0x5603ceb2e840 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fea8c0274e0 tx=0x7fea8c014000 comp rx=0 tx=0).stop 2024-05-09T04:52:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.106+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1954467930 shutdown_connections 2024-05-09T04:52:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.106+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/1954467930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5603cea851e0 0x5603ceb2a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.106+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/1954467930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5603ce99b520 0x5603ceb2e840 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.106+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/1954467930 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5603ceb18720 0x5603ceb9ef30 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.106+0000 7feaa66cbb80 1 --2- 172.21.15.46:0/1954467930 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5603ce9fb720 0x5603ceb29ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.106+0000 7fea797fa700 0 client.15498 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.106+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1954467930 >> 172.21.15.46:0/1954467930 conn(0x5603ceb1ed30 msgr2=0x5603ceb1d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.106+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1954467930 shutdown_connections 2024-05-09T04:52:08.110 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.106+0000 7feaa66cbb80 1 -- 172.21.15.46:0/1954467930 wait complete. 2024-05-09T04:52:08.111 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.107+0000 7feaa66cbb80 20 client.15498 trim_cache size 0 max 16384 2024-05-09T04:52:08.111 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:52:08.111 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:52:08.111 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "fc6a1f86-83cf-477e-8e5d-eee67f306fda", 2024-05-09T04:52:08.111 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:52:08.111 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:52:08.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.107+0000 7fea97cb5700 1 -- 172.21.15.46:0/4029849767 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fea7c0b2b90 msgr2=0x7fea7c0b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:08.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.107+0000 7fea97cb5700 1 --2- 172.21.15.46:0/4029849767 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fea7c0b2b90 0x7fea7c0b50a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fea8c009c60 tx=0x7fea8c009260 comp rx=0 tx=0).stop 2024-05-09T04:52:08.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.107+0000 7fea97cb5700 1 -- 172.21.15.46:0/4029849767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea90105e30 msgr2=0x7fea90103710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:08.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.107+0000 7fea97cb5700 1 --2- 172.21.15.46:0/4029849767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea90105e30 0x7fea90103710 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fea80000c00 tx=0x7fea8000bfd0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.107+0000 7fea97cb5700 1 -- 172.21.15.46:0/4029849767 shutdown_connections 2024-05-09T04:52:08.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.107+0000 7fea97cb5700 1 --2- 172.21.15.46:0/4029849767 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fea7c0b2b90 0x7fea7c0b50a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.107+0000 7fea97cb5700 1 --2- 172.21.15.46:0/4029849767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea9010e910 0x7fea90103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.107+0000 7fea97cb5700 1 --2- 172.21.15.46:0/4029849767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea901054e0 0x7fea901031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.107+0000 7fea97cb5700 1 --2- 172.21.15.46:0/4029849767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea90105e30 0x7fea90103710 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:08.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.107+0000 7fea97cb5700 1 -- 172.21.15.46:0/4029849767 >> 172.21.15.46:0/4029849767 conn(0x7fea900fedc0 msgr2=0x7fea900ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:08.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.107+0000 7fea97cb5700 1 -- 172.21.15.46:0/4029849767 shutdown_connections 2024-05-09T04:52:08.112 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:08.108+0000 7fea97cb5700 1 -- 172.21.15.46:0/4029849767 wait complete. 2024-05-09T04:52:08.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:08 smithi046 ceph-mon[30136]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-09T04:52:09.001 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:52:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:08 smithi110 ceph-mon[34904]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-09T04:52:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:08 smithi072 ceph-mon[34088]: pgmap v34: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-09T04:52:09.273 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:10.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:09 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:52:10.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:09 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:10.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:09 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:52:10.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:09 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:10.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:09 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:52:10.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:09 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:10.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:09 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:52:10.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:09 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:09 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:52:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:09 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:09 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:52:10.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:09 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:10.724 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:10 smithi046 ceph-mon[30136]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-09T04:52:10.855 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.855+0000 7f5859429640 1 -- 172.21.15.46:0/3658802533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58541032f0 msgr2=0x7f585410f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.855+0000 7f5859429640 1 --2- 172.21.15.46:0/3658802533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58541032f0 0x7f585410f6c0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f584c009ca0 tx=0x7f584c02a8c0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.855+0000 7f5859429640 1 -- 172.21.15.46:0/3658802533 shutdown_connections 2024-05-09T04:52:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.855+0000 7f5859429640 1 --2- 172.21.15.46:0/3658802533 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5854109040 0x7f5854102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.855+0000 7f5859429640 1 --2- 172.21.15.46:0/3658802533 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58541032f0 0x7f585410f6c0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.855+0000 7f5859429640 1 --2- 172.21.15.46:0/3658802533 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5854108ba0 0x7f5854102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.855+0000 7f5859429640 1 -- 172.21.15.46:0/3658802533 >> 172.21.15.46:0/3658802533 conn(0x7f58540782e0 msgr2=0x7f5854078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.855+0000 7f5859429640 1 -- 172.21.15.46:0/3658802533 shutdown_connections 2024-05-09T04:52:10.856 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.855+0000 7f5859429640 1 -- 172.21.15.46:0/3658802533 wait complete. 2024-05-09T04:52:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.856+0000 7f5859429640 1 Processor -- start 2024-05-09T04:52:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.857+0000 7f5859429640 1 -- start start 2024-05-09T04:52:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.857+0000 7f5859429640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58541032f0 0x7f585419f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.857+0000 7f5859429640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5854108ba0 0x7f585419fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.857+0000 7f5859429640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5854109040 0x7f5854199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.857+0000 7f5859429640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5854112d90 con 0x7f58541032f0 2024-05-09T04:52:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.857+0000 7f5859429640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5854112c10 con 0x7f5854109040 2024-05-09T04:52:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.857+0000 7f5859429640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5854112f10 con 0x7f5854108ba0 2024-05-09T04:52:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.857+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58541032f0 0x7f585419f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.859 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.857+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58541032f0 0x7f585419f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34506/0 (socket says 172.21.15.46:34506) 2024-05-09T04:52:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.857+0000 7f5852ffd640 1 -- 172.21.15.46:0/3761421870 learned_addr learned my addr 172.21.15.46:0/3761421870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.857+0000 7f58527fc640 1 --2- 172.21.15.46:0/3761421870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5854108ba0 0x7f585419fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.857+0000 7f58537fe640 1 --2- 172.21.15.46:0/3761421870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5854109040 0x7f5854199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.858+0000 7f5852ffd640 1 -- 172.21.15.46:0/3761421870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5854108ba0 msgr2=0x7f585419fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.858+0000 7f5852ffd640 1 --2- 172.21.15.46:0/3761421870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5854108ba0 0x7f585419fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.858+0000 7f5852ffd640 1 -- 172.21.15.46:0/3761421870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5854109040 msgr2=0x7f5854199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.858+0000 7f5852ffd640 1 --2- 172.21.15.46:0/3761421870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5854109040 0x7f5854199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.858+0000 7f5852ffd640 1 -- 172.21.15.46:0/3761421870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5854199d80 con 0x7f58541032f0 2024-05-09T04:52:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.858+0000 7f5852ffd640 1 --2- 172.21.15.46:0/3761421870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58541032f0 0x7f585419f4d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5844009860 tx=0x7f5844009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.858+0000 7f583bfff640 1 -- 172.21.15.46:0/3761421870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584401d070 con 0x7f58541032f0 2024-05-09T04:52:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.858+0000 7f5859429640 1 -- 172.21.15.46:0/3761421870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f585419a010 con 0x7f58541032f0 2024-05-09T04:52:10.860 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.858+0000 7f583bfff640 1 -- 172.21.15.46:0/3761421870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5844007c50 con 0x7f58541032f0 2024-05-09T04:52:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.858+0000 7f583bfff640 1 -- 172.21.15.46:0/3761421870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844018d30 con 0x7f58541032f0 2024-05-09T04:52:10.861 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.858+0000 7f5859429640 1 -- 172.21.15.46:0/3761421870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f585406a4f0 con 0x7f58541032f0 2024-05-09T04:52:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.861+0000 7f583bfff640 1 -- 172.21.15.46:0/3761421870 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f5844020070 con 0x7f58541032f0 2024-05-09T04:52:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.861+0000 7f58603b3740 1 Processor -- start 2024-05-09T04:52:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.861+0000 7f58603b3740 1 -- start start 2024-05-09T04:52:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f58603b3740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650d91b0 0x556f65480a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f58603b3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650b02f0 0x556f65480fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.863 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f58603b3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f65481510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f58603b3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556f653e17c0 con 0x556f650b84d0 2024-05-09T04:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f58603b3740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556f6537d0e0 con 0x556f650d91b0 2024-05-09T04:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f58603b3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556f653dd780 con 0x556f650b02f0 2024-05-09T04:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f58527fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f65481510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f58527fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f65481510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34510/0 (socket says 172.21.15.46:34510) 2024-05-09T04:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f58527fc640 1 -- 172.21.15.46:0/3582333996 learned_addr learned my addr 172.21.15.46:0/3582333996 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f58537fe640 1 --2- 172.21.15.46:0/3582333996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650d91b0 0x556f65480a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f5852ffd640 1 --2- 172.21.15.46:0/3582333996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650b02f0 0x556f65480fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f58527fc640 1 -- 172.21.15.46:0/3582333996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650b02f0 msgr2=0x556f65480fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f58527fc640 1 --2- 172.21.15.46:0/3582333996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650b02f0 0x556f65480fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f58527fc640 1 -- 172.21.15.46:0/3582333996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650d91b0 msgr2=0x556f65480a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f58527fc640 1 --2- 172.21.15.46:0/3582333996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650d91b0 0x556f65480a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f58527fc640 1 -- 172.21.15.46:0/3582333996 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f653a67c0 con 0x556f650b84d0 2024-05-09T04:52:10.864 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.862+0000 7f5852ffd640 1 --2- 172.21.15.46:0/3582333996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650b02f0 0x556f65480fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:52:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.863+0000 7f583bfff640 1 --2- 172.21.15.46:0/3761421870 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f583c0b2a70 0x7f583c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.863+0000 7f58527fc640 1 --2- 172.21.15.46:0/3582333996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f65481510 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f584800bb70 tx=0x7f58480107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11376966552b1f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.863+0000 7f58397fa640 1 -- 172.21.15.46:0/3582333996 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848004030 con 0x556f650b84d0 2024-05-09T04:52:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.863+0000 7f583bfff640 1 -- 172.21.15.46:0/3761421870 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f584400a250 con 0x7f58541032f0 2024-05-09T04:52:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.863+0000 7f58397fa640 1 -- 172.21.15.46:0/3582333996 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5848005020 con 0x556f650b84d0 2024-05-09T04:52:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.863+0000 7f58397fa640 1 -- 172.21.15.46:0/3582333996 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5848004be0 con 0x556f650b84d0 2024-05-09T04:52:10.865 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.863+0000 7f58537fe640 1 --2- 172.21.15.46:0/3761421870 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f583c0b2a70 0x7f583c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.864+0000 7f58603b3740 1 -- 172.21.15.46:0/3582333996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 msgr2=0x556f65481510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.864+0000 7f58603b3740 1 --2- 172.21.15.46:0/3582333996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f65481510 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f584800bb70 tx=0x7f58480107a0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.864+0000 7f58537fe640 1 --2- 172.21.15.46:0/3761421870 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f583c0b2a70 0x7f583c0b4f80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x556f6547dc00 tx=0x7f584c02f390 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:10.870 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.864+0000 7f58603b3740 1 -- 172.21.15.46:0/3582333996 shutdown_connections 2024-05-09T04:52:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.864+0000 7f58603b3740 1 --2- 172.21.15.46:0/3582333996 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650b02f0 0x556f65480fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.864+0000 7f58603b3740 1 --2- 172.21.15.46:0/3582333996 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f65481510 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.864+0000 7f58603b3740 1 --2- 172.21.15.46:0/3582333996 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650d91b0 0x556f65480a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.864+0000 7f58603b3740 1 -- 172.21.15.46:0/3582333996 >> 172.21.15.46:0/3582333996 conn(0x556f650cd300 msgr2=0x556f6547d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.864+0000 7f58603b3740 1 -- 172.21.15.46:0/3582333996 shutdown_connections 2024-05-09T04:52:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.864+0000 7f58603b3740 1 -- 172.21.15.46:0/3582333996 wait complete. 2024-05-09T04:52:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.865+0000 7f58603b3740 1 -- start start 2024-05-09T04:52:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.865+0000 7f58603b3740 1 Processor -- start 2024-05-09T04:52:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.865+0000 7f58603b3740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650b02f0 0x556f6548b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.865+0000 7f58603b3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f6548b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.865+0000 7f58603b3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650d91b0 0x556f654910b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.865+0000 7f58603b3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556f6548dbc0 con 0x556f650b84d0 2024-05-09T04:52:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.865+0000 7f58603b3740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556f653689c0 con 0x556f650b02f0 2024-05-09T04:52:10.871 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.865+0000 7f58603b3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556f6548dd40 con 0x556f650d91b0 2024-05-09T04:52:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.865+0000 7f58603b3740 10 client.0 _ll_register_callbacks cb 0x556f65395020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.865+0000 7f58527fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f6548b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.865+0000 7f58537fe640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650d91b0 0x556f654910b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.865+0000 7f58527fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f6548b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34512/0 (socket says 172.21.15.46:34512) 2024-05-09T04:52:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.865+0000 7f58527fc640 1 -- 172.21.15.46:0/4227458179 learned_addr learned my addr 172.21.15.46:0/4227458179 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.865+0000 7f5852ffd640 1 --2- 172.21.15.46:0/4227458179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650b02f0 0x556f6548b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.866+0000 7f58537fe640 1 -- 172.21.15.46:0/4227458179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650b02f0 msgr2=0x556f6548b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.866+0000 7f58537fe640 1 --2- 172.21.15.46:0/4227458179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650b02f0 0x556f6548b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.866+0000 7f58537fe640 1 -- 172.21.15.46:0/4227458179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 msgr2=0x556f6548b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.866+0000 7f58537fe640 1 --2- 172.21.15.46:0/4227458179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f6548b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.866+0000 7f58537fe640 1 -- 172.21.15.46:0/4227458179 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f6538f1f0 con 0x556f650d91b0 2024-05-09T04:52:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.866+0000 7f58527fc640 1 --2- 172.21.15.46:0/4227458179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f6548b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.866+0000 7f5852ffd640 1 --2- 172.21.15.46:0/4227458179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650b02f0 0x556f6548b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:52:10.872 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.866+0000 7f58537fe640 1 --2- 172.21.15.46:0/4227458179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650d91b0 0x556f654910b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f584c005e30 tx=0x7f584c02a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa6f53a1dbfde5d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.866+0000 7f5836ff5640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:52:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.866+0000 7f5836ff5640 1 -- 172.21.15.46:0/4227458179 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c003e00 con 0x556f650d91b0 2024-05-09T04:52:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.866+0000 7f5836ff5640 1 -- 172.21.15.46:0/4227458179 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f584c004040 con 0x556f650d91b0 2024-05-09T04:52:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.867+0000 7f58603b3740 1 -- 172.21.15.46:0/4227458179 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5848005020 con 0x556f650d91b0 2024-05-09T04:52:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.867+0000 7f5836ff5640 1 -- 172.21.15.46:0/4227458179 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f584c004240 con 0x556f650d91b0 2024-05-09T04:52:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.867+0000 7f58603b3740 10 client.25225 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.867+0000 7f58603b3740 1 -- 172.21.15.46:0/4227458179 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556f65144790 con 0x556f650d91b0 2024-05-09T04:52:10.873 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.867+0000 7f5836ff5640 1 -- 172.21.15.46:0/4227458179 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5848005020 con 0x556f650d91b0 2024-05-09T04:52:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.867+0000 7f5836ff5640 1 -- 172.21.15.46:0/4227458179 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f584c051070 con 0x556f650d91b0 2024-05-09T04:52:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.867+0000 7f58603b3740 10 client.25225 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.867+0000 7f58603b3740 10 client.25225 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.867+0000 7f58603b3740 20 client.25225 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.867+0000 7f58603b3740 1 --2- 172.21.15.46:0/4227458179 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556f65097d80 0x556f6547bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.868+0000 7f58603b3740 4 client.25225 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.868+0000 7f58603b3740 1 -- 172.21.15.46:0/4227458179 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f653ef3d0 con 0x556f65097d80 2024-05-09T04:52:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.868+0000 7f5852ffd640 1 --2- 172.21.15.46:0/4227458179 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556f65097d80 0x556f6547bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.868+0000 7f5852ffd640 1 --2- 172.21.15.46:0/4227458179 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556f65097d80 0x556f6547bb40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c5340b24a2c69ce server_cookie=341581c18f9e3f28 in_seq=0 out_seq=0 2024-05-09T04:52:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.868+0000 7f5836ff5640 10 client.25225 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.869+0000 7f5836ff5640 1 -- 172.21.15.46:0/4227458179 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556f653ef3d0 con 0x556f65097d80 2024-05-09T04:52:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.869+0000 7f5836ff5640 10 client.25225 handle_command_reply: tid=0 2024-05-09T04:52:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.869+0000 7f58603b3740 1 client.25225 shutdown 2024-05-09T04:52:10.874 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.869+0000 7f58603b3740 1 -- 172.21.15.46:0/4227458179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650d91b0 msgr2=0x556f654910b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.869+0000 7f58603b3740 1 --2- 172.21.15.46:0/4227458179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650d91b0 0x556f654910b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f584c005e30 tx=0x7f584c02a8c0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.869+0000 7f58603b3740 1 -- 172.21.15.46:0/4227458179 shutdown_connections 2024-05-09T04:52:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.869+0000 7f58603b3740 1 --2- 172.21.15.46:0/4227458179 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556f65097d80 0x556f6547bb40 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.869+0000 7f58603b3740 1 --2- 172.21.15.46:0/4227458179 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650d91b0 0x556f654910b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.869+0000 7f58603b3740 1 --2- 172.21.15.46:0/4227458179 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f6548b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.869+0000 7f58603b3740 1 --2- 172.21.15.46:0/4227458179 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650b02f0 0x556f6548b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.869+0000 7f58603b3740 1 -- 172.21.15.46:0/4227458179 >> 172.21.15.46:0/4227458179 conn(0x556f650cd300 msgr2=0x556f65477700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.869+0000 7f5836ff5640 0 client.25225 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.870+0000 7f58603b3740 1 -- 172.21.15.46:0/4227458179 shutdown_connections 2024-05-09T04:52:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.870+0000 7f58603b3740 1 -- 172.21.15.46:0/4227458179 wait complete. 2024-05-09T04:52:10.875 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.870+0000 7f58603b3740 20 client.25225 trim_cache size 0 max 16384 2024-05-09T04:52:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58603b3740 1 Processor -- start 2024-05-09T04:52:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58603b3740 1 -- start start 2024-05-09T04:52:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58603b3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650b02f0 0x556f6547d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58603b3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f65489800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.889 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58603b3740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650d91b0 0x556f65489d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58603b3740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58541b0280 con 0x556f650b84d0 2024-05-09T04:52:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58603b3740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f58541adb50 con 0x556f650d91b0 2024-05-09T04:52:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58603b3740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f58541ab9a0 con 0x556f650b02f0 2024-05-09T04:52:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58537fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650d91b0 0x556f65489d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58537fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650d91b0 0x556f65489d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:39156/0 (socket says 172.21.15.46:39156) 2024-05-09T04:52:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58537fe640 1 -- 172.21.15.46:0/830866064 learned_addr learned my addr 172.21.15.46:0/830866064 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f5852ffd640 1 --2- 172.21.15.46:0/830866064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650b02f0 0x556f6547d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58527fc640 1 --2- 172.21.15.46:0/830866064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f65489800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58537fe640 1 -- 172.21.15.46:0/830866064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650b02f0 msgr2=0x556f6547d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58537fe640 1 --2- 172.21.15.46:0/830866064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650b02f0 0x556f6547d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58537fe640 1 -- 172.21.15.46:0/830866064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 msgr2=0x556f65489800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58537fe640 1 --2- 172.21.15.46:0/830866064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f65489800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.886+0000 7f58537fe640 1 -- 172.21.15.46:0/830866064 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f651bb5f0 con 0x556f650d91b0 2024-05-09T04:52:10.890 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5852ffd640 1 --2- 172.21.15.46:0/830866064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650b02f0 0x556f6547d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:52:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58527fc640 1 --2- 172.21.15.46:0/830866064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f65489800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:52:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58537fe640 1 --2- 172.21.15.46:0/830866064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650d91b0 0x556f65489d40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f584c02ade0 tx=0x7f584c0095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32f8c16c86e32672 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5838ff9640 1 -- 172.21.15.46:0/830866064 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c0305c0 con 0x556f650d91b0 2024-05-09T04:52:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5838ff9640 1 -- 172.21.15.46:0/830866064 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f584c036050 con 0x556f650d91b0 2024-05-09T04:52:10.891 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5838ff9640 1 -- 172.21.15.46:0/830866064 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f584c030940 con 0x556f650d91b0 2024-05-09T04:52:10.894 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 -- 172.21.15.46:0/830866064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650d91b0 msgr2=0x556f65489d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 --2- 172.21.15.46:0/830866064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650d91b0 0x556f65489d40 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f584c02ade0 tx=0x7f584c0095c0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 -- 172.21.15.46:0/830866064 shutdown_connections 2024-05-09T04:52:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 --2- 172.21.15.46:0/830866064 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650d91b0 0x556f65489d40 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 --2- 172.21.15.46:0/830866064 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f65489800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 --2- 172.21.15.46:0/830866064 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650b02f0 0x556f6547d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 -- 172.21.15.46:0/830866064 >> 172.21.15.46:0/830866064 conn(0x556f650cd300 msgr2=0x556f65484780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 -- 172.21.15.46:0/830866064 shutdown_connections 2024-05-09T04:52:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 -- 172.21.15.46:0/830866064 wait complete. 2024-05-09T04:52:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 -- start start 2024-05-09T04:52:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 Processor -- start 2024-05-09T04:52:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650b02f0 0x556f65496f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f654974c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650d91b0 0x556f65491420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.895 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650b02f0 0x556f65496f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58527fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f654974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5852ffd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650b02f0 0x556f65496f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:39168/0 (socket says 172.21.15.46:39168) 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5852ffd640 1 -- 172.21.15.46:0/2650067356 learned_addr learned my addr 172.21.15.46:0/2650067356 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 -- 172.21.15.46:0/2650067356 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58541ac010 con 0x556f650b84d0 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 -- 172.21.15.46:0/2650067356 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f58541ad480 con 0x556f650b02f0 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 -- 172.21.15.46:0/2650067356 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f58541ac6e0 con 0x556f650d91b0 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 10 client.0 _ll_register_callbacks cb 0x7f584c031450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58537fe640 1 --2- 172.21.15.46:0/2650067356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650d91b0 0x556f65491420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5852ffd640 1 -- 172.21.15.46:0/2650067356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650d91b0 msgr2=0x556f65491420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5852ffd640 1 --2- 172.21.15.46:0/2650067356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650d91b0 0x556f65491420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5852ffd640 1 -- 172.21.15.46:0/2650067356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 msgr2=0x556f654974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5852ffd640 1 --2- 172.21.15.46:0/2650067356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f654974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5852ffd640 1 -- 172.21.15.46:0/2650067356 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f653a5ed0 con 0x556f650b02f0 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5852ffd640 1 --2- 172.21.15.46:0/2650067356 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650b02f0 0x556f65496f80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f584401a000 tx=0x7f5844009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a488b44b03c934a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5836ff5640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:52:10.896 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5836ff5640 1 -- 172.21.15.46:0/2650067356 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58440b3330 con 0x556f650b02f0 2024-05-09T04:52:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 -- 172.21.15.46:0/2650067356 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f584c036050 con 0x556f650b02f0 2024-05-09T04:52:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5836ff5640 1 -- 172.21.15.46:0/2650067356 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f584400d0b0 con 0x556f650b02f0 2024-05-09T04:52:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5836ff5640 1 -- 172.21.15.46:0/2650067356 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f584400d2b0 con 0x556f650b02f0 2024-05-09T04:52:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 10 client.25280 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 -- 172.21.15.46:0/2650067356 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556f65508dd0 con 0x556f650b02f0 2024-05-09T04:52:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5836ff5640 1 -- 172.21.15.46:0/2650067356 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f584c036050 con 0x556f650b02f0 2024-05-09T04:52:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5836ff5640 1 -- 172.21.15.46:0/2650067356 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f58440e13b0 con 0x556f650b02f0 2024-05-09T04:52:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 10 client.25280 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 10 client.25280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 20 client.25280 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 --2- 172.21.15.46:0/2650067356 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556f6506d1c0 0x556f6550b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 4 client.25280 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T04:52:10.897 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58603b3740 1 -- 172.21.15.46:0/2650067356 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556f653e9240 con 0x556f6506d1c0 2024-05-09T04:52:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58527fc640 1 --2- 172.21.15.46:0/2650067356 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556f6506d1c0 0x556f6550b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f5836ff5640 10 client.25280 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:10.898 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.889+0000 7f58527fc640 1 --2- 172.21.15.46:0/2650067356 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556f6506d1c0 0x556f6550b510 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc3570ad001e8b60 server_cookie=72ce0a421ed07b15 in_seq=0 out_seq=0 2024-05-09T04:52:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f5836ff5640 1 -- 172.21.15.46:0/2650067356 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556f653e9240 con 0x556f6506d1c0 2024-05-09T04:52:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f5836ff5640 10 client.25280 handle_command_reply: tid=0 2024-05-09T04:52:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f58603b3740 1 client.25280 shutdown 2024-05-09T04:52:10.902 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f58603b3740 1 -- 172.21.15.46:0/2650067356 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650b02f0 msgr2=0x556f65496f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f58603b3740 1 --2- 172.21.15.46:0/2650067356 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650b02f0 0x556f65496f80 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f584401a000 tx=0x7f5844009f70 comp rx=0 tx=0).stop 2024-05-09T04:52:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f58603b3740 1 -- 172.21.15.46:0/2650067356 shutdown_connections 2024-05-09T04:52:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f58603b3740 1 --2- 172.21.15.46:0/2650067356 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556f6506d1c0 0x556f6550b510 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f58603b3740 1 --2- 172.21.15.46:0/2650067356 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556f650d91b0 0x556f65491420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f58603b3740 1 --2- 172.21.15.46:0/2650067356 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556f650b84d0 0x556f654974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f58603b3740 1 --2- 172.21.15.46:0/2650067356 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556f650b02f0 0x556f65496f80 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f58603b3740 1 -- 172.21.15.46:0/2650067356 >> 172.21.15.46:0/2650067356 conn(0x556f650cd300 msgr2=0x556f65477700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f5836ff5640 0 client.25280 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f58603b3740 1 -- 172.21.15.46:0/2650067356 shutdown_connections 2024-05-09T04:52:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f58603b3740 1 -- 172.21.15.46:0/2650067356 wait complete. 2024-05-09T04:52:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f58603b3740 20 client.25280 trim_cache size 0 max 16384 2024-05-09T04:52:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-05-09T04:52:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-05-09T04:52:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-05-09T04:52:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f5859429640 1 -- 172.21.15.46:0/3761421870 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f583c0b2a70 msgr2=0x7f583c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f5859429640 1 --2- 172.21.15.46:0/3761421870 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f583c0b2a70 0x7f583c0b4f80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x556f6547dc00 tx=0x7f584c02f390 comp rx=0 tx=0).stop 2024-05-09T04:52:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f5859429640 1 -- 172.21.15.46:0/3761421870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58541032f0 msgr2=0x7f585419f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f5859429640 1 --2- 172.21.15.46:0/3761421870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58541032f0 0x7f585419f4d0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5844009860 tx=0x7f5844009d20 comp rx=0 tx=0).stop 2024-05-09T04:52:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f5859429640 1 -- 172.21.15.46:0/3761421870 shutdown_connections 2024-05-09T04:52:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f5859429640 1 --2- 172.21.15.46:0/3761421870 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f583c0b2a70 0x7f583c0b4f80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f5859429640 1 --2- 172.21.15.46:0/3761421870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5854108ba0 0x7f585419fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f5859429640 1 --2- 172.21.15.46:0/3761421870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5854109040 0x7f5854199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f5859429640 1 --2- 172.21.15.46:0/3761421870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f58541032f0 0x7f585419f4d0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f5859429640 1 -- 172.21.15.46:0/3761421870 >> 172.21.15.46:0/3761421870 conn(0x7f58540782e0 msgr2=0x7f58540fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f5859429640 1 -- 172.21.15.46:0/3761421870 shutdown_connections 2024-05-09T04:52:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:10.901+0000 7f5859429640 1 -- 172.21.15.46:0/3761421870 wait complete. 2024-05-09T04:52:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:10 smithi110 ceph-mon[34904]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-09T04:52:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:10 smithi072 ceph-mon[34088]: pgmap v35: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 KiB/s wr, 22 op/s 2024-05-09T04:52:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.170+0000 7fee952f9700 1 -- 172.21.15.46:0/604341475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee90108760 msgr2=0x7fee901100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.170+0000 7fee952f9700 1 --2- 172.21.15.46:0/604341475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee90108760 0x7fee901100c0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fee8800b0d0 tx=0x7fee8800b3e0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.170+0000 7fee952f9700 1 -- 172.21.15.46:0/604341475 shutdown_connections 2024-05-09T04:52:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.170+0000 7fee952f9700 1 --2- 172.21.15.46:0/604341475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee90108760 0x7fee901100c0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.170+0000 7fee952f9700 1 --2- 172.21.15.46:0/604341475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee901034e0 0x7fee901038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.170+0000 7fee952f9700 1 --2- 172.21.15.46:0/604341475 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee90107e10 0x7fee90108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.170+0000 7fee952f9700 1 -- 172.21.15.46:0/604341475 >> 172.21.15.46:0/604341475 conn(0x7fee900fef30 msgr2=0x7fee90101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.170+0000 7fee952f9700 1 -- 172.21.15.46:0/604341475 shutdown_connections 2024-05-09T04:52:11.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.170+0000 7fee952f9700 1 -- 172.21.15.46:0/604341475 wait complete. 2024-05-09T04:52:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.171+0000 7fee952f9700 1 Processor -- start 2024-05-09T04:52:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.171+0000 7fee952f9700 1 -- start start 2024-05-09T04:52:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee952f9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee901034e0 0x7fee900ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee952f9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee90107e10 0x7fee900ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee952f9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee90108760 0x7fee900ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee952f9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee90113390 con 0x7fee90107e10 2024-05-09T04:52:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee952f9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fee90113210 con 0x7fee901034e0 2024-05-09T04:52:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee952f9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee90113510 con 0x7fee90108760 2024-05-09T04:52:11.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee8e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee90107e10 0x7fee900ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee8e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee90107e10 0x7fee900ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34538/0 (socket says 172.21.15.46:34538) 2024-05-09T04:52:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee8e7fc700 1 -- 172.21.15.46:0/236926352 learned_addr learned my addr 172.21.15.46:0/236926352 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee8f7fe700 1 --2- 172.21.15.46:0/236926352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee90108760 0x7fee900ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee8effd700 1 --2- 172.21.15.46:0/236926352 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee901034e0 0x7fee900ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee8e7fc700 1 -- 172.21.15.46:0/236926352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee90108760 msgr2=0x7fee900ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/236926352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee90108760 0x7fee900ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee8e7fc700 1 -- 172.21.15.46:0/236926352 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee901034e0 msgr2=0x7fee900ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/236926352 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee901034e0 0x7fee900ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.172+0000 7fee8e7fc700 1 -- 172.21.15.46:0/236926352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee901a91c0 con 0x7fee90107e10 2024-05-09T04:52:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.173+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/236926352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee90107e10 0x7fee900ff950 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fee8400c9e0 tx=0x7fee8400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.173+0000 7fee77fff700 1 -- 172.21.15.46:0/236926352 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee84007aa0 con 0x7fee90107e10 2024-05-09T04:52:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.173+0000 7fee952f9700 1 -- 172.21.15.46:0/236926352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee901a94b0 con 0x7fee90107e10 2024-05-09T04:52:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.173+0000 7fee77fff700 1 -- 172.21.15.46:0/236926352 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee84007c40 con 0x7fee90107e10 2024-05-09T04:52:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.173+0000 7fee77fff700 1 -- 172.21.15.46:0/236926352 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8400fe60 con 0x7fee90107e10 2024-05-09T04:52:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.173+0000 7fee952f9700 1 -- 172.21.15.46:0/236926352 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee901a99f0 con 0x7fee90107e10 2024-05-09T04:52:11.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.176+0000 7fee77fff700 1 -- 172.21.15.46:0/236926352 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fee84004870 con 0x7fee90107e10 2024-05-09T04:52:11.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.177+0000 7fee77fff700 1 --2- 172.21.15.46:0/236926352 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fee780b2b10 0x7fee780b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.177+0000 7fee8effd700 1 --2- 172.21.15.46:0/236926352 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fee780b2b10 0x7fee780b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.177+0000 7fee77fff700 1 -- 172.21.15.46:0/236926352 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fee840e27a0 con 0x7fee90107e10 2024-05-09T04:52:11.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.177+0000 7feea3d0fb80 1 Processor -- start 2024-05-09T04:52:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.177+0000 7fee8effd700 1 --2- 172.21.15.46:0/236926352 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fee780b2b10 0x7fee780b5020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fee80007be0 tx=0x7fee800073d0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.177+0000 7feea3d0fb80 1 -- start start 2024-05-09T04:52:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7feea3d0fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e48808d00 0x559e48911240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7feea3d0fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e48911780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7feea3d0fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e4890b400 0x559e4890b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7feea3d0fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559e4882e0a0 con 0x559e4878e4a0 2024-05-09T04:52:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7feea3d0fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559e487d39a0 con 0x559e4890b400 2024-05-09T04:52:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7feea3d0fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559e4882a060 con 0x559e48808d00 2024-05-09T04:52:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7fee8e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e48911780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7fee8e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e48911780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34550/0 (socket says 172.21.15.46:34550) 2024-05-09T04:52:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7fee8e7fc700 1 -- 172.21.15.46:0/1980583256 learned_addr learned my addr 172.21.15.46:0/1980583256 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7fee8f7fe700 1 --2- 172.21.15.46:0/1980583256 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e4890b400 0x559e4890b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7fee8effd700 1 --2- 172.21.15.46:0/1980583256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e48808d00 0x559e48911240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7fee8e7fc700 1 -- 172.21.15.46:0/1980583256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e48808d00 msgr2=0x559e48911240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:11.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/1980583256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e48808d00 0x559e48911240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7fee8e7fc700 1 -- 172.21.15.46:0/1980583256 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e4890b400 msgr2=0x559e4890b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/1980583256 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e4890b400 0x559e4890b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.178+0000 7fee8e7fc700 1 -- 172.21.15.46:0/1980583256 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e487c4500 con 0x559e4878e4a0 2024-05-09T04:52:11.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.179+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/1980583256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e48911780 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fee840a9510 tx=0x7fee840a9160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1276dddc2d55f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:11.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.179+0000 7fee757fa700 1 -- 172.21.15.46:0/1980583256 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840a9320 con 0x559e4878e4a0 2024-05-09T04:52:11.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.179+0000 7fee757fa700 1 -- 172.21.15.46:0/1980583256 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee84007df0 con 0x559e4878e4a0 2024-05-09T04:52:11.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.179+0000 7fee757fa700 1 -- 172.21.15.46:0/1980583256 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840a9a70 con 0x559e4878e4a0 2024-05-09T04:52:11.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.180+0000 7feea3d0fb80 1 -- 172.21.15.46:0/1980583256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 msgr2=0x559e48911780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:11.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.180+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/1980583256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e48911780 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fee840a9510 tx=0x7fee840a9160 comp rx=0 tx=0).stop 2024-05-09T04:52:11.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.180+0000 7feea3d0fb80 1 -- 172.21.15.46:0/1980583256 shutdown_connections 2024-05-09T04:52:11.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.180+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/1980583256 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e4890b400 0x559e4890b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.180+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/1980583256 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e48808d00 0x559e48911240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.180+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/1980583256 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e48911780 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.180+0000 7feea3d0fb80 1 -- 172.21.15.46:0/1980583256 >> 172.21.15.46:0/1980583256 conn(0x559e48791f70 msgr2=0x559e48906220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.180+0000 7feea3d0fb80 1 -- 172.21.15.46:0/1980583256 shutdown_connections 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.180+0000 7feea3d0fb80 1 -- 172.21.15.46:0/1980583256 wait complete. 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.180+0000 7feea3d0fb80 1 -- start start 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7feea3d0fb80 1 Processor -- start 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7feea3d0fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e48914f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7feea3d0fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e48808d00 0x559e489154d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7feea3d0fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4890b400 0x559e48915a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7feea3d0fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559e4890c160 con 0x559e4878e4a0 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7feea3d0fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559e487c8b20 con 0x559e48808d00 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7feea3d0fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559e48918020 con 0x559e4890b400 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7fee8effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e48914f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7fee8effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e48914f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34564/0 (socket says 172.21.15.46:34564) 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7fee8effd700 1 -- 172.21.15.46:0/355660703 learned_addr learned my addr 172.21.15.46:0/355660703 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7feea3d0fb80 10 client.0 _ll_register_callbacks cb 0x559e4869f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/355660703 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e48808d00 0x559e489154d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7fee8f7fe700 1 --2- 172.21.15.46:0/355660703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4890b400 0x559e48915a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7fee8effd700 1 -- 172.21.15.46:0/355660703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4890b400 msgr2=0x559e48915a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7fee8effd700 1 --2- 172.21.15.46:0/355660703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4890b400 0x559e48915a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7fee8effd700 1 -- 172.21.15.46:0/355660703 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e48808d00 msgr2=0x559e489154d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7fee8effd700 1 --2- 172.21.15.46:0/355660703 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e48808d00 0x559e489154d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7fee8effd700 1 -- 172.21.15.46:0/355660703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e486a9490 con 0x559e4878e4a0 2024-05-09T04:52:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.181+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/355660703 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e48808d00 0x559e489154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:52:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.182+0000 7fee8effd700 1 --2- 172.21.15.46:0/355660703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e48914f90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fee800042e0 tx=0x7fee800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9af1b6787d0185 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.182+0000 7fee72ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.182+0000 7fee72ff5700 1 -- 172.21.15.46:0/355660703 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80024070 con 0x559e4878e4a0 2024-05-09T04:52:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.182+0000 7fee72ff5700 1 -- 172.21.15.46:0/355660703 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fee80003d10 con 0x559e4878e4a0 2024-05-09T04:52:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.182+0000 7feea3d0fb80 1 -- 172.21.15.46:0/355660703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee84007df0 con 0x559e4878e4a0 2024-05-09T04:52:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.182+0000 7fee72ff5700 1 -- 172.21.15.46:0/355660703 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fee80029440 con 0x559e4878e4a0 2024-05-09T04:52:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.182+0000 7feea3d0fb80 10 client.15528 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:11.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.182+0000 7feea3d0fb80 1 -- 172.21.15.46:0/355660703 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559e489229a0 con 0x559e4878e4a0 2024-05-09T04:52:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.182+0000 7fee72ff5700 1 -- 172.21.15.46:0/355660703 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee84007df0 con 0x559e4878e4a0 2024-05-09T04:52:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.183+0000 7fee72ff5700 1 -- 172.21.15.46:0/355660703 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fee80020410 con 0x559e4878e4a0 2024-05-09T04:52:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.183+0000 7feea3d0fb80 10 client.15528 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.183+0000 7feea3d0fb80 10 client.15528 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.183+0000 7feea3d0fb80 20 client.15528 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.183+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/355660703 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559e48925030 0x559e48927450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.183+0000 7feea3d0fb80 4 client.15528 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.183+0000 7feea3d0fb80 1 -- 172.21.15.46:0/355660703 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559e4883ba80 con 0x559e48925030 2024-05-09T04:52:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.183+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/355660703 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559e48925030 0x559e48927450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.183+0000 7fee72ff5700 10 client.15528 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:11.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.184+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/355660703 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559e48925030 0x559e48927450 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=667c1ad8a19e3d73 server_cookie=dd437f57ffb04ae3 in_seq=0 out_seq=0 2024-05-09T04:52:11.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.184+0000 7fee72ff5700 1 -- 172.21.15.46:0/355660703 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559e4883ba80 con 0x559e48925030 2024-05-09T04:52:11.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.184+0000 7fee72ff5700 10 client.15528 handle_command_reply: tid=0 2024-05-09T04:52:11.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.185+0000 7feea3d0fb80 1 client.15528 shutdown 2024-05-09T04:52:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.185+0000 7feea3d0fb80 1 -- 172.21.15.46:0/355660703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 msgr2=0x559e48914f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.185+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/355660703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e48914f90 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fee800042e0 tx=0x7fee800047c0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.185+0000 7feea3d0fb80 1 -- 172.21.15.46:0/355660703 shutdown_connections 2024-05-09T04:52:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.185+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/355660703 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4890b400 0x559e48915a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.185+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/355660703 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e48914f90 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.185+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/355660703 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559e48925030 0x559e48927450 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.185+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/355660703 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e48808d00 0x559e489154d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.185+0000 7fee72ff5700 0 client.15528 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.185+0000 7feea3d0fb80 1 -- 172.21.15.46:0/355660703 >> 172.21.15.46:0/355660703 conn(0x559e48791f70 msgr2=0x559e488ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.185+0000 7feea3d0fb80 1 -- 172.21.15.46:0/355660703 shutdown_connections 2024-05-09T04:52:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.185+0000 7feea3d0fb80 1 -- 172.21.15.46:0/355660703 wait complete. 2024-05-09T04:52:11.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.185+0000 7feea3d0fb80 20 client.15528 trim_cache size 0 max 16384 2024-05-09T04:52:11.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.206+0000 7feea3d0fb80 1 Processor -- start 2024-05-09T04:52:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.206+0000 7feea3d0fb80 1 -- start start 2024-05-09T04:52:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.206+0000 7feea3d0fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4878e4a0 0x559e48920e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.206+0000 7feea3d0fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e48808d00 0x559e4892cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.206+0000 7feea3d0fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e4890b400 0x559e4892d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.206+0000 7feea3d0fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee901b3a60 con 0x559e48808d00 2024-05-09T04:52:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.206+0000 7feea3d0fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fee901b12f0 con 0x559e4890b400 2024-05-09T04:52:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.206+0000 7feea3d0fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee901af140 con 0x559e4878e4a0 2024-05-09T04:52:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee8e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e48808d00 0x559e4892cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee8e7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e48808d00 0x559e4892cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34568/0 (socket says 172.21.15.46:34568) 2024-05-09T04:52:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee8e7fc700 1 -- 172.21.15.46:0/2753029205 learned_addr learned my addr 172.21.15.46:0/2753029205 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee8effd700 1 --2- 172.21.15.46:0/2753029205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4878e4a0 0x559e48920e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee8f7fe700 1 --2- 172.21.15.46:0/2753029205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e4890b400 0x559e4892d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee8e7fc700 1 -- 172.21.15.46:0/2753029205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4878e4a0 msgr2=0x559e48920e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/2753029205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4878e4a0 0x559e48920e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee8e7fc700 1 -- 172.21.15.46:0/2753029205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e4890b400 msgr2=0x559e4892d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/2753029205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e4890b400 0x559e4892d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee8e7fc700 1 -- 172.21.15.46:0/2753029205 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559e489029d0 con 0x559e48808d00 2024-05-09T04:52:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee8effd700 1 --2- 172.21.15.46:0/2753029205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4878e4a0 0x559e48920e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:52:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee8f7fe700 1 --2- 172.21.15.46:0/2753029205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e4890b400 0x559e4892d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:52:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/2753029205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e48808d00 0x559e4892cc10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fee840dfe60 tx=0x7fee840a50d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b661bf3cbd25eb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee74ff9700 1 -- 172.21.15.46:0/2753029205 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee8406b030 con 0x559e48808d00 2024-05-09T04:52:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee74ff9700 1 -- 172.21.15.46:0/2753029205 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee840adbc0 con 0x559e48808d00 2024-05-09T04:52:11.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.207+0000 7fee74ff9700 1 -- 172.21.15.46:0/2753029205 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee840c2070 con 0x559e48808d00 2024-05-09T04:52:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.208+0000 7feea3d0fb80 1 -- 172.21.15.46:0/2753029205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e48808d00 msgr2=0x559e4892cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.208+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/2753029205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e48808d00 0x559e4892cc10 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fee840dfe60 tx=0x7fee840a50d0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.208+0000 7feea3d0fb80 1 -- 172.21.15.46:0/2753029205 shutdown_connections 2024-05-09T04:52:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.208+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/2753029205 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e4890b400 0x559e4892d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.208+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/2753029205 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4878e4a0 0x559e48920e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.208+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/2753029205 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e48808d00 0x559e4892cc10 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.208+0000 7feea3d0fb80 1 -- 172.21.15.46:0/2753029205 >> 172.21.15.46:0/2753029205 conn(0x559e48791f70 msgr2=0x559e48913880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.209+0000 7feea3d0fb80 1 -- 172.21.15.46:0/2753029205 shutdown_connections 2024-05-09T04:52:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.209+0000 7feea3d0fb80 1 -- 172.21.15.46:0/2753029205 wait complete. 2024-05-09T04:52:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.209+0000 7feea3d0fb80 1 -- start start 2024-05-09T04:52:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.209+0000 7feea3d0fb80 1 Processor -- start 2024-05-09T04:52:11.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.209+0000 7feea3d0fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e4891d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.209+0000 7feea3d0fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e48808d00 0x559e4891d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.209+0000 7feea3d0fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4890b400 0x559e48911640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7feea3d0fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fee901af7b0 con 0x559e4878e4a0 2024-05-09T04:52:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7feea3d0fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fee901b0c20 con 0x559e48808d00 2024-05-09T04:52:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7feea3d0fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fee901afe80 con 0x559e4890b400 2024-05-09T04:52:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7feea3d0fb80 10 client.0 _ll_register_callbacks cb 0x559e4869f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7fee8effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e4891d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7fee8effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e4891d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34578/0 (socket says 172.21.15.46:34578) 2024-05-09T04:52:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7fee8effd700 1 -- 172.21.15.46:0/2506379559 learned_addr learned my addr 172.21.15.46:0/2506379559 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/2506379559 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e48808d00 0x559e4891d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7fee8f7fe700 1 --2- 172.21.15.46:0/2506379559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4890b400 0x559e48911640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7fee8effd700 1 -- 172.21.15.46:0/2506379559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4890b400 msgr2=0x559e48911640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:11.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7fee8effd700 1 --2- 172.21.15.46:0/2506379559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4890b400 0x559e48911640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7fee8effd700 1 -- 172.21.15.46:0/2506379559 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e48808d00 msgr2=0x559e4891d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7fee8effd700 1 --2- 172.21.15.46:0/2506379559 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e48808d00 0x559e4891d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7fee8effd700 1 -- 172.21.15.46:0/2506379559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559e48911d40 con 0x559e4878e4a0 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7fee8effd700 1 --2- 172.21.15.46:0/2506379559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e4891d100 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fee80000c00 tx=0x7fee80004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4947db90f421fcae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7fee72ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.210+0000 7fee72ff5700 1 -- 172.21.15.46:0/2506379559 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee80039070 con 0x559e4878e4a0 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.211+0000 7fee72ff5700 1 -- 172.21.15.46:0/2506379559 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fee80003b60 con 0x559e4878e4a0 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.211+0000 7feea3d0fb80 1 -- 172.21.15.46:0/2506379559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x559e48835b20 con 0x559e4878e4a0 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.211+0000 7fee72ff5700 1 -- 172.21.15.46:0/2506379559 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fee80024040 con 0x559e4878e4a0 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.211+0000 7feea3d0fb80 10 client.15534 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.211+0000 7feea3d0fb80 1 -- 172.21.15.46:0/2506379559 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559e4898f010 con 0x559e4878e4a0 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.211+0000 7fee72ff5700 1 -- 172.21.15.46:0/2506379559 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x559e48835b20 con 0x559e4878e4a0 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.211+0000 7fee72ff5700 1 -- 172.21.15.46:0/2506379559 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fee80027960 con 0x559e4878e4a0 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.211+0000 7feea3d0fb80 10 client.15534 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.211+0000 7feea3d0fb80 10 client.15534 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.211+0000 7feea3d0fb80 20 client.15534 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:11.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.211+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/2506379559 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559e4898f720 0x559e48991b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.211+0000 7feea3d0fb80 4 client.15534 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:52:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.211+0000 7feea3d0fb80 1 -- 172.21.15.46:0/2506379559 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559e486a9970 con 0x559e4898f720 2024-05-09T04:52:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.212+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/2506379559 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559e4898f720 0x559e48991b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.212+0000 7fee8e7fc700 1 --2- 172.21.15.46:0/2506379559 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559e4898f720 0x559e48991b40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0e853c7860b176a server_cookie=d6d2d1f8fa6689d2 in_seq=0 out_seq=0 2024-05-09T04:52:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.212+0000 7fee72ff5700 10 client.15534 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.212+0000 7fee72ff5700 1 -- 172.21.15.46:0/2506379559 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x559e486a9970 con 0x559e4898f720 2024-05-09T04:52:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.212+0000 7fee72ff5700 10 client.15534 handle_command_reply: tid=0 2024-05-09T04:52:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.212+0000 7feea3d0fb80 1 client.15534 shutdown 2024-05-09T04:52:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.212+0000 7feea3d0fb80 1 -- 172.21.15.46:0/2506379559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 msgr2=0x559e4891d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.212+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/2506379559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e4891d100 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fee80000c00 tx=0x7fee80004680 comp rx=0 tx=0).stop 2024-05-09T04:52:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.212+0000 7feea3d0fb80 1 -- 172.21.15.46:0/2506379559 shutdown_connections 2024-05-09T04:52:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.212+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/2506379559 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559e4890b400 0x559e48911640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.212+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/2506379559 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559e4878e4a0 0x559e4891d100 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.212+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/2506379559 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559e4898f720 0x559e48991b40 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.212+0000 7fee72ff5700 0 client.15534 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:11.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.212+0000 7feea3d0fb80 1 --2- 172.21.15.46:0/2506379559 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559e48808d00 0x559e4891d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.212+0000 7feea3d0fb80 1 -- 172.21.15.46:0/2506379559 >> 172.21.15.46:0/2506379559 conn(0x559e48791f70 msgr2=0x559e488ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:11.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.213+0000 7feea3d0fb80 1 -- 172.21.15.46:0/2506379559 shutdown_connections 2024-05-09T04:52:11.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.213+0000 7feea3d0fb80 1 -- 172.21.15.46:0/2506379559 wait complete. 2024-05-09T04:52:11.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.213+0000 7feea3d0fb80 20 client.15534 trim_cache size 0 max 16384 2024-05-09T04:52:11.217 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:52:11.217 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:52:11.217 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:52:11.217 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:52:11.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.213+0000 7fee952f9700 1 -- 172.21.15.46:0/236926352 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fee780b2b10 msgr2=0x7fee780b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:11.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.213+0000 7fee952f9700 1 --2- 172.21.15.46:0/236926352 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fee780b2b10 0x7fee780b5020 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fee80007be0 tx=0x7fee800073d0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.213+0000 7fee952f9700 1 -- 172.21.15.46:0/236926352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee90107e10 msgr2=0x7fee900ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:11.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.213+0000 7fee952f9700 1 --2- 172.21.15.46:0/236926352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee90107e10 0x7fee900ff950 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fee8400c9e0 tx=0x7fee8400ccf0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.213+0000 7fee952f9700 1 -- 172.21.15.46:0/236926352 shutdown_connections 2024-05-09T04:52:11.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.213+0000 7fee952f9700 1 --2- 172.21.15.46:0/236926352 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fee780b2b10 0x7fee780b5020 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.213+0000 7fee952f9700 1 --2- 172.21.15.46:0/236926352 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fee90108760 0x7fee900ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.213+0000 7fee952f9700 1 --2- 172.21.15.46:0/236926352 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fee901034e0 0x7fee900ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.213+0000 7fee952f9700 1 --2- 172.21.15.46:0/236926352 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fee90107e10 0x7fee900ff950 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:11.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.213+0000 7fee952f9700 1 -- 172.21.15.46:0/236926352 >> 172.21.15.46:0/236926352 conn(0x7fee900fef30 msgr2=0x7fee9006c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.213+0000 7fee952f9700 1 -- 172.21.15.46:0/236926352 shutdown_connections 2024-05-09T04:52:11.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:11.214+0000 7fee952f9700 1 -- 172.21.15.46:0/236926352 wait complete. 2024-05-09T04:52:11.958 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:11 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:52:11.985 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:11 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:52:12.013 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:52:12.013 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:52:12.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:11 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:52:12.276 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:13.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:12 smithi110 ceph-mon[34904]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-09T04:52:13.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:12 smithi046 ceph-mon[30136]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-09T04:52:13.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:12 smithi072 ceph-mon[34088]: pgmap v36: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-09T04:52:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.072+0000 7fa39add4700 1 -- 172.21.15.46:0/2256642562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa394107df0 msgr2=0x7fa394108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.072+0000 7fa39add4700 1 --2- 172.21.15.46:0/2256642562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa394107df0 0x7fa394108200 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fa3800099c0 tx=0x7fa380009cd0 comp rx=0 tx=0).stop 2024-05-09T04:52:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.072+0000 7fa39add4700 1 -- 172.21.15.46:0/2256642562 shutdown_connections 2024-05-09T04:52:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.072+0000 7fa39add4700 1 --2- 172.21.15.46:0/2256642562 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa394108740 0x7fa3941100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.072+0000 7fa39add4700 1 --2- 172.21.15.46:0/2256642562 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3941034c0 0x7fa3941038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.072+0000 7fa39add4700 1 --2- 172.21.15.46:0/2256642562 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa394107df0 0x7fa394108200 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.072+0000 7fa39add4700 1 -- 172.21.15.46:0/2256642562 >> 172.21.15.46:0/2256642562 conn(0x7fa3940fedc0 msgr2=0x7fa3941011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.072+0000 7fa39add4700 1 -- 172.21.15.46:0/2256642562 shutdown_connections 2024-05-09T04:52:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.072+0000 7fa39add4700 1 -- 172.21.15.46:0/2256642562 wait complete. 2024-05-09T04:52:14.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.073+0000 7fa39add4700 1 Processor -- start 2024-05-09T04:52:14.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.074+0000 7fa39add4700 1 -- start start 2024-05-09T04:52:14.076 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.074+0000 7fa39add4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3941034c0 0x7fa39419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:14.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.074+0000 7fa39add4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa394107df0 0x7fa39419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:14.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.074+0000 7fa39add4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa394108740 0x7fa39419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:14.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.074+0000 7fa39add4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa394114480 con 0x7fa394108740 2024-05-09T04:52:14.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.074+0000 7fa39add4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa394114300 con 0x7fa394107df0 2024-05-09T04:52:14.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.074+0000 7fa39add4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa394114600 con 0x7fa3941034c0 2024-05-09T04:52:14.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.074+0000 7fa399371700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa394108740 0x7fa39419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:14.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.074+0000 7fa398b70700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3941034c0 0x7fa39419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:14.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.074+0000 7fa398b70700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3941034c0 0x7fa39419add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:44346/0 (socket says 172.21.15.46:44346) 2024-05-09T04:52:14.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.074+0000 7fa399371700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa394108740 0x7fa39419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55522/0 (socket says 172.21.15.46:55522) 2024-05-09T04:52:14.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.074+0000 7fa398b70700 1 -- 172.21.15.46:0/202277220 learned_addr learned my addr 172.21.15.46:0/202277220 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:14.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.074+0000 7fa393fff700 1 --2- 172.21.15.46:0/202277220 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa394107df0 0x7fa39419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:14.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.075+0000 7fa398b70700 1 -- 172.21.15.46:0/202277220 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa394107df0 msgr2=0x7fa39419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:14.077 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.075+0000 7fa398b70700 1 --2- 172.21.15.46:0/202277220 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa394107df0 0x7fa39419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:14.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.075+0000 7fa398b70700 1 -- 172.21.15.46:0/202277220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa394108740 msgr2=0x7fa39419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:14.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.075+0000 7fa398b70700 1 --2- 172.21.15.46:0/202277220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa394108740 0x7fa39419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:14.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.075+0000 7fa398b70700 1 -- 172.21.15.46:0/202277220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39419fca0 con 0x7fa3941034c0 2024-05-09T04:52:14.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.075+0000 7fa398b70700 1 --2- 172.21.15.46:0/202277220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3941034c0 0x7fa39419add0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa38800e9a0 tx=0x7fa38800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:14.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.075+0000 7fa391ffb700 1 -- 172.21.15.46:0/202277220 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38800cbd0 con 0x7fa3941034c0 2024-05-09T04:52:14.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.075+0000 7fa39add4700 1 -- 172.21.15.46:0/202277220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39419ff90 con 0x7fa3941034c0 2024-05-09T04:52:14.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.076+0000 7fa39add4700 1 -- 172.21.15.46:0/202277220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3941109b0 con 0x7fa3941034c0 2024-05-09T04:52:14.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.076+0000 7fa391ffb700 1 -- 172.21.15.46:0/202277220 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa38800cd70 con 0x7fa3941034c0 2024-05-09T04:52:14.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.076+0000 7fa391ffb700 1 -- 172.21.15.46:0/202277220 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa388018680 con 0x7fa3941034c0 2024-05-09T04:52:14.078 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.077+0000 7fa39add4700 1 -- 172.21.15.46:0/202277220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa394104970 con 0x7fa3941034c0 2024-05-09T04:52:14.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.079+0000 7fa391ffb700 1 -- 172.21.15.46:0/202277220 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fa388010430 con 0x7fa3941034c0 2024-05-09T04:52:14.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.080+0000 7fa391ffb700 1 --2- 172.21.15.46:0/202277220 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3840b2b10 0x7fa3840b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:14.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.080+0000 7fa393fff700 1 --2- 172.21.15.46:0/202277220 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3840b2b10 0x7fa3840b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:14.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.081+0000 7fa391ffb700 1 -- 172.21.15.46:0/202277220 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa3880e2b30 con 0x7fa3941034c0 2024-05-09T04:52:14.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.081+0000 7fa393fff700 1 --2- 172.21.15.46:0/202277220 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3840b2b10 0x7fa3840b5020 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa380000c00 tx=0x7fa38001a040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:14.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.086+0000 7fa391ffb700 1 -- 172.21.15.46:0/202277220 <== mon.1 v2:172.21.15.110:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3880aa2a0 con 0x7fa3941034c0 2024-05-09T04:52:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.552+0000 7fa39add4700 1 -- 172.21.15.46:0/202277220 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa39419c130 con 0x7fa3941034c0 2024-05-09T04:52:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.553+0000 7fa391ffb700 1 -- 172.21.15.46:0/202277220 <== mon.1 v2:172.21.15.110:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7fa3880b2320 con 0x7fa3941034c0 2024-05-09T04:52:14.553 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:52:14.553 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:52:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.556+0000 7fa39add4700 1 -- 172.21.15.46:0/202277220 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3840b2b10 msgr2=0x7fa3840b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.556+0000 7fa39add4700 1 --2- 172.21.15.46:0/202277220 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3840b2b10 0x7fa3840b5020 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa380000c00 tx=0x7fa38001a040 comp rx=0 tx=0).stop 2024-05-09T04:52:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.556+0000 7fa39add4700 1 -- 172.21.15.46:0/202277220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3941034c0 msgr2=0x7fa39419add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.556+0000 7fa39add4700 1 --2- 172.21.15.46:0/202277220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3941034c0 0x7fa39419add0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa38800e9a0 tx=0x7fa38800ecb0 comp rx=0 tx=0).stop 2024-05-09T04:52:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.557+0000 7fa39add4700 1 -- 172.21.15.46:0/202277220 shutdown_connections 2024-05-09T04:52:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.557+0000 7fa39add4700 1 --2- 172.21.15.46:0/202277220 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3840b2b10 0x7fa3840b5020 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.557+0000 7fa39add4700 1 --2- 172.21.15.46:0/202277220 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa394108740 0x7fa39419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.557+0000 7fa39add4700 1 --2- 172.21.15.46:0/202277220 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3941034c0 0x7fa39419add0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.557+0000 7fa39add4700 1 --2- 172.21.15.46:0/202277220 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa394107df0 0x7fa39419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.557+0000 7fa39add4700 1 -- 172.21.15.46:0/202277220 >> 172.21.15.46:0/202277220 conn(0x7fa3940fedc0 msgr2=0x7fa3941096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.557+0000 7fa39add4700 1 -- 172.21.15.46:0/202277220 shutdown_connections 2024-05-09T04:52:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:14.557+0000 7fa39add4700 1 -- 172.21.15.46:0/202277220 wait complete. 2024-05-09T04:52:14.558 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:52:14.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:14 smithi046 ceph-mon[30136]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T04:52:14.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:14 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/202277220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:52:15.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:14 smithi110 ceph-mon[34904]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T04:52:15.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:14 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/202277220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:52:15.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:14 smithi072 ceph-mon[34088]: pgmap v37: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T04:52:15.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:14 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/202277220' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:52:15.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:52:15.635 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:16.979 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:16 smithi046 ceph-mon[30136]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T04:52:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:16 smithi110 ceph-mon[34904]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T04:52:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:16 smithi072 ceph-mon[34088]: pgmap v38: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T04:52:17.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.648+0000 7f2dcbb2a700 1 -- 172.21.15.46:0/2255505121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc4102b30 msgr2=0x7f2dc4102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:17.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.648+0000 7f2dcbb2a700 1 --2- 172.21.15.46:0/2255505121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc4102b30 0x7f2dc4102f40 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2dc00099c0 tx=0x7f2dc0009cd0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.648+0000 7f2dcbb2a700 1 -- 172.21.15.46:0/2255505121 shutdown_connections 2024-05-09T04:52:17.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.648+0000 7f2dcbb2a700 1 --2- 172.21.15.46:0/2255505121 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2dc41098a0 0x7f2dc4111240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.648+0000 7f2dcbb2a700 1 --2- 172.21.15.46:0/2255505121 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc4102b30 0x7f2dc4102f40 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.648+0000 7f2dcbb2a700 1 --2- 172.21.15.46:0/2255505121 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dc4108f50 0x7f2dc4109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.648+0000 7f2dcbb2a700 1 -- 172.21.15.46:0/2255505121 >> 172.21.15.46:0/2255505121 conn(0x7f2dc40fe6f0 msgr2=0x7f2dc4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:17.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.648+0000 7f2dcbb2a700 1 -- 172.21.15.46:0/2255505121 shutdown_connections 2024-05-09T04:52:17.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.648+0000 7f2dcbb2a700 1 -- 172.21.15.46:0/2255505121 wait complete. 2024-05-09T04:52:17.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.649+0000 7f2dcbb2a700 1 Processor -- start 2024-05-09T04:52:17.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.649+0000 7f2dcbb2a700 1 -- start start 2024-05-09T04:52:17.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dcbb2a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dc4108f50 0x7f2dc419b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dcbb2a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc41098a0 0x7f2dc419b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dcbb2a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2dc419f820 0x7f2dc419fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dcbb2a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc4114580 con 0x7f2dc41098a0 2024-05-09T04:52:17.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dcbb2a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc4114400 con 0x7f2dc4108f50 2024-05-09T04:52:17.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dcbb2a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc4114700 con 0x7f2dc419f820 2024-05-09T04:52:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dc90c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc41098a0 0x7f2dc419b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dc90c5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc41098a0 0x7f2dc419b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55536/0 (socket says 172.21.15.46:55536) 2024-05-09T04:52:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dc90c5700 1 -- 172.21.15.46:0/2334911499 learned_addr learned my addr 172.21.15.46:0/2334911499 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dc98c6700 1 --2- 172.21.15.46:0/2334911499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dc4108f50 0x7f2dc419b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dca0c7700 1 --2- 172.21.15.46:0/2334911499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2dc419f820 0x7f2dc419fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dc90c5700 1 -- 172.21.15.46:0/2334911499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2dc419f820 msgr2=0x7f2dc419fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dc90c5700 1 --2- 172.21.15.46:0/2334911499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2dc419f820 0x7f2dc419fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dc90c5700 1 -- 172.21.15.46:0/2334911499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dc4108f50 msgr2=0x7f2dc419b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dc90c5700 1 --2- 172.21.15.46:0/2334911499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dc4108f50 0x7f2dc419b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.650+0000 7f2dc90c5700 1 -- 172.21.15.46:0/2334911499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2dc41a0370 con 0x7f2dc41098a0 2024-05-09T04:52:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.651+0000 7f2dc90c5700 1 --2- 172.21.15.46:0/2334911499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc41098a0 0x7f2dc419b650 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2db400e970 tx=0x7f2db400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.651+0000 7f2dbaffd700 1 -- 172.21.15.46:0/2334911499 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db400cbd0 con 0x7f2dc41098a0 2024-05-09T04:52:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.651+0000 7f2dbaffd700 1 -- 172.21.15.46:0/2334911499 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2db400cd70 con 0x7f2dc41098a0 2024-05-09T04:52:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.651+0000 7f2dbaffd700 1 -- 172.21.15.46:0/2334911499 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db4010430 con 0x7f2dc41098a0 2024-05-09T04:52:17.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.651+0000 7f2dcbb2a700 1 -- 172.21.15.46:0/2334911499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2dc41a0660 con 0x7f2dc41098a0 2024-05-09T04:52:17.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.651+0000 7f2dcbb2a700 1 -- 172.21.15.46:0/2334911499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2dc41a7df0 con 0x7f2dc41098a0 2024-05-09T04:52:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.655+0000 7f2dbaffd700 1 -- 172.21.15.46:0/2334911499 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f2db4004d70 con 0x7f2dc41098a0 2024-05-09T04:52:17.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.656+0000 7f2dda540b80 1 Processor -- start 2024-05-09T04:52:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.657+0000 7f2dda540b80 1 -- start start 2024-05-09T04:52:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.657+0000 7f2dbaffd700 1 --2- 172.21.15.46:0/2334911499 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f2db00b2b90 0x7f2db00b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.657+0000 7f2dda540b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c2940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.657+0000 7f2dda540b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd13b4a0 0x561fcd2c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.657+0000 7f2dca0c7700 1 --2- 172.21.15.46:0/2334911499 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f2db00b2b90 0x7f2db00b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.657+0000 7f2dda540b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd2bcab0 0x561fcd2bcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.657+0000 7f2dda540b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561fcd1daff0 con 0x561fcd2bcab0 2024-05-09T04:52:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.657+0000 7f2dda540b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x561fcd1797f0 con 0x561fcd225160 2024-05-09T04:52:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.657+0000 7f2dda540b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561fcd1d6fb0 con 0x561fcd13b4a0 2024-05-09T04:52:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.657+0000 7f2dbaffd700 1 -- 172.21.15.46:0/2334911499 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f2db40e2b90 con 0x7f2dc41098a0 2024-05-09T04:52:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.657+0000 7f2dc98c6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.657+0000 7f2dc98c6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:51382/0 (socket says 172.21.15.46:51382) 2024-05-09T04:52:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.657+0000 7f2dc98c6700 1 -- 172.21.15.46:0/3310426914 learned_addr learned my addr 172.21.15.46:0/3310426914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.658+0000 7f2dc90c5700 1 --2- 172.21.15.46:0/3310426914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd13b4a0 0x561fcd2c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.658+0000 7f2dc98c6700 1 -- 172.21.15.46:0/3310426914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd13b4a0 msgr2=0x561fcd2c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.658+0000 7f2dc98c6700 1 --2- 172.21.15.46:0/3310426914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd13b4a0 0x561fcd2c2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.658+0000 7f2dc98c6700 1 -- 172.21.15.46:0/3310426914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd2bcab0 msgr2=0x561fcd2bcec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T04:52:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.658+0000 7f2dc98c6700 1 --2- 172.21.15.46:0/3310426914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd2bcab0 0x561fcd2bcec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.658+0000 7f2dc98c6700 1 -- 172.21.15.46:0/3310426914 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fcd171370 con 0x561fcd225160 2024-05-09T04:52:17.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.658+0000 7f2dc90c5700 1 --2- 172.21.15.46:0/3310426914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd13b4a0 0x561fcd2c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:52:17.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.658+0000 7f2dc98c6700 1 --2- 172.21.15.46:0/3310426914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c2940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0000c00 tx=0x7f2dc0005040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f625f25a920b74d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:17.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.658+0000 7f2daffff700 1 -- 172.21.15.46:0/3310426914 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc00052b0 con 0x561fcd225160 2024-05-09T04:52:17.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.658+0000 7f2daffff700 1 -- 172.21.15.46:0/3310426914 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dc0009590 con 0x561fcd225160 2024-05-09T04:52:17.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.658+0000 7f2daffff700 1 -- 172.21.15.46:0/3310426914 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc000b9f0 con 0x561fcd225160 2024-05-09T04:52:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.659+0000 7f2dda540b80 1 -- 172.21.15.46:0/3310426914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 msgr2=0x561fcd2c2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.659+0000 7f2dca0c7700 1 --2- 172.21.15.46:0/2334911499 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f2db00b2b90 0x7f2db00b50a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc009380 tx=0x7f2dbc00b040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.659+0000 7f2dda540b80 1 --2- 172.21.15.46:0/3310426914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c2940 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0000c00 tx=0x7f2dc0005040 comp rx=0 tx=0).stop 2024-05-09T04:52:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.659+0000 7f2dda540b80 1 -- 172.21.15.46:0/3310426914 shutdown_connections 2024-05-09T04:52:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.659+0000 7f2dda540b80 1 --2- 172.21.15.46:0/3310426914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd2bcab0 0x561fcd2bcec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.659+0000 7f2dda540b80 1 --2- 172.21.15.46:0/3310426914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c2940 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.659+0000 7f2dda540b80 1 --2- 172.21.15.46:0/3310426914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd13b4a0 0x561fcd2c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.659+0000 7f2dda540b80 1 -- 172.21.15.46:0/3310426914 >> 172.21.15.46:0/3310426914 conn(0x561fcd19b6a0 msgr2=0x561fcd2b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.660+0000 7f2dda540b80 1 -- 172.21.15.46:0/3310426914 shutdown_connections 2024-05-09T04:52:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.660+0000 7f2dda540b80 1 -- 172.21.15.46:0/3310426914 wait complete. 2024-05-09T04:52:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.660+0000 7f2dda540b80 1 -- start start 2024-05-09T04:52:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.660+0000 7f2dda540b80 1 Processor -- start 2024-05-09T04:52:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.661+0000 7f2dda540b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2c65d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.661+0000 7f2dc98c6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2c65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.661+0000 7f2dc98c6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2c65d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55542/0 (socket says 172.21.15.46:55542) 2024-05-09T04:52:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.661+0000 7f2dc98c6700 1 -- 172.21.15.46:0/3054683004 learned_addr learned my addr 172.21.15.46:0/3054683004 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.662+0000 7f2dda540b80 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.662+0000 7f2dda540b80 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd2bcab0 0x561fcd2c7050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.662+0000 7f2dda540b80 1 -- 172.21.15.46:0/3054683004 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561fcd056ba0 con 0x561fcd13b4a0 2024-05-09T04:52:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.662+0000 7f2dc90c5700 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.662+0000 7f2dda540b80 1 -- 172.21.15.46:0/3054683004 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x561fcd195150 con 0x561fcd225160 2024-05-09T04:52:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.662+0000 7f2dda540b80 1 -- 172.21.15.46:0/3054683004 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561fcd2bd820 con 0x561fcd2bcab0 2024-05-09T04:52:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.662+0000 7f2dca0c7700 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd2bcab0 0x561fcd2c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.663+0000 7f2dca0c7700 1 -- 172.21.15.46:0/3054683004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 msgr2=0x561fcd2c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:17.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.663+0000 7f2dca0c7700 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.663+0000 7f2dca0c7700 1 -- 172.21.15.46:0/3054683004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 msgr2=0x561fcd2c65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:17.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.663+0000 7f2dda540b80 10 client.0 _ll_register_callbacks cb 0x561fcd04c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:17.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.663+0000 7f2dca0c7700 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2c65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.663+0000 7f2dca0c7700 1 -- 172.21.15.46:0/3054683004 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fcd056490 con 0x561fcd2bcab0 2024-05-09T04:52:17.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.663+0000 7f2dda540b80 1 -- 172.21.15.46:0/3054683004 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2dc0009590 con 0x561fcd2bcab0 2024-05-09T04:52:17.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.663+0000 7f2dc98c6700 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:17.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.664+0000 7f2dca0c7700 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd2bcab0 0x561fcd2c7050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc015e10 tx=0x7f2dbc004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d48725c49782cfe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.664+0000 7f2dad7fa700 10 client.25261 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:52:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.664+0000 7f2dad7fa700 1 -- 172.21.15.46:0/3054683004 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc01c050 con 0x561fcd2bcab0 2024-05-09T04:52:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.664+0000 7f2dad7fa700 1 -- 172.21.15.46:0/3054683004 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f2dbc016c90 con 0x561fcd2bcab0 2024-05-09T04:52:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.664+0000 7f2dad7fa700 1 -- 172.21.15.46:0/3054683004 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2dbc015e40 con 0x561fcd2bcab0 2024-05-09T04:52:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.664+0000 7f2dda540b80 10 client.25261 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.664+0000 7f2dda540b80 1 -- 172.21.15.46:0/3054683004 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=25}) -- 0x561fcd2d3ce0 con 0x561fcd2bcab0 2024-05-09T04:52:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.665+0000 7f2dad7fa700 1 -- 172.21.15.46:0/3054683004 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2dbc021440 con 0x561fcd2bcab0 2024-05-09T04:52:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.665+0000 7f2dda540b80 10 client.25261 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.665+0000 7f2dda540b80 10 client.25261 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:52:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.665+0000 7f2dda540b80 20 client.25261 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.665+0000 7f2dda540b80 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x561fcd2d6210 0x561fcd2d8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.665+0000 7f2dda540b80 4 client.25261 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.665+0000 7f2dda540b80 1 -- 172.21.15.46:0/3054683004 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561fcd1e89d0 con 0x561fcd2d6210 2024-05-09T04:52:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.665+0000 7f2dc98c6700 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x561fcd2d6210 0x561fcd2d8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.666+0000 7f2dad7fa700 10 client.25261 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:52:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.666+0000 7f2dc98c6700 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x561fcd2d6210 0x561fcd2d8670 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1fc6fa2e818ac0d6 server_cookie=75a56ebb297db59f in_seq=0 out_seq=0 2024-05-09T04:52:17.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.667+0000 7f2dad7fa700 1 -- 172.21.15.46:0/3054683004 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561fcd1e89d0 con 0x561fcd2d6210 2024-05-09T04:52:17.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.667+0000 7f2dad7fa700 10 client.25261 handle_command_reply: tid=0 2024-05-09T04:52:17.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.667+0000 7f2dda540b80 1 client.25261 shutdown 2024-05-09T04:52:17.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.667+0000 7f2dda540b80 1 -- 172.21.15.46:0/3054683004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd2bcab0 msgr2=0x561fcd2c7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:17.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.667+0000 7f2dda540b80 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd2bcab0 0x561fcd2c7050 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc015e10 tx=0x7f2dbc004ad0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.667+0000 7f2dda540b80 1 -- 172.21.15.46:0/3054683004 shutdown_connections 2024-05-09T04:52:17.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.667+0000 7f2dda540b80 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x561fcd2d6210 0x561fcd2d8670 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.667+0000 7f2dda540b80 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd2bcab0 0x561fcd2c7050 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.667+0000 7f2dda540b80 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2c65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.667+0000 7f2dda540b80 1 --2- 172.21.15.46:0/3054683004 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.667+0000 7f2dad7fa700 0 client.25261 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:52:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.667+0000 7f2dda540b80 1 -- 172.21.15.46:0/3054683004 >> 172.21.15.46:0/3054683004 conn(0x561fcd19b6a0 msgr2=0x561fcd2b31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.668+0000 7f2dda540b80 1 -- 172.21.15.46:0/3054683004 shutdown_connections 2024-05-09T04:52:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.668+0000 7f2dda540b80 1 -- 172.21.15.46:0/3054683004 wait complete. 2024-05-09T04:52:17.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.668+0000 7f2dda540b80 20 client.25261 trim_cache size 0 max 16384 2024-05-09T04:52:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.696+0000 7f2dda540b80 1 Processor -- start 2024-05-09T04:52:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.696+0000 7f2dda540b80 1 -- start start 2024-05-09T04:52:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.696+0000 7f2dda540b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2c1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.696+0000 7f2dda540b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.696+0000 7f2dda540b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd2bcab0 0x561fcd2d22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.696+0000 7f2dda540b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc41afc10 con 0x561fcd13b4a0 2024-05-09T04:52:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.696+0000 7f2dda540b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc41ad4a0 con 0x561fcd225160 2024-05-09T04:52:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.696+0000 7f2dda540b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc41ab2f0 con 0x561fcd2bcab0 2024-05-09T04:52:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.696+0000 7f2dc90c5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.696+0000 7f2dc98c6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2c1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.696+0000 7f2dc90c5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:51412/0 (socket says 172.21.15.46:51412) 2024-05-09T04:52:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.696+0000 7f2dc90c5700 1 -- 172.21.15.46:0/2521781477 learned_addr learned my addr 172.21.15.46:0/2521781477 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.696+0000 7f2dca0c7700 1 --2- 172.21.15.46:0/2521781477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd2bcab0 0x561fcd2d22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.697+0000 7f2dc90c5700 1 -- 172.21.15.46:0/2521781477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd2bcab0 msgr2=0x561fcd2d22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.697+0000 7f2dc90c5700 1 --2- 172.21.15.46:0/2521781477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd2bcab0 0x561fcd2d22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.697+0000 7f2dc90c5700 1 -- 172.21.15.46:0/2521781477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 msgr2=0x561fcd2c1f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.697+0000 7f2dc90c5700 1 --2- 172.21.15.46:0/2521781477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2c1f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.697+0000 7f2dc90c5700 1 -- 172.21.15.46:0/2521781477 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561fcd2acf60 con 0x561fcd225160 2024-05-09T04:52:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.697+0000 7f2dca0c7700 1 --2- 172.21.15.46:0/2521781477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd2bcab0 0x561fcd2d22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.697+0000 7f2dc98c6700 1 --2- 172.21.15.46:0/2521781477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2c1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.697+0000 7f2dc90c5700 1 --2- 172.21.15.46:0/2521781477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c6e90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2db40e0250 tx=0x7f2db40a9470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=751d468ad591458e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.697+0000 7f2daf7fe700 1 -- 172.21.15.46:0/2521781477 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db40c42e0 con 0x561fcd225160 2024-05-09T04:52:17.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.698+0000 7f2daf7fe700 1 -- 172.21.15.46:0/2521781477 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2db40b23a0 con 0x561fcd225160 2024-05-09T04:52:17.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.698+0000 7f2daf7fe700 1 -- 172.21.15.46:0/2521781477 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2db40aa0c0 con 0x561fcd225160 2024-05-09T04:52:17.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.699+0000 7f2dda540b80 1 -- 172.21.15.46:0/2521781477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 msgr2=0x561fcd2c6e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:17.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.699+0000 7f2dda540b80 1 --2- 172.21.15.46:0/2521781477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c6e90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2db40e0250 tx=0x7f2db40a9470 comp rx=0 tx=0).stop 2024-05-09T04:52:17.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.699+0000 7f2dda540b80 1 -- 172.21.15.46:0/2521781477 shutdown_connections 2024-05-09T04:52:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.699+0000 7f2dda540b80 1 --2- 172.21.15.46:0/2521781477 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd2bcab0 0x561fcd2d22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.699+0000 7f2dda540b80 1 --2- 172.21.15.46:0/2521781477 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2c1f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.699+0000 7f2dda540b80 1 --2- 172.21.15.46:0/2521781477 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd225160 0x561fcd2c6e90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.699+0000 7f2dda540b80 1 -- 172.21.15.46:0/2521781477 >> 172.21.15.46:0/2521781477 conn(0x561fcd19b6a0 msgr2=0x561fcd2c1840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.699+0000 7f2dda540b80 1 -- 172.21.15.46:0/2521781477 shutdown_connections 2024-05-09T04:52:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.699+0000 7f2dda540b80 1 -- 172.21.15.46:0/2521781477 wait complete. 2024-05-09T04:52:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.699+0000 7f2dda540b80 1 -- start start 2024-05-09T04:52:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.700+0000 7f2dda540b80 1 Processor -- start 2024-05-09T04:52:17.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.700+0000 7f2dda540b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2ce500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.701+0000 7f2dc98c6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2ce500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.701+0000 7f2dc98c6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2ce500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55572/0 (socket says 172.21.15.46:55572) 2024-05-09T04:52:17.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.701+0000 7f2dc98c6700 1 -- 172.21.15.46:0/2983304047 learned_addr learned my addr 172.21.15.46:0/2983304047 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:17.701 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.701+0000 7f2dda540b80 1 --2- 172.21.15.46:0/2983304047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd225160 0x561fcd2cea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.701+0000 7f2dda540b80 1 --2- 172.21.15.46:0/2983304047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd2bcab0 0x561fcd2dce20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.701+0000 7f2dda540b80 1 -- 172.21.15.46:0/2983304047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc41ab960 con 0x561fcd13b4a0 2024-05-09T04:52:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.701+0000 7f2dda540b80 1 -- 172.21.15.46:0/2983304047 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc41acdd0 con 0x561fcd2bcab0 2024-05-09T04:52:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.701+0000 7f2dda540b80 1 -- 172.21.15.46:0/2983304047 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2dc41ac030 con 0x561fcd225160 2024-05-09T04:52:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.701+0000 7f2dca0c7700 1 --2- 172.21.15.46:0/2983304047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd2bcab0 0x561fcd2dce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.702 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.701+0000 7f2dc90c5700 1 --2- 172.21.15.46:0/2983304047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd225160 0x561fcd2cea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.701+0000 7f2dc98c6700 1 -- 172.21.15.46:0/2983304047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd225160 msgr2=0x561fcd2cea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.701+0000 7f2dc98c6700 1 --2- 172.21.15.46:0/2983304047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd225160 0x561fcd2cea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.702+0000 7f2dc98c6700 1 -- 172.21.15.46:0/2983304047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd2bcab0 msgr2=0x561fcd2dce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.702+0000 7f2dc98c6700 1 --2- 172.21.15.46:0/2983304047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd2bcab0 0x561fcd2dce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.702+0000 7f2dda540b80 10 client.0 _ll_register_callbacks cb 0x561fcd04c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.702+0000 7f2dc98c6700 1 -- 172.21.15.46:0/2983304047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561fcd2dd360 con 0x561fcd13b4a0 2024-05-09T04:52:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.702+0000 7f2dda540b80 1 -- 172.21.15.46:0/2983304047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x561fcd1e2a70 con 0x561fcd13b4a0 2024-05-09T04:52:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.702+0000 7f2dc98c6700 1 --2- 172.21.15.46:0/2983304047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2ce500 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0000c00 tx=0x7f2dc00039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eaec27c4c0c5bcd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.702+0000 7f2dad7fa700 10 client.15555 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.702+0000 7f2dad7fa700 1 -- 172.21.15.46:0/2983304047 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dc0003cd0 con 0x561fcd13b4a0 2024-05-09T04:52:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.702+0000 7f2dad7fa700 1 -- 172.21.15.46:0/2983304047 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f2dc000b730 con 0x561fcd13b4a0 2024-05-09T04:52:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.702+0000 7f2dad7fa700 1 -- 172.21.15.46:0/2983304047 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2dc000b930 con 0x561fcd13b4a0 2024-05-09T04:52:17.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.702+0000 7f2dda540b80 10 client.15555 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:17.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.702+0000 7f2dda540b80 1 -- 172.21.15.46:0/2983304047 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=25}) -- 0x561fcd33ffb0 con 0x561fcd13b4a0 2024-05-09T04:52:17.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.703+0000 7f2dad7fa700 1 -- 172.21.15.46:0/2983304047 <== mon.0 v2:172.21.15.46:3300/0 4 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2dc000bad0 con 0x561fcd13b4a0 2024-05-09T04:52:17.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.703+0000 7f2dda540b80 10 client.15555 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:17.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.703+0000 7f2dda540b80 10 client.15555 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:52:17.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.703+0000 7f2dda540b80 20 client.15555 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:17.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.703+0000 7f2dda540b80 1 --2- 172.21.15.46:0/2983304047 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x561fcd340180 0x561fcd3425a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:17.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.703+0000 7f2dda540b80 4 client.15555 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:52:17.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.703+0000 7f2dda540b80 1 -- 172.21.15.46:0/2983304047 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561fcd056970 con 0x561fcd340180 2024-05-09T04:52:17.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.703+0000 7f2dc90c5700 1 --2- 172.21.15.46:0/2983304047 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x561fcd340180 0x561fcd3425a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:17.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.704+0000 7f2dad7fa700 10 client.15555 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:52:17.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.704+0000 7f2dc90c5700 1 --2- 172.21.15.46:0/2983304047 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x561fcd340180 0x561fcd3425a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8dc62dcd64bb3fa server_cookie=a937a34a62a70dcd in_seq=0 out_seq=0 2024-05-09T04:52:17.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.704+0000 7f2dad7fa700 1 -- 172.21.15.46:0/2983304047 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561fcd056970 con 0x561fcd340180 2024-05-09T04:52:17.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.704+0000 7f2dad7fa700 10 client.15555 handle_command_reply: tid=0 2024-05-09T04:52:17.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.704+0000 7f2dda540b80 1 client.15555 shutdown 2024-05-09T04:52:17.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.704+0000 7f2dda540b80 1 -- 172.21.15.46:0/2983304047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 msgr2=0x561fcd2ce500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:17.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.704+0000 7f2dda540b80 1 --2- 172.21.15.46:0/2983304047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2ce500 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0000c00 tx=0x7f2dc00039f0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.704+0000 7f2dda540b80 1 -- 172.21.15.46:0/2983304047 shutdown_connections 2024-05-09T04:52:17.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.704+0000 7f2dda540b80 1 --2- 172.21.15.46:0/2983304047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561fcd2bcab0 0x561fcd2dce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.704+0000 7f2dda540b80 1 --2- 172.21.15.46:0/2983304047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561fcd13b4a0 0x561fcd2ce500 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.704+0000 7f2dda540b80 1 --2- 172.21.15.46:0/2983304047 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x561fcd340180 0x561fcd3425a0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.704+0000 7f2dda540b80 1 --2- 172.21.15.46:0/2983304047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561fcd225160 0x561fcd2cea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.704+0000 7f2dad7fa700 0 client.15555 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:52:17.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.704+0000 7f2dda540b80 1 -- 172.21.15.46:0/2983304047 >> 172.21.15.46:0/2983304047 conn(0x561fcd19b6a0 msgr2=0x561fcd2c4ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:17.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.705+0000 7f2dda540b80 1 -- 172.21.15.46:0/2983304047 shutdown_connections 2024-05-09T04:52:17.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.705+0000 7f2dda540b80 1 -- 172.21.15.46:0/2983304047 wait complete. 2024-05-09T04:52:17.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.705+0000 7f2dda540b80 20 client.15555 trim_cache size 0 max 16384 2024-05-09T04:52:17.707 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:52:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.706+0000 7f2dcbb2a700 1 -- 172.21.15.46:0/2334911499 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f2db00b2b90 msgr2=0x7f2db00b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.706+0000 7f2dcbb2a700 1 --2- 172.21.15.46:0/2334911499 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f2db00b2b90 0x7f2db00b50a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc009380 tx=0x7f2dbc00b040 comp rx=0 tx=0).stop 2024-05-09T04:52:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.706+0000 7f2dcbb2a700 1 -- 172.21.15.46:0/2334911499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc41098a0 msgr2=0x7f2dc419b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.706+0000 7f2dcbb2a700 1 --2- 172.21.15.46:0/2334911499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc41098a0 0x7f2dc419b650 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2db400e970 tx=0x7f2db400ed30 comp rx=0 tx=0).stop 2024-05-09T04:52:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.706+0000 7f2dcbb2a700 1 -- 172.21.15.46:0/2334911499 shutdown_connections 2024-05-09T04:52:17.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.706+0000 7f2dcbb2a700 1 --2- 172.21.15.46:0/2334911499 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f2db00b2b90 0x7f2db00b50a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.706+0000 7f2dcbb2a700 1 --2- 172.21.15.46:0/2334911499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2dc419f820 0x7f2dc419fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.706+0000 7f2dcbb2a700 1 --2- 172.21.15.46:0/2334911499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2dc4108f50 0x7f2dc419b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.706+0000 7f2dcbb2a700 1 --2- 172.21.15.46:0/2334911499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2dc41098a0 0x7f2dc419b650 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.706+0000 7f2dcbb2a700 1 -- 172.21.15.46:0/2334911499 >> 172.21.15.46:0/2334911499 conn(0x7f2dc40fe6f0 msgr2=0x7f2dc4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.707+0000 7f2dcbb2a700 1 -- 172.21.15.46:0/2334911499 shutdown_connections 2024-05-09T04:52:17.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:17.707+0000 7f2dcbb2a700 1 -- 172.21.15.46:0/2334911499 wait complete. 2024-05-09T04:52:18.020 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:17 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:52:18.020 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:17 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:52:18.020 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:17 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:52:18.020 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:17 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:52:18.031 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:17 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:52:18.031 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:17 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:52:18.031 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:17 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:52:18.031 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:17 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:52:18.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:17 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:52:18.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:17 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:52:18.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:17 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:52:18.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:17 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:52:18.390 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:52:18.694 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:18 smithi046 ceph-mon[30136]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-09T04:52:18.725 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:18 smithi110 ceph-mon[34904]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-09T04:52:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:18 smithi072 ceph-mon[34088]: pgmap v39: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-09T04:52:20.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.668+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/3544845415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c1029c0 msgr2=0x7f4f9c102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:20.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.668+0000 7f4fa0c2a700 1 --2- 172.21.15.46:0/3544845415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c1029c0 0x7f4f9c102dd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f4f9000aaa0 tx=0x7f4f9000adb0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.669+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/3544845415 shutdown_connections 2024-05-09T04:52:20.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.669+0000 7f4fa0c2a700 1 --2- 172.21.15.46:0/3544845415 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f9c103310 0x7f4f9c10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.669+0000 7f4fa0c2a700 1 --2- 172.21.15.46:0/3544845415 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f9c1089f0 0x7f4f9c108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.669+0000 7f4fa0c2a700 1 --2- 172.21.15.46:0/3544845415 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c1029c0 0x7f4f9c102dd0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.669+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/3544845415 >> 172.21.15.46:0/3544845415 conn(0x7f4f9c0fe6f0 msgr2=0x7f4f9c100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:20.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.669+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/3544845415 shutdown_connections 2024-05-09T04:52:20.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.669+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/3544845415 wait complete. 2024-05-09T04:52:20.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.671+0000 7f4fa0c2a700 1 Processor -- start 2024-05-09T04:52:20.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.671+0000 7f4fa0c2a700 1 -- start start 2024-05-09T04:52:20.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.671+0000 7f4fa0c2a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f9c1029c0 0x7f4f9c19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.671+0000 7f4fa0c2a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c103310 0x7f4f9c19ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.672+0000 7f4f99d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c103310 0x7f4f9c19ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.672+0000 7f4f9a59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f9c1029c0 0x7f4f9c19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.672+0000 7f4f9a59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f9c1029c0 0x7f4f9c19e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:51436/0 (socket says 172.21.15.46:51436) 2024-05-09T04:52:20.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.672+0000 7f4f99d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c103310 0x7f4f9c19ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55598/0 (socket says 172.21.15.46:55598) 2024-05-09T04:52:20.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.672+0000 7f4f9a59c700 1 -- 172.21.15.46:0/1429891914 learned_addr learned my addr 172.21.15.46:0/1429891914 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:20.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.672+0000 7f4fa0c2a700 1 --2- 172.21.15.46:0/1429891914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f9c1089f0 0x7f4f9c1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.672+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/1429891914 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c112340 con 0x7f4f9c103310 2024-05-09T04:52:20.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.672+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/1429891914 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1121c0 con 0x7f4f9c1029c0 2024-05-09T04:52:20.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.672+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/1429891914 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1124c0 con 0x7f4f9c1089f0 2024-05-09T04:52:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.672+0000 7f4f9ad9d700 1 --2- 172.21.15.46:0/1429891914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f9c1089f0 0x7f4f9c1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.673+0000 7f4f9ad9d700 1 -- 172.21.15.46:0/1429891914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f9c1029c0 msgr2=0x7f4f9c19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.673+0000 7f4f9ad9d700 1 --2- 172.21.15.46:0/1429891914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f9c1029c0 0x7f4f9c19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.673+0000 7f4f9ad9d700 1 -- 172.21.15.46:0/1429891914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c103310 msgr2=0x7f4f9c19ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.673+0000 7f4f9ad9d700 1 --2- 172.21.15.46:0/1429891914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c103310 0x7f4f9c19ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.673+0000 7f4f9ad9d700 1 -- 172.21.15.46:0/1429891914 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f9c198f70 con 0x7f4f9c1089f0 2024-05-09T04:52:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.673+0000 7f4f99d9b700 1 --2- 172.21.15.46:0/1429891914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c103310 0x7f4f9c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.673+0000 7f4f9ad9d700 1 --2- 172.21.15.46:0/1429891914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f9c1089f0 0x7f4f9c1988c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4f94007f10 tx=0x7f4f9400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.674+0000 7f4f8b7fe700 1 -- 172.21.15.46:0/1429891914 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94004db0 con 0x7f4f9c1089f0 2024-05-09T04:52:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.674+0000 7f4f8b7fe700 1 -- 172.21.15.46:0/1429891914 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f94009e20 con 0x7f4f9c1089f0 2024-05-09T04:52:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.674+0000 7f4f8b7fe700 1 -- 172.21.15.46:0/1429891914 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f94015400 con 0x7f4f9c1089f0 2024-05-09T04:52:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.674+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/1429891914 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f9c199260 con 0x7f4f9c1089f0 2024-05-09T04:52:20.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.674+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/1429891914 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f9c199690 con 0x7f4f9c1089f0 2024-05-09T04:52:20.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.678+0000 7f4f8b7fe700 1 -- 172.21.15.46:0/1429891914 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f4f940155a0 con 0x7f4f9c1089f0 2024-05-09T04:52:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.679+0000 7f4faf640b80 1 Processor -- start 2024-05-09T04:52:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.679+0000 7f4f8b7fe700 1 --2- 172.21.15.46:0/1429891914 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4f840b2b10 0x7f4f840b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.680+0000 7f4faf640b80 1 -- start start 2024-05-09T04:52:20.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.680+0000 7f4f8b7fe700 1 -- 172.21.15.46:0/1429891914 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f4f940e22a0 con 0x7f4f9c1089f0 2024-05-09T04:52:20.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.680+0000 7f4f99d9b700 1 --2- 172.21.15.46:0/1429891914 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4f840b2b10 0x7f4f840b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4f99d9b700 1 --2- 172.21.15.46:0/1429891914 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4f840b2b10 0x7f4f840b5020 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c102090 tx=0x7f4f9001f040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4faf640b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9d5fd00 0x5622b9e68240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4faf640b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9ce54a0 0x5622b9e68780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4faf640b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9e62400 0x5622b9e62810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4faf640b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5622b9d850a0 con 0x5622b9e62400 2024-05-09T04:52:20.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4faf640b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5622b9d2a9a0 con 0x5622b9d5fd00 2024-05-09T04:52:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4faf640b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5622b9d81060 con 0x5622b9ce54a0 2024-05-09T04:52:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4f9ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9e62400 0x5622b9e62810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4f9a59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9d5fd00 0x5622b9e68240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4f9a59c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9d5fd00 0x5622b9e68240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:51448/0 (socket says 172.21.15.46:51448) 2024-05-09T04:52:20.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4f9ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9e62400 0x5622b9e62810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55602/0 (socket says 172.21.15.46:55602) 2024-05-09T04:52:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4f9a59c700 1 -- 172.21.15.46:0/3788264060 learned_addr learned my addr 172.21.15.46:0/3788264060 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4f99d9b700 1 --2- 172.21.15.46:0/3788264060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9ce54a0 0x5622b9e68780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4f9a59c700 1 -- 172.21.15.46:0/3788264060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9ce54a0 msgr2=0x5622b9e68780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:20.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4f9a59c700 1 --2- 172.21.15.46:0/3788264060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9ce54a0 0x5622b9e68780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4f9a59c700 1 -- 172.21.15.46:0/3788264060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9e62400 msgr2=0x5622b9e62810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4f9a59c700 1 --2- 172.21.15.46:0/3788264060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9e62400 0x5622b9e62810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4f9a59c700 1 -- 172.21.15.46:0/3788264060 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622b9d1b500 con 0x5622b9d5fd00 2024-05-09T04:52:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.681+0000 7f4f9ad9d700 1 --2- 172.21.15.46:0/3788264060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9e62400 0x5622b9e62810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:52:20.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.682+0000 7f4f9a59c700 1 --2- 172.21.15.46:0/3788264060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9d5fd00 0x5622b9e68240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00dd80 tx=0x7f4f8c00c2f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cc375021560dbda server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.682+0000 7f4f88ff9700 1 -- 172.21.15.46:0/3788264060 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c01a030 con 0x5622b9d5fd00 2024-05-09T04:52:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.682+0000 7f4f88ff9700 1 -- 172.21.15.46:0/3788264060 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f8c00d020 con 0x5622b9d5fd00 2024-05-09T04:52:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.682+0000 7f4f88ff9700 1 -- 172.21.15.46:0/3788264060 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c00cbb0 con 0x5622b9d5fd00 2024-05-09T04:52:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.683+0000 7f4faf640b80 1 -- 172.21.15.46:0/3788264060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9d5fd00 msgr2=0x5622b9e68240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.683+0000 7f4faf640b80 1 --2- 172.21.15.46:0/3788264060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9d5fd00 0x5622b9e68240 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00dd80 tx=0x7f4f8c00c2f0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.683+0000 7f4faf640b80 1 -- 172.21.15.46:0/3788264060 shutdown_connections 2024-05-09T04:52:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.683+0000 7f4faf640b80 1 --2- 172.21.15.46:0/3788264060 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9e62400 0x5622b9e62810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.683+0000 7f4faf640b80 1 --2- 172.21.15.46:0/3788264060 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9d5fd00 0x5622b9e68240 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.683+0000 7f4faf640b80 1 --2- 172.21.15.46:0/3788264060 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9ce54a0 0x5622b9e68780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.683+0000 7f4faf640b80 1 -- 172.21.15.46:0/3788264060 >> 172.21.15.46:0/3788264060 conn(0x5622b9ce8f70 msgr2=0x5622b9e5d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:20.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.683+0000 7f4faf640b80 1 -- 172.21.15.46:0/3788264060 shutdown_connections 2024-05-09T04:52:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.683+0000 7f4faf640b80 1 -- 172.21.15.46:0/3788264060 wait complete. 2024-05-09T04:52:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.684+0000 7f4faf640b80 1 -- start start 2024-05-09T04:52:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.684+0000 7f4faf640b80 1 Processor -- start 2024-05-09T04:52:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4faf640b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9ce54a0 0x5622b9e6bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4faf640b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9d5fd00 0x5622b9e6c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4faf640b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9e62400 0x5622b9e6ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4faf640b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5622b9e63200 con 0x5622b9ce54a0 2024-05-09T04:52:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4faf640b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5622b9d1fb20 con 0x5622b9e62400 2024-05-09T04:52:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4faf640b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5622b9e6f020 con 0x5622b9d5fd00 2024-05-09T04:52:20.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4faf640b80 10 client.0 _ll_register_callbacks cb 0x5622b9bf66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:20.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4f9a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9ce54a0 0x5622b9e6bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4f9a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9ce54a0 0x5622b9e6bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55610/0 (socket says 172.21.15.46:55610) 2024-05-09T04:52:20.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4f9a59c700 1 -- 172.21.15.46:0/2276572033 learned_addr learned my addr 172.21.15.46:0/2276572033 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:20.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4f9ad9d700 1 --2- 172.21.15.46:0/2276572033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9e62400 0x5622b9e6ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4f99d9b700 1 --2- 172.21.15.46:0/2276572033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9d5fd00 0x5622b9e6c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4f9ad9d700 1 -- 172.21.15.46:0/2276572033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9d5fd00 msgr2=0x5622b9e6c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:20.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4f9ad9d700 1 --2- 172.21.15.46:0/2276572033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9d5fd00 0x5622b9e6c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4f9ad9d700 1 -- 172.21.15.46:0/2276572033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9ce54a0 msgr2=0x5622b9e6bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:20.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4f9ad9d700 1 --2- 172.21.15.46:0/2276572033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9ce54a0 0x5622b9e6bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4f9ad9d700 1 -- 172.21.15.46:0/2276572033 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622b9c00490 con 0x5622b9e62400 2024-05-09T04:52:20.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.685+0000 7f4f9a59c700 1 --2- 172.21.15.46:0/2276572033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9ce54a0 0x5622b9e6bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:20.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.686+0000 7f4f9ad9d700 1 --2- 172.21.15.46:0/2276572033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9e62400 0x5622b9e6ca70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4f940df850 tx=0x7f4f94005560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed2e052e1bf7ceb8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:20.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.686+0000 7f4f81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:52:20.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.686+0000 7f4f81ffb700 1 -- 172.21.15.46:0/2276572033 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f940a9310 con 0x5622b9e62400 2024-05-09T04:52:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.686+0000 7f4faf640b80 1 -- 172.21.15.46:0/2276572033 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f8c00d020 con 0x5622b9e62400 2024-05-09T04:52:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.686+0000 7f4f81ffb700 1 -- 172.21.15.46:0/2276572033 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f4f940b1ef0 con 0x5622b9e62400 2024-05-09T04:52:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.686+0000 7f4f81ffb700 1 -- 172.21.15.46:0/2276572033 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f940ad6c0 con 0x5622b9e62400 2024-05-09T04:52:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.686+0000 7f4faf640b80 10 client.25328 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.686+0000 7f4faf640b80 1 -- 172.21.15.46:0/2276572033 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5622b9e89330 con 0x5622b9e62400 2024-05-09T04:52:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.686+0000 7f4f81ffb700 1 -- 172.21.15.46:0/2276572033 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f8c00d020 con 0x5622b9e62400 2024-05-09T04:52:20.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.686+0000 7f4f81ffb700 1 -- 172.21.15.46:0/2276572033 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f4f940ad160 con 0x5622b9e62400 2024-05-09T04:52:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.686+0000 7f4faf640b80 10 client.25328 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.687+0000 7f4faf640b80 10 client.25328 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:52:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.687+0000 7f4faf640b80 20 client.25328 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.687+0000 7f4faf640b80 1 --2- 172.21.15.46:0/2276572033 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5622b9e8b8c0 0x5622b9e8dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.687+0000 7f4faf640b80 4 client.25328 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.687+0000 7f4faf640b80 1 -- 172.21.15.46:0/2276572033 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622b9d92a80 con 0x5622b9e8b8c0 2024-05-09T04:52:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.687+0000 7f4f9a59c700 1 --2- 172.21.15.46:0/2276572033 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5622b9e8b8c0 0x5622b9e8dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.687+0000 7f4f81ffb700 10 client.25328 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:52:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.688+0000 7f4f9a59c700 1 --2- 172.21.15.46:0/2276572033 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5622b9e8b8c0 0x5622b9e8dce0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17bb68594292505e server_cookie=9ae18d0f30641f65 in_seq=0 out_seq=0 2024-05-09T04:52:20.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.689+0000 7f4f81ffb700 1 -- 172.21.15.46:0/2276572033 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5622b9d92a80 con 0x5622b9e8b8c0 2024-05-09T04:52:20.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.689+0000 7f4f81ffb700 10 client.25328 handle_command_reply: tid=0 2024-05-09T04:52:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.689+0000 7f4faf640b80 1 client.25328 shutdown 2024-05-09T04:52:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.689+0000 7f4faf640b80 1 -- 172.21.15.46:0/2276572033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9e62400 msgr2=0x5622b9e6ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.689+0000 7f4faf640b80 1 --2- 172.21.15.46:0/2276572033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9e62400 0x5622b9e6ca70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4f940df850 tx=0x7f4f94005560 comp rx=0 tx=0).stop 2024-05-09T04:52:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.689+0000 7f4faf640b80 1 -- 172.21.15.46:0/2276572033 shutdown_connections 2024-05-09T04:52:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.689+0000 7f4faf640b80 1 --2- 172.21.15.46:0/2276572033 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5622b9e8b8c0 0x5622b9e8dce0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.689+0000 7f4faf640b80 1 --2- 172.21.15.46:0/2276572033 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9e62400 0x5622b9e6ca70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.689+0000 7f4faf640b80 1 --2- 172.21.15.46:0/2276572033 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9ce54a0 0x5622b9e6bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.689+0000 7f4faf640b80 1 --2- 172.21.15.46:0/2276572033 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9d5fd00 0x5622b9e6c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.689+0000 7f4faf640b80 1 -- 172.21.15.46:0/2276572033 >> 172.21.15.46:0/2276572033 conn(0x5622b9ce8f70 msgr2=0x5622b9e56ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.689+0000 7f4f81ffb700 0 client.25328 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:52:20.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.689+0000 7f4faf640b80 1 -- 172.21.15.46:0/2276572033 shutdown_connections 2024-05-09T04:52:20.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.689+0000 7f4faf640b80 1 -- 172.21.15.46:0/2276572033 wait complete. 2024-05-09T04:52:20.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.689+0000 7f4faf640b80 20 client.25328 trim_cache size 0 max 16384 2024-05-09T04:52:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.717+0000 7f4faf640b80 1 Processor -- start 2024-05-09T04:52:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.717+0000 7f4faf640b80 1 -- start start 2024-05-09T04:52:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.718+0000 7f4faf640b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9ce8f70 0x5622b9e67810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.718+0000 7f4faf640b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9ce54a0 0x5622b9e6c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.718+0000 7f4f9a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9ce8f70 0x5622b9e67810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.718+0000 7f4f9a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9ce8f70 0x5622b9e67810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55616/0 (socket says 172.21.15.46:55616) 2024-05-09T04:52:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.718+0000 7f4f9a59c700 1 -- 172.21.15.46:0/2113029803 learned_addr learned my addr 172.21.15.46:0/2113029803 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.718+0000 7f4faf640b80 1 --2- 172.21.15.46:0/2113029803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9d5fd00 0x5622b9e56ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.718+0000 7f4faf640b80 1 -- 172.21.15.46:0/2113029803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c1a6c30 con 0x5622b9ce8f70 2024-05-09T04:52:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.718+0000 7f4faf640b80 1 -- 172.21.15.46:0/2113029803 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c0748c0 con 0x5622b9ce54a0 2024-05-09T04:52:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.718+0000 7f4faf640b80 1 -- 172.21.15.46:0/2113029803 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c072710 con 0x5622b9d5fd00 2024-05-09T04:52:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.718+0000 7f4f99d9b700 1 --2- 172.21.15.46:0/2113029803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9ce54a0 0x5622b9e6c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.719+0000 7f4f9ad9d700 1 --2- 172.21.15.46:0/2113029803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9d5fd00 0x5622b9e56ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.719+0000 7f4f9a59c700 1 -- 172.21.15.46:0/2113029803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9d5fd00 msgr2=0x5622b9e56ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.719+0000 7f4f9a59c700 1 --2- 172.21.15.46:0/2113029803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9d5fd00 0x5622b9e56ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.719+0000 7f4f9a59c700 1 -- 172.21.15.46:0/2113029803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9ce54a0 msgr2=0x5622b9e6c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.719+0000 7f4f9a59c700 1 --2- 172.21.15.46:0/2113029803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9ce54a0 0x5622b9e6c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.719+0000 7f4f9a59c700 1 -- 172.21.15.46:0/2113029803 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622b9e6a560 con 0x5622b9ce8f70 2024-05-09T04:52:20.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.719+0000 7f4f9a59c700 1 --2- 172.21.15.46:0/2113029803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9ce8f70 0x5622b9e67810 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00d500 tx=0x7f4f8c00c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0946e757f71d760 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.719+0000 7f4f83fff700 1 -- 172.21.15.46:0/2113029803 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c01a030 con 0x5622b9ce8f70 2024-05-09T04:52:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.719+0000 7f4f83fff700 1 -- 172.21.15.46:0/2113029803 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f8c009e50 con 0x5622b9ce8f70 2024-05-09T04:52:20.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.719+0000 7f4f83fff700 1 -- 172.21.15.46:0/2113029803 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f8c02aa80 con 0x5622b9ce8f70 2024-05-09T04:52:20.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.721+0000 7f4faf640b80 1 -- 172.21.15.46:0/2113029803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9ce8f70 msgr2=0x5622b9e67810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.721+0000 7f4faf640b80 1 --2- 172.21.15.46:0/2113029803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9ce8f70 0x5622b9e67810 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f4f8c00d500 tx=0x7f4f8c00c710 comp rx=0 tx=0).stop 2024-05-09T04:52:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.721+0000 7f4faf640b80 1 -- 172.21.15.46:0/2113029803 shutdown_connections 2024-05-09T04:52:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.721+0000 7f4faf640b80 1 --2- 172.21.15.46:0/2113029803 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9d5fd00 0x5622b9e56ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.721+0000 7f4faf640b80 1 --2- 172.21.15.46:0/2113029803 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9ce8f70 0x5622b9e67810 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.721+0000 7f4faf640b80 1 --2- 172.21.15.46:0/2113029803 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9ce54a0 0x5622b9e6c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.721+0000 7f4faf640b80 1 -- 172.21.15.46:0/2113029803 >> 172.21.15.46:0/2113029803 conn(0x5622b9e62ae0 msgr2=0x5622b9e67100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.721+0000 7f4faf640b80 1 -- 172.21.15.46:0/2113029803 shutdown_connections 2024-05-09T04:52:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.721+0000 7f4faf640b80 1 -- 172.21.15.46:0/2113029803 wait complete. 2024-05-09T04:52:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.721+0000 7f4faf640b80 1 -- start start 2024-05-09T04:52:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.722+0000 7f4faf640b80 1 Processor -- start 2024-05-09T04:52:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.722+0000 7f4faf640b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9ce54a0 0x5622b9e854c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.722+0000 7f4faf640b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9ce8f70 0x5622b9e925f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.722+0000 7f4faf640b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9d5fd00 0x5622b9e92b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.722+0000 7f4f9ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9d5fd00 0x5622b9e92b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.722+0000 7f4f9ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9d5fd00 0x5622b9e92b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55622/0 (socket says 172.21.15.46:55622) 2024-05-09T04:52:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.722+0000 7f4f9ad9d700 1 -- 172.21.15.46:0/1127532169 learned_addr learned my addr 172.21.15.46:0/1127532169 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.722+0000 7f4faf640b80 1 -- 172.21.15.46:0/1127532169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c072d80 con 0x5622b9d5fd00 2024-05-09T04:52:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.722+0000 7f4faf640b80 1 -- 172.21.15.46:0/1127532169 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c0741f0 con 0x5622b9ce54a0 2024-05-09T04:52:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.722+0000 7f4f9a59c700 1 --2- 172.21.15.46:0/1127532169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9ce54a0 0x5622b9e854c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.722+0000 7f4f99d9b700 1 --2- 172.21.15.46:0/1127532169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9ce8f70 0x5622b9e925f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.722+0000 7f4faf640b80 1 -- 172.21.15.46:0/1127532169 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4f9c073450 con 0x5622b9ce8f70 2024-05-09T04:52:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.723+0000 7f4faf640b80 10 client.0 _ll_register_callbacks cb 0x5622b9bf66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.723+0000 7f4f9ad9d700 1 -- 172.21.15.46:0/1127532169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9ce8f70 msgr2=0x5622b9e925f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.723+0000 7f4f9ad9d700 1 --2- 172.21.15.46:0/1127532169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9ce8f70 0x5622b9e925f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.723+0000 7f4f9ad9d700 1 -- 172.21.15.46:0/1127532169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9ce54a0 msgr2=0x5622b9e854c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.723+0000 7f4f9ad9d700 1 --2- 172.21.15.46:0/1127532169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9ce54a0 0x5622b9e854c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.723+0000 7f4f9ad9d700 1 -- 172.21.15.46:0/1127532169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622b9e93070 con 0x5622b9d5fd00 2024-05-09T04:52:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.723+0000 7f4f9ad9d700 1 --2- 172.21.15.46:0/1127532169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9d5fd00 0x5622b9e92b30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4f940a58b0 tx=0x7f4f94005560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3185afebb9ab12ab server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.723+0000 7f4f81ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.723+0000 7f4faf640b80 1 -- 172.21.15.46:0/1127532169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4f8c009e50 con 0x5622b9d5fd00 2024-05-09T04:52:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.723+0000 7f4f81ffb700 1 -- 172.21.15.46:0/1127532169 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f940c7070 con 0x5622b9d5fd00 2024-05-09T04:52:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.723+0000 7f4f81ffb700 1 -- 172.21.15.46:0/1127532169 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f4f940dfb30 con 0x5622b9d5fd00 2024-05-09T04:52:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.724+0000 7f4f81ffb700 1 -- 172.21.15.46:0/1127532169 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4f940dfd30 con 0x5622b9d5fd00 2024-05-09T04:52:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.724+0000 7f4faf640b80 10 client.15570 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.724+0000 7f4faf640b80 1 -- 172.21.15.46:0/1127532169 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5622b9ef75a0 con 0x5622b9d5fd00 2024-05-09T04:52:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.724+0000 7f4f81ffb700 1 -- 172.21.15.46:0/1127532169 <== mon.0 v2:172.21.15.46:3300/0 4 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f4f94005be0 con 0x5622b9d5fd00 2024-05-09T04:52:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.724+0000 7f4faf640b80 10 client.15570 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.724+0000 7f4faf640b80 10 client.15570 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:52:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.724+0000 7f4faf640b80 20 client.15570 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.724+0000 7f4faf640b80 1 --2- 172.21.15.46:0/1127532169 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5622b9e62400 0x5622b9ef9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.724+0000 7f4faf640b80 4 client.15570 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:52:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.724+0000 7f4faf640b80 1 -- 172.21.15.46:0/1127532169 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f4f8c009e50 con 0x5622b9e62400 2024-05-09T04:52:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.725+0000 7f4f9a59c700 1 --2- 172.21.15.46:0/1127532169 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5622b9e62400 0x5622b9ef9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.725+0000 7f4f81ffb700 10 client.15570 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:52:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.725+0000 7f4f9a59c700 1 --2- 172.21.15.46:0/1127532169 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5622b9e62400 0x5622b9ef9970 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=599a686a6a1210cb server_cookie=8195c28317d59355 in_seq=0 out_seq=0 2024-05-09T04:52:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.726+0000 7f4f81ffb700 1 -- 172.21.15.46:0/1127532169 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f4f8c009e50 con 0x5622b9e62400 2024-05-09T04:52:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.726+0000 7f4f81ffb700 10 client.15570 handle_command_reply: tid=0 2024-05-09T04:52:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.726+0000 7f4faf640b80 1 client.15570 shutdown 2024-05-09T04:52:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.726+0000 7f4faf640b80 1 -- 172.21.15.46:0/1127532169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9d5fd00 msgr2=0x5622b9e92b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.726+0000 7f4faf640b80 1 --2- 172.21.15.46:0/1127532169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9d5fd00 0x5622b9e92b30 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f4f940a58b0 tx=0x7f4f94005560 comp rx=0 tx=0).stop 2024-05-09T04:52:20.730 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:20 smithi046 ceph-mon[30136]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-09T04:52:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.727+0000 7f4faf640b80 1 -- 172.21.15.46:0/1127532169 shutdown_connections 2024-05-09T04:52:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.727+0000 7f4faf640b80 1 --2- 172.21.15.46:0/1127532169 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5622b9e62400 0x5622b9ef9970 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.727+0000 7f4faf640b80 1 --2- 172.21.15.46:0/1127532169 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5622b9d5fd00 0x5622b9e92b30 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.727+0000 7f4faf640b80 1 --2- 172.21.15.46:0/1127532169 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5622b9ce54a0 0x5622b9e854c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.727+0000 7f4faf640b80 1 --2- 172.21.15.46:0/1127532169 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5622b9ce8f70 0x5622b9e925f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.727+0000 7f4faf640b80 1 -- 172.21.15.46:0/1127532169 >> 172.21.15.46:0/1127532169 conn(0x5622b9e62ae0 msgr2=0x5622b9e56ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.728+0000 7f4f81ffb700 0 client.15570 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:52:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.728+0000 7f4faf640b80 1 -- 172.21.15.46:0/1127532169 shutdown_connections 2024-05-09T04:52:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.728+0000 7f4faf640b80 1 -- 172.21.15.46:0/1127532169 wait complete. 2024-05-09T04:52:20.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.728+0000 7f4faf640b80 20 client.15570 trim_cache size 0 max 16384 2024-05-09T04:52:20.732 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:52:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.731+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/1429891914 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4f840b2b10 msgr2=0x7f4f840b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.731+0000 7f4fa0c2a700 1 --2- 172.21.15.46:0/1429891914 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4f840b2b10 0x7f4f840b5020 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f4f9c102090 tx=0x7f4f9001f040 comp rx=0 tx=0).stop 2024-05-09T04:52:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.731+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/1429891914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f9c1089f0 msgr2=0x7f4f9c1988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.731+0000 7f4fa0c2a700 1 --2- 172.21.15.46:0/1429891914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f9c1089f0 0x7f4f9c1988c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4f94007f10 tx=0x7f4f9400d3b0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.731+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/1429891914 shutdown_connections 2024-05-09T04:52:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.731+0000 7f4fa0c2a700 1 --2- 172.21.15.46:0/1429891914 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4f840b2b10 0x7f4f840b5020 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.731+0000 7f4fa0c2a700 1 --2- 172.21.15.46:0/1429891914 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4f9c1089f0 0x7f4f9c1988c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.731+0000 7f4fa0c2a700 1 --2- 172.21.15.46:0/1429891914 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4f9c1029c0 0x7f4f9c19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.731+0000 7f4fa0c2a700 1 --2- 172.21.15.46:0/1429891914 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4f9c103310 0x7f4f9c19ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.731+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/1429891914 >> 172.21.15.46:0/1429891914 conn(0x7f4f9c0fe6f0 msgr2=0x7f4f9c106c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.732+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/1429891914 shutdown_connections 2024-05-09T04:52:20.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:20.732+0000 7f4fa0c2a700 1 -- 172.21.15.46:0/1429891914 wait complete. 2024-05-09T04:52:21.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:20 smithi110 ceph-mon[34904]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-09T04:52:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:20 smithi072 ceph-mon[34088]: pgmap v40: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 511 B/s wr, 6 op/s 2024-05-09T04:52:21.506 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:52:21.907 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:22.967 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:22 smithi046 ceph-mon[30136]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-09T04:52:23.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:22 smithi110 ceph-mon[34904]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-09T04:52:23.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:22 smithi072 ceph-mon[34088]: pgmap v41: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 511 B/s wr, 9 op/s 2024-05-09T04:52:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.058+0000 7fdb3f1ea640 1 -- 172.21.15.46:0/1594175290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb38102580 msgr2=0x7fdb38102990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.058+0000 7fdb3f1ea640 1 --2- 172.21.15.46:0/1594175290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb38102580 0x7fdb38102990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c009900 tx=0x7fdb2c02a410 comp rx=0 tx=0).stop 2024-05-09T04:52:24.059 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.059+0000 7fdb3f1ea640 1 -- 172.21.15.46:0/1594175290 shutdown_connections 2024-05-09T04:52:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.059+0000 7fdb3f1ea640 1 --2- 172.21.15.46:0/1594175290 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb381075d0 0x7fdb3810ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.059+0000 7fdb3f1ea640 1 --2- 172.21.15.46:0/1594175290 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb38106b10 0x7fdb38106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.059+0000 7fdb3f1ea640 1 --2- 172.21.15.46:0/1594175290 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb38102580 0x7fdb38102990 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.059+0000 7fdb3f1ea640 1 -- 172.21.15.46:0/1594175290 >> 172.21.15.46:0/1594175290 conn(0x7fdb380779e0 msgr2=0x7fdb38077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.059+0000 7fdb3f1ea640 1 -- 172.21.15.46:0/1594175290 shutdown_connections 2024-05-09T04:52:24.060 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.059+0000 7fdb3f1ea640 1 -- 172.21.15.46:0/1594175290 wait complete. 2024-05-09T04:52:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.060+0000 7fdb3f1ea640 1 Processor -- start 2024-05-09T04:52:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.060+0000 7fdb3f1ea640 1 -- start start 2024-05-09T04:52:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.061+0000 7fdb3f1ea640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb38102580 0x7fdb381986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.061+0000 7fdb3f1ea640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38106b10 0x7fdb38198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.061+0000 7fdb3f1ea640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb381075d0 0x7fdb3819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.061+0000 7fdb3f1ea640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdb38111c60 con 0x7fdb38102580 2024-05-09T04:52:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.061+0000 7fdb3f1ea640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdb38111ae0 con 0x7fdb381075d0 2024-05-09T04:52:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.061+0000 7fdb3f1ea640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb38111de0 con 0x7fdb38106b10 2024-05-09T04:52:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.061+0000 7fdb3cf86640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb38102580 0x7fdb381986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.061+0000 7fdb3cf86640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb38102580 0x7fdb381986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52446/0 (socket says 172.21.15.46:52446) 2024-05-09T04:52:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.061+0000 7fdb3cf86640 1 -- 172.21.15.46:0/792409800 learned_addr learned my addr 172.21.15.46:0/792409800 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.061+0000 7fdb37fff640 1 --2- 172.21.15.46:0/792409800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38106b10 0x7fdb38198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.062 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.061+0000 7fdb3d787640 1 --2- 172.21.15.46:0/792409800 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb381075d0 0x7fdb3819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.062+0000 7fdb3cf86640 1 -- 172.21.15.46:0/792409800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38106b10 msgr2=0x7fdb38198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.062+0000 7fdb3cf86640 1 --2- 172.21.15.46:0/792409800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38106b10 0x7fdb38198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.062+0000 7fdb3cf86640 1 -- 172.21.15.46:0/792409800 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb381075d0 msgr2=0x7fdb3819cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.062+0000 7fdb3cf86640 1 --2- 172.21.15.46:0/792409800 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb381075d0 0x7fdb3819cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.062+0000 7fdb3cf86640 1 -- 172.21.15.46:0/792409800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb3819d520 con 0x7fdb38102580 2024-05-09T04:52:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.062+0000 7fdb37fff640 1 --2- 172.21.15.46:0/792409800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38106b10 0x7fdb38198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:24.063 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.062+0000 7fdb3cf86640 1 --2- 172.21.15.46:0/792409800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb38102580 0x7fdb381986c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c005940 tx=0x7fdb2c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.062+0000 7fdb35ffb640 1 -- 172.21.15.46:0/792409800 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c03f030 con 0x7fdb38102580 2024-05-09T04:52:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.062+0000 7fdb35ffb640 1 -- 172.21.15.46:0/792409800 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb2c003cb0 con 0x7fdb38102580 2024-05-09T04:52:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.062+0000 7fdb3f1ea640 1 -- 172.21.15.46:0/792409800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb3819d7b0 con 0x7fdb38102580 2024-05-09T04:52:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.062+0000 7fdb35ffb640 1 -- 172.21.15.46:0/792409800 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c035070 con 0x7fdb38102580 2024-05-09T04:52:24.064 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.063+0000 7fdb3f1ea640 1 -- 172.21.15.46:0/792409800 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb3819dc30 con 0x7fdb38102580 2024-05-09T04:52:24.066 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.066+0000 7fdb46178740 1 Processor -- start 2024-05-09T04:52:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.066+0000 7fdb46178740 1 -- start start 2024-05-09T04:52:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.067+0000 7fdb35ffb640 1 -- 172.21.15.46:0/792409800 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7fdb2c042030 con 0x7fdb38102580 2024-05-09T04:52:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.067+0000 7fdb46178740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cef1b0 0x559c78093cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.067+0000 7fdb46178740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cc62f0 0x559c780924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.067 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.067+0000 7fdb46178740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cce4d0 0x559c78092a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.067+0000 7fdb3d787640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cef1b0 0x559c78093cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.067+0000 7fdb37fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cce4d0 0x559c78092a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.067+0000 7fdb3d787640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cef1b0 0x559c78093cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:60776/0 (socket says 172.21.15.46:60776) 2024-05-09T04:52:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.067+0000 7fdb37fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cce4d0 0x559c78092a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52452/0 (socket says 172.21.15.46:52452) 2024-05-09T04:52:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.067+0000 7fdb3d787640 1 -- 172.21.15.46:0/986287931 learned_addr learned my addr 172.21.15.46:0/986287931 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.067+0000 7fdb46178740 1 -- 172.21.15.46:0/986287931 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559c77ff77c0 con 0x559c77cce4d0 2024-05-09T04:52:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.067+0000 7fdb46178740 1 -- 172.21.15.46:0/986287931 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559c77f930e0 con 0x559c77cc62f0 2024-05-09T04:52:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.067+0000 7fdb46178740 1 -- 172.21.15.46:0/986287931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559c77ff3780 con 0x559c77cef1b0 2024-05-09T04:52:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.067+0000 7fdb3cf86640 1 --2- 172.21.15.46:0/986287931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cc62f0 0x559c780924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.068+0000 7fdb3d787640 1 -- 172.21.15.46:0/986287931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cc62f0 msgr2=0x559c780924d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.068+0000 7fdb3d787640 1 --2- 172.21.15.46:0/986287931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cc62f0 0x559c780924d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.069 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.068+0000 7fdb3d787640 1 -- 172.21.15.46:0/986287931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cce4d0 msgr2=0x559c78092a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.068+0000 7fdb3d787640 1 --2- 172.21.15.46:0/986287931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cce4d0 0x559c78092a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.068+0000 7fdb3d787640 1 -- 172.21.15.46:0/986287931 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c77fbc7c0 con 0x559c77cef1b0 2024-05-09T04:52:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.068+0000 7fdb37fff640 1 --2- 172.21.15.46:0/986287931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cce4d0 0x559c78092a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.068+0000 7fdb35ffb640 1 --2- 172.21.15.46:0/792409800 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdb280b2af0 0x7fdb280b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.068+0000 7fdb3d787640 1 --2- 172.21.15.46:0/986287931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cef1b0 0x559c78093cb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009f70 tx=0x7fdb30010800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8b1db4466a14089 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.068+0000 7fdb3cf86640 1 --2- 172.21.15.46:0/792409800 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdb280b2af0 0x7fdb280b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.068+0000 7fdb35ffb640 1 -- 172.21.15.46:0/792409800 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fdb2c102fe0 con 0x7fdb38102580 2024-05-09T04:52:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.068+0000 7fdb22ffd640 1 -- 172.21.15.46:0/986287931 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30016070 con 0x559c77cef1b0 2024-05-09T04:52:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.068+0000 7fdb22ffd640 1 -- 172.21.15.46:0/986287931 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb30004d20 con 0x559c77cef1b0 2024-05-09T04:52:24.070 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.068+0000 7fdb22ffd640 1 -- 172.21.15.46:0/986287931 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30012790 con 0x559c77cef1b0 2024-05-09T04:52:24.073 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.070+0000 7fdb46178740 1 -- 172.21.15.46:0/986287931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cef1b0 msgr2=0x559c78093cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.070+0000 7fdb3cf86640 1 --2- 172.21.15.46:0/792409800 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdb280b2af0 0x7fdb280b5000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c1005f0 tx=0x7fdb2c106af0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.070+0000 7fdb46178740 1 --2- 172.21.15.46:0/986287931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cef1b0 0x559c78093cb0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009f70 tx=0x7fdb30010800 comp rx=0 tx=0).stop 2024-05-09T04:52:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.070+0000 7fdb46178740 1 -- 172.21.15.46:0/986287931 shutdown_connections 2024-05-09T04:52:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.070+0000 7fdb46178740 1 --2- 172.21.15.46:0/986287931 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cce4d0 0x559c78092a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.070+0000 7fdb46178740 1 --2- 172.21.15.46:0/986287931 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cc62f0 0x559c780924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.070+0000 7fdb46178740 1 --2- 172.21.15.46:0/986287931 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cef1b0 0x559c78093cb0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.070+0000 7fdb46178740 1 -- 172.21.15.46:0/986287931 >> 172.21.15.46:0/986287931 conn(0x559c77ce3300 msgr2=0x559c7808ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.070+0000 7fdb46178740 1 -- 172.21.15.46:0/986287931 shutdown_connections 2024-05-09T04:52:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.070+0000 7fdb46178740 1 -- 172.21.15.46:0/986287931 wait complete. 2024-05-09T04:52:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.071+0000 7fdb46178740 1 -- start start 2024-05-09T04:52:24.074 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.071+0000 7fdb46178740 1 Processor -- start 2024-05-09T04:52:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb46178740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cc62f0 0x559c780a2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb46178740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cce4d0 0x559c780a3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb46178740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cef1b0 0x559c780a11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb46178740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559c780a3c30 con 0x559c77cef1b0 2024-05-09T04:52:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb46178740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559c77f7e9c0 con 0x559c77cce4d0 2024-05-09T04:52:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb46178740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559c780a3db0 con 0x559c77cc62f0 2024-05-09T04:52:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb3d787640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cef1b0 0x559c780a11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb3d787640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cef1b0 0x559c780a11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52466/0 (socket says 172.21.15.46:52466) 2024-05-09T04:52:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb3d787640 1 -- 172.21.15.46:0/792672851 learned_addr learned my addr 172.21.15.46:0/792672851 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb46178740 10 client.0 _ll_register_callbacks cb 0x559c77fab020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb37fff640 1 --2- 172.21.15.46:0/792672851 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cce4d0 0x559c780a3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb3d787640 1 --2- 172.21.15.46:0/792672851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cc62f0 0x559c780a2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.075 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb3d787640 1 -- 172.21.15.46:0/792672851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cc62f0 msgr2=0x559c780a2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb3d787640 1 --2- 172.21.15.46:0/792672851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cc62f0 0x559c780a2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb3d787640 1 -- 172.21.15.46:0/792672851 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cce4d0 msgr2=0x559c780a3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb3d787640 1 --2- 172.21.15.46:0/792672851 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cce4d0 0x559c780a3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb3d787640 1 -- 172.21.15.46:0/792672851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c77fa51f0 con 0x559c77cef1b0 2024-05-09T04:52:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.072+0000 7fdb37fff640 1 --2- 172.21.15.46:0/792672851 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cce4d0 0x559c780a3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:52:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.073+0000 7fdb3d787640 1 --2- 172.21.15.46:0/792672851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cef1b0 0x559c780a11d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009d20 tx=0x7fdb30005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f33ae5c172b34e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.073+0000 7fdb17fff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.073+0000 7fdb17fff640 1 -- 172.21.15.46:0/792672851 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30016030 con 0x559c77cef1b0 2024-05-09T04:52:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.073+0000 7fdb17fff640 1 -- 172.21.15.46:0/792672851 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fdb30003970 con 0x559c77cef1b0 2024-05-09T04:52:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.073+0000 7fdb46178740 1 -- 172.21.15.46:0/792672851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb30004d20 con 0x559c77cef1b0 2024-05-09T04:52:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.073+0000 7fdb17fff640 1 -- 172.21.15.46:0/792672851 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdb3002e050 con 0x559c77cef1b0 2024-05-09T04:52:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.073+0000 7fdb46178740 10 client.15582 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.073+0000 7fdb46178740 1 -- 172.21.15.46:0/792672851 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559c77d5a790 con 0x559c77cef1b0 2024-05-09T04:52:24.076 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.073+0000 7fdb17fff640 1 -- 172.21.15.46:0/792672851 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb30004d20 con 0x559c77cef1b0 2024-05-09T04:52:24.078 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.074+0000 7fdb17fff640 1 -- 172.21.15.46:0/792672851 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fdb3000f040 con 0x559c77cef1b0 2024-05-09T04:52:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.074+0000 7fdb46178740 10 client.15582 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.074+0000 7fdb46178740 10 client.15582 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.074+0000 7fdb46178740 20 client.15582 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.074+0000 7fdb46178740 1 --2- 172.21.15.46:0/792672851 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559c77cadd80 0x559c780b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.074+0000 7fdb46178740 4 client.15582 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.074+0000 7fdb46178740 1 -- 172.21.15.46:0/792672851 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c780053d0 con 0x559c77cadd80 2024-05-09T04:52:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.075+0000 7fdb37fff640 1 --2- 172.21.15.46:0/792672851 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559c77cadd80 0x559c780b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.075+0000 7fdb17fff640 10 client.15582 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.075+0000 7fdb37fff640 1 --2- 172.21.15.46:0/792672851 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559c77cadd80 0x559c780b38d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de5419ce4d8795bc server_cookie=3caf094b9c5b5584 in_seq=0 out_seq=0 2024-05-09T04:52:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.076+0000 7fdb17fff640 1 -- 172.21.15.46:0/792672851 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559c780053d0 con 0x559c77cadd80 2024-05-09T04:52:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.076+0000 7fdb17fff640 10 client.15582 handle_command_reply: tid=0 2024-05-09T04:52:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.076+0000 7fdb46178740 1 client.15582 shutdown 2024-05-09T04:52:24.079 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.076+0000 7fdb46178740 1 -- 172.21.15.46:0/792672851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cef1b0 msgr2=0x559c780a11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.076+0000 7fdb46178740 1 --2- 172.21.15.46:0/792672851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cef1b0 0x559c780a11d0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fdb30009d20 tx=0x7fdb30005a40 comp rx=0 tx=0).stop 2024-05-09T04:52:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.076+0000 7fdb46178740 1 -- 172.21.15.46:0/792672851 shutdown_connections 2024-05-09T04:52:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.076+0000 7fdb46178740 1 --2- 172.21.15.46:0/792672851 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559c77cadd80 0x559c780b38d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.077+0000 7fdb46178740 1 --2- 172.21.15.46:0/792672851 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cef1b0 0x559c780a11d0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.077+0000 7fdb17fff640 0 client.15582 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.077+0000 7fdb46178740 1 --2- 172.21.15.46:0/792672851 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cce4d0 0x559c780a3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.077+0000 7fdb46178740 1 --2- 172.21.15.46:0/792672851 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cc62f0 0x559c780a2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.077+0000 7fdb46178740 1 -- 172.21.15.46:0/792672851 >> 172.21.15.46:0/792672851 conn(0x559c77ce3300 msgr2=0x559c78094330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.077+0000 7fdb46178740 1 -- 172.21.15.46:0/792672851 shutdown_connections 2024-05-09T04:52:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.077+0000 7fdb46178740 1 -- 172.21.15.46:0/792672851 wait complete. 2024-05-09T04:52:24.080 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.077+0000 7fdb46178740 20 client.15582 trim_cache size 0 max 16384 2024-05-09T04:52:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.103+0000 7fdb46178740 1 Processor -- start 2024-05-09T04:52:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.103+0000 7fdb46178740 1 -- start start 2024-05-09T04:52:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.103+0000 7fdb46178740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cadd80 0x559c780a1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.103+0000 7fdb46178740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cc62f0 0x559c780962e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.103+0000 7fdb46178740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cce4d0 0x559c78096820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.103+0000 7fdb46178740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381b0360 con 0x559c77cadd80 2024-05-09T04:52:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.103+0000 7fdb46178740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381adc30 con 0x559c77cc62f0 2024-05-09T04:52:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.103+0000 7fdb46178740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381aba80 con 0x559c77cce4d0 2024-05-09T04:52:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.103+0000 7fdb3d787640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cadd80 0x559c780a1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.104+0000 7fdb3d787640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cadd80 0x559c780a1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52476/0 (socket says 172.21.15.46:52476) 2024-05-09T04:52:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.104+0000 7fdb3d787640 1 -- 172.21.15.46:0/899918795 learned_addr learned my addr 172.21.15.46:0/899918795 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:24.104 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.104+0000 7fdb37fff640 1 --2- 172.21.15.46:0/899918795 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cc62f0 0x559c780962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.104+0000 7fdb3d787640 1 -- 172.21.15.46:0/899918795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cce4d0 msgr2=0x559c78096820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.104+0000 7fdb3d787640 1 --2- 172.21.15.46:0/899918795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cce4d0 0x559c78096820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.104+0000 7fdb3d787640 1 -- 172.21.15.46:0/899918795 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cc62f0 msgr2=0x559c780962e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.104+0000 7fdb3d787640 1 --2- 172.21.15.46:0/899918795 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cc62f0 0x559c780962e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.104+0000 7fdb3d787640 1 -- 172.21.15.46:0/899918795 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c77dd15f0 con 0x559c77cadd80 2024-05-09T04:52:24.105 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.104+0000 7fdb3d787640 1 --2- 172.21.15.46:0/899918795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cadd80 0x559c780a1720 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdb30004a10 tx=0x7fdb30005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a76cf571a71ac03c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.104+0000 7fdb227fc640 1 -- 172.21.15.46:0/899918795 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb30016040 con 0x559c77cadd80 2024-05-09T04:52:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.104+0000 7fdb227fc640 1 -- 172.21.15.46:0/899918795 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb3001f070 con 0x559c77cadd80 2024-05-09T04:52:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.104+0000 7fdb227fc640 1 -- 172.21.15.46:0/899918795 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb3001a990 con 0x559c77cadd80 2024-05-09T04:52:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.105+0000 7fdb46178740 1 -- 172.21.15.46:0/899918795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cadd80 msgr2=0x559c780a1720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.105+0000 7fdb46178740 1 --2- 172.21.15.46:0/899918795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cadd80 0x559c780a1720 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fdb30004a10 tx=0x7fdb30005790 comp rx=0 tx=0).stop 2024-05-09T04:52:24.106 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.105+0000 7fdb46178740 1 -- 172.21.15.46:0/899918795 shutdown_connections 2024-05-09T04:52:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.105+0000 7fdb46178740 1 --2- 172.21.15.46:0/899918795 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cce4d0 0x559c78096820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.105+0000 7fdb46178740 1 --2- 172.21.15.46:0/899918795 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cc62f0 0x559c780962e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.105+0000 7fdb46178740 1 --2- 172.21.15.46:0/899918795 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cadd80 0x559c780a1720 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.105+0000 7fdb46178740 1 -- 172.21.15.46:0/899918795 >> 172.21.15.46:0/899918795 conn(0x559c77ce3300 msgr2=0x559c7809faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.106+0000 7fdb46178740 1 -- 172.21.15.46:0/899918795 shutdown_connections 2024-05-09T04:52:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.106+0000 7fdb46178740 1 -- 172.21.15.46:0/899918795 wait complete. 2024-05-09T04:52:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.106+0000 7fdb46178740 1 -- start start 2024-05-09T04:52:24.107 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.106+0000 7fdb46178740 1 Processor -- start 2024-05-09T04:52:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb46178740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cadd80 0x559c780a6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb46178740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cc62f0 0x559c780a6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb46178740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cce4d0 0x559c78119c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.110 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb46178740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381ac0f0 con 0x559c77cce4d0 2024-05-09T04:52:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb46178740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381ad560 con 0x559c77cadd80 2024-05-09T04:52:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb46178740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb381ac7c0 con 0x559c77cc62f0 2024-05-09T04:52:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb3d787640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cce4d0 0x559c78119c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb3d787640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cce4d0 0x559c78119c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52480/0 (socket says 172.21.15.46:52480) 2024-05-09T04:52:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb3d787640 1 -- 172.21.15.46:0/2769686402 learned_addr learned my addr 172.21.15.46:0/2769686402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb37fff640 1 --2- 172.21.15.46:0/2769686402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cc62f0 0x559c780a6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb46178740 10 client.0 _ll_register_callbacks cb 0x7fdb2400cc30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb37fff640 1 -- 172.21.15.46:0/2769686402 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cadd80 msgr2=0x559c780a6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb37fff640 1 --2- 172.21.15.46:0/2769686402 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cadd80 0x559c780a6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb37fff640 1 -- 172.21.15.46:0/2769686402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cce4d0 msgr2=0x559c78119c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb37fff640 1 --2- 172.21.15.46:0/2769686402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cce4d0 0x559c78119c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb37fff640 1 -- 172.21.15.46:0/2769686402 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c77fbbed0 con 0x559c77cc62f0 2024-05-09T04:52:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.107+0000 7fdb3d787640 1 --2- 172.21.15.46:0/2769686402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cce4d0 0x559c78119c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.108+0000 7fdb37fff640 1 --2- 172.21.15.46:0/2769686402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cc62f0 0x559c780a6550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x559c780b4000 tx=0x7fdb2400a780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=889fdf594d853e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:24.111 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.108+0000 7fdb17fff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:52:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.108+0000 7fdb17fff640 1 -- 172.21.15.46:0/2769686402 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb24028070 con 0x559c77cc62f0 2024-05-09T04:52:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.108+0000 7fdb17fff640 1 -- 172.21.15.46:0/2769686402 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fdb24004220 con 0x559c77cc62f0 2024-05-09T04:52:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.108+0000 7fdb46178740 1 -- 172.21.15.46:0/2769686402 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb3001f070 con 0x559c77cc62f0 2024-05-09T04:52:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.108+0000 7fdb17fff640 1 -- 172.21.15.46:0/2769686402 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdb24033050 con 0x559c77cc62f0 2024-05-09T04:52:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.108+0000 7fdb46178740 10 client.25300 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.108+0000 7fdb46178740 1 -- 172.21.15.46:0/2769686402 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559c78097830 con 0x559c77cc62f0 2024-05-09T04:52:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.108+0000 7fdb17fff640 1 -- 172.21.15.46:0/2769686402 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb24004440 con 0x559c77cc62f0 2024-05-09T04:52:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.109+0000 7fdb17fff640 1 -- 172.21.15.46:0/2769686402 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fdb2400b040 con 0x559c77cc62f0 2024-05-09T04:52:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.109+0000 7fdb46178740 10 client.25300 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.109+0000 7fdb46178740 10 client.25300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.109+0000 7fdb46178740 20 client.25300 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.109+0000 7fdb46178740 1 --2- 172.21.15.46:0/2769686402 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559c77cef1b0 0x559c7809a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.109+0000 7fdb46178740 4 client.25300 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T04:52:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.109+0000 7fdb46178740 1 -- 172.21.15.46:0/2769686402 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7fdb3001f070 con 0x559c77cef1b0 2024-05-09T04:52:24.112 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.109+0000 7fdb3d787640 1 --2- 172.21.15.46:0/2769686402 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559c77cef1b0 0x559c7809a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.109+0000 7fdb17fff640 10 client.25300 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:24.113 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.110+0000 7fdb3d787640 1 --2- 172.21.15.46:0/2769686402 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559c77cef1b0 0x559c7809a010 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f80f700b9ad5bab server_cookie=fe83f9689acaaa8f in_seq=0 out_seq=0 2024-05-09T04:52:24.114 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.114+0000 7fdb17fff640 1 -- 172.21.15.46:0/2769686402 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x7fdb3001f070 con 0x559c77cef1b0 2024-05-09T04:52:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.114+0000 7fdb17fff640 10 client.25300 handle_command_reply: tid=0 2024-05-09T04:52:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.114+0000 7fdb46178740 1 client.25300 shutdown 2024-05-09T04:52:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.114+0000 7fdb46178740 1 -- 172.21.15.46:0/2769686402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cc62f0 msgr2=0x559c780a6550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.114+0000 7fdb46178740 1 --2- 172.21.15.46:0/2769686402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cc62f0 0x559c780a6550 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x559c780b4000 tx=0x7fdb2400a780 comp rx=0 tx=0).stop 2024-05-09T04:52:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.114+0000 7fdb46178740 1 -- 172.21.15.46:0/2769686402 shutdown_connections 2024-05-09T04:52:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.114+0000 7fdb46178740 1 --2- 172.21.15.46:0/2769686402 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559c77cef1b0 0x559c7809a010 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.114+0000 7fdb17fff640 0 client.25300 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.114+0000 7fdb46178740 1 --2- 172.21.15.46:0/2769686402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559c77cc62f0 0x559c780a6550 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.114+0000 7fdb46178740 1 --2- 172.21.15.46:0/2769686402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559c77cce4d0 0x559c78119c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.114+0000 7fdb46178740 1 --2- 172.21.15.46:0/2769686402 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559c77cadd80 0x559c780a6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.114+0000 7fdb46178740 1 -- 172.21.15.46:0/2769686402 >> 172.21.15.46:0/2769686402 conn(0x559c77ce3300 msgr2=0x559c780962e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.114+0000 7fdb46178740 1 -- 172.21.15.46:0/2769686402 shutdown_connections 2024-05-09T04:52:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.114+0000 7fdb46178740 1 -- 172.21.15.46:0/2769686402 wait complete. 2024-05-09T04:52:24.115 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.114+0000 7fdb46178740 20 client.25300 trim_cache size 0 max 16384 2024-05-09T04:52:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-05-09T04:52:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-05-09T04:52:24.116 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-05-09T04:52:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.115+0000 7fdb3f1ea640 1 -- 172.21.15.46:0/792409800 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdb280b2af0 msgr2=0x7fdb280b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.115+0000 7fdb3f1ea640 1 --2- 172.21.15.46:0/792409800 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdb280b2af0 0x7fdb280b5000 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c1005f0 tx=0x7fdb2c106af0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.115+0000 7fdb3f1ea640 1 -- 172.21.15.46:0/792409800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb38102580 msgr2=0x7fdb381986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.115+0000 7fdb3f1ea640 1 --2- 172.21.15.46:0/792409800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb38102580 0x7fdb381986c0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c005940 tx=0x7fdb2c0042f0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.115+0000 7fdb3f1ea640 1 -- 172.21.15.46:0/792409800 shutdown_connections 2024-05-09T04:52:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.115+0000 7fdb3f1ea640 1 --2- 172.21.15.46:0/792409800 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdb280b2af0 0x7fdb280b5000 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.115+0000 7fdb3f1ea640 1 --2- 172.21.15.46:0/792409800 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb38106b10 0x7fdb38198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.115+0000 7fdb3f1ea640 1 --2- 172.21.15.46:0/792409800 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb381075d0 0x7fdb3819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.115+0000 7fdb3f1ea640 1 --2- 172.21.15.46:0/792409800 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb38102580 0x7fdb381986c0 secure :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c005940 tx=0x7fdb2c0042f0 comp rx=0 tx=0).stop 2024-05-09T04:52:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.115+0000 7fdb3f1ea640 1 -- 172.21.15.46:0/792409800 >> 172.21.15.46:0/792409800 conn(0x7fdb380779e0 msgr2=0x7fdb38100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.116+0000 7fdb3f1ea640 1 -- 172.21.15.46:0/792409800 shutdown_connections 2024-05-09T04:52:24.117 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:24.116+0000 7fdb3f1ea640 1 -- 172.21.15.46:0/792409800 wait complete. 2024-05-09T04:52:25.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:24 smithi110 ceph-mon[34904]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T04:52:25.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:24 smithi046 ceph-mon[30136]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T04:52:25.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:24 smithi072 ceph-mon[34088]: pgmap v42: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T04:52:26.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.375+0000 7ff94f845700 1 -- 172.21.15.46:0/2231983372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9481034e0 msgr2=0x7ff9481038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:26.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.375+0000 7ff94f845700 1 --2- 172.21.15.46:0/2231983372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9481034e0 0x7ff9481038f0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ff9440099c0 tx=0x7ff944009cd0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.375+0000 7ff94f845700 1 -- 172.21.15.46:0/2231983372 shutdown_connections 2024-05-09T04:52:26.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.375+0000 7ff94f845700 1 --2- 172.21.15.46:0/2231983372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff948108760 0x7ff9481100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.375+0000 7ff94f845700 1 --2- 172.21.15.46:0/2231983372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff9481034e0 0x7ff9481038f0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.375+0000 7ff94f845700 1 --2- 172.21.15.46:0/2231983372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff948107e10 0x7ff948108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.375+0000 7ff94f845700 1 -- 172.21.15.46:0/2231983372 >> 172.21.15.46:0/2231983372 conn(0x7ff9480fef30 msgr2=0x7ff948101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:26.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.375+0000 7ff94f845700 1 -- 172.21.15.46:0/2231983372 shutdown_connections 2024-05-09T04:52:26.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.376+0000 7ff94f845700 1 -- 172.21.15.46:0/2231983372 wait complete. 2024-05-09T04:52:26.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.376+0000 7ff94f845700 1 Processor -- start 2024-05-09T04:52:26.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.376+0000 7ff94f845700 1 -- start start 2024-05-09T04:52:26.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.377+0000 7ff94f845700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9481034e0 0x7ff948199d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.377+0000 7ff94f845700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff948107e10 0x7ff94819a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.377+0000 7ff94f845700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff948108760 0x7ff94819e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.377+0000 7ff94f845700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff948113390 con 0x7ff948107e10 2024-05-09T04:52:26.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.377+0000 7ff94f845700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff948113210 con 0x7ff9481034e0 2024-05-09T04:52:26.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.377+0000 7ff94cde0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff948107e10 0x7ff94819a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.377+0000 7ff94f845700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff948113510 con 0x7ff948108760 2024-05-09T04:52:26.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.377+0000 7ff94d5e1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9481034e0 0x7ff948199d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.377+0000 7ff94cde0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff948107e10 0x7ff94819a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52512/0 (socket says 172.21.15.46:52512) 2024-05-09T04:52:26.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.377+0000 7ff94cde0700 1 -- 172.21.15.46:0/3580286328 learned_addr learned my addr 172.21.15.46:0/3580286328 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:26.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.377+0000 7ff94dde2700 1 --2- 172.21.15.46:0/3580286328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff948108760 0x7ff94819e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.378+0000 7ff94cde0700 1 -- 172.21.15.46:0/3580286328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff948108760 msgr2=0x7ff94819e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:26.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.378+0000 7ff94cde0700 1 --2- 172.21.15.46:0/3580286328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff948108760 0x7ff94819e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.378+0000 7ff94cde0700 1 -- 172.21.15.46:0/3580286328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9481034e0 msgr2=0x7ff948199d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:26.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.378+0000 7ff94cde0700 1 --2- 172.21.15.46:0/3580286328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9481034e0 0x7ff948199d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.378+0000 7ff94cde0700 1 -- 172.21.15.46:0/3580286328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff94819ec10 con 0x7ff948107e10 2024-05-09T04:52:26.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.378+0000 7ff94cde0700 1 --2- 172.21.15.46:0/3580286328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff948107e10 0x7ff94819a2e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff938007f00 tx=0x7ff93800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:26.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.378+0000 7ff93e7fc700 1 -- 172.21.15.46:0/3580286328 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938004220 con 0x7ff948107e10 2024-05-09T04:52:26.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.378+0000 7ff94f845700 1 -- 172.21.15.46:0/3580286328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff94819ef00 con 0x7ff948107e10 2024-05-09T04:52:26.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.378+0000 7ff94f845700 1 -- 172.21.15.46:0/3580286328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff94806c040 con 0x7ff948107e10 2024-05-09T04:52:26.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.378+0000 7ff93e7fc700 1 -- 172.21.15.46:0/3580286328 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9380043c0 con 0x7ff948107e10 2024-05-09T04:52:26.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.378+0000 7ff93e7fc700 1 -- 172.21.15.46:0/3580286328 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9380055c0 con 0x7ff948107e10 2024-05-09T04:52:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.382+0000 7ff95e25bb80 1 Processor -- start 2024-05-09T04:52:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.383+0000 7ff95e25bb80 1 -- start start 2024-05-09T04:52:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.383+0000 7ff95e25bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace1b0160 0x55dace2494f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.383+0000 7ff95e25bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace0c64a0 0x55dace249a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.383+0000 7ff95e25bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace2436a0 0x55dace243ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.383+0000 7ff95e25bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dace165ff0 con 0x55dace0c64a0 2024-05-09T04:52:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.383+0000 7ff95e25bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55dace1047f0 con 0x55dace2436a0 2024-05-09T04:52:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.383+0000 7ff95e25bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dace161fb0 con 0x55dace1b0160 2024-05-09T04:52:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.383+0000 7ff94dde2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace1b0160 0x55dace2494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.383+0000 7ff94dde2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace1b0160 0x55dace2494f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:60818/0 (socket says 172.21.15.46:60818) 2024-05-09T04:52:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.383+0000 7ff94dde2700 1 -- 172.21.15.46:0/4005392300 learned_addr learned my addr 172.21.15.46:0/4005392300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.383+0000 7ff93e7fc700 1 -- 172.21.15.46:0/3580286328 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7ff938004830 con 0x7ff948107e10 2024-05-09T04:52:26.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.383+0000 7ff94d5e1700 1 --2- 172.21.15.46:0/4005392300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace0c64a0 0x55dace249a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.384+0000 7ff94dde2700 1 -- 172.21.15.46:0/4005392300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace2436a0 msgr2=0x55dace243ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:52:26.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.384+0000 7ff94dde2700 1 --2- 172.21.15.46:0/4005392300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace2436a0 0x55dace243ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.384+0000 7ff94dde2700 1 -- 172.21.15.46:0/4005392300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace0c64a0 msgr2=0x55dace249a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:26.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.384+0000 7ff94dde2700 1 --2- 172.21.15.46:0/4005392300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace0c64a0 0x55dace249a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.384+0000 7ff94dde2700 1 -- 172.21.15.46:0/4005392300 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dace0fc370 con 0x55dace1b0160 2024-05-09T04:52:26.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.384+0000 7ff94dde2700 1 --2- 172.21.15.46:0/4005392300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace1b0160 0x55dace2494f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff940011680 tx=0x7ff940009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6bb150fedddfed7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.384+0000 7ff9337fe700 1 -- 172.21.15.46:0/4005392300 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940019030 con 0x55dace1b0160 2024-05-09T04:52:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.385+0000 7ff9337fe700 1 -- 172.21.15.46:0/4005392300 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9400093a0 con 0x55dace1b0160 2024-05-09T04:52:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.385+0000 7ff9337fe700 1 -- 172.21.15.46:0/4005392300 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff940007500 con 0x55dace1b0160 2024-05-09T04:52:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.385+0000 7ff93e7fc700 1 --2- 172.21.15.46:0/3580286328 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff9340b2b10 0x7ff9340b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.385+0000 7ff94d5e1700 1 --2- 172.21.15.46:0/3580286328 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff9340b2b10 0x7ff9340b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.385+0000 7ff93e7fc700 1 -- 172.21.15.46:0/3580286328 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7ff9380ea870 con 0x7ff948107e10 2024-05-09T04:52:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.386+0000 7ff95e25bb80 1 -- 172.21.15.46:0/4005392300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace1b0160 msgr2=0x55dace2494f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.386+0000 7ff94d5e1700 1 --2- 172.21.15.46:0/3580286328 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff9340b2b10 0x7ff9340b5020 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff944006010 tx=0x7ff94401e040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.386+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/4005392300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace1b0160 0x55dace2494f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff940011680 tx=0x7ff940009920 comp rx=0 tx=0).stop 2024-05-09T04:52:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.386+0000 7ff95e25bb80 1 -- 172.21.15.46:0/4005392300 shutdown_connections 2024-05-09T04:52:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.386+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/4005392300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace2436a0 0x55dace243ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.386+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/4005392300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace1b0160 0x55dace2494f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.386+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/4005392300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace0c64a0 0x55dace249a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.386+0000 7ff95e25bb80 1 -- 172.21.15.46:0/4005392300 >> 172.21.15.46:0/4005392300 conn(0x55dace1266a0 msgr2=0x55dace23e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.386+0000 7ff95e25bb80 1 -- 172.21.15.46:0/4005392300 shutdown_connections 2024-05-09T04:52:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.386+0000 7ff95e25bb80 1 -- 172.21.15.46:0/4005392300 wait complete. 2024-05-09T04:52:26.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.387+0000 7ff95e25bb80 1 -- start start 2024-05-09T04:52:26.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.387+0000 7ff95e25bb80 1 Processor -- start 2024-05-09T04:52:26.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.388+0000 7ff95e25bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace0c64a0 0x55dace24d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.388+0000 7ff95e25bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace1b0160 0x55dace24d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.388+0000 7ff95e25bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace2436a0 0x55dace24dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.388+0000 7ff94dde2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace2436a0 0x55dace24dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.388+0000 7ff94d5e1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace0c64a0 0x55dace24d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.388+0000 7ff94dde2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace2436a0 0x55dace24dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52534/0 (socket says 172.21.15.46:52534) 2024-05-09T04:52:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.388+0000 7ff94dde2700 1 -- 172.21.15.46:0/1772475574 learned_addr learned my addr 172.21.15.46:0/1772475574 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.388+0000 7ff95e25bb80 1 -- 172.21.15.46:0/1772475574 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dacdfe1ba0 con 0x55dace2436a0 2024-05-09T04:52:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.388+0000 7ff94cde0700 1 --2- 172.21.15.46:0/1772475574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace1b0160 0x55dace24d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.388+0000 7ff95e25bb80 1 -- 172.21.15.46:0/1772475574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55dace120150 con 0x55dace0c64a0 2024-05-09T04:52:26.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.388+0000 7ff95e25bb80 1 -- 172.21.15.46:0/1772475574 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dace2501d0 con 0x55dace1b0160 2024-05-09T04:52:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.389+0000 7ff94d5e1700 1 -- 172.21.15.46:0/1772475574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace1b0160 msgr2=0x55dace24d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.389+0000 7ff94d5e1700 1 --2- 172.21.15.46:0/1772475574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace1b0160 0x55dace24d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.389+0000 7ff94d5e1700 1 -- 172.21.15.46:0/1772475574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace2436a0 msgr2=0x55dace24dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.389+0000 7ff94d5e1700 1 --2- 172.21.15.46:0/1772475574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace2436a0 0x55dace24dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.389+0000 7ff94d5e1700 1 -- 172.21.15.46:0/1772475574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dacdfe1490 con 0x55dace0c64a0 2024-05-09T04:52:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.389+0000 7ff94dde2700 1 --2- 172.21.15.46:0/1772475574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace2436a0 0x55dace24dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.389+0000 7ff94cde0700 1 --2- 172.21.15.46:0/1772475574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace1b0160 0x55dace24d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:52:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.389+0000 7ff95e25bb80 10 client.0 _ll_register_callbacks cb 0x55dacdfd76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.389+0000 7ff94d5e1700 1 --2- 172.21.15.46:0/1772475574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace0c64a0 0x55dace24d2c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff944009990 tx=0x7ff944009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3450634c4240176 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.389+0000 7ff95e25bb80 1 -- 172.21.15.46:0/1772475574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dace1739d0 con 0x55dace0c64a0 2024-05-09T04:52:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.389+0000 7ff930ff9700 10 client.25355 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:52:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.389+0000 7ff930ff9700 1 -- 172.21.15.46:0/1772475574 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94400ba60 con 0x55dace0c64a0 2024-05-09T04:52:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.389+0000 7ff930ff9700 1 -- 172.21.15.46:0/1772475574 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7ff94400bc00 con 0x55dace0c64a0 2024-05-09T04:52:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.390+0000 7ff930ff9700 1 -- 172.21.15.46:0/1772475574 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff94400be40 con 0x55dace0c64a0 2024-05-09T04:52:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.390+0000 7ff95e25bb80 10 client.25355 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.390+0000 7ff95e25bb80 1 -- 172.21.15.46:0/1772475574 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55dace25aa80 con 0x55dace0c64a0 2024-05-09T04:52:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.390+0000 7ff930ff9700 1 -- 172.21.15.46:0/1772475574 <== mon.2 v2:172.21.15.72:3300/0 4 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff94403b420 con 0x55dace0c64a0 2024-05-09T04:52:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.390+0000 7ff95e25bb80 10 client.25355 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:26.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.390+0000 7ff95e25bb80 10 client.25355 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:26.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.390+0000 7ff95e25bb80 20 client.25355 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:26.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.390+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/1772475574 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dace25d530 0x55dace25fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.390+0000 7ff95e25bb80 4 client.25355 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:26.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.390+0000 7ff95e25bb80 1 -- 172.21.15.46:0/1772475574 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dace16da70 con 0x55dace25d530 2024-05-09T04:52:26.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.390+0000 7ff94cde0700 1 --2- 172.21.15.46:0/1772475574 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dace25d530 0x55dace25fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.391+0000 7ff930ff9700 10 client.25355 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:26.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.391+0000 7ff94cde0700 1 --2- 172.21.15.46:0/1772475574 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dace25d530 0x55dace25fa40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11483171f02bf455 server_cookie=27c2601cf55f4b77 in_seq=0 out_seq=0 2024-05-09T04:52:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.392+0000 7ff930ff9700 1 -- 172.21.15.46:0/1772475574 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dace16da70 con 0x55dace25d530 2024-05-09T04:52:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.392+0000 7ff930ff9700 10 client.25355 handle_command_reply: tid=0 2024-05-09T04:52:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.392+0000 7ff95e25bb80 1 client.25355 shutdown 2024-05-09T04:52:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.392+0000 7ff95e25bb80 1 -- 172.21.15.46:0/1772475574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace0c64a0 msgr2=0x55dace24d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.392+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/1772475574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace0c64a0 0x55dace24d2c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff944009990 tx=0x7ff944009cd0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.392+0000 7ff95e25bb80 1 -- 172.21.15.46:0/1772475574 shutdown_connections 2024-05-09T04:52:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.392+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/1772475574 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace2436a0 0x55dace24dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.392+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/1772475574 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dace25d530 0x55dace25fa40 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.392+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/1772475574 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace0c64a0 0x55dace24d2c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.392+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/1772475574 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace1b0160 0x55dace24d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.392+0000 7ff95e25bb80 1 -- 172.21.15.46:0/1772475574 >> 172.21.15.46:0/1772475574 conn(0x55dace1266a0 msgr2=0x55dace23e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.392+0000 7ff930ff9700 0 client.25355 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.393+0000 7ff95e25bb80 1 -- 172.21.15.46:0/1772475574 shutdown_connections 2024-05-09T04:52:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.393+0000 7ff95e25bb80 1 -- 172.21.15.46:0/1772475574 wait complete. 2024-05-09T04:52:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.393+0000 7ff95e25bb80 20 client.25355 trim_cache size 0 max 16384 2024-05-09T04:52:26.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.408+0000 7ff95e25bb80 1 Processor -- start 2024-05-09T04:52:26.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.409+0000 7ff95e25bb80 1 -- start start 2024-05-09T04:52:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.409+0000 7ff95e25bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace0c64a0 0x55dace2488e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.409+0000 7ff95e25bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace1b0160 0x55dace24ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.409+0000 7ff95e25bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace2436a0 0x55dace24db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.409+0000 7ff95e25bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c00fef0 con 0x55dace1b0160 2024-05-09T04:52:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.409+0000 7ff95e25bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c00d780 con 0x55dace0c64a0 2024-05-09T04:52:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.409+0000 7ff95e25bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c00b5d0 con 0x55dace2436a0 2024-05-09T04:52:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.409+0000 7ff94d5e1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace0c64a0 0x55dace2488e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.409+0000 7ff94cde0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace1b0160 0x55dace24ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.409+0000 7ff94d5e1700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace0c64a0 0x55dace2488e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:38064/0 (socket says 172.21.15.46:38064) 2024-05-09T04:52:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.409+0000 7ff94d5e1700 1 -- 172.21.15.46:0/3476868882 learned_addr learned my addr 172.21.15.46:0/3476868882 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.409+0000 7ff94dde2700 1 --2- 172.21.15.46:0/3476868882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace2436a0 0x55dace24db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.410+0000 7ff94d5e1700 1 -- 172.21.15.46:0/3476868882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace2436a0 msgr2=0x55dace24db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.410+0000 7ff94d5e1700 1 --2- 172.21.15.46:0/3476868882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace2436a0 0x55dace24db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.410+0000 7ff94d5e1700 1 -- 172.21.15.46:0/3476868882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace1b0160 msgr2=0x55dace24ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.410+0000 7ff94d5e1700 1 --2- 172.21.15.46:0/3476868882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace1b0160 0x55dace24ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.410+0000 7ff94d5e1700 1 -- 172.21.15.46:0/3476868882 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dace237f60 con 0x55dace0c64a0 2024-05-09T04:52:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.410+0000 7ff94cde0700 1 --2- 172.21.15.46:0/3476868882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace1b0160 0x55dace24ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:52:26.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.410+0000 7ff94d5e1700 1 --2- 172.21.15.46:0/3476868882 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace0c64a0 0x55dace2488e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff944009940 tx=0x7ff94402a070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af5462425caf2fc2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:26.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.410+0000 7ff932ffd700 1 -- 172.21.15.46:0/3476868882 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff94403b420 con 0x55dace0c64a0 2024-05-09T04:52:26.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.410+0000 7ff932ffd700 1 -- 172.21.15.46:0/3476868882 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55dace1739d0 con 0x55dace0c64a0 2024-05-09T04:52:26.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.410+0000 7ff932ffd700 1 -- 172.21.15.46:0/3476868882 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff944028410 con 0x55dace0c64a0 2024-05-09T04:52:26.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.411+0000 7ff95e25bb80 1 -- 172.21.15.46:0/3476868882 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace0c64a0 msgr2=0x55dace2488e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:26.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.411+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/3476868882 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace0c64a0 0x55dace2488e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff944009940 tx=0x7ff94402a070 comp rx=0 tx=0).stop 2024-05-09T04:52:26.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.411+0000 7ff95e25bb80 1 -- 172.21.15.46:0/3476868882 shutdown_connections 2024-05-09T04:52:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.411+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/3476868882 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace2436a0 0x55dace24db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.412+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/3476868882 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace0c64a0 0x55dace2488e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.412+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/3476868882 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace1b0160 0x55dace24ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.412+0000 7ff95e25bb80 1 -- 172.21.15.46:0/3476868882 >> 172.21.15.46:0/3476868882 conn(0x55dace1266a0 msgr2=0x55dace246e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.412+0000 7ff95e25bb80 1 -- 172.21.15.46:0/3476868882 shutdown_connections 2024-05-09T04:52:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.412+0000 7ff95e25bb80 1 -- 172.21.15.46:0/3476868882 wait complete. 2024-05-09T04:52:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.412+0000 7ff95e25bb80 1 -- start start 2024-05-09T04:52:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.413+0000 7ff95e25bb80 1 Processor -- start 2024-05-09T04:52:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.414+0000 7ff95e25bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace0c64a0 0x55dace255320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.414+0000 7ff95e25bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace1b0160 0x55dace24fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.414+0000 7ff94d5e1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace0c64a0 0x55dace255320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.414+0000 7ff94d5e1700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace0c64a0 0x55dace255320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:60836/0 (socket says 172.21.15.46:60836) 2024-05-09T04:52:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.414+0000 7ff94cde0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace1b0160 0x55dace24fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.414+0000 7ff94cde0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace1b0160 0x55dace24fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52548/0 (socket says 172.21.15.46:52548) 2024-05-09T04:52:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.414+0000 7ff94d5e1700 1 -- 172.21.15.46:0/601001289 learned_addr learned my addr 172.21.15.46:0/601001289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.414+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/601001289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace2436a0 0x55dace250410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.414+0000 7ff95e25bb80 1 -- 172.21.15.46:0/601001289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c00bc40 con 0x55dace1b0160 2024-05-09T04:52:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.414+0000 7ff95e25bb80 1 -- 172.21.15.46:0/601001289 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c00d0b0 con 0x55dace2436a0 2024-05-09T04:52:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.415+0000 7ff95e25bb80 1 -- 172.21.15.46:0/601001289 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff92c00c310 con 0x55dace0c64a0 2024-05-09T04:52:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.415+0000 7ff94dde2700 1 --2- 172.21.15.46:0/601001289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace2436a0 0x55dace250410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.415+0000 7ff94cde0700 1 -- 172.21.15.46:0/601001289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace0c64a0 msgr2=0x55dace255320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.415+0000 7ff94cde0700 1 --2- 172.21.15.46:0/601001289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace0c64a0 0x55dace255320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.415+0000 7ff94cde0700 1 -- 172.21.15.46:0/601001289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace2436a0 msgr2=0x55dace250410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.415+0000 7ff94cde0700 1 --2- 172.21.15.46:0/601001289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace2436a0 0x55dace250410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.415+0000 7ff94cde0700 1 -- 172.21.15.46:0/601001289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dace250a00 con 0x55dace1b0160 2024-05-09T04:52:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.416+0000 7ff95e25bb80 10 client.0 _ll_register_callbacks cb 0x55dacdfd76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.416+0000 7ff94cde0700 1 --2- 172.21.15.46:0/601001289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace1b0160 0x55dace24fed0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff9380e7f30 tx=0x7ff9380eab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf9da2fdae0534f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.416+0000 7ff930ff9700 10 client.15606 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.416+0000 7ff95e25bb80 1 -- 172.21.15.46:0/601001289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55dace1739d0 con 0x55dace1b0160 2024-05-09T04:52:26.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.416+0000 7ff930ff9700 1 -- 172.21.15.46:0/601001289 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff938025040 con 0x55dace1b0160 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.416+0000 7ff930ff9700 1 -- 172.21.15.46:0/601001289 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7ff9380ba3c0 con 0x55dace1b0160 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.416+0000 7ff930ff9700 1 -- 172.21.15.46:0/601001289 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff9380b1160 con 0x55dace1b0160 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.416+0000 7ff95e25bb80 10 client.15606 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.416+0000 7ff95e25bb80 1 -- 172.21.15.46:0/601001289 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=25}) -- 0x55dace2c6a60 con 0x55dace1b0160 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.417+0000 7ff930ff9700 1 -- 172.21.15.46:0/601001289 <== mon.0 v2:172.21.15.46:3300/0 4 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff9380e8100 con 0x55dace1b0160 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.417+0000 7ff95e25bb80 10 client.15606 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.417+0000 7ff95e25bb80 10 client.15606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.417+0000 7ff95e25bb80 20 client.15606 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.417+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/601001289 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dace2c6f20 0x55dace2c9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.417+0000 7ff95e25bb80 4 client.15606 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.417+0000 7ff95e25bb80 1 -- 172.21.15.46:0/601001289 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dace1739d0 con 0x55dace2c6f20 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.417+0000 7ff94d5e1700 1 --2- 172.21.15.46:0/601001289 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dace2c6f20 0x55dace2c9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.418+0000 7ff94d5e1700 1 --2- 172.21.15.46:0/601001289 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dace2c6f20 0x55dace2c9390 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa7a934320d8b45 server_cookie=f3c693c095060355 in_seq=0 out_seq=0 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.418+0000 7ff930ff9700 10 client.15606 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.418+0000 7ff930ff9700 1 -- 172.21.15.46:0/601001289 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dace1739d0 con 0x55dace2c6f20 2024-05-09T04:52:26.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.418+0000 7ff930ff9700 10 client.15606 handle_command_reply: tid=0 2024-05-09T04:52:26.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.418+0000 7ff95e25bb80 1 client.15606 shutdown 2024-05-09T04:52:26.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.418+0000 7ff95e25bb80 1 -- 172.21.15.46:0/601001289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace1b0160 msgr2=0x55dace24fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:26.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.418+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/601001289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace1b0160 0x55dace24fed0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7ff9380e7f30 tx=0x7ff9380eab70 comp rx=0 tx=0).stop 2024-05-09T04:52:26.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.418+0000 7ff95e25bb80 1 -- 172.21.15.46:0/601001289 shutdown_connections 2024-05-09T04:52:26.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.418+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/601001289 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dace2c6f20 0x55dace2c9390 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.418+0000 7ff930ff9700 0 client.15606 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:26.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.418+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/601001289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dace2436a0 0x55dace250410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.418+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/601001289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dace0c64a0 0x55dace255320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.418+0000 7ff95e25bb80 1 --2- 172.21.15.46:0/601001289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dace1b0160 0x55dace24fed0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.418+0000 7ff95e25bb80 1 -- 172.21.15.46:0/601001289 >> 172.21.15.46:0/601001289 conn(0x55dace1266a0 msgr2=0x55dace242690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:26.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.419+0000 7ff95e25bb80 1 -- 172.21.15.46:0/601001289 shutdown_connections 2024-05-09T04:52:26.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.419+0000 7ff95e25bb80 1 -- 172.21.15.46:0/601001289 wait complete. 2024-05-09T04:52:26.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.419+0000 7ff95e25bb80 20 client.15606 trim_cache size 0 max 16384 2024-05-09T04:52:26.423 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:52:26.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.420+0000 7ff94f845700 1 -- 172.21.15.46:0/3580286328 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff9340b2b10 msgr2=0x7ff9340b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:26.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.420+0000 7ff94f845700 1 --2- 172.21.15.46:0/3580286328 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff9340b2b10 0x7ff9340b5020 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff944006010 tx=0x7ff94401e040 comp rx=0 tx=0).stop 2024-05-09T04:52:26.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.420+0000 7ff94f845700 1 -- 172.21.15.46:0/3580286328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff948107e10 msgr2=0x7ff94819a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:26.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.420+0000 7ff94f845700 1 --2- 172.21.15.46:0/3580286328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff948107e10 0x7ff94819a2e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff938007f00 tx=0x7ff93800dbb0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.420+0000 7ff94f845700 1 -- 172.21.15.46:0/3580286328 shutdown_connections 2024-05-09T04:52:26.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.420+0000 7ff94f845700 1 --2- 172.21.15.46:0/3580286328 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff9340b2b10 0x7ff9340b5020 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.420+0000 7ff94f845700 1 --2- 172.21.15.46:0/3580286328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff948108760 0x7ff94819e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.420+0000 7ff94f845700 1 --2- 172.21.15.46:0/3580286328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff9481034e0 0x7ff948199d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.420+0000 7ff94f845700 1 --2- 172.21.15.46:0/3580286328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff948107e10 0x7ff94819a2e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:26.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.420+0000 7ff94f845700 1 -- 172.21.15.46:0/3580286328 >> 172.21.15.46:0/3580286328 conn(0x7ff9480fef30 msgr2=0x7ff9481063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:26.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.420+0000 7ff94f845700 1 -- 172.21.15.46:0/3580286328 shutdown_connections 2024-05-09T04:52:26.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:26.420+0000 7ff94f845700 1 -- 172.21.15.46:0/3580286328 wait complete. 2024-05-09T04:52:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:26 smithi110 ceph-mon[34904]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T04:52:27.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:26 smithi046 ceph-mon[30136]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T04:52:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:26 smithi072 ceph-mon[34088]: pgmap v43: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T04:52:28.248 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:52:28.249 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:52:28.606 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:29.020 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:28 smithi046 ceph-mon[30136]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-09T04:52:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:28 smithi110 ceph-mon[34904]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-09T04:52:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:28 smithi072 ceph-mon[34088]: pgmap v44: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2024-05-09T04:52:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.596+0000 7f44c7ff5700 1 -- 172.21.15.46:0/2077626005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c01034c0 msgr2=0x7f44c01038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.596+0000 7f44c7ff5700 1 --2- 172.21.15.46:0/2077626005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c01034c0 0x7f44c01038d0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0099c0 tx=0x7f44bc009cd0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.596+0000 7f44c7ff5700 1 -- 172.21.15.46:0/2077626005 shutdown_connections 2024-05-09T04:52:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.596+0000 7f44c7ff5700 1 --2- 172.21.15.46:0/2077626005 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44c0108740 0x7f44c01100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.596+0000 7f44c7ff5700 1 --2- 172.21.15.46:0/2077626005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c01034c0 0x7f44c01038d0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.596+0000 7f44c7ff5700 1 --2- 172.21.15.46:0/2077626005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44c0107df0 0x7f44c0108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.596+0000 7f44c7ff5700 1 -- 172.21.15.46:0/2077626005 >> 172.21.15.46:0/2077626005 conn(0x7f44c00fedc0 msgr2=0x7f44c01011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.597+0000 7f44c7ff5700 1 -- 172.21.15.46:0/2077626005 shutdown_connections 2024-05-09T04:52:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.597+0000 7f44c7ff5700 1 -- 172.21.15.46:0/2077626005 wait complete. 2024-05-09T04:52:30.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.598+0000 7f44c7ff5700 1 Processor -- start 2024-05-09T04:52:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.598+0000 7f44c7ff5700 1 -- start start 2024-05-09T04:52:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.599+0000 7f44c7ff5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44c01034c0 0x7f44c019f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.599+0000 7f44c7ff5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c0107df0 0x7f44c019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.599+0000 7f44c7ff5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44c0108740 0x7f44c0199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.599+0000 7f44c5590700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c0107df0 0x7f44c019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.599+0000 7f44c5d91700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44c01034c0 0x7f44c019f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.599+0000 7f44c5590700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c0107df0 0x7f44c019fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52572/0 (socket says 172.21.15.46:52572) 2024-05-09T04:52:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.599+0000 7f44c5d91700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44c01034c0 0x7f44c019f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:38080/0 (socket says 172.21.15.46:38080) 2024-05-09T04:52:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.599+0000 7f44c6592700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44c0108740 0x7f44c0199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.599+0000 7f44c6592700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44c0108740 0x7f44c0199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:60850/0 (socket says 172.21.15.46:60850) 2024-05-09T04:52:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.600+0000 7f44c5590700 1 -- 172.21.15.46:0/3719139534 learned_addr learned my addr 172.21.15.46:0/3719139534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.600+0000 7f44c7ff5700 1 -- 172.21.15.46:0/3719139534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01133c0 con 0x7f44c0107df0 2024-05-09T04:52:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.600+0000 7f44c7ff5700 1 -- 172.21.15.46:0/3719139534 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f44c0113240 con 0x7f44c01034c0 2024-05-09T04:52:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.600+0000 7f44c7ff5700 1 -- 172.21.15.46:0/3719139534 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f44c0113540 con 0x7f44c0108740 2024-05-09T04:52:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.600+0000 7f44c5d91700 1 -- 172.21.15.46:0/3719139534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44c0108740 msgr2=0x7f44c0199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.600+0000 7f44c5d91700 1 --2- 172.21.15.46:0/3719139534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44c0108740 0x7f44c0199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.600+0000 7f44c5d91700 1 -- 172.21.15.46:0/3719139534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c0107df0 msgr2=0x7f44c019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.600+0000 7f44c5d91700 1 --2- 172.21.15.46:0/3719139534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c0107df0 0x7f44c019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.602 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.600+0000 7f44c5d91700 1 -- 172.21.15.46:0/3719139534 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44c019a0b0 con 0x7f44c01034c0 2024-05-09T04:52:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.600+0000 7f44c5590700 1 --2- 172.21.15.46:0/3719139534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c0107df0 0x7f44c019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.601+0000 7f44c5d91700 1 --2- 172.21.15.46:0/3719139534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44c01034c0 0x7f44c019f7e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0098e0 tx=0x7f44bc009290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.601+0000 7f44b6ffd700 1 -- 172.21.15.46:0/3719139534 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc02c070 con 0x7f44c01034c0 2024-05-09T04:52:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.601+0000 7f44b6ffd700 1 -- 172.21.15.46:0/3719139534 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44bc004660 con 0x7f44c01034c0 2024-05-09T04:52:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.601+0000 7f44b6ffd700 1 -- 172.21.15.46:0/3719139534 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc00f6a0 con 0x7f44c01034c0 2024-05-09T04:52:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.601+0000 7f44c7ff5700 1 -- 172.21.15.46:0/3719139534 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44c019a340 con 0x7f44c01034c0 2024-05-09T04:52:30.603 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.601+0000 7f44c7ff5700 1 -- 172.21.15.46:0/3719139534 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44c019a7d0 con 0x7f44c01034c0 2024-05-09T04:52:30.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.605+0000 7f44b6ffd700 1 -- 172.21.15.46:0/3719139534 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f44bc02f070 con 0x7f44c01034c0 2024-05-09T04:52:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.606+0000 7f44b6ffd700 1 --2- 172.21.15.46:0/3719139534 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f44ac0b2b10 0x7f44ac0b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.606+0000 7f44d6a0bb80 1 Processor -- start 2024-05-09T04:52:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.607+0000 7f44b6ffd700 1 -- 172.21.15.46:0/3719139534 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f44bc0eb530 con 0x7f44c01034c0 2024-05-09T04:52:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.607+0000 7f44c5590700 1 --2- 172.21.15.46:0/3719139534 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f44ac0b2b10 0x7f44ac0b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.607+0000 7f44d6a0bb80 1 -- start start 2024-05-09T04:52:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.608+0000 7f44c5590700 1 --2- 172.21.15.46:0/3719139534 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f44ac0b2b10 0x7f44ac0b5020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f44c0102710 tx=0x7f44b0009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.608+0000 7f44d6a0bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c76561e0 0x5583c76ef570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.608+0000 7f44d6a0bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c756c520 0x5583c76efab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.608+0000 7f44d6a0bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76e9720 0x5583c76e9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.608+0000 7f44d6a0bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5583c760c070 con 0x5583c76e9720 2024-05-09T04:52:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.608+0000 7f44d6a0bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5583c75aa870 con 0x5583c756c520 2024-05-09T04:52:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.608+0000 7f44d6a0bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5583c7608030 con 0x5583c76561e0 2024-05-09T04:52:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.608+0000 7f44c5d91700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c76561e0 0x5583c76ef570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.608+0000 7f44c5d91700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c76561e0 0x5583c76ef570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:60854/0 (socket says 172.21.15.46:60854) 2024-05-09T04:52:30.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.608+0000 7f44c5d91700 1 -- 172.21.15.46:0/1800489885 learned_addr learned my addr 172.21.15.46:0/1800489885 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.608+0000 7f44c6592700 1 --2- 172.21.15.46:0/1800489885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76e9720 0x5583c76e9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.608+0000 7f44c5d91700 1 -- 172.21.15.46:0/1800489885 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c756c520 msgr2=0x5583c76efab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.608+0000 7f44c5d91700 1 --2- 172.21.15.46:0/1800489885 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c756c520 0x5583c76efab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.608+0000 7f44c5d91700 1 -- 172.21.15.46:0/1800489885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76e9720 msgr2=0x5583c76e9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.609+0000 7f44c5d91700 1 --2- 172.21.15.46:0/1800489885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76e9720 0x5583c76e9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.609+0000 7f44c5d91700 1 -- 172.21.15.46:0/1800489885 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583c75a23f0 con 0x5583c76561e0 2024-05-09T04:52:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.609+0000 7f44c6592700 1 --2- 172.21.15.46:0/1800489885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76e9720 0x5583c76e9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:30.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.609+0000 7f44c5d91700 1 --2- 172.21.15.46:0/1800489885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c76561e0 0x5583c76ef570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f44bc006010 tx=0x7f44bc0b2000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80cfffda2c0ed2b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.609+0000 7f44abfff700 1 -- 172.21.15.46:0/1800489885 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc0b2140 con 0x5583c76561e0 2024-05-09T04:52:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.609+0000 7f44abfff700 1 -- 172.21.15.46:0/1800489885 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44bc0b2d30 con 0x5583c76561e0 2024-05-09T04:52:30.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.609+0000 7f44abfff700 1 -- 172.21.15.46:0/1800489885 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44bc0bb1d0 con 0x5583c76561e0 2024-05-09T04:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.610+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/1800489885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c76561e0 msgr2=0x5583c76ef570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.610+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/1800489885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c76561e0 0x5583c76ef570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f44bc006010 tx=0x7f44bc0b2000 comp rx=0 tx=0).stop 2024-05-09T04:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.611+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/1800489885 shutdown_connections 2024-05-09T04:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.611+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/1800489885 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76e9720 0x5583c76e9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.611+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/1800489885 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c76561e0 0x5583c76ef570 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.611+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/1800489885 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c756c520 0x5583c76efab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.611+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/1800489885 >> 172.21.15.46:0/1800489885 conn(0x5583c75cc720 msgr2=0x5583c76e4480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.611+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/1800489885 shutdown_connections 2024-05-09T04:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.611+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/1800489885 wait complete. 2024-05-09T04:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.611+0000 7f44d6a0bb80 1 -- start start 2024-05-09T04:52:30.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.612+0000 7f44d6a0bb80 1 Processor -- start 2024-05-09T04:52:30.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.612+0000 7f44d6a0bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c756c520 0x5583c76f8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.612+0000 7f44d6a0bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c76561e0 0x5583c76f90f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.612+0000 7f44d6a0bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76e9720 0x5583c76f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.612+0000 7f44d6a0bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5583c7487c20 con 0x5583c76e9720 2024-05-09T04:52:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.613+0000 7f44d6a0bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5583c75c61d0 con 0x5583c76561e0 2024-05-09T04:52:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.613+0000 7f44d6a0bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5583c7608030 con 0x5583c756c520 2024-05-09T04:52:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.613+0000 7f44c6592700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76e9720 0x5583c76f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.613+0000 7f44c6592700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76e9720 0x5583c76f2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52598/0 (socket says 172.21.15.46:52598) 2024-05-09T04:52:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.613+0000 7f44c6592700 1 -- 172.21.15.46:0/2428141276 learned_addr learned my addr 172.21.15.46:0/2428141276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.613+0000 7f44c5d91700 1 --2- 172.21.15.46:0/2428141276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c756c520 0x5583c76f8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.613+0000 7f44c6592700 1 -- 172.21.15.46:0/2428141276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c756c520 msgr2=0x5583c76f8bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.613+0000 7f44c6592700 1 --2- 172.21.15.46:0/2428141276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c756c520 0x5583c76f8bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.613+0000 7f44c6592700 1 -- 172.21.15.46:0/2428141276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c76561e0 msgr2=0x5583c76f90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:30.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.613+0000 7f44c6592700 1 --2- 172.21.15.46:0/2428141276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c76561e0 0x5583c76f90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.613+0000 7f44c6592700 1 -- 172.21.15.46:0/2428141276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583c7487510 con 0x5583c76e9720 2024-05-09T04:52:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.613+0000 7f44d6a0bb80 10 client.0 _ll_register_callbacks cb 0x5583c747d740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.613+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/2428141276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44bc0b2d30 con 0x5583c76e9720 2024-05-09T04:52:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.613+0000 7f44c6592700 1 --2- 172.21.15.46:0/2428141276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76e9720 0x5583c76f2fb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f44b8007f70 tx=0x7f44b8009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14acc31261f6fec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.613+0000 7f44a97fa700 10 client.15618 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.614+0000 7f44a97fa700 1 -- 172.21.15.46:0/2428141276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b801c070 con 0x5583c76e9720 2024-05-09T04:52:30.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.614+0000 7f44a97fa700 1 -- 172.21.15.46:0/2428141276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f44b8003c00 con 0x5583c76e9720 2024-05-09T04:52:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.614+0000 7f44a97fa700 1 -- 172.21.15.46:0/2428141276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f44b8003e40 con 0x5583c76e9720 2024-05-09T04:52:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.614+0000 7f44d6a0bb80 10 client.15618 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.614+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/2428141276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5583c7700730 con 0x5583c76e9720 2024-05-09T04:52:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.615+0000 7f44a97fa700 1 -- 172.21.15.46:0/2428141276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f44b80138d0 con 0x5583c76e9720 2024-05-09T04:52:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.615+0000 7f44d6a0bb80 10 client.15618 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.615+0000 7f44d6a0bb80 10 client.15618 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.615+0000 7f44d6a0bb80 20 client.15618 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.615+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/2428141276 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583c7702cb0 0x5583c7705130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.615+0000 7f44d6a0bb80 4 client.15618 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.615+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/2428141276 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583c7619a50 con 0x5583c7702cb0 2024-05-09T04:52:30.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.615+0000 7f44c5d91700 1 --2- 172.21.15.46:0/2428141276 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583c7702cb0 0x5583c7705130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.616+0000 7f44c5d91700 1 --2- 172.21.15.46:0/2428141276 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583c7702cb0 0x5583c7705130 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7c7e9e92c7f87d3 server_cookie=295908c203387bca in_seq=0 out_seq=0 2024-05-09T04:52:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.616+0000 7f44a97fa700 10 client.15618 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:30.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.617+0000 7f44a97fa700 1 -- 172.21.15.46:0/2428141276 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583c7619a50 con 0x5583c7702cb0 2024-05-09T04:52:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.617+0000 7f44a97fa700 10 client.15618 handle_command_reply: tid=0 2024-05-09T04:52:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.617+0000 7f44d6a0bb80 1 client.15618 shutdown 2024-05-09T04:52:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.617+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/2428141276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76e9720 msgr2=0x5583c76f2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.617+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/2428141276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76e9720 0x5583c76f2fb0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f44b8007f70 tx=0x7f44b8009300 comp rx=0 tx=0).stop 2024-05-09T04:52:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.617+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/2428141276 shutdown_connections 2024-05-09T04:52:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.617+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/2428141276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76e9720 0x5583c76f2fb0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.617+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/2428141276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c756c520 0x5583c76f8bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.617+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/2428141276 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583c7702cb0 0x5583c7705130 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.617+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/2428141276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c76561e0 0x5583c76f90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.617+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/2428141276 >> 172.21.15.46:0/2428141276 conn(0x5583c75cc720 msgr2=0x5583c76e4240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.617+0000 7f44a97fa700 0 client.15618 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.617+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/2428141276 shutdown_connections 2024-05-09T04:52:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.617+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/2428141276 wait complete. 2024-05-09T04:52:30.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.618+0000 7f44d6a0bb80 20 client.15618 trim_cache size 0 max 16384 2024-05-09T04:52:30.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.660+0000 7f44d6a0bb80 1 Processor -- start 2024-05-09T04:52:30.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.660+0000 7f44d6a0bb80 1 -- start start 2024-05-09T04:52:30.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.661+0000 7f44d6a0bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c756c520 0x5583c76ee6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.661+0000 7f44d6a0bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76561e0 0x5583c770a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.661+0000 7f44d6a0bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c76e9720 0x5583c770ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.661+0000 7f44d6a0bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01af2d0 con 0x5583c76561e0 2024-05-09T04:52:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.661+0000 7f44c5590700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76561e0 0x5583c770a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.661+0000 7f44d6a0bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01acb60 con 0x5583c76e9720 2024-05-09T04:52:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.661+0000 7f44c5590700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76561e0 0x5583c770a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52600/0 (socket says 172.21.15.46:52600) 2024-05-09T04:52:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.661+0000 7f44c5590700 1 -- 172.21.15.46:0/1928014793 learned_addr learned my addr 172.21.15.46:0/1928014793 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.661+0000 7f44c5d91700 1 --2- 172.21.15.46:0/1928014793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c756c520 0x5583c76ee6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.661+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/1928014793 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01aa9b0 con 0x5583c756c520 2024-05-09T04:52:30.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.662+0000 7f44c6592700 1 --2- 172.21.15.46:0/1928014793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c76e9720 0x5583c770ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.662+0000 7f44c5590700 1 -- 172.21.15.46:0/1928014793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c756c520 msgr2=0x5583c76ee6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:30.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.662+0000 7f44c5590700 1 --2- 172.21.15.46:0/1928014793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c756c520 0x5583c76ee6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.662+0000 7f44c5590700 1 -- 172.21.15.46:0/1928014793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c76e9720 msgr2=0x5583c770ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:30.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.662+0000 7f44c5590700 1 --2- 172.21.15.46:0/1928014793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c76e9720 0x5583c770ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.662+0000 7f44c5590700 1 -- 172.21.15.46:0/1928014793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583c76e5d40 con 0x5583c76561e0 2024-05-09T04:52:30.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.662+0000 7f44c5d91700 1 --2- 172.21.15.46:0/1928014793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c756c520 0x5583c76ee6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:30.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.662+0000 7f44c6592700 1 --2- 172.21.15.46:0/1928014793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c76e9720 0x5583c770ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.662+0000 7f44c5590700 1 --2- 172.21.15.46:0/1928014793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76561e0 0x5583c770a8f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009cf0 tx=0x7f44b0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51ccfa72096a1575 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.663+0000 7f44ab7fe700 1 -- 172.21.15.46:0/1928014793 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0003a60 con 0x5583c76561e0 2024-05-09T04:52:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.663+0000 7f44ab7fe700 1 -- 172.21.15.46:0/1928014793 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b0023030 con 0x5583c76561e0 2024-05-09T04:52:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.663+0000 7f44ab7fe700 1 -- 172.21.15.46:0/1928014793 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b001f410 con 0x5583c76561e0 2024-05-09T04:52:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.664+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/1928014793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76561e0 msgr2=0x5583c770a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:30.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.664+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/1928014793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76561e0 0x5583c770a8f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009cf0 tx=0x7f44b0004750 comp rx=0 tx=0).stop 2024-05-09T04:52:30.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.664+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/1928014793 shutdown_connections 2024-05-09T04:52:30.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.664+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/1928014793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c76e9720 0x5583c770ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.664+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/1928014793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c756c520 0x5583c76ee6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.664+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/1928014793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c76561e0 0x5583c770a8f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.665+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/1928014793 >> 172.21.15.46:0/1928014793 conn(0x5583c75cc720 msgr2=0x5583c76edfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:30.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.665+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/1928014793 shutdown_connections 2024-05-09T04:52:30.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.665+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/1928014793 wait complete. 2024-05-09T04:52:30.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.665+0000 7f44d6a0bb80 1 -- start start 2024-05-09T04:52:30.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.666+0000 7f44d6a0bb80 1 Processor -- start 2024-05-09T04:52:30.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.666+0000 7f44d6a0bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c756c520 0x5583c76ff0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.666+0000 7f44d6a0bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c76561e0 0x5583c76fd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.667+0000 7f44d6a0bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c76e9720 0x5583c76fd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.667+0000 7f44d6a0bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01ab020 con 0x5583c756c520 2024-05-09T04:52:30.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.667+0000 7f44c5d91700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c756c520 0x5583c76ff0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.667+0000 7f44d6a0bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01ac490 con 0x5583c76561e0 2024-05-09T04:52:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.667+0000 7f44c5d91700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c756c520 0x5583c76ff0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52602/0 (socket says 172.21.15.46:52602) 2024-05-09T04:52:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.667+0000 7f44c5d91700 1 -- 172.21.15.46:0/3328377938 learned_addr learned my addr 172.21.15.46:0/3328377938 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.667+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/3328377938 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f44c01ab6f0 con 0x5583c76e9720 2024-05-09T04:52:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.667+0000 7f44c5590700 1 --2- 172.21.15.46:0/3328377938 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c76561e0 0x5583c76fd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.667+0000 7f44c6592700 1 --2- 172.21.15.46:0/3328377938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c76e9720 0x5583c76fd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.667+0000 7f44c6592700 1 -- 172.21.15.46:0/3328377938 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c76561e0 msgr2=0x5583c76fd460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.667+0000 7f44c6592700 1 --2- 172.21.15.46:0/3328377938 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c76561e0 0x5583c76fd460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.667+0000 7f44c6592700 1 -- 172.21.15.46:0/3328377938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c756c520 msgr2=0x5583c76ff0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.667+0000 7f44c6592700 1 --2- 172.21.15.46:0/3328377938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c756c520 0x5583c76ff0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.667+0000 7f44c6592700 1 -- 172.21.15.46:0/3328377938 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583c76ddfe0 con 0x5583c76e9720 2024-05-09T04:52:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.668+0000 7f44c6592700 1 --2- 172.21.15.46:0/3328377938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c76e9720 0x5583c76fd9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009340 tx=0x7f44b8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da6e117ff6c8ff8b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.668+0000 7f44a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:52:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.668+0000 7f44d6a0bb80 10 client.0 _ll_register_callbacks cb 0x5583c747d740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.668+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/3328377938 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b0023030 con 0x5583c76e9720 2024-05-09T04:52:30.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.668+0000 7f44a97fa700 1 -- 172.21.15.46:0/3328377938 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b8038050 con 0x5583c76e9720 2024-05-09T04:52:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.668+0000 7f44a97fa700 1 -- 172.21.15.46:0/3328377938 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f44b8005e60 con 0x5583c76e9720 2024-05-09T04:52:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.668+0000 7f44a97fa700 1 -- 172.21.15.46:0/3328377938 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f44b8003c90 con 0x5583c76e9720 2024-05-09T04:52:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.669+0000 7f44d6a0bb80 10 client.25333 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.669+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/3328377938 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5583c776de60 con 0x5583c76e9720 2024-05-09T04:52:30.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.669+0000 7f44a97fa700 1 -- 172.21.15.46:0/3328377938 <== mon.1 v2:172.21.15.110:3300/0 4 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f44b8003fe0 con 0x5583c76e9720 2024-05-09T04:52:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.669+0000 7f44d6a0bb80 10 client.25333 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.669+0000 7f44d6a0bb80 10 client.25333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.669+0000 7f44d6a0bb80 20 client.25333 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.669+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/3328377938 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583c776e430 0x5583c77708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.669+0000 7f44d6a0bb80 4 client.25333 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:52:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.669+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/3328377938 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f44b0023030 con 0x5583c776e430 2024-05-09T04:52:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.670+0000 7f44c5d91700 1 --2- 172.21.15.46:0/3328377938 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583c776e430 0x5583c77708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:30.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.671+0000 7f44a97fa700 10 client.25333 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:30.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.671+0000 7f44c5d91700 1 --2- 172.21.15.46:0/3328377938 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583c776e430 0x5583c77708b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=255820c2e43da953 server_cookie=a0fe8cf8cbba9ba9 in_seq=0 out_seq=0 2024-05-09T04:52:30.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.672+0000 7f44a97fa700 1 -- 172.21.15.46:0/3328377938 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f44b0023030 con 0x5583c776e430 2024-05-09T04:52:30.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.672+0000 7f44a97fa700 10 client.25333 handle_command_reply: tid=0 2024-05-09T04:52:30.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.672+0000 7f44d6a0bb80 1 client.25333 shutdown 2024-05-09T04:52:30.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.672+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/3328377938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c76e9720 msgr2=0x5583c76fd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:30.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.672+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/3328377938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c76e9720 0x5583c76fd9a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f44b8009340 tx=0x7f44b8004ad0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.673+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/3328377938 shutdown_connections 2024-05-09T04:52:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.673+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/3328377938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583c76e9720 0x5583c76fd9a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.673+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/3328377938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583c756c520 0x5583c76ff0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.673+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/3328377938 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583c776e430 0x5583c77708b0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.673+0000 7f44d6a0bb80 1 --2- 172.21.15.46:0/3328377938 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583c76561e0 0x5583c76fd460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.673+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/3328377938 >> 172.21.15.46:0/3328377938 conn(0x5583c75cc720 msgr2=0x5583c76edfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.673+0000 7f44a97fa700 0 client.25333 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.673+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/3328377938 shutdown_connections 2024-05-09T04:52:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.673+0000 7f44d6a0bb80 1 -- 172.21.15.46:0/3328377938 wait complete. 2024-05-09T04:52:30.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.673+0000 7f44d6a0bb80 20 client.25333 trim_cache size 0 max 16384 2024-05-09T04:52:30.675 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:52:30.675 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:52:30.675 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "c8d85d01-e06e-4c74-a1a6-1baf8832be70", 2024-05-09T04:52:30.675 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:52:30.675 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:52:30.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.675+0000 7f44c7ff5700 1 -- 172.21.15.46:0/3719139534 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f44ac0b2b10 msgr2=0x7f44ac0b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:30.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.675+0000 7f44c7ff5700 1 --2- 172.21.15.46:0/3719139534 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f44ac0b2b10 0x7f44ac0b5020 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f44c0102710 tx=0x7f44b0009260 comp rx=0 tx=0).stop 2024-05-09T04:52:30.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.675+0000 7f44c7ff5700 1 -- 172.21.15.46:0/3719139534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44c01034c0 msgr2=0x7f44c019f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:30.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.675+0000 7f44c7ff5700 1 --2- 172.21.15.46:0/3719139534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44c01034c0 0x7f44c019f7e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f44bc0098e0 tx=0x7f44bc009290 comp rx=0 tx=0).stop 2024-05-09T04:52:30.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.675+0000 7f44c7ff5700 1 -- 172.21.15.46:0/3719139534 shutdown_connections 2024-05-09T04:52:30.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.675+0000 7f44c7ff5700 1 --2- 172.21.15.46:0/3719139534 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f44ac0b2b10 0x7f44ac0b5020 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.675+0000 7f44c7ff5700 1 --2- 172.21.15.46:0/3719139534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f44c0108740 0x7f44c0199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.675+0000 7f44c7ff5700 1 --2- 172.21.15.46:0/3719139534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f44c01034c0 0x7f44c019f7e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.675+0000 7f44c7ff5700 1 --2- 172.21.15.46:0/3719139534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f44c0107df0 0x7f44c019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:30.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.675+0000 7f44c7ff5700 1 -- 172.21.15.46:0/3719139534 >> 172.21.15.46:0/3719139534 conn(0x7f44c00fedc0 msgr2=0x7f44c01073f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:30.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.676+0000 7f44c7ff5700 1 -- 172.21.15.46:0/3719139534 shutdown_connections 2024-05-09T04:52:30.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:30.676+0000 7f44c7ff5700 1 -- 172.21.15.46:0/3719139534 wait complete. 2024-05-09T04:52:31.017 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:30 smithi046 ceph-mon[30136]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-09T04:52:31.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:30 smithi110 ceph-mon[34904]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-09T04:52:31.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:30 smithi072 ceph-mon[34088]: pgmap v45: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-09T04:52:31.438 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:52:31.713 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:31.984 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:31 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:52:31.984 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:31 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:31.984 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:31 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:52:31.984 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:31 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:31.984 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:31 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:52:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:31 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:52:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:31 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:31 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:52:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:31 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:31 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:52:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:31 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:52:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:31 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:31 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:52:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:31 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:31 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:52:33.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:32 smithi110 ceph-mon[34904]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T04:52:33.065 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:32 smithi046 ceph-mon[30136]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T04:52:33.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:32 smithi072 ceph-mon[34088]: pgmap v46: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-09T04:52:33.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.582+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/4094223829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff81034e0 msgr2=0x7f0ff81038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:33.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.582+0000 7f0ffd9bc700 1 --2- 172.21.15.46:0/4094223829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff81034e0 0x7f0ff81038f0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80099c0 tx=0x7f0fe8009cd0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.583+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/4094223829 shutdown_connections 2024-05-09T04:52:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.583+0000 7f0ffd9bc700 1 --2- 172.21.15.46:0/4094223829 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ff8108760 0x7f0ff81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.583+0000 7f0ffd9bc700 1 --2- 172.21.15.46:0/4094223829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff81034e0 0x7f0ff81038f0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.583+0000 7f0ffd9bc700 1 --2- 172.21.15.46:0/4094223829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ff8107e10 0x7f0ff8108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.583+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/4094223829 >> 172.21.15.46:0/4094223829 conn(0x7f0ff80fef30 msgr2=0x7f0ff8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.584+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/4094223829 shutdown_connections 2024-05-09T04:52:33.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.584+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/4094223829 wait complete. 2024-05-09T04:52:33.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.585+0000 7f0ffd9bc700 1 Processor -- start 2024-05-09T04:52:33.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.585+0000 7f0ffd9bc700 1 -- start start 2024-05-09T04:52:33.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.586+0000 7f0ffd9bc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff81034e0 0x7f0ff8199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.586+0000 7f0ffd9bc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ff8107e10 0x7f0ff819a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.586+0000 7f0ffd9bc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ff8108760 0x7f0ff819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.586+0000 7f0ff6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff81034e0 0x7f0ff8199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.586+0000 7f0ff6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff81034e0 0x7f0ff8199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52624/0 (socket says 172.21.15.46:52624) 2024-05-09T04:52:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.586+0000 7f0ff6ffd700 1 -- 172.21.15.46:0/345509446 learned_addr learned my addr 172.21.15.46:0/345509446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.586+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/345509446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff8113390 con 0x7f0ff81034e0 2024-05-09T04:52:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.586+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/345509446 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff8113210 con 0x7f0ff8107e10 2024-05-09T04:52:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.586+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/345509446 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff8113510 con 0x7f0ff8108760 2024-05-09T04:52:33.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.586+0000 7f0ff77fe700 1 --2- 172.21.15.46:0/345509446 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ff8108760 0x7f0ff819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.587+0000 7f0ff67fc700 1 --2- 172.21.15.46:0/345509446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ff8107e10 0x7f0ff819a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.587+0000 7f0ff77fe700 1 -- 172.21.15.46:0/345509446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ff8107e10 msgr2=0x7f0ff819a300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.587+0000 7f0ff77fe700 1 --2- 172.21.15.46:0/345509446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ff8107e10 0x7f0ff819a300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.587+0000 7f0ff77fe700 1 -- 172.21.15.46:0/345509446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff81034e0 msgr2=0x7f0ff8199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.587+0000 7f0ff77fe700 1 --2- 172.21.15.46:0/345509446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff81034e0 0x7f0ff8199d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.587+0000 7f0ff77fe700 1 -- 172.21.15.46:0/345509446 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff819ec30 con 0x7f0ff8108760 2024-05-09T04:52:33.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.587+0000 7f0ff77fe700 1 --2- 172.21.15.46:0/345509446 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ff8108760 0x7f0ff819e590 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000efb0 tx=0x7f0ff000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.588+0000 7f0ffc9ba700 1 -- 172.21.15.46:0/345509446 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0003a00 con 0x7f0ff8108760 2024-05-09T04:52:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.588+0000 7f0ffc9ba700 1 -- 172.21.15.46:0/345509446 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ff000ce50 con 0x7f0ff8108760 2024-05-09T04:52:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.588+0000 7f0ffc9ba700 1 -- 172.21.15.46:0/345509446 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00047c0 con 0x7f0ff8108760 2024-05-09T04:52:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.589+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/345509446 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff819ef20 con 0x7f0ff8108760 2024-05-09T04:52:33.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.589+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/345509446 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff819f3a0 con 0x7f0ff8108760 2024-05-09T04:52:33.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.593+0000 7f0ffc9ba700 1 -- 172.21.15.46:0/345509446 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f0ff0003de0 con 0x7f0ff8108760 2024-05-09T04:52:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.594+0000 7f0ffc9ba700 1 --2- 172.21.15.46:0/345509446 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0fe00b2b90 0x7f0fe00b50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.594+0000 7f100c3d2b80 1 Processor -- start 2024-05-09T04:52:33.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.595+0000 7f0ffc9ba700 1 -- 172.21.15.46:0/345509446 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f0ff00e6450 con 0x7f0ff8108760 2024-05-09T04:52:33.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.595+0000 7f100c3d2b80 1 -- start start 2024-05-09T04:52:33.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.595+0000 7f0ff6ffd700 1 --2- 172.21.15.46:0/345509446 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0fe00b2b90 0x7f0fe00b50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:33.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.595+0000 7f100c3d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c070240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.595+0000 7f100c3d2b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c070780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.595+0000 7f100c3d2b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 0x55d83c06a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.595+0000 7f100c3d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d83bf8d0a0 con 0x55d83bf67d00 2024-05-09T04:52:33.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.595+0000 7f100c3d2b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d83bf329a0 con 0x55d83beed4a0 2024-05-09T04:52:33.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.595+0000 7f100c3d2b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d83bf89060 con 0x55d83c06a400 2024-05-09T04:52:33.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.595+0000 7f0ff67fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c070780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:33.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.595+0000 7f0ff67fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c070780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:38154/0 (socket says 172.21.15.46:38154) 2024-05-09T04:52:33.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.595+0000 7f0ff67fc700 1 -- 172.21.15.46:0/1185820462 learned_addr learned my addr 172.21.15.46:0/1185820462 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:33.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.596+0000 7f0ff6ffd700 1 --2- 172.21.15.46:0/1185820462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c070240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:33.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.596+0000 7f0ff67fc700 1 -- 172.21.15.46:0/1185820462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 msgr2=0x55d83c06a810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:52:33.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.596+0000 7f0ff67fc700 1 --2- 172.21.15.46:0/1185820462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 0x55d83c06a810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.596+0000 7f0ff67fc700 1 -- 172.21.15.46:0/1185820462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 msgr2=0x55d83c070240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:33.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.596+0000 7f0ff67fc700 1 --2- 172.21.15.46:0/1185820462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c070240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.596+0000 7f0ff67fc700 1 -- 172.21.15.46:0/1185820462 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d83bf23500 con 0x55d83beed4a0 2024-05-09T04:52:33.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.596+0000 7f0ff6ffd700 1 --2- 172.21.15.46:0/1185820462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:52:33.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.597+0000 7f0ff67fc700 1 --2- 172.21.15.46:0/1185820462 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c070780 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00bcf0 tx=0x7f0fec0103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b855d1e9b33c35ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:33.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.597+0000 7f0ff6ffd700 1 --2- 172.21.15.46:0/345509446 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0fe00b2b90 0x7f0fe00b50a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8003760 tx=0x7f0fe8005040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.597+0000 7f0fddffb700 1 -- 172.21.15.46:0/1185820462 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec010dc0 con 0x55d83beed4a0 2024-05-09T04:52:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.597+0000 7f0fddffb700 1 -- 172.21.15.46:0/1185820462 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fec0044e0 con 0x55d83beed4a0 2024-05-09T04:52:33.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.597+0000 7f0fddffb700 1 -- 172.21.15.46:0/1185820462 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0116a0 con 0x55d83beed4a0 2024-05-09T04:52:33.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.598+0000 7f100c3d2b80 1 -- 172.21.15.46:0/1185820462 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 msgr2=0x55d83c070780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.598+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/1185820462 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c070780 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00bcf0 tx=0x7f0fec0103d0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.599+0000 7f100c3d2b80 1 -- 172.21.15.46:0/1185820462 shutdown_connections 2024-05-09T04:52:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.599+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/1185820462 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 0x55d83c06a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.599+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/1185820462 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c070240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.599+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/1185820462 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c070780 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.599+0000 7f100c3d2b80 1 -- 172.21.15.46:0/1185820462 >> 172.21.15.46:0/1185820462 conn(0x55d83bef0f70 msgr2=0x55d83c065220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.599+0000 7f100c3d2b80 1 -- 172.21.15.46:0/1185820462 shutdown_connections 2024-05-09T04:52:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.599+0000 7f100c3d2b80 1 -- 172.21.15.46:0/1185820462 wait complete. 2024-05-09T04:52:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.600+0000 7f100c3d2b80 1 -- start start 2024-05-09T04:52:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.600+0000 7f100c3d2b80 1 Processor -- start 2024-05-09T04:52:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.601+0000 7f100c3d2b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c074090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.601+0000 7f100c3d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c0745d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.605 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.601+0000 7f0ff6ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c074090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.601+0000 7f100c3d2b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 0x55d83c074b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.601+0000 7f0ff6ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c074090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:38168/0 (socket says 172.21.15.46:38168) 2024-05-09T04:52:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.601+0000 7f0ff6ffd700 1 -- 172.21.15.46:0/3123569212 learned_addr learned my addr 172.21.15.46:0/3123569212 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.601+0000 7f0ff67fc700 1 --2- 172.21.15.46:0/3123569212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c0745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.601+0000 7f0ff77fe700 1 --2- 172.21.15.46:0/3123569212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 0x55d83c074b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.601+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3123569212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d83c06b160 con 0x55d83bf67d00 2024-05-09T04:52:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.601+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3123569212 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d83bf27b20 con 0x55d83beed4a0 2024-05-09T04:52:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.601+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3123569212 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d83c077020 con 0x55d83c06a400 2024-05-09T04:52:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.602+0000 7f100c3d2b80 10 client.0 _ll_register_callbacks cb 0x55d83bdfe6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.602+0000 7f0ff77fe700 1 -- 172.21.15.46:0/3123569212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 msgr2=0x55d83c074090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:33.606 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.602+0000 7f0ff77fe700 1 --2- 172.21.15.46:0/3123569212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c074090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.602+0000 7f0ff77fe700 1 -- 172.21.15.46:0/3123569212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 msgr2=0x55d83c0745d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.602+0000 7f0ff77fe700 1 --2- 172.21.15.46:0/3123569212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c0745d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.602+0000 7f0ff77fe700 1 -- 172.21.15.46:0/3123569212 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d83be08490 con 0x55d83c06a400 2024-05-09T04:52:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.602+0000 7f0ff67fc700 1 --2- 172.21.15.46:0/3123569212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c0745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.603+0000 7f0ff77fe700 1 --2- 172.21.15.46:0/3123569212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 0x55d83c074b10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00e3b10 tx=0x7f0ff00ad3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79bfe3efe31d7881 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.603+0000 7f0fd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:52:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.603+0000 7f0fd6ffd700 1 -- 172.21.15.46:0/3123569212 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00ad0e0 con 0x55d83c06a400 2024-05-09T04:52:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.603+0000 7f0fd6ffd700 1 -- 172.21.15.46:0/3123569212 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f0ff0019340 con 0x55d83c06a400 2024-05-09T04:52:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.603+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3123569212 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fec0044e0 con 0x55d83c06a400 2024-05-09T04:52:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.603+0000 7f0fd6ffd700 1 -- 172.21.15.46:0/3123569212 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0ff00adcd0 con 0x55d83c06a400 2024-05-09T04:52:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.603+0000 7f100c3d2b80 10 client.25342 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.603+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3123569212 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d83c0817e0 con 0x55d83c06a400 2024-05-09T04:52:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.604+0000 7f0fd6ffd700 1 -- 172.21.15.46:0/3123569212 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fec0044e0 con 0x55d83c06a400 2024-05-09T04:52:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.604+0000 7f0fd6ffd700 1 -- 172.21.15.46:0/3123569212 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f0ff00b5fa0 con 0x55d83c06a400 2024-05-09T04:52:33.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.604+0000 7f100c3d2b80 10 client.25342 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:33.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.604+0000 7f100c3d2b80 10 client.25342 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:33.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.604+0000 7f100c3d2b80 20 client.25342 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:33.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.605+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/3123569212 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d83c083d70 0x55d83c086190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.605+0000 7f100c3d2b80 4 client.25342 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:33.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.605+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3123569212 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d83bf9aa80 con 0x55d83c083d70 2024-05-09T04:52:33.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.605+0000 7f0ff6ffd700 1 --2- 172.21.15.46:0/3123569212 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d83c083d70 0x55d83c086190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:33.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.605+0000 7f0fd6ffd700 10 client.25342 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:33.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.605+0000 7f0ff6ffd700 1 --2- 172.21.15.46:0/3123569212 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d83c083d70 0x55d83c086190 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcea4f9382764a1b server_cookie=1c265ae0439784ec in_seq=0 out_seq=0 2024-05-09T04:52:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.606+0000 7f0fd6ffd700 1 -- 172.21.15.46:0/3123569212 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d83bf9aa80 con 0x55d83c083d70 2024-05-09T04:52:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.606+0000 7f0fd6ffd700 10 client.25342 handle_command_reply: tid=0 2024-05-09T04:52:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.606+0000 7f100c3d2b80 1 client.25342 shutdown 2024-05-09T04:52:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.607+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3123569212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 msgr2=0x55d83c074b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.607+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/3123569212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 0x55d83c074b10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0ff00e3b10 tx=0x7f0ff00ad3a0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.607+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3123569212 shutdown_connections 2024-05-09T04:52:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.607+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/3123569212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 0x55d83c074b10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.609 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.607+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/3123569212 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d83c083d70 0x55d83c086190 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.607+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/3123569212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c074090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.607+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/3123569212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c0745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.607+0000 7f0fd6ffd700 0 client.25342 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.607+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3123569212 >> 172.21.15.46:0/3123569212 conn(0x55d83bef0f70 msgr2=0x55d83c05eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.607+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3123569212 shutdown_connections 2024-05-09T04:52:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.607+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3123569212 wait complete. 2024-05-09T04:52:33.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.608+0000 7f100c3d2b80 20 client.25342 trim_cache size 0 max 16384 2024-05-09T04:52:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.649+0000 7f100c3d2b80 1 Processor -- start 2024-05-09T04:52:33.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.649+0000 7f100c3d2b80 1 -- start start 2024-05-09T04:52:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.649+0000 7f100c3d2b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.649+0000 7f100c3d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c08b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.649+0000 7f100c3d2b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 0x55d83c08bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.649+0000 7f100c3d2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff81af5e0 con 0x55d83bf67d00 2024-05-09T04:52:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.649+0000 7f100c3d2b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff81ace70 con 0x55d83beed4a0 2024-05-09T04:52:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.649+0000 7f100c3d2b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff81aaca0 con 0x55d83c06a400 2024-05-09T04:52:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.650+0000 7f0ff67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c08b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.650+0000 7f0ff67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c08b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52658/0 (socket says 172.21.15.46:52658) 2024-05-09T04:52:33.650 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.650+0000 7f0ff67fc700 1 -- 172.21.15.46:0/3264363063 learned_addr learned my addr 172.21.15.46:0/3264363063 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:33.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.650+0000 7f0ff67fc700 1 -- 172.21.15.46:0/3264363063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 msgr2=0x55d83c08bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:33.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.650+0000 7f0ff67fc700 1 --2- 172.21.15.46:0/3264363063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 0x55d83c08bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.650+0000 7f0ff67fc700 1 -- 172.21.15.46:0/3264363063 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 msgr2=0x55d83c069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:33.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.650+0000 7f0ff6ffd700 1 --2- 172.21.15.46:0/3264363063 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:33.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.650+0000 7f0ff67fc700 1 --2- 172.21.15.46:0/3264363063 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.650+0000 7f0ff67fc700 1 -- 172.21.15.46:0/3264363063 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d83c0619d0 con 0x55d83bf67d00 2024-05-09T04:52:33.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.651+0000 7f0ff67fc700 1 --2- 172.21.15.46:0/3264363063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c08b8b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55d83c066c90 tx=0x7f0fec0038c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46bf5a2c5e75e3e2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:33.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.651+0000 7f0ff6ffd700 1 --2- 172.21.15.46:0/3264363063 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:52:33.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.651+0000 7f0fdd7fa700 1 -- 172.21.15.46:0/3264363063 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec00b410 con 0x55d83bf67d00 2024-05-09T04:52:33.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.651+0000 7f0fdd7fa700 1 -- 172.21.15.46:0/3264363063 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fec0054f0 con 0x55d83bf67d00 2024-05-09T04:52:33.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.651+0000 7f0fdd7fa700 1 -- 172.21.15.46:0/3264363063 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec011d40 con 0x55d83bf67d00 2024-05-09T04:52:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.653+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3264363063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 msgr2=0x55d83c08b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.653+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/3264363063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c08b8b0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55d83c066c90 tx=0x7f0fec0038c0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.653+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3264363063 shutdown_connections 2024-05-09T04:52:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.653+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/3264363063 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 0x55d83c08bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.653+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/3264363063 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.653+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/3264363063 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c08b8b0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.653+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3264363063 >> 172.21.15.46:0/3264363063 conn(0x55d83bef0f70 msgr2=0x55d83c05f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.653+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3264363063 shutdown_connections 2024-05-09T04:52:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.653+0000 7f100c3d2b80 1 -- 172.21.15.46:0/3264363063 wait complete. 2024-05-09T04:52:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.653+0000 7f100c3d2b80 1 -- start start 2024-05-09T04:52:33.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.654+0000 7f100c3d2b80 1 Processor -- start 2024-05-09T04:52:33.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.654+0000 7f100c3d2b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c07c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.654+0000 7f100c3d2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c07a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.655+0000 7f0ff67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c07a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:33.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.655+0000 7f0ff67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c07a920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52668/0 (socket says 172.21.15.46:52668) 2024-05-09T04:52:33.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.655+0000 7f0ff67fc700 1 -- 172.21.15.46:0/2478769686 learned_addr learned my addr 172.21.15.46:0/2478769686 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:33.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.655+0000 7f0ff6ffd700 1 --2- 172.21.15.46:0/2478769686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c07c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:33.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.655+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/2478769686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 0x55d83c07ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.655+0000 7f100c3d2b80 1 -- 172.21.15.46:0/2478769686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff81ab330 con 0x55d83bf67d00 2024-05-09T04:52:33.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.655+0000 7f100c3d2b80 1 -- 172.21.15.46:0/2478769686 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff81ac7a0 con 0x55d83beed4a0 2024-05-09T04:52:33.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.655+0000 7f100c3d2b80 1 -- 172.21.15.46:0/2478769686 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff81aba00 con 0x55d83c06a400 2024-05-09T04:52:33.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.655+0000 7f100c3d2b80 10 client.0 _ll_register_callbacks cb 0x55d83bdfe6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:33.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.655+0000 7f0ff67fc700 1 -- 172.21.15.46:0/2478769686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 msgr2=0x55d83c07ae60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:52:33.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.655+0000 7f0ff67fc700 1 --2- 172.21.15.46:0/2478769686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 0x55d83c07ae60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.655+0000 7f0ff67fc700 1 -- 172.21.15.46:0/2478769686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 msgr2=0x55d83c07c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:33.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.655+0000 7f0ff67fc700 1 --2- 172.21.15.46:0/2478769686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c07c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.655+0000 7f0ff67fc700 1 -- 172.21.15.46:0/2478769686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d83c07b3f0 con 0x55d83bf67d00 2024-05-09T04:52:33.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.656+0000 7f0ff67fc700 1 --2- 172.21.15.46:0/2478769686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c07a920 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00bc10 tx=0x7f0fec00b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdb458babfa34ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:33.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.656+0000 7f0fd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:33.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.656+0000 7f0fd6ffd700 1 -- 172.21.15.46:0/2478769686 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fec035060 con 0x55d83bf67d00 2024-05-09T04:52:33.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.656+0000 7f0fd6ffd700 1 -- 172.21.15.46:0/2478769686 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f0fec011940 con 0x55d83bf67d00 2024-05-09T04:52:33.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.656+0000 7f100c3d2b80 1 -- 172.21.15.46:0/2478769686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fec0054f0 con 0x55d83bf67d00 2024-05-09T04:52:33.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.656+0000 7f0fd6ffd700 1 -- 172.21.15.46:0/2478769686 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0fec0056a0 con 0x55d83bf67d00 2024-05-09T04:52:33.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.657+0000 7f100c3d2b80 10 client.15639 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:33.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.657+0000 7f100c3d2b80 1 -- 172.21.15.46:0/2478769686 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d83c0ef120 con 0x55d83bf67d00 2024-05-09T04:52:33.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.657+0000 7f0fd6ffd700 1 -- 172.21.15.46:0/2478769686 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fec030460 con 0x55d83bf67d00 2024-05-09T04:52:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.657+0000 7f0fd6ffd700 1 -- 172.21.15.46:0/2478769686 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f0fec01b420 con 0x55d83bf67d00 2024-05-09T04:52:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.657+0000 7f100c3d2b80 10 client.15639 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.657+0000 7f100c3d2b80 10 client.15639 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.657+0000 7f100c3d2b80 20 client.15639 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.658+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/2478769686 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d83c0ef500 0x55d83c0f1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.658+0000 7f100c3d2b80 4 client.15639 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:52:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.658+0000 7f100c3d2b80 1 -- 172.21.15.46:0/2478769686 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f0fec0054f0 con 0x55d83c0ef500 2024-05-09T04:52:33.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.658+0000 7f0ff6ffd700 1 --2- 172.21.15.46:0/2478769686 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d83c0ef500 0x55d83c0f1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.659+0000 7f0fd6ffd700 10 client.15639 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.659+0000 7f0ff6ffd700 1 --2- 172.21.15.46:0/2478769686 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d83c0ef500 0x55d83c0f1920 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37bf90a5a4657e78 server_cookie=ee814c8c9b8fcce5 in_seq=0 out_seq=0 2024-05-09T04:52:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.659+0000 7f0fd6ffd700 1 -- 172.21.15.46:0/2478769686 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x7f0fec0054f0 con 0x55d83c0ef500 2024-05-09T04:52:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.659+0000 7f0fd6ffd700 10 client.15639 handle_command_reply: tid=0 2024-05-09T04:52:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.659+0000 7f100c3d2b80 1 client.15639 shutdown 2024-05-09T04:52:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.660+0000 7f100c3d2b80 1 -- 172.21.15.46:0/2478769686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 msgr2=0x55d83c07a920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.660+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/2478769686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c07a920 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0fec00bc10 tx=0x7f0fec00b540 comp rx=0 tx=0).stop 2024-05-09T04:52:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.660+0000 7f100c3d2b80 1 -- 172.21.15.46:0/2478769686 shutdown_connections 2024-05-09T04:52:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.660+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/2478769686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d83c06a400 0x55d83c07ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.660+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/2478769686 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d83c0ef500 0x55d83c0f1920 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.660+0000 7f0fd6ffd700 0 client.15639 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.660+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/2478769686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d83beed4a0 0x55d83c07c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.660+0000 7f100c3d2b80 1 --2- 172.21.15.46:0/2478769686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d83bf67d00 0x55d83c07a920 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.665 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:52:33.665 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:52:33.665 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:52:33.665 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:52:33.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.660+0000 7f100c3d2b80 1 -- 172.21.15.46:0/2478769686 >> 172.21.15.46:0/2478769686 conn(0x55d83bef0f70 msgr2=0x55d83c06dc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:33.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.660+0000 7f100c3d2b80 1 -- 172.21.15.46:0/2478769686 shutdown_connections 2024-05-09T04:52:33.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.660+0000 7f100c3d2b80 1 -- 172.21.15.46:0/2478769686 wait complete. 2024-05-09T04:52:33.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.660+0000 7f100c3d2b80 20 client.15639 trim_cache size 0 max 16384 2024-05-09T04:52:33.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.661+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/345509446 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0fe00b2b90 msgr2=0x7f0fe00b50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:33.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.661+0000 7f0ffd9bc700 1 --2- 172.21.15.46:0/345509446 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0fe00b2b90 0x7f0fe00b50a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8003760 tx=0x7f0fe8005040 comp rx=0 tx=0).stop 2024-05-09T04:52:33.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.661+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/345509446 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ff8108760 msgr2=0x7f0ff819e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:33.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.661+0000 7f0ffd9bc700 1 --2- 172.21.15.46:0/345509446 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ff8108760 0x7f0ff819e590 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000efb0 tx=0x7f0ff000c3a0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.661+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/345509446 shutdown_connections 2024-05-09T04:52:33.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.661+0000 7f0ffd9bc700 1 --2- 172.21.15.46:0/345509446 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0fe00b2b90 0x7f0fe00b50a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.661+0000 7f0ffd9bc700 1 --2- 172.21.15.46:0/345509446 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0ff8108760 0x7f0ff819e590 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.661+0000 7f0ffd9bc700 1 --2- 172.21.15.46:0/345509446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0ff81034e0 0x7f0ff8199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.661+0000 7f0ffd9bc700 1 --2- 172.21.15.46:0/345509446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0ff8107e10 0x7f0ff819a300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.661+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/345509446 >> 172.21.15.46:0/345509446 conn(0x7f0ff80fef30 msgr2=0x7f0ff81096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.662+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/345509446 shutdown_connections 2024-05-09T04:52:33.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:33.662+0000 7f0ffd9bc700 1 -- 172.21.15.46:0/345509446 wait complete. 2024-05-09T04:52:34.316 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:52:34.316 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:52:34.669 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:34 smithi110 ceph-mon[34904]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T04:52:35.075 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:34 smithi046 ceph-mon[30136]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T04:52:35.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:34 smithi072 ceph-mon[34088]: pgmap v47: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T04:52:36.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.718+0000 7f279375e700 1 -- 172.21.15.46:0/1473473553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c108760 msgr2=0x7f278c1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.718+0000 7f279375e700 1 --2- 172.21.15.46:0/1473473553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c108760 0x7f278c1100c0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f278400b0a0 tx=0x7f278400b3b0 comp rx=0 tx=0).stop 2024-05-09T04:52:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.718+0000 7f279375e700 1 -- 172.21.15.46:0/1473473553 shutdown_connections 2024-05-09T04:52:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.718+0000 7f279375e700 1 --2- 172.21.15.46:0/1473473553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c108760 0x7f278c1100c0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.718+0000 7f279375e700 1 --2- 172.21.15.46:0/1473473553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f278c1034e0 0x7f278c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.718+0000 7f279375e700 1 --2- 172.21.15.46:0/1473473553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f278c107e10 0x7f278c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.718+0000 7f279375e700 1 -- 172.21.15.46:0/1473473553 >> 172.21.15.46:0/1473473553 conn(0x7f278c0fef30 msgr2=0x7f278c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.719+0000 7f279375e700 1 -- 172.21.15.46:0/1473473553 shutdown_connections 2024-05-09T04:52:36.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.719+0000 7f279375e700 1 -- 172.21.15.46:0/1473473553 wait complete. 2024-05-09T04:52:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.720+0000 7f279375e700 1 Processor -- start 2024-05-09T04:52:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.720+0000 7f279375e700 1 -- start start 2024-05-09T04:52:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.720+0000 7f279375e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f278c1034e0 0x7f278c1106d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.720+0000 7f279375e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c107e10 0x7f278c110c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.720+0000 7f279375e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f278c108760 0x7f278c111150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.720+0000 7f2790cf9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c107e10 0x7f278c110c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:36.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.720+0000 7f27914fa700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f278c1034e0 0x7f278c1106d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.720+0000 7f279375e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f278c114530 con 0x7f278c107e10 2024-05-09T04:52:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.720+0000 7f279375e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1143b0 con 0x7f278c1034e0 2024-05-09T04:52:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.720+0000 7f279375e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f278c1146b0 con 0x7f278c108760 2024-05-09T04:52:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.720+0000 7f27914fa700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f278c1034e0 0x7f278c1106d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:52822/0 (socket says 172.21.15.46:52822) 2024-05-09T04:52:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.721+0000 7f2790cf9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c107e10 0x7f278c110c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47218/0 (socket says 172.21.15.46:47218) 2024-05-09T04:52:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.721+0000 7f27914fa700 1 -- 172.21.15.46:0/2804032581 learned_addr learned my addr 172.21.15.46:0/2804032581 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:36.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.721+0000 7f2791cfb700 1 --2- 172.21.15.46:0/2804032581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f278c108760 0x7f278c111150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.721+0000 7f27914fa700 1 -- 172.21.15.46:0/2804032581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f278c108760 msgr2=0x7f278c111150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.721+0000 7f27914fa700 1 --2- 172.21.15.46:0/2804032581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f278c108760 0x7f278c111150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.721+0000 7f27914fa700 1 -- 172.21.15.46:0/2804032581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c107e10 msgr2=0x7f278c110c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.721+0000 7f27914fa700 1 --2- 172.21.15.46:0/2804032581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c107e10 0x7f278c110c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.721+0000 7f27914fa700 1 -- 172.21.15.46:0/2804032581 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f278c1a91a0 con 0x7f278c1034e0 2024-05-09T04:52:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.721+0000 7f2790cf9700 1 --2- 172.21.15.46:0/2804032581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c107e10 0x7f278c110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.722+0000 7f27914fa700 1 --2- 172.21.15.46:0/2804032581 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f278c1034e0 0x7f278c1106d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f278800e9a0 tx=0x7f278800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:36.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.722+0000 7f27827fc700 1 -- 172.21.15.46:0/2804032581 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2788024070 con 0x7f278c1034e0 2024-05-09T04:52:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.722+0000 7f279375e700 1 -- 172.21.15.46:0/2804032581 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f278c19abc0 con 0x7f278c1034e0 2024-05-09T04:52:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.722+0000 7f27827fc700 1 -- 172.21.15.46:0/2804032581 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f278800cb90 con 0x7f278c1034e0 2024-05-09T04:52:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.722+0000 7f27827fc700 1 -- 172.21.15.46:0/2804032581 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278801fbb0 con 0x7f278c1034e0 2024-05-09T04:52:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.722+0000 7f279375e700 1 -- 172.21.15.46:0/2804032581 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f278c19b150 con 0x7f278c1034e0 2024-05-09T04:52:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.724+0000 7f2777fff700 1 -- 172.21.15.46:0/2804032581 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f278c0512c0 con 0x7f278c1034e0 2024-05-09T04:52:36.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.726+0000 7f27827fc700 1 -- 172.21.15.46:0/2804032581 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f2788027070 con 0x7f278c1034e0 2024-05-09T04:52:36.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.727+0000 7f27827fc700 1 --2- 172.21.15.46:0/2804032581 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f27780b2b10 0x7f27780b5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:36.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.728+0000 7f2790cf9700 1 --2- 172.21.15.46:0/2804032581 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f27780b2b10 0x7f27780b5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:36.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.728+0000 7f27827fc700 1 -- 172.21.15.46:0/2804032581 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f27880eae70 con 0x7f278c1034e0 2024-05-09T04:52:36.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.728+0000 7f2790cf9700 1 --2- 172.21.15.46:0/2804032581 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f27780b2b10 0x7f27780b5020 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f278c1066e0 tx=0x7f277c009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:36.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:36.729+0000 7f27827fc700 1 -- 172.21.15.46:0/2804032581 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f27880b26f0 con 0x7f278c1034e0 2024-05-09T04:52:36.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:36 smithi046 ceph-mon[30136]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T04:52:37.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:36 smithi110 ceph-mon[34904]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T04:52:37.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:36 smithi072 ceph-mon[34088]: pgmap v48: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 6 op/s 2024-05-09T04:52:37.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:37.209+0000 7f2777fff700 1 -- 172.21.15.46:0/2804032581 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f278c0643d0 con 0x7f278c1034e0 2024-05-09T04:52:37.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:37.210+0000 7f27827fc700 1 -- 172.21.15.46:0/2804032581 <== mon.2 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7f27880b1ff0 con 0x7f278c1034e0 2024-05-09T04:52:37.211 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:52:37.211 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:52:37.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:37.214+0000 7f2777fff700 1 -- 172.21.15.46:0/2804032581 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f27780b2b10 msgr2=0x7f27780b5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:37.214+0000 7f2777fff700 1 --2- 172.21.15.46:0/2804032581 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f27780b2b10 0x7f27780b5020 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f278c1066e0 tx=0x7f277c009260 comp rx=0 tx=0).stop 2024-05-09T04:52:37.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:37.214+0000 7f2777fff700 1 -- 172.21.15.46:0/2804032581 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f278c1034e0 msgr2=0x7f278c1106d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:37.214+0000 7f2777fff700 1 --2- 172.21.15.46:0/2804032581 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f278c1034e0 0x7f278c1106d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f278800e9a0 tx=0x7f278800ecb0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:37.214+0000 7f2777fff700 1 -- 172.21.15.46:0/2804032581 shutdown_connections 2024-05-09T04:52:37.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:37.214+0000 7f2777fff700 1 --2- 172.21.15.46:0/2804032581 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f27780b2b10 0x7f27780b5020 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:37.214+0000 7f2777fff700 1 --2- 172.21.15.46:0/2804032581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f278c108760 0x7f278c111150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:37.214+0000 7f2777fff700 1 --2- 172.21.15.46:0/2804032581 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f278c1034e0 0x7f278c1106d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:37.214+0000 7f2777fff700 1 --2- 172.21.15.46:0/2804032581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f278c107e10 0x7f278c110c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:37.214+0000 7f2777fff700 1 -- 172.21.15.46:0/2804032581 >> 172.21.15.46:0/2804032581 conn(0x7f278c0fef30 msgr2=0x7f278c107550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:37.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:37.215+0000 7f2777fff700 1 -- 172.21.15.46:0/2804032581 shutdown_connections 2024-05-09T04:52:37.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:37.215+0000 7f2777fff700 1 -- 172.21.15.46:0/2804032581 wait complete. 2024-05-09T04:52:37.216 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:52:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.276+0000 7f8a77b00640 1 -- 172.21.15.46:0/164415011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a70106b10 msgr2=0x7f8a70106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.276+0000 7f8a77b00640 1 --2- 172.21.15.46:0/164415011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a70106b10 0x7f8a70106f40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f8a6000aa10 tx=0x7f8a6002b720 comp rx=0 tx=0).stop 2024-05-09T04:52:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.277+0000 7f8a77b00640 1 -- 172.21.15.46:0/164415011 shutdown_connections 2024-05-09T04:52:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.277+0000 7f8a77b00640 1 --2- 172.21.15.46:0/164415011 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a701075d0 0x7f8a7010ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.277+0000 7f8a77b00640 1 --2- 172.21.15.46:0/164415011 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a70106b10 0x7f8a70106f40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.277+0000 7f8a77b00640 1 --2- 172.21.15.46:0/164415011 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a70102580 0x7f8a70102990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.277+0000 7f8a77b00640 1 -- 172.21.15.46:0/164415011 >> 172.21.15.46:0/164415011 conn(0x7f8a700779e0 msgr2=0x7f8a70077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.277+0000 7f8a77b00640 1 -- 172.21.15.46:0/164415011 shutdown_connections 2024-05-09T04:52:37.278 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.277+0000 7f8a77b00640 1 -- 172.21.15.46:0/164415011 wait complete. 2024-05-09T04:52:37.279 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.278+0000 7f8a77b00640 1 Processor -- start 2024-05-09T04:52:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.278+0000 7f8a77b00640 1 -- start start 2024-05-09T04:52:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.278+0000 7f8a77b00640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a70102580 0x7f8a701986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.278+0000 7f8a77b00640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a70106b10 0x7f8a70198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.278+0000 7f8a77b00640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a701075d0 0x7f8a7019ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.278+0000 7f8a77b00640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8a70111cb0 con 0x7f8a70106b10 2024-05-09T04:52:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.278+0000 7f8a77b00640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8a70111b30 con 0x7f8a70102580 2024-05-09T04:52:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.278+0000 7f8a77b00640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8a70111e30 con 0x7f8a701075d0 2024-05-09T04:52:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.278+0000 7f8a7509b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a70106b10 0x7f8a70198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.278+0000 7f8a7509b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a70106b10 0x7f8a70198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47228/0 (socket says 172.21.15.46:47228) 2024-05-09T04:52:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.278+0000 7f8a7509b640 1 -- 172.21.15.46:0/3971007131 learned_addr learned my addr 172.21.15.46:0/3971007131 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.278+0000 7f8a7589c640 1 --2- 172.21.15.46:0/3971007131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a70102580 0x7f8a701986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.279+0000 7f8a7609d640 1 --2- 172.21.15.46:0/3971007131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a701075d0 0x7f8a7019ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.279+0000 7f8a7509b640 1 -- 172.21.15.46:0/3971007131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a701075d0 msgr2=0x7f8a7019ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.279+0000 7f8a7509b640 1 --2- 172.21.15.46:0/3971007131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a701075d0 0x7f8a7019ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.280 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.279+0000 7f8a7509b640 1 -- 172.21.15.46:0/3971007131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a70102580 msgr2=0x7f8a701986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.279+0000 7f8a7509b640 1 --2- 172.21.15.46:0/3971007131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a70102580 0x7f8a701986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.279+0000 7f8a7509b640 1 -- 172.21.15.46:0/3971007131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a7019d510 con 0x7f8a70106b10 2024-05-09T04:52:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.279+0000 7f8a7589c640 1 --2- 172.21.15.46:0/3971007131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a70102580 0x7f8a701986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.279+0000 7f8a7509b640 1 --2- 172.21.15.46:0/3971007131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a70106b10 0x7f8a70198c10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8a60005ab0 tx=0x7f8a60003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.279+0000 7f8a66ffd640 1 -- 172.21.15.46:0/3971007131 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a6003f070 con 0x7f8a70106b10 2024-05-09T04:52:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.280+0000 7f8a66ffd640 1 -- 172.21.15.46:0/3971007131 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a60030c60 con 0x7f8a70106b10 2024-05-09T04:52:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.280+0000 7f8a77b00640 1 -- 172.21.15.46:0/3971007131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a7019d7a0 con 0x7f8a70106b10 2024-05-09T04:52:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.280+0000 7f8a77b00640 1 -- 172.21.15.46:0/3971007131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a7019dce0 con 0x7f8a70106b10 2024-05-09T04:52:37.281 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.280+0000 7f8a66ffd640 1 -- 172.21.15.46:0/3971007131 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60036040 con 0x7f8a70106b10 2024-05-09T04:52:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.284+0000 7f8a7e9ba740 1 Processor -- start 2024-05-09T04:52:37.283 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.284+0000 7f8a7e9ba740 1 -- start start 2024-05-09T04:52:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.284+0000 7f8a7e9ba740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f6e1b0 0x563313319e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.284+0000 7f8a7e9ba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f452f0 0x56331331a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.284+0000 7f8a7e9ba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f4d4d0 0x56331331a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.284+0000 7f8a7e9ba740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5633132767c0 con 0x563312f452f0 2024-05-09T04:52:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.284+0000 7f8a7e9ba740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5633132120e0 con 0x563312f6e1b0 2024-05-09T04:52:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.284+0000 7f8a7e9ba740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563313272780 con 0x563312f4d4d0 2024-05-09T04:52:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.284+0000 7f8a66ffd640 1 -- 172.21.15.46:0/3971007131 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 25) ==== 149885+0+0 (secure 0 0 0) 0x7f8a60042070 con 0x7f8a70106b10 2024-05-09T04:52:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.284+0000 7f8a7589c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f452f0 0x56331331a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.285+0000 7f8a7589c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f452f0 0x56331331a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47232/0 (socket says 172.21.15.46:47232) 2024-05-09T04:52:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.285+0000 7f8a7589c640 1 -- 172.21.15.46:0/4146473258 learned_addr learned my addr 172.21.15.46:0/4146473258 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.285+0000 7f8a7609d640 1 --2- 172.21.15.46:0/4146473258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f6e1b0 0x563313319e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.285+0000 7f8a7509b640 1 --2- 172.21.15.46:0/4146473258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f4d4d0 0x56331331a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.285+0000 7f8a7609d640 1 -- 172.21.15.46:0/4146473258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f4d4d0 msgr2=0x56331331a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.285+0000 7f8a7609d640 1 --2- 172.21.15.46:0/4146473258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f4d4d0 0x56331331a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.285+0000 7f8a7609d640 1 -- 172.21.15.46:0/4146473258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f452f0 msgr2=0x56331331a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.285+0000 7f8a7609d640 1 --2- 172.21.15.46:0/4146473258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f452f0 0x56331331a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.285+0000 7f8a7609d640 1 -- 172.21.15.46:0/4146473258 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56331323b7c0 con 0x563312f6e1b0 2024-05-09T04:52:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.285+0000 7f8a7589c640 1 --2- 172.21.15.46:0/4146473258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f452f0 0x56331331a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:37.286 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.285+0000 7f8a7609d640 1 --2- 172.21.15.46:0/4146473258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f6e1b0 0x563313319e90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009f00 tx=0x7f8a68011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55258bca536465f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.285+0000 7f8a5bfff640 1 -- 172.21.15.46:0/4146473258 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68004d20 con 0x563312f6e1b0 2024-05-09T04:52:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.286+0000 7f8a5bfff640 1 -- 172.21.15.46:0/4146473258 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a68004050 con 0x563312f6e1b0 2024-05-09T04:52:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.286+0000 7f8a66ffd640 1 --2- 172.21.15.46:0/3971007131 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8a5c0b2b70 0x7f8a5c0b5080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.286+0000 7f8a5bfff640 1 -- 172.21.15.46:0/4146473258 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a68005280 con 0x563312f6e1b0 2024-05-09T04:52:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.286+0000 7f8a7589c640 1 --2- 172.21.15.46:0/3971007131 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8a5c0b2b70 0x7f8a5c0b5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.287+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/4146473258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f6e1b0 msgr2=0x563313319e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.287+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/4146473258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f6e1b0 0x563313319e90 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8a68009f00 tx=0x7f8a68011810 comp rx=0 tx=0).stop 2024-05-09T04:52:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.287+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/4146473258 shutdown_connections 2024-05-09T04:52:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.287+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/4146473258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f4d4d0 0x56331331a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.287+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/4146473258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f452f0 0x56331331a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.287+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/4146473258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f6e1b0 0x563313319e90 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.287+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/4146473258 >> 172.21.15.46:0/4146473258 conn(0x563312f62300 msgr2=0x563313316640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.287+0000 7f8a7589c640 1 --2- 172.21.15.46:0/3971007131 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8a5c0b2b70 0x7f8a5c0b5080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x56331331b240 tx=0x7f8a6c012040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.287+0000 7f8a66ffd640 1 -- 172.21.15.46:0/3971007131 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f8a601027d0 con 0x7f8a70106b10 2024-05-09T04:52:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.287+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/4146473258 shutdown_connections 2024-05-09T04:52:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.287+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/4146473258 wait complete. 2024-05-09T04:52:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.288+0000 7f8a7e9ba740 1 -- start start 2024-05-09T04:52:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.288+0000 7f8a7e9ba740 1 Processor -- start 2024-05-09T04:52:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7e9ba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f452f0 0x5633133244c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7e9ba740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f4d4d0 0x563313324a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7e9ba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f6e1b0 0x56331332a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7609d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f6e1b0 0x56331332a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7609d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f6e1b0 0x56331332a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47238/0 (socket says 172.21.15.46:47238) 2024-05-09T04:52:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7609d640 1 -- 172.21.15.46:0/2456336892 learned_addr learned my addr 172.21.15.46:0/2456336892 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/2456336892 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563313327060 con 0x563312f6e1b0 2024-05-09T04:52:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7509b640 1 --2- 172.21.15.46:0/2456336892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f4d4d0 0x563313324a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/2456336892 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5633131fd9c0 con 0x563312f4d4d0 2024-05-09T04:52:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/2456336892 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5633133271e0 con 0x563312f452f0 2024-05-09T04:52:37.291 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7589c640 1 --2- 172.21.15.46:0/2456336892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f452f0 0x5633133244c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7e9ba740 10 client.0 _ll_register_callbacks cb 0x56331322a020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7509b640 1 -- 172.21.15.46:0/2456336892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f452f0 msgr2=0x5633133244c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7509b640 1 --2- 172.21.15.46:0/2456336892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f452f0 0x5633133244c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7509b640 1 -- 172.21.15.46:0/2456336892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f6e1b0 msgr2=0x56331332a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7509b640 1 --2- 172.21.15.46:0/2456336892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f6e1b0 0x56331332a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.289+0000 7f8a7509b640 1 -- 172.21.15.46:0/2456336892 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633132241f0 con 0x563312f4d4d0 2024-05-09T04:52:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.290+0000 7f8a7509b640 1 --2- 172.21.15.46:0/2456336892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f4d4d0 0x563313324a00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8a600ffd20 tx=0x7f8a6000a730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28179d53ac55e10f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.290+0000 7f8a597fa640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:52:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.290+0000 7f8a597fa640 1 -- 172.21.15.46:0/2456336892 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60049070 con 0x563312f4d4d0 2024-05-09T04:52:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.290+0000 7f8a597fa640 1 -- 172.21.15.46:0/2456336892 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f8a60032400 con 0x563312f4d4d0 2024-05-09T04:52:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.290+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/2456336892 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a68004050 con 0x563312f4d4d0 2024-05-09T04:52:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.290+0000 7f8a597fa640 1 -- 172.21.15.46:0/2456336892 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8a600c92b0 con 0x563312f4d4d0 2024-05-09T04:52:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.290+0000 7f8a7e9ba740 10 client.25406 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:37.292 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.290+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/2456336892 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563312fd9790 con 0x563312f4d4d0 2024-05-09T04:52:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.290+0000 7f8a597fa640 1 -- 172.21.15.46:0/2456336892 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a68004050 con 0x563312f4d4d0 2024-05-09T04:52:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.290+0000 7f8a597fa640 1 -- 172.21.15.46:0/2456336892 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f8a6004c060 con 0x563312f4d4d0 2024-05-09T04:52:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.290+0000 7f8a7e9ba740 10 client.25406 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.291+0000 7f8a7e9ba740 10 client.25406 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.291+0000 7f8a7e9ba740 20 client.25406 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.291+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/2456336892 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563312f2cd80 0x563313336930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.291+0000 7f8a7e9ba740 4 client.25406 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.291+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/2456336892 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5633132843d0 con 0x563312f2cd80 2024-05-09T04:52:37.293 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.291+0000 7f8a7589c640 1 --2- 172.21.15.46:0/2456336892 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563312f2cd80 0x563313336930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.291+0000 7f8a597fa640 10 client.25406 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:37.294 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.291+0000 7f8a7589c640 1 --2- 172.21.15.46:0/2456336892 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563312f2cd80 0x563313336930 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f08c816beedb7619 server_cookie=7ba1381abc51cfdd in_seq=0 out_seq=0 2024-05-09T04:52:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.292+0000 7f8a597fa640 1 -- 172.21.15.46:0/2456336892 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5633132843d0 con 0x563312f2cd80 2024-05-09T04:52:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.292+0000 7f8a597fa640 10 client.25406 handle_command_reply: tid=0 2024-05-09T04:52:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.292+0000 7f8a7e9ba740 1 client.25406 shutdown 2024-05-09T04:52:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.292+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/2456336892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f4d4d0 msgr2=0x563313324a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.292+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/2456336892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f4d4d0 0x563313324a00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f8a600ffd20 tx=0x7f8a6000a730 comp rx=0 tx=0).stop 2024-05-09T04:52:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.293+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/2456336892 shutdown_connections 2024-05-09T04:52:37.295 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.293+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/2456336892 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563312f2cd80 0x563313336930 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.293+0000 7f8a597fa640 0 client.25406 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.293+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/2456336892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f6e1b0 0x56331332a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.293+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/2456336892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f4d4d0 0x563313324a00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.293+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/2456336892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f452f0 0x5633133244c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.293+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/2456336892 >> 172.21.15.46:0/2456336892 conn(0x563312f62300 msgr2=0x563313319cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.293+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/2456336892 shutdown_connections 2024-05-09T04:52:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.293+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/2456336892 wait complete. 2024-05-09T04:52:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.293+0000 7f8a7e9ba740 20 client.25406 trim_cache size 0 max 16384 2024-05-09T04:52:37.323 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.323+0000 7f8a7e9ba740 1 Processor -- start 2024-05-09T04:52:37.324 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.324+0000 7f8a7e9ba740 1 -- start start 2024-05-09T04:52:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.324+0000 7f8a7e9ba740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f2cd80 0x563313323e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.324+0000 7f8a7e9ba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f452f0 0x563313316640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.324+0000 7f8a7e9ba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f4d4d0 0x56331331dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.324+0000 7f8a7509b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f452f0 0x563313316640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.324+0000 7f8a7589c640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f2cd80 0x563313323e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.324+0000 7f8a7509b640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f452f0 0x563313316640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47254/0 (socket says 172.21.15.46:47254) 2024-05-09T04:52:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.324+0000 7f8a7509b640 1 -- 172.21.15.46:0/928551677 learned_addr learned my addr 172.21.15.46:0/928551677 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:37.325 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.324+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/928551677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8a54011010 con 0x563312f452f0 2024-05-09T04:52:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.324+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/928551677 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5400e8e0 con 0x563312f2cd80 2024-05-09T04:52:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.324+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/928551677 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5400c730 con 0x563312f4d4d0 2024-05-09T04:52:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.325+0000 7f8a7609d640 1 --2- 172.21.15.46:0/928551677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f4d4d0 0x56331331dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.325+0000 7f8a7509b640 1 -- 172.21.15.46:0/928551677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f4d4d0 msgr2=0x56331331dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.325+0000 7f8a7509b640 1 --2- 172.21.15.46:0/928551677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f4d4d0 0x56331331dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.325+0000 7f8a7509b640 1 -- 172.21.15.46:0/928551677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f2cd80 msgr2=0x563313323e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.325+0000 7f8a7509b640 1 --2- 172.21.15.46:0/928551677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f2cd80 0x563313323e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.325+0000 7f8a7509b640 1 -- 172.21.15.46:0/928551677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633130505f0 con 0x563312f452f0 2024-05-09T04:52:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.325+0000 7f8a7609d640 1 --2- 172.21.15.46:0/928551677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f4d4d0 0x56331331dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:52:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.325+0000 7f8a7589c640 1 --2- 172.21.15.46:0/928551677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f2cd80 0x563313323e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:52:37.326 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.325+0000 7f8a7509b640 1 --2- 172.21.15.46:0/928551677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f452f0 0x563313316640 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8a600c59e0 tx=0x7f8a600d2050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a52f2f0be4b93273 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.326+0000 7f8a5b7fe640 1 -- 172.21.15.46:0/928551677 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60049030 con 0x563312f452f0 2024-05-09T04:52:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.326+0000 7f8a5b7fe640 1 -- 172.21.15.46:0/928551677 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a600c9470 con 0x563312f452f0 2024-05-09T04:52:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.326+0000 7f8a5b7fe640 1 -- 172.21.15.46:0/928551677 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60032600 con 0x563312f452f0 2024-05-09T04:52:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.327+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/928551677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f452f0 msgr2=0x563313316640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.327+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/928551677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f452f0 0x563313316640 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f8a600c59e0 tx=0x7f8a600d2050 comp rx=0 tx=0).stop 2024-05-09T04:52:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.327+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/928551677 shutdown_connections 2024-05-09T04:52:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.327+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/928551677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f4d4d0 0x56331331dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.328 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.327+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/928551677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f452f0 0x563313316640 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.327+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/928551677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f2cd80 0x563313323e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.327+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/928551677 >> 172.21.15.46:0/928551677 conn(0x563312f62300 msgr2=0x563313315f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.327+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/928551677 shutdown_connections 2024-05-09T04:52:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.327+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/928551677 wait complete. 2024-05-09T04:52:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.328+0000 7f8a7e9ba740 1 -- start start 2024-05-09T04:52:37.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.328+0000 7f8a7e9ba740 1 Processor -- start 2024-05-09T04:52:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7e9ba740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f2cd80 0x563313330570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7e9ba740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f452f0 0x563313329930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7e9ba740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f4d4d0 0x563313329e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7e9ba740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5400cda0 con 0x563312f4d4d0 2024-05-09T04:52:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7e9ba740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5400e210 con 0x563312f452f0 2024-05-09T04:52:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7e9ba740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8a5400d470 con 0x563312f2cd80 2024-05-09T04:52:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7509b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f452f0 0x563313329930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7509b640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f452f0 0x563313329930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:52864/0 (socket says 172.21.15.46:52864) 2024-05-09T04:52:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7509b640 1 -- 172.21.15.46:0/1469291982 learned_addr learned my addr 172.21.15.46:0/1469291982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7589c640 1 --2- 172.21.15.46:0/1469291982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f2cd80 0x563313330570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7609d640 1 --2- 172.21.15.46:0/1469291982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f4d4d0 0x563313329e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7e9ba740 10 client.0 _ll_register_callbacks cb 0x7f8a60005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7509b640 1 -- 172.21.15.46:0/1469291982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f2cd80 msgr2=0x563313330570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7509b640 1 --2- 172.21.15.46:0/1469291982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f2cd80 0x563313330570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7509b640 1 -- 172.21.15.46:0/1469291982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f4d4d0 msgr2=0x563313329e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7509b640 1 --2- 172.21.15.46:0/1469291982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f4d4d0 0x563313329e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7509b640 1 -- 172.21.15.46:0/1469291982 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56331323aed0 con 0x563312f452f0 2024-05-09T04:52:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7609d640 1 --2- 172.21.15.46:0/1469291982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f4d4d0 0x563313329e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:52:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.329+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/1469291982 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a600c9470 con 0x563312f452f0 2024-05-09T04:52:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.330+0000 7f8a7509b640 1 --2- 172.21.15.46:0/1469291982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f452f0 0x563313329930 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8a600d2560 tx=0x7f8a600d2590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b7821f10a19d925 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.330+0000 7f8a597fa640 10 client.25409 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:52:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.330+0000 7f8a597fa640 1 -- 172.21.15.46:0/1469291982 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a60049030 con 0x563312f452f0 2024-05-09T04:52:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.330+0000 7f8a597fa640 1 -- 172.21.15.46:0/1469291982 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f8a600ead70 con 0x563312f452f0 2024-05-09T04:52:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.330+0000 7f8a597fa640 1 -- 172.21.15.46:0/1469291982 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8a600c90d0 con 0x563312f452f0 2024-05-09T04:52:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.330+0000 7f8a7e9ba740 10 client.25409 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.330+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/1469291982 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({fsmap=25}) -- 0x56331323aed0 con 0x563312f452f0 2024-05-09T04:52:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.331+0000 7f8a597fa640 1 -- 172.21.15.46:0/1469291982 <== mon.2 v2:172.21.15.72:3300/0 4 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f8a6000b040 con 0x563312f452f0 2024-05-09T04:52:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.331+0000 7f8a7e9ba740 10 client.25409 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.331+0000 7f8a7e9ba740 10 client.25409 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.331+0000 7f8a7e9ba740 20 client.25409 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.331+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/1469291982 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563312f6e1b0 0x5633133a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.331+0000 7f8a7e9ba740 4 client.25409 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T04:52:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.331+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/1469291982 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56331327e240 con 0x563312f6e1b0 2024-05-09T04:52:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.331+0000 7f8a7589c640 1 --2- 172.21.15.46:0/1469291982 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563312f6e1b0 0x5633133a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.332+0000 7f8a597fa640 10 client.25409 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:37.334 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.332+0000 7f8a7589c640 1 --2- 172.21.15.46:0/1469291982 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563312f6e1b0 0x5633133a0bf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13c678eec7419806 server_cookie=dc7ec0399904b16e in_seq=0 out_seq=0 2024-05-09T04:52:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.338+0000 7f8a597fa640 1 -- 172.21.15.46:0/1469291982 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56331327e240 con 0x563312f6e1b0 2024-05-09T04:52:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.338+0000 7f8a597fa640 10 client.25409 handle_command_reply: tid=0 2024-05-09T04:52:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.338+0000 7f8a7e9ba740 1 client.25409 shutdown 2024-05-09T04:52:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.338+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/1469291982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f452f0 msgr2=0x563313329930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.340 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.338+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/1469291982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f452f0 0x563313329930 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8a600d2560 tx=0x7f8a600d2590 comp rx=0 tx=0).stop 2024-05-09T04:52:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.338+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/1469291982 shutdown_connections 2024-05-09T04:52:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.338+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/1469291982 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563312f6e1b0 0x5633133a0bf0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.338+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/1469291982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563312f4d4d0 0x563313329e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.338+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/1469291982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563312f452f0 0x563313329930 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.338+0000 7f8a7e9ba740 1 --2- 172.21.15.46:0/1469291982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563312f2cd80 0x563313330570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.338+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/1469291982 >> 172.21.15.46:0/1469291982 conn(0x563312f62300 msgr2=0x563313322740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:37.341 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.338+0000 7f8a597fa640 0 client.25409 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.338+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/1469291982 shutdown_connections 2024-05-09T04:52:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.338+0000 7f8a7e9ba740 1 -- 172.21.15.46:0/1469291982 wait complete. 2024-05-09T04:52:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.339+0000 7f8a7e9ba740 20 client.25409 trim_cache size 0 max 16384 2024-05-09T04:52:37.342 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-05-09T04:52:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-05-09T04:52:37.343 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-05-09T04:52:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.339+0000 7f8a77b00640 1 -- 172.21.15.46:0/3971007131 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8a5c0b2b70 msgr2=0x7f8a5c0b5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.339+0000 7f8a77b00640 1 --2- 172.21.15.46:0/3971007131 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8a5c0b2b70 0x7f8a5c0b5080 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x56331331b240 tx=0x7f8a6c012040 comp rx=0 tx=0).stop 2024-05-09T04:52:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.339+0000 7f8a77b00640 1 -- 172.21.15.46:0/3971007131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a70106b10 msgr2=0x7f8a70198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:37.344 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.339+0000 7f8a77b00640 1 --2- 172.21.15.46:0/3971007131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a70106b10 0x7f8a70198c10 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f8a60005ab0 tx=0x7f8a60003b90 comp rx=0 tx=0).stop 2024-05-09T04:52:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.339+0000 7f8a77b00640 1 -- 172.21.15.46:0/3971007131 shutdown_connections 2024-05-09T04:52:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.339+0000 7f8a77b00640 1 --2- 172.21.15.46:0/3971007131 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8a5c0b2b70 0x7f8a5c0b5080 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.339+0000 7f8a77b00640 1 --2- 172.21.15.46:0/3971007131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8a701075d0 0x7f8a7019ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.339+0000 7f8a77b00640 1 --2- 172.21.15.46:0/3971007131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8a70106b10 0x7f8a70198c10 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.339+0000 7f8a77b00640 1 --2- 172.21.15.46:0/3971007131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8a70102580 0x7f8a701986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:37.345 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.339+0000 7f8a77b00640 1 -- 172.21.15.46:0/3971007131 >> 172.21.15.46:0/3971007131 conn(0x7f8a700779e0 msgr2=0x7f8a70100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.340+0000 7f8a77b00640 1 -- 172.21.15.46:0/3971007131 shutdown_connections 2024-05-09T04:52:37.346 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:37.340+0000 7f8a77b00640 1 -- 172.21.15.46:0/3971007131 wait complete. 2024-05-09T04:52:37.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:37 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2804032581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:52:37.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:52:38.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:37 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2804032581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:52:38.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:37 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2804032581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:52:38.263 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:39.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:38 smithi110 ceph-mon[34904]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-09T04:52:39.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:38 smithi110 ceph-mon[34904]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-09T04:52:39.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:38 smithi046 ceph-mon[30136]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-09T04:52:39.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:38 smithi046 ceph-mon[30136]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-09T04:52:39.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:38 smithi072 ceph-mon[34088]: pgmap v49: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-09T04:52:39.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:38 smithi072 ceph-mon[34088]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-05-09T04:52:40.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.248+0000 7fc58cc94700 1 -- 172.21.15.46:0/3160864913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc588105dd0 msgr2=0x7fc58810e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:40.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.248+0000 7fc58cc94700 1 --2- 172.21.15.46:0/3160864913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc588105dd0 0x7fc58810e320 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc57c009990 tx=0x7fc57c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.249+0000 7fc58cc94700 1 -- 172.21.15.46:0/3160864913 shutdown_connections 2024-05-09T04:52:40.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.249+0000 7fc58cc94700 1 --2- 172.21.15.46:0/3160864913 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc58810e8f0 0x7fc588110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.250+0000 7fc58cc94700 1 --2- 172.21.15.46:0/3160864913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc588105480 0x7fc588105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.250+0000 7fc58cc94700 1 --2- 172.21.15.46:0/3160864913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc588105dd0 0x7fc58810e320 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.250+0000 7fc58cc94700 1 -- 172.21.15.46:0/3160864913 >> 172.21.15.46:0/3160864913 conn(0x7fc5880fef30 msgr2=0x7fc588101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:40.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.250+0000 7fc58cc94700 1 -- 172.21.15.46:0/3160864913 shutdown_connections 2024-05-09T04:52:40.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.250+0000 7fc58cc94700 1 -- 172.21.15.46:0/3160864913 wait complete. 2024-05-09T04:52:40.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.251+0000 7fc58cc94700 1 Processor -- start 2024-05-09T04:52:40.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.251+0000 7fc58cc94700 1 -- start start 2024-05-09T04:52:40.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.251+0000 7fc58cc94700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc588105480 0x7fc58819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.251+0000 7fc58cc94700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc588105dd0 0x7fc58819b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.251+0000 7fc58cc94700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc58810e8f0 0x7fc58819f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.251+0000 7fc58cc94700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc588114450 con 0x7fc588105480 2024-05-09T04:52:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.251+0000 7fc58cc94700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc5881142d0 con 0x7fc588105dd0 2024-05-09T04:52:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.251+0000 7fc58cc94700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc5881145d0 con 0x7fc58810e8f0 2024-05-09T04:52:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.251+0000 7fc58659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc588105480 0x7fc58819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.251+0000 7fc58659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc588105480 0x7fc58819ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47264/0 (socket says 172.21.15.46:47264) 2024-05-09T04:52:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.251+0000 7fc58659c700 1 -- 172.21.15.46:0/2056211504 learned_addr learned my addr 172.21.15.46:0/2056211504 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.251+0000 7fc585d9b700 1 --2- 172.21.15.46:0/2056211504 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc588105dd0 0x7fc58819b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.252+0000 7fc586d9d700 1 --2- 172.21.15.46:0/2056211504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc58810e8f0 0x7fc58819f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.252+0000 7fc58659c700 1 -- 172.21.15.46:0/2056211504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc58810e8f0 msgr2=0x7fc58819f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:40.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.252+0000 7fc58659c700 1 --2- 172.21.15.46:0/2056211504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc58810e8f0 0x7fc58819f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.252+0000 7fc58659c700 1 -- 172.21.15.46:0/2056211504 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc588105dd0 msgr2=0x7fc58819b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:40.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.252+0000 7fc58659c700 1 --2- 172.21.15.46:0/2056211504 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc588105dd0 0x7fc58819b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.252+0000 7fc58659c700 1 -- 172.21.15.46:0/2056211504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc58819fcb0 con 0x7fc588105480 2024-05-09T04:52:40.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.252+0000 7fc58659c700 1 --2- 172.21.15.46:0/2056211504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc588105480 0x7fc58819ade0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc57800d980 tx=0x7fc57800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:40.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.252+0000 7fc5777fe700 1 -- 172.21.15.46:0/2056211504 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578016070 con 0x7fc588105480 2024-05-09T04:52:40.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.252+0000 7fc58cc94700 1 -- 172.21.15.46:0/2056211504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc58819ffa0 con 0x7fc588105480 2024-05-09T04:52:40.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.252+0000 7fc58cc94700 1 -- 172.21.15.46:0/2056211504 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5881a78c0 con 0x7fc588105480 2024-05-09T04:52:40.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.252+0000 7fc5777fe700 1 -- 172.21.15.46:0/2056211504 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc578004830 con 0x7fc588105480 2024-05-09T04:52:40.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.252+0000 7fc5777fe700 1 -- 172.21.15.46:0/2056211504 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc578005580 con 0x7fc588105480 2024-05-09T04:52:40.256 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.257+0000 7fc5777fe700 1 -- 172.21.15.46:0/2056211504 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc57800b750 con 0x7fc588105480 2024-05-09T04:52:40.257 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.257+0000 7fc5777fe700 1 --2- 172.21.15.46:0/2056211504 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5700b2a70 0x7fc5700b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.257+0000 7fc5777fe700 1 -- 172.21.15.46:0/2056211504 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fc5780e22e0 con 0x7fc588105480 2024-05-09T04:52:40.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.258+0000 7fc59b6aab80 1 Processor -- start 2024-05-09T04:52:40.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.258+0000 7fc585d9b700 1 --2- 172.21.15.46:0/2056211504 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5700b2a70 0x7fc5700b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.258+0000 7fc59b6aab80 1 -- start start 2024-05-09T04:52:40.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.258+0000 7fc585d9b700 1 --2- 172.21.15.46:0/2056211504 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5700b2a70 0x7fc5700b4f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0098e0 tx=0x7fc57c005960 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:40.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc59b6aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e6277160 0x5614e63104f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc59b6aab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 0x5614e6310a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc59b6aab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e630a6a0 0x5614e630aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc59b6aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5614e622cff0 con 0x5614e6277160 2024-05-09T04:52:40.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc59b6aab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5614e61cb7f0 con 0x5614e618d4a0 2024-05-09T04:52:40.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc59b6aab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5614e6228fb0 con 0x5614e630a6a0 2024-05-09T04:52:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc58659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e6277160 0x5614e63104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc58659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e6277160 0x5614e63104f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47276/0 (socket says 172.21.15.46:47276) 2024-05-09T04:52:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc58659c700 1 -- 172.21.15.46:0/4066809792 learned_addr learned my addr 172.21.15.46:0/4066809792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc586d9d700 1 --2- 172.21.15.46:0/4066809792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e630a6a0 0x5614e630aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc58659c700 1 -- 172.21.15.46:0/4066809792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e630a6a0 msgr2=0x5614e630aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc58659c700 1 --2- 172.21.15.46:0/4066809792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e630a6a0 0x5614e630aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc58659c700 1 -- 172.21.15.46:0/4066809792 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 msgr2=0x5614e6310a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc58659c700 1 --2- 172.21.15.46:0/4066809792 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 0x5614e6310a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc58659c700 1 -- 172.21.15.46:0/4066809792 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e61c3370 con 0x5614e6277160 2024-05-09T04:52:40.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.259+0000 7fc58659c700 1 --2- 172.21.15.46:0/4066809792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e6277160 0x5614e63104f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc5780a92a0 tx=0x7fc5780a8e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82255214b81143 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:40.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.260+0000 7fc574ff9700 1 -- 172.21.15.46:0/4066809792 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5780a95a0 con 0x5614e6277160 2024-05-09T04:52:40.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.260+0000 7fc574ff9700 1 -- 172.21.15.46:0/4066809792 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5780ad950 con 0x5614e6277160 2024-05-09T04:52:40.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.260+0000 7fc574ff9700 1 -- 172.21.15.46:0/4066809792 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5780a9a60 con 0x5614e6277160 2024-05-09T04:52:40.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.261+0000 7fc59b6aab80 1 -- 172.21.15.46:0/4066809792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e6277160 msgr2=0x5614e63104f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:40.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.261+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/4066809792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e6277160 0x5614e63104f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc5780a92a0 tx=0x7fc5780a8e00 comp rx=0 tx=0).stop 2024-05-09T04:52:40.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.261+0000 7fc59b6aab80 1 -- 172.21.15.46:0/4066809792 shutdown_connections 2024-05-09T04:52:40.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.261+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/4066809792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e630a6a0 0x5614e630aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.261+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/4066809792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e6277160 0x5614e63104f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.261+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/4066809792 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 0x5614e6310a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.261+0000 7fc59b6aab80 1 -- 172.21.15.46:0/4066809792 >> 172.21.15.46:0/4066809792 conn(0x5614e61ed6a0 msgr2=0x5614e6305400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:40.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.261+0000 7fc59b6aab80 1 -- 172.21.15.46:0/4066809792 shutdown_connections 2024-05-09T04:52:40.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.261+0000 7fc59b6aab80 1 -- 172.21.15.46:0/4066809792 wait complete. 2024-05-09T04:52:40.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.262+0000 7fc59b6aab80 1 -- start start 2024-05-09T04:52:40.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.262+0000 7fc59b6aab80 1 Processor -- start 2024-05-09T04:52:40.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc59b6aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e618d4a0 0x5614e6314230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc59b6aab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e6277160 0x5614e6314770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc59b6aab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e630a6a0 0x5614e6314cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc59b6aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5614e60a8ba0 con 0x5614e618d4a0 2024-05-09T04:52:40.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc59b6aab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5614e61e7150 con 0x5614e6277160 2024-05-09T04:52:40.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc59b6aab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5614e63171d0 con 0x5614e630a6a0 2024-05-09T04:52:40.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc59b6aab80 10 client.0 _ll_register_callbacks cb 0x5614e609e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:40.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc58659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e618d4a0 0x5614e6314230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc585d9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e6277160 0x5614e6314770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc58659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e618d4a0 0x5614e6314230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47286/0 (socket says 172.21.15.46:47286) 2024-05-09T04:52:40.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc58659c700 1 -- 172.21.15.46:0/311886543 learned_addr learned my addr 172.21.15.46:0/311886543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc586d9d700 1 --2- 172.21.15.46:0/311886543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e630a6a0 0x5614e6314cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc585d9b700 1 -- 172.21.15.46:0/311886543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e630a6a0 msgr2=0x5614e6314cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc585d9b700 1 --2- 172.21.15.46:0/311886543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e630a6a0 0x5614e6314cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc585d9b700 1 -- 172.21.15.46:0/311886543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e618d4a0 msgr2=0x5614e6314230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc585d9b700 1 --2- 172.21.15.46:0/311886543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e618d4a0 0x5614e6314230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc585d9b700 1 -- 172.21.15.46:0/311886543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e60a8490 con 0x5614e6277160 2024-05-09T04:52:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.263+0000 7fc58659c700 1 --2- 172.21.15.46:0/311886543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e618d4a0 0x5614e6314230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.264+0000 7fc585d9b700 1 --2- 172.21.15.46:0/311886543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e6277160 0x5614e6314770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc57c006010 tx=0x7fc57c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd2308bb199c4e7e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.264+0000 7fc56dffb700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:52:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.264+0000 7fc56dffb700 1 -- 172.21.15.46:0/311886543 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc57c00b9e0 con 0x5614e6277160 2024-05-09T04:52:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.264+0000 7fc59b6aab80 1 -- 172.21.15.46:0/311886543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5780ad950 con 0x5614e6277160 2024-05-09T04:52:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.264+0000 7fc56dffb700 1 -- 172.21.15.46:0/311886543 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fc57c00bb80 con 0x5614e6277160 2024-05-09T04:52:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.264+0000 7fc56dffb700 1 -- 172.21.15.46:0/311886543 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc57c00bd80 con 0x5614e6277160 2024-05-09T04:52:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.264+0000 7fc59b6aab80 10 client.25418 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.264+0000 7fc59b6aab80 1 -- 172.21.15.46:0/311886543 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614e6321c60 con 0x5614e6277160 2024-05-09T04:52:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.265+0000 7fc56dffb700 1 -- 172.21.15.46:0/311886543 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5780ad950 con 0x5614e6277160 2024-05-09T04:52:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.265+0000 7fc56dffb700 1 -- 172.21.15.46:0/311886543 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc57c022070 con 0x5614e6277160 2024-05-09T04:52:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.265+0000 7fc59b6aab80 10 client.25418 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.265+0000 7fc59b6aab80 10 client.25418 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:52:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.265+0000 7fc59b6aab80 20 client.25418 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:40.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.265+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/311886543 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5614e6324240 0x5614e6326680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.265+0000 7fc59b6aab80 4 client.25418 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.265+0000 7fc59b6aab80 1 -- 172.21.15.46:0/311886543 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614e623a9d0 con 0x5614e6324240 2024-05-09T04:52:40.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.265+0000 7fc58659c700 1 --2- 172.21.15.46:0/311886543 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5614e6324240 0x5614e6326680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.266+0000 7fc56dffb700 10 client.25418 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:52:40.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.266+0000 7fc58659c700 1 --2- 172.21.15.46:0/311886543 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5614e6324240 0x5614e6326680 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dac49f745a722611 server_cookie=fedc380bdc5a2f07 in_seq=0 out_seq=0 2024-05-09T04:52:40.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.266+0000 7fc56dffb700 1 -- 172.21.15.46:0/311886543 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5614e623a9d0 con 0x5614e6324240 2024-05-09T04:52:40.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.266+0000 7fc56dffb700 10 client.25418 handle_command_reply: tid=0 2024-05-09T04:52:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.267+0000 7fc59b6aab80 1 client.25418 shutdown 2024-05-09T04:52:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.267+0000 7fc59b6aab80 1 -- 172.21.15.46:0/311886543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e6277160 msgr2=0x5614e6314770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.267+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/311886543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e6277160 0x5614e6314770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc57c006010 tx=0x7fc57c009310 comp rx=0 tx=0).stop 2024-05-09T04:52:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.267+0000 7fc59b6aab80 1 -- 172.21.15.46:0/311886543 shutdown_connections 2024-05-09T04:52:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.267+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/311886543 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5614e6324240 0x5614e6326680 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.267+0000 7fc56dffb700 0 client.25418 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:52:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.267+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/311886543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e630a6a0 0x5614e6314cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.267+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/311886543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e618d4a0 0x5614e6314230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.267+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/311886543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e6277160 0x5614e6314770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.267+0000 7fc59b6aab80 1 -- 172.21.15.46:0/311886543 >> 172.21.15.46:0/311886543 conn(0x5614e61ed6a0 msgr2=0x5614e63051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.267+0000 7fc59b6aab80 1 -- 172.21.15.46:0/311886543 shutdown_connections 2024-05-09T04:52:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.267+0000 7fc59b6aab80 1 -- 172.21.15.46:0/311886543 wait complete. 2024-05-09T04:52:40.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.267+0000 7fc59b6aab80 20 client.25418 trim_cache size 0 max 16384 2024-05-09T04:52:40.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.283+0000 7fc59b6aab80 1 Processor -- start 2024-05-09T04:52:40.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.283+0000 7fc59b6aab80 1 -- start start 2024-05-09T04:52:40.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.283+0000 7fc59b6aab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 0x5614e630e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.284+0000 7fc59b6aab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e6277160 0x5614e630cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.284+0000 7fc59b6aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e630a6a0 0x5614e630d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.284+0000 7fc59b6aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc56800ff40 con 0x5614e630a6a0 2024-05-09T04:52:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.284+0000 7fc59b6aab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc56800d7d0 con 0x5614e618d4a0 2024-05-09T04:52:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.284+0000 7fc59b6aab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc56800b620 con 0x5614e6277160 2024-05-09T04:52:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.284+0000 7fc586d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e630a6a0 0x5614e630d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.284+0000 7fc58659c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 0x5614e630e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.284+0000 7fc58659c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 0x5614e630e7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:52912/0 (socket says 172.21.15.46:52912) 2024-05-09T04:52:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.284+0000 7fc58659c700 1 -- 172.21.15.46:0/549233684 learned_addr learned my addr 172.21.15.46:0/549233684 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.284+0000 7fc585d9b700 1 --2- 172.21.15.46:0/549233684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e6277160 0x5614e630cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.284+0000 7fc58659c700 1 -- 172.21.15.46:0/549233684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e6277160 msgr2=0x5614e630cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.284+0000 7fc58659c700 1 --2- 172.21.15.46:0/549233684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e6277160 0x5614e630cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.284+0000 7fc58659c700 1 -- 172.21.15.46:0/549233684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e630a6a0 msgr2=0x5614e630d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.284+0000 7fc58659c700 1 --2- 172.21.15.46:0/549233684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e630a6a0 0x5614e630d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.284+0000 7fc58659c700 1 -- 172.21.15.46:0/549233684 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614e630b640 con 0x5614e618d4a0 2024-05-09T04:52:40.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.285+0000 7fc58659c700 1 --2- 172.21.15.46:0/549233684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 0x5614e630e7c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc5780910b0 tx=0x7fc578091040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d029c7ec8d6535e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.285+0000 7fc56ffff700 1 -- 172.21.15.46:0/549233684 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5780ad950 con 0x5614e618d4a0 2024-05-09T04:52:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.285+0000 7fc56ffff700 1 -- 172.21.15.46:0/549233684 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5780a95a0 con 0x5614e618d4a0 2024-05-09T04:52:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.285+0000 7fc56ffff700 1 -- 172.21.15.46:0/549233684 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5780a9a40 con 0x5614e618d4a0 2024-05-09T04:52:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.286+0000 7fc59b6aab80 1 -- 172.21.15.46:0/549233684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 msgr2=0x5614e630e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:40.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.286+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/549233684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 0x5614e630e7c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc5780910b0 tx=0x7fc578091040 comp rx=0 tx=0).stop 2024-05-09T04:52:40.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.286+0000 7fc59b6aab80 1 -- 172.21.15.46:0/549233684 shutdown_connections 2024-05-09T04:52:40.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.286+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/549233684 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e630a6a0 0x5614e630d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.286+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/549233684 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 0x5614e630e7c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.286+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/549233684 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e6277160 0x5614e630cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.286+0000 7fc59b6aab80 1 -- 172.21.15.46:0/549233684 >> 172.21.15.46:0/549233684 conn(0x5614e61ed6a0 msgr2=0x5614e630f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:40.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.286+0000 7fc59b6aab80 1 -- 172.21.15.46:0/549233684 shutdown_connections 2024-05-09T04:52:40.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.286+0000 7fc59b6aab80 1 -- 172.21.15.46:0/549233684 wait complete. 2024-05-09T04:52:40.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.287+0000 7fc59b6aab80 1 -- start start 2024-05-09T04:52:40.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.287+0000 7fc59b6aab80 1 Processor -- start 2024-05-09T04:52:40.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.287+0000 7fc59b6aab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 0x5614e632ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.287+0000 7fc59b6aab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e6277160 0x5614e632b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.287+0000 7fc59b6aab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e630a6a0 0x5614e632b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.287+0000 7fc59b6aab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc56800bc90 con 0x5614e630a6a0 2024-05-09T04:52:40.288 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.287+0000 7fc59b6aab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc56800d100 con 0x5614e618d4a0 2024-05-09T04:52:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.287+0000 7fc59b6aab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc56800c360 con 0x5614e6277160 2024-05-09T04:52:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.287+0000 7fc59b6aab80 10 client.0 _ll_register_callbacks cb 0x5614e609e6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.287+0000 7fc585d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e6277160 0x5614e632b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.287+0000 7fc58659c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 0x5614e632ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.287+0000 7fc585d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e6277160 0x5614e632b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:46766/0 (socket says 172.21.15.46:46766) 2024-05-09T04:52:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.287+0000 7fc585d9b700 1 -- 172.21.15.46:0/4123354615 learned_addr learned my addr 172.21.15.46:0/4123354615 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:40.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.287+0000 7fc586d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e630a6a0 0x5614e632b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.288+0000 7fc586d9d700 1 -- 172.21.15.46:0/4123354615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e6277160 msgr2=0x5614e632b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:40.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.288+0000 7fc586d9d700 1 --2- 172.21.15.46:0/4123354615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e6277160 0x5614e632b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.288+0000 7fc586d9d700 1 -- 172.21.15.46:0/4123354615 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 msgr2=0x5614e632ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:40.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.288+0000 7fc586d9d700 1 --2- 172.21.15.46:0/4123354615 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 0x5614e632ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.288+0000 7fc586d9d700 1 -- 172.21.15.46:0/4123354615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614e62fef60 con 0x5614e630a6a0 2024-05-09T04:52:40.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.288+0000 7fc585d9b700 1 --2- 172.21.15.46:0/4123354615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e6277160 0x5614e632b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:40.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.288+0000 7fc586d9d700 1 --2- 172.21.15.46:0/4123354615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e630a6a0 0x5614e632b8b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x5614e6300400 tx=0x7fc5800077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f96a74eb92c3914 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:40.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.288+0000 7fc56dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:40.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.288+0000 7fc56dffb700 1 -- 172.21.15.46:0/4123354615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc58000f070 con 0x5614e630a6a0 2024-05-09T04:52:40.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.288+0000 7fc59b6aab80 1 -- 172.21.15.46:0/4123354615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5780a95a0 con 0x5614e630a6a0 2024-05-09T04:52:40.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.288+0000 7fc56dffb700 1 -- 172.21.15.46:0/4123354615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fc580004030 con 0x5614e630a6a0 2024-05-09T04:52:40.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.289+0000 7fc56dffb700 1 -- 172.21.15.46:0/4123354615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc580026050 con 0x5614e630a6a0 2024-05-09T04:52:40.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.289+0000 7fc59b6aab80 10 client.15681 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:40.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.289+0000 7fc59b6aab80 1 -- 172.21.15.46:0/4123354615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5614e6306eb0 con 0x5614e630a6a0 2024-05-09T04:52:40.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.289+0000 7fc56dffb700 1 -- 172.21.15.46:0/4123354615 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5780a95a0 con 0x5614e630a6a0 2024-05-09T04:52:40.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.289+0000 7fc56dffb700 1 -- 172.21.15.46:0/4123354615 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc580018700 con 0x5614e630a6a0 2024-05-09T04:52:40.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.289+0000 7fc59b6aab80 10 client.15681 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:40.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.289+0000 7fc59b6aab80 10 client.15681 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:52:40.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.289+0000 7fc59b6aab80 20 client.15681 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:40.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.289+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/4123354615 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5614e6324240 0x5614e6394e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:40.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.289+0000 7fc59b6aab80 4 client.15681 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:52:40.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.289+0000 7fc59b6aab80 1 -- 172.21.15.46:0/4123354615 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614e6234a70 con 0x5614e6324240 2024-05-09T04:52:40.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.290+0000 7fc58659c700 1 --2- 172.21.15.46:0/4123354615 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5614e6324240 0x5614e6394e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:40.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.290+0000 7fc56dffb700 10 client.15681 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:52:40.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.290+0000 7fc58659c700 1 --2- 172.21.15.46:0/4123354615 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5614e6324240 0x5614e6394e10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fc25252364c1bb89 server_cookie=e36ca9dd5858ec4c in_seq=0 out_seq=0 2024-05-09T04:52:40.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.290+0000 7fc56dffb700 1 -- 172.21.15.46:0/4123354615 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614e6234a70 con 0x5614e6324240 2024-05-09T04:52:40.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.290+0000 7fc56dffb700 10 client.15681 handle_command_reply: tid=0 2024-05-09T04:52:40.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.290+0000 7fc59b6aab80 1 client.15681 shutdown 2024-05-09T04:52:40.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.290+0000 7fc59b6aab80 1 -- 172.21.15.46:0/4123354615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e630a6a0 msgr2=0x5614e632b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:40.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.290+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/4123354615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e630a6a0 0x5614e632b8b0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x5614e6300400 tx=0x7fc5800077e0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.291+0000 7fc59b6aab80 1 -- 172.21.15.46:0/4123354615 shutdown_connections 2024-05-09T04:52:40.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.291+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/4123354615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5614e630a6a0 0x5614e632b8b0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.291+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/4123354615 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5614e618d4a0 0x5614e632ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.291+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/4123354615 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5614e6324240 0x5614e6394e10 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.291+0000 7fc59b6aab80 1 --2- 172.21.15.46:0/4123354615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5614e6277160 0x5614e632b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.291+0000 7fc59b6aab80 1 -- 172.21.15.46:0/4123354615 >> 172.21.15.46:0/4123354615 conn(0x5614e61ed6a0 msgr2=0x5614e6312b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:40.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.291+0000 7fc56dffb700 0 client.15681 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:52:40.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.291+0000 7fc59b6aab80 1 -- 172.21.15.46:0/4123354615 shutdown_connections 2024-05-09T04:52:40.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.291+0000 7fc59b6aab80 1 -- 172.21.15.46:0/4123354615 wait complete. 2024-05-09T04:52:40.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.291+0000 7fc59b6aab80 20 client.15681 trim_cache size 0 max 16384 2024-05-09T04:52:40.294 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:52:40.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.291+0000 7fc58cc94700 1 -- 172.21.15.46:0/2056211504 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5700b2a70 msgr2=0x7fc5700b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:40.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.291+0000 7fc58cc94700 1 --2- 172.21.15.46:0/2056211504 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5700b2a70 0x7fc5700b4f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc57c0098e0 tx=0x7fc57c005960 comp rx=0 tx=0).stop 2024-05-09T04:52:40.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.291+0000 7fc58cc94700 1 -- 172.21.15.46:0/2056211504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc588105480 msgr2=0x7fc58819ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:40.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.291+0000 7fc58cc94700 1 --2- 172.21.15.46:0/2056211504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc588105480 0x7fc58819ade0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fc57800d980 tx=0x7fc57800dc90 comp rx=0 tx=0).stop 2024-05-09T04:52:40.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.292+0000 7fc58cc94700 1 -- 172.21.15.46:0/2056211504 shutdown_connections 2024-05-09T04:52:40.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.292+0000 7fc58cc94700 1 --2- 172.21.15.46:0/2056211504 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5700b2a70 0x7fc5700b4f80 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.292+0000 7fc58cc94700 1 --2- 172.21.15.46:0/2056211504 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc58810e8f0 0x7fc58819f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.292+0000 7fc58cc94700 1 --2- 172.21.15.46:0/2056211504 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc588105480 0x7fc58819ade0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.292+0000 7fc58cc94700 1 --2- 172.21.15.46:0/2056211504 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc588105dd0 0x7fc58819b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:40.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.292+0000 7fc58cc94700 1 -- 172.21.15.46:0/2056211504 >> 172.21.15.46:0/2056211504 conn(0x7fc5880fef30 msgr2=0x7fc5880ff5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:40.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.292+0000 7fc58cc94700 1 -- 172.21.15.46:0/2056211504 shutdown_connections 2024-05-09T04:52:40.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:40.292+0000 7fc58cc94700 1 -- 172.21.15.46:0/2056211504 wait complete. 2024-05-09T04:52:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:40 smithi110 ceph-mon[34904]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-09T04:52:41.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:40 smithi046 ceph-mon[30136]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-09T04:52:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:40 smithi072 ceph-mon[34088]: pgmap v50: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-05-09T04:52:41.187 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:52:41.462 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:42.934 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:42 smithi046 ceph-mon[30136]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-09T04:52:43.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:42 smithi110 ceph-mon[34904]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-09T04:52:43.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:42 smithi072 ceph-mon[34088]: pgmap v51: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 853 B/s wr, 10 op/s 2024-05-09T04:52:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.340+0000 7f0487f23700 1 -- 172.21.15.46:0/1260497525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0480105480 msgr2=0x7f0480105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.340+0000 7f0487f23700 1 --2- 172.21.15.46:0/1260497525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0480105480 0x7f0480105890 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f047c009990 tx=0x7f047c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.340+0000 7f0487f23700 1 -- 172.21.15.46:0/1260497525 shutdown_connections 2024-05-09T04:52:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.340+0000 7f0487f23700 1 --2- 172.21.15.46:0/1260497525 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f048010e8f0 0x7f0480110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.340+0000 7f0487f23700 1 --2- 172.21.15.46:0/1260497525 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0480105480 0x7f0480105890 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.340+0000 7f0487f23700 1 --2- 172.21.15.46:0/1260497525 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0480105dd0 0x7f048010e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.340+0000 7f0487f23700 1 -- 172.21.15.46:0/1260497525 >> 172.21.15.46:0/1260497525 conn(0x7f04800fef30 msgr2=0x7f0480101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:43.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.340+0000 7f0487f23700 1 -- 172.21.15.46:0/1260497525 shutdown_connections 2024-05-09T04:52:43.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.340+0000 7f0487f23700 1 -- 172.21.15.46:0/1260497525 wait complete. 2024-05-09T04:52:43.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.341+0000 7f0487f23700 1 Processor -- start 2024-05-09T04:52:43.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.341+0000 7f0487f23700 1 -- start start 2024-05-09T04:52:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.341+0000 7f0487f23700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0480105480 0x7f04800753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.341+0000 7f0487f23700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0480105dd0 0x7f048006e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.342+0000 7f0487f23700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f048010e8f0 0x7f048006eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.342+0000 7f0487f23700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0480114440 con 0x7f0480105dd0 2024-05-09T04:52:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.342+0000 7f0487f23700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f04801142c0 con 0x7f048010e8f0 2024-05-09T04:52:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.342+0000 7f0487f23700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04801145c0 con 0x7f0480105480 2024-05-09T04:52:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.342+0000 7f04854be700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0480105dd0 0x7f048006e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.342+0000 7f04854be700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0480105dd0 0x7f048006e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47332/0 (socket says 172.21.15.46:47332) 2024-05-09T04:52:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.342+0000 7f04854be700 1 -- 172.21.15.46:0/1986835640 learned_addr learned my addr 172.21.15.46:0/1986835640 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.342+0000 7f0485cbf700 1 --2- 172.21.15.46:0/1986835640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0480105480 0x7f04800753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.342+0000 7f04864c0700 1 --2- 172.21.15.46:0/1986835640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f048010e8f0 0x7f048006eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.342+0000 7f04854be700 1 -- 172.21.15.46:0/1986835640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0480105480 msgr2=0x7f04800753a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.342+0000 7f04854be700 1 --2- 172.21.15.46:0/1986835640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0480105480 0x7f04800753a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.342+0000 7f04854be700 1 -- 172.21.15.46:0/1986835640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f048010e8f0 msgr2=0x7f048006eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.342+0000 7f04854be700 1 --2- 172.21.15.46:0/1986835640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f048010e8f0 0x7f048006eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.342+0000 7f04854be700 1 -- 172.21.15.46:0/1986835640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0480072ed0 con 0x7f0480105dd0 2024-05-09T04:52:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.342+0000 7f04854be700 1 --2- 172.21.15.46:0/1986835640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0480105dd0 0x7f048006e4d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f047000d980 tx=0x7f047000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.343+0000 7f0476ffd700 1 -- 172.21.15.46:0/1986835640 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470016070 con 0x7f0480105dd0 2024-05-09T04:52:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.343+0000 7f0476ffd700 1 -- 172.21.15.46:0/1986835640 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0470004830 con 0x7f0480105dd0 2024-05-09T04:52:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.343+0000 7f0476ffd700 1 -- 172.21.15.46:0/1986835640 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0470005540 con 0x7f0480105dd0 2024-05-09T04:52:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.343+0000 7f0487f23700 1 -- 172.21.15.46:0/1986835640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04800731c0 con 0x7f0480105dd0 2024-05-09T04:52:43.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.343+0000 7f0487f23700 1 -- 172.21.15.46:0/1986835640 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04800736b0 con 0x7f0480105dd0 2024-05-09T04:52:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.346+0000 7f0476ffd700 1 -- 172.21.15.46:0/1986835640 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0470004220 con 0x7f0480105dd0 2024-05-09T04:52:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.347+0000 7f0476ffd700 1 --2- 172.21.15.46:0/1986835640 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f046c0b2b40 0x7f046c0b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.347+0000 7f0485cbf700 1 --2- 172.21.15.46:0/1986835640 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f046c0b2b40 0x7f046c0b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.347+0000 7f0476ffd700 1 -- 172.21.15.46:0/1986835640 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f04700e2360 con 0x7f0480105dd0 2024-05-09T04:52:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.348+0000 7f0485cbf700 1 --2- 172.21.15.46:0/1986835640 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f046c0b2b40 0x7f046c0b5050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f048006fd20 tx=0x7f047c005960 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:43.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.350+0000 7f0496939b80 1 Processor -- start 2024-05-09T04:52:43.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.350+0000 7f0496939b80 1 -- start start 2024-05-09T04:52:43.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.350+0000 7f0496939b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec3c7160 0x5630ec4604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.351+0000 7f0496939b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec2dd4a0 0x5630ec460a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.351+0000 7f0496939b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec45a6a0 0x5630ec45aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.351+0000 7f0496939b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5630ec37cff0 con 0x5630ec3c7160 2024-05-09T04:52:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.351+0000 7f0496939b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5630ec31b7f0 con 0x5630ec2dd4a0 2024-05-09T04:52:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.351+0000 7f0496939b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5630ec378fb0 con 0x5630ec45a6a0 2024-05-09T04:52:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.351+0000 7f0485cbf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec3c7160 0x5630ec4604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.351+0000 7f0485cbf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec3c7160 0x5630ec4604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47344/0 (socket says 172.21.15.46:47344) 2024-05-09T04:52:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.351+0000 7f0485cbf700 1 -- 172.21.15.46:0/1813684132 learned_addr learned my addr 172.21.15.46:0/1813684132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:43.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.351+0000 7f04864c0700 1 --2- 172.21.15.46:0/1813684132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec45a6a0 0x5630ec45aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.351+0000 7f04854be700 1 --2- 172.21.15.46:0/1813684132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec2dd4a0 0x5630ec460a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.351+0000 7f04864c0700 1 -- 172.21.15.46:0/1813684132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec2dd4a0 msgr2=0x5630ec460a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:43.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.351+0000 7f04864c0700 1 --2- 172.21.15.46:0/1813684132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec2dd4a0 0x5630ec460a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.351+0000 7f04864c0700 1 -- 172.21.15.46:0/1813684132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec3c7160 msgr2=0x5630ec4604f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.352+0000 7f04864c0700 1 --2- 172.21.15.46:0/1813684132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec3c7160 0x5630ec4604f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.352+0000 7f04864c0700 1 -- 172.21.15.46:0/1813684132 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ec313370 con 0x5630ec45a6a0 2024-05-09T04:52:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.352+0000 7f0485cbf700 1 --2- 172.21.15.46:0/1813684132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec3c7160 0x5630ec4604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.352+0000 7f04864c0700 1 --2- 172.21.15.46:0/1813684132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec45a6a0 0x5630ec45aad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0478003db0 tx=0x7f0478016c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=56441a88c94525b0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.352+0000 7f046bfff700 1 -- 172.21.15.46:0/1813684132 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478009c70 con 0x5630ec45a6a0 2024-05-09T04:52:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.352+0000 7f046bfff700 1 -- 172.21.15.46:0/1813684132 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0478009e10 con 0x5630ec45a6a0 2024-05-09T04:52:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.352+0000 7f046bfff700 1 -- 172.21.15.46:0/1813684132 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0478007760 con 0x5630ec45a6a0 2024-05-09T04:52:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.353+0000 7f0496939b80 1 -- 172.21.15.46:0/1813684132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec45a6a0 msgr2=0x5630ec45aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.353+0000 7f0496939b80 1 --2- 172.21.15.46:0/1813684132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec45a6a0 0x5630ec45aad0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0478003db0 tx=0x7f0478016c30 comp rx=0 tx=0).stop 2024-05-09T04:52:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.353+0000 7f0496939b80 1 -- 172.21.15.46:0/1813684132 shutdown_connections 2024-05-09T04:52:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.353+0000 7f0496939b80 1 --2- 172.21.15.46:0/1813684132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec45a6a0 0x5630ec45aad0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.353+0000 7f0496939b80 1 --2- 172.21.15.46:0/1813684132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec3c7160 0x5630ec4604f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.353+0000 7f0496939b80 1 --2- 172.21.15.46:0/1813684132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec2dd4a0 0x5630ec460a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.353+0000 7f0496939b80 1 -- 172.21.15.46:0/1813684132 >> 172.21.15.46:0/1813684132 conn(0x5630ec33d6a0 msgr2=0x5630ec455400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:43.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.354+0000 7f0496939b80 1 -- 172.21.15.46:0/1813684132 shutdown_connections 2024-05-09T04:52:43.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.354+0000 7f0496939b80 1 -- 172.21.15.46:0/1813684132 wait complete. 2024-05-09T04:52:43.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.354+0000 7f0496939b80 1 -- start start 2024-05-09T04:52:43.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.354+0000 7f0496939b80 1 Processor -- start 2024-05-09T04:52:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.355+0000 7f0496939b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec2dd4a0 0x5630ec464260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.355+0000 7f0496939b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec3c7160 0x5630ec4647a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.355+0000 7f0496939b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 0x5630ec464ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.355+0000 7f0496939b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5630ec1f8ba0 con 0x5630ec2dd4a0 2024-05-09T04:52:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.355+0000 7f0496939b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5630ec337150 con 0x5630ec45a6a0 2024-05-09T04:52:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.355+0000 7f0496939b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5630ec4671d0 con 0x5630ec3c7160 2024-05-09T04:52:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.355+0000 7f0496939b80 10 client.0 _ll_register_callbacks cb 0x5630ec1ee6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.355+0000 7f0485cbf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec2dd4a0 0x5630ec464260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.355+0000 7f04864c0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 0x5630ec464ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.355+0000 7f0485cbf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec2dd4a0 0x5630ec464260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47360/0 (socket says 172.21.15.46:47360) 2024-05-09T04:52:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.355+0000 7f0485cbf700 1 -- 172.21.15.46:0/3549926216 learned_addr learned my addr 172.21.15.46:0/3549926216 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.355+0000 7f04854be700 1 --2- 172.21.15.46:0/3549926216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec3c7160 0x5630ec4647a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.356+0000 7f0485cbf700 1 -- 172.21.15.46:0/3549926216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec3c7160 msgr2=0x5630ec4647a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.356+0000 7f0485cbf700 1 --2- 172.21.15.46:0/3549926216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec3c7160 0x5630ec4647a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.356+0000 7f0485cbf700 1 -- 172.21.15.46:0/3549926216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 msgr2=0x5630ec464ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.356+0000 7f0485cbf700 1 --2- 172.21.15.46:0/3549926216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 0x5630ec464ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.356+0000 7f0485cbf700 1 -- 172.21.15.46:0/3549926216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630ec1f8490 con 0x5630ec2dd4a0 2024-05-09T04:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.356+0000 7f04854be700 1 --2- 172.21.15.46:0/3549926216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec3c7160 0x5630ec4647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.356+0000 7f0485cbf700 1 --2- 172.21.15.46:0/3549926216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec2dd4a0 0x5630ec464260 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f047c006010 tx=0x7f047c01a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f556699c244140 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.356+0000 7f04697fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.356+0000 7f04697fa700 1 -- 172.21.15.46:0/3549926216 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c00f460 con 0x5630ec2dd4a0 2024-05-09T04:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.356+0000 7f04697fa700 1 -- 172.21.15.46:0/3549926216 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f047c00f600 con 0x5630ec2dd4a0 2024-05-09T04:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.356+0000 7f0496939b80 1 -- 172.21.15.46:0/3549926216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0478009e10 con 0x5630ec2dd4a0 2024-05-09T04:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.357+0000 7f04697fa700 1 -- 172.21.15.46:0/3549926216 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f047c03d050 con 0x5630ec2dd4a0 2024-05-09T04:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.357+0000 7f0496939b80 10 client.15693 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.357+0000 7f0496939b80 1 -- 172.21.15.46:0/3549926216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630ec471c40 con 0x5630ec2dd4a0 2024-05-09T04:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.357+0000 7f04697fa700 1 -- 172.21.15.46:0/3549926216 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0478009e10 con 0x5630ec2dd4a0 2024-05-09T04:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.357+0000 7f04697fa700 1 -- 172.21.15.46:0/3549926216 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f047c022070 con 0x5630ec2dd4a0 2024-05-09T04:52:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.358+0000 7f0496939b80 10 client.15693 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.358+0000 7f0496939b80 10 client.15693 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:52:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.358+0000 7f0496939b80 20 client.15693 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.358+0000 7f0496939b80 1 --2- 172.21.15.46:0/3549926216 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5630ec474320 0x5630ec476780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.358+0000 7f0496939b80 4 client.15693 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.358+0000 7f0496939b80 1 -- 172.21.15.46:0/3549926216 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630ec38a9d0 con 0x5630ec474320 2024-05-09T04:52:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.358+0000 7f04854be700 1 --2- 172.21.15.46:0/3549926216 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5630ec474320 0x5630ec476780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.359+0000 7f04697fa700 10 client.15693 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:52:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.359+0000 7f04854be700 1 --2- 172.21.15.46:0/3549926216 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5630ec474320 0x5630ec476780 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=5bfb3c60a6867bfd server_cookie=bb4cd0b3f3033d58 in_seq=0 out_seq=0 2024-05-09T04:52:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.359+0000 7f04697fa700 1 -- 172.21.15.46:0/3549926216 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5630ec38a9d0 con 0x5630ec474320 2024-05-09T04:52:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.359+0000 7f04697fa700 10 client.15693 handle_command_reply: tid=0 2024-05-09T04:52:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.360+0000 7f0496939b80 1 client.15693 shutdown 2024-05-09T04:52:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.360+0000 7f0496939b80 1 -- 172.21.15.46:0/3549926216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec2dd4a0 msgr2=0x5630ec464260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.360+0000 7f0496939b80 1 --2- 172.21.15.46:0/3549926216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec2dd4a0 0x5630ec464260 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f047c006010 tx=0x7f047c01a050 comp rx=0 tx=0).stop 2024-05-09T04:52:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.360+0000 7f0496939b80 1 -- 172.21.15.46:0/3549926216 shutdown_connections 2024-05-09T04:52:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.360+0000 7f0496939b80 1 --2- 172.21.15.46:0/3549926216 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5630ec474320 0x5630ec476780 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.360+0000 7f0496939b80 1 --2- 172.21.15.46:0/3549926216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 0x5630ec464ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.360+0000 7f04697fa700 0 client.15693 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:52:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.360+0000 7f0496939b80 1 --2- 172.21.15.46:0/3549926216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec2dd4a0 0x5630ec464260 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.360+0000 7f0496939b80 1 --2- 172.21.15.46:0/3549926216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec3c7160 0x5630ec4647a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.360+0000 7f0496939b80 1 -- 172.21.15.46:0/3549926216 >> 172.21.15.46:0/3549926216 conn(0x5630ec33d6a0 msgr2=0x5630ec4551c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.360+0000 7f0496939b80 1 -- 172.21.15.46:0/3549926216 shutdown_connections 2024-05-09T04:52:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.360+0000 7f0496939b80 1 -- 172.21.15.46:0/3549926216 wait complete. 2024-05-09T04:52:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.361+0000 7f0496939b80 20 client.15693 trim_cache size 0 max 16384 2024-05-09T04:52:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.376+0000 7f0496939b80 1 Processor -- start 2024-05-09T04:52:43.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.376+0000 7f0496939b80 1 -- start start 2024-05-09T04:52:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.376+0000 7f0496939b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec2dd4a0 0x5630ec45f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f0496939b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec3c7160 0x5630ec463580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f0496939b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 0x5630ec459da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f0496939b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f04801b7d50 con 0x5630ec3c7160 2024-05-09T04:52:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f0496939b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f04801b55e0 con 0x5630ec45a6a0 2024-05-09T04:52:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f0496939b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04801b3450 con 0x5630ec2dd4a0 2024-05-09T04:52:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f04854be700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec3c7160 0x5630ec463580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f04854be700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec3c7160 0x5630ec463580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47376/0 (socket says 172.21.15.46:47376) 2024-05-09T04:52:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f04854be700 1 -- 172.21.15.46:0/1550802458 learned_addr learned my addr 172.21.15.46:0/1550802458 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f0485cbf700 1 --2- 172.21.15.46:0/1550802458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec2dd4a0 0x5630ec45f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f04864c0700 1 --2- 172.21.15.46:0/1550802458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 0x5630ec459da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f04854be700 1 -- 172.21.15.46:0/1550802458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec2dd4a0 msgr2=0x5630ec45f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f04854be700 1 --2- 172.21.15.46:0/1550802458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec2dd4a0 0x5630ec45f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f04854be700 1 -- 172.21.15.46:0/1550802458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 msgr2=0x5630ec459da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f04854be700 1 --2- 172.21.15.46:0/1550802458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 0x5630ec459da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f04854be700 1 -- 172.21.15.46:0/1550802458 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630ec44ef60 con 0x5630ec3c7160 2024-05-09T04:52:43.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.377+0000 7f04854be700 1 --2- 172.21.15.46:0/1550802458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec3c7160 0x5630ec463580 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0470007e80 tx=0x7f0470013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e824e717fd85da server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:43.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.378+0000 7f046b7fe700 1 -- 172.21.15.46:0/1550802458 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04700ad970 con 0x5630ec3c7160 2024-05-09T04:52:43.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.378+0000 7f046b7fe700 1 -- 172.21.15.46:0/1550802458 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04700e1040 con 0x5630ec3c7160 2024-05-09T04:52:43.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.378+0000 7f046b7fe700 1 -- 172.21.15.46:0/1550802458 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04700a9820 con 0x5630ec3c7160 2024-05-09T04:52:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.379+0000 7f0496939b80 1 -- 172.21.15.46:0/1550802458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec3c7160 msgr2=0x5630ec463580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.379+0000 7f0496939b80 1 --2- 172.21.15.46:0/1550802458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec3c7160 0x5630ec463580 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0470007e80 tx=0x7f0470013040 comp rx=0 tx=0).stop 2024-05-09T04:52:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.379+0000 7f0496939b80 1 -- 172.21.15.46:0/1550802458 shutdown_connections 2024-05-09T04:52:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.379+0000 7f0496939b80 1 --2- 172.21.15.46:0/1550802458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 0x5630ec459da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.379+0000 7f0496939b80 1 --2- 172.21.15.46:0/1550802458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec2dd4a0 0x5630ec45f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.379+0000 7f0496939b80 1 --2- 172.21.15.46:0/1550802458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec3c7160 0x5630ec463580 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.379+0000 7f0496939b80 1 -- 172.21.15.46:0/1550802458 >> 172.21.15.46:0/1550802458 conn(0x5630ec33d6a0 msgr2=0x5630ec459690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.379+0000 7f0496939b80 1 -- 172.21.15.46:0/1550802458 shutdown_connections 2024-05-09T04:52:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.379+0000 7f0496939b80 1 -- 172.21.15.46:0/1550802458 wait complete. 2024-05-09T04:52:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.380+0000 7f0496939b80 1 -- start start 2024-05-09T04:52:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.380+0000 7f0496939b80 1 Processor -- start 2024-05-09T04:52:43.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.381+0000 7f0496939b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec2dd4a0 0x5630ec46c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.381+0000 7f0485cbf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec2dd4a0 0x5630ec46c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.381+0000 7f0485cbf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec2dd4a0 0x5630ec46c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47382/0 (socket says 172.21.15.46:47382) 2024-05-09T04:52:43.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.381+0000 7f0485cbf700 1 -- 172.21.15.46:0/2127395192 learned_addr learned my addr 172.21.15.46:0/2127395192 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:43.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.381+0000 7f0496939b80 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 0x5630ec46c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.381+0000 7f0496939b80 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec474320 0x5630ec47af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.381+0000 7f0496939b80 1 -- 172.21.15.46:0/2127395192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f04801b3aa0 con 0x5630ec2dd4a0 2024-05-09T04:52:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.381+0000 7f0496939b80 1 -- 172.21.15.46:0/2127395192 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f04801b4f10 con 0x5630ec45a6a0 2024-05-09T04:52:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.381+0000 7f0496939b80 1 -- 172.21.15.46:0/2127395192 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04801b4170 con 0x5630ec474320 2024-05-09T04:52:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.381+0000 7f04854be700 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 0x5630ec46c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.381+0000 7f04864c0700 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec474320 0x5630ec47af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.382+0000 7f0496939b80 10 client.0 _ll_register_callbacks cb 0x5630ec1ee6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.382+0000 7f04854be700 1 -- 172.21.15.46:0/2127395192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec474320 msgr2=0x5630ec47af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.382+0000 7f04854be700 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec474320 0x5630ec47af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.382+0000 7f04854be700 1 -- 172.21.15.46:0/2127395192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec2dd4a0 msgr2=0x5630ec46c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.382+0000 7f04854be700 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec2dd4a0 0x5630ec46c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.382+0000 7f04854be700 1 -- 172.21.15.46:0/2127395192 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630ec47b4f0 con 0x5630ec45a6a0 2024-05-09T04:52:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.382+0000 7f0485cbf700 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec2dd4a0 0x5630ec46c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.382+0000 7f04854be700 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 0x5630ec46c890 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5630ec460e10 tx=0x7f0470004f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1a6fe214e546432 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.382+0000 7f04697fa700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:52:43.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.382+0000 7f0496939b80 1 -- 172.21.15.46:0/2127395192 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f04700e1040 con 0x5630ec45a6a0 2024-05-09T04:52:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.382+0000 7f04697fa700 1 -- 172.21.15.46:0/2127395192 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04700b2420 con 0x5630ec45a6a0 2024-05-09T04:52:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.382+0000 7f04697fa700 1 -- 172.21.15.46:0/2127395192 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f04700a9820 con 0x5630ec45a6a0 2024-05-09T04:52:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.383+0000 7f04697fa700 1 -- 172.21.15.46:0/2127395192 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f04700a9a20 con 0x5630ec45a6a0 2024-05-09T04:52:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.383+0000 7f0496939b80 10 client.25442 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.383+0000 7f0496939b80 1 -- 172.21.15.46:0/2127395192 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({fsmap=25}) -- 0x5630ec4de1d0 con 0x5630ec45a6a0 2024-05-09T04:52:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.383+0000 7f04697fa700 1 -- 172.21.15.46:0/2127395192 <== mon.2 v2:172.21.15.72:3300/0 4 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f04700a3ce0 con 0x5630ec45a6a0 2024-05-09T04:52:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.383+0000 7f0496939b80 10 client.25442 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.383+0000 7f0496939b80 10 client.25442 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:52:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.383+0000 7f0496939b80 20 client.25442 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.383+0000 7f0496939b80 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5630ec4de5d0 0x5630ec4e09f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.383+0000 7f0496939b80 4 client.25442 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:52:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.383+0000 7f0496939b80 1 -- 172.21.15.46:0/2127395192 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5630ec384a70 con 0x5630ec4de5d0 2024-05-09T04:52:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.383+0000 7f0485cbf700 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5630ec4de5d0 0x5630ec4e09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.384+0000 7f04697fa700 10 client.25442 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:52:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.384+0000 7f0485cbf700 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5630ec4de5d0 0x5630ec4e09f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=dfec00c8fea28b47 server_cookie=6166ed3f53297d5d in_seq=0 out_seq=0 2024-05-09T04:52:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.384+0000 7f04697fa700 1 -- 172.21.15.46:0/2127395192 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5630ec384a70 con 0x5630ec4de5d0 2024-05-09T04:52:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.384+0000 7f04697fa700 10 client.25442 handle_command_reply: tid=0 2024-05-09T04:52:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.384+0000 7f0496939b80 1 client.25442 shutdown 2024-05-09T04:52:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.384+0000 7f0496939b80 1 -- 172.21.15.46:0/2127395192 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 msgr2=0x5630ec46c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.384+0000 7f0496939b80 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 0x5630ec46c890 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x5630ec460e10 tx=0x7f0470004f10 comp rx=0 tx=0).stop 2024-05-09T04:52:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.384+0000 7f0496939b80 1 -- 172.21.15.46:0/2127395192 shutdown_connections 2024-05-09T04:52:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.384+0000 7f0496939b80 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5630ec4de5d0 0x5630ec4e09f0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.384+0000 7f0496939b80 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630ec474320 0x5630ec47af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.384+0000 7f04697fa700 0 client.25442 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:52:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.384+0000 7f0496939b80 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630ec2dd4a0 0x5630ec46c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.384+0000 7f0496939b80 1 --2- 172.21.15.46:0/2127395192 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630ec45a6a0 0x5630ec46c890 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.385+0000 7f0496939b80 1 -- 172.21.15.46:0/2127395192 >> 172.21.15.46:0/2127395192 conn(0x5630ec33d6a0 msgr2=0x5630ec4632e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.385+0000 7f0496939b80 1 -- 172.21.15.46:0/2127395192 shutdown_connections 2024-05-09T04:52:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.385+0000 7f0496939b80 1 -- 172.21.15.46:0/2127395192 wait complete. 2024-05-09T04:52:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.385+0000 7f0496939b80 20 client.25442 trim_cache size 0 max 16384 2024-05-09T04:52:43.388 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:52:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.386+0000 7f0487f23700 1 -- 172.21.15.46:0/1986835640 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f046c0b2b40 msgr2=0x7f046c0b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.386+0000 7f0487f23700 1 --2- 172.21.15.46:0/1986835640 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f046c0b2b40 0x7f046c0b5050 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f048006fd20 tx=0x7f047c005960 comp rx=0 tx=0).stop 2024-05-09T04:52:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.386+0000 7f0487f23700 1 -- 172.21.15.46:0/1986835640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0480105dd0 msgr2=0x7f048006e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.386+0000 7f0487f23700 1 --2- 172.21.15.46:0/1986835640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0480105dd0 0x7f048006e4d0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f047000d980 tx=0x7f047000dc90 comp rx=0 tx=0).stop 2024-05-09T04:52:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.386+0000 7f0487f23700 1 -- 172.21.15.46:0/1986835640 shutdown_connections 2024-05-09T04:52:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.386+0000 7f0487f23700 1 --2- 172.21.15.46:0/1986835640 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f046c0b2b40 0x7f046c0b5050 secure :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f048006fd20 tx=0x7f047c005960 comp rx=0 tx=0).stop 2024-05-09T04:52:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.386+0000 7f0487f23700 1 --2- 172.21.15.46:0/1986835640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f048010e8f0 0x7f048006eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.386+0000 7f0487f23700 1 --2- 172.21.15.46:0/1986835640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0480105480 0x7f04800753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.386+0000 7f0487f23700 1 --2- 172.21.15.46:0/1986835640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0480105dd0 0x7f048006e4d0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:43.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.386+0000 7f0487f23700 1 -- 172.21.15.46:0/1986835640 >> 172.21.15.46:0/1986835640 conn(0x7f04800fef30 msgr2=0x7f04800ff680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:43.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.387+0000 7f0487f23700 1 -- 172.21.15.46:0/1986835640 shutdown_connections 2024-05-09T04:52:43.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:43.387+0000 7f0487f23700 1 -- 172.21.15.46:0/1986835640 wait complete. 2024-05-09T04:52:44.130 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:52:44.414 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:44.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:44 smithi046 ceph-mon[30136]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-09T04:52:45.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:44 smithi110 ceph-mon[34904]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-09T04:52:45.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:44 smithi072 ceph-mon[34088]: pgmap v52: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-09T04:52:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.577+0000 7fb032ce6700 1 -- 172.21.15.46:0/3196347266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c109870 msgr2=0x7fb02c1111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.577+0000 7fb032ce6700 1 --2- 172.21.15.46:0/3196347266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c109870 0x7fb02c1111d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fb02400b0d0 tx=0x7fb02400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.577+0000 7fb032ce6700 1 -- 172.21.15.46:0/3196347266 shutdown_connections 2024-05-09T04:52:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.577+0000 7fb032ce6700 1 --2- 172.21.15.46:0/3196347266 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c109870 0x7fb02c1111d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.577+0000 7fb032ce6700 1 --2- 172.21.15.46:0/3196347266 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb02c1045f0 0x7fb02c104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.577+0000 7fb032ce6700 1 --2- 172.21.15.46:0/3196347266 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb02c108f20 0x7fb02c109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.577+0000 7fb032ce6700 1 -- 172.21.15.46:0/3196347266 >> 172.21.15.46:0/3196347266 conn(0x7fb02c100040 msgr2=0x7fb02c102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:46.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.577+0000 7fb032ce6700 1 -- 172.21.15.46:0/3196347266 shutdown_connections 2024-05-09T04:52:46.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.577+0000 7fb032ce6700 1 -- 172.21.15.46:0/3196347266 wait complete. 2024-05-09T04:52:46.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb032ce6700 1 Processor -- start 2024-05-09T04:52:46.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb032ce6700 1 -- start start 2024-05-09T04:52:46.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb032ce6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c1045f0 0x7fb02c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb032ce6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb02c108f20 0x7fb02c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb032ce6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb02c109870 0x7fb02c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb032ce6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c1144a0 con 0x7fb02c1045f0 2024-05-09T04:52:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb032ce6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c114320 con 0x7fb02c109870 2024-05-09T04:52:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb032ce6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c114620 con 0x7fb02c108f20 2024-05-09T04:52:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb030a82700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c1045f0 0x7fb02c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb030a82700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c1045f0 0x7fb02c19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56368/0 (socket says 172.21.15.46:56368) 2024-05-09T04:52:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb030a82700 1 -- 172.21.15.46:0/3090365866 learned_addr learned my addr 172.21.15.46:0/3090365866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb031283700 1 --2- 172.21.15.46:0/3090365866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb02c109870 0x7fb02c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb02bfff700 1 --2- 172.21.15.46:0/3090365866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb02c108f20 0x7fb02c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb030a82700 1 -- 172.21.15.46:0/3090365866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb02c108f20 msgr2=0x7fb02c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb030a82700 1 --2- 172.21.15.46:0/3090365866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb02c108f20 0x7fb02c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb030a82700 1 -- 172.21.15.46:0/3090365866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb02c109870 msgr2=0x7fb02c19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:46.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.578+0000 7fb030a82700 1 --2- 172.21.15.46:0/3090365866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb02c109870 0x7fb02c19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.579+0000 7fb030a82700 1 -- 172.21.15.46:0/3090365866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb02c19fc80 con 0x7fb02c1045f0 2024-05-09T04:52:46.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.579+0000 7fb030a82700 1 --2- 172.21.15.46:0/3090365866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c1045f0 0x7fb02c19adb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb02000e9a0 tx=0x7fb02000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:46.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.579+0000 7fb029ffb700 1 -- 172.21.15.46:0/3090365866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb02000ca10 con 0x7fb02c1045f0 2024-05-09T04:52:46.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.579+0000 7fb029ffb700 1 -- 172.21.15.46:0/3090365866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb02000cbb0 con 0x7fb02c1045f0 2024-05-09T04:52:46.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.579+0000 7fb032ce6700 1 -- 172.21.15.46:0/3090365866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb02c19ff70 con 0x7fb02c1045f0 2024-05-09T04:52:46.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.579+0000 7fb029ffb700 1 -- 172.21.15.46:0/3090365866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb020018680 con 0x7fb02c1045f0 2024-05-09T04:52:46.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.579+0000 7fb032ce6700 1 -- 172.21.15.46:0/3090365866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb02c1a7890 con 0x7fb02c1045f0 2024-05-09T04:52:46.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.582+0000 7fb029ffb700 1 -- 172.21.15.46:0/3090365866 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb020010430 con 0x7fb02c1045f0 2024-05-09T04:52:46.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.583+0000 7fb029ffb700 1 --2- 172.21.15.46:0/3090365866 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb01c0b29f0 0x7fb01c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.583+0000 7fb02bfff700 1 --2- 172.21.15.46:0/3090365866 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb01c0b29f0 0x7fb01c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.583+0000 7fb029ffb700 1 -- 172.21.15.46:0/3090365866 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fb0200e2790 con 0x7fb02c1045f0 2024-05-09T04:52:46.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.583+0000 7fb02bfff700 1 --2- 172.21.15.46:0/3090365866 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb01c0b29f0 0x7fb01c0b4f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb02c103a40 tx=0x7fb018009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.584+0000 7fb0416fcb80 1 Processor -- start 2024-05-09T04:52:46.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.584+0000 7fb0416fcb80 1 -- start start 2024-05-09T04:52:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb0416fcb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84ed2d00 0x557e84fdb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb0416fcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84e584a0 0x557e84fdb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb0416fcb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84fd5400 0x557e84fd5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb0416fcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557e84ef80a0 con 0x557e84e584a0 2024-05-09T04:52:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb0416fcb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557e84e9d9a0 con 0x557e84fd5400 2024-05-09T04:52:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb0416fcb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557e84ef4060 con 0x557e84ed2d00 2024-05-09T04:52:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb02bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84e584a0 0x557e84fdb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb02bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84e584a0 0x557e84fdb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56378/0 (socket says 172.21.15.46:56378) 2024-05-09T04:52:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb02bfff700 1 -- 172.21.15.46:0/1864473578 learned_addr learned my addr 172.21.15.46:0/1864473578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb030a82700 1 --2- 172.21.15.46:0/1864473578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84ed2d00 0x557e84fdb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb031283700 1 --2- 172.21.15.46:0/1864473578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84fd5400 0x557e84fd5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb02bfff700 1 -- 172.21.15.46:0/1864473578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84ed2d00 msgr2=0x557e84fdb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb02bfff700 1 --2- 172.21.15.46:0/1864473578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84ed2d00 0x557e84fdb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb02bfff700 1 -- 172.21.15.46:0/1864473578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84fd5400 msgr2=0x557e84fd5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb02bfff700 1 --2- 172.21.15.46:0/1864473578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84fd5400 0x557e84fd5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb02bfff700 1 -- 172.21.15.46:0/1864473578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e84e8e500 con 0x557e84e584a0 2024-05-09T04:52:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb031283700 1 --2- 172.21.15.46:0/1864473578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84fd5400 0x557e84fd5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:52:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.585+0000 7fb030a82700 1 --2- 172.21.15.46:0/1864473578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84ed2d00 0x557e84fdb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.586+0000 7fb02bfff700 1 --2- 172.21.15.46:0/1864473578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84e584a0 0x557e84fdb780 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb018011d10 tx=0x7fb018014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515eeb6717e71ad6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.586+0000 7fb016ffd700 1 -- 172.21.15.46:0/1864473578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018011d40 con 0x557e84e584a0 2024-05-09T04:52:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.586+0000 7fb016ffd700 1 -- 172.21.15.46:0/1864473578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb018004020 con 0x557e84e584a0 2024-05-09T04:52:46.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.586+0000 7fb016ffd700 1 -- 172.21.15.46:0/1864473578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018003bc0 con 0x557e84e584a0 2024-05-09T04:52:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.587+0000 7fb0416fcb80 1 -- 172.21.15.46:0/1864473578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84e584a0 msgr2=0x557e84fdb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.587+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/1864473578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84e584a0 0x557e84fdb780 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fb018011d10 tx=0x7fb018014000 comp rx=0 tx=0).stop 2024-05-09T04:52:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.587+0000 7fb0416fcb80 1 -- 172.21.15.46:0/1864473578 shutdown_connections 2024-05-09T04:52:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.587+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/1864473578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84fd5400 0x557e84fd5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.587+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/1864473578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84ed2d00 0x557e84fdb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.587+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/1864473578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84e584a0 0x557e84fdb780 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.587+0000 7fb0416fcb80 1 -- 172.21.15.46:0/1864473578 >> 172.21.15.46:0/1864473578 conn(0x557e84e5bf70 msgr2=0x557e84fd0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.587+0000 7fb0416fcb80 1 -- 172.21.15.46:0/1864473578 shutdown_connections 2024-05-09T04:52:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.588+0000 7fb0416fcb80 1 -- 172.21.15.46:0/1864473578 wait complete. 2024-05-09T04:52:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.588+0000 7fb0416fcb80 1 -- start start 2024-05-09T04:52:46.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.588+0000 7fb0416fcb80 1 Processor -- start 2024-05-09T04:52:46.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.589+0000 7fb0416fcb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84e584a0 0x557e84fdf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.589+0000 7fb0416fcb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84ed2d00 0x557e84fdf650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.589+0000 7fb0416fcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84fd5400 0x557e84fdfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.589+0000 7fb0416fcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557e84fd6160 con 0x557e84fd5400 2024-05-09T04:52:46.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.589+0000 7fb0416fcb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557e84e92b20 con 0x557e84ed2d00 2024-05-09T04:52:46.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.589+0000 7fb0416fcb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557e84fe2020 con 0x557e84e584a0 2024-05-09T04:52:46.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.589+0000 7fb0416fcb80 10 client.0 _ll_register_callbacks cb 0x557e84d696c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:46.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.589+0000 7fb031283700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84fd5400 0x557e84fdfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.589+0000 7fb02bfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84ed2d00 0x557e84fdf650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.589+0000 7fb031283700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84fd5400 0x557e84fdfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56394/0 (socket says 172.21.15.46:56394) 2024-05-09T04:52:46.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.589+0000 7fb031283700 1 -- 172.21.15.46:0/2780425640 learned_addr learned my addr 172.21.15.46:0/2780425640 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:46.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.589+0000 7fb030a82700 1 --2- 172.21.15.46:0/2780425640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84e584a0 0x557e84fdf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.590+0000 7fb02bfff700 1 -- 172.21.15.46:0/2780425640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84e584a0 msgr2=0x557e84fdf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:46.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.590+0000 7fb02bfff700 1 --2- 172.21.15.46:0/2780425640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84e584a0 0x557e84fdf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.590+0000 7fb02bfff700 1 -- 172.21.15.46:0/2780425640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84fd5400 msgr2=0x557e84fdfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.590+0000 7fb02bfff700 1 --2- 172.21.15.46:0/2780425640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84fd5400 0x557e84fdfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.590+0000 7fb02bfff700 1 -- 172.21.15.46:0/2780425640 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e84d73490 con 0x557e84ed2d00 2024-05-09T04:52:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.590+0000 7fb031283700 1 --2- 172.21.15.46:0/2780425640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84fd5400 0x557e84fdfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.590+0000 7fb02bfff700 1 --2- 172.21.15.46:0/2780425640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84ed2d00 0x557e84fdf650 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb018005950 tx=0x7fb018005550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38e6730d5f7de811 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.590+0000 7fb00bfff700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:52:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.590+0000 7fb00bfff700 1 -- 172.21.15.46:0/2780425640 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb018003a60 con 0x557e84ed2d00 2024-05-09T04:52:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.590+0000 7fb00bfff700 1 -- 172.21.15.46:0/2780425640 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fb018003c00 con 0x557e84ed2d00 2024-05-09T04:52:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.590+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2780425640 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb018004020 con 0x557e84ed2d00 2024-05-09T04:52:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.590+0000 7fb00bfff700 1 -- 172.21.15.46:0/2780425640 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb018003e00 con 0x557e84ed2d00 2024-05-09T04:52:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.590+0000 7fb0416fcb80 10 client.25454 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.590+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2780425640 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557e84feca90 con 0x557e84ed2d00 2024-05-09T04:52:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.591+0000 7fb00bfff700 1 -- 172.21.15.46:0/2780425640 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb018004020 con 0x557e84ed2d00 2024-05-09T04:52:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.591+0000 7fb00bfff700 1 -- 172.21.15.46:0/2780425640 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fb01802e070 con 0x557e84ed2d00 2024-05-09T04:52:46.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.591+0000 7fb0416fcb80 10 client.25454 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:46.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.591+0000 7fb0416fcb80 10 client.25454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:46.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.591+0000 7fb0416fcb80 20 client.25454 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:46.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.591+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2780425640 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e84feef60 0x557e84ff1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.591+0000 7fb0416fcb80 4 client.25454 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:46.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.591+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2780425640 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e84f05a80 con 0x557e84feef60 2024-05-09T04:52:46.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.591+0000 7fb030a82700 1 --2- 172.21.15.46:0/2780425640 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e84feef60 0x557e84ff1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.592+0000 7fb00bfff700 10 client.25454 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:46.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.592+0000 7fb030a82700 1 --2- 172.21.15.46:0/2780425640 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e84feef60 0x557e84ff1380 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d89ab129c57a373 server_cookie=43fc08e59f6990e in_seq=0 out_seq=0 2024-05-09T04:52:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.592+0000 7fb00bfff700 1 -- 172.21.15.46:0/2780425640 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e84f05a80 con 0x557e84feef60 2024-05-09T04:52:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.592+0000 7fb00bfff700 10 client.25454 handle_command_reply: tid=0 2024-05-09T04:52:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.593+0000 7fb0416fcb80 1 client.25454 shutdown 2024-05-09T04:52:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.593+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2780425640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84ed2d00 msgr2=0x557e84fdf650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:46.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.593+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2780425640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84ed2d00 0x557e84fdf650 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb018005950 tx=0x7fb018005550 comp rx=0 tx=0).stop 2024-05-09T04:52:46.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.593+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2780425640 shutdown_connections 2024-05-09T04:52:46.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.593+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2780425640 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84fd5400 0x557e84fdfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.593+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2780425640 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84e584a0 0x557e84fdf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.593+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2780425640 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e84feef60 0x557e84ff1380 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.593+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2780425640 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84ed2d00 0x557e84fdf650 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.593+0000 7fb00bfff700 0 client.25454 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:46.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.593+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2780425640 >> 172.21.15.46:0/2780425640 conn(0x557e84e5bf70 msgr2=0x557e84fc9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:46.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.593+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2780425640 shutdown_connections 2024-05-09T04:52:46.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.593+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2780425640 wait complete. 2024-05-09T04:52:46.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.593+0000 7fb0416fcb80 20 client.25454 trim_cache size 0 max 16384 2024-05-09T04:52:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.608+0000 7fb0416fcb80 1 Processor -- start 2024-05-09T04:52:46.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.608+0000 7fb0416fcb80 1 -- start start 2024-05-09T04:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb0416fcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84e584a0 0x557e84fdccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb0416fcb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84ed2d00 0x557e84fca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb0416fcb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84fd5400 0x557e84ff43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb0416fcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c1b1810 con 0x557e84e584a0 2024-05-09T04:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb0416fcb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c1af0a0 con 0x557e84ed2d00 2024-05-09T04:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb0416fcb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c1acef0 con 0x557e84fd5400 2024-05-09T04:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb030a82700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84e584a0 0x557e84fdccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb030a82700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84e584a0 0x557e84fdccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56402/0 (socket says 172.21.15.46:56402) 2024-05-09T04:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb030a82700 1 -- 172.21.15.46:0/2758905528 learned_addr learned my addr 172.21.15.46:0/2758905528 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb02bfff700 1 --2- 172.21.15.46:0/2758905528 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84ed2d00 0x557e84fca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb031283700 1 --2- 172.21.15.46:0/2758905528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84fd5400 0x557e84ff43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb030a82700 1 -- 172.21.15.46:0/2758905528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84fd5400 msgr2=0x557e84ff43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:46.610 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb030a82700 1 --2- 172.21.15.46:0/2758905528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84fd5400 0x557e84ff43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb030a82700 1 -- 172.21.15.46:0/2758905528 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84ed2d00 msgr2=0x557e84fca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb030a82700 1 --2- 172.21.15.46:0/2758905528 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84ed2d00 0x557e84fca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb030a82700 1 -- 172.21.15.46:0/2758905528 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e84fcc9d0 con 0x557e84e584a0 2024-05-09T04:52:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb031283700 1 --2- 172.21.15.46:0/2758905528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84fd5400 0x557e84ff43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:52:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.609+0000 7fb030a82700 1 --2- 172.21.15.46:0/2758905528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84e584a0 0x557e84fdccf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb02000e970 tx=0x7fb020010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75b7857ef05e54b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.610+0000 7fb0167fc700 1 -- 172.21.15.46:0/2758905528 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0200a5040 con 0x557e84e584a0 2024-05-09T04:52:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.610+0000 7fb0167fc700 1 -- 172.21.15.46:0/2758905528 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0200b2040 con 0x557e84e584a0 2024-05-09T04:52:46.611 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.610+0000 7fb0167fc700 1 -- 172.21.15.46:0/2758905528 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0200b2420 con 0x557e84e584a0 2024-05-09T04:52:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.611+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2758905528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84e584a0 msgr2=0x557e84fdccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.611+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2758905528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84e584a0 0x557e84fdccf0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fb02000e970 tx=0x7fb020010c10 comp rx=0 tx=0).stop 2024-05-09T04:52:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.611+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2758905528 shutdown_connections 2024-05-09T04:52:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.611+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2758905528 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84fd5400 0x557e84ff43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.611+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2758905528 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84e584a0 0x557e84fdccf0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.611+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2758905528 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84ed2d00 0x557e84fca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.611+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2758905528 >> 172.21.15.46:0/2758905528 conn(0x557e84e5bf70 msgr2=0x557e84fc9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.611+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2758905528 shutdown_connections 2024-05-09T04:52:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.611+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2758905528 wait complete. 2024-05-09T04:52:46.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.611+0000 7fb0416fcb80 1 -- start start 2024-05-09T04:52:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.611+0000 7fb0416fcb80 1 Processor -- start 2024-05-09T04:52:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb0416fcb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84e584a0 0x557e84ff32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb0416fcb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84ed2d00 0x557e84ff37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb0416fcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84fd5400 0x557e850574b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb0416fcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c1ad560 con 0x557e84fd5400 2024-05-09T04:52:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb0416fcb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c1ae9d0 con 0x557e84e584a0 2024-05-09T04:52:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb0416fcb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb02c1adc30 con 0x557e84ed2d00 2024-05-09T04:52:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb0416fcb80 10 client.0 _ll_register_callbacks cb 0x557e84d696c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb031283700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84fd5400 0x557e850574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb031283700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84fd5400 0x557e850574b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56404/0 (socket says 172.21.15.46:56404) 2024-05-09T04:52:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb031283700 1 -- 172.21.15.46:0/2412230372 learned_addr learned my addr 172.21.15.46:0/2412230372 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb030a82700 1 --2- 172.21.15.46:0/2412230372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84e584a0 0x557e84ff32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb02bfff700 1 --2- 172.21.15.46:0/2412230372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84ed2d00 0x557e84ff37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb031283700 1 -- 172.21.15.46:0/2412230372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84ed2d00 msgr2=0x557e84ff37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:46.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb031283700 1 --2- 172.21.15.46:0/2412230372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84ed2d00 0x557e84ff37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb031283700 1 -- 172.21.15.46:0/2412230372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84e584a0 msgr2=0x557e84ff32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb031283700 1 --2- 172.21.15.46:0/2412230372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84e584a0 0x557e84ff32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb031283700 1 -- 172.21.15.46:0/2412230372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e84fd6600 con 0x557e84fd5400 2024-05-09T04:52:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb02bfff700 1 --2- 172.21.15.46:0/2412230372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84ed2d00 0x557e84ff37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:52:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.612+0000 7fb031283700 1 --2- 172.21.15.46:0/2412230372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84fd5400 0x557e850574b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb024006130 tx=0x7fb024004c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e9f18f083c21125 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.613+0000 7fb00bfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.613+0000 7fb00bfff700 1 -- 172.21.15.46:0/2412230372 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0240079d0 con 0x557e84fd5400 2024-05-09T04:52:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.613+0000 7fb00bfff700 1 -- 172.21.15.46:0/2412230372 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fb024007b70 con 0x557e84fd5400 2024-05-09T04:52:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.613+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2412230372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0200b2040 con 0x557e84fd5400 2024-05-09T04:52:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.613+0000 7fb00bfff700 1 -- 172.21.15.46:0/2412230372 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb024009450 con 0x557e84fd5400 2024-05-09T04:52:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.613+0000 7fb0416fcb80 10 client.15717 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.613+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2412230372 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557e84fdbaf0 con 0x557e84fd5400 2024-05-09T04:52:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.613+0000 7fb00bfff700 1 -- 172.21.15.46:0/2412230372 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0200b2040 con 0x557e84fd5400 2024-05-09T04:52:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.613+0000 7fb00bfff700 1 -- 172.21.15.46:0/2412230372 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fb024024070 con 0x557e84fd5400 2024-05-09T04:52:46.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.613+0000 7fb0416fcb80 10 client.15717 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.613+0000 7fb0416fcb80 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.613+0000 7fb0416fcb80 20 client.15717 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.613+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2412230372 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e85058d20 0x557e8505b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.614+0000 7fb0416fcb80 4 client.15717 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:52:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.614+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2412230372 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557e84effb20 con 0x557e85058d20 2024-05-09T04:52:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.614+0000 7fb030a82700 1 --2- 172.21.15.46:0/2412230372 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e85058d20 0x557e8505b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.614+0000 7fb00bfff700 10 client.15717 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.614+0000 7fb030a82700 1 --2- 172.21.15.46:0/2412230372 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e85058d20 0x557e8505b140 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3e00728026f4571 server_cookie=487d60151c69d606 in_seq=0 out_seq=0 2024-05-09T04:52:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.614+0000 7fb00bfff700 1 -- 172.21.15.46:0/2412230372 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557e84effb20 con 0x557e85058d20 2024-05-09T04:52:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.614+0000 7fb00bfff700 10 client.15717 handle_command_reply: tid=0 2024-05-09T04:52:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.614+0000 7fb0416fcb80 1 client.15717 shutdown 2024-05-09T04:52:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.615+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2412230372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84fd5400 msgr2=0x557e850574b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:46.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.615+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2412230372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84fd5400 0x557e850574b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fb024006130 tx=0x7fb024004c20 comp rx=0 tx=0).stop 2024-05-09T04:52:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.615+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2412230372 shutdown_connections 2024-05-09T04:52:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.615+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2412230372 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e84fd5400 0x557e850574b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.615+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2412230372 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e85058d20 0x557e8505b140 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.615+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2412230372 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e84e584a0 0x557e84ff32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.615+0000 7fb0416fcb80 1 --2- 172.21.15.46:0/2412230372 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e84ed2d00 0x557e84ff37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.615+0000 7fb00bfff700 0 client.15717 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.615+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2412230372 >> 172.21.15.46:0/2412230372 conn(0x557e84e5bf70 msgr2=0x557e84fd8ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.615+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2412230372 shutdown_connections 2024-05-09T04:52:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.615+0000 7fb0416fcb80 1 -- 172.21.15.46:0/2412230372 wait complete. 2024-05-09T04:52:46.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.615+0000 7fb0416fcb80 20 client.15717 trim_cache size 0 max 16384 2024-05-09T04:52:46.619 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:52:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.616+0000 7fb032ce6700 1 -- 172.21.15.46:0/3090365866 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb01c0b29f0 msgr2=0x7fb01c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.616+0000 7fb032ce6700 1 --2- 172.21.15.46:0/3090365866 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb01c0b29f0 0x7fb01c0b4f00 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb02c103a40 tx=0x7fb018009260 comp rx=0 tx=0).stop 2024-05-09T04:52:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.616+0000 7fb032ce6700 1 -- 172.21.15.46:0/3090365866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c1045f0 msgr2=0x7fb02c19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.616+0000 7fb032ce6700 1 --2- 172.21.15.46:0/3090365866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c1045f0 0x7fb02c19adb0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fb02000e9a0 tx=0x7fb02000ecb0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.616+0000 7fb032ce6700 1 -- 172.21.15.46:0/3090365866 shutdown_connections 2024-05-09T04:52:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.616+0000 7fb032ce6700 1 --2- 172.21.15.46:0/3090365866 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb01c0b29f0 0x7fb01c0b4f00 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.616+0000 7fb032ce6700 1 --2- 172.21.15.46:0/3090365866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb02c109870 0x7fb02c19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.616+0000 7fb032ce6700 1 --2- 172.21.15.46:0/3090365866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb02c1045f0 0x7fb02c19adb0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.616+0000 7fb032ce6700 1 --2- 172.21.15.46:0/3090365866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb02c108f20 0x7fb02c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:46.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.616+0000 7fb032ce6700 1 -- 172.21.15.46:0/3090365866 >> 172.21.15.46:0/3090365866 conn(0x7fb02c100040 msgr2=0x7fb02c108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:46.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.616+0000 7fb032ce6700 1 -- 172.21.15.46:0/3090365866 shutdown_connections 2024-05-09T04:52:46.621 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:46.616+0000 7fb032ce6700 1 -- 172.21.15.46:0/3090365866 wait complete. 2024-05-09T04:52:46.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:46 smithi046 ceph-mon[30136]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-09T04:52:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:46 smithi110 ceph-mon[34904]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-09T04:52:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:46 smithi072 ceph-mon[34088]: pgmap v53: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 853 B/s wr, 6 op/s 2024-05-09T04:52:47.649 INFO:tasks.workunit.client.0.smithi046.stderr:Updating files: 89% (10967/12271) Updating files: 90% (11044/12271) Updating files: 91% (11167/12271) Updating files: 92% (11290/12271) Updating files: 93% (11413/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-09T04:52:48.445 INFO:tasks.workunit.client.0.smithi046.stderr:Note: switching to 'c9da4049f638c0f84bdd9b4e33c42e86046ac554'. 2024-05-09T04:52:48.446 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-09T04:52:48.446 INFO:tasks.workunit.client.0.smithi046.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-09T04:52:48.446 INFO:tasks.workunit.client.0.smithi046.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-09T04:52:48.446 INFO:tasks.workunit.client.0.smithi046.stderr:state without impacting any branches by switching back to a branch. 2024-05-09T04:52:48.446 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-09T04:52:48.446 INFO:tasks.workunit.client.0.smithi046.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-09T04:52:48.446 INFO:tasks.workunit.client.0.smithi046.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-09T04:52:48.446 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-09T04:52:48.446 INFO:tasks.workunit.client.0.smithi046.stderr: git switch -c 2024-05-09T04:52:48.446 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-09T04:52:48.446 INFO:tasks.workunit.client.0.smithi046.stderr:Or undo this operation with: 2024-05-09T04:52:48.446 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-09T04:52:48.446 INFO:tasks.workunit.client.0.smithi046.stderr: git switch - 2024-05-09T04:52:48.447 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-09T04:52:48.447 INFO:tasks.workunit.client.0.smithi046.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-09T04:52:48.447 INFO:tasks.workunit.client.0.smithi046.stderr: 2024-05-09T04:52:48.447 INFO:tasks.workunit.client.0.smithi046.stderr:HEAD is now at c9da4049f63 Merge PR #57081 into wip-pdonnell-testing-20240508.183908-debug 2024-05-09T04:52:48.452 DEBUG:teuthology.orchestra.run.smithi046:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-09T04:52:48.518 INFO:tasks.workunit.client.0.smithi046.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-09T04:52:48.520 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T04:52:48.521 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-09T04:52:48.552 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:52:48.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:52:48.806 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:48.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:48 smithi046 ceph-mon[30136]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-09T04:52:49.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:48 smithi110 ceph-mon[34904]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-09T04:52:49.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:48 smithi072 ceph-mon[34088]: pgmap v54: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 853 B/s wr, 9 op/s 2024-05-09T04:52:50.317 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-09T04:52:50.378 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-09T04:52:50.425 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-09T04:52:50.426 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T04:52:50.426 INFO:tasks.workunit.client.0.smithi046.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-09T04:52:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.459+0000 7f4a21784640 1 -- 172.21.15.46:0/102615445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1c10e650 msgr2=0x7f4a1c110a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.459+0000 7f4a21784640 1 --2- 172.21.15.46:0/102615445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1c10e650 0x7f4a1c110a70 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f4a14009ca0 tx=0x7f4a1402a8c0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.459+0000 7f4a21784640 1 -- 172.21.15.46:0/102615445 shutdown_connections 2024-05-09T04:52:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.459+0000 7f4a21784640 1 --2- 172.21.15.46:0/102615445 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a1c1022b0 0x7f4a1c1026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.459+0000 7f4a21784640 1 --2- 172.21.15.46:0/102615445 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1c10e650 0x7f4a1c110a70 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.459+0000 7f4a21784640 1 --2- 172.21.15.46:0/102615445 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a1c108c00 0x7f4a1c101d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.459+0000 7f4a21784640 1 -- 172.21.15.46:0/102615445 >> 172.21.15.46:0/102615445 conn(0x7f4a1c0779e0 msgr2=0x7f4a1c077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.459+0000 7f4a21784640 1 -- 172.21.15.46:0/102615445 shutdown_connections 2024-05-09T04:52:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.459+0000 7f4a21784640 1 -- 172.21.15.46:0/102615445 wait complete. 2024-05-09T04:52:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.460+0000 7f4a21784640 1 Processor -- start 2024-05-09T04:52:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.460+0000 7f4a21784640 1 -- start start 2024-05-09T04:52:50.462 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.460+0000 7f4a21784640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1c1022b0 0x7f4a1c102f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.460+0000 7f4a21784640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a1c108c00 0x7f4a1c103490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.460+0000 7f4a21784640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a1c10e650 0x7f4a1c1039d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.460+0000 7f4a21784640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c113fd0 con 0x7f4a1c1022b0 2024-05-09T04:52:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.460+0000 7f4a21784640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c113e50 con 0x7f4a1c108c00 2024-05-09T04:52:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.460+0000 7f4a21784640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c114150 con 0x7f4a1c10e650 2024-05-09T04:52:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.460+0000 7f4a1affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1c1022b0 0x7f4a1c102f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.460+0000 7f4a1affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1c1022b0 0x7f4a1c102f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56418/0 (socket says 172.21.15.46:56418) 2024-05-09T04:52:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.460+0000 7f4a1affd640 1 -- 172.21.15.46:0/1724294514 learned_addr learned my addr 172.21.15.46:0/1724294514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.460+0000 7f4a1a7fc640 1 --2- 172.21.15.46:0/1724294514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a1c108c00 0x7f4a1c103490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.461+0000 7f4a1b7fe640 1 --2- 172.21.15.46:0/1724294514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a1c10e650 0x7f4a1c1039d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.461+0000 7f4a1affd640 1 -- 172.21.15.46:0/1724294514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a1c10e650 msgr2=0x7f4a1c1039d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.461+0000 7f4a1affd640 1 --2- 172.21.15.46:0/1724294514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a1c10e650 0x7f4a1c1039d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.463 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.461+0000 7f4a1affd640 1 -- 172.21.15.46:0/1724294514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a1c108c00 msgr2=0x7f4a1c103490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.461+0000 7f4a1affd640 1 --2- 172.21.15.46:0/1724294514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a1c108c00 0x7f4a1c103490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.461+0000 7f4a1affd640 1 -- 172.21.15.46:0/1724294514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a1c1a8cb0 con 0x7f4a1c1022b0 2024-05-09T04:52:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.461+0000 7f4a1affd640 1 --2- 172.21.15.46:0/1724294514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1c1022b0 0x7f4a1c102f50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00c970 tx=0x7f4a0c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.461+0000 7f4a03fff640 1 -- 172.21.15.46:0/1724294514 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c007bb0 con 0x7f4a1c1022b0 2024-05-09T04:52:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.461+0000 7f4a03fff640 1 -- 172.21.15.46:0/1724294514 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a0c007d50 con 0x7f4a1c1022b0 2024-05-09T04:52:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.461+0000 7f4a21784640 1 -- 172.21.15.46:0/1724294514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a1c1a8fa0 con 0x7f4a1c1022b0 2024-05-09T04:52:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.461+0000 7f4a03fff640 1 -- 172.21.15.46:0/1724294514 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c004dd0 con 0x7f4a1c1022b0 2024-05-09T04:52:50.464 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.461+0000 7f4a21784640 1 -- 172.21.15.46:0/1724294514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a1c1a9400 con 0x7f4a1c1022b0 2024-05-09T04:52:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.464+0000 7f4a2863a740 1 Processor -- start 2024-05-09T04:52:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.464+0000 7f4a2863a740 1 -- start start 2024-05-09T04:52:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.464+0000 7f4a03fff640 1 -- 172.21.15.46:0/1724294514 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4a0c004fb0 con 0x7f4a1c1022b0 2024-05-09T04:52:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.464+0000 7f4a2863a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b541b0 0x55f4f8ef7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.464+0000 7f4a2863a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8ef7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.464+0000 7f4a2863a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b334d0 0x55f4f8ef8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.464+0000 7f4a2863a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f4f8e5c7c0 con 0x55f4f8b334d0 2024-05-09T04:52:50.471 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.465+0000 7f4a2863a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f4f8df80e0 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.465+0000 7f4a2863a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f4f8e58780 con 0x55f4f8b541b0 2024-05-09T04:52:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.465+0000 7f4a1a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b334d0 0x55f4f8ef8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.465+0000 7f4a1a7fc640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b334d0 0x55f4f8ef8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56426/0 (socket says 172.21.15.46:56426) 2024-05-09T04:52:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.465+0000 7f4a1a7fc640 1 -- 172.21.15.46:0/3438259621 learned_addr learned my addr 172.21.15.46:0/3438259621 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.465+0000 7f4a1affd640 1 --2- 172.21.15.46:0/3438259621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8ef7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.465+0000 7f4a1b7fe640 1 --2- 172.21.15.46:0/3438259621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b541b0 0x55f4f8ef7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.465+0000 7f4a1a7fc640 1 -- 172.21.15.46:0/3438259621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b541b0 msgr2=0x55f4f8ef7680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.465+0000 7f4a1a7fc640 1 --2- 172.21.15.46:0/3438259621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b541b0 0x55f4f8ef7680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.465+0000 7f4a1a7fc640 1 -- 172.21.15.46:0/3438259621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b2b2f0 msgr2=0x55f4f8ef7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.465+0000 7f4a1a7fc640 1 --2- 172.21.15.46:0/3438259621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8ef7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.465+0000 7f4a1a7fc640 1 -- 172.21.15.46:0/3438259621 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f8e217c0 con 0x55f4f8b334d0 2024-05-09T04:52:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.465+0000 7f4a1a7fc640 1 --2- 172.21.15.46:0/3438259621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b334d0 0x55f4f8ef8100 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4a10009a00 tx=0x7f4a10009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd01421f8748e3e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.465+0000 7f4a03fff640 1 --2- 172.21.15.46:0/1724294514 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4a040b2950 0x7f4a040b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.466+0000 7f4a1b7fe640 1 --2- 172.21.15.46:0/1724294514 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4a040b2950 0x7f4a040b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.466+0000 7f4a017fa640 1 -- 172.21.15.46:0/3438259621 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a10017070 con 0x55f4f8b334d0 2024-05-09T04:52:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.466+0000 7f4a03fff640 1 -- 172.21.15.46:0/1724294514 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f4a0c0e2340 con 0x7f4a1c1022b0 2024-05-09T04:52:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.466+0000 7f4a017fa640 1 -- 172.21.15.46:0/3438259621 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a10013470 con 0x55f4f8b334d0 2024-05-09T04:52:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.466+0000 7f4a017fa640 1 -- 172.21.15.46:0/3438259621 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a10012630 con 0x55f4f8b334d0 2024-05-09T04:52:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.467+0000 7f4a2863a740 1 -- 172.21.15.46:0/3438259621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b334d0 msgr2=0x55f4f8ef8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.467+0000 7f4a1b7fe640 1 --2- 172.21.15.46:0/1724294514 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4a040b2950 0x7f4a040b4e60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4a140094e0 tx=0x7f4a1402f3a0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.467+0000 7f4a2863a740 1 --2- 172.21.15.46:0/3438259621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b334d0 0x55f4f8ef8100 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4a10009a00 tx=0x7f4a10009ec0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.467+0000 7f4a2863a740 1 -- 172.21.15.46:0/3438259621 shutdown_connections 2024-05-09T04:52:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.467+0000 7f4a2863a740 1 --2- 172.21.15.46:0/3438259621 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b334d0 0x55f4f8ef8100 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.467+0000 7f4a2863a740 1 --2- 172.21.15.46:0/3438259621 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8ef7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.467+0000 7f4a2863a740 1 --2- 172.21.15.46:0/3438259621 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b541b0 0x55f4f8ef7680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.467+0000 7f4a2863a740 1 -- 172.21.15.46:0/3438259621 >> 172.21.15.46:0/3438259621 conn(0x55f4f8b48300 msgr2=0x55f4f8ef3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.467+0000 7f4a2863a740 1 -- 172.21.15.46:0/3438259621 shutdown_connections 2024-05-09T04:52:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.467+0000 7f4a2863a740 1 -- 172.21.15.46:0/3438259621 wait complete. 2024-05-09T04:52:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.467+0000 7f4a2863a740 1 -- start start 2024-05-09T04:52:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.467+0000 7f4a2863a740 1 Processor -- start 2024-05-09T04:52:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.467+0000 7f4a2863a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8f036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a2863a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 0x55f4f8f03be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a2863a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b541b0 0x55f4f8f01b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a2863a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f4f8f047b0 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a2863a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f4f8de39c0 con 0x55f4f8b541b0 2024-05-09T04:52:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a2863a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f4f8f04930 con 0x55f4f8b334d0 2024-05-09T04:52:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a2863a740 10 client.0 _ll_register_callbacks cb 0x55f4f8e10020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a1affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8f036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a1affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8f036a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56432/0 (socket says 172.21.15.46:56432) 2024-05-09T04:52:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a1affd640 1 -- 172.21.15.46:0/819904969 learned_addr learned my addr 172.21.15.46:0/819904969 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a1b7fe640 1 --2- 172.21.15.46:0/819904969 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b541b0 0x55f4f8f01b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a1a7fc640 1 --2- 172.21.15.46:0/819904969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 0x55f4f8f03be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a1affd640 1 -- 172.21.15.46:0/819904969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 msgr2=0x55f4f8f03be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a1affd640 1 --2- 172.21.15.46:0/819904969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 0x55f4f8f03be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a1affd640 1 -- 172.21.15.46:0/819904969 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b541b0 msgr2=0x55f4f8f01b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a1affd640 1 --2- 172.21.15.46:0/819904969 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b541b0 0x55f4f8f01b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a1affd640 1 -- 172.21.15.46:0/819904969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f8e0a1f0 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a1affd640 1 --2- 172.21.15.46:0/819904969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8f036a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0a9300 tx=0x7f4a0c00a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5229d7e72994f8a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f49feff5640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f49feff5640 1 -- 172.21.15.46:0/819904969 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c01d070 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.476 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f49feff5640 1 -- 172.21.15.46:0/819904969 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f4a0c0a5890 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f4a2863a740 1 -- 172.21.15.46:0/819904969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a10013470 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.468+0000 7f49feff5640 1 -- 172.21.15.46:0/819904969 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4a0c0a8f50 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.469+0000 7f4a2863a740 10 client.15729 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.469+0000 7f4a2863a740 1 -- 172.21.15.46:0/819904969 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f4f8bbf790 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.469+0000 7f49feff5640 1 -- 172.21.15.46:0/819904969 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a10013470 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.469+0000 7f49feff5640 1 -- 172.21.15.46:0/819904969 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f4a0c0052a0 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.469+0000 7f4a2863a740 10 client.15729 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.469+0000 7f4a2863a740 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.469+0000 7f4a2863a740 20 client.15729 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.469+0000 7f4a2863a740 1 --2- 172.21.15.46:0/819904969 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f4f8b12d80 0x55f4f8f13f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.469+0000 7f4a2863a740 4 client.15729 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.469+0000 7f4a2863a740 1 -- 172.21.15.46:0/819904969 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4f8e6a3d0 con 0x55f4f8b12d80 2024-05-09T04:52:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.469+0000 7f4a1a7fc640 1 --2- 172.21.15.46:0/819904969 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f4f8b12d80 0x55f4f8f13f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.470+0000 7f4a1a7fc640 1 --2- 172.21.15.46:0/819904969 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f4f8b12d80 0x55f4f8f13f80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2370fa2e8eaec498 server_cookie=f60cf4abc0f68c3e in_seq=0 out_seq=0 2024-05-09T04:52:50.477 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.470+0000 7f49feff5640 10 client.15729 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.470+0000 7f49feff5640 1 -- 172.21.15.46:0/819904969 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4f8e6a3d0 con 0x55f4f8b12d80 2024-05-09T04:52:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.470+0000 7f49feff5640 10 client.15729 handle_command_reply: tid=0 2024-05-09T04:52:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.471+0000 7f4a2863a740 1 client.15729 shutdown 2024-05-09T04:52:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.471+0000 7f4a2863a740 1 -- 172.21.15.46:0/819904969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b2b2f0 msgr2=0x55f4f8f036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.471+0000 7f4a2863a740 1 --2- 172.21.15.46:0/819904969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8f036a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0a9300 tx=0x7f4a0c00a4a0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.471+0000 7f4a2863a740 1 -- 172.21.15.46:0/819904969 shutdown_connections 2024-05-09T04:52:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.471+0000 7f4a2863a740 1 --2- 172.21.15.46:0/819904969 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f4f8b12d80 0x55f4f8f13f80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.471+0000 7f4a2863a740 1 --2- 172.21.15.46:0/819904969 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 0x55f4f8f03be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.471+0000 7f4a2863a740 1 --2- 172.21.15.46:0/819904969 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b541b0 0x55f4f8f01b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.471+0000 7f49feff5640 0 client.15729 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.471+0000 7f4a2863a740 1 --2- 172.21.15.46:0/819904969 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8f036a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.478 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.471+0000 7f4a2863a740 1 -- 172.21.15.46:0/819904969 >> 172.21.15.46:0/819904969 conn(0x55f4f8b48300 msgr2=0x55f4f8ef93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:50.479 INFO:tasks.workunit.client.0.smithi046.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-09T04:52:50.479 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.471+0000 7f4a2863a740 1 -- 172.21.15.46:0/819904969 shutdown_connections 2024-05-09T04:52:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.471+0000 7f4a2863a740 1 -- 172.21.15.46:0/819904969 wait complete. 2024-05-09T04:52:50.480 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.471+0000 7f4a2863a740 20 client.15729 trim_cache size 0 max 16384 2024-05-09T04:52:50.480 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T04:52:50.481 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-09T04:52:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.487+0000 7f4a2863a740 1 Processor -- start 2024-05-09T04:52:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.487+0000 7f4a2863a740 1 -- start start 2024-05-09T04:52:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a2863a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8ef8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a2863a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 0x55f4f8f037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a2863a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b541b0 0x55f4f8f19780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a2863a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c1b4780 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a2863a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c1b2050 con 0x55f4f8b541b0 2024-05-09T04:52:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a2863a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c1afea0 con 0x55f4f8b334d0 2024-05-09T04:52:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a1affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8ef8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a1affd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8ef8420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56446/0 (socket says 172.21.15.46:56446) 2024-05-09T04:52:50.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a1affd640 1 -- 172.21.15.46:0/1534378167 learned_addr learned my addr 172.21.15.46:0/1534378167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a1b7fe640 1 --2- 172.21.15.46:0/1534378167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b541b0 0x55f4f8f19780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a1a7fc640 1 --2- 172.21.15.46:0/1534378167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 0x55f4f8f037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a1affd640 1 -- 172.21.15.46:0/1534378167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 msgr2=0x55f4f8f037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a1affd640 1 --2- 172.21.15.46:0/1534378167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 0x55f4f8f037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a1affd640 1 -- 172.21.15.46:0/1534378167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b541b0 msgr2=0x55f4f8f19780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a1affd640 1 --2- 172.21.15.46:0/1534378167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b541b0 0x55f4f8f19780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a1affd640 1 -- 172.21.15.46:0/1534378167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4f8c365f0 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a1a7fc640 1 --2- 172.21.15.46:0/1534378167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 0x55f4f8f037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:52:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.488+0000 7f4a1affd640 1 --2- 172.21.15.46:0/1534378167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8ef8420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0a9590 tx=0x7f4a0c0dff00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3000a5a7b4f54d74 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.489+0000 7f4a00ff9640 1 -- 172.21.15.46:0/1534378167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c01d070 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.489+0000 7f4a00ff9640 1 -- 172.21.15.46:0/1534378167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a0c0a9110 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.489+0000 7f4a00ff9640 1 -- 172.21.15.46:0/1534378167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c0a8d70 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.489+0000 7f4a2863a740 1 -- 172.21.15.46:0/1534378167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b2b2f0 msgr2=0x55f4f8ef8420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.489+0000 7f4a2863a740 1 --2- 172.21.15.46:0/1534378167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8ef8420 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0a9590 tx=0x7f4a0c0dff00 comp rx=0 tx=0).stop 2024-05-09T04:52:50.494 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.489+0000 7f4a2863a740 1 -- 172.21.15.46:0/1534378167 shutdown_connections 2024-05-09T04:52:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.489+0000 7f4a2863a740 1 --2- 172.21.15.46:0/1534378167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 0x55f4f8f037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.489+0000 7f4a2863a740 1 --2- 172.21.15.46:0/1534378167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b541b0 0x55f4f8f19780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.489+0000 7f4a2863a740 1 --2- 172.21.15.46:0/1534378167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8ef8420 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.490+0000 7f4a2863a740 1 -- 172.21.15.46:0/1534378167 >> 172.21.15.46:0/1534378167 conn(0x55f4f8b48300 msgr2=0x55f4f8f030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.490+0000 7f4a2863a740 1 -- 172.21.15.46:0/1534378167 shutdown_connections 2024-05-09T04:52:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.490+0000 7f4a2863a740 1 -- 172.21.15.46:0/1534378167 wait complete. 2024-05-09T04:52:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.490+0000 7f4a2863a740 1 -- start start 2024-05-09T04:52:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.490+0000 7f4a2863a740 1 Processor -- start 2024-05-09T04:52:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.490+0000 7f4a2863a740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8f09e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.490+0000 7f4a2863a740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 0x55f4f8f05160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.495 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.490+0000 7f4a2863a740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b541b0 0x55f4f8f056a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.490+0000 7f4a2863a740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c1b0510 con 0x55f4f8b541b0 2024-05-09T04:52:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.490+0000 7f4a2863a740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c1b1980 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.490+0000 7f4a2863a740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4a1c1b0be0 con 0x55f4f8b334d0 2024-05-09T04:52:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.490+0000 7f4a2863a740 10 client.0 _ll_register_callbacks cb 0x7f4a0c00c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.490+0000 7f4a1b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b541b0 0x55f4f8f056a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.491+0000 7f4a1b7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b541b0 0x55f4f8f056a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56452/0 (socket says 172.21.15.46:56452) 2024-05-09T04:52:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.491+0000 7f4a1affd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8f09e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.491+0000 7f4a1b7fe640 1 -- 172.21.15.46:0/1479629723 learned_addr learned my addr 172.21.15.46:0/1479629723 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.491+0000 7f4a1a7fc640 1 --2- 172.21.15.46:0/1479629723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 0x55f4f8f05160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.491+0000 7f4a1affd640 1 -- 172.21.15.46:0/1479629723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 msgr2=0x55f4f8f05160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:50.496 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.491+0000 7f4a1affd640 1 --2- 172.21.15.46:0/1479629723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 0x55f4f8f05160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.491+0000 7f4a1affd640 1 -- 172.21.15.46:0/1479629723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b541b0 msgr2=0x55f4f8f056a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.491+0000 7f4a1affd640 1 --2- 172.21.15.46:0/1479629723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b541b0 0x55f4f8f056a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.491+0000 7f4a1affd640 1 -- 172.21.15.46:0/1479629723 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4f8e20ed0 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.491+0000 7f4a1b7fe640 1 --2- 172.21.15.46:0/1479629723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b541b0 0x55f4f8f056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:52:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.491+0000 7f4a1affd640 1 --2- 172.21.15.46:0/1479629723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8f09e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0e0070 tx=0x7f4a0c08fb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8089eed7bd5394c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.491+0000 7f49feff5640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:52:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.491+0000 7f49feff5640 1 -- 172.21.15.46:0/1479629723 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a0c01d040 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.491+0000 7f49feff5640 1 -- 172.21.15.46:0/1479629723 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f4a0c0a8a70 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.491+0000 7f4a2863a740 1 -- 172.21.15.46:0/1479629723 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a0c0a9110 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.497 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.492+0000 7f49feff5640 1 -- 172.21.15.46:0/1479629723 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4a0c0b2460 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.492+0000 7f4a2863a740 10 client.25475 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.492+0000 7f4a2863a740 1 -- 172.21.15.46:0/1479629723 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f4f8e20ed0 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.492+0000 7f49feff5640 1 -- 172.21.15.46:0/1479629723 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a0c0a9110 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.492+0000 7f49feff5640 1 -- 172.21.15.46:0/1479629723 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f4a0c0a82d0 con 0x55f4f8b2b2f0 2024-05-09T04:52:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.492+0000 7f4a2863a740 10 client.25475 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.492+0000 7f4a2863a740 10 client.25475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.492+0000 7f4a2863a740 20 client.25475 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.492+0000 7f4a2863a740 1 --2- 172.21.15.46:0/1479629723 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f4f8ae81c0 0x55f4f8f7dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.492+0000 7f4a2863a740 4 client.25475 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T04:52:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.492+0000 7f4a2863a740 1 -- 172.21.15.46:0/1479629723 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4f8e64240 con 0x55f4f8ae81c0 2024-05-09T04:52:50.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.492+0000 7f4a1a7fc640 1 --2- 172.21.15.46:0/1479629723 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f4f8ae81c0 0x55f4f8f7dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.493+0000 7f49feff5640 10 client.25475 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:50.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.493+0000 7f4a1a7fc640 1 --2- 172.21.15.46:0/1479629723 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f4f8ae81c0 0x55f4f8f7dfe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c2f84b236a57aba server_cookie=7a2ff6c0dd23dc5e in_seq=0 out_seq=0 2024-05-09T04:52:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.496+0000 7f49feff5640 1 -- 172.21.15.46:0/1479629723 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55f4f8e64240 con 0x55f4f8ae81c0 2024-05-09T04:52:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.496+0000 7f49feff5640 10 client.25475 handle_command_reply: tid=0 2024-05-09T04:52:50.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.496+0000 7f4a2863a740 1 client.25475 shutdown 2024-05-09T04:52:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a2863a740 1 -- 172.21.15.46:0/1479629723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b2b2f0 msgr2=0x55f4f8f09e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a2863a740 1 --2- 172.21.15.46:0/1479629723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8f09e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c0e0070 tx=0x7f4a0c08fb20 comp rx=0 tx=0).stop 2024-05-09T04:52:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a2863a740 1 -- 172.21.15.46:0/1479629723 shutdown_connections 2024-05-09T04:52:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a2863a740 1 --2- 172.21.15.46:0/1479629723 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f4f8ae81c0 0x55f4f8f7dfe0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a2863a740 1 --2- 172.21.15.46:0/1479629723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f4f8b334d0 0x55f4f8f05160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f49feff5640 0 client.25475 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a2863a740 1 --2- 172.21.15.46:0/1479629723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f4f8b541b0 0x55f4f8f056a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a2863a740 1 --2- 172.21.15.46:0/1479629723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f4f8b2b2f0 0x55f4f8f09e10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a2863a740 1 -- 172.21.15.46:0/1479629723 >> 172.21.15.46:0/1479629723 conn(0x55f4f8b48300 msgr2=0x55f4f8f03000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a2863a740 1 -- 172.21.15.46:0/1479629723 shutdown_connections 2024-05-09T04:52:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a2863a740 1 -- 172.21.15.46:0/1479629723 wait complete. 2024-05-09T04:52:50.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a2863a740 20 client.25475 trim_cache size 0 max 16384 2024-05-09T04:52:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-05-09T04:52:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-05-09T04:52:50.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-05-09T04:52:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a21784640 1 -- 172.21.15.46:0/1724294514 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4a040b2950 msgr2=0x7f4a040b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a21784640 1 --2- 172.21.15.46:0/1724294514 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4a040b2950 0x7f4a040b4e60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4a140094e0 tx=0x7f4a1402f3a0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a21784640 1 -- 172.21.15.46:0/1724294514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1c1022b0 msgr2=0x7f4a1c102f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a21784640 1 --2- 172.21.15.46:0/1724294514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1c1022b0 0x7f4a1c102f50 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4a0c00c970 tx=0x7f4a0c00cd70 comp rx=0 tx=0).stop 2024-05-09T04:52:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a21784640 1 -- 172.21.15.46:0/1724294514 shutdown_connections 2024-05-09T04:52:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a21784640 1 --2- 172.21.15.46:0/1724294514 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4a040b2950 0x7f4a040b4e60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a21784640 1 --2- 172.21.15.46:0/1724294514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4a1c10e650 0x7f4a1c1039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a21784640 1 --2- 172.21.15.46:0/1724294514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4a1c108c00 0x7f4a1c103490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a21784640 1 --2- 172.21.15.46:0/1724294514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4a1c1022b0 0x7f4a1c102f50 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:50.503 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.497+0000 7f4a21784640 1 -- 172.21.15.46:0/1724294514 >> 172.21.15.46:0/1724294514 conn(0x7f4a1c0779e0 msgr2=0x7f4a1c100a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:50.505 INFO:tasks.workunit:Running workunits matching suites/iozone.sh on client.0... 2024-05-09T04:52:50.507 INFO:tasks.workunit:Running workunit suites/iozone.sh... 2024-05-09T04:52:50.507 DEBUG:teuthology.orchestra.run.smithi046:workunit test suites/iozone.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c9da4049f638c0f84bdd9b4e33c42e86046ac554 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iozone.sh 2024-05-09T04:52:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.498+0000 7f4a21784640 1 -- 172.21.15.46:0/1724294514 shutdown_connections 2024-05-09T04:52:50.507 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:52:50.498+0000 7f4a21784640 1 -- 172.21.15.46:0/1724294514 wait complete. 2024-05-09T04:52:50.568 INFO:tasks.workunit.client.0.smithi046.stderr:+ iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-05-09T04:52:50.690 INFO:tasks.workunit.client.0.smithi046.stdout: Iozone: Performance Test of File I/O 2024-05-09T04:52:50.691 INFO:tasks.workunit.client.0.smithi046.stdout: Version $Revision: 3.494 $ 2024-05-09T04:52:50.691 INFO:tasks.workunit.client.0.smithi046.stdout: Compiled for 64 bit mode. 2024-05-09T04:52:50.691 INFO:tasks.workunit.client.0.smithi046.stdout: Build: linux-AMD64 2024-05-09T04:52:50.691 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:52:50.691 INFO:tasks.workunit.client.0.smithi046.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-05-09T04:52:50.691 INFO:tasks.workunit.client.0.smithi046.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-05-09T04:52:50.691 INFO:tasks.workunit.client.0.smithi046.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-05-09T04:52:50.691 INFO:tasks.workunit.client.0.smithi046.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-05-09T04:52:50.691 INFO:tasks.workunit.client.0.smithi046.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-05-09T04:52:50.691 INFO:tasks.workunit.client.0.smithi046.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-05-09T04:52:50.691 INFO:tasks.workunit.client.0.smithi046.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-05-09T04:52:50.691 INFO:tasks.workunit.client.0.smithi046.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-05-09T04:52:50.691 INFO:tasks.workunit.client.0.smithi046.stdout: Alexey Skidanov, Sudhir Kumar. 2024-05-09T04:52:50.692 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:52:50.692 INFO:tasks.workunit.client.0.smithi046.stdout: Run began: Thu May 9 04:52:50 2024 2024-05-09T04:52:50.692 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:52:50.692 INFO:tasks.workunit.client.0.smithi046.stdout: Include close in write timing 2024-05-09T04:52:50.692 INFO:tasks.workunit.client.0.smithi046.stdout: Include fsync in write timing 2024-05-09T04:52:50.692 INFO:tasks.workunit.client.0.smithi046.stdout: File size set to 1048576 kB 2024-05-09T04:52:50.692 INFO:tasks.workunit.client.0.smithi046.stdout: Record Size 16 kB 2024-05-09T04:52:50.692 INFO:tasks.workunit.client.0.smithi046.stdout: Command line used: iozone -c -e -s 1024M -r 16K -t 1 -F f1 -i 0 -i 1 2024-05-09T04:52:50.693 INFO:tasks.workunit.client.0.smithi046.stdout: Output is in kBytes/sec 2024-05-09T04:52:50.693 INFO:tasks.workunit.client.0.smithi046.stdout: Time Resolution = 0.000001 seconds. 2024-05-09T04:52:50.693 INFO:tasks.workunit.client.0.smithi046.stdout: Processor cache size set to 1024 kBytes. 2024-05-09T04:52:50.693 INFO:tasks.workunit.client.0.smithi046.stdout: Processor cache line size set to 32 bytes. 2024-05-09T04:52:50.693 INFO:tasks.workunit.client.0.smithi046.stdout: File stride size set to 17 * record size. 2024-05-09T04:52:50.693 INFO:tasks.workunit.client.0.smithi046.stdout: Throughput test with 1 process 2024-05-09T04:52:50.693 INFO:tasks.workunit.client.0.smithi046.stdout: Each process writes a 1048576 kByte file in 16 kByte records 2024-05-09T04:52:51.052 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:50 smithi046 ceph-mon[30136]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-09T04:52:51.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:50 smithi110 ceph-mon[34904]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-09T04:52:51.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:50 smithi072 ceph-mon[34088]: pgmap v55: 97 pgs: 97 active+clean; 1.3 MiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2024-05-09T04:52:51.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.349+0000 7f85ff550700 1 -- 172.21.15.46:0/607025047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85f810b110 msgr2=0x7f85f8117620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:51.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.349+0000 7f85ff550700 1 --2- 172.21.15.46:0/607025047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85f810b110 0x7f85f8117620 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f85f000b0a0 tx=0x7f85f000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.355+0000 7f85ff550700 1 -- 172.21.15.46:0/607025047 shutdown_connections 2024-05-09T04:52:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.355+0000 7f85ff550700 1 --2- 172.21.15.46:0/607025047 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85f810b110 0x7f85f8117620 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.355+0000 7f85ff550700 1 --2- 172.21.15.46:0/607025047 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85f810f6c0 0x7f85f810fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.355+0000 7f85ff550700 1 --2- 172.21.15.46:0/607025047 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85f810a7c0 0x7f85f810abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.355+0000 7f85ff550700 1 -- 172.21.15.46:0/607025047 >> 172.21.15.46:0/607025047 conn(0x7f85f8076b00 msgr2=0x7f85f8076f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:51.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.360+0000 7f85ff550700 1 -- 172.21.15.46:0/607025047 shutdown_connections 2024-05-09T04:52:51.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.361+0000 7f85ff550700 1 -- 172.21.15.46:0/607025047 wait complete. 2024-05-09T04:52:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.363+0000 7f85ff550700 1 Processor -- start 2024-05-09T04:52:51.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.363+0000 7f85ff550700 1 -- start start 2024-05-09T04:52:51.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.365+0000 7f85ff550700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85f810a7c0 0x7f85f81a1300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.366+0000 7f85ff550700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85f810b110 0x7f85f81a18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.366+0000 7f85ff550700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85f810f6c0 0x7f85f81a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.366+0000 7f85ff550700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85f811a990 con 0x7f85f810b110 2024-05-09T04:52:51.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.366+0000 7f85ff550700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f85f811a810 con 0x7f85f810f6c0 2024-05-09T04:52:51.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.366+0000 7f85ff550700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85f811ab10 con 0x7f85f810a7c0 2024-05-09T04:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.367+0000 7f85fe54e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85f810a7c0 0x7f85f81a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.367+0000 7f85fe54e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85f810a7c0 0x7f85f81a1300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:42260/0 (socket says 172.21.15.46:42260) 2024-05-09T04:52:51.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.367+0000 7f85fe54e700 1 -- 172.21.15.46:0/3142380322 learned_addr learned my addr 172.21.15.46:0/3142380322 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:51.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.369+0000 7f85fdd4d700 1 --2- 172.21.15.46:0/3142380322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85f810b110 0x7f85f81a18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:51.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.371+0000 7f85fdd4d700 1 -- 172.21.15.46:0/3142380322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85f810a7c0 msgr2=0x7f85f81a1300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.371+0000 7f85fdd4d700 1 --2- 172.21.15.46:0/3142380322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85f810a7c0 0x7f85f81a1300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.371+0000 7f85fdd4d700 1 -- 172.21.15.46:0/3142380322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85f810f6c0 msgr2=0x7f85f81a5b30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T04:52:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.371+0000 7f85fdd4d700 1 --2- 172.21.15.46:0/3142380322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85f810f6c0 0x7f85f81a5b30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.371+0000 7f85fdd4d700 1 -- 172.21.15.46:0/3142380322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85f81a61d0 con 0x7f85f810b110 2024-05-09T04:52:51.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.372+0000 7f85fdd4d700 1 --2- 172.21.15.46:0/3142380322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85f810b110 0x7f85f81a18a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f85e800c970 tx=0x7f85e800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.373+0000 7f85ef7fe700 1 -- 172.21.15.46:0/3142380322 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e800ce60 con 0x7f85f810b110 2024-05-09T04:52:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.373+0000 7f85ff550700 1 -- 172.21.15.46:0/3142380322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85f81a64c0 con 0x7f85f810b110 2024-05-09T04:52:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.373+0000 7f85ff550700 1 -- 172.21.15.46:0/3142380322 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85f81adde0 con 0x7f85f810b110 2024-05-09T04:52:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.373+0000 7f85ef7fe700 1 -- 172.21.15.46:0/3142380322 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85e8004830 con 0x7f85f810b110 2024-05-09T04:52:51.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.373+0000 7f85ef7fe700 1 -- 172.21.15.46:0/3142380322 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e8005580 con 0x7f85f810b110 2024-05-09T04:52:51.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.381+0000 7f860f1c8b80 1 Processor -- start 2024-05-09T04:52:51.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.387+0000 7f860f1c8b80 1 -- start start 2024-05-09T04:52:51.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.388+0000 7f860f1c8b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3b791e0 0x5630e3c127c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.388+0000 7f860f1c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3a8f520 0x5630e3c12d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.388+0000 7f860f1c8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3c0c880 0x5630e3c0ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.388+0000 7f860f1c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5630e3b2f070 con 0x5630e3a8f520 2024-05-09T04:52:51.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.388+0000 7f860f1c8b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5630e3acd870 con 0x5630e3b791e0 2024-05-09T04:52:51.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.388+0000 7f860f1c8b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5630e3b2b030 con 0x5630e3c0c880 2024-05-09T04:52:51.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.389+0000 7f85ef7fe700 1 -- 172.21.15.46:0/3142380322 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f85e800f450 con 0x7f85f810b110 2024-05-09T04:52:51.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.389+0000 7f85fed4f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3b791e0 0x5630e3c127c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:51.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.389+0000 7f85fed4f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3b791e0 0x5630e3c127c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:51494/0 (socket says 172.21.15.46:51494) 2024-05-09T04:52:51.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.389+0000 7f85fed4f700 1 -- 172.21.15.46:0/2576181374 learned_addr learned my addr 172.21.15.46:0/2576181374 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:51.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.390+0000 7f85fed4f700 1 -- 172.21.15.46:0/2576181374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3c0c880 msgr2=0x5630e3c0ccb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:52:51.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.390+0000 7f85fed4f700 1 --2- 172.21.15.46:0/2576181374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3c0c880 0x5630e3c0ccb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.390+0000 7f85fed4f700 1 -- 172.21.15.46:0/2576181374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3a8f520 msgr2=0x5630e3c12d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:51.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.390+0000 7f85fed4f700 1 --2- 172.21.15.46:0/2576181374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3a8f520 0x5630e3c12d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.390+0000 7f85fed4f700 1 -- 172.21.15.46:0/2576181374 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630e3ac53f0 con 0x5630e3b791e0 2024-05-09T04:52:51.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.391+0000 7f85fed4f700 1 --2- 172.21.15.46:0/2576181374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3b791e0 0x5630e3c127c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f85f0009900 tx=0x7f85f000cd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=401079bbf30f6b02 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:51.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.391+0000 7f85ef7fe700 1 --2- 172.21.15.46:0/3142380322 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f85e40b2b20 0x7f85e40b5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.391+0000 7f85ecff9700 1 -- 172.21.15.46:0/2576181374 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0021030 con 0x5630e3b791e0 2024-05-09T04:52:51.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.391+0000 7f85ecff9700 1 -- 172.21.15.46:0/2576181374 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f0003a00 con 0x5630e3b791e0 2024-05-09T04:52:51.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.393+0000 7f860f1c8b80 1 -- 172.21.15.46:0/2576181374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3b791e0 msgr2=0x5630e3c127c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:51.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.393+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/2576181374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3b791e0 0x5630e3c127c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f85f0009900 tx=0x7f85f000cd80 comp rx=0 tx=0).stop 2024-05-09T04:52:51.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.393+0000 7f85fe54e700 1 --2- 172.21.15.46:0/3142380322 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f85e40b2b20 0x7f85e40b5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:51.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.394+0000 7f85ef7fe700 1 -- 172.21.15.46:0/3142380322 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f85e80e1ff0 con 0x7f85f810b110 2024-05-09T04:52:51.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.396+0000 7f860f1c8b80 1 -- 172.21.15.46:0/2576181374 shutdown_connections 2024-05-09T04:52:51.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.396+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/2576181374 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3c0c880 0x5630e3c0ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.396+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/2576181374 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3b791e0 0x5630e3c127c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.396+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/2576181374 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3a8f520 0x5630e3c12d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.396+0000 7f860f1c8b80 1 -- 172.21.15.46:0/2576181374 >> 172.21.15.46:0/2576181374 conn(0x5630e3aef720 msgr2=0x5630e3c07750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:51.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.397+0000 7f85fe54e700 1 --2- 172.21.15.46:0/3142380322 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f85e40b2b20 0x7f85e40b5030 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f85f4006ae0 tx=0x7f85f4006a70 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:51.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.399+0000 7f860f1c8b80 1 -- 172.21.15.46:0/2576181374 shutdown_connections 2024-05-09T04:52:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.399+0000 7f860f1c8b80 1 -- 172.21.15.46:0/2576181374 wait complete. 2024-05-09T04:52:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.400+0000 7f860f1c8b80 1 -- start start 2024-05-09T04:52:51.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.400+0000 7f860f1c8b80 1 Processor -- start 2024-05-09T04:52:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.401+0000 7f860f1c8b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3a8f520 0x5630e3c16320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.401+0000 7f860f1c8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3b791e0 0x5630e3c16860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.401+0000 7f860f1c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3c0c880 0x5630e3c16da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.401+0000 7f860f1c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5630e39aac20 con 0x5630e3c0c880 2024-05-09T04:52:51.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.401+0000 7f860f1c8b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5630e3ae91d0 con 0x5630e3a8f520 2024-05-09T04:52:51.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.401+0000 7f860f1c8b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5630e3c2a4b0 con 0x5630e3b791e0 2024-05-09T04:52:51.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.401+0000 7f860f1c8b80 10 client.0 _ll_register_callbacks cb 0x5630e39a0740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:51.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.404+0000 7f85fe54e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3a8f520 0x5630e3c16320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:51.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.406+0000 7f85fe54e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3a8f520 0x5630e3c16320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:51506/0 (socket says 172.21.15.46:51506) 2024-05-09T04:52:51.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.406+0000 7f85fe54e700 1 -- 172.21.15.46:0/3507123915 learned_addr learned my addr 172.21.15.46:0/3507123915 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:51.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.406+0000 7f85fed4f700 1 --2- 172.21.15.46:0/3507123915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3c0c880 0x5630e3c16da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.406+0000 7f85fdd4d700 1 --2- 172.21.15.46:0/3507123915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3b791e0 0x5630e3c16860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:51.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.407+0000 7f85fed4f700 1 -- 172.21.15.46:0/3507123915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3b791e0 msgr2=0x5630e3c16860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.407+0000 7f85fed4f700 1 --2- 172.21.15.46:0/3507123915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3b791e0 0x5630e3c16860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.407+0000 7f85fed4f700 1 -- 172.21.15.46:0/3507123915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3a8f520 msgr2=0x5630e3c16320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.407+0000 7f85fed4f700 1 --2- 172.21.15.46:0/3507123915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3a8f520 0x5630e3c16320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.407+0000 7f85fed4f700 1 -- 172.21.15.46:0/3507123915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630e39aa510 con 0x5630e3c0c880 2024-05-09T04:52:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.407+0000 7f85fed4f700 1 --2- 172.21.15.46:0/3507123915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3c0c880 0x5630e3c16da0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f85f000b0a0 tx=0x7f85f0007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=770283efb03fb5b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.408+0000 7f85e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.408+0000 7f85e1ffb700 1 -- 172.21.15.46:0/3507123915 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0003dc0 con 0x5630e3c0c880 2024-05-09T04:52:51.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.408+0000 7f85e1ffb700 1 -- 172.21.15.46:0/3507123915 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f85f00050e0 con 0x5630e3c0c880 2024-05-09T04:52:51.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.415+0000 7f860f1c8b80 1 -- 172.21.15.46:0/3507123915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85f0003a00 con 0x5630e3c0c880 2024-05-09T04:52:51.422 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.422+0000 7f85e1ffb700 1 -- 172.21.15.46:0/3507123915 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f85f00052e0 con 0x5630e3c0c880 2024-05-09T04:52:51.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.423+0000 7f860f1c8b80 10 client.15744 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:51.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.423+0000 7f860f1c8b80 1 -- 172.21.15.46:0/3507123915 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630e3c34de0 con 0x5630e3c0c880 2024-05-09T04:52:51.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.425+0000 7f85e1ffb700 1 -- 172.21.15.46:0/3507123915 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f0003a00 con 0x5630e3c0c880 2024-05-09T04:52:51.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.425+0000 7f85e1ffb700 1 -- 172.21.15.46:0/3507123915 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f85f0011070 con 0x5630e3c0c880 2024-05-09T04:52:51.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.425+0000 7f860f1c8b80 10 client.15744 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:51.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.425+0000 7f860f1c8b80 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:51.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.425+0000 7f860f1c8b80 20 client.15744 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:51.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.425+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/3507123915 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5630e3c372b0 0x5630e3c396d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.425+0000 7f860f1c8b80 4 client.15744 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:51.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.426+0000 7f860f1c8b80 1 -- 172.21.15.46:0/3507123915 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630e3b3ca50 con 0x5630e3c372b0 2024-05-09T04:52:51.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.435+0000 7f85fe54e700 1 --2- 172.21.15.46:0/3507123915 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5630e3c372b0 0x5630e3c396d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:51.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.440+0000 7f85fe54e700 1 --2- 172.21.15.46:0/3507123915 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5630e3c372b0 0x5630e3c396d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5f184b3cebb505 server_cookie=45806824cc97fe in_seq=0 out_seq=0 2024-05-09T04:52:51.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.440+0000 7f85e1ffb700 10 client.15744 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:51.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.444+0000 7f85e1ffb700 1 -- 172.21.15.46:0/3507123915 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5630e3b3ca50 con 0x5630e3c372b0 2024-05-09T04:52:51.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.444+0000 7f85e1ffb700 10 client.15744 handle_command_reply: tid=0 2024-05-09T04:52:51.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.446+0000 7f860f1c8b80 1 client.15744 shutdown 2024-05-09T04:52:51.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.446+0000 7f860f1c8b80 1 -- 172.21.15.46:0/3507123915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3c0c880 msgr2=0x5630e3c16da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:51.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.447+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/3507123915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3c0c880 0x5630e3c16da0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f85f000b0a0 tx=0x7f85f0007f40 comp rx=0 tx=0).stop 2024-05-09T04:52:51.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.448+0000 7f860f1c8b80 1 -- 172.21.15.46:0/3507123915 shutdown_connections 2024-05-09T04:52:51.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.448+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/3507123915 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3c0c880 0x5630e3c16da0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.448+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/3507123915 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5630e3c372b0 0x5630e3c396d0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.448+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/3507123915 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3a8f520 0x5630e3c16320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.448+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/3507123915 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3b791e0 0x5630e3c16860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.448+0000 7f860f1c8b80 1 -- 172.21.15.46:0/3507123915 >> 172.21.15.46:0/3507123915 conn(0x5630e3aef720 msgr2=0x5630e3c07450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:51.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.448+0000 7f85e1ffb700 0 client.15744 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:51.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.450+0000 7f860f1c8b80 1 -- 172.21.15.46:0/3507123915 shutdown_connections 2024-05-09T04:52:51.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.450+0000 7f860f1c8b80 1 -- 172.21.15.46:0/3507123915 wait complete. 2024-05-09T04:52:51.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.451+0000 7f860f1c8b80 20 client.15744 trim_cache size 0 max 16384 2024-05-09T04:52:51.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.504+0000 7f860f1c8b80 1 Processor -- start 2024-05-09T04:52:51.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.504+0000 7f860f1c8b80 1 -- start start 2024-05-09T04:52:51.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.505+0000 7f860f1c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3a8f520 0x5630e3c0c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.505+0000 7f860f1c8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3b791e0 0x5630e3c16be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.505+0000 7f860f1c8b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3c0c880 0x5630e3c133c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.505+0000 7f85fe54e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3a8f520 0x5630e3c0c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.505+0000 7f85fe54e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3a8f520 0x5630e3c0c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56514/0 (socket says 172.21.15.46:56514) 2024-05-09T04:52:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.506+0000 7f85fe54e700 1 -- 172.21.15.46:0/1512295479 learned_addr learned my addr 172.21.15.46:0/1512295479 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.506+0000 7f860f1c8b80 1 -- 172.21.15.46:0/1512295479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc00ff90 con 0x5630e3a8f520 2024-05-09T04:52:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.506+0000 7f860f1c8b80 1 -- 172.21.15.46:0/1512295479 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc00d820 con 0x5630e3c0c880 2024-05-09T04:52:51.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.506+0000 7f860f1c8b80 1 -- 172.21.15.46:0/1512295479 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc00b670 con 0x5630e3b791e0 2024-05-09T04:52:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.506+0000 7f85fe54e700 1 -- 172.21.15.46:0/1512295479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3b791e0 msgr2=0x5630e3c16be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.506+0000 7f85fe54e700 1 --2- 172.21.15.46:0/1512295479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3b791e0 0x5630e3c16be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.506+0000 7f85fe54e700 1 -- 172.21.15.46:0/1512295479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3c0c880 msgr2=0x5630e3c133c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.506+0000 7f85fe54e700 1 --2- 172.21.15.46:0/1512295479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3c0c880 0x5630e3c133c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.506+0000 7f85fe54e700 1 -- 172.21.15.46:0/1512295479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630e3c17120 con 0x5630e3a8f520 2024-05-09T04:52:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.507+0000 7f85fe54e700 1 --2- 172.21.15.46:0/1512295479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3a8f520 0x5630e3c0c2a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009560 tx=0x7f85f40047e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff2449ccd245141b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.507+0000 7f85e3fff700 1 -- 172.21.15.46:0/1512295479 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f401b040 con 0x5630e3a8f520 2024-05-09T04:52:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.507+0000 7f85e3fff700 1 -- 172.21.15.46:0/1512295479 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f402b030 con 0x5630e3a8f520 2024-05-09T04:52:51.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.507+0000 7f85e3fff700 1 -- 172.21.15.46:0/1512295479 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f4015a40 con 0x5630e3a8f520 2024-05-09T04:52:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.508+0000 7f860f1c8b80 1 -- 172.21.15.46:0/1512295479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3a8f520 msgr2=0x5630e3c0c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.508+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/1512295479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3a8f520 0x5630e3c0c2a0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009560 tx=0x7f85f40047e0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.508+0000 7f860f1c8b80 1 -- 172.21.15.46:0/1512295479 shutdown_connections 2024-05-09T04:52:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.508+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/1512295479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3c0c880 0x5630e3c133c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.508+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/1512295479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3a8f520 0x5630e3c0c2a0 secure :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f85f4009560 tx=0x7f85f40047e0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.508+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/1512295479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3b791e0 0x5630e3c16be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.508+0000 7f860f1c8b80 1 -- 172.21.15.46:0/1512295479 >> 172.21.15.46:0/1512295479 conn(0x5630e3aef720 msgr2=0x5630e3c10740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.509+0000 7f860f1c8b80 1 -- 172.21.15.46:0/1512295479 shutdown_connections 2024-05-09T04:52:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.509+0000 7f860f1c8b80 1 -- 172.21.15.46:0/1512295479 wait complete. 2024-05-09T04:52:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.509+0000 7f860f1c8b80 1 -- start start 2024-05-09T04:52:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.509+0000 7f860f1c8b80 1 Processor -- start 2024-05-09T04:52:51.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.510+0000 7f860f1c8b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3b791e0 0x5630e3c3a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.510+0000 7f860f1c8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3c0c880 0x5630e3c3ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.510+0000 7f860f1c8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3c9e7f0 0x5630e3ca0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.510+0000 7f860f1c8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc00bce0 con 0x5630e3c9e7f0 2024-05-09T04:52:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.510+0000 7f860f1c8b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc00d150 con 0x5630e3b791e0 2024-05-09T04:52:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.510+0000 7f860f1c8b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc00c3b0 con 0x5630e3c0c880 2024-05-09T04:52:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.510+0000 7f860f1c8b80 10 client.0 _ll_register_callbacks cb 0x5630e39a0740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.511+0000 7f85fed4f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3c9e7f0 0x5630e3ca0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.511+0000 7f85fed4f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3c9e7f0 0x5630e3ca0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56518/0 (socket says 172.21.15.46:56518) 2024-05-09T04:52:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.511+0000 7f85fed4f700 1 -- 172.21.15.46:0/928161122 learned_addr learned my addr 172.21.15.46:0/928161122 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:51.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.511+0000 7f85fdd4d700 1 --2- 172.21.15.46:0/928161122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3c0c880 0x5630e3c3ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.511+0000 7f85fe54e700 1 --2- 172.21.15.46:0/928161122 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3b791e0 0x5630e3c3a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.511+0000 7f85fed4f700 1 -- 172.21.15.46:0/928161122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3c0c880 msgr2=0x5630e3c3ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.511+0000 7f85fed4f700 1 --2- 172.21.15.46:0/928161122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3c0c880 0x5630e3c3ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.511+0000 7f85fed4f700 1 -- 172.21.15.46:0/928161122 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3b791e0 msgr2=0x5630e3c3a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.511+0000 7f85fed4f700 1 --2- 172.21.15.46:0/928161122 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3b791e0 0x5630e3c3a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.511+0000 7f85fed4f700 1 -- 172.21.15.46:0/928161122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630e3ca1150 con 0x5630e3c9e7f0 2024-05-09T04:52:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.511+0000 7f85fe54e700 1 --2- 172.21.15.46:0/928161122 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3b791e0 0x5630e3c3a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:52:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.512+0000 7f85fed4f700 1 --2- 172.21.15.46:0/928161122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3c9e7f0 0x5630e3ca0c10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f85f000b070 tx=0x7f85f0009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61182d2d7053610 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.512+0000 7f85e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.513+0000 7f85e1ffb700 1 -- 172.21.15.46:0/928161122 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0021070 con 0x5630e3c9e7f0 2024-05-09T04:52:51.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.513+0000 7f85e1ffb700 1 -- 172.21.15.46:0/928161122 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f85f001d450 con 0x5630e3c9e7f0 2024-05-09T04:52:51.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.513+0000 7f860f1c8b80 1 -- 172.21.15.46:0/928161122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85f402b030 con 0x5630e3c9e7f0 2024-05-09T04:52:51.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.513+0000 7f85e1ffb700 1 -- 172.21.15.46:0/928161122 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f85f0003a00 con 0x5630e3c9e7f0 2024-05-09T04:52:51.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.513+0000 7f860f1c8b80 10 client.15750 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:51.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.513+0000 7f860f1c8b80 1 -- 172.21.15.46:0/928161122 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5630e3ca2370 con 0x5630e3c9e7f0 2024-05-09T04:52:51.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.514+0000 7f85e1ffb700 1 -- 172.21.15.46:0/928161122 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f0007e10 con 0x5630e3c9e7f0 2024-05-09T04:52:51.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.514+0000 7f85e1ffb700 1 -- 172.21.15.46:0/928161122 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f85f0011040 con 0x5630e3c9e7f0 2024-05-09T04:52:51.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.514+0000 7f860f1c8b80 10 client.15750 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:51.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.514+0000 7f860f1c8b80 10 client.15750 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:51.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.514+0000 7f860f1c8b80 20 client.15750 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:51.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.514+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/928161122 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5630e3ca2910 0x5630e3ca4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:51.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.514+0000 7f860f1c8b80 4 client.15750 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:52:51.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.514+0000 7f860f1c8b80 1 -- 172.21.15.46:0/928161122 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f85f402b030 con 0x5630e3ca2910 2024-05-09T04:52:51.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.516+0000 7f85fe54e700 1 --2- 172.21.15.46:0/928161122 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5630e3ca2910 0x5630e3ca4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:51.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.517+0000 7f85fe54e700 1 --2- 172.21.15.46:0/928161122 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5630e3ca2910 0x5630e3ca4d30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd9ed731d80c3e45 server_cookie=a8181298e87c64d7 in_seq=0 out_seq=0 2024-05-09T04:52:51.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.517+0000 7f85e1ffb700 10 client.15750 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.519+0000 7f85e1ffb700 1 -- 172.21.15.46:0/928161122 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f85f402b030 con 0x5630e3ca2910 2024-05-09T04:52:51.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.519+0000 7f85e1ffb700 10 client.15750 handle_command_reply: tid=0 2024-05-09T04:52:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.519+0000 7f860f1c8b80 1 client.15750 shutdown 2024-05-09T04:52:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.520+0000 7f860f1c8b80 1 -- 172.21.15.46:0/928161122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3c9e7f0 msgr2=0x5630e3ca0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.520+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/928161122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3c9e7f0 0x5630e3ca0c10 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f85f000b070 tx=0x7f85f0009450 comp rx=0 tx=0).stop 2024-05-09T04:52:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.520+0000 7f860f1c8b80 1 -- 172.21.15.46:0/928161122 shutdown_connections 2024-05-09T04:52:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.520+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/928161122 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5630e3c9e7f0 0x5630e3ca0c10 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.520+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/928161122 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5630e3ca2910 0x5630e3ca4d30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.520+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/928161122 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5630e3b791e0 0x5630e3c3a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.520+0000 7f85e1ffb700 0 client.15750 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.520+0000 7f860f1c8b80 1 --2- 172.21.15.46:0/928161122 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5630e3c0c880 0x5630e3c3ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.520+0000 7f860f1c8b80 1 -- 172.21.15.46:0/928161122 >> 172.21.15.46:0/928161122 conn(0x5630e3aef720 msgr2=0x5630e3c10740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.520+0000 7f860f1c8b80 1 -- 172.21.15.46:0/928161122 shutdown_connections 2024-05-09T04:52:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.520+0000 7f860f1c8b80 1 -- 172.21.15.46:0/928161122 wait complete. 2024-05-09T04:52:51.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.520+0000 7f860f1c8b80 20 client.15750 trim_cache size 0 max 16384 2024-05-09T04:52:51.522 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:52:51.523 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:52:51.523 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "74b24236-45ba-4b7a-b83b-6a4abea038fa", 2024-05-09T04:52:51.523 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:52:51.523 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:52:51.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.521+0000 7f85ff550700 1 -- 172.21.15.46:0/3142380322 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f85e40b2b20 msgr2=0x7f85e40b5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:51.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.521+0000 7f85ff550700 1 --2- 172.21.15.46:0/3142380322 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f85e40b2b20 0x7f85e40b5030 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f85f4006ae0 tx=0x7f85f4006a70 comp rx=0 tx=0).stop 2024-05-09T04:52:51.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.522+0000 7f85ff550700 1 -- 172.21.15.46:0/3142380322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85f810b110 msgr2=0x7f85f81a18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:51.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.522+0000 7f85ff550700 1 --2- 172.21.15.46:0/3142380322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85f810b110 0x7f85f81a18a0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f85e800c970 tx=0x7f85e800cc80 comp rx=0 tx=0).stop 2024-05-09T04:52:51.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.522+0000 7f85ff550700 1 -- 172.21.15.46:0/3142380322 shutdown_connections 2024-05-09T04:52:51.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.522+0000 7f85ff550700 1 --2- 172.21.15.46:0/3142380322 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f85e40b2b20 0x7f85e40b5030 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.522+0000 7f85ff550700 1 --2- 172.21.15.46:0/3142380322 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85f810f6c0 0x7f85f81a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.522+0000 7f85ff550700 1 --2- 172.21.15.46:0/3142380322 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85f810a7c0 0x7f85f81a1300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.522+0000 7f85ff550700 1 --2- 172.21.15.46:0/3142380322 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f85f810b110 0x7f85f81a18a0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:51.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.522+0000 7f85ff550700 1 -- 172.21.15.46:0/3142380322 >> 172.21.15.46:0/3142380322 conn(0x7f85f8076b00 msgr2=0x7f85f8108860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:51.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.522+0000 7f85ff550700 1 -- 172.21.15.46:0/3142380322 shutdown_connections 2024-05-09T04:52:51.525 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:51.522+0000 7f85ff550700 1 -- 172.21.15.46:0/3142380322 wait complete. 2024-05-09T04:52:52.259 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:52:52.259 INFO:tasks.workunit.client.0.smithi046.stdout: Children see throughput for 1 initial writers = 669724.81 kB/sec 2024-05-09T04:52:52.259 INFO:tasks.workunit.client.0.smithi046.stdout: Parent sees throughput for 1 initial writers = 669661.48 kB/sec 2024-05-09T04:52:52.259 INFO:tasks.workunit.client.0.smithi046.stdout: Min throughput per process = 669724.81 kB/sec 2024-05-09T04:52:52.259 INFO:tasks.workunit.client.0.smithi046.stdout: Max throughput per process = 669724.81 kB/sec 2024-05-09T04:52:52.259 INFO:tasks.workunit.client.0.smithi046.stdout: Avg throughput per process = 669724.81 kB/sec 2024-05-09T04:52:52.259 INFO:tasks.workunit.client.0.smithi046.stdout: Min xfer = 1048576.00 kB 2024-05-09T04:52:53.011 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:52 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:52:53.011 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:52 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:53.012 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:52 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:52:53.012 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:52 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:53.012 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:52 smithi046 ceph-mon[30136]: pgmap v56: 97 pgs: 97 active+clean; 533 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 44 MiB/s wr, 22 op/s 2024-05-09T04:52:53.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:52 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:52:53.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:52 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:53.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:52 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:52:53.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:52 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:53.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:52 smithi110 ceph-mon[34904]: pgmap v56: 97 pgs: 97 active+clean; 533 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 44 MiB/s wr, 22 op/s 2024-05-09T04:52:53.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:52 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:52:53.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:52 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:53.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:52 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:52:53.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:52 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:52:53.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:52 smithi072 ceph-mon[34088]: pgmap v56: 97 pgs: 97 active+clean; 533 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 44 MiB/s wr, 22 op/s 2024-05-09T04:52:53.926 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:52:54.226 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:55.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:54 smithi110 ceph-mon[34904]: pgmap v57: 97 pgs: 97 active+clean; 533 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 44 MiB/s wr, 19 op/s 2024-05-09T04:52:55.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:54 smithi046 ceph-mon[30136]: pgmap v57: 97 pgs: 97 active+clean; 533 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 44 MiB/s wr, 19 op/s 2024-05-09T04:52:55.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:54 smithi072 ceph-mon[34088]: pgmap v57: 97 pgs: 97 active+clean; 533 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 44 MiB/s wr, 19 op/s 2024-05-09T04:52:56.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.206+0000 7fb91f980700 1 -- 172.21.15.46:0/2661141450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb918107df0 msgr2=0x7fb918108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:56.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.206+0000 7fb91f980700 1 --2- 172.21.15.46:0/2661141450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb918107df0 0x7fb918108200 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0099c0 tx=0x7fb90c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.207+0000 7fb91f980700 1 -- 172.21.15.46:0/2661141450 shutdown_connections 2024-05-09T04:52:56.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.207+0000 7fb91f980700 1 --2- 172.21.15.46:0/2661141450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb918108740 0x7fb9181100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.207+0000 7fb91f980700 1 --2- 172.21.15.46:0/2661141450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb9181034c0 0x7fb9181038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.207+0000 7fb91f980700 1 --2- 172.21.15.46:0/2661141450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb918107df0 0x7fb918108200 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.207+0000 7fb91f980700 1 -- 172.21.15.46:0/2661141450 >> 172.21.15.46:0/2661141450 conn(0x7fb9180fedf0 msgr2=0x7fb918101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:56.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.207+0000 7fb91f980700 1 -- 172.21.15.46:0/2661141450 shutdown_connections 2024-05-09T04:52:56.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.207+0000 7fb91f980700 1 -- 172.21.15.46:0/2661141450 wait complete. 2024-05-09T04:52:56.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.207+0000 7fb91f980700 1 Processor -- start 2024-05-09T04:52:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.207+0000 7fb91f980700 1 -- start start 2024-05-09T04:52:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91f980700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9181034c0 0x7fb91819aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91f980700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb918107df0 0x7fb91819b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91f980700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb918108740 0x7fb91819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91f980700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb918113350 con 0x7fb9181034c0 2024-05-09T04:52:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91f980700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb9181131d0 con 0x7fb918108740 2024-05-09T04:52:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91f980700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb9181134d0 con 0x7fb918107df0 2024-05-09T04:52:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91d71c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9181034c0 0x7fb91819aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91d71c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9181034c0 0x7fb91819aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44310/0 (socket says 172.21.15.46:44310) 2024-05-09T04:52:56.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91d71c700 1 -- 172.21.15.46:0/1527691131 learned_addr learned my addr 172.21.15.46:0/1527691131 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91df1d700 1 --2- 172.21.15.46:0/1527691131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb918108740 0x7fb91819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91cf1b700 1 --2- 172.21.15.46:0/1527691131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb918107df0 0x7fb91819b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91d71c700 1 -- 172.21.15.46:0/1527691131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb918107df0 msgr2=0x7fb91819b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91d71c700 1 --2- 172.21.15.46:0/1527691131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb918107df0 0x7fb91819b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91d71c700 1 -- 172.21.15.46:0/1527691131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb918108740 msgr2=0x7fb91819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91d71c700 1 --2- 172.21.15.46:0/1527691131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb918108740 0x7fb91819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.208+0000 7fb91d71c700 1 -- 172.21.15.46:0/1527691131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb91819fd50 con 0x7fb9181034c0 2024-05-09T04:52:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.209+0000 7fb91d71c700 1 --2- 172.21.15.46:0/1527691131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9181034c0 0x7fb91819aed0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb9080060b0 tx=0x7fb90800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.209+0000 7fb9167fc700 1 -- 172.21.15.46:0/1527691131 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90801a070 con 0x7fb9181034c0 2024-05-09T04:52:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.209+0000 7fb91f980700 1 -- 172.21.15.46:0/1527691131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9181a0040 con 0x7fb9181034c0 2024-05-09T04:52:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.209+0000 7fb9167fc700 1 -- 172.21.15.46:0/1527691131 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb90800ddd0 con 0x7fb9181034c0 2024-05-09T04:52:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.209+0000 7fb9167fc700 1 -- 172.21.15.46:0/1527691131 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb908014600 con 0x7fb9181034c0 2024-05-09T04:52:56.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.209+0000 7fb91f980700 1 -- 172.21.15.46:0/1527691131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9181a7960 con 0x7fb9181034c0 2024-05-09T04:52:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.211+0000 7fb9167fc700 1 -- 172.21.15.46:0/1527691131 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb90801d070 con 0x7fb9181034c0 2024-05-09T04:52:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.212+0000 7fb9167fc700 1 --2- 172.21.15.46:0/1527691131 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb9000b29f0 0x7fb9000b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.212+0000 7fb91cf1b700 1 --2- 172.21.15.46:0/1527691131 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb9000b29f0 0x7fb9000b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.212+0000 7fb9167fc700 1 -- 172.21.15.46:0/1527691131 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fb9080e1e30 con 0x7fb9181034c0 2024-05-09T04:52:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.213+0000 7fb92e396b80 1 Processor -- start 2024-05-09T04:52:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.213+0000 7fb91cf1b700 1 --2- 172.21.15.46:0/1527691131 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb9000b29f0 0x7fb9000b4f00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb90c006010 tx=0x7fb90c000bc0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.213+0000 7fb92e396b80 1 -- start start 2024-05-09T04:52:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.213+0000 7fb92e396b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca511d00 0x5559ca61a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.213+0000 7fb92e396b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca4974a0 0x5559ca61a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.213+0000 7fb92e396b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca614400 0x5559ca614810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.213+0000 7fb92e396b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5559ca5370a0 con 0x5559ca614400 2024-05-09T04:52:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.213+0000 7fb92e396b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5559ca4dc9a0 con 0x5559ca4974a0 2024-05-09T04:52:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.213+0000 7fb92e396b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5559ca533060 con 0x5559ca511d00 2024-05-09T04:52:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.213+0000 7fb91df1d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca614400 0x5559ca614810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.213+0000 7fb91df1d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca614400 0x5559ca614810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44326/0 (socket says 172.21.15.46:44326) 2024-05-09T04:52:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.213+0000 7fb91df1d700 1 -- 172.21.15.46:0/1410005896 learned_addr learned my addr 172.21.15.46:0/1410005896 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.213+0000 7fb91d71c700 1 --2- 172.21.15.46:0/1410005896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca511d00 0x5559ca61a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.213+0000 7fb91cf1b700 1 --2- 172.21.15.46:0/1410005896 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca4974a0 0x5559ca61a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.214+0000 7fb91df1d700 1 -- 172.21.15.46:0/1410005896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca511d00 msgr2=0x5559ca61a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.214+0000 7fb91df1d700 1 --2- 172.21.15.46:0/1410005896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca511d00 0x5559ca61a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.214+0000 7fb91df1d700 1 -- 172.21.15.46:0/1410005896 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca4974a0 msgr2=0x5559ca61a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.214+0000 7fb91df1d700 1 --2- 172.21.15.46:0/1410005896 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca4974a0 0x5559ca61a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.214+0000 7fb91df1d700 1 -- 172.21.15.46:0/1410005896 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ca4cd500 con 0x5559ca614400 2024-05-09T04:52:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.214+0000 7fb91cf1b700 1 --2- 172.21.15.46:0/1410005896 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca4974a0 0x5559ca61a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:52:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.214+0000 7fb91df1d700 1 --2- 172.21.15.46:0/1410005896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca614400 0x5559ca614810 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb910007c10 tx=0x7fb910007f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ef1e7be1bbbb0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:56.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.214+0000 7fb8ff7fe700 1 -- 172.21.15.46:0/1410005896 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb910010b80 con 0x5559ca614400 2024-05-09T04:52:56.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.214+0000 7fb8ff7fe700 1 -- 172.21.15.46:0/1410005896 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb910010d20 con 0x5559ca614400 2024-05-09T04:52:56.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.214+0000 7fb8ff7fe700 1 -- 172.21.15.46:0/1410005896 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb910019670 con 0x5559ca614400 2024-05-09T04:52:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.215+0000 7fb92e396b80 1 -- 172.21.15.46:0/1410005896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca614400 msgr2=0x5559ca614810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.215+0000 7fb92e396b80 1 --2- 172.21.15.46:0/1410005896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca614400 0x5559ca614810 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb910007c10 tx=0x7fb910007f20 comp rx=0 tx=0).stop 2024-05-09T04:52:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.215+0000 7fb92e396b80 1 -- 172.21.15.46:0/1410005896 shutdown_connections 2024-05-09T04:52:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.215+0000 7fb92e396b80 1 --2- 172.21.15.46:0/1410005896 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca614400 0x5559ca614810 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.215+0000 7fb92e396b80 1 --2- 172.21.15.46:0/1410005896 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca511d00 0x5559ca61a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.215+0000 7fb92e396b80 1 --2- 172.21.15.46:0/1410005896 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca4974a0 0x5559ca61a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.215+0000 7fb92e396b80 1 -- 172.21.15.46:0/1410005896 >> 172.21.15.46:0/1410005896 conn(0x5559ca49af70 msgr2=0x5559ca60f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.215+0000 7fb92e396b80 1 -- 172.21.15.46:0/1410005896 shutdown_connections 2024-05-09T04:52:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.215+0000 7fb92e396b80 1 -- 172.21.15.46:0/1410005896 wait complete. 2024-05-09T04:52:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.215+0000 7fb92e396b80 1 -- start start 2024-05-09T04:52:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.216+0000 7fb92e396b80 1 Processor -- start 2024-05-09T04:52:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.216+0000 7fb92e396b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca4974a0 0x5559ca61e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.216+0000 7fb92e396b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca511d00 0x5559ca61e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.216+0000 7fb92e396b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca614400 0x5559ca61ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.216+0000 7fb92e396b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5559ca615160 con 0x5559ca511d00 2024-05-09T04:52:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.216+0000 7fb92e396b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5559ca4d1b20 con 0x5559ca4974a0 2024-05-09T04:52:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.216+0000 7fb92e396b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5559ca621020 con 0x5559ca614400 2024-05-09T04:52:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.216+0000 7fb92e396b80 10 client.0 _ll_register_callbacks cb 0x5559ca3a86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.216+0000 7fb91cf1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca511d00 0x5559ca61e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.216+0000 7fb91cf1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca511d00 0x5559ca61e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44328/0 (socket says 172.21.15.46:44328) 2024-05-09T04:52:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.216+0000 7fb91cf1b700 1 -- 172.21.15.46:0/1740201734 learned_addr learned my addr 172.21.15.46:0/1740201734 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.216+0000 7fb91df1d700 1 --2- 172.21.15.46:0/1740201734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca614400 0x5559ca61ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.216+0000 7fb91d71c700 1 --2- 172.21.15.46:0/1740201734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca4974a0 0x5559ca61e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.217+0000 7fb91cf1b700 1 -- 172.21.15.46:0/1740201734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca614400 msgr2=0x5559ca61ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.217+0000 7fb91cf1b700 1 --2- 172.21.15.46:0/1740201734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca614400 0x5559ca61ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.217+0000 7fb91cf1b700 1 -- 172.21.15.46:0/1740201734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca4974a0 msgr2=0x5559ca61e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.217+0000 7fb91cf1b700 1 --2- 172.21.15.46:0/1740201734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca4974a0 0x5559ca61e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.217+0000 7fb91cf1b700 1 -- 172.21.15.46:0/1740201734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559ca3b2490 con 0x5559ca511d00 2024-05-09T04:52:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.217+0000 7fb91cf1b700 1 --2- 172.21.15.46:0/1740201734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca511d00 0x5559ca61e6b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0099c0 tx=0x7fb90c009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee009f329da8a25d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.217+0000 7fb8fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.217+0000 7fb8fcff9700 1 -- 172.21.15.46:0/1740201734 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb90c034060 con 0x5559ca511d00 2024-05-09T04:52:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.217+0000 7fb8fcff9700 1 -- 172.21.15.46:0/1740201734 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fb90c00bdf0 con 0x5559ca511d00 2024-05-09T04:52:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.217+0000 7fb92e396b80 1 -- 172.21.15.46:0/1740201734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb910010d20 con 0x5559ca511d00 2024-05-09T04:52:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.217+0000 7fb8fcff9700 1 -- 172.21.15.46:0/1740201734 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb90c00f460 con 0x5559ca511d00 2024-05-09T04:52:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.217+0000 7fb92e396b80 10 client.15762 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.217+0000 7fb92e396b80 1 -- 172.21.15.46:0/1740201734 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5559ca62bb50 con 0x5559ca511d00 2024-05-09T04:52:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.217+0000 7fb8fcff9700 1 -- 172.21.15.46:0/1740201734 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb910010d20 con 0x5559ca511d00 2024-05-09T04:52:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.218+0000 7fb8fcff9700 1 -- 172.21.15.46:0/1740201734 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fb90c02e070 con 0x5559ca511d00 2024-05-09T04:52:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.218+0000 7fb92e396b80 10 client.15762 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.218+0000 7fb92e396b80 10 client.15762 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.218+0000 7fb92e396b80 20 client.15762 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:56.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.218+0000 7fb92e396b80 1 --2- 172.21.15.46:0/1740201734 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5559ca62e190 0x5559ca630610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.218+0000 7fb92e396b80 4 client.15762 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:52:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.218+0000 7fb92e396b80 1 -- 172.21.15.46:0/1740201734 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5559ca544a80 con 0x5559ca62e190 2024-05-09T04:52:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.218+0000 7fb91d71c700 1 --2- 172.21.15.46:0/1740201734 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5559ca62e190 0x5559ca630610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.218+0000 7fb8fcff9700 10 client.15762 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:56.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.218+0000 7fb91d71c700 1 --2- 172.21.15.46:0/1740201734 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5559ca62e190 0x5559ca630610 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e45edba28de9f1f server_cookie=d627866d213a0eaa in_seq=0 out_seq=0 2024-05-09T04:52:56.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.219+0000 7fb8fcff9700 1 -- 172.21.15.46:0/1740201734 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5559ca544a80 con 0x5559ca62e190 2024-05-09T04:52:56.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.219+0000 7fb8fcff9700 10 client.15762 handle_command_reply: tid=0 2024-05-09T04:52:56.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.219+0000 7fb92e396b80 1 client.15762 shutdown 2024-05-09T04:52:56.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.219+0000 7fb92e396b80 1 -- 172.21.15.46:0/1740201734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca511d00 msgr2=0x5559ca61e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:56.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.219+0000 7fb92e396b80 1 --2- 172.21.15.46:0/1740201734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca511d00 0x5559ca61e6b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0099c0 tx=0x7fb90c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.219+0000 7fb92e396b80 1 -- 172.21.15.46:0/1740201734 shutdown_connections 2024-05-09T04:52:56.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.219+0000 7fb92e396b80 1 --2- 172.21.15.46:0/1740201734 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca614400 0x5559ca61ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.219+0000 7fb92e396b80 1 --2- 172.21.15.46:0/1740201734 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5559ca62e190 0x5559ca630610 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.219+0000 7fb92e396b80 1 --2- 172.21.15.46:0/1740201734 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca4974a0 0x5559ca61e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.219+0000 7fb92e396b80 1 --2- 172.21.15.46:0/1740201734 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca511d00 0x5559ca61e6b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.220+0000 7fb92e396b80 1 -- 172.21.15.46:0/1740201734 >> 172.21.15.46:0/1740201734 conn(0x5559ca49af70 msgr2=0x5559ca608ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:56.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.220+0000 7fb8fcff9700 0 client.15762 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:56.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.220+0000 7fb92e396b80 1 -- 172.21.15.46:0/1740201734 shutdown_connections 2024-05-09T04:52:56.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.220+0000 7fb92e396b80 1 -- 172.21.15.46:0/1740201734 wait complete. 2024-05-09T04:52:56.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.220+0000 7fb92e396b80 20 client.15762 trim_cache size 0 max 16384 2024-05-09T04:52:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.240+0000 7fb92e396b80 1 Processor -- start 2024-05-09T04:52:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.240+0000 7fb92e396b80 1 -- start start 2024-05-09T04:52:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.240+0000 7fb92e396b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca4974a0 0x5559ca609700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.240+0000 7fb92e396b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca511d00 0x5559ca629e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.240+0000 7fb92e396b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca614400 0x5559ca62a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.240+0000 7fb92e396b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb9181b1790 con 0x5559ca4974a0 2024-05-09T04:52:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.240+0000 7fb92e396b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb9181af020 con 0x5559ca614400 2024-05-09T04:52:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.240+0000 7fb92e396b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb9181ace70 con 0x5559ca511d00 2024-05-09T04:52:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.241+0000 7fb91d71c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca4974a0 0x5559ca609700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.241+0000 7fb91d71c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca4974a0 0x5559ca609700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44344/0 (socket says 172.21.15.46:44344) 2024-05-09T04:52:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.241+0000 7fb91d71c700 1 -- 172.21.15.46:0/2989758660 learned_addr learned my addr 172.21.15.46:0/2989758660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.241+0000 7fb91d71c700 1 -- 172.21.15.46:0/2989758660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca511d00 msgr2=0x5559ca629e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.241+0000 7fb91cf1b700 1 --2- 172.21.15.46:0/2989758660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca511d00 0x5559ca629e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.241+0000 7fb91d71c700 1 --2- 172.21.15.46:0/2989758660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca511d00 0x5559ca629e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.241+0000 7fb91df1d700 1 --2- 172.21.15.46:0/2989758660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca614400 0x5559ca62a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.241+0000 7fb91d71c700 1 -- 172.21.15.46:0/2989758660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca614400 msgr2=0x5559ca62a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.241+0000 7fb91d71c700 1 --2- 172.21.15.46:0/2989758660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca614400 0x5559ca62a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.241+0000 7fb91d71c700 1 -- 172.21.15.46:0/2989758660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5559ca60eb80 con 0x5559ca4974a0 2024-05-09T04:52:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.241+0000 7fb91d71c700 1 --2- 172.21.15.46:0/2989758660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca4974a0 0x5559ca609700 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb9080a6300 tx=0x7fb908021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6384ef25cb0663c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:56.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.241+0000 7fb8feffd700 1 -- 172.21.15.46:0/2989758660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9080c8070 con 0x5559ca4974a0 2024-05-09T04:52:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.241+0000 7fb8feffd700 1 -- 172.21.15.46:0/2989758660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9080b1ef0 con 0x5559ca4974a0 2024-05-09T04:52:56.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.241+0000 7fb8feffd700 1 -- 172.21.15.46:0/2989758660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9080c2070 con 0x5559ca4974a0 2024-05-09T04:52:56.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.242+0000 7fb92e396b80 1 -- 172.21.15.46:0/2989758660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca4974a0 msgr2=0x5559ca609700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:56.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.242+0000 7fb92e396b80 1 --2- 172.21.15.46:0/2989758660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca4974a0 0x5559ca609700 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fb9080a6300 tx=0x7fb908021040 comp rx=0 tx=0).stop 2024-05-09T04:52:56.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.242+0000 7fb92e396b80 1 -- 172.21.15.46:0/2989758660 shutdown_connections 2024-05-09T04:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.242+0000 7fb92e396b80 1 --2- 172.21.15.46:0/2989758660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca614400 0x5559ca62a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.242+0000 7fb92e396b80 1 --2- 172.21.15.46:0/2989758660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca4974a0 0x5559ca609700 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.242+0000 7fb92e396b80 1 --2- 172.21.15.46:0/2989758660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca511d00 0x5559ca629e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.242+0000 7fb92e396b80 1 -- 172.21.15.46:0/2989758660 >> 172.21.15.46:0/2989758660 conn(0x5559ca49af70 msgr2=0x5559ca608ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.242+0000 7fb92e396b80 1 -- 172.21.15.46:0/2989758660 shutdown_connections 2024-05-09T04:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.242+0000 7fb92e396b80 1 -- 172.21.15.46:0/2989758660 wait complete. 2024-05-09T04:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.242+0000 7fb92e396b80 1 -- start start 2024-05-09T04:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.243+0000 7fb92e396b80 1 Processor -- start 2024-05-09T04:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.243+0000 7fb92e396b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca4974a0 0x5559ca6356c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.243+0000 7fb92e396b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca511d00 0x5559ca635c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.243+0000 7fb92e396b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca614400 0x5559ca636140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.243+0000 7fb92e396b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb9181ad4e0 con 0x5559ca4974a0 2024-05-09T04:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.243+0000 7fb92e396b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb9181ae950 con 0x5559ca511d00 2024-05-09T04:52:56.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.243+0000 7fb92e396b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb9181adbb0 con 0x5559ca614400 2024-05-09T04:52:56.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.243+0000 7fb92e396b80 10 client.0 _ll_register_callbacks cb 0x5559ca614dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:52:56.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.243+0000 7fb91d71c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca4974a0 0x5559ca6356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.243+0000 7fb91d71c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca4974a0 0x5559ca6356c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44354/0 (socket says 172.21.15.46:44354) 2024-05-09T04:52:56.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.243+0000 7fb91d71c700 1 -- 172.21.15.46:0/613006829 learned_addr learned my addr 172.21.15.46:0/613006829 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:56.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.243+0000 7fb91cf1b700 1 --2- 172.21.15.46:0/613006829 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca511d00 0x5559ca635c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.243+0000 7fb91df1d700 1 --2- 172.21.15.46:0/613006829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca614400 0x5559ca636140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.244+0000 7fb91d71c700 1 -- 172.21.15.46:0/613006829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca614400 msgr2=0x5559ca636140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:56.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.244+0000 7fb91d71c700 1 --2- 172.21.15.46:0/613006829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca614400 0x5559ca636140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.244+0000 7fb91d71c700 1 -- 172.21.15.46:0/613006829 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca511d00 msgr2=0x5559ca635c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:52:56.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.244+0000 7fb91d71c700 1 --2- 172.21.15.46:0/613006829 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca511d00 0x5559ca635c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.244+0000 7fb91d71c700 1 -- 172.21.15.46:0/613006829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559ca6979c0 con 0x5559ca4974a0 2024-05-09T04:52:56.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.244+0000 7fb91d71c700 1 --2- 172.21.15.46:0/613006829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca4974a0 0x5559ca6356c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb9080ad600 tx=0x7fb9080dfa60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5454d1edca552a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:56.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.244+0000 7fb8fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:52:56.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.244+0000 7fb8fcff9700 1 -- 172.21.15.46:0/613006829 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9080c8070 con 0x5559ca4974a0 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.244+0000 7fb8fcff9700 1 -- 172.21.15.46:0/613006829 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fb9080b1a60 con 0x5559ca4974a0 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.244+0000 7fb92e396b80 1 -- 172.21.15.46:0/613006829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9080b1ef0 con 0x5559ca4974a0 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.244+0000 7fb8fcff9700 1 -- 172.21.15.46:0/613006829 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb9080c2070 con 0x5559ca4974a0 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.244+0000 7fb92e396b80 10 client.15768 fetch_fsmap learned FSMap version 25 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.244+0000 7fb92e396b80 1 -- 172.21.15.46:0/613006829 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5559ca698bb0 con 0x5559ca4974a0 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.244+0000 7fb8fcff9700 1 -- 172.21.15.46:0/613006829 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9080b1ef0 con 0x5559ca4974a0 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.245+0000 7fb8fcff9700 1 -- 172.21.15.46:0/613006829 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fb90808f180 con 0x5559ca4974a0 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.245+0000 7fb92e396b80 10 client.15768 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.245+0000 7fb92e396b80 10 client.15768 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.245+0000 7fb92e396b80 20 client.15768 populate_metadata read hostname 'smithi046' 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.245+0000 7fb92e396b80 1 --2- 172.21.15.46:0/613006829 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5559ca699260 0x5559ca69b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.245+0000 7fb92e396b80 4 client.15768 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.245+0000 7fb92e396b80 1 -- 172.21.15.46:0/613006829 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5559ca53eb20 con 0x5559ca699260 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.245+0000 7fb91cf1b700 1 --2- 172.21.15.46:0/613006829 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5559ca699260 0x5559ca69b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.245+0000 7fb91cf1b700 1 --2- 172.21.15.46:0/613006829 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5559ca699260 0x5559ca69b680 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5e9423cd29050d server_cookie=d324812436a66b1f in_seq=0 out_seq=0 2024-05-09T04:52:56.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.245+0000 7fb8fcff9700 10 client.15768 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:56.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.246+0000 7fb8fcff9700 1 -- 172.21.15.46:0/613006829 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5559ca53eb20 con 0x5559ca699260 2024-05-09T04:52:56.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.246+0000 7fb8fcff9700 10 client.15768 handle_command_reply: tid=0 2024-05-09T04:52:56.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.246+0000 7fb92e396b80 1 client.15768 shutdown 2024-05-09T04:52:56.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.246+0000 7fb92e396b80 1 -- 172.21.15.46:0/613006829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca4974a0 msgr2=0x5559ca6356c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:56.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.246+0000 7fb92e396b80 1 --2- 172.21.15.46:0/613006829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca4974a0 0x5559ca6356c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb9080ad600 tx=0x7fb9080dfa60 comp rx=0 tx=0).stop 2024-05-09T04:52:56.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.246+0000 7fb92e396b80 1 -- 172.21.15.46:0/613006829 shutdown_connections 2024-05-09T04:52:56.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.246+0000 7fb92e396b80 1 --2- 172.21.15.46:0/613006829 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5559ca614400 0x5559ca636140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.246+0000 7fb92e396b80 1 --2- 172.21.15.46:0/613006829 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5559ca4974a0 0x5559ca6356c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.246+0000 7fb92e396b80 1 --2- 172.21.15.46:0/613006829 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5559ca699260 0x5559ca69b680 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.246+0000 7fb92e396b80 1 --2- 172.21.15.46:0/613006829 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5559ca511d00 0x5559ca635c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.246+0000 7fb8fcff9700 0 client.15768 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:52:56.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.246+0000 7fb92e396b80 1 -- 172.21.15.46:0/613006829 >> 172.21.15.46:0/613006829 conn(0x5559ca49af70 msgr2=0x5559ca61e0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:56.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.246+0000 7fb92e396b80 1 -- 172.21.15.46:0/613006829 shutdown_connections 2024-05-09T04:52:56.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.246+0000 7fb92e396b80 1 -- 172.21.15.46:0/613006829 wait complete. 2024-05-09T04:52:56.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.246+0000 7fb92e396b80 20 client.15768 trim_cache size 0 max 16384 2024-05-09T04:52:56.251 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:52:56.251 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:52:56.251 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:52:56.251 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:52:56.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.247+0000 7fb91f980700 1 -- 172.21.15.46:0/1527691131 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb9000b29f0 msgr2=0x7fb9000b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:56.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.247+0000 7fb91f980700 1 --2- 172.21.15.46:0/1527691131 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb9000b29f0 0x7fb9000b4f00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb90c006010 tx=0x7fb90c000bc0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.247+0000 7fb91f980700 1 -- 172.21.15.46:0/1527691131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9181034c0 msgr2=0x7fb91819aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:56.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.247+0000 7fb91f980700 1 --2- 172.21.15.46:0/1527691131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9181034c0 0x7fb91819aed0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb9080060b0 tx=0x7fb90800d3b0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.247+0000 7fb91f980700 1 -- 172.21.15.46:0/1527691131 shutdown_connections 2024-05-09T04:52:56.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.247+0000 7fb91f980700 1 --2- 172.21.15.46:0/1527691131 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb9000b29f0 0x7fb9000b4f00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.247+0000 7fb91f980700 1 --2- 172.21.15.46:0/1527691131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb918108740 0x7fb91819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.247+0000 7fb91f980700 1 --2- 172.21.15.46:0/1527691131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb9181034c0 0x7fb91819aed0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.247+0000 7fb91f980700 1 --2- 172.21.15.46:0/1527691131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb918107df0 0x7fb91819b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:56.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.247+0000 7fb91f980700 1 -- 172.21.15.46:0/1527691131 >> 172.21.15.46:0/1527691131 conn(0x7fb9180fedf0 msgr2=0x7fb9181096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:56.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.247+0000 7fb91f980700 1 -- 172.21.15.46:0/1527691131 shutdown_connections 2024-05-09T04:52:56.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:56.247+0000 7fb91f980700 1 -- 172.21.15.46:0/1527691131 wait complete. 2024-05-09T04:52:56.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:56 smithi046 ceph-mon[30136]: pgmap v58: 97 pgs: 97 active+clean; 533 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 44 MiB/s wr, 19 op/s 2024-05-09T04:52:56.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:56 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:52:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:56 smithi110 ceph-mon[34904]: pgmap v58: 97 pgs: 97 active+clean; 533 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 44 MiB/s wr, 19 op/s 2024-05-09T04:52:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:56 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:52:57.105 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:52:57.106 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:52:57.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:56 smithi072 ceph-mon[34088]: pgmap v58: 97 pgs: 97 active+clean; 533 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 44 MiB/s wr, 19 op/s 2024-05-09T04:52:57.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:56 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:52:57.441 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:52:57.770 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:52:57.770 INFO:tasks.workunit.client.0.smithi046.stdout: Children see throughput for 1 rewriters = 731994.81 kB/sec 2024-05-09T04:52:57.770 INFO:tasks.workunit.client.0.smithi046.stdout: Parent sees throughput for 1 rewriters = 731893.71 kB/sec 2024-05-09T04:52:57.770 INFO:tasks.workunit.client.0.smithi046.stdout: Min throughput per process = 731994.81 kB/sec 2024-05-09T04:52:57.770 INFO:tasks.workunit.client.0.smithi046.stdout: Max throughput per process = 731994.81 kB/sec 2024-05-09T04:52:57.770 INFO:tasks.workunit.client.0.smithi046.stdout: Avg throughput per process = 731994.81 kB/sec 2024-05-09T04:52:57.770 INFO:tasks.workunit.client.0.smithi046.stdout: Min xfer = 1048576.00 kB 2024-05-09T04:52:59.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:52:58 smithi110 ceph-mon[34904]: pgmap v59: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 85 MiB/s wr, 32 op/s 2024-05-09T04:52:59.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:52:58 smithi046 ceph-mon[30136]: pgmap v59: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 85 MiB/s wr, 32 op/s 2024-05-09T04:52:59.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:52:58 smithi072 ceph-mon[34088]: pgmap v59: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 85 MiB/s wr, 32 op/s 2024-05-09T04:52:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.884+0000 7fc28ace0700 1 -- 172.21.15.46:0/2060220506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc284105480 msgr2=0x7fc284105890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.884+0000 7fc28ace0700 1 --2- 172.21.15.46:0/2060220506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc284105480 0x7fc284105890 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fc274009990 tx=0x7fc274009ca0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.884+0000 7fc28ace0700 1 -- 172.21.15.46:0/2060220506 shutdown_connections 2024-05-09T04:52:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.884+0000 7fc28ace0700 1 --2- 172.21.15.46:0/2060220506 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc28410e8f0 0x7fc284110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.884+0000 7fc28ace0700 1 --2- 172.21.15.46:0/2060220506 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc284105480 0x7fc284105890 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.884+0000 7fc28ace0700 1 --2- 172.21.15.46:0/2060220506 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc284105dd0 0x7fc28410e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.884+0000 7fc28ace0700 1 -- 172.21.15.46:0/2060220506 >> 172.21.15.46:0/2060220506 conn(0x7fc2840fef30 msgr2=0x7fc284101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:52:59.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.884+0000 7fc28ace0700 1 -- 172.21.15.46:0/2060220506 shutdown_connections 2024-05-09T04:52:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.884+0000 7fc28ace0700 1 -- 172.21.15.46:0/2060220506 wait complete. 2024-05-09T04:52:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.885+0000 7fc28ace0700 1 Processor -- start 2024-05-09T04:52:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.885+0000 7fc28ace0700 1 -- start start 2024-05-09T04:52:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc28ace0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc284105480 0x7fc284103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc28ace0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc284105dd0 0x7fc2841036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc28ace0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc28410e8f0 0x7fc284103bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc28ace0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc284114490 con 0x7fc284105480 2024-05-09T04:52:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc28ace0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc284114310 con 0x7fc284105dd0 2024-05-09T04:52:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc28ace0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc284114610 con 0x7fc28410e8f0 2024-05-09T04:52:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc288a7c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc284105480 0x7fc284103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc288a7c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc284105480 0x7fc284103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44378/0 (socket says 172.21.15.46:44378) 2024-05-09T04:52:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc288a7c700 1 -- 172.21.15.46:0/1493172132 learned_addr learned my addr 172.21.15.46:0/1493172132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:52:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc283fff700 1 --2- 172.21.15.46:0/1493172132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc284105dd0 0x7fc2841036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc28927d700 1 --2- 172.21.15.46:0/1493172132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc28410e8f0 0x7fc284103bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:59.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc288a7c700 1 -- 172.21.15.46:0/1493172132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc28410e8f0 msgr2=0x7fc284103bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc288a7c700 1 --2- 172.21.15.46:0/1493172132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc28410e8f0 0x7fc284103bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc288a7c700 1 -- 172.21.15.46:0/1493172132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc284105dd0 msgr2=0x7fc2841036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:52:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc288a7c700 1 --2- 172.21.15.46:0/1493172132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc284105dd0 0x7fc2841036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:52:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc288a7c700 1 -- 172.21.15.46:0/1493172132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2841a91c0 con 0x7fc284105480 2024-05-09T04:52:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.886+0000 7fc288a7c700 1 --2- 172.21.15.46:0/1493172132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc284105480 0x7fc284103170 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc274005910 tx=0x7fc2740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.887+0000 7fc281ffb700 1 -- 172.21.15.46:0/1493172132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274027070 con 0x7fc284105480 2024-05-09T04:52:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.887+0000 7fc281ffb700 1 -- 172.21.15.46:0/1493172132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc274003f20 con 0x7fc284105480 2024-05-09T04:52:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.887+0000 7fc28ace0700 1 -- 172.21.15.46:0/1493172132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2841a9450 con 0x7fc284105480 2024-05-09T04:52:59.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.887+0000 7fc281ffb700 1 -- 172.21.15.46:0/1493172132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc274022c70 con 0x7fc284105480 2024-05-09T04:52:59.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.887+0000 7fc28ace0700 1 -- 172.21.15.46:0/1493172132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2841a98b0 con 0x7fc284105480 2024-05-09T04:52:59.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.888+0000 7fc28ace0700 1 -- 172.21.15.46:0/1493172132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2840512c0 con 0x7fc284105480 2024-05-09T04:52:59.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.889+0000 7fc281ffb700 1 -- 172.21.15.46:0/1493172132 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc27402a070 con 0x7fc284105480 2024-05-09T04:52:59.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.890+0000 7fc281ffb700 1 --2- 172.21.15.46:0/1493172132 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc26c0b2a70 0x7fc26c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:52:59.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.890+0000 7fc283fff700 1 --2- 172.21.15.46:0/1493172132 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc26c0b2a70 0x7fc26c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:52:59.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.890+0000 7fc281ffb700 1 -- 172.21.15.46:0/1493172132 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fc2740e3460 con 0x7fc284105480 2024-05-09T04:52:59.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.891+0000 7fc283fff700 1 --2- 172.21.15.46:0/1493172132 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc26c0b2a70 0x7fc26c0b4f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc2841024e0 tx=0x7fc2780073d0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:52:59.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:52:59.895+0000 7fc281ffb700 1 -- 172.21.15.46:0/1493172132 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc274013070 con 0x7fc284105480 2024-05-09T04:53:00.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:00.158+0000 7fc28ace0700 1 -- 172.21.15.46:0/1493172132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc28402e5f0 con 0x7fc284105480 2024-05-09T04:53:00.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:00.159+0000 7fc281ffb700 1 -- 172.21.15.46:0/1493172132 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7fc2740aaeb0 con 0x7fc284105480 2024-05-09T04:53:00.159 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:53:00.160 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:53:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:00.161+0000 7fc28ace0700 1 -- 172.21.15.46:0/1493172132 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc26c0b2a70 msgr2=0x7fc26c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:00.161+0000 7fc28ace0700 1 --2- 172.21.15.46:0/1493172132 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc26c0b2a70 0x7fc26c0b4f80 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fc2841024e0 tx=0x7fc2780073d0 comp rx=0 tx=0).stop 2024-05-09T04:53:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:00.161+0000 7fc28ace0700 1 -- 172.21.15.46:0/1493172132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc284105480 msgr2=0x7fc284103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:00.161+0000 7fc28ace0700 1 --2- 172.21.15.46:0/1493172132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc284105480 0x7fc284103170 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fc274005910 tx=0x7fc2740047b0 comp rx=0 tx=0).stop 2024-05-09T04:53:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:00.161+0000 7fc28ace0700 1 -- 172.21.15.46:0/1493172132 shutdown_connections 2024-05-09T04:53:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:00.161+0000 7fc28ace0700 1 --2- 172.21.15.46:0/1493172132 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc26c0b2a70 0x7fc26c0b4f80 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:00.161+0000 7fc28ace0700 1 --2- 172.21.15.46:0/1493172132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc28410e8f0 0x7fc284103bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:00.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:00.161+0000 7fc28ace0700 1 --2- 172.21.15.46:0/1493172132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc284105480 0x7fc284103170 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:00.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:00.161+0000 7fc28ace0700 1 --2- 172.21.15.46:0/1493172132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc284105dd0 0x7fc2841036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:00.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:00.162+0000 7fc28ace0700 1 -- 172.21.15.46:0/1493172132 >> 172.21.15.46:0/1493172132 conn(0x7fc2840fef30 msgr2=0x7fc2840ff580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:00.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:00.162+0000 7fc28ace0700 1 -- 172.21.15.46:0/1493172132 shutdown_connections 2024-05-09T04:53:00.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:00.162+0000 7fc28ace0700 1 -- 172.21.15.46:0/1493172132 wait complete. 2024-05-09T04:53:00.163 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:53:00.590 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:00.591 INFO:tasks.workunit.client.0.smithi046.stdout: Children see throughput for 1 readers = 7256630.50 kB/sec 2024-05-09T04:53:00.591 INFO:tasks.workunit.client.0.smithi046.stdout: Parent sees throughput for 1 readers = 7247351.35 kB/sec 2024-05-09T04:53:00.591 INFO:tasks.workunit.client.0.smithi046.stdout: Min throughput per process = 7256630.50 kB/sec 2024-05-09T04:53:00.591 INFO:tasks.workunit.client.0.smithi046.stdout: Max throughput per process = 7256630.50 kB/sec 2024-05-09T04:53:00.591 INFO:tasks.workunit.client.0.smithi046.stdout: Avg throughput per process = 7256630.50 kB/sec 2024-05-09T04:53:00.591 INFO:tasks.workunit.client.0.smithi046.stdout: Min xfer = 1048576.00 kB 2024-05-09T04:53:01.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:00 smithi110 ceph-mon[34904]: pgmap v60: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 85 MiB/s wr, 29 op/s 2024-05-09T04:53:01.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:00 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1493172132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:00 smithi046 ceph-mon[30136]: pgmap v60: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 85 MiB/s wr, 29 op/s 2024-05-09T04:53:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:00 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1493172132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:01.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:00 smithi072 ceph-mon[34088]: pgmap v60: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 85 MiB/s wr, 29 op/s 2024-05-09T04:53:01.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:00 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1493172132' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:01.227 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:53:01.489 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:02.906 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:02 smithi046 ceph-mon[30136]: pgmap v61: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 171 MiB/s wr, 68 op/s 2024-05-09T04:53:02.998 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:02.998 INFO:tasks.workunit.client.0.smithi046.stdout: Children see throughput for 1 re-readers = 7079136.00 kB/sec 2024-05-09T04:53:02.998 INFO:tasks.workunit.client.0.smithi046.stdout: Parent sees throughput for 1 re-readers = 7069498.11 kB/sec 2024-05-09T04:53:02.998 INFO:tasks.workunit.client.0.smithi046.stdout: Min throughput per process = 7079136.00 kB/sec 2024-05-09T04:53:02.998 INFO:tasks.workunit.client.0.smithi046.stdout: Max throughput per process = 7079136.00 kB/sec 2024-05-09T04:53:02.998 INFO:tasks.workunit.client.0.smithi046.stdout: Avg throughput per process = 7079136.00 kB/sec 2024-05-09T04:53:02.999 INFO:tasks.workunit.client.0.smithi046.stdout: Min xfer = 1048576.00 kB 2024-05-09T04:53:03.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:02 smithi110 ceph-mon[34904]: pgmap v61: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 171 MiB/s wr, 68 op/s 2024-05-09T04:53:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:02 smithi072 ceph-mon[34088]: pgmap v61: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 171 MiB/s wr, 68 op/s 2024-05-09T04:53:03.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.417+0000 7f406ca8e700 1 -- 172.21.15.46:0/802932465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40681034c0 msgr2=0x7f40681038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.417+0000 7f406ca8e700 1 --2- 172.21.15.46:0/802932465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40681034c0 0x7f40681038d0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f40580099c0 tx=0x7f4058009cd0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.417+0000 7f406ca8e700 1 -- 172.21.15.46:0/802932465 shutdown_connections 2024-05-09T04:53:03.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.417+0000 7f406ca8e700 1 --2- 172.21.15.46:0/802932465 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4068108740 0x7f40681100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.417+0000 7f406ca8e700 1 --2- 172.21.15.46:0/802932465 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f40681034c0 0x7f40681038d0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.417+0000 7f406ca8e700 1 --2- 172.21.15.46:0/802932465 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4068107df0 0x7f4068108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.417+0000 7f406ca8e700 1 -- 172.21.15.46:0/802932465 >> 172.21.15.46:0/802932465 conn(0x7f40680fedc0 msgr2=0x7f40681011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.417+0000 7f406ca8e700 1 -- 172.21.15.46:0/802932465 shutdown_connections 2024-05-09T04:53:03.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.417+0000 7f406ca8e700 1 -- 172.21.15.46:0/802932465 wait complete. 2024-05-09T04:53:03.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.418+0000 7f406ca8e700 1 Processor -- start 2024-05-09T04:53:03.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.418+0000 7f406ca8e700 1 -- start start 2024-05-09T04:53:03.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.418+0000 7f406ca8e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40681034c0 0x7f406819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.418+0000 7f406ca8e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4068107df0 0x7f406819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.418+0000 7f406ca8e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4068108740 0x7f406819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.418+0000 7f406ca8e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f40681144c0 con 0x7f4068107df0 2024-05-09T04:53:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.418+0000 7f406ca8e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4068114340 con 0x7f40681034c0 2024-05-09T04:53:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.418+0000 7f406ca8e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4068114640 con 0x7f4068108740 2024-05-09T04:53:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.418+0000 7f4065d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4068107df0 0x7f406819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.418+0000 7f4065d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4068107df0 0x7f406819b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44406/0 (socket says 172.21.15.46:44406) 2024-05-09T04:53:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.418+0000 7f4065d9b700 1 -- 172.21.15.46:0/884093509 learned_addr learned my addr 172.21.15.46:0/884093509 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.418+0000 7f406659c700 1 --2- 172.21.15.46:0/884093509 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40681034c0 0x7f406819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.418+0000 7f4066d9d700 1 --2- 172.21.15.46:0/884093509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4068108740 0x7f406819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.419+0000 7f4065d9b700 1 -- 172.21.15.46:0/884093509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4068108740 msgr2=0x7f406819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.419+0000 7f4065d9b700 1 --2- 172.21.15.46:0/884093509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4068108740 0x7f406819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.419+0000 7f4065d9b700 1 -- 172.21.15.46:0/884093509 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40681034c0 msgr2=0x7f406819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.419+0000 7f4065d9b700 1 --2- 172.21.15.46:0/884093509 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40681034c0 0x7f406819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.419+0000 7f4065d9b700 1 -- 172.21.15.46:0/884093509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406819fce0 con 0x7f4068107df0 2024-05-09T04:53:03.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.419+0000 7f4065d9b700 1 --2- 172.21.15.46:0/884093509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4068107df0 0x7f406819b3b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f405c00d950 tx=0x7f405c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:03.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.419+0000 7f40577fe700 1 -- 172.21.15.46:0/884093509 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c017070 con 0x7f4068107df0 2024-05-09T04:53:03.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.419+0000 7f40577fe700 1 -- 172.21.15.46:0/884093509 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f405c005020 con 0x7f4068107df0 2024-05-09T04:53:03.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.419+0000 7f406ca8e700 1 -- 172.21.15.46:0/884093509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406819ffd0 con 0x7f4068107df0 2024-05-09T04:53:03.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.419+0000 7f40577fe700 1 -- 172.21.15.46:0/884093509 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c00fa10 con 0x7f4068107df0 2024-05-09T04:53:03.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.419+0000 7f406ca8e700 1 -- 172.21.15.46:0/884093509 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40681a78f0 con 0x7f4068107df0 2024-05-09T04:53:03.421 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.422+0000 7f40577fe700 1 -- 172.21.15.46:0/884093509 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f405c00fbd0 con 0x7f4068107df0 2024-05-09T04:53:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.422+0000 7f40577fe700 1 --2- 172.21.15.46:0/884093509 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f40500b2a70 0x7f40500b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.423+0000 7f406659c700 1 --2- 172.21.15.46:0/884093509 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f40500b2a70 0x7f40500b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.423+0000 7f40577fe700 1 -- 172.21.15.46:0/884093509 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f405c0e1e90 con 0x7f4068107df0 2024-05-09T04:53:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.423+0000 7f407b4a4b80 1 Processor -- start 2024-05-09T04:53:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.423+0000 7f407b4a4b80 1 -- start start 2024-05-09T04:53:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.423+0000 7f406659c700 1 --2- 172.21.15.46:0/884093509 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f40500b2a70 0x7f40500b4f80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4058009990 tx=0x7f40580036e0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.423+0000 7f407b4a4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb53e1160 0x55ddb5478d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.423+0000 7f407b4a4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb52f74a0 0x55ddb54792f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.423+0000 7f407b4a4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb547d400 0x55ddb547d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.423+0000 7f407b4a4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ddb5396ff0 con 0x55ddb52f74a0 2024-05-09T04:53:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.423+0000 7f407b4a4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ddb53357f0 con 0x55ddb53e1160 2024-05-09T04:53:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.423+0000 7f407b4a4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ddb5392fb0 con 0x55ddb547d400 2024-05-09T04:53:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.424+0000 7f4065d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb52f74a0 0x55ddb54792f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.424+0000 7f4065d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb52f74a0 0x55ddb54792f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44416/0 (socket says 172.21.15.46:44416) 2024-05-09T04:53:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.424+0000 7f4065d9b700 1 -- 172.21.15.46:0/3962968643 learned_addr learned my addr 172.21.15.46:0/3962968643 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.424+0000 7f4066d9d700 1 --2- 172.21.15.46:0/3962968643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb547d400 0x55ddb547d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.424+0000 7f406659c700 1 --2- 172.21.15.46:0/3962968643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb53e1160 0x55ddb5478d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.424+0000 7f4065d9b700 1 -- 172.21.15.46:0/3962968643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb547d400 msgr2=0x55ddb547d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.424+0000 7f4065d9b700 1 --2- 172.21.15.46:0/3962968643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb547d400 0x55ddb547d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.424+0000 7f4065d9b700 1 -- 172.21.15.46:0/3962968643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb53e1160 msgr2=0x55ddb5478d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.424+0000 7f4065d9b700 1 --2- 172.21.15.46:0/3962968643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb53e1160 0x55ddb5478d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.424+0000 7f4065d9b700 1 -- 172.21.15.46:0/3962968643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb532d370 con 0x55ddb52f74a0 2024-05-09T04:53:03.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.424+0000 7f4065d9b700 1 --2- 172.21.15.46:0/3962968643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb52f74a0 0x55ddb54792f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f405c0a8e50 tx=0x7f405c0a9850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=276d36054836d82f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:03.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.424+0000 7f4054ff9700 1 -- 172.21.15.46:0/3962968643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c020070 con 0x55ddb52f74a0 2024-05-09T04:53:03.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.424+0000 7f4054ff9700 1 -- 172.21.15.46:0/3962968643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f405c0ad500 con 0x55ddb52f74a0 2024-05-09T04:53:03.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.424+0000 7f4054ff9700 1 -- 172.21.15.46:0/3962968643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c0a9610 con 0x55ddb52f74a0 2024-05-09T04:53:03.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.425+0000 7f407b4a4b80 1 -- 172.21.15.46:0/3962968643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb52f74a0 msgr2=0x55ddb54792f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.425+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/3962968643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb52f74a0 0x55ddb54792f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f405c0a8e50 tx=0x7f405c0a9850 comp rx=0 tx=0).stop 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.425+0000 7f407b4a4b80 1 -- 172.21.15.46:0/3962968643 shutdown_connections 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.425+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/3962968643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb547d400 0x55ddb547d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.425+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/3962968643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb53e1160 0x55ddb5478d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.425+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/3962968643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb52f74a0 0x55ddb54792f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.425+0000 7f407b4a4b80 1 -- 172.21.15.46:0/3962968643 >> 172.21.15.46:0/3962968643 conn(0x55ddb53576a0 msgr2=0x55ddb546f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.425+0000 7f407b4a4b80 1 -- 172.21.15.46:0/3962968643 shutdown_connections 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.425+0000 7f407b4a4b80 1 -- 172.21.15.46:0/3962968643 wait complete. 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.426+0000 7f407b4a4b80 1 -- start start 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.426+0000 7f407b4a4b80 1 Processor -- start 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.426+0000 7f407b4a4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb52f74a0 0x55ddb5482680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.426+0000 7f407b4a4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb53e1160 0x55ddb5482bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.426+0000 7f407b4a4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb547d400 0x55ddb5483100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.426+0000 7f407b4a4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ddb5212ba0 con 0x55ddb547d400 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.426+0000 7f407b4a4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ddb5351150 con 0x55ddb53e1160 2024-05-09T04:53:03.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.426+0000 7f407b4a4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ddb547e1b0 con 0x55ddb52f74a0 2024-05-09T04:53:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.426+0000 7f407b4a4b80 10 client.0 _ll_register_callbacks cb 0x55ddb52086c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.426+0000 7f4066d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb547d400 0x55ddb5483100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.426+0000 7f4066d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb547d400 0x55ddb5483100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44420/0 (socket says 172.21.15.46:44420) 2024-05-09T04:53:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.426+0000 7f4066d9d700 1 -- 172.21.15.46:0/1331493963 learned_addr learned my addr 172.21.15.46:0/1331493963 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f4065d9b700 1 --2- 172.21.15.46:0/1331493963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb53e1160 0x55ddb5482bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f406659c700 1 --2- 172.21.15.46:0/1331493963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb52f74a0 0x55ddb5482680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f4066d9d700 1 -- 172.21.15.46:0/1331493963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb52f74a0 msgr2=0x55ddb5482680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f4066d9d700 1 --2- 172.21.15.46:0/1331493963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb52f74a0 0x55ddb5482680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f4066d9d700 1 -- 172.21.15.46:0/1331493963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb53e1160 msgr2=0x55ddb5482bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f4066d9d700 1 --2- 172.21.15.46:0/1331493963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb53e1160 0x55ddb5482bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f4066d9d700 1 -- 172.21.15.46:0/1331493963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb5212490 con 0x55ddb547d400 2024-05-09T04:53:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f406659c700 1 --2- 172.21.15.46:0/1331493963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb52f74a0 0x55ddb5482680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f4066d9d700 1 --2- 172.21.15.46:0/1331493963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb547d400 0x55ddb5483100 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f40600042f0 tx=0x7f4060004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1744f12b0190c3e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f404dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:03.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f404dffb700 1 -- 172.21.15.46:0/1331493963 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f406001a070 con 0x55ddb547d400 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f404dffb700 1 -- 172.21.15.46:0/1331493963 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f4060007910 con 0x55ddb547d400 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1331493963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f405c0ad500 con 0x55ddb547d400 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f404dffb700 1 -- 172.21.15.46:0/1331493963 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4060026050 con 0x55ddb547d400 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f407b4a4b80 10 client.15786 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.427+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1331493963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ddb5490000 con 0x55ddb547d400 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.428+0000 7f404dffb700 1 -- 172.21.15.46:0/1331493963 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f405c0ad500 con 0x55ddb547d400 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.428+0000 7f404dffb700 1 -- 172.21.15.46:0/1331493963 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f4060016af0 con 0x55ddb547d400 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.428+0000 7f407b4a4b80 10 client.15786 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.428+0000 7f407b4a4b80 10 client.15786 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.428+0000 7f407b4a4b80 20 client.15786 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.428+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/1331493963 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ddb5492590 0x55ddb54949b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.428+0000 7f407b4a4b80 4 client.15786 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.428+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1331493963 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ddb53a49d0 con 0x55ddb5492590 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.428+0000 7f406659c700 1 --2- 172.21.15.46:0/1331493963 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ddb5492590 0x55ddb54949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.429+0000 7f404dffb700 10 client.15786 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.429+0000 7f406659c700 1 --2- 172.21.15.46:0/1331493963 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ddb5492590 0x55ddb54949b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a8993e87275072b server_cookie=160186dfa475b354 in_seq=0 out_seq=0 2024-05-09T04:53:03.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.430+0000 7f404dffb700 1 -- 172.21.15.46:0/1331493963 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ddb53a49d0 con 0x55ddb5492590 2024-05-09T04:53:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.430+0000 7f404dffb700 10 client.15786 handle_command_reply: tid=0 2024-05-09T04:53:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.430+0000 7f407b4a4b80 1 client.15786 shutdown 2024-05-09T04:53:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.430+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1331493963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb547d400 msgr2=0x55ddb5483100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.430+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/1331493963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb547d400 0x55ddb5483100 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f40600042f0 tx=0x7f4060004320 comp rx=0 tx=0).stop 2024-05-09T04:53:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.430+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1331493963 shutdown_connections 2024-05-09T04:53:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.430+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/1331493963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb547d400 0x55ddb5483100 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.430+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/1331493963 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ddb5492590 0x55ddb54949b0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.430+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/1331493963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb52f74a0 0x55ddb5482680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.430+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/1331493963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb53e1160 0x55ddb5482bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.430+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1331493963 >> 172.21.15.46:0/1331493963 conn(0x55ddb53576a0 msgr2=0x55ddb546f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.430+0000 7f404dffb700 0 client.15786 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:53:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.430+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1331493963 shutdown_connections 2024-05-09T04:53:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.430+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1331493963 wait complete. 2024-05-09T04:53:03.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.430+0000 7f407b4a4b80 20 client.15786 trim_cache size 0 max 16384 2024-05-09T04:53:03.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.445+0000 7f407b4a4b80 1 Processor -- start 2024-05-09T04:53:03.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.445+0000 7f407b4a4b80 1 -- start start 2024-05-09T04:53:03.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.445+0000 7f407b4a4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb52f74a0 0x55ddb5482f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.445+0000 7f407b4a4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb53e1160 0x55ddb548e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.445+0000 7f407b4a4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb547d400 0x55ddb547c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.445+0000 7f407b4a4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f40681af5d0 con 0x55ddb53e1160 2024-05-09T04:53:03.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.445+0000 7f407b4a4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f40681ace60 con 0x55ddb547d400 2024-05-09T04:53:03.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.445+0000 7f407b4a4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f40681aacd0 con 0x55ddb52f74a0 2024-05-09T04:53:03.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.446+0000 7f4065d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb53e1160 0x55ddb548e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.446+0000 7f4065d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb53e1160 0x55ddb548e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44428/0 (socket says 172.21.15.46:44428) 2024-05-09T04:53:03.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.446+0000 7f4065d9b700 1 -- 172.21.15.46:0/767303272 learned_addr learned my addr 172.21.15.46:0/767303272 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:03.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.446+0000 7f406659c700 1 --2- 172.21.15.46:0/767303272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb52f74a0 0x55ddb5482f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.446+0000 7f4065d9b700 1 -- 172.21.15.46:0/767303272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb52f74a0 msgr2=0x55ddb5482f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.446+0000 7f4065d9b700 1 --2- 172.21.15.46:0/767303272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb52f74a0 0x55ddb5482f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.446+0000 7f4065d9b700 1 -- 172.21.15.46:0/767303272 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb547d400 msgr2=0x55ddb547c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.446+0000 7f4065d9b700 1 --2- 172.21.15.46:0/767303272 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb547d400 0x55ddb547c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.446+0000 7f4065d9b700 1 -- 172.21.15.46:0/767303272 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddb5480650 con 0x55ddb53e1160 2024-05-09T04:53:03.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.446+0000 7f4065d9b700 1 --2- 172.21.15.46:0/767303272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb53e1160 0x55ddb548e400 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f405c0a9310 tx=0x7f405c0a8f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fa85bb22dcb398 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:03.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.446+0000 7f404ffff700 1 -- 172.21.15.46:0/767303272 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c0c8070 con 0x55ddb53e1160 2024-05-09T04:53:03.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.446+0000 7f404ffff700 1 -- 172.21.15.46:0/767303272 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f405c0ad500 con 0x55ddb53e1160 2024-05-09T04:53:03.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.446+0000 7f404ffff700 1 -- 172.21.15.46:0/767303272 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405c020040 con 0x55ddb53e1160 2024-05-09T04:53:03.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.447+0000 7f407b4a4b80 1 -- 172.21.15.46:0/767303272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb53e1160 msgr2=0x55ddb548e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.447+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/767303272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb53e1160 0x55ddb548e400 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f405c0a9310 tx=0x7f405c0a8f20 comp rx=0 tx=0).stop 2024-05-09T04:53:03.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.447+0000 7f407b4a4b80 1 -- 172.21.15.46:0/767303272 shutdown_connections 2024-05-09T04:53:03.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.447+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/767303272 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb547d400 0x55ddb547c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.447+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/767303272 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb52f74a0 0x55ddb5482f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.447+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/767303272 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb53e1160 0x55ddb548e400 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.447+0000 7f407b4a4b80 1 -- 172.21.15.46:0/767303272 >> 172.21.15.46:0/767303272 conn(0x55ddb53576a0 msgr2=0x55ddb5480f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:03.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.447+0000 7f407b4a4b80 1 -- 172.21.15.46:0/767303272 shutdown_connections 2024-05-09T04:53:03.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.447+0000 7f407b4a4b80 1 -- 172.21.15.46:0/767303272 wait complete. 2024-05-09T04:53:03.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.447+0000 7f407b4a4b80 1 -- start start 2024-05-09T04:53:03.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f407b4a4b80 1 Processor -- start 2024-05-09T04:53:03.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f407b4a4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb52f74a0 0x55ddb548a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f407b4a4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb53e1160 0x55ddb548acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f407b4a4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb547d400 0x55ddb547b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f407b4a4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f40681ab320 con 0x55ddb52f74a0 2024-05-09T04:53:03.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f407b4a4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f40681ac790 con 0x55ddb547d400 2024-05-09T04:53:03.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f407b4a4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f40681ab9f0 con 0x55ddb53e1160 2024-05-09T04:53:03.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f407b4a4b80 10 client.0 _ll_register_callbacks cb 0x55ddb52086c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f406659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb52f74a0 0x55ddb548a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f406659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb52f74a0 0x55ddb548a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44438/0 (socket says 172.21.15.46:44438) 2024-05-09T04:53:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f406659c700 1 -- 172.21.15.46:0/1180369923 learned_addr learned my addr 172.21.15.46:0/1180369923 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f4065d9b700 1 --2- 172.21.15.46:0/1180369923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb53e1160 0x55ddb548acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f4066d9d700 1 --2- 172.21.15.46:0/1180369923 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb547d400 0x55ddb547b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f406659c700 1 -- 172.21.15.46:0/1180369923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb53e1160 msgr2=0x55ddb548acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f406659c700 1 --2- 172.21.15.46:0/1180369923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb53e1160 0x55ddb548acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f406659c700 1 -- 172.21.15.46:0/1180369923 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb547d400 msgr2=0x55ddb547b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.448+0000 7f406659c700 1 --2- 172.21.15.46:0/1180369923 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb547d400 0x55ddb547b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.449+0000 7f406659c700 1 -- 172.21.15.46:0/1180369923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddb547b870 con 0x55ddb52f74a0 2024-05-09T04:53:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.449+0000 7f406659c700 1 --2- 172.21.15.46:0/1180369923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb52f74a0 0x55ddb548a780 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f40580098e0 tx=0x7f4058009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d934a0d1b78561fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.449+0000 7f404dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.449+0000 7f404dffb700 1 -- 172.21.15.46:0/1180369923 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4058037070 con 0x55ddb52f74a0 2024-05-09T04:53:03.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.449+0000 7f404dffb700 1 -- 172.21.15.46:0/1180369923 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f405800b990 con 0x55ddb52f74a0 2024-05-09T04:53:03.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.449+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1180369923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f405c0ad500 con 0x55ddb52f74a0 2024-05-09T04:53:03.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.449+0000 7f404dffb700 1 -- 172.21.15.46:0/1180369923 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4058043050 con 0x55ddb52f74a0 2024-05-09T04:53:03.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.449+0000 7f407b4a4b80 10 client.15792 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:03.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.449+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1180369923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ddb5499ae0 con 0x55ddb52f74a0 2024-05-09T04:53:03.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.449+0000 7f404dffb700 1 -- 172.21.15.46:0/1180369923 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f405c0ad500 con 0x55ddb52f74a0 2024-05-09T04:53:03.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.449+0000 7f404dffb700 1 -- 172.21.15.46:0/1180369923 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f4058022070 con 0x55ddb52f74a0 2024-05-09T04:53:03.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.450+0000 7f407b4a4b80 10 client.15792 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:03.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.450+0000 7f407b4a4b80 10 client.15792 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:53:03.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.450+0000 7f407b4a4b80 20 client.15792 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:03.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.450+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/1180369923 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ddb54fcdb0 0x55ddb54ff1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.450+0000 7f407b4a4b80 4 client.15792 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:53:03.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.450+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1180369923 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ddb539ea70 con 0x55ddb54fcdb0 2024-05-09T04:53:03.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.450+0000 7f4065d9b700 1 --2- 172.21.15.46:0/1180369923 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ddb54fcdb0 0x55ddb54ff1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.450+0000 7f404dffb700 10 client.15792 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:53:03.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.450+0000 7f4065d9b700 1 --2- 172.21.15.46:0/1180369923 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ddb54fcdb0 0x55ddb54ff1d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=873f57aa822ded0f server_cookie=68aa8ceb076ab089 in_seq=0 out_seq=0 2024-05-09T04:53:03.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.450+0000 7f404dffb700 1 -- 172.21.15.46:0/1180369923 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ddb539ea70 con 0x55ddb54fcdb0 2024-05-09T04:53:03.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.450+0000 7f404dffb700 10 client.15792 handle_command_reply: tid=0 2024-05-09T04:53:03.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.450+0000 7f407b4a4b80 1 client.15792 shutdown 2024-05-09T04:53:03.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.450+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1180369923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb52f74a0 msgr2=0x55ddb548a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/1180369923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb52f74a0 0x55ddb548a780 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f40580098e0 tx=0x7f4058009cd0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1180369923 shutdown_connections 2024-05-09T04:53:03.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/1180369923 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ddb547d400 0x55ddb547b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/1180369923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ddb52f74a0 0x55ddb548a780 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/1180369923 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ddb54fcdb0 0x55ddb54ff1d0 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f407b4a4b80 1 --2- 172.21.15.46:0/1180369923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ddb53e1160 0x55ddb548acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1180369923 >> 172.21.15.46:0/1180369923 conn(0x55ddb53576a0 msgr2=0x55ddb5480bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:03.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f404dffb700 0 client.15792 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:53:03.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1180369923 shutdown_connections 2024-05-09T04:53:03.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f407b4a4b80 1 -- 172.21.15.46:0/1180369923 wait complete. 2024-05-09T04:53:03.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f407b4a4b80 20 client.15792 trim_cache size 0 max 16384 2024-05-09T04:53:03.456 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:53:03.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f406ca8e700 1 -- 172.21.15.46:0/884093509 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f40500b2a70 msgr2=0x7f40500b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f406ca8e700 1 --2- 172.21.15.46:0/884093509 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f40500b2a70 0x7f40500b4f80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f4058009990 tx=0x7f40580036e0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f406ca8e700 1 -- 172.21.15.46:0/884093509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4068107df0 msgr2=0x7f406819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f406ca8e700 1 --2- 172.21.15.46:0/884093509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4068107df0 0x7f406819b3b0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f405c00d950 tx=0x7f405c00dc60 comp rx=0 tx=0).stop 2024-05-09T04:53:03.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.451+0000 7f406ca8e700 1 -- 172.21.15.46:0/884093509 shutdown_connections 2024-05-09T04:53:03.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.452+0000 7f406ca8e700 1 --2- 172.21.15.46:0/884093509 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f40500b2a70 0x7f40500b4f80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.452+0000 7f406ca8e700 1 --2- 172.21.15.46:0/884093509 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4068108740 0x7f406819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.452+0000 7f406ca8e700 1 --2- 172.21.15.46:0/884093509 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f40681034c0 0x7f406819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.452+0000 7f406ca8e700 1 --2- 172.21.15.46:0/884093509 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4068107df0 0x7f406819b3b0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.452+0000 7f406ca8e700 1 -- 172.21.15.46:0/884093509 >> 172.21.15.46:0/884093509 conn(0x7f40680fedc0 msgr2=0x7f4068101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:03.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.452+0000 7f406ca8e700 1 -- 172.21.15.46:0/884093509 shutdown_connections 2024-05-09T04:53:03.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:03.452+0000 7f406ca8e700 1 -- 172.21.15.46:0/884093509 wait complete. 2024-05-09T04:53:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.600+0000 7f37cecbd640 1 -- 172.21.15.46:0/4182537479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37c8109ca0 msgr2=0x7f37c8103840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.600+0000 7f37cecbd640 1 --2- 172.21.15.46:0/4182537479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37c8109ca0 0x7f37c8103840 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f37bc009900 tx=0x7f37bc02a410 comp rx=0 tx=0).stop 2024-05-09T04:53:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.600+0000 7f37cecbd640 1 -- 172.21.15.46:0/4182537479 shutdown_connections 2024-05-09T04:53:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.600+0000 7f37cecbd640 1 --2- 172.21.15.46:0/4182537479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37c810a140 0x7f37c8103d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.600+0000 7f37cecbd640 1 --2- 172.21.15.46:0/4182537479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f37c81043f0 0x7f37c81107c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.600+0000 7f37cecbd640 1 --2- 172.21.15.46:0/4182537479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37c8109ca0 0x7f37c8103840 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.600+0000 7f37cecbd640 1 -- 172.21.15.46:0/4182537479 >> 172.21.15.46:0/4182537479 conn(0x7f37c80ff340 msgr2=0x7f37c8101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.600+0000 7f37cecbd640 1 -- 172.21.15.46:0/4182537479 shutdown_connections 2024-05-09T04:53:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.600+0000 7f37cecbd640 1 -- 172.21.15.46:0/4182537479 wait complete. 2024-05-09T04:53:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cecbd640 1 Processor -- start 2024-05-09T04:53:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cecbd640 1 -- start start 2024-05-09T04:53:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cecbd640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37c81043f0 0x7f37c81a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cecbd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f37c8109ca0 0x7f37c81a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cecbd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37c810a140 0x7f37c819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cecbd640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8113ee0 con 0x7f37c810a140 2024-05-09T04:53:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cecbd640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8113d60 con 0x7f37c8109ca0 2024-05-09T04:53:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cecbd640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f37c8114060 con 0x7f37c81043f0 2024-05-09T04:53:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cd25a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37c810a140 0x7f37c819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cd25a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37c810a140 0x7f37c819a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44456/0 (socket says 172.21.15.46:44456) 2024-05-09T04:53:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cd25a640 1 -- 172.21.15.46:0/2889080963 learned_addr learned my addr 172.21.15.46:0/2889080963 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cca59640 1 --2- 172.21.15.46:0/2889080963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37c81043f0 0x7f37c81a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37c7fff640 1 --2- 172.21.15.46:0/2889080963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f37c8109ca0 0x7f37c81a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cd25a640 1 -- 172.21.15.46:0/2889080963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37c81043f0 msgr2=0x7f37c81a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cd25a640 1 --2- 172.21.15.46:0/2889080963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37c81043f0 0x7f37c81a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cd25a640 1 -- 172.21.15.46:0/2889080963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f37c8109ca0 msgr2=0x7f37c81a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cd25a640 1 --2- 172.21.15.46:0/2889080963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f37c8109ca0 0x7f37c81a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37cd25a640 1 -- 172.21.15.46:0/2889080963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37c819ae80 con 0x7f37c810a140 2024-05-09T04:53:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.601+0000 7f37c7fff640 1 --2- 172.21.15.46:0/2889080963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f37c8109ca0 0x7f37c81a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.602+0000 7f37cd25a640 1 --2- 172.21.15.46:0/2889080963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37c810a140 0x7f37c819a7e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009fd0 tx=0x7f37c000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.602+0000 7f37c5ffb640 1 -- 172.21.15.46:0/2889080963 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0017070 con 0x7f37c810a140 2024-05-09T04:53:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.602+0000 7f37c5ffb640 1 -- 172.21.15.46:0/2889080963 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37c0003e20 con 0x7f37c810a140 2024-05-09T04:53:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.602+0000 7f37c5ffb640 1 -- 172.21.15.46:0/2889080963 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37c0010460 con 0x7f37c810a140 2024-05-09T04:53:03.604 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.602+0000 7f37cecbd640 1 -- 172.21.15.46:0/2889080963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37c819b170 con 0x7f37c810a140 2024-05-09T04:53:03.605 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.602+0000 7f37cecbd640 1 -- 172.21.15.46:0/2889080963 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37c81a7400 con 0x7f37c810a140 2024-05-09T04:53:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.605+0000 7f37d5b7b740 1 Processor -- start 2024-05-09T04:53:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.605+0000 7f37c5ffb640 1 -- 172.21.15.46:0/2889080963 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f37c0003970 con 0x7f37c810a140 2024-05-09T04:53:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.605+0000 7f37d5b7b740 1 -- start start 2024-05-09T04:53:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37d5b7b740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b40e1b0 0x56495b7b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37d5b7b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3e52f0 0x56495b7b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37d5b7b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b3ed4d0 0x56495b7b1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37d5b7b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56495b7167c0 con 0x56495b3ed4d0 2024-05-09T04:53:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37d5b7b740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56495b6b20e0 con 0x56495b40e1b0 2024-05-09T04:53:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37d5b7b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56495b712780 con 0x56495b3e52f0 2024-05-09T04:53:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b3ed4d0 0x56495b7b1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b3ed4d0 0x56495b7b1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44462/0 (socket says 172.21.15.46:44462) 2024-05-09T04:53:03.607 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37c7fff640 1 -- 172.21.15.46:0/396656618 learned_addr learned my addr 172.21.15.46:0/396656618 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37cca59640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3e52f0 0x56495b7b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37c7fff640 1 --2- 172.21.15.46:0/396656618 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b40e1b0 0x56495b7b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37c5ffb640 1 --2- 172.21.15.46:0/2889080963 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f37b80b29d0 0x7f37b80b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37cca59640 1 -- 172.21.15.46:0/396656618 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b40e1b0 msgr2=0x56495b7b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37cca59640 1 --2- 172.21.15.46:0/396656618 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b40e1b0 0x56495b7b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37cca59640 1 -- 172.21.15.46:0/396656618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b3ed4d0 msgr2=0x56495b7b1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37cca59640 1 --2- 172.21.15.46:0/396656618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b3ed4d0 0x56495b7b1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37cca59640 1 -- 172.21.15.46:0/396656618 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56495b6db7c0 con 0x56495b3e52f0 2024-05-09T04:53:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37c7fff640 1 --2- 172.21.15.46:0/396656618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b3ed4d0 0x56495b7b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.606+0000 7f37cd25a640 1 --2- 172.21.15.46:0/2889080963 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f37b80b29d0 0x7f37b80b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.607+0000 7f37c5ffb640 1 -- 172.21.15.46:0/2889080963 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f37c00e23e0 con 0x7f37c810a140 2024-05-09T04:53:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.607+0000 7f37cca59640 1 --2- 172.21.15.46:0/396656618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3e52f0 0x56495b7b1a70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f37bc02a930 tx=0x7f37bc004390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2578fc983e3b75ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.607+0000 7f37b2ffd640 1 -- 172.21.15.46:0/396656618 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc035040 con 0x56495b3e52f0 2024-05-09T04:53:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.607+0000 7f37cd25a640 1 --2- 172.21.15.46:0/2889080963 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f37b80b29d0 0x7f37b80b4ee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f37c00a5670 tx=0x7f37c00a55e0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:03.608 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.607+0000 7f37b2ffd640 1 -- 172.21.15.46:0/396656618 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37bc004750 con 0x56495b3e52f0 2024-05-09T04:53:03.609 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.607+0000 7f37b2ffd640 1 -- 172.21.15.46:0/396656618 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37bc003830 con 0x56495b3e52f0 2024-05-09T04:53:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.608+0000 7f37d5b7b740 1 -- 172.21.15.46:0/396656618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3e52f0 msgr2=0x56495b7b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.608+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/396656618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3e52f0 0x56495b7b1a70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f37bc02a930 tx=0x7f37bc004390 comp rx=0 tx=0).stop 2024-05-09T04:53:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.608+0000 7f37d5b7b740 1 -- 172.21.15.46:0/396656618 shutdown_connections 2024-05-09T04:53:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.608+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/396656618 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3e52f0 0x56495b7b1a70 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.608+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/396656618 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b3ed4d0 0x56495b7b1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.608+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/396656618 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b40e1b0 0x56495b7b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.608+0000 7f37d5b7b740 1 -- 172.21.15.46:0/396656618 >> 172.21.15.46:0/396656618 conn(0x56495b402300 msgr2=0x56495b7ade30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.608+0000 7f37d5b7b740 1 -- 172.21.15.46:0/396656618 shutdown_connections 2024-05-09T04:53:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.608+0000 7f37d5b7b740 1 -- 172.21.15.46:0/396656618 wait complete. 2024-05-09T04:53:03.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.608+0000 7f37d5b7b740 1 -- start start 2024-05-09T04:53:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.608+0000 7f37d5b7b740 1 Processor -- start 2024-05-09T04:53:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37d5b7b740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b3e52f0 0x56495b7c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37d5b7b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3ed4d0 0x56495b7c2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37d5b7b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b40e1b0 0x56495b7bfff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37d5b7b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56495b7c2b60 con 0x56495b40e1b0 2024-05-09T04:53:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37d5b7b740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56495b69d9c0 con 0x56495b3e52f0 2024-05-09T04:53:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37d5b7b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56495b7c2ce0 con 0x56495b3ed4d0 2024-05-09T04:53:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37d5b7b740 10 client.0 _ll_register_callbacks cb 0x56495b6ca020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b40e1b0 0x56495b7bfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b40e1b0 0x56495b7bfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44478/0 (socket says 172.21.15.46:44478) 2024-05-09T04:53:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37c7fff640 1 -- 172.21.15.46:0/3537206287 learned_addr learned my addr 172.21.15.46:0/3537206287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37c7fff640 1 --2- 172.21.15.46:0/3537206287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b3e52f0 0x56495b7c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37cca59640 1 --2- 172.21.15.46:0/3537206287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3ed4d0 0x56495b7c2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37c7fff640 1 -- 172.21.15.46:0/3537206287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3ed4d0 msgr2=0x56495b7c2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37c7fff640 1 --2- 172.21.15.46:0/3537206287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3ed4d0 0x56495b7c2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37c7fff640 1 -- 172.21.15.46:0/3537206287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b3e52f0 msgr2=0x56495b7c1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37c7fff640 1 --2- 172.21.15.46:0/3537206287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b3e52f0 0x56495b7c1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.609+0000 7f37c7fff640 1 -- 172.21.15.46:0/3537206287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56495b6c41f0 con 0x56495b40e1b0 2024-05-09T04:53:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.610+0000 7f37c7fff640 1 --2- 172.21.15.46:0/3537206287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b40e1b0 0x56495b7bfff0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f37b400e760 tx=0x7f37b400eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c749b151d10d52f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.610+0000 7f37a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.610+0000 7f37a7fff640 1 -- 172.21.15.46:0/3537206287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4013310 con 0x56495b40e1b0 2024-05-09T04:53:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.610+0000 7f37a7fff640 1 -- 172.21.15.46:0/3537206287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f37b400c940 con 0x56495b40e1b0 2024-05-09T04:53:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.610+0000 7f37d5b7b740 1 -- 172.21.15.46:0/3537206287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37bc004750 con 0x56495b40e1b0 2024-05-09T04:53:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.610+0000 7f37a7fff640 1 -- 172.21.15.46:0/3537206287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37b400cb40 con 0x56495b40e1b0 2024-05-09T04:53:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.610+0000 7f37d5b7b740 10 client.15804 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.610+0000 7f37d5b7b740 1 -- 172.21.15.46:0/3537206287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56495b479790 con 0x56495b40e1b0 2024-05-09T04:53:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.610+0000 7f37a7fff640 1 -- 172.21.15.46:0/3537206287 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37bc004750 con 0x56495b40e1b0 2024-05-09T04:53:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.610+0000 7f37a7fff640 1 -- 172.21.15.46:0/3537206287 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f37b4011b90 con 0x56495b40e1b0 2024-05-09T04:53:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.610+0000 7f37d5b7b740 10 client.15804 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:03.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.610+0000 7f37d5b7b740 10 client.15804 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.610+0000 7f37d5b7b740 20 client.15804 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.611+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/3537206287 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56495b3ccd80 0x56495b7d23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.611+0000 7f37d5b7b740 4 client.15804 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.611+0000 7f37d5b7b740 1 -- 172.21.15.46:0/3537206287 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56495b7243d0 con 0x56495b3ccd80 2024-05-09T04:53:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.611+0000 7f37cca59640 1 --2- 172.21.15.46:0/3537206287 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56495b3ccd80 0x56495b7d23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.611+0000 7f37cca59640 1 --2- 172.21.15.46:0/3537206287 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56495b3ccd80 0x56495b7d23b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d6a1a7e0c3c2cf5 server_cookie=2f30611e65af6205 in_seq=0 out_seq=0 2024-05-09T04:53:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.611+0000 7f37a7fff640 10 client.15804 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.612+0000 7f37a7fff640 1 -- 172.21.15.46:0/3537206287 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56495b7243d0 con 0x56495b3ccd80 2024-05-09T04:53:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.612+0000 7f37a7fff640 10 client.15804 handle_command_reply: tid=0 2024-05-09T04:53:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.612+0000 7f37d5b7b740 1 client.15804 shutdown 2024-05-09T04:53:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.612+0000 7f37d5b7b740 1 -- 172.21.15.46:0/3537206287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b40e1b0 msgr2=0x56495b7bfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.612+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/3537206287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b40e1b0 0x56495b7bfff0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f37b400e760 tx=0x7f37b400eb30 comp rx=0 tx=0).stop 2024-05-09T04:53:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.612+0000 7f37d5b7b740 1 -- 172.21.15.46:0/3537206287 shutdown_connections 2024-05-09T04:53:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.612+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/3537206287 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56495b3ccd80 0x56495b7d23b0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.612+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/3537206287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3ed4d0 0x56495b7c2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.612+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/3537206287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b40e1b0 0x56495b7bfff0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.612+0000 7f37a7fff640 0 client.15804 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.612+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/3537206287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b3e52f0 0x56495b7c1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.612+0000 7f37d5b7b740 1 -- 172.21.15.46:0/3537206287 >> 172.21.15.46:0/3537206287 conn(0x56495b402300 msgr2=0x56495b7b1410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.612+0000 7f37d5b7b740 1 -- 172.21.15.46:0/3537206287 shutdown_connections 2024-05-09T04:53:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.612+0000 7f37d5b7b740 1 -- 172.21.15.46:0/3537206287 wait complete. 2024-05-09T04:53:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.613+0000 7f37d5b7b740 20 client.15804 trim_cache size 0 max 16384 2024-05-09T04:53:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.628+0000 7f37d5b7b740 1 Processor -- start 2024-05-09T04:53:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.628+0000 7f37d5b7b740 1 -- start start 2024-05-09T04:53:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.628+0000 7f37d5b7b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b3e52f0 0x56495b7b1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.628+0000 7f37d5b7b740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b3ed4d0 0x56495b7c0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.628+0000 7f37d5b7b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b40e1b0 0x56495b7b3a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.628+0000 7f37d5b7b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81b02c0 con 0x56495b3e52f0 2024-05-09T04:53:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.628+0000 7f37d5b7b740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81adb90 con 0x56495b3ed4d0 2024-05-09T04:53:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.628+0000 7f37d5b7b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81ab9e0 con 0x56495b40e1b0 2024-05-09T04:53:03.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.628+0000 7f37c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b3e52f0 0x56495b7b1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.628+0000 7f37c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b3e52f0 0x56495b7b1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44486/0 (socket says 172.21.15.46:44486) 2024-05-09T04:53:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.628+0000 7f37c7fff640 1 -- 172.21.15.46:0/3966989918 learned_addr learned my addr 172.21.15.46:0/3966989918 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.629+0000 7f37c7fff640 1 --2- 172.21.15.46:0/3966989918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b40e1b0 0x56495b7b3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.629+0000 7f37cca59640 1 --2- 172.21.15.46:0/3966989918 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b3ed4d0 0x56495b7c0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.629+0000 7f37c7fff640 1 -- 172.21.15.46:0/3966989918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b40e1b0 msgr2=0x56495b7b3a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.629+0000 7f37c7fff640 1 --2- 172.21.15.46:0/3966989918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b40e1b0 0x56495b7b3a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.629+0000 7f37c7fff640 1 -- 172.21.15.46:0/3966989918 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b3ed4d0 msgr2=0x56495b7c0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.629+0000 7f37c7fff640 1 --2- 172.21.15.46:0/3966989918 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b3ed4d0 0x56495b7c0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.629+0000 7f37c7fff640 1 -- 172.21.15.46:0/3966989918 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56495b4f05f0 con 0x56495b3e52f0 2024-05-09T04:53:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.629+0000 7f37c7fff640 1 --2- 172.21.15.46:0/3966989918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b3e52f0 0x56495b7b1e40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f37b4010660 tx=0x7f37b400ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b049add8a957013 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.629+0000 7f37b27fc640 1 -- 172.21.15.46:0/3966989918 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4000e60 con 0x56495b3e52f0 2024-05-09T04:53:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.629+0000 7f37b27fc640 1 -- 172.21.15.46:0/3966989918 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b40138b0 con 0x56495b3e52f0 2024-05-09T04:53:03.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.629+0000 7f37b27fc640 1 -- 172.21.15.46:0/3966989918 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b40035f0 con 0x56495b3e52f0 2024-05-09T04:53:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.630+0000 7f37d5b7b740 1 -- 172.21.15.46:0/3966989918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b3e52f0 msgr2=0x56495b7b1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.630+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/3966989918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b3e52f0 0x56495b7b1e40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f37b4010660 tx=0x7f37b400ec20 comp rx=0 tx=0).stop 2024-05-09T04:53:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.630+0000 7f37d5b7b740 1 -- 172.21.15.46:0/3966989918 shutdown_connections 2024-05-09T04:53:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.630+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/3966989918 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b40e1b0 0x56495b7b3a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.630+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/3966989918 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b3ed4d0 0x56495b7c0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.630+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/3966989918 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b3e52f0 0x56495b7b1e40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.630+0000 7f37d5b7b740 1 -- 172.21.15.46:0/3966989918 >> 172.21.15.46:0/3966989918 conn(0x56495b402300 msgr2=0x56495b7b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.630+0000 7f37d5b7b740 1 -- 172.21.15.46:0/3966989918 shutdown_connections 2024-05-09T04:53:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.630+0000 7f37d5b7b740 1 -- 172.21.15.46:0/3966989918 wait complete. 2024-05-09T04:53:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.630+0000 7f37d5b7b740 1 -- start start 2024-05-09T04:53:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.630+0000 7f37d5b7b740 1 Processor -- start 2024-05-09T04:53:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37d5b7b740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b3e52f0 0x56495b7c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37d5b7b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3ed4d0 0x56495b7c3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37d5b7b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b40e1b0 0x56495b7c3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37d5b7b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81ac050 con 0x56495b40e1b0 2024-05-09T04:53:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37d5b7b740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81ad4c0 con 0x56495b3e52f0 2024-05-09T04:53:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37d5b7b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f37c81ac720 con 0x56495b3ed4d0 2024-05-09T04:53:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37d5b7b740 10 client.0 _ll_register_callbacks cb 0x7f37b4013630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b40e1b0 0x56495b7c3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37c7fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b40e1b0 0x56495b7c3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44498/0 (socket says 172.21.15.46:44498) 2024-05-09T04:53:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37c7fff640 1 -- 172.21.15.46:0/145115306 learned_addr learned my addr 172.21.15.46:0/145115306 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37c7fff640 1 --2- 172.21.15.46:0/145115306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b3e52f0 0x56495b7c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37cca59640 1 --2- 172.21.15.46:0/145115306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3ed4d0 0x56495b7c3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37c7fff640 1 -- 172.21.15.46:0/145115306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3ed4d0 msgr2=0x56495b7c3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37c7fff640 1 --2- 172.21.15.46:0/145115306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3ed4d0 0x56495b7c3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37c7fff640 1 -- 172.21.15.46:0/145115306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b3e52f0 msgr2=0x56495b7c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:03.635 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37c7fff640 1 --2- 172.21.15.46:0/145115306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b3e52f0 0x56495b7c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37c7fff640 1 -- 172.21.15.46:0/145115306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56495b6daed0 con 0x56495b40e1b0 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37cca59640 1 --2- 172.21.15.46:0/145115306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3ed4d0 0x56495b7c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37c7fff640 1 --2- 172.21.15.46:0/145115306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b40e1b0 0x56495b7c3b90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f37b4005940 tx=0x7f37b400ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4bfaf7776363959 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.631+0000 7f37a7fff640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.632+0000 7f37a7fff640 1 -- 172.21.15.46:0/145115306 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37b4004030 con 0x56495b40e1b0 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.632+0000 7f37a7fff640 1 -- 172.21.15.46:0/145115306 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f37b40041d0 con 0x56495b40e1b0 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.632+0000 7f37d5b7b740 1 -- 172.21.15.46:0/145115306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37b40138b0 con 0x56495b40e1b0 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.632+0000 7f37a7fff640 1 -- 172.21.15.46:0/145115306 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f37b4046050 con 0x56495b40e1b0 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.632+0000 7f37d5b7b740 10 client.15810 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.632+0000 7f37d5b7b740 1 -- 172.21.15.46:0/145115306 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56495b83a290 con 0x56495b40e1b0 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.632+0000 7f37a7fff640 1 -- 172.21.15.46:0/145115306 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37b40138b0 con 0x56495b40e1b0 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.632+0000 7f37a7fff640 1 -- 172.21.15.46:0/145115306 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f37b4003780 con 0x56495b40e1b0 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.632+0000 7f37d5b7b740 10 client.15810 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.632+0000 7f37d5b7b740 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.632+0000 7f37d5b7b740 20 client.15810 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:03.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.632+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/145115306 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56495b3a21c0 0x56495b83c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.632+0000 7f37d5b7b740 4 client.15810 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T04:53:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.632+0000 7f37d5b7b740 1 -- 172.21.15.46:0/145115306 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56495b71e240 con 0x56495b3a21c0 2024-05-09T04:53:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.633+0000 7f37cca59640 1 --2- 172.21.15.46:0/145115306 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56495b3a21c0 0x56495b83c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.633+0000 7f37a7fff640 10 client.15810 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:03.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.633+0000 7f37cca59640 1 --2- 172.21.15.46:0/145115306 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56495b3a21c0 0x56495b83c8a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=89b9ae4353db012e server_cookie=626ffd6c430f326f in_seq=0 out_seq=0 2024-05-09T04:53:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.644+0000 7f37a7fff640 1 -- 172.21.15.46:0/145115306 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x56495b71e240 con 0x56495b3a21c0 2024-05-09T04:53:03.644 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.644+0000 7f37a7fff640 10 client.15810 handle_command_reply: tid=0 2024-05-09T04:53:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.644+0000 7f37d5b7b740 1 client.15810 shutdown 2024-05-09T04:53:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.644+0000 7f37d5b7b740 1 -- 172.21.15.46:0/145115306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b40e1b0 msgr2=0x56495b7c3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.644+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/145115306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b40e1b0 0x56495b7c3b90 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f37b4005940 tx=0x7f37b400ec90 comp rx=0 tx=0).stop 2024-05-09T04:53:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.644+0000 7f37d5b7b740 1 -- 172.21.15.46:0/145115306 shutdown_connections 2024-05-09T04:53:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.644+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/145115306 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56495b3a21c0 0x56495b83c8a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.644+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/145115306 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56495b3ed4d0 0x56495b7c3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.644+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/145115306 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56495b40e1b0 0x56495b7c3b90 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.644+0000 7f37d5b7b740 1 --2- 172.21.15.46:0/145115306 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56495b3e52f0 0x56495b7c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.644+0000 7f37a7fff640 0 client.15810 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.644+0000 7f37d5b7b740 1 -- 172.21.15.46:0/145115306 >> 172.21.15.46:0/145115306 conn(0x56495b402300 msgr2=0x56495b7b3d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.644+0000 7f37d5b7b740 1 -- 172.21.15.46:0/145115306 shutdown_connections 2024-05-09T04:53:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.644+0000 7f37d5b7b740 1 -- 172.21.15.46:0/145115306 wait complete. 2024-05-09T04:53:03.645 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.644+0000 7f37d5b7b740 20 client.15810 trim_cache size 0 max 16384 2024-05-09T04:53:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-05-09T04:53:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-05-09T04:53:03.646 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-05-09T04:53:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.645+0000 7f37cecbd640 1 -- 172.21.15.46:0/2889080963 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f37b80b29d0 msgr2=0x7f37b80b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.645+0000 7f37cecbd640 1 --2- 172.21.15.46:0/2889080963 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f37b80b29d0 0x7f37b80b4ee0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f37c00a5670 tx=0x7f37c00a55e0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.645+0000 7f37cecbd640 1 -- 172.21.15.46:0/2889080963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37c810a140 msgr2=0x7f37c819a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.645+0000 7f37cecbd640 1 --2- 172.21.15.46:0/2889080963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37c810a140 0x7f37c819a7e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f37c0009fd0 tx=0x7f37c000b260 comp rx=0 tx=0).stop 2024-05-09T04:53:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.645+0000 7f37cecbd640 1 -- 172.21.15.46:0/2889080963 shutdown_connections 2024-05-09T04:53:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.645+0000 7f37cecbd640 1 --2- 172.21.15.46:0/2889080963 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f37b80b29d0 0x7f37b80b4ee0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.645+0000 7f37cecbd640 1 --2- 172.21.15.46:0/2889080963 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f37c810a140 0x7f37c819a7e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.645+0000 7f37cecbd640 1 --2- 172.21.15.46:0/2889080963 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f37c8109ca0 0x7f37c81a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.645+0000 7f37cecbd640 1 --2- 172.21.15.46:0/2889080963 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f37c81043f0 0x7f37c81a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.645+0000 7f37cecbd640 1 -- 172.21.15.46:0/2889080963 >> 172.21.15.46:0/2889080963 conn(0x7f37c80ff340 msgr2=0x7f37c80ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.645+0000 7f37cecbd640 1 -- 172.21.15.46:0/2889080963 shutdown_connections 2024-05-09T04:53:03.647 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:03.645+0000 7f37cecbd640 1 -- 172.21.15.46:0/2889080963 wait complete. 2024-05-09T04:53:04.344 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:53:04.601 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:04.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:04 smithi046 ceph-mon[30136]: pgmap v62: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 126 MiB/s wr, 52 op/s 2024-05-09T04:53:05.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:04 smithi110 ceph-mon[34904]: pgmap v62: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 126 MiB/s wr, 52 op/s 2024-05-09T04:53:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:04 smithi072 ceph-mon[34088]: pgmap v62: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 126 MiB/s wr, 52 op/s 2024-05-09T04:53:06.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:05 smithi046 ceph-mon[30136]: pgmap v63: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 126 MiB/s wr, 52 op/s 2024-05-09T04:53:06.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:05 smithi110 ceph-mon[34904]: pgmap v63: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 126 MiB/s wr, 52 op/s 2024-05-09T04:53:06.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:05 smithi072 ceph-mon[34088]: pgmap v63: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 126 MiB/s wr, 52 op/s 2024-05-09T04:53:06.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.430+0000 7fad462b7700 1 -- 172.21.15.46:0/1129297683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad40109850 msgr2=0x7fad401111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:06.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.430+0000 7fad462b7700 1 --2- 172.21.15.46:0/1129297683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad40109850 0x7fad401111b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fad3800b0d0 tx=0x7fad3800b3e0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.431+0000 7fad462b7700 1 -- 172.21.15.46:0/1129297683 shutdown_connections 2024-05-09T04:53:06.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.431+0000 7fad462b7700 1 --2- 172.21.15.46:0/1129297683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad40109850 0x7fad401111b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.431+0000 7fad462b7700 1 --2- 172.21.15.46:0/1129297683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad401045d0 0x7fad401049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.431+0000 7fad462b7700 1 --2- 172.21.15.46:0/1129297683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad40108f00 0x7fad40109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.431+0000 7fad462b7700 1 -- 172.21.15.46:0/1129297683 >> 172.21.15.46:0/1129297683 conn(0x7fad400ffea0 msgr2=0x7fad401022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:06.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.431+0000 7fad462b7700 1 -- 172.21.15.46:0/1129297683 shutdown_connections 2024-05-09T04:53:06.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.431+0000 7fad462b7700 1 -- 172.21.15.46:0/1129297683 wait complete. 2024-05-09T04:53:06.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.431+0000 7fad462b7700 1 Processor -- start 2024-05-09T04:53:06.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.431+0000 7fad462b7700 1 -- start start 2024-05-09T04:53:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad462b7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad401045d0 0x7fad4019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad462b7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad40108f00 0x7fad4019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad462b7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad40109850 0x7fad4019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad462b7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad40114480 con 0x7fad401045d0 2024-05-09T04:53:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad462b7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fad40114300 con 0x7fad40109850 2024-05-09T04:53:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad462b7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fad40114600 con 0x7fad40108f00 2024-05-09T04:53:06.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad3ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad401045d0 0x7fad4019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad3ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad401045d0 0x7fad4019adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47224/0 (socket says 172.21.15.46:47224) 2024-05-09T04:53:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad3ffff700 1 -- 172.21.15.46:0/2327201329 learned_addr learned my addr 172.21.15.46:0/2327201329 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad44854700 1 --2- 172.21.15.46:0/2327201329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad40109850 0x7fad4019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/2327201329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad40108f00 0x7fad4019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad3ffff700 1 -- 172.21.15.46:0/2327201329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad40108f00 msgr2=0x7fad4019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad3ffff700 1 --2- 172.21.15.46:0/2327201329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad40108f00 0x7fad4019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad3ffff700 1 -- 172.21.15.46:0/2327201329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad40109850 msgr2=0x7fad4019f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad3ffff700 1 --2- 172.21.15.46:0/2327201329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad40109850 0x7fad4019f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad3ffff700 1 -- 172.21.15.46:0/2327201329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad4019fcc0 con 0x7fad401045d0 2024-05-09T04:53:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.432+0000 7fad3ffff700 1 --2- 172.21.15.46:0/2327201329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad401045d0 0x7fad4019adf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fad3000d950 tx=0x7fad3000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.433+0000 7fad3d7fa700 1 -- 172.21.15.46:0/2327201329 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30016070 con 0x7fad401045d0 2024-05-09T04:53:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.433+0000 7fad3d7fa700 1 -- 172.21.15.46:0/2327201329 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad30009df0 con 0x7fad401045d0 2024-05-09T04:53:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.433+0000 7fad462b7700 1 -- 172.21.15.46:0/2327201329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad4019ffb0 con 0x7fad401045d0 2024-05-09T04:53:06.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.433+0000 7fad3d7fa700 1 -- 172.21.15.46:0/2327201329 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad3000f3c0 con 0x7fad401045d0 2024-05-09T04:53:06.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.433+0000 7fad462b7700 1 -- 172.21.15.46:0/2327201329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad401a78d0 con 0x7fad401045d0 2024-05-09T04:53:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.435+0000 7fad3d7fa700 1 -- 172.21.15.46:0/2327201329 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fad30019070 con 0x7fad401045d0 2024-05-09T04:53:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.436+0000 7fad3d7fa700 1 --2- 172.21.15.46:0/2327201329 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fad280b2a70 0x7fad280b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.436+0000 7fad3d7fa700 1 -- 172.21.15.46:0/2327201329 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fad300e2110 con 0x7fad401045d0 2024-05-09T04:53:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.436+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/2327201329 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fad280b2a70 0x7fad280b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.436+0000 7fad54ccdb80 1 Processor -- start 2024-05-09T04:53:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad54ccdb80 1 -- start start 2024-05-09T04:53:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/2327201329 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fad280b2a70 0x7fad280b4f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad34005950 tx=0x7fad340058e0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:06.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad54ccdb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21fc5160 0x561f2205e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad54ccdb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f21edb4a0 0x561f2205ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad54ccdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f220586a0 0x561f22058ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad54ccdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561f21f7aff0 con 0x561f220586a0 2024-05-09T04:53:06.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad54ccdb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x561f21f197f0 con 0x561f21edb4a0 2024-05-09T04:53:06.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad54ccdb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561f21f76fb0 con 0x561f21fc5160 2024-05-09T04:53:06.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad44854700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f220586a0 0x561f22058ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad44854700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f220586a0 0x561f22058ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47234/0 (socket says 172.21.15.46:47234) 2024-05-09T04:53:06.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad44854700 1 -- 172.21.15.46:0/2859974463 learned_addr learned my addr 172.21.15.46:0/2859974463 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:06.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad3ffff700 1 --2- 172.21.15.46:0/2859974463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21fc5160 0x561f2205e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/2859974463 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f21edb4a0 0x561f2205ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad44854700 1 -- 172.21.15.46:0/2859974463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21fc5160 msgr2=0x561f2205e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:06.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad44854700 1 --2- 172.21.15.46:0/2859974463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21fc5160 0x561f2205e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad44854700 1 -- 172.21.15.46:0/2859974463 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f21edb4a0 msgr2=0x561f2205ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:06.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad44854700 1 --2- 172.21.15.46:0/2859974463 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f21edb4a0 0x561f2205ea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad44854700 1 -- 172.21.15.46:0/2859974463 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f21f11370 con 0x561f220586a0 2024-05-09T04:53:06.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.437+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/2859974463 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f21edb4a0 0x561f2205ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:06.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.438+0000 7fad44854700 1 --2- 172.21.15.46:0/2859974463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f220586a0 0x561f22058ad0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fad38009cb0 tx=0x7fad38009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4e5ebec725e139 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.438+0000 7fad267fc700 1 -- 172.21.15.46:0/2859974463 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad38009350 con 0x561f220586a0 2024-05-09T04:53:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.438+0000 7fad267fc700 1 -- 172.21.15.46:0/2859974463 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad38006000 con 0x561f220586a0 2024-05-09T04:53:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.438+0000 7fad267fc700 1 -- 172.21.15.46:0/2859974463 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad380075c0 con 0x561f220586a0 2024-05-09T04:53:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.439+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2859974463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f220586a0 msgr2=0x561f22058ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.439+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/2859974463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f220586a0 0x561f22058ad0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fad38009cb0 tx=0x7fad38009ce0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.439+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2859974463 shutdown_connections 2024-05-09T04:53:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.439+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/2859974463 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f220586a0 0x561f22058ad0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.439+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/2859974463 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21fc5160 0x561f2205e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.439+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/2859974463 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f21edb4a0 0x561f2205ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.439+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2859974463 >> 172.21.15.46:0/2859974463 conn(0x561f21f3b6a0 msgr2=0x561f22053400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.439+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2859974463 shutdown_connections 2024-05-09T04:53:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.439+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2859974463 wait complete. 2024-05-09T04:53:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.439+0000 7fad54ccdb80 1 -- start start 2024-05-09T04:53:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad54ccdb80 1 Processor -- start 2024-05-09T04:53:06.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad54ccdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21edb4a0 0x561f22062230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad54ccdb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21fc5160 0x561f22062770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad54ccdb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f220586a0 0x561f22062cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad54ccdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561f21df6ba0 con 0x561f21edb4a0 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad54ccdb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x561f21f35150 con 0x561f220586a0 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad54ccdb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561f220651d0 con 0x561f21fc5160 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad3ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21edb4a0 0x561f22062230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad3ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21edb4a0 0x561f22062230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47244/0 (socket says 172.21.15.46:47244) 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad3ffff700 1 -- 172.21.15.46:0/2830096050 learned_addr learned my addr 172.21.15.46:0/2830096050 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad54ccdb80 10 client.0 _ll_register_callbacks cb 0x561f21dec6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/2830096050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21fc5160 0x561f22062770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad44854700 1 --2- 172.21.15.46:0/2830096050 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f220586a0 0x561f22062cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad3ffff700 1 -- 172.21.15.46:0/2830096050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21fc5160 msgr2=0x561f22062770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad3ffff700 1 --2- 172.21.15.46:0/2830096050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21fc5160 0x561f22062770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad3ffff700 1 -- 172.21.15.46:0/2830096050 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f220586a0 msgr2=0x561f22062cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad3ffff700 1 --2- 172.21.15.46:0/2830096050 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f220586a0 0x561f22062cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad3ffff700 1 -- 172.21.15.46:0/2830096050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f21df6490 con 0x561f21edb4a0 2024-05-09T04:53:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.440+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/2830096050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21fc5160 0x561f22062770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad3ffff700 1 --2- 172.21.15.46:0/2830096050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21edb4a0 0x561f22062230 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fad300a9170 tx=0x7fad30010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860b89dd26e445ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad1f7fe700 1 -- 172.21.15.46:0/2830096050 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30020070 con 0x561f21edb4a0 2024-05-09T04:53:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad1f7fe700 1 -- 172.21.15.46:0/2830096050 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fad300a98e0 con 0x561f21edb4a0 2024-05-09T04:53:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2830096050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad38006000 con 0x561f21edb4a0 2024-05-09T04:53:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad1f7fe700 1 -- 172.21.15.46:0/2830096050 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fad300ad820 con 0x561f21edb4a0 2024-05-09T04:53:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad54ccdb80 10 client.15822 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2830096050 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f2206fc10 con 0x561f21edb4a0 2024-05-09T04:53:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad1f7fe700 1 -- 172.21.15.46:0/2830096050 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad38006000 con 0x561f21edb4a0 2024-05-09T04:53:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad1f7fe700 1 -- 172.21.15.46:0/2830096050 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fad300ca300 con 0x561f21edb4a0 2024-05-09T04:53:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad54ccdb80 10 client.15822 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:06.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad54ccdb80 10 client.15822 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:53:06.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad54ccdb80 20 client.15822 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:06.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/2830096050 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x561f220721f0 0x561f22074650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad54ccdb80 4 client.15822 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:06.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.441+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2830096050 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f21f889d0 con 0x561f220721f0 2024-05-09T04:53:06.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.442+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/2830096050 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x561f220721f0 0x561f22074650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.442+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/2830096050 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x561f220721f0 0x561f22074650 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b312c23ddcd4a4cc server_cookie=4d4c50d6656e7764 in_seq=0 out_seq=0 2024-05-09T04:53:06.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.442+0000 7fad1f7fe700 10 client.15822 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:53:06.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.443+0000 7fad1f7fe700 1 -- 172.21.15.46:0/2830096050 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561f21f889d0 con 0x561f220721f0 2024-05-09T04:53:06.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.443+0000 7fad1f7fe700 10 client.15822 handle_command_reply: tid=0 2024-05-09T04:53:06.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.443+0000 7fad54ccdb80 1 client.15822 shutdown 2024-05-09T04:53:06.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.443+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2830096050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21edb4a0 msgr2=0x561f22062230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.443+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/2830096050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21edb4a0 0x561f22062230 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fad300a9170 tx=0x7fad30010940 comp rx=0 tx=0).stop 2024-05-09T04:53:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.443+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2830096050 shutdown_connections 2024-05-09T04:53:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.443+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/2830096050 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x561f220721f0 0x561f22074650 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.443+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/2830096050 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f220586a0 0x561f22062cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.443+0000 7fad1f7fe700 0 client.15822 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:53:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.443+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/2830096050 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21edb4a0 0x561f22062230 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.443+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/2830096050 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21fc5160 0x561f22062770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.443+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2830096050 >> 172.21.15.46:0/2830096050 conn(0x561f21f3b6a0 msgr2=0x561f220531c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.444+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2830096050 shutdown_connections 2024-05-09T04:53:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.444+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2830096050 wait complete. 2024-05-09T04:53:06.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.444+0000 7fad54ccdb80 20 client.15822 trim_cache size 0 max 16384 2024-05-09T04:53:06.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad54ccdb80 1 Processor -- start 2024-05-09T04:53:06.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad54ccdb80 1 -- start start 2024-05-09T04:53:06.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad54ccdb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21edb4a0 0x561f2205e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad54ccdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21fc5160 0x561f2205c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad54ccdb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f220586a0 0x561f22062af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad54ccdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad401b15a0 con 0x561f21fc5160 2024-05-09T04:53:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad54ccdb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fad401aee30 con 0x561f220586a0 2024-05-09T04:53:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad54ccdb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fad401acc80 con 0x561f21edb4a0 2024-05-09T04:53:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad3f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21fc5160 0x561f2205c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad3f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21fc5160 0x561f2205c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47246/0 (socket says 172.21.15.46:47246) 2024-05-09T04:53:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad3f7fe700 1 -- 172.21.15.46:0/2230211997 learned_addr learned my addr 172.21.15.46:0/2230211997 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad3ffff700 1 --2- 172.21.15.46:0/2230211997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21edb4a0 0x561f2205e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad3f7fe700 1 -- 172.21.15.46:0/2230211997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21edb4a0 msgr2=0x561f2205e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/2230211997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21edb4a0 0x561f2205e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad3f7fe700 1 -- 172.21.15.46:0/2230211997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f220586a0 msgr2=0x561f22062af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/2230211997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f220586a0 0x561f22062af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad3f7fe700 1 -- 172.21.15.46:0/2230211997 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f22059640 con 0x561f21fc5160 2024-05-09T04:53:06.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.459+0000 7fad3ffff700 1 --2- 172.21.15.46:0/2230211997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21edb4a0 0x561f2205e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:53:06.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.460+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/2230211997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21fc5160 0x561f2205c7c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fad3400b9c0 tx=0x7fad34015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd5a8fd0b3fb2bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:06.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.460+0000 7fad25ffb700 1 -- 172.21.15.46:0/2230211997 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34021e40 con 0x561f21fc5160 2024-05-09T04:53:06.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.460+0000 7fad25ffb700 1 -- 172.21.15.46:0/2230211997 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad3402f070 con 0x561f21fc5160 2024-05-09T04:53:06.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.460+0000 7fad25ffb700 1 -- 172.21.15.46:0/2230211997 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad34011b20 con 0x561f21fc5160 2024-05-09T04:53:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.461+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2230211997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21fc5160 msgr2=0x561f2205c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.461+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/2230211997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21fc5160 0x561f2205c7c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fad3400b9c0 tx=0x7fad34015000 comp rx=0 tx=0).stop 2024-05-09T04:53:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.461+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2230211997 shutdown_connections 2024-05-09T04:53:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.461+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/2230211997 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f220586a0 0x561f22062af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.461+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/2230211997 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f21edb4a0 0x561f2205e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.461+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/2230211997 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21fc5160 0x561f2205c7c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.461+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2230211997 >> 172.21.15.46:0/2230211997 conn(0x561f21f3b6a0 msgr2=0x561f22060b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.461+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2230211997 shutdown_connections 2024-05-09T04:53:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.461+0000 7fad54ccdb80 1 -- 172.21.15.46:0/2230211997 wait complete. 2024-05-09T04:53:06.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.461+0000 7fad54ccdb80 1 -- start start 2024-05-09T04:53:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.461+0000 7fad54ccdb80 1 Processor -- start 2024-05-09T04:53:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad54ccdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21edb4a0 0x561f2206aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad54ccdb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f21fc5160 0x561f22061ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad54ccdb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f220586a0 0x561f22062040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad54ccdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fad401ad2f0 con 0x561f21edb4a0 2024-05-09T04:53:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad54ccdb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fad401ae760 con 0x561f21fc5160 2024-05-09T04:53:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad54ccdb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fad401ad9c0 con 0x561f220586a0 2024-05-09T04:53:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad54ccdb80 10 client.0 _ll_register_callbacks cb 0x561f21dec6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad3ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21edb4a0 0x561f2206aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad3ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21edb4a0 0x561f2206aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47262/0 (socket says 172.21.15.46:47262) 2024-05-09T04:53:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad3ffff700 1 -- 172.21.15.46:0/4247591593 learned_addr learned my addr 172.21.15.46:0/4247591593 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/4247591593 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f21fc5160 0x561f22061ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad44854700 1 --2- 172.21.15.46:0/4247591593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f220586a0 0x561f22062040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.467 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad3ffff700 1 -- 172.21.15.46:0/4247591593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f220586a0 msgr2=0x561f22062040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad3ffff700 1 --2- 172.21.15.46:0/4247591593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f220586a0 0x561f22062040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad3ffff700 1 -- 172.21.15.46:0/4247591593 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f21fc5160 msgr2=0x561f22061ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad3ffff700 1 --2- 172.21.15.46:0/4247591593 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f21fc5160 0x561f22061ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad3ffff700 1 -- 172.21.15.46:0/4247591593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f2204cf60 con 0x561f21edb4a0 2024-05-09T04:53:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/4247591593 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f21fc5160 0x561f22061ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad44854700 1 --2- 172.21.15.46:0/4247591593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f220586a0 0x561f22062040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:53:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad3ffff700 1 --2- 172.21.15.46:0/4247591593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21edb4a0 0x561f2206aa00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fad300a9400 tx=0x7fad30005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b309982fa9a91111 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.462+0000 7fad1f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.463+0000 7fad1f7fe700 1 -- 172.21.15.46:0/4247591593 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad30020070 con 0x561f21edb4a0 2024-05-09T04:53:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.463+0000 7fad1f7fe700 1 -- 172.21.15.46:0/4247591593 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fad300ca350 con 0x561f21edb4a0 2024-05-09T04:53:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.463+0000 7fad54ccdb80 1 -- 172.21.15.46:0/4247591593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad3402f070 con 0x561f21edb4a0 2024-05-09T04:53:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.463+0000 7fad1f7fe700 1 -- 172.21.15.46:0/4247591593 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fad300b22d0 con 0x561f21edb4a0 2024-05-09T04:53:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.463+0000 7fad54ccdb80 10 client.15828 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.463+0000 7fad54ccdb80 1 -- 172.21.15.46:0/4247591593 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f22054d00 con 0x561f21edb4a0 2024-05-09T04:53:06.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.463+0000 7fad1f7fe700 1 -- 172.21.15.46:0/4247591593 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad3402f070 con 0x561f21edb4a0 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.463+0000 7fad1f7fe700 1 -- 172.21.15.46:0/4247591593 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fad300a4de0 con 0x561f21edb4a0 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.463+0000 7fad54ccdb80 10 client.15828 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.463+0000 7fad54ccdb80 10 client.15828 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.463+0000 7fad54ccdb80 20 client.15828 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.463+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/4247591593 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x561f220721f0 0x561f220dea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.463+0000 7fad54ccdb80 4 client.15828 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.463+0000 7fad54ccdb80 1 -- 172.21.15.46:0/4247591593 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561f21f82a70 con 0x561f220721f0 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.464+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/4247591593 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x561f220721f0 0x561f220dea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.464+0000 7fad1f7fe700 10 client.15828 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.464+0000 7fad3f7fe700 1 --2- 172.21.15.46:0/4247591593 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x561f220721f0 0x561f220dea20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=19ddc785f7359aeb server_cookie=b32ae2673c432d64 in_seq=0 out_seq=0 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.464+0000 7fad1f7fe700 1 -- 172.21.15.46:0/4247591593 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561f21f82a70 con 0x561f220721f0 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.464+0000 7fad1f7fe700 10 client.15828 handle_command_reply: tid=0 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.464+0000 7fad54ccdb80 1 client.15828 shutdown 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.464+0000 7fad54ccdb80 1 -- 172.21.15.46:0/4247591593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21edb4a0 msgr2=0x561f2206aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/4247591593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21edb4a0 0x561f2206aa00 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fad300a9400 tx=0x7fad30005910 comp rx=0 tx=0).stop 2024-05-09T04:53:06.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad54ccdb80 1 -- 172.21.15.46:0/4247591593 shutdown_connections 2024-05-09T04:53:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/4247591593 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x561f220721f0 0x561f220dea20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/4247591593 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561f220586a0 0x561f22062040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/4247591593 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561f21edb4a0 0x561f2206aa00 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad54ccdb80 1 --2- 172.21.15.46:0/4247591593 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561f21fc5160 0x561f22061ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad54ccdb80 1 -- 172.21.15.46:0/4247591593 >> 172.21.15.46:0/4247591593 conn(0x561f21f3b6a0 msgr2=0x561f2205be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad1f7fe700 0 client.15828 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:53:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad54ccdb80 1 -- 172.21.15.46:0/4247591593 shutdown_connections 2024-05-09T04:53:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad54ccdb80 1 -- 172.21.15.46:0/4247591593 wait complete. 2024-05-09T04:53:06.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad54ccdb80 20 client.15828 trim_cache size 0 max 16384 2024-05-09T04:53:06.471 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:53:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad462b7700 1 -- 172.21.15.46:0/2327201329 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fad280b2a70 msgr2=0x7fad280b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad462b7700 1 --2- 172.21.15.46:0/2327201329 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fad280b2a70 0x7fad280b4f80 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fad34005950 tx=0x7fad340058e0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad462b7700 1 -- 172.21.15.46:0/2327201329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad401045d0 msgr2=0x7fad4019adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad462b7700 1 --2- 172.21.15.46:0/2327201329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad401045d0 0x7fad4019adf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fad3000d950 tx=0x7fad3000dc60 comp rx=0 tx=0).stop 2024-05-09T04:53:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad462b7700 1 -- 172.21.15.46:0/2327201329 shutdown_connections 2024-05-09T04:53:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.465+0000 7fad462b7700 1 --2- 172.21.15.46:0/2327201329 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fad280b2a70 0x7fad280b4f80 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.466+0000 7fad462b7700 1 --2- 172.21.15.46:0/2327201329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fad40109850 0x7fad4019f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.466+0000 7fad462b7700 1 --2- 172.21.15.46:0/2327201329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fad401045d0 0x7fad4019adf0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.466+0000 7fad462b7700 1 --2- 172.21.15.46:0/2327201329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fad40108f00 0x7fad4019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.466+0000 7fad462b7700 1 -- 172.21.15.46:0/2327201329 >> 172.21.15.46:0/2327201329 conn(0x7fad400ffea0 msgr2=0x7fad40108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.466+0000 7fad462b7700 1 -- 172.21.15.46:0/2327201329 shutdown_connections 2024-05-09T04:53:06.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:06.466+0000 7fad462b7700 1 -- 172.21.15.46:0/2327201329 wait complete. 2024-05-09T04:53:07.176 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:53:07.435 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:08.970 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:08 smithi046 ceph-mon[30136]: pgmap v64: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 126 MiB/s wr, 57 op/s 2024-05-09T04:53:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:08 smithi110 ceph-mon[34904]: pgmap v64: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 126 MiB/s wr, 57 op/s 2024-05-09T04:53:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:08 smithi072 ceph-mon[34088]: pgmap v64: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 126 MiB/s wr, 57 op/s 2024-05-09T04:53:09.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.203+0000 7fa1b01fd700 1 -- 172.21.15.46:0/1770947450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a8107e10 msgr2=0x7fa1a8108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:09.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.203+0000 7fa1b01fd700 1 --2- 172.21.15.46:0/1770947450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a8107e10 0x7fa1a8108220 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fa1980099c0 tx=0x7fa198009cd0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.203+0000 7fa1b01fd700 1 -- 172.21.15.46:0/1770947450 shutdown_connections 2024-05-09T04:53:09.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.203+0000 7fa1b01fd700 1 --2- 172.21.15.46:0/1770947450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8108760 0x7fa1a81100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.203+0000 7fa1b01fd700 1 --2- 172.21.15.46:0/1770947450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1a81034e0 0x7fa1a81038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.203+0000 7fa1b01fd700 1 --2- 172.21.15.46:0/1770947450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a8107e10 0x7fa1a8108220 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.203+0000 7fa1b01fd700 1 -- 172.21.15.46:0/1770947450 >> 172.21.15.46:0/1770947450 conn(0x7fa1a80fef30 msgr2=0x7fa1a8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:09.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.203+0000 7fa1b01fd700 1 -- 172.21.15.46:0/1770947450 shutdown_connections 2024-05-09T04:53:09.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.203+0000 7fa1b01fd700 1 -- 172.21.15.46:0/1770947450 wait complete. 2024-05-09T04:53:09.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.204+0000 7fa1b01fd700 1 Processor -- start 2024-05-09T04:53:09.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.204+0000 7fa1b01fd700 1 -- start start 2024-05-09T04:53:09.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.204+0000 7fa1b01fd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a81034e0 0x7fa1a819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.204+0000 7fa1b01fd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1a8107e10 0x7fa1a819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.204+0000 7fa1b01fd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8108760 0x7fa1a819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.204+0000 7fa1b01fd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a8113370 con 0x7fa1a81034e0 2024-05-09T04:53:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.204+0000 7fa1b01fd700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81131f0 con 0x7fa1a8107e10 2024-05-09T04:53:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.204+0000 7fa1b01fd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81134f0 con 0x7fa1a8108760 2024-05-09T04:53:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.204+0000 7fa1adf99700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a81034e0 0x7fa1a819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.204+0000 7fa1adf99700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a81034e0 0x7fa1a819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47274/0 (socket says 172.21.15.46:47274) 2024-05-09T04:53:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.204+0000 7fa1adf99700 1 -- 172.21.15.46:0/78408607 learned_addr learned my addr 172.21.15.46:0/78408607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.204+0000 7fa1ad798700 1 --2- 172.21.15.46:0/78408607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1a8107e10 0x7fa1a819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.204+0000 7fa1ae79a700 1 --2- 172.21.15.46:0/78408607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8108760 0x7fa1a819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.204+0000 7fa1adf99700 1 -- 172.21.15.46:0/78408607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8108760 msgr2=0x7fa1a819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.205+0000 7fa1adf99700 1 --2- 172.21.15.46:0/78408607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8108760 0x7fa1a819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.205+0000 7fa1adf99700 1 -- 172.21.15.46:0/78408607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1a8107e10 msgr2=0x7fa1a819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.205+0000 7fa1adf99700 1 --2- 172.21.15.46:0/78408607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1a8107e10 0x7fa1a819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.205+0000 7fa1adf99700 1 -- 172.21.15.46:0/78408607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a819fce0 con 0x7fa1a81034e0 2024-05-09T04:53:09.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.205+0000 7fa1adf99700 1 --2- 172.21.15.46:0/78408607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a81034e0 0x7fa1a819ae10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400c920 tx=0x7fa1a400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:09.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.205+0000 7fa19effd700 1 -- 172.21.15.46:0/78408607 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4007b00 con 0x7fa1a81034e0 2024-05-09T04:53:09.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.205+0000 7fa1b01fd700 1 -- 172.21.15.46:0/78408607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a819ffd0 con 0x7fa1a81034e0 2024-05-09T04:53:09.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.205+0000 7fa19effd700 1 -- 172.21.15.46:0/78408607 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1a4004ce0 con 0x7fa1a81034e0 2024-05-09T04:53:09.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.205+0000 7fa19effd700 1 -- 172.21.15.46:0/78408607 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4005500 con 0x7fa1a81034e0 2024-05-09T04:53:09.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.205+0000 7fa1b01fd700 1 -- 172.21.15.46:0/78408607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a81a78f0 con 0x7fa1a81034e0 2024-05-09T04:53:09.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.208+0000 7fa19effd700 1 -- 172.21.15.46:0/78408607 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa1a4007ca0 con 0x7fa1a81034e0 2024-05-09T04:53:09.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.208+0000 7fa19effd700 1 --2- 172.21.15.46:0/78408607 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa1940b29f0 0x7fa1940b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.209+0000 7fa1ad798700 1 --2- 172.21.15.46:0/78408607 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa1940b29f0 0x7fa1940b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.209+0000 7fa19effd700 1 -- 172.21.15.46:0/78408607 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa1a40e2110 con 0x7fa1a81034e0 2024-05-09T04:53:09.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.209+0000 7fa1bec13b80 1 Processor -- start 2024-05-09T04:53:09.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.209+0000 7fa1bec13b80 1 -- start start 2024-05-09T04:53:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.209+0000 7fa1ad798700 1 --2- 172.21.15.46:0/78408607 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa1940b29f0 0x7fa1940b4f00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa198006010 tx=0x7fa198000bc0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.209+0000 7fa1bec13b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140d09d00 0x563140e10ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.209+0000 7fa1bec13b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140c8f4a0 0x563140e11030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.209+0000 7fa1bec13b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140e151e0 0x563140e155f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.209+0000 7fa1bec13b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563140d2f0a0 con 0x563140d09d00 2024-05-09T04:53:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.209+0000 7fa1bec13b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x563140cd49a0 con 0x563140e151e0 2024-05-09T04:53:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.209+0000 7fa1bec13b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563140d2b060 con 0x563140c8f4a0 2024-05-09T04:53:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.209+0000 7fa1adf99700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140d09d00 0x563140e10ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.210+0000 7fa1adf99700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140d09d00 0x563140e10ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47286/0 (socket says 172.21.15.46:47286) 2024-05-09T04:53:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.210+0000 7fa1adf99700 1 -- 172.21.15.46:0/217228558 learned_addr learned my addr 172.21.15.46:0/217228558 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.210+0000 7fa1ad798700 1 --2- 172.21.15.46:0/217228558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140c8f4a0 0x563140e11030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.210+0000 7fa1ae79a700 1 --2- 172.21.15.46:0/217228558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140e151e0 0x563140e155f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.210+0000 7fa1adf99700 1 -- 172.21.15.46:0/217228558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140c8f4a0 msgr2=0x563140e11030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.210+0000 7fa1adf99700 1 --2- 172.21.15.46:0/217228558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140c8f4a0 0x563140e11030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.210+0000 7fa1adf99700 1 -- 172.21.15.46:0/217228558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140e151e0 msgr2=0x563140e155f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.210+0000 7fa1adf99700 1 --2- 172.21.15.46:0/217228558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140e151e0 0x563140e155f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.210+0000 7fa1adf99700 1 -- 172.21.15.46:0/217228558 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563140cc5500 con 0x563140d09d00 2024-05-09T04:53:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.210+0000 7fa1adf99700 1 --2- 172.21.15.46:0/217228558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140d09d00 0x563140e10ad0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4005950 tx=0x7fa1a40a8cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49763120374ee533 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.210+0000 7fa193fff700 1 -- 172.21.15.46:0/217228558 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a401d040 con 0x563140d09d00 2024-05-09T04:53:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.210+0000 7fa193fff700 1 -- 172.21.15.46:0/217228558 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1a40ad820 con 0x563140d09d00 2024-05-09T04:53:09.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.210+0000 7fa193fff700 1 -- 172.21.15.46:0/217228558 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40b1e40 con 0x563140d09d00 2024-05-09T04:53:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.211+0000 7fa1bec13b80 1 -- 172.21.15.46:0/217228558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140d09d00 msgr2=0x563140e10ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.211+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/217228558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140d09d00 0x563140e10ad0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4005950 tx=0x7fa1a40a8cb0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.211+0000 7fa1bec13b80 1 -- 172.21.15.46:0/217228558 shutdown_connections 2024-05-09T04:53:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.211+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/217228558 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140e151e0 0x563140e155f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.211+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/217228558 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140d09d00 0x563140e10ad0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.211+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/217228558 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140c8f4a0 0x563140e11030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.211+0000 7fa1bec13b80 1 -- 172.21.15.46:0/217228558 >> 172.21.15.46:0/217228558 conn(0x563140c92f70 msgr2=0x563140e07220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.211+0000 7fa1bec13b80 1 -- 172.21.15.46:0/217228558 shutdown_connections 2024-05-09T04:53:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.211+0000 7fa1bec13b80 1 -- 172.21.15.46:0/217228558 wait complete. 2024-05-09T04:53:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.212+0000 7fa1bec13b80 1 -- start start 2024-05-09T04:53:09.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.212+0000 7fa1bec13b80 1 Processor -- start 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.212+0000 7fa1bec13b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140c8f4a0 0x563140e1a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.212+0000 7fa1bec13b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140d09d00 0x563140e1aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.212+0000 7fa1bec13b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140e151e0 0x563140e1af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.212+0000 7fa1bec13b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563140e15e40 con 0x563140c8f4a0 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.212+0000 7fa1bec13b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x563140cc9b20 con 0x563140d09d00 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.212+0000 7fa1bec13b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563140e15fc0 con 0x563140e151e0 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.212+0000 7fa1bec13b80 10 client.0 _ll_register_callbacks cb 0x563140ba06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.212+0000 7fa1adf99700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140c8f4a0 0x563140e1a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.212+0000 7fa1adf99700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140c8f4a0 0x563140e1a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47300/0 (socket says 172.21.15.46:47300) 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1adf99700 1 -- 172.21.15.46:0/3725203970 learned_addr learned my addr 172.21.15.46:0/3725203970 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1ae79a700 1 --2- 172.21.15.46:0/3725203970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140e151e0 0x563140e1af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1ad798700 1 --2- 172.21.15.46:0/3725203970 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140d09d00 0x563140e1aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1adf99700 1 -- 172.21.15.46:0/3725203970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140e151e0 msgr2=0x563140e1af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1adf99700 1 --2- 172.21.15.46:0/3725203970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140e151e0 0x563140e1af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1adf99700 1 -- 172.21.15.46:0/3725203970 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140d09d00 msgr2=0x563140e1aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:09.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1adf99700 1 --2- 172.21.15.46:0/3725203970 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140d09d00 0x563140e1aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1adf99700 1 -- 172.21.15.46:0/3725203970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563140baa490 con 0x563140c8f4a0 2024-05-09T04:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1adf99700 1 --2- 172.21.15.46:0/3725203970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140c8f4a0 0x563140e1a4e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40df990 tx=0x7fa1a400ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90f20643bf59591 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1917fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1917fa700 1 -- 172.21.15.46:0/3725203970 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40c2020 con 0x563140c8f4a0 2024-05-09T04:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1917fa700 1 -- 172.21.15.46:0/3725203970 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa1a40b2050 con 0x563140c8f4a0 2024-05-09T04:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1bec13b80 1 -- 172.21.15.46:0/3725203970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a40ad820 con 0x563140c8f4a0 2024-05-09T04:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1917fa700 1 -- 172.21.15.46:0/3725203970 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa1a401d070 con 0x563140c8f4a0 2024-05-09T04:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1bec13b80 10 client.15840 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.213+0000 7fa1bec13b80 1 -- 172.21.15.46:0/3725203970 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563140e27dc0 con 0x563140c8f4a0 2024-05-09T04:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.214+0000 7fa1917fa700 1 -- 172.21.15.46:0/3725203970 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1a40ad820 con 0x563140c8f4a0 2024-05-09T04:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.214+0000 7fa1917fa700 1 -- 172.21.15.46:0/3725203970 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa1a40a4de0 con 0x563140c8f4a0 2024-05-09T04:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.214+0000 7fa1bec13b80 10 client.15840 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.214+0000 7fa1bec13b80 10 client.15840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.214+0000 7fa1bec13b80 20 client.15840 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:09.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.214+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/3725203970 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563140e2a290 0x563140e2c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.214+0000 7fa1bec13b80 4 client.15840 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:09.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.214+0000 7fa1bec13b80 1 -- 172.21.15.46:0/3725203970 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563140d3ca80 con 0x563140e2a290 2024-05-09T04:53:09.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.214+0000 7fa1ad798700 1 --2- 172.21.15.46:0/3725203970 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563140e2a290 0x563140e2c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.215+0000 7fa1ad798700 1 --2- 172.21.15.46:0/3725203970 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563140e2a290 0x563140e2c6b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d4fd3257220f1b6 server_cookie=9392cfc2c8c33a49 in_seq=0 out_seq=0 2024-05-09T04:53:09.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.215+0000 7fa1917fa700 10 client.15840 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:09.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.215+0000 7fa1917fa700 1 -- 172.21.15.46:0/3725203970 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563140d3ca80 con 0x563140e2a290 2024-05-09T04:53:09.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.215+0000 7fa1917fa700 10 client.15840 handle_command_reply: tid=0 2024-05-09T04:53:09.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.215+0000 7fa1bec13b80 1 client.15840 shutdown 2024-05-09T04:53:09.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.216+0000 7fa1bec13b80 1 -- 172.21.15.46:0/3725203970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140c8f4a0 msgr2=0x563140e1a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:09.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.216+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/3725203970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140c8f4a0 0x563140e1a4e0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40df990 tx=0x7fa1a400ce70 comp rx=0 tx=0).stop 2024-05-09T04:53:09.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.216+0000 7fa1bec13b80 1 -- 172.21.15.46:0/3725203970 shutdown_connections 2024-05-09T04:53:09.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.216+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/3725203970 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140e151e0 0x563140e1af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.216+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/3725203970 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140c8f4a0 0x563140e1a4e0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.216+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/3725203970 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563140e2a290 0x563140e2c6b0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.216+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/3725203970 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140d09d00 0x563140e1aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.216+0000 7fa1917fa700 0 client.15840 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.216+0000 7fa1bec13b80 1 -- 172.21.15.46:0/3725203970 >> 172.21.15.46:0/3725203970 conn(0x563140c92f70 msgr2=0x563140e00ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.216+0000 7fa1bec13b80 1 -- 172.21.15.46:0/3725203970 shutdown_connections 2024-05-09T04:53:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.216+0000 7fa1bec13b80 1 -- 172.21.15.46:0/3725203970 wait complete. 2024-05-09T04:53:09.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.216+0000 7fa1bec13b80 20 client.15840 trim_cache size 0 max 16384 2024-05-09T04:53:09.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.231+0000 7fa1bec13b80 1 Processor -- start 2024-05-09T04:53:09.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.231+0000 7fa1bec13b80 1 -- start start 2024-05-09T04:53:09.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.231+0000 7fa1bec13b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140c8f4a0 0x563140e01a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.231+0000 7fa1bec13b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140d09d00 0x563140e14b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.231+0000 7fa1bec13b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140e151e0 0x563140e1ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.231+0000 7fa1bec13b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81af6b0 con 0x563140c8f4a0 2024-05-09T04:53:09.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.231+0000 7fa1bec13b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81acf40 con 0x563140d09d00 2024-05-09T04:53:09.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.231+0000 7fa1bec13b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81aad90 con 0x563140e151e0 2024-05-09T04:53:09.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.231+0000 7fa1adf99700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140c8f4a0 0x563140e01a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.231+0000 7fa1adf99700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140c8f4a0 0x563140e01a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47310/0 (socket says 172.21.15.46:47310) 2024-05-09T04:53:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.231+0000 7fa1adf99700 1 -- 172.21.15.46:0/988776555 learned_addr learned my addr 172.21.15.46:0/988776555 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.232+0000 7fa1ad798700 1 --2- 172.21.15.46:0/988776555 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140d09d00 0x563140e14b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.232+0000 7fa1adf99700 1 -- 172.21.15.46:0/988776555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140e151e0 msgr2=0x563140e1ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.232+0000 7fa1adf99700 1 --2- 172.21.15.46:0/988776555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140e151e0 0x563140e1ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.232+0000 7fa1adf99700 1 -- 172.21.15.46:0/988776555 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140d09d00 msgr2=0x563140e14b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.232+0000 7fa1adf99700 1 --2- 172.21.15.46:0/988776555 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140d09d00 0x563140e14b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.232+0000 7fa1adf99700 1 -- 172.21.15.46:0/988776555 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563140e1b2e0 con 0x563140c8f4a0 2024-05-09T04:53:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.232+0000 7fa1adf99700 1 --2- 172.21.15.46:0/988776555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140c8f4a0 0x563140e01a20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40a9400 tx=0x7fa1a40a8cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6286bb136f00a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.232+0000 7fa1937fe700 1 -- 172.21.15.46:0/988776555 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40c2070 con 0x563140c8f4a0 2024-05-09T04:53:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.232+0000 7fa1937fe700 1 -- 172.21.15.46:0/988776555 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1a40ad820 con 0x563140c8f4a0 2024-05-09T04:53:09.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.232+0000 7fa1937fe700 1 -- 172.21.15.46:0/988776555 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40b0e00 con 0x563140c8f4a0 2024-05-09T04:53:09.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.233+0000 7fa1bec13b80 1 -- 172.21.15.46:0/988776555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140c8f4a0 msgr2=0x563140e01a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:09.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.233+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/988776555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140c8f4a0 0x563140e01a20 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fa1a40a9400 tx=0x7fa1a40a8cb0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.233+0000 7fa1bec13b80 1 -- 172.21.15.46:0/988776555 shutdown_connections 2024-05-09T04:53:09.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.233+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/988776555 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140e151e0 0x563140e1ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.233+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/988776555 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140c8f4a0 0x563140e01a20 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.233+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/988776555 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140d09d00 0x563140e14b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.233+0000 7fa1bec13b80 1 -- 172.21.15.46:0/988776555 >> 172.21.15.46:0/988776555 conn(0x563140c92f70 msgr2=0x563140e25540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.233+0000 7fa1bec13b80 1 -- 172.21.15.46:0/988776555 shutdown_connections 2024-05-09T04:53:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.234+0000 7fa1bec13b80 1 -- 172.21.15.46:0/988776555 wait complete. 2024-05-09T04:53:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.234+0000 7fa1bec13b80 1 -- start start 2024-05-09T04:53:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.234+0000 7fa1bec13b80 1 Processor -- start 2024-05-09T04:53:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.234+0000 7fa1bec13b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140c8f4a0 0x563140e1a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1bec13b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140d09d00 0x563140e1aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1bec13b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140e151e0 0x563140e8c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1bec13b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81ab400 con 0x563140d09d00 2024-05-09T04:53:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1bec13b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81ac870 con 0x563140e151e0 2024-05-09T04:53:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1bec13b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a81abad0 con 0x563140c8f4a0 2024-05-09T04:53:09.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1ad798700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140d09d00 0x563140e1aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1bec13b80 10 client.0 _ll_register_callbacks cb 0x563140ba06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:09.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1ad798700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140d09d00 0x563140e1aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47320/0 (socket says 172.21.15.46:47320) 2024-05-09T04:53:09.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1ad798700 1 -- 172.21.15.46:0/174827875 learned_addr learned my addr 172.21.15.46:0/174827875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:09.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1adf99700 1 --2- 172.21.15.46:0/174827875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140c8f4a0 0x563140e1a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1ae79a700 1 --2- 172.21.15.46:0/174827875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140e151e0 0x563140e8c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1ad798700 1 -- 172.21.15.46:0/174827875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140c8f4a0 msgr2=0x563140e1a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:09.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1ad798700 1 --2- 172.21.15.46:0/174827875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140c8f4a0 0x563140e1a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1ad798700 1 -- 172.21.15.46:0/174827875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140e151e0 msgr2=0x563140e8c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:09.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1ad798700 1 --2- 172.21.15.46:0/174827875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140e151e0 0x563140e8c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1ad798700 1 -- 172.21.15.46:0/174827875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563140e8ce70 con 0x563140d09d00 2024-05-09T04:53:09.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1ae79a700 1 --2- 172.21.15.46:0/174827875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140e151e0 0x563140e8c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:09.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1ad798700 1 --2- 172.21.15.46:0/174827875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140d09d00 0x563140e1aa90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa198009990 tx=0x7fa198009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b0c5803029c4d72 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:09.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.235+0000 7fa1917fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:09.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.236+0000 7fa1917fa700 1 -- 172.21.15.46:0/174827875 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198038070 con 0x563140d09d00 2024-05-09T04:53:09.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.236+0000 7fa1917fa700 1 -- 172.21.15.46:0/174827875 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa19800bd70 con 0x563140d09d00 2024-05-09T04:53:09.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.236+0000 7fa1bec13b80 1 -- 172.21.15.46:0/174827875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1a40ad820 con 0x563140d09d00 2024-05-09T04:53:09.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.236+0000 7fa1917fa700 1 -- 172.21.15.46:0/174827875 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa198044050 con 0x563140d09d00 2024-05-09T04:53:09.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.236+0000 7fa1bec13b80 10 client.15846 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:09.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.236+0000 7fa1bec13b80 1 -- 172.21.15.46:0/174827875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563140e94220 con 0x563140d09d00 2024-05-09T04:53:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.236+0000 7fa1917fa700 1 -- 172.21.15.46:0/174827875 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1a40ad820 con 0x563140d09d00 2024-05-09T04:53:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.236+0000 7fa1917fa700 1 -- 172.21.15.46:0/174827875 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa198022070 con 0x563140d09d00 2024-05-09T04:53:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.237+0000 7fa1bec13b80 10 client.15846 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.237+0000 7fa1bec13b80 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:09.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.237+0000 7fa1bec13b80 20 client.15846 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.237+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/174827875 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563140e948c0 0x563140e96ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.237+0000 7fa1bec13b80 4 client.15846 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:53:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.237+0000 7fa1bec13b80 1 -- 172.21.15.46:0/174827875 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563140d36b20 con 0x563140e948c0 2024-05-09T04:53:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.237+0000 7fa1adf99700 1 --2- 172.21.15.46:0/174827875 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563140e948c0 0x563140e96ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.237+0000 7fa1adf99700 1 --2- 172.21.15.46:0/174827875 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563140e948c0 0x563140e96ce0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e48fdd672fe5715b server_cookie=a172dc3ff1eadba6 in_seq=0 out_seq=0 2024-05-09T04:53:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.237+0000 7fa1917fa700 10 client.15846 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.237+0000 7fa1917fa700 1 -- 172.21.15.46:0/174827875 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563140d36b20 con 0x563140e948c0 2024-05-09T04:53:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.238+0000 7fa1917fa700 10 client.15846 handle_command_reply: tid=0 2024-05-09T04:53:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.238+0000 7fa1bec13b80 1 client.15846 shutdown 2024-05-09T04:53:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.238+0000 7fa1bec13b80 1 -- 172.21.15.46:0/174827875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140d09d00 msgr2=0x563140e1aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.238+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/174827875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140d09d00 0x563140e1aa90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fa198009990 tx=0x7fa198009cd0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.238+0000 7fa1bec13b80 1 -- 172.21.15.46:0/174827875 shutdown_connections 2024-05-09T04:53:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.238+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/174827875 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563140e948c0 0x563140e96ce0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.243 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.238+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/174827875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563140e151e0 0x563140e8c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.238+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/174827875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563140c8f4a0 0x563140e1a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.238+0000 7fa1bec13b80 1 --2- 172.21.15.46:0/174827875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563140d09d00 0x563140e1aa90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.238+0000 7fa1917fa700 0 client.15846 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:09.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.238+0000 7fa1bec13b80 1 -- 172.21.15.46:0/174827875 >> 172.21.15.46:0/174827875 conn(0x563140c92f70 msgr2=0x563140e01780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:09.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.238+0000 7fa1bec13b80 1 -- 172.21.15.46:0/174827875 shutdown_connections 2024-05-09T04:53:09.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.238+0000 7fa1bec13b80 1 -- 172.21.15.46:0/174827875 wait complete. 2024-05-09T04:53:09.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.238+0000 7fa1bec13b80 20 client.15846 trim_cache size 0 max 16384 2024-05-09T04:53:09.244 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:53:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.239+0000 7fa1b01fd700 1 -- 172.21.15.46:0/78408607 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa1940b29f0 msgr2=0x7fa1940b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.239+0000 7fa1b01fd700 1 --2- 172.21.15.46:0/78408607 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa1940b29f0 0x7fa1940b4f00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa198006010 tx=0x7fa198000bc0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.239+0000 7fa1b01fd700 1 -- 172.21.15.46:0/78408607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a81034e0 msgr2=0x7fa1a819ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.239+0000 7fa1b01fd700 1 --2- 172.21.15.46:0/78408607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a81034e0 0x7fa1a819ae10 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400c920 tx=0x7fa1a400cc30 comp rx=0 tx=0).stop 2024-05-09T04:53:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.239+0000 7fa1b01fd700 1 -- 172.21.15.46:0/78408607 shutdown_connections 2024-05-09T04:53:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.239+0000 7fa1b01fd700 1 --2- 172.21.15.46:0/78408607 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa1940b29f0 0x7fa1940b4f00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.239+0000 7fa1b01fd700 1 --2- 172.21.15.46:0/78408607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa1a8108760 0x7fa1a819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.239+0000 7fa1b01fd700 1 --2- 172.21.15.46:0/78408607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa1a81034e0 0x7fa1a819ae10 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.239+0000 7fa1b01fd700 1 --2- 172.21.15.46:0/78408607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa1a8107e10 0x7fa1a819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:09.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.239+0000 7fa1b01fd700 1 -- 172.21.15.46:0/78408607 >> 172.21.15.46:0/78408607 conn(0x7fa1a80fef30 msgr2=0x7fa1a806c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:09.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.239+0000 7fa1b01fd700 1 -- 172.21.15.46:0/78408607 shutdown_connections 2024-05-09T04:53:09.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:09.239+0000 7fa1b01fd700 1 -- 172.21.15.46:0/78408607 wait complete. 2024-05-09T04:53:09.640 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:09.640 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:09.640 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:09.640 INFO:tasks.workunit.client.0.smithi046.stdout:iozone test complete. 2024-05-09T04:53:09.641 INFO:tasks.workunit.client.0.smithi046.stderr:+ iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-05-09T04:53:09.725 INFO:tasks.workunit.client.0.smithi046.stdout: Iozone: Performance Test of File I/O 2024-05-09T04:53:09.725 INFO:tasks.workunit.client.0.smithi046.stdout: Version $Revision: 3.494 $ 2024-05-09T04:53:09.725 INFO:tasks.workunit.client.0.smithi046.stdout: Compiled for 64 bit mode. 2024-05-09T04:53:09.725 INFO:tasks.workunit.client.0.smithi046.stdout: Build: linux-AMD64 2024-05-09T04:53:09.726 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:09.726 INFO:tasks.workunit.client.0.smithi046.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-05-09T04:53:09.726 INFO:tasks.workunit.client.0.smithi046.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-05-09T04:53:09.726 INFO:tasks.workunit.client.0.smithi046.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-05-09T04:53:09.726 INFO:tasks.workunit.client.0.smithi046.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-05-09T04:53:09.726 INFO:tasks.workunit.client.0.smithi046.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-05-09T04:53:09.726 INFO:tasks.workunit.client.0.smithi046.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-05-09T04:53:09.726 INFO:tasks.workunit.client.0.smithi046.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-05-09T04:53:09.726 INFO:tasks.workunit.client.0.smithi046.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-05-09T04:53:09.726 INFO:tasks.workunit.client.0.smithi046.stdout: Alexey Skidanov, Sudhir Kumar. 2024-05-09T04:53:09.726 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:09.726 INFO:tasks.workunit.client.0.smithi046.stdout: Run began: Thu May 9 04:53:09 2024 2024-05-09T04:53:09.726 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:09.726 INFO:tasks.workunit.client.0.smithi046.stdout: Include close in write timing 2024-05-09T04:53:09.727 INFO:tasks.workunit.client.0.smithi046.stdout: Include fsync in write timing 2024-05-09T04:53:09.727 INFO:tasks.workunit.client.0.smithi046.stdout: File size set to 1048576 kB 2024-05-09T04:53:09.727 INFO:tasks.workunit.client.0.smithi046.stdout: Record Size 1024 kB 2024-05-09T04:53:09.727 INFO:tasks.workunit.client.0.smithi046.stdout: Command line used: iozone -c -e -s 1024M -r 1M -t 1 -F f2 -i 0 -i 1 2024-05-09T04:53:09.727 INFO:tasks.workunit.client.0.smithi046.stdout: Output is in kBytes/sec 2024-05-09T04:53:09.727 INFO:tasks.workunit.client.0.smithi046.stdout: Time Resolution = 0.000001 seconds. 2024-05-09T04:53:09.727 INFO:tasks.workunit.client.0.smithi046.stdout: Processor cache size set to 1024 kBytes. 2024-05-09T04:53:09.727 INFO:tasks.workunit.client.0.smithi046.stdout: Processor cache line size set to 32 bytes. 2024-05-09T04:53:09.727 INFO:tasks.workunit.client.0.smithi046.stdout: File stride size set to 17 * record size. 2024-05-09T04:53:09.727 INFO:tasks.workunit.client.0.smithi046.stdout: Throughput test with 1 process 2024-05-09T04:53:09.727 INFO:tasks.workunit.client.0.smithi046.stdout: Each process writes a 1048576 kByte file in 1024 kByte records 2024-05-09T04:53:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:10 smithi110 ceph-mon[34904]: pgmap v65: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 85 MiB/s wr, 44 op/s 2024-05-09T04:53:11.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:10 smithi046 ceph-mon[30136]: pgmap v65: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 85 MiB/s wr, 44 op/s 2024-05-09T04:53:11.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:10 smithi072 ceph-mon[34088]: pgmap v65: 97 pgs: 97 active+clean; 1.0 GiB data, 553 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 85 MiB/s wr, 44 op/s 2024-05-09T04:53:11.282 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:53:11.283 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:53:11.293 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:11.294 INFO:tasks.workunit.client.0.smithi046.stdout: Children see throughput for 1 initial writers = 684972.12 kB/sec 2024-05-09T04:53:11.294 INFO:tasks.workunit.client.0.smithi046.stdout: Parent sees throughput for 1 initial writers = 684887.15 kB/sec 2024-05-09T04:53:11.294 INFO:tasks.workunit.client.0.smithi046.stdout: Min throughput per process = 684972.12 kB/sec 2024-05-09T04:53:11.294 INFO:tasks.workunit.client.0.smithi046.stdout: Max throughput per process = 684972.12 kB/sec 2024-05-09T04:53:11.294 INFO:tasks.workunit.client.0.smithi046.stdout: Avg throughput per process = 684972.12 kB/sec 2024-05-09T04:53:11.294 INFO:tasks.workunit.client.0.smithi046.stdout: Min xfer = 1048576.00 kB 2024-05-09T04:53:11.541 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:13.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:12 smithi110 ceph-mon[34904]: pgmap v66: 97 pgs: 97 active+clean; 2.0 GiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 171 MiB/s wr, 92 op/s 2024-05-09T04:53:13.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:12 smithi046 ceph-mon[30136]: pgmap v66: 97 pgs: 97 active+clean; 2.0 GiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 171 MiB/s wr, 92 op/s 2024-05-09T04:53:13.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:12 smithi072 ceph-mon[34088]: pgmap v66: 97 pgs: 97 active+clean; 2.0 GiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 171 MiB/s wr, 92 op/s 2024-05-09T04:53:13.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.990+0000 7f65fe688700 1 -- 172.21.15.46:0/703003556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81029c0 msgr2=0x7f65f8102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:13.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.990+0000 7f65fe688700 1 --2- 172.21.15.46:0/703003556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81029c0 0x7f65f8102dd0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f65e0009990 tx=0x7f65e0009ca0 comp rx=0 tx=0).stop 2024-05-09T04:53:13.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.990+0000 7f65fe688700 1 -- 172.21.15.46:0/703003556 shutdown_connections 2024-05-09T04:53:13.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.990+0000 7f65fe688700 1 --2- 172.21.15.46:0/703003556 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65f8103310 0x7f65f810f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:13.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.990+0000 7f65fe688700 1 --2- 172.21.15.46:0/703003556 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65f81089f0 0x7f65f8108e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:13.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.990+0000 7f65fe688700 1 --2- 172.21.15.46:0/703003556 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81029c0 0x7f65f8102dd0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:13.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.990+0000 7f65fe688700 1 -- 172.21.15.46:0/703003556 >> 172.21.15.46:0/703003556 conn(0x7f65f80fe540 msgr2=0x7f65f8100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:13.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.990+0000 7f65fe688700 1 -- 172.21.15.46:0/703003556 shutdown_connections 2024-05-09T04:53:13.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.990+0000 7f65fe688700 1 -- 172.21.15.46:0/703003556 wait complete. 2024-05-09T04:53:13.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.991+0000 7f65fe688700 1 Processor -- start 2024-05-09T04:53:13.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.991+0000 7f65fe688700 1 -- start start 2024-05-09T04:53:13.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.991+0000 7f65fe688700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81029c0 0x7f65f819ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:13.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.991+0000 7f65fe688700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65f8103310 0x7f65f819b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:13.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.991+0000 7f65fe688700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65f81089f0 0x7f65f819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:13.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.991+0000 7f65fe688700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65f81122f0 con 0x7f65f81029c0 2024-05-09T04:53:13.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.991+0000 7f65fe688700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f65f8112170 con 0x7f65f81089f0 2024-05-09T04:53:13.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.991+0000 7f65fe688700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f65f8112470 con 0x7f65f8103310 2024-05-09T04:53:13.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.991+0000 7f65f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81029c0 0x7f65f819ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:13.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.991+0000 7f65f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81029c0 0x7f65f819ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45610/0 (socket says 172.21.15.46:45610) 2024-05-09T04:53:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.991+0000 7f65f7fff700 1 -- 172.21.15.46:0/763466367 learned_addr learned my addr 172.21.15.46:0/763466367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.991+0000 7f65effff700 1 --2- 172.21.15.46:0/763466367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65f8103310 0x7f65f819b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.991+0000 7f65fcc25700 1 --2- 172.21.15.46:0/763466367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65f81089f0 0x7f65f819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.991+0000 7f65f7fff700 1 -- 172.21.15.46:0/763466367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65f8103310 msgr2=0x7f65f819b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.991+0000 7f65f7fff700 1 --2- 172.21.15.46:0/763466367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65f8103310 0x7f65f819b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.992+0000 7f65f7fff700 1 -- 172.21.15.46:0/763466367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65f81089f0 msgr2=0x7f65f819f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.992+0000 7f65f7fff700 1 --2- 172.21.15.46:0/763466367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65f81089f0 0x7f65f819f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.992+0000 7f65f7fff700 1 -- 172.21.15.46:0/763466367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f819fd60 con 0x7f65f81029c0 2024-05-09T04:53:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.992+0000 7f65effff700 1 --2- 172.21.15.46:0/763466367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65f8103310 0x7f65f819b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.992+0000 7f65f7fff700 1 --2- 172.21.15.46:0/763466367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81029c0 0x7f65f819ae90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f65e800ddf0 tx=0x7f65e800a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.992+0000 7f65f5ffb700 1 -- 172.21.15.46:0/763466367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e801b070 con 0x7f65f81029c0 2024-05-09T04:53:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.992+0000 7f65f5ffb700 1 -- 172.21.15.46:0/763466367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e800e900 con 0x7f65f81029c0 2024-05-09T04:53:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.992+0000 7f65fe688700 1 -- 172.21.15.46:0/763466367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f81a0050 con 0x7f65f81029c0 2024-05-09T04:53:13.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.992+0000 7f65f5ffb700 1 -- 172.21.15.46:0/763466367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e80168e0 con 0x7f65f81029c0 2024-05-09T04:53:13.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.992+0000 7f65fe688700 1 -- 172.21.15.46:0/763466367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f81a7970 con 0x7f65f81029c0 2024-05-09T04:53:13.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.995+0000 7f65f5ffb700 1 -- 172.21.15.46:0/763466367 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f65e801e070 con 0x7f65f81029c0 2024-05-09T04:53:13.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.995+0000 7f65f5ffb700 1 --2- 172.21.15.46:0/763466367 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f65e40b29f0 0x7f65e40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:13.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.996+0000 7f65effff700 1 --2- 172.21.15.46:0/763466367 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f65e40b29f0 0x7f65e40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:13.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.996+0000 7f65f5ffb700 1 -- 172.21.15.46:0/763466367 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f65e80e2210 con 0x7f65f81029c0 2024-05-09T04:53:13.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.996+0000 7f660d09eb80 1 Processor -- start 2024-05-09T04:53:13.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.996+0000 7f660d09eb80 1 -- start start 2024-05-09T04:53:13.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.996+0000 7f65effff700 1 --2- 172.21.15.46:0/763466367 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f65e40b29f0 0x7f65e40b4f00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f65e0006010 tx=0x7f65e001a040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.996+0000 7f660d09eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524ce5d80 0x55b524dee2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.996+0000 7f660d09eb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524c6b520 0x55b524dee800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.996+0000 7f660d09eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524de8480 0x55b524de8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.996+0000 7f660d09eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b524d0b120 con 0x55b524ce5d80 2024-05-09T04:53:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.996+0000 7f660d09eb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55b524cb0a20 con 0x55b524c6b520 2024-05-09T04:53:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.996+0000 7f660d09eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b524d070e0 con 0x55b524de8480 2024-05-09T04:53:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.996+0000 7f65f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524ce5d80 0x55b524dee2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.996+0000 7f65f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524ce5d80 0x55b524dee2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45622/0 (socket says 172.21.15.46:45622) 2024-05-09T04:53:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.996+0000 7f65f7fff700 1 -- 172.21.15.46:0/1114052384 learned_addr learned my addr 172.21.15.46:0/1114052384 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.997+0000 7f65effff700 1 --2- 172.21.15.46:0/1114052384 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524c6b520 0x55b524dee800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.997+0000 7f65fcc25700 1 --2- 172.21.15.46:0/1114052384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524de8480 0x55b524de8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.997+0000 7f65f7fff700 1 -- 172.21.15.46:0/1114052384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524de8480 msgr2=0x55b524de8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.997+0000 7f65f7fff700 1 --2- 172.21.15.46:0/1114052384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524de8480 0x55b524de8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.997+0000 7f65f7fff700 1 -- 172.21.15.46:0/1114052384 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524c6b520 msgr2=0x55b524dee800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:13.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.997+0000 7f65f7fff700 1 --2- 172.21.15.46:0/1114052384 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524c6b520 0x55b524dee800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.997+0000 7f65f7fff700 1 -- 172.21.15.46:0/1114052384 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b524ca1580 con 0x55b524ce5d80 2024-05-09T04:53:14.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.997+0000 7f65f7fff700 1 --2- 172.21.15.46:0/1114052384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524ce5d80 0x55b524dee2c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f65e80a5660 tx=0x7f65e80a90f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482cdaffd1095cfc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:14.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.997+0000 7f65ee7fc700 1 -- 172.21.15.46:0/1114052384 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e80ad990 con 0x55b524ce5d80 2024-05-09T04:53:14.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.997+0000 7f65ee7fc700 1 -- 172.21.15.46:0/1114052384 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e80b2440 con 0x55b524ce5d80 2024-05-09T04:53:14.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.997+0000 7f65ee7fc700 1 -- 172.21.15.46:0/1114052384 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e80b1fb0 con 0x55b524ce5d80 2024-05-09T04:53:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.998+0000 7f660d09eb80 1 -- 172.21.15.46:0/1114052384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524ce5d80 msgr2=0x55b524dee2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.998+0000 7f660d09eb80 1 --2- 172.21.15.46:0/1114052384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524ce5d80 0x55b524dee2c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f65e80a5660 tx=0x7f65e80a90f0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.998+0000 7f660d09eb80 1 -- 172.21.15.46:0/1114052384 shutdown_connections 2024-05-09T04:53:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.998+0000 7f660d09eb80 1 --2- 172.21.15.46:0/1114052384 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524de8480 0x55b524de8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.998+0000 7f660d09eb80 1 --2- 172.21.15.46:0/1114052384 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524ce5d80 0x55b524dee2c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.998+0000 7f660d09eb80 1 --2- 172.21.15.46:0/1114052384 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524c6b520 0x55b524dee800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.998+0000 7f660d09eb80 1 -- 172.21.15.46:0/1114052384 >> 172.21.15.46:0/1114052384 conn(0x55b524c6eff0 msgr2=0x55b524de32a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.998+0000 7f660d09eb80 1 -- 172.21.15.46:0/1114052384 shutdown_connections 2024-05-09T04:53:14.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.998+0000 7f660d09eb80 1 -- 172.21.15.46:0/1114052384 wait complete. 2024-05-09T04:53:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.999+0000 7f660d09eb80 1 -- start start 2024-05-09T04:53:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.999+0000 7f660d09eb80 1 Processor -- start 2024-05-09T04:53:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.999+0000 7f660d09eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524df20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.999+0000 7f660d09eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524ce5d80 0x55b524df2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.999+0000 7f660d09eb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524de8480 0x55b524df2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.999+0000 7f660d09eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b524de91e0 con 0x55b524c6b520 2024-05-09T04:53:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.999+0000 7f660d09eb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55b524ca5ba0 con 0x55b524de8480 2024-05-09T04:53:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.999+0000 7f660d09eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b524df50a0 con 0x55b524ce5d80 2024-05-09T04:53:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.999+0000 7f660d09eb80 10 client.0 _ll_register_callbacks cb 0x55b524b7c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.999+0000 7f65f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524df20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.999+0000 7f65f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524df20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45626/0 (socket says 172.21.15.46:45626) 2024-05-09T04:53:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.999+0000 7f65f7fff700 1 -- 172.21.15.46:0/1619239149 learned_addr learned my addr 172.21.15.46:0/1619239149 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:13.999+0000 7f65effff700 1 --2- 172.21.15.46:0/1619239149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524ce5d80 0x55b524df2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f65fcc25700 1 --2- 172.21.15.46:0/1619239149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524de8480 0x55b524df2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:14.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f65f7fff700 1 -- 172.21.15.46:0/1619239149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524ce5d80 msgr2=0x55b524df2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f65f7fff700 1 --2- 172.21.15.46:0/1619239149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524ce5d80 0x55b524df2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f65f7fff700 1 -- 172.21.15.46:0/1619239149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524de8480 msgr2=0x55b524df2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f65f7fff700 1 --2- 172.21.15.46:0/1619239149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524de8480 0x55b524df2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f65f7fff700 1 -- 172.21.15.46:0/1619239149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b524b86510 con 0x55b524c6b520 2024-05-09T04:53:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f65fcc25700 1 --2- 172.21.15.46:0/1619239149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524de8480 0x55b524df2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f65f7fff700 1 --2- 172.21.15.46:0/1619239149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524df20e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f65e80dfb00 tx=0x7f65e8005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1610d846ffc7f88 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f65d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f65d77fe700 1 -- 172.21.15.46:0/1619239149 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e80c3060 con 0x55b524c6b520 2024-05-09T04:53:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f65d77fe700 1 -- 172.21.15.46:0/1619239149 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f65e80a9a50 con 0x55b524c6b520 2024-05-09T04:53:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f660d09eb80 1 -- 172.21.15.46:0/1619239149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e80b2440 con 0x55b524c6b520 2024-05-09T04:53:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f65d77fe700 1 -- 172.21.15.46:0/1619239149 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65e80b1fb0 con 0x55b524c6b520 2024-05-09T04:53:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f660d09eb80 10 client.15858 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f660d09eb80 1 -- 172.21.15.46:0/1619239149 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b524dffab0 con 0x55b524c6b520 2024-05-09T04:53:14.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.000+0000 7f65d77fe700 1 -- 172.21.15.46:0/1619239149 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e80b2440 con 0x55b524c6b520 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.001+0000 7f65d77fe700 1 -- 172.21.15.46:0/1619239149 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f65e800b040 con 0x55b524c6b520 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.001+0000 7f660d09eb80 10 client.15858 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.001+0000 7f660d09eb80 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.001+0000 7f660d09eb80 20 client.15858 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.001+0000 7f660d09eb80 1 --2- 172.21.15.46:0/1619239149 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b524e01f80 0x55b524e043a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.001+0000 7f660d09eb80 4 client.15858 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.001+0000 7f660d09eb80 1 -- 172.21.15.46:0/1619239149 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b524d18b00 con 0x55b524e01f80 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.001+0000 7f65effff700 1 --2- 172.21.15.46:0/1619239149 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b524e01f80 0x55b524e043a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.002+0000 7f65effff700 1 --2- 172.21.15.46:0/1619239149 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b524e01f80 0x55b524e043a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7960efef30ceae7b server_cookie=c498609fcd80e6c0 in_seq=0 out_seq=0 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.002+0000 7f65d77fe700 10 client.15858 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.002+0000 7f65d77fe700 1 -- 172.21.15.46:0/1619239149 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b524d18b00 con 0x55b524e01f80 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.002+0000 7f65d77fe700 10 client.15858 handle_command_reply: tid=0 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.002+0000 7f660d09eb80 1 client.15858 shutdown 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.003+0000 7f660d09eb80 1 -- 172.21.15.46:0/1619239149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 msgr2=0x55b524df20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.003+0000 7f660d09eb80 1 --2- 172.21.15.46:0/1619239149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524df20e0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f65e80dfb00 tx=0x7f65e8005ca0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.003+0000 7f660d09eb80 1 -- 172.21.15.46:0/1619239149 shutdown_connections 2024-05-09T04:53:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.003+0000 7f660d09eb80 1 --2- 172.21.15.46:0/1619239149 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b524e01f80 0x55b524e043a0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.003+0000 7f660d09eb80 1 --2- 172.21.15.46:0/1619239149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524de8480 0x55b524df2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.003+0000 7f660d09eb80 1 --2- 172.21.15.46:0/1619239149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524df20e0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.003+0000 7f660d09eb80 1 --2- 172.21.15.46:0/1619239149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524ce5d80 0x55b524df2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.003+0000 7f660d09eb80 1 -- 172.21.15.46:0/1619239149 >> 172.21.15.46:0/1619239149 conn(0x55b524c6eff0 msgr2=0x55b524ddd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.003+0000 7f65d77fe700 0 client.15858 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.003+0000 7f660d09eb80 1 -- 172.21.15.46:0/1619239149 shutdown_connections 2024-05-09T04:53:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.003+0000 7f660d09eb80 1 -- 172.21.15.46:0/1619239149 wait complete. 2024-05-09T04:53:14.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.003+0000 7f660d09eb80 20 client.15858 trim_cache size 0 max 16384 2024-05-09T04:53:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f660d09eb80 1 Processor -- start 2024-05-09T04:53:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f660d09eb80 1 -- start start 2024-05-09T04:53:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f660d09eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524de7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f660d09eb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524ce5d80 0x55b524df29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f660d09eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524de8480 0x55b524df08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f660d09eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65f81b19a0 con 0x55b524c6b520 2024-05-09T04:53:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f660d09eb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f65f81af230 con 0x55b524ce5d80 2024-05-09T04:53:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f660d09eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f65f81ad080 con 0x55b524de8480 2024-05-09T04:53:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f65f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524de7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f65f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524de7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45634/0 (socket says 172.21.15.46:45634) 2024-05-09T04:53:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f65f7fff700 1 -- 172.21.15.46:0/281005302 learned_addr learned my addr 172.21.15.46:0/281005302 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:14.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f65effff700 1 --2- 172.21.15.46:0/281005302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524ce5d80 0x55b524df29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f65f7fff700 1 -- 172.21.15.46:0/281005302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524de8480 msgr2=0x55b524df08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f65f7fff700 1 --2- 172.21.15.46:0/281005302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524de8480 0x55b524df08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f65f7fff700 1 -- 172.21.15.46:0/281005302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524ce5d80 msgr2=0x55b524df29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f65f7fff700 1 --2- 172.21.15.46:0/281005302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524ce5d80 0x55b524df29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f65f7fff700 1 -- 172.21.15.46:0/281005302 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b524ddfa50 con 0x55b524c6b520 2024-05-09T04:53:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.024+0000 7f65effff700 1 --2- 172.21.15.46:0/281005302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524ce5d80 0x55b524df29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:53:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.025+0000 7f65f7fff700 1 --2- 172.21.15.46:0/281005302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524de7ea0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f65e80a5660 tx=0x7f65e800b5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=414687f9f5fb931 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.025+0000 7f65edffb700 1 -- 172.21.15.46:0/281005302 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e80c3070 con 0x55b524c6b520 2024-05-09T04:53:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.025+0000 7f65edffb700 1 -- 172.21.15.46:0/281005302 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e80b2440 con 0x55b524c6b520 2024-05-09T04:53:14.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.025+0000 7f65edffb700 1 -- 172.21.15.46:0/281005302 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e80a8840 con 0x55b524c6b520 2024-05-09T04:53:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.026+0000 7f660d09eb80 1 -- 172.21.15.46:0/281005302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 msgr2=0x55b524de7ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:14.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.026+0000 7f660d09eb80 1 --2- 172.21.15.46:0/281005302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524de7ea0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f65e80a5660 tx=0x7f65e800b5f0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.026+0000 7f660d09eb80 1 -- 172.21.15.46:0/281005302 shutdown_connections 2024-05-09T04:53:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.026+0000 7f660d09eb80 1 --2- 172.21.15.46:0/281005302 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524de8480 0x55b524df08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.026+0000 7f660d09eb80 1 --2- 172.21.15.46:0/281005302 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524de7ea0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.026+0000 7f660d09eb80 1 --2- 172.21.15.46:0/281005302 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524ce5d80 0x55b524df29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.026+0000 7f660d09eb80 1 -- 172.21.15.46:0/281005302 >> 172.21.15.46:0/281005302 conn(0x55b524c6eff0 msgr2=0x55b524ddd6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.026+0000 7f660d09eb80 1 -- 172.21.15.46:0/281005302 shutdown_connections 2024-05-09T04:53:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.026+0000 7f660d09eb80 1 -- 172.21.15.46:0/281005302 wait complete. 2024-05-09T04:53:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.026+0000 7f660d09eb80 1 -- start start 2024-05-09T04:53:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.026+0000 7f660d09eb80 1 Processor -- start 2024-05-09T04:53:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f660d09eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524e073c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f660d09eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524ce5d80 0x55b524e047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f660d09eb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524de8480 0x55b524e04d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f660d09eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f65f81ad6f0 con 0x55b524c6b520 2024-05-09T04:53:14.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f660d09eb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f65f81aeb60 con 0x55b524de8480 2024-05-09T04:53:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f660d09eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f65f81addc0 con 0x55b524ce5d80 2024-05-09T04:53:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f660d09eb80 10 client.0 _ll_register_callbacks cb 0x55b524b7c740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f65f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524e073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f65f7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524e073c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45644/0 (socket says 172.21.15.46:45644) 2024-05-09T04:53:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f65f7fff700 1 -- 172.21.15.46:0/526159214 learned_addr learned my addr 172.21.15.46:0/526159214 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f65fcc25700 1 --2- 172.21.15.46:0/526159214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524de8480 0x55b524e04d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f65effff700 1 --2- 172.21.15.46:0/526159214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524ce5d80 0x55b524e047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f65f7fff700 1 -- 172.21.15.46:0/526159214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524ce5d80 msgr2=0x55b524e047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f65f7fff700 1 --2- 172.21.15.46:0/526159214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524ce5d80 0x55b524e047e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f65f7fff700 1 -- 172.21.15.46:0/526159214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524de8480 msgr2=0x55b524e04d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f65f7fff700 1 --2- 172.21.15.46:0/526159214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524de8480 0x55b524e04d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f65f7fff700 1 -- 172.21.15.46:0/526159214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b524e05360 con 0x55b524c6b520 2024-05-09T04:53:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f65effff700 1 --2- 172.21.15.46:0/526159214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524ce5d80 0x55b524e047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:53:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f65f7fff700 1 --2- 172.21.15.46:0/526159214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524e073c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f65e80dfec0 tx=0x7f65e8005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d352c388735165 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:14.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f65d77fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.027+0000 7f65d77fe700 1 -- 172.21.15.46:0/526159214 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e80c3050 con 0x55b524c6b520 2024-05-09T04:53:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.028+0000 7f65d77fe700 1 -- 172.21.15.46:0/526159214 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f65e80a5130 con 0x55b524c6b520 2024-05-09T04:53:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.028+0000 7f660d09eb80 1 -- 172.21.15.46:0/526159214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e80b2440 con 0x55b524c6b520 2024-05-09T04:53:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.028+0000 7f65d77fe700 1 -- 172.21.15.46:0/526159214 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65e800e450 con 0x55b524c6b520 2024-05-09T04:53:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.028+0000 7f660d09eb80 10 client.15864 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.028+0000 7f660d09eb80 1 -- 172.21.15.46:0/526159214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b524e05360 con 0x55b524c6b520 2024-05-09T04:53:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.028+0000 7f65d77fe700 1 -- 172.21.15.46:0/526159214 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e80b2440 con 0x55b524c6b520 2024-05-09T04:53:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.028+0000 7f65d77fe700 1 -- 172.21.15.46:0/526159214 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f65e80aeb20 con 0x55b524c6b520 2024-05-09T04:53:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.028+0000 7f660d09eb80 10 client.15864 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.028+0000 7f660d09eb80 10 client.15864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.028+0000 7f660d09eb80 20 client.15864 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.028+0000 7f660d09eb80 1 --2- 172.21.15.46:0/526159214 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b524e70900 0x55b524e72d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.028+0000 7f660d09eb80 4 client.15864 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:53:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.028+0000 7f660d09eb80 1 -- 172.21.15.46:0/526159214 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b524d12ba0 con 0x55b524e70900 2024-05-09T04:53:14.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.028+0000 7f65effff700 1 --2- 172.21.15.46:0/526159214 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b524e70900 0x55b524e72d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.029+0000 7f65d77fe700 10 client.15864 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:14.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.029+0000 7f65effff700 1 --2- 172.21.15.46:0/526159214 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b524e70900 0x55b524e72d20 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4bc7b7591ac8d3a server_cookie=203c5540311108dd in_seq=0 out_seq=0 2024-05-09T04:53:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.030+0000 7f65d77fe700 1 -- 172.21.15.46:0/526159214 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b524d12ba0 con 0x55b524e70900 2024-05-09T04:53:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.030+0000 7f65d77fe700 10 client.15864 handle_command_reply: tid=0 2024-05-09T04:53:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.030+0000 7f660d09eb80 1 client.15864 shutdown 2024-05-09T04:53:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.030+0000 7f660d09eb80 1 -- 172.21.15.46:0/526159214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 msgr2=0x55b524e073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.030+0000 7f660d09eb80 1 --2- 172.21.15.46:0/526159214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524e073c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f65e80dfec0 tx=0x7f65e8005c40 comp rx=0 tx=0).stop 2024-05-09T04:53:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.030+0000 7f660d09eb80 1 -- 172.21.15.46:0/526159214 shutdown_connections 2024-05-09T04:53:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.030+0000 7f660d09eb80 1 --2- 172.21.15.46:0/526159214 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b524e70900 0x55b524e72d20 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.030+0000 7f660d09eb80 1 --2- 172.21.15.46:0/526159214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b524de8480 0x55b524e04d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.030+0000 7f660d09eb80 1 --2- 172.21.15.46:0/526159214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b524c6b520 0x55b524e073c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.030+0000 7f65d77fe700 0 client.15864 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.030+0000 7f660d09eb80 1 --2- 172.21.15.46:0/526159214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b524ce5d80 0x55b524e047e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.030+0000 7f660d09eb80 1 -- 172.21.15.46:0/526159214 >> 172.21.15.46:0/526159214 conn(0x55b524c6eff0 msgr2=0x55b524df08c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:14.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.030+0000 7f660d09eb80 1 -- 172.21.15.46:0/526159214 shutdown_connections 2024-05-09T04:53:14.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.030+0000 7f660d09eb80 1 -- 172.21.15.46:0/526159214 wait complete. 2024-05-09T04:53:14.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.031+0000 7f660d09eb80 20 client.15864 trim_cache size 0 max 16384 2024-05-09T04:53:14.036 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:53:14.036 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:53:14.036 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "b671edcf-59ba-4a0d-8db8-3130d3f241f4", 2024-05-09T04:53:14.036 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:53:14.036 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:53:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.031+0000 7f65fe688700 1 -- 172.21.15.46:0/763466367 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f65e40b29f0 msgr2=0x7f65e40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.031+0000 7f65fe688700 1 --2- 172.21.15.46:0/763466367 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f65e40b29f0 0x7f65e40b4f00 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f65e0006010 tx=0x7f65e001a040 comp rx=0 tx=0).stop 2024-05-09T04:53:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.031+0000 7f65fe688700 1 -- 172.21.15.46:0/763466367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81029c0 msgr2=0x7f65f819ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.031+0000 7f65fe688700 1 --2- 172.21.15.46:0/763466367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81029c0 0x7f65f819ae90 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f65e800ddf0 tx=0x7f65e800a530 comp rx=0 tx=0).stop 2024-05-09T04:53:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.031+0000 7f65fe688700 1 -- 172.21.15.46:0/763466367 shutdown_connections 2024-05-09T04:53:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.031+0000 7f65fe688700 1 --2- 172.21.15.46:0/763466367 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f65e40b29f0 0x7f65e40b4f00 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.031+0000 7f65fe688700 1 --2- 172.21.15.46:0/763466367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f65f81089f0 0x7f65f819f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.031+0000 7f65fe688700 1 --2- 172.21.15.46:0/763466367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f65f81029c0 0x7f65f819ae90 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.031+0000 7f65fe688700 1 --2- 172.21.15.46:0/763466367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f65f8103310 0x7f65f819b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.031+0000 7f65fe688700 1 -- 172.21.15.46:0/763466367 >> 172.21.15.46:0/763466367 conn(0x7f65f80fe540 msgr2=0x7f65f8100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.031+0000 7f65fe688700 1 -- 172.21.15.46:0/763466367 shutdown_connections 2024-05-09T04:53:14.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:14.031+0000 7f65fe688700 1 -- 172.21.15.46:0/763466367 wait complete. 2024-05-09T04:53:14.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:14 smithi046 ceph-mon[30136]: pgmap v67: 97 pgs: 97 active+clean; 2.0 GiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-05-09T04:53:14.825 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:53:15.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:14 smithi110 ceph-mon[34904]: pgmap v67: 97 pgs: 97 active+clean; 2.0 GiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-05-09T04:53:15.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:14 smithi072 ceph-mon[34088]: pgmap v67: 97 pgs: 97 active+clean; 2.0 GiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-05-09T04:53:15.200 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:15.568 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:15.568 INFO:tasks.workunit.client.0.smithi046.stdout: Children see throughput for 1 rewriters = 758403.25 kB/sec 2024-05-09T04:53:15.569 INFO:tasks.workunit.client.0.smithi046.stdout: Parent sees throughput for 1 rewriters = 758274.89 kB/sec 2024-05-09T04:53:15.569 INFO:tasks.workunit.client.0.smithi046.stdout: Min throughput per process = 758403.25 kB/sec 2024-05-09T04:53:15.569 INFO:tasks.workunit.client.0.smithi046.stdout: Max throughput per process = 758403.25 kB/sec 2024-05-09T04:53:15.569 INFO:tasks.workunit.client.0.smithi046.stdout: Avg throughput per process = 758403.25 kB/sec 2024-05-09T04:53:15.569 INFO:tasks.workunit.client.0.smithi046.stdout: Min xfer = 1048576.00 kB 2024-05-09T04:53:16.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:15 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:53:16.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:15 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:53:16.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:15 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:53:16.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:15 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:53:16.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:15 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:53:16.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:15 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:53:16.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:15 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:53:16.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:15 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:53:16.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:15 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:53:16.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:15 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:53:16.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:15 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:53:16.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:15 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:53:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.759+0000 7fcff0784640 1 -- 172.21.15.46:0/3857646020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe8108ba0 msgr2=0x7fcfe8102740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.759+0000 7fcff0784640 1 --2- 172.21.15.46:0/3857646020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe8108ba0 0x7fcfe8102740 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009900 tx=0x7fcfe402a410 comp rx=0 tx=0).stop 2024-05-09T04:53:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.759+0000 7fcff0784640 1 -- 172.21.15.46:0/3857646020 shutdown_connections 2024-05-09T04:53:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.759+0000 7fcff0784640 1 --2- 172.21.15.46:0/3857646020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe8109040 0x7fcfe8102c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.759+0000 7fcff0784640 1 --2- 172.21.15.46:0/3857646020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfe81032f0 0x7fcfe810f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.761 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.759+0000 7fcff0784640 1 --2- 172.21.15.46:0/3857646020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe8108ba0 0x7fcfe8102740 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.759+0000 7fcff0784640 1 -- 172.21.15.46:0/3857646020 >> 172.21.15.46:0/3857646020 conn(0x7fcfe80782e0 msgr2=0x7fcfe8078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.759+0000 7fcff0784640 1 -- 172.21.15.46:0/3857646020 shutdown_connections 2024-05-09T04:53:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.759+0000 7fcff0784640 1 -- 172.21.15.46:0/3857646020 wait complete. 2024-05-09T04:53:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.760+0000 7fcff0784640 1 Processor -- start 2024-05-09T04:53:16.762 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.760+0000 7fcff0784640 1 -- start start 2024-05-09T04:53:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.760+0000 7fcff0784640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe81032f0 0x7fcfe819b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcff0784640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe8108ba0 0x7fcfe819bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcff0784640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfe8109040 0x7fcfe819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcff0784640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe8112d90 con 0x7fcfe81032f0 2024-05-09T04:53:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcff0784640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe8112c10 con 0x7fcfe8109040 2024-05-09T04:53:16.764 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcff0784640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe8112f10 con 0x7fcfe8108ba0 2024-05-09T04:53:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcfee520640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe81032f0 0x7fcfe819b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcfee520640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe81032f0 0x7fcfe819b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45668/0 (socket says 172.21.15.46:45668) 2024-05-09T04:53:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcfee520640 1 -- 172.21.15.46:0/3890478056 learned_addr learned my addr 172.21.15.46:0/3890478056 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcfeed21640 1 --2- 172.21.15.46:0/3890478056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfe8109040 0x7fcfe819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcfedd1f640 1 --2- 172.21.15.46:0/3890478056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe8108ba0 0x7fcfe819bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcfee520640 1 -- 172.21.15.46:0/3890478056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe8108ba0 msgr2=0x7fcfe819bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcfee520640 1 --2- 172.21.15.46:0/3890478056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe8108ba0 0x7fcfe819bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcfee520640 1 -- 172.21.15.46:0/3890478056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfe8109040 msgr2=0x7fcfe819c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcfee520640 1 --2- 172.21.15.46:0/3890478056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfe8109040 0x7fcfe819c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcfee520640 1 -- 172.21.15.46:0/3890478056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcfe81a6ab0 con 0x7fcfe81032f0 2024-05-09T04:53:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.761+0000 7fcfee520640 1 --2- 172.21.15.46:0/3890478056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe81032f0 0x7fcfe819b770 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4032fd0 tx=0x7fcfe4031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.762+0000 7fcfdf7fe640 1 -- 172.21.15.46:0/3890478056 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe403f030 con 0x7fcfe81032f0 2024-05-09T04:53:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.762+0000 7fcff0784640 1 -- 172.21.15.46:0/3890478056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcfe81a6d40 con 0x7fcfe81032f0 2024-05-09T04:53:16.765 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.762+0000 7fcfdf7fe640 1 -- 172.21.15.46:0/3890478056 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfe4004490 con 0x7fcfe81032f0 2024-05-09T04:53:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.762+0000 7fcfdf7fe640 1 -- 172.21.15.46:0/3890478056 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4035070 con 0x7fcfe81032f0 2024-05-09T04:53:16.766 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.762+0000 7fcff0784640 1 -- 172.21.15.46:0/3890478056 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcfe81a71d0 con 0x7fcfe81032f0 2024-05-09T04:53:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.764+0000 7fcff7637740 1 Processor -- start 2024-05-09T04:53:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.765+0000 7fcfdf7fe640 1 -- 172.21.15.46:0/3890478056 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcfe4042030 con 0x7fcfe81032f0 2024-05-09T04:53:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.765+0000 7fcff7637740 1 -- start start 2024-05-09T04:53:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.765+0000 7fcff7637740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8341b0 0x5573febd7680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.765+0000 7fcff7637740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 0x5573febd7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.765+0000 7fcff7637740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8134d0 0x5573febd8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.765+0000 7fcff7637740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5573feb3c7c0 con 0x5573fe8341b0 2024-05-09T04:53:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.765+0000 7fcff7637740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5573fead80e0 con 0x5573fe80b2f0 2024-05-09T04:53:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.765+0000 7fcff7637740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5573feb38780 con 0x5573fe8134d0 2024-05-09T04:53:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.765+0000 7fcfeed21640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8341b0 0x5573febd7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.765+0000 7fcfeed21640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8341b0 0x5573febd7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45682/0 (socket says 172.21.15.46:45682) 2024-05-09T04:53:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.765+0000 7fcfeed21640 1 -- 172.21.15.46:0/3817094523 learned_addr learned my addr 172.21.15.46:0/3817094523 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:16.772 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.765+0000 7fcfee520640 1 --2- 172.21.15.46:0/3817094523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 0x5573febd7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.765+0000 7fcfedd1f640 1 --2- 172.21.15.46:0/3817094523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8134d0 0x5573febd8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.766+0000 7fcfdf7fe640 1 --2- 172.21.15.46:0/3890478056 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fcfd40b2950 0x7fcfd40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.766+0000 7fcfeed21640 1 -- 172.21.15.46:0/3817094523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8134d0 msgr2=0x5573febd8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.766+0000 7fcfeed21640 1 --2- 172.21.15.46:0/3817094523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8134d0 0x5573febd8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.766+0000 7fcfeed21640 1 -- 172.21.15.46:0/3817094523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 msgr2=0x5573febd7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.766+0000 7fcfeed21640 1 --2- 172.21.15.46:0/3817094523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 0x5573febd7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.766+0000 7fcfeed21640 1 -- 172.21.15.46:0/3817094523 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573feb017c0 con 0x5573fe8341b0 2024-05-09T04:53:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.766+0000 7fcfdf7fe640 1 -- 172.21.15.46:0/3890478056 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fcfe4102910 con 0x7fcfe81032f0 2024-05-09T04:53:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.766+0000 7fcfeed21640 1 --2- 172.21.15.46:0/3890478056 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fcfd40b2950 0x7fcfd40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.766+0000 7fcfeed21640 1 --2- 172.21.15.46:0/3817094523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8341b0 0x5573febd7680 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0009f50 tx=0x7fcfe00118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=440934f960c6075c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.766+0000 7fcfdcff9640 1 -- 172.21.15.46:0/3817094523 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0004250 con 0x5573fe8341b0 2024-05-09T04:53:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.766+0000 7fcfdcff9640 1 -- 172.21.15.46:0/3817094523 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfe000fe00 con 0x5573fe8341b0 2024-05-09T04:53:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.766+0000 7fcfdcff9640 1 -- 172.21.15.46:0/3817094523 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe001baa0 con 0x5573fe8341b0 2024-05-09T04:53:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.767+0000 7fcff7637740 1 -- 172.21.15.46:0/3817094523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8341b0 msgr2=0x5573febd7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:16.773 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.767+0000 7fcfeed21640 1 --2- 172.21.15.46:0/3890478056 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fcfd40b2950 0x7fcfd40b4e60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0012fd0 tx=0x7fcfe0021000 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.767+0000 7fcff7637740 1 --2- 172.21.15.46:0/3817094523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8341b0 0x5573febd7680 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0009f50 tx=0x7fcfe00118d0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.767+0000 7fcff7637740 1 -- 172.21.15.46:0/3817094523 shutdown_connections 2024-05-09T04:53:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.767+0000 7fcff7637740 1 --2- 172.21.15.46:0/3817094523 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8134d0 0x5573febd8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.767+0000 7fcff7637740 1 --2- 172.21.15.46:0/3817094523 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 0x5573febd7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.767+0000 7fcff7637740 1 --2- 172.21.15.46:0/3817094523 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8341b0 0x5573febd7680 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.767+0000 7fcff7637740 1 -- 172.21.15.46:0/3817094523 >> 172.21.15.46:0/3817094523 conn(0x5573fe828300 msgr2=0x5573febd3e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.767+0000 7fcff7637740 1 -- 172.21.15.46:0/3817094523 shutdown_connections 2024-05-09T04:53:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.767+0000 7fcff7637740 1 -- 172.21.15.46:0/3817094523 wait complete. 2024-05-09T04:53:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.768+0000 7fcff7637740 1 -- start start 2024-05-09T04:53:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.768+0000 7fcff7637740 1 Processor -- start 2024-05-09T04:53:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.768+0000 7fcff7637740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 0x5573febe3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.768+0000 7fcff7637740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.768+0000 7fcff7637740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8341b0 0x5573febe1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.774 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.768+0000 7fcff7637740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5573febe47b0 con 0x5573fe8134d0 2024-05-09T04:53:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.768+0000 7fcff7637740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5573feac39c0 con 0x5573fe80b2f0 2024-05-09T04:53:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.768+0000 7fcff7637740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5573febe4930 con 0x5573fe8341b0 2024-05-09T04:53:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.768+0000 7fcff7637740 10 client.0 _ll_register_callbacks cb 0x5573feaf0020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.768+0000 7fcfedd1f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.768+0000 7fcfedd1f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45688/0 (socket says 172.21.15.46:45688) 2024-05-09T04:53:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.768+0000 7fcfedd1f640 1 -- 172.21.15.46:0/1264122974 learned_addr learned my addr 172.21.15.46:0/1264122974 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcfee520640 1 --2- 172.21.15.46:0/1264122974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 0x5573febe3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcfeed21640 1 --2- 172.21.15.46:0/1264122974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8341b0 0x5573febe1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcfedd1f640 1 -- 172.21.15.46:0/1264122974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8341b0 msgr2=0x5573febe1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcfedd1f640 1 --2- 172.21.15.46:0/1264122974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8341b0 0x5573febe1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcfedd1f640 1 -- 172.21.15.46:0/1264122974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 msgr2=0x5573febe3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcfedd1f640 1 --2- 172.21.15.46:0/1264122974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 0x5573febe3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.775 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcfedd1f640 1 -- 172.21.15.46:0/1264122974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573feaea1f0 con 0x5573fe8134d0 2024-05-09T04:53:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcfeed21640 1 --2- 172.21.15.46:0/1264122974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8341b0 0x5573febe1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:53:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcfedd1f640 1 --2- 172.21.15.46:0/1264122974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe3cb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8012700 tx=0x7fcfd8012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a8b8d7ef5c036c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcfd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcfd1ffb640 1 -- 172.21.15.46:0/1264122974 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8004050 con 0x5573fe8134d0 2024-05-09T04:53:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcfd1ffb640 1 -- 172.21.15.46:0/1264122974 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fcfd80041f0 con 0x5573fe8134d0 2024-05-09T04:53:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcff7637740 1 -- 172.21.15.46:0/1264122974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfe000fe00 con 0x5573fe8134d0 2024-05-09T04:53:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcfd1ffb640 1 -- 172.21.15.46:0/1264122974 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcfd80043f0 con 0x5573fe8134d0 2024-05-09T04:53:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcff7637740 10 client.15876 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.769+0000 7fcff7637740 1 -- 172.21.15.46:0/1264122974 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5573fe89f790 con 0x5573fe8134d0 2024-05-09T04:53:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.770+0000 7fcfd1ffb640 1 -- 172.21.15.46:0/1264122974 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfe000fe00 con 0x5573fe8134d0 2024-05-09T04:53:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.770+0000 7fcfd1ffb640 1 -- 172.21.15.46:0/1264122974 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fcfd8003c70 con 0x5573fe8134d0 2024-05-09T04:53:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.770+0000 7fcff7637740 10 client.15876 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.770+0000 7fcff7637740 10 client.15876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.770+0000 7fcff7637740 20 client.15876 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:16.776 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.770+0000 7fcff7637740 1 --2- 172.21.15.46:0/1264122974 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5573fe7f2d80 0x5573febf4010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.770+0000 7fcff7637740 4 client.15876 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.770+0000 7fcff7637740 1 -- 172.21.15.46:0/1264122974 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5573feb4a3d0 con 0x5573fe7f2d80 2024-05-09T04:53:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.770+0000 7fcfee520640 1 --2- 172.21.15.46:0/1264122974 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5573fe7f2d80 0x5573febf4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.771+0000 7fcfd1ffb640 10 client.15876 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:16.777 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.771+0000 7fcfee520640 1 --2- 172.21.15.46:0/1264122974 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5573fe7f2d80 0x5573febf4010 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=974917c7224e1b3f server_cookie=863e71cf424c2814 in_seq=0 out_seq=0 2024-05-09T04:53:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.771+0000 7fcfd1ffb640 1 -- 172.21.15.46:0/1264122974 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5573feb4a3d0 con 0x5573fe7f2d80 2024-05-09T04:53:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.771+0000 7fcfd1ffb640 10 client.15876 handle_command_reply: tid=0 2024-05-09T04:53:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.771+0000 7fcff7637740 1 client.15876 shutdown 2024-05-09T04:53:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.771+0000 7fcff7637740 1 -- 172.21.15.46:0/1264122974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 msgr2=0x5573febe3cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.771+0000 7fcff7637740 1 --2- 172.21.15.46:0/1264122974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe3cb0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8012700 tx=0x7fcfd8012bc0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.771+0000 7fcff7637740 1 -- 172.21.15.46:0/1264122974 shutdown_connections 2024-05-09T04:53:16.778 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.771+0000 7fcff7637740 1 --2- 172.21.15.46:0/1264122974 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5573fe7f2d80 0x5573febf4010 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.771+0000 7fcff7637740 1 --2- 172.21.15.46:0/1264122974 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8341b0 0x5573febe1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.771+0000 7fcff7637740 1 --2- 172.21.15.46:0/1264122974 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe3cb0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.772+0000 7fcff7637740 1 --2- 172.21.15.46:0/1264122974 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 0x5573febe3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.772+0000 7fcfd1ffb640 0 client.15876 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.772+0000 7fcff7637740 1 -- 172.21.15.46:0/1264122974 >> 172.21.15.46:0/1264122974 conn(0x5573fe828300 msgr2=0x5573febd2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.772+0000 7fcff7637740 1 -- 172.21.15.46:0/1264122974 shutdown_connections 2024-05-09T04:53:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.772+0000 7fcff7637740 1 -- 172.21.15.46:0/1264122974 wait complete. 2024-05-09T04:53:16.779 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.772+0000 7fcff7637740 20 client.15876 trim_cache size 0 max 16384 2024-05-09T04:53:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.787+0000 7fcff7637740 1 Processor -- start 2024-05-09T04:53:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.787+0000 7fcff7637740 1 -- start start 2024-05-09T04:53:16.789 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.787+0000 7fcff7637740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 0x5573febdba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.787+0000 7fcff7637740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.787+0000 7fcff7637740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8341b0 0x5573febea100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.787+0000 7fcff7637740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe81b2580 con 0x5573fe8134d0 2024-05-09T04:53:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.787+0000 7fcff7637740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe81afe50 con 0x5573fe80b2f0 2024-05-09T04:53:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.787+0000 7fcff7637740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe81adca0 con 0x5573fe8341b0 2024-05-09T04:53:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.787+0000 7fcfedd1f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.787+0000 7fcfedd1f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45690/0 (socket says 172.21.15.46:45690) 2024-05-09T04:53:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.787+0000 7fcfedd1f640 1 -- 172.21.15.46:0/1041398790 learned_addr learned my addr 172.21.15.46:0/1041398790 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.788+0000 7fcfeed21640 1 --2- 172.21.15.46:0/1041398790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8341b0 0x5573febea100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.788+0000 7fcfee520640 1 --2- 172.21.15.46:0/1041398790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 0x5573febdba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.788+0000 7fcfedd1f640 1 -- 172.21.15.46:0/1041398790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8341b0 msgr2=0x5573febea100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.788+0000 7fcfedd1f640 1 --2- 172.21.15.46:0/1041398790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8341b0 0x5573febea100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.788+0000 7fcfedd1f640 1 -- 172.21.15.46:0/1041398790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 msgr2=0x5573febdba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:16.790 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.788+0000 7fcfedd1f640 1 --2- 172.21.15.46:0/1041398790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 0x5573febdba50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.788+0000 7fcfedd1f640 1 -- 172.21.15.46:0/1041398790 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5573fe9165f0 con 0x5573fe8134d0 2024-05-09T04:53:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.788+0000 7fcfee520640 1 --2- 172.21.15.46:0/1041398790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 0x5573febdba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:53:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.788+0000 7fcfedd1f640 1 --2- 172.21.15.46:0/1041398790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe9bc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8000c00 tx=0x7fcfd8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90deebac730bc582 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.788+0000 7fcfd3fff640 1 -- 172.21.15.46:0/1041398790 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8029040 con 0x5573fe8134d0 2024-05-09T04:53:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.788+0000 7fcfd3fff640 1 -- 172.21.15.46:0/1041398790 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd800f920 con 0x5573fe8134d0 2024-05-09T04:53:16.791 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.788+0000 7fcfd3fff640 1 -- 172.21.15.46:0/1041398790 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8005460 con 0x5573fe8134d0 2024-05-09T04:53:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.789+0000 7fcff7637740 1 -- 172.21.15.46:0/1041398790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 msgr2=0x5573febe9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.789+0000 7fcff7637740 1 --2- 172.21.15.46:0/1041398790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe9bc0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8000c00 tx=0x7fcfd8010860 comp rx=0 tx=0).stop 2024-05-09T04:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.789+0000 7fcff7637740 1 -- 172.21.15.46:0/1041398790 shutdown_connections 2024-05-09T04:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.789+0000 7fcff7637740 1 --2- 172.21.15.46:0/1041398790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe8341b0 0x5573febea100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.789+0000 7fcff7637740 1 --2- 172.21.15.46:0/1041398790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe9bc0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.789+0000 7fcff7637740 1 --2- 172.21.15.46:0/1041398790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe80b2f0 0x5573febdba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.789+0000 7fcff7637740 1 -- 172.21.15.46:0/1041398790 >> 172.21.15.46:0/1041398790 conn(0x5573fe828300 msgr2=0x5573febdb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.789+0000 7fcff7637740 1 -- 172.21.15.46:0/1041398790 shutdown_connections 2024-05-09T04:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.789+0000 7fcff7637740 1 -- 172.21.15.46:0/1041398790 wait complete. 2024-05-09T04:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.789+0000 7fcff7637740 1 -- start start 2024-05-09T04:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcff7637740 1 Processor -- start 2024-05-09T04:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcff7637740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe80b2f0 0x5573febe7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcff7637740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcff7637740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe8341b0 0x5573fec54400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcff7637740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe81ae310 con 0x5573fe8134d0 2024-05-09T04:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcff7637740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe81af780 con 0x5573fe8341b0 2024-05-09T04:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcff7637740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcfe81ae9e0 con 0x5573fe80b2f0 2024-05-09T04:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcff7637740 10 client.0 _ll_register_callbacks cb 0x7fcfd8009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcfedd1f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcfedd1f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe75b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45694/0 (socket says 172.21.15.46:45694) 2024-05-09T04:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcfedd1f640 1 -- 172.21.15.46:0/3737173676 learned_addr learned my addr 172.21.15.46:0/3737173676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcfee520640 1 --2- 172.21.15.46:0/3737173676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe80b2f0 0x5573febe7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcfeed21640 1 --2- 172.21.15.46:0/3737173676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe8341b0 0x5573fec54400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcfedd1f640 1 -- 172.21.15.46:0/3737173676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe80b2f0 msgr2=0x5573febe7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcfedd1f640 1 --2- 172.21.15.46:0/3737173676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe80b2f0 0x5573febe7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcfedd1f640 1 -- 172.21.15.46:0/3737173676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe8341b0 msgr2=0x5573fec54400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcfedd1f640 1 --2- 172.21.15.46:0/3737173676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe8341b0 0x5573fec54400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.790+0000 7fcfedd1f640 1 -- 172.21.15.46:0/3737173676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5573feb00ed0 con 0x5573fe8134d0 2024-05-09T04:53:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcfedd1f640 1 --2- 172.21.15.46:0/3737173676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe75b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009eb0 tx=0x7fcfd8010620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e4858786649d5a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcfd1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcfd1ffb640 1 -- 172.21.15.46:0/3737173676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfd8033040 con 0x5573fe8134d0 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcfd1ffb640 1 -- 172.21.15.46:0/3737173676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fcfd800fc90 con 0x5573fe8134d0 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcff7637740 1 -- 172.21.15.46:0/3737173676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcfd800f920 con 0x5573fe8134d0 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcfd1ffb640 1 -- 172.21.15.46:0/3737173676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fcfd8044050 con 0x5573fe8134d0 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcff7637740 10 client.15882 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcff7637740 1 -- 172.21.15.46:0/3737173676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5573fec5ba20 con 0x5573fe8134d0 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcfd1ffb640 1 -- 172.21.15.46:0/3737173676 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcfd800f920 con 0x5573fe8134d0 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcfd1ffb640 1 -- 172.21.15.46:0/3737173676 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fcfd803f030 con 0x5573fe8134d0 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcff7637740 10 client.15882 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcff7637740 10 client.15882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcff7637740 20 client.15882 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcff7637740 1 --2- 172.21.15.46:0/3737173676 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5573fe7c81c0 0x5573fec5e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcff7637740 4 client.15882 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.791+0000 7fcff7637740 1 -- 172.21.15.46:0/3737173676 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5573feb44240 con 0x5573fe7c81c0 2024-05-09T04:53:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.792+0000 7fcfee520640 1 --2- 172.21.15.46:0/3737173676 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5573fe7c81c0 0x5573fec5e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.792+0000 7fcfee520640 1 --2- 172.21.15.46:0/3737173676 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5573fe7c81c0 0x5573fec5e170 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e23409f48426b857 server_cookie=b250e155eeb2e8f5 in_seq=0 out_seq=0 2024-05-09T04:53:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.792+0000 7fcfd1ffb640 10 client.15882 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.803+0000 7fcfd1ffb640 1 -- 172.21.15.46:0/3737173676 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x5573feb44240 con 0x5573fe7c81c0 2024-05-09T04:53:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.803+0000 7fcfd1ffb640 10 client.15882 handle_command_reply: tid=0 2024-05-09T04:53:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.803+0000 7fcff7637740 1 client.15882 shutdown 2024-05-09T04:53:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.803+0000 7fcff7637740 1 -- 172.21.15.46:0/3737173676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 msgr2=0x5573febe75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.803+0000 7fcff7637740 1 --2- 172.21.15.46:0/3737173676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe75b0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fcfd8009eb0 tx=0x7fcfd8010620 comp rx=0 tx=0).stop 2024-05-09T04:53:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.803+0000 7fcff7637740 1 -- 172.21.15.46:0/3737173676 shutdown_connections 2024-05-09T04:53:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.803+0000 7fcff7637740 1 --2- 172.21.15.46:0/3737173676 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5573fe7c81c0 0x5573fec5e170 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.803+0000 7fcff7637740 1 --2- 172.21.15.46:0/3737173676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5573fe8341b0 0x5573fec54400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.803+0000 7fcff7637740 1 --2- 172.21.15.46:0/3737173676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5573fe8134d0 0x5573febe75b0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.803+0000 7fcfd1ffb640 0 client.15882 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.803+0000 7fcff7637740 1 --2- 172.21.15.46:0/3737173676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5573fe80b2f0 0x5573febe7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.803+0000 7fcff7637740 1 -- 172.21.15.46:0/3737173676 >> 172.21.15.46:0/3737173676 conn(0x5573fe828300 msgr2=0x5573febd2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.803+0000 7fcff7637740 1 -- 172.21.15.46:0/3737173676 shutdown_connections 2024-05-09T04:53:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.803+0000 7fcff7637740 1 -- 172.21.15.46:0/3737173676 wait complete. 2024-05-09T04:53:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.803+0000 7fcff7637740 20 client.15882 trim_cache size 0 max 16384 2024-05-09T04:53:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-05-09T04:53:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-05-09T04:53:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-05-09T04:53:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.804+0000 7fcff0784640 1 -- 172.21.15.46:0/3890478056 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fcfd40b2950 msgr2=0x7fcfd40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.804+0000 7fcff0784640 1 --2- 172.21.15.46:0/3890478056 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fcfd40b2950 0x7fcfd40b4e60 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fcfe0012fd0 tx=0x7fcfe0021000 comp rx=0 tx=0).stop 2024-05-09T04:53:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.804+0000 7fcff0784640 1 -- 172.21.15.46:0/3890478056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe81032f0 msgr2=0x7fcfe819b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.804+0000 7fcff0784640 1 --2- 172.21.15.46:0/3890478056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe81032f0 0x7fcfe819b770 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4032fd0 tx=0x7fcfe4031a20 comp rx=0 tx=0).stop 2024-05-09T04:53:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.804+0000 7fcff0784640 1 -- 172.21.15.46:0/3890478056 shutdown_connections 2024-05-09T04:53:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.804+0000 7fcff0784640 1 --2- 172.21.15.46:0/3890478056 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fcfd40b2950 0x7fcfd40b4e60 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.804+0000 7fcff0784640 1 --2- 172.21.15.46:0/3890478056 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcfe8108ba0 0x7fcfe819bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.804+0000 7fcff0784640 1 --2- 172.21.15.46:0/3890478056 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcfe8109040 0x7fcfe819c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.804+0000 7fcff0784640 1 --2- 172.21.15.46:0/3890478056 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcfe81032f0 0x7fcfe819b770 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.804+0000 7fcff0784640 1 -- 172.21.15.46:0/3890478056 >> 172.21.15.46:0/3890478056 conn(0x7fcfe80782e0 msgr2=0x7fcfe80ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.804+0000 7fcff0784640 1 -- 172.21.15.46:0/3890478056 shutdown_connections 2024-05-09T04:53:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:16.804+0000 7fcff0784640 1 -- 172.21.15.46:0/3890478056 wait complete. 2024-05-09T04:53:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:16 smithi110 ceph-mon[34904]: pgmap v68: 97 pgs: 97 active+clean; 2.0 GiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-05-09T04:53:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:16 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:53:17.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:16 smithi046 ceph-mon[30136]: pgmap v68: 97 pgs: 97 active+clean; 2.0 GiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-05-09T04:53:17.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:16 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:53:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:16 smithi072 ceph-mon[34088]: pgmap v68: 97 pgs: 97 active+clean; 2.0 GiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 85 MiB/s wr, 53 op/s 2024-05-09T04:53:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:16 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:53:17.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.649+0000 7f6ffe8d8700 1 -- 172.21.15.46:0/2918575419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ff8109860 msgr2=0x7f6ff81111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:17.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.649+0000 7f6ffe8d8700 1 --2- 172.21.15.46:0/2918575419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ff8109860 0x7f6ff81111c0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6ff000b0d0 tx=0x7f6ff000b3e0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.649+0000 7f6ffe8d8700 1 -- 172.21.15.46:0/2918575419 shutdown_connections 2024-05-09T04:53:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.649+0000 7f6ffe8d8700 1 --2- 172.21.15.46:0/2918575419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ff8109860 0x7f6ff81111c0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.649+0000 7f6ffe8d8700 1 --2- 172.21.15.46:0/2918575419 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ff81045e0 0x7f6ff81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.649+0000 7f6ffe8d8700 1 --2- 172.21.15.46:0/2918575419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ff8108f10 0x7f6ff8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.649+0000 7f6ffe8d8700 1 -- 172.21.15.46:0/2918575419 >> 172.21.15.46:0/2918575419 conn(0x7f6ff80ffe70 msgr2=0x7f6ff81022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.649+0000 7f6ffe8d8700 1 -- 172.21.15.46:0/2918575419 shutdown_connections 2024-05-09T04:53:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.649+0000 7f6ffe8d8700 1 -- 172.21.15.46:0/2918575419 wait complete. 2024-05-09T04:53:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.650+0000 7f6ffe8d8700 1 Processor -- start 2024-05-09T04:53:17.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.650+0000 7f6ffe8d8700 1 -- start start 2024-05-09T04:53:17.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.650+0000 7f6ffe8d8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ff81045e0 0x7f6ff819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.650+0000 7f6ffe8d8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ff8108f10 0x7f6ff819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6ffe8d8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ff8109860 0x7f6ff819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6ffe8d8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8114490 con 0x7f6ff8109860 2024-05-09T04:53:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6ffe8d8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8114310 con 0x7f6ff8108f10 2024-05-09T04:53:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6ffe8d8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff8114610 con 0x7f6ff81045e0 2024-05-09T04:53:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6ffce75700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ff8109860 0x7f6ff819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6ffce75700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ff8109860 0x7f6ff819f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45720/0 (socket says 172.21.15.46:45720) 2024-05-09T04:53:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6ffce75700 1 -- 172.21.15.46:0/1576927192 learned_addr learned my addr 172.21.15.46:0/1576927192 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6ff7fff700 1 --2- 172.21.15.46:0/1576927192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ff81045e0 0x7f6ff819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6feffff700 1 --2- 172.21.15.46:0/1576927192 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ff8108f10 0x7f6ff819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6ffce75700 1 -- 172.21.15.46:0/1576927192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ff81045e0 msgr2=0x7f6ff819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:17.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6ffce75700 1 --2- 172.21.15.46:0/1576927192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ff81045e0 0x7f6ff819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6ffce75700 1 -- 172.21.15.46:0/1576927192 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ff8108f10 msgr2=0x7f6ff819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6ffce75700 1 --2- 172.21.15.46:0/1576927192 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ff8108f10 0x7f6ff819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6ffce75700 1 -- 172.21.15.46:0/1576927192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff819fbc0 con 0x7f6ff8109860 2024-05-09T04:53:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6ffce75700 1 --2- 172.21.15.46:0/1576927192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ff8109860 0x7f6ff819f5d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009850 tx=0x7f6ff0007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.651+0000 7f6ff5ffb700 1 -- 172.21.15.46:0/1576927192 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff001f070 con 0x7f6ff8109860 2024-05-09T04:53:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.652+0000 7f6ff5ffb700 1 -- 172.21.15.46:0/1576927192 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ff0004c50 con 0x7f6ff8109860 2024-05-09T04:53:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.652+0000 7f6ff5ffb700 1 -- 172.21.15.46:0/1576927192 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff001b690 con 0x7f6ff8109860 2024-05-09T04:53:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.652+0000 7f6ffe8d8700 1 -- 172.21.15.46:0/1576927192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff819fe50 con 0x7f6ff8109860 2024-05-09T04:53:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.652+0000 7f6ffe8d8700 1 -- 172.21.15.46:0/1576927192 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff81a02e0 con 0x7f6ff8109860 2024-05-09T04:53:17.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.655+0000 7f6ff5ffb700 1 -- 172.21.15.46:0/1576927192 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6ff0022070 con 0x7f6ff8109860 2024-05-09T04:53:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.655+0000 7f700d2eeb80 1 Processor -- start 2024-05-09T04:53:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f6ff5ffb700 1 --2- 172.21.15.46:0/1576927192 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f6fe40b2a70 0x7f6fe40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f700d2eeb80 1 -- start start 2024-05-09T04:53:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f6feffff700 1 --2- 172.21.15.46:0/1576927192 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f6fe40b2a70 0x7f6fe40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f6ff5ffb700 1 -- 172.21.15.46:0/1576927192 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f6ff000f020 con 0x7f6ff8109860 2024-05-09T04:53:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f700d2eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e8183d00 0x5562e828c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f700d2eeb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e81094a0 0x5562e828c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f700d2eeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8286400 0x5562e8286810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f700d2eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5562e81a90a0 con 0x5562e8183d00 2024-05-09T04:53:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f700d2eeb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5562e814e9a0 con 0x5562e81094a0 2024-05-09T04:53:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f700d2eeb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5562e81a5060 con 0x5562e8286400 2024-05-09T04:53:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f6ff7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e8183d00 0x5562e828c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f6ff7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e8183d00 0x5562e828c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45730/0 (socket says 172.21.15.46:45730) 2024-05-09T04:53:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f6ff7fff700 1 -- 172.21.15.46:0/1719855624 learned_addr learned my addr 172.21.15.46:0/1719855624 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f6ffce75700 1 --2- 172.21.15.46:0/1719855624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8286400 0x5562e8286810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f6feffff700 1 --2- 172.21.15.46:0/1719855624 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e81094a0 0x5562e828c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.656+0000 7f6feffff700 1 --2- 172.21.15.46:0/1576927192 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f6fe40b2a70 0x7f6fe40b4f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6ff81037e0 tx=0x7f6fe0011440 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.657+0000 7f6ff7fff700 1 -- 172.21.15.46:0/1719855624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8286400 msgr2=0x5562e8286810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.657+0000 7f6ff7fff700 1 --2- 172.21.15.46:0/1719855624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8286400 0x5562e8286810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.657+0000 7f6ff7fff700 1 -- 172.21.15.46:0/1719855624 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e81094a0 msgr2=0x5562e828c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.657+0000 7f6ff7fff700 1 --2- 172.21.15.46:0/1719855624 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e81094a0 0x5562e828c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.657+0000 7f6ff7fff700 1 -- 172.21.15.46:0/1719855624 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e813f500 con 0x5562e8183d00 2024-05-09T04:53:17.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.657+0000 7f6feffff700 1 --2- 172.21.15.46:0/1719855624 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e81094a0 0x5562e828c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:17.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.657+0000 7f6ff7fff700 1 --2- 172.21.15.46:0/1719855624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e8183d00 0x5562e828c240 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800ad70 tx=0x7f6fe800f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fd03d3321bc4b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:17.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.657+0000 7f6fee7fc700 1 -- 172.21.15.46:0/1719855624 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8004020 con 0x5562e8183d00 2024-05-09T04:53:17.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.657+0000 7f6fee7fc700 1 -- 172.21.15.46:0/1719855624 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe8003680 con 0x5562e8183d00 2024-05-09T04:53:17.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.657+0000 7f6fee7fc700 1 -- 172.21.15.46:0/1719855624 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe80186a0 con 0x5562e8183d00 2024-05-09T04:53:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.658+0000 7f700d2eeb80 1 -- 172.21.15.46:0/1719855624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e8183d00 msgr2=0x5562e828c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.658+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/1719855624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e8183d00 0x5562e828c240 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800ad70 tx=0x7f6fe800f6a0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.658+0000 7f700d2eeb80 1 -- 172.21.15.46:0/1719855624 shutdown_connections 2024-05-09T04:53:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.658+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/1719855624 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8286400 0x5562e8286810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.658+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/1719855624 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e8183d00 0x5562e828c240 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.658+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/1719855624 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e81094a0 0x5562e828c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.658+0000 7f700d2eeb80 1 -- 172.21.15.46:0/1719855624 >> 172.21.15.46:0/1719855624 conn(0x5562e810cf70 msgr2=0x5562e8281220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.658+0000 7f700d2eeb80 1 -- 172.21.15.46:0/1719855624 shutdown_connections 2024-05-09T04:53:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.658+0000 7f700d2eeb80 1 -- 172.21.15.46:0/1719855624 wait complete. 2024-05-09T04:53:17.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.658+0000 7f700d2eeb80 1 -- start start 2024-05-09T04:53:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.659+0000 7f700d2eeb80 1 Processor -- start 2024-05-09T04:53:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.659+0000 7f700d2eeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e81094a0 0x5562e828ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.659+0000 7f700d2eeb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e8183d00 0x5562e82904d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.659+0000 7f700d2eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e8286400 0x5562e8290a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.659+0000 7f700d2eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5562e8287160 con 0x5562e8286400 2024-05-09T04:53:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.659+0000 7f700d2eeb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5562e8143b20 con 0x5562e8183d00 2024-05-09T04:53:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.659+0000 7f700d2eeb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5562e8293020 con 0x5562e81094a0 2024-05-09T04:53:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.659+0000 7f700d2eeb80 10 client.0 _ll_register_callbacks cb 0x5562e801a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.659+0000 7f6ffce75700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e8286400 0x5562e8290a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.659+0000 7f6ffce75700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e8286400 0x5562e8290a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45740/0 (socket says 172.21.15.46:45740) 2024-05-09T04:53:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.659+0000 7f6ffce75700 1 -- 172.21.15.46:0/2841089540 learned_addr learned my addr 172.21.15.46:0/2841089540 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.659+0000 7f6feffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e8183d00 0x5562e82904d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.659+0000 7f6ff7fff700 1 --2- 172.21.15.46:0/2841089540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e81094a0 0x5562e828ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.659+0000 7f6ffce75700 1 -- 172.21.15.46:0/2841089540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e81094a0 msgr2=0x5562e828ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:17.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.660+0000 7f6ffce75700 1 --2- 172.21.15.46:0/2841089540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e81094a0 0x5562e828ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.660+0000 7f6ffce75700 1 -- 172.21.15.46:0/2841089540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e8183d00 msgr2=0x5562e82904d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.660+0000 7f6ffce75700 1 --2- 172.21.15.46:0/2841089540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e8183d00 0x5562e82904d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.660+0000 7f6ffce75700 1 -- 172.21.15.46:0/2841089540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e8024490 con 0x5562e8286400 2024-05-09T04:53:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.660+0000 7f6ff7fff700 1 --2- 172.21.15.46:0/2841089540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e81094a0 0x5562e828ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.660+0000 7f6feffff700 1 --2- 172.21.15.46:0/2841089540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e8183d00 0x5562e82904d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.660+0000 7f6ffce75700 1 --2- 172.21.15.46:0/2841089540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e8286400 0x5562e8290a10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00a63f0 tx=0x7f6ff00aa880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b41008f844a42b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.660+0000 7f6fd77fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.660+0000 7f6fd77fe700 1 -- 172.21.15.46:0/2841089540 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ff0004e00 con 0x5562e8286400 2024-05-09T04:53:17.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.660+0000 7f700d2eeb80 1 -- 172.21.15.46:0/2841089540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fe8003680 con 0x5562e8286400 2024-05-09T04:53:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.663+0000 7f6fd77fe700 1 -- 172.21.15.46:0/2841089540 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f6ff00b2ec0 con 0x5562e8286400 2024-05-09T04:53:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.663+0000 7f6fd77fe700 1 -- 172.21.15.46:0/2841089540 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6ff00a9e80 con 0x5562e8286400 2024-05-09T04:53:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.663+0000 7f700d2eeb80 10 client.15894 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.663+0000 7f700d2eeb80 1 -- 172.21.15.46:0/2841089540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5562e829d960 con 0x5562e8286400 2024-05-09T04:53:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.663+0000 7f6fd77fe700 1 -- 172.21.15.46:0/2841089540 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe8003680 con 0x5562e8286400 2024-05-09T04:53:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.664+0000 7f6fd77fe700 1 -- 172.21.15.46:0/2841089540 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f6ff0011070 con 0x5562e8286400 2024-05-09T04:53:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.664+0000 7f700d2eeb80 10 client.15894 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.664+0000 7f700d2eeb80 10 client.15894 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.664+0000 7f700d2eeb80 20 client.15894 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.664+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/2841089540 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5562e829ff20 0x5562e82a2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.664+0000 7f700d2eeb80 4 client.15894 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:17.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.664+0000 7f700d2eeb80 1 -- 172.21.15.46:0/2841089540 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562e81b6a80 con 0x5562e829ff20 2024-05-09T04:53:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.664+0000 7f6ff7fff700 1 --2- 172.21.15.46:0/2841089540 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5562e829ff20 0x5562e82a2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.664+0000 7f6ff7fff700 1 --2- 172.21.15.46:0/2841089540 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5562e829ff20 0x5562e82a2390 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8046e96a5df305c3 server_cookie=1c0a06babc580178 in_seq=0 out_seq=0 2024-05-09T04:53:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.664+0000 7f6fd77fe700 10 client.15894 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.665+0000 7f6fd77fe700 1 -- 172.21.15.46:0/2841089540 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5562e81b6a80 con 0x5562e829ff20 2024-05-09T04:53:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.665+0000 7f6fd77fe700 10 client.15894 handle_command_reply: tid=0 2024-05-09T04:53:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.665+0000 7f700d2eeb80 1 client.15894 shutdown 2024-05-09T04:53:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.665+0000 7f700d2eeb80 1 -- 172.21.15.46:0/2841089540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e8286400 msgr2=0x5562e8290a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.665+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/2841089540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e8286400 0x5562e8290a10 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f6ff00a63f0 tx=0x7f6ff00aa880 comp rx=0 tx=0).stop 2024-05-09T04:53:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.665+0000 7f700d2eeb80 1 -- 172.21.15.46:0/2841089540 shutdown_connections 2024-05-09T04:53:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.665+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/2841089540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e8286400 0x5562e8290a10 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.665+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/2841089540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e81094a0 0x5562e828ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.665+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/2841089540 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5562e829ff20 0x5562e82a2390 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.665+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/2841089540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e8183d00 0x5562e82904d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.665+0000 7f6fd77fe700 0 client.15894 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:17.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.665+0000 7f700d2eeb80 1 -- 172.21.15.46:0/2841089540 >> 172.21.15.46:0/2841089540 conn(0x5562e810cf70 msgr2=0x5562e827aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:17.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.666+0000 7f700d2eeb80 1 -- 172.21.15.46:0/2841089540 shutdown_connections 2024-05-09T04:53:17.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.666+0000 7f700d2eeb80 1 -- 172.21.15.46:0/2841089540 wait complete. 2024-05-09T04:53:17.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.666+0000 7f700d2eeb80 20 client.15894 trim_cache size 0 max 16384 2024-05-09T04:53:17.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.686+0000 7f700d2eeb80 1 Processor -- start 2024-05-09T04:53:17.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.686+0000 7f700d2eeb80 1 -- start start 2024-05-09T04:53:17.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.686+0000 7f700d2eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e810cf70 0x5562e828b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.686+0000 7f700d2eeb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e81094a0 0x5562e827ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.686+0000 7f700d2eeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8183d00 0x5562e8290850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.686+0000 7f700d2eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff81b16f0 con 0x5562e810cf70 2024-05-09T04:53:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.686+0000 7f700d2eeb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff81aef80 con 0x5562e81094a0 2024-05-09T04:53:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.686+0000 7f700d2eeb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff81acdd0 con 0x5562e8183d00 2024-05-09T04:53:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.686+0000 7f6ff7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e810cf70 0x5562e828b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.686+0000 7f6ff7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e810cf70 0x5562e828b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45742/0 (socket says 172.21.15.46:45742) 2024-05-09T04:53:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.686+0000 7f6ff7fff700 1 -- 172.21.15.46:0/4224962534 learned_addr learned my addr 172.21.15.46:0/4224962534 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.687+0000 7f6feffff700 1 --2- 172.21.15.46:0/4224962534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e81094a0 0x5562e827ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.687+0000 7f6ffce75700 1 --2- 172.21.15.46:0/4224962534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8183d00 0x5562e8290850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.687+0000 7f6ff7fff700 1 -- 172.21.15.46:0/4224962534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8183d00 msgr2=0x5562e8290850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.687+0000 7f6ff7fff700 1 --2- 172.21.15.46:0/4224962534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8183d00 0x5562e8290850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.687+0000 7f6ff7fff700 1 -- 172.21.15.46:0/4224962534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e81094a0 msgr2=0x5562e827ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.687+0000 7f6ff7fff700 1 --2- 172.21.15.46:0/4224962534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e81094a0 0x5562e827ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.687+0000 7f6ff7fff700 1 -- 172.21.15.46:0/4224962534 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562e828bc80 con 0x5562e810cf70 2024-05-09T04:53:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.687+0000 7f6ffce75700 1 --2- 172.21.15.46:0/4224962534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8183d00 0x5562e8290850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:53:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.687+0000 7f6ff7fff700 1 --2- 172.21.15.46:0/4224962534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e810cf70 0x5562e828b460 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800a3c0 tx=0x7f6fe8005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d4fe2b4ba433cc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.687+0000 7f6fedffb700 1 -- 172.21.15.46:0/4224962534 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8003680 con 0x5562e810cf70 2024-05-09T04:53:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.687+0000 7f6fedffb700 1 -- 172.21.15.46:0/4224962534 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe802f070 con 0x5562e810cf70 2024-05-09T04:53:17.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.687+0000 7f6fedffb700 1 -- 172.21.15.46:0/4224962534 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe80046d0 con 0x5562e810cf70 2024-05-09T04:53:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.688+0000 7f700d2eeb80 1 -- 172.21.15.46:0/4224962534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e810cf70 msgr2=0x5562e828b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.688+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/4224962534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e810cf70 0x5562e828b460 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800a3c0 tx=0x7f6fe8005ed0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.688+0000 7f700d2eeb80 1 -- 172.21.15.46:0/4224962534 shutdown_connections 2024-05-09T04:53:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.688+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/4224962534 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8183d00 0x5562e8290850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.688+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/4224962534 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e810cf70 0x5562e828b460 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.688+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/4224962534 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e81094a0 0x5562e827ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.688+0000 7f700d2eeb80 1 -- 172.21.15.46:0/4224962534 >> 172.21.15.46:0/4224962534 conn(0x5562e8281520 msgr2=0x5562e82853f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.688+0000 7f700d2eeb80 1 -- 172.21.15.46:0/4224962534 shutdown_connections 2024-05-09T04:53:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.688+0000 7f700d2eeb80 1 -- 172.21.15.46:0/4224962534 wait complete. 2024-05-09T04:53:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.688+0000 7f700d2eeb80 1 -- start start 2024-05-09T04:53:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.689+0000 7f700d2eeb80 1 Processor -- start 2024-05-09T04:53:17.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.689+0000 7f700d2eeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e81094a0 0x5562e82984e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.689+0000 7f700d2eeb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e810cf70 0x5562e82a7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.689+0000 7f700d2eeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8183d00 0x5562e82a7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.689+0000 7f700d2eeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff81ad440 con 0x5562e81094a0 2024-05-09T04:53:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.689+0000 7f700d2eeb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff81ae8b0 con 0x5562e810cf70 2024-05-09T04:53:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.689+0000 7f700d2eeb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff81adb10 con 0x5562e8183d00 2024-05-09T04:53:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.689+0000 7f700d2eeb80 10 client.0 _ll_register_callbacks cb 0x5562e801a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.689+0000 7f6ff7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e81094a0 0x5562e82984e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.689+0000 7f6ff7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e81094a0 0x5562e82984e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45752/0 (socket says 172.21.15.46:45752) 2024-05-09T04:53:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.689+0000 7f6ff7fff700 1 -- 172.21.15.46:0/896766661 learned_addr learned my addr 172.21.15.46:0/896766661 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.689+0000 7f6feffff700 1 --2- 172.21.15.46:0/896766661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e810cf70 0x5562e82a7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.689+0000 7f6ffce75700 1 --2- 172.21.15.46:0/896766661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8183d00 0x5562e82a7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.689+0000 7f6ff7fff700 1 -- 172.21.15.46:0/896766661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8183d00 msgr2=0x5562e82a7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.689+0000 7f6ff7fff700 1 --2- 172.21.15.46:0/896766661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8183d00 0x5562e82a7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.690+0000 7f6ff7fff700 1 -- 172.21.15.46:0/896766661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e810cf70 msgr2=0x5562e82a7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.690+0000 7f6ff7fff700 1 --2- 172.21.15.46:0/896766661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e810cf70 0x5562e82a7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.690+0000 7f6ff7fff700 1 -- 172.21.15.46:0/896766661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562e82a7f40 con 0x5562e81094a0 2024-05-09T04:53:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.690+0000 7f6ffce75700 1 --2- 172.21.15.46:0/896766661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8183d00 0x5562e82a7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:53:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.690+0000 7f6feffff700 1 --2- 172.21.15.46:0/896766661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e810cf70 0x5562e82a7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.690+0000 7f6ff7fff700 1 --2- 172.21.15.46:0/896766661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e81094a0 0x5562e82984e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800ffe0 tx=0x7f6fe8003930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcec0ea027bbceff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.690+0000 7f6fd77fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.690+0000 7f6fd77fe700 1 -- 172.21.15.46:0/896766661 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8003e60 con 0x5562e81094a0 2024-05-09T04:53:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.690+0000 7f6fd77fe700 1 -- 172.21.15.46:0/896766661 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f6fe8005480 con 0x5562e81094a0 2024-05-09T04:53:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.690+0000 7f700d2eeb80 1 -- 172.21.15.46:0/896766661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6fe802f070 con 0x5562e81094a0 2024-05-09T04:53:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.690+0000 7f6fd77fe700 1 -- 172.21.15.46:0/896766661 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f6fe8005680 con 0x5562e81094a0 2024-05-09T04:53:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.690+0000 7f700d2eeb80 10 client.15900 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.690+0000 7f700d2eeb80 1 -- 172.21.15.46:0/896766661 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5562e830a3b0 con 0x5562e81094a0 2024-05-09T04:53:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.690+0000 7f6fd77fe700 1 -- 172.21.15.46:0/896766661 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe802f070 con 0x5562e81094a0 2024-05-09T04:53:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.691+0000 7f6fd77fe700 1 -- 172.21.15.46:0/896766661 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f6fe802ace0 con 0x5562e81094a0 2024-05-09T04:53:17.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.691+0000 7f700d2eeb80 10 client.15900 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.691+0000 7f700d2eeb80 10 client.15900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.691+0000 7f700d2eeb80 20 client.15900 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.691+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/896766661 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5562e8286400 0x5562e830c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.691+0000 7f700d2eeb80 4 client.15900 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:53:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.691+0000 7f700d2eeb80 1 -- 172.21.15.46:0/896766661 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5562e81b0b20 con 0x5562e8286400 2024-05-09T04:53:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.691+0000 7f6feffff700 1 --2- 172.21.15.46:0/896766661 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5562e8286400 0x5562e830c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.691+0000 7f6feffff700 1 --2- 172.21.15.46:0/896766661 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5562e8286400 0x5562e830c8d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d14fe34b626abd84 server_cookie=363df02d5fcf7097 in_seq=0 out_seq=0 2024-05-09T04:53:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.691+0000 7f6fd77fe700 10 client.15900 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.691+0000 7f6fd77fe700 1 -- 172.21.15.46:0/896766661 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5562e81b0b20 con 0x5562e8286400 2024-05-09T04:53:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.691+0000 7f6fd77fe700 10 client.15900 handle_command_reply: tid=0 2024-05-09T04:53:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.691+0000 7f700d2eeb80 1 client.15900 shutdown 2024-05-09T04:53:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f700d2eeb80 1 -- 172.21.15.46:0/896766661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e81094a0 msgr2=0x5562e82984e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/896766661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e81094a0 0x5562e82984e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f6fe800ffe0 tx=0x7f6fe8003930 comp rx=0 tx=0).stop 2024-05-09T04:53:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f700d2eeb80 1 -- 172.21.15.46:0/896766661 shutdown_connections 2024-05-09T04:53:17.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/896766661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5562e8183d00 0x5562e82a7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/896766661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5562e81094a0 0x5562e82984e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/896766661 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5562e8286400 0x5562e830c8d0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f700d2eeb80 1 --2- 172.21.15.46:0/896766661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5562e810cf70 0x5562e82a7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f700d2eeb80 1 -- 172.21.15.46:0/896766661 >> 172.21.15.46:0/896766661 conn(0x5562e8281520 msgr2=0x5562e827b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f6fd77fe700 0 client.15900 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f700d2eeb80 1 -- 172.21.15.46:0/896766661 shutdown_connections 2024-05-09T04:53:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f700d2eeb80 1 -- 172.21.15.46:0/896766661 wait complete. 2024-05-09T04:53:17.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f700d2eeb80 20 client.15900 trim_cache size 0 max 16384 2024-05-09T04:53:17.698 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:53:17.698 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:53:17.698 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:53:17.699 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:53:17.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f6ffe8d8700 1 -- 172.21.15.46:0/1576927192 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f6fe40b2a70 msgr2=0x7f6fe40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:17.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f6ffe8d8700 1 --2- 172.21.15.46:0/1576927192 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f6fe40b2a70 0x7f6fe40b4f80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6ff81037e0 tx=0x7f6fe0011440 comp rx=0 tx=0).stop 2024-05-09T04:53:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f6ffe8d8700 1 -- 172.21.15.46:0/1576927192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ff8109860 msgr2=0x7f6ff819f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.692+0000 7f6ffe8d8700 1 --2- 172.21.15.46:0/1576927192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ff8109860 0x7f6ff819f5d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6ff0009850 tx=0x7f6ff0007a20 comp rx=0 tx=0).stop 2024-05-09T04:53:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.693+0000 7f6ffe8d8700 1 -- 172.21.15.46:0/1576927192 shutdown_connections 2024-05-09T04:53:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.693+0000 7f6ffe8d8700 1 --2- 172.21.15.46:0/1576927192 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f6fe40b2a70 0x7f6fe40b4f80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.693+0000 7f6ffe8d8700 1 --2- 172.21.15.46:0/1576927192 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6ff8109860 0x7f6ff819f5d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.693+0000 7f6ffe8d8700 1 --2- 172.21.15.46:0/1576927192 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6ff81045e0 0x7f6ff819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.693+0000 7f6ffe8d8700 1 --2- 172.21.15.46:0/1576927192 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6ff8108f10 0x7f6ff819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.693+0000 7f6ffe8d8700 1 -- 172.21.15.46:0/1576927192 >> 172.21.15.46:0/1576927192 conn(0x7f6ff80ffe70 msgr2=0x7f6ff8108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.693+0000 7f6ffe8d8700 1 -- 172.21.15.46:0/1576927192 shutdown_connections 2024-05-09T04:53:17.700 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:17.693+0000 7f6ffe8d8700 1 -- 172.21.15.46:0/1576927192 wait complete. 2024-05-09T04:53:18.333 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:18.333 INFO:tasks.workunit.client.0.smithi046.stdout: Children see throughput for 1 readers = 8961490.00 kB/sec 2024-05-09T04:53:18.333 INFO:tasks.workunit.client.0.smithi046.stdout: Parent sees throughput for 1 readers = 8945049.46 kB/sec 2024-05-09T04:53:18.334 INFO:tasks.workunit.client.0.smithi046.stdout: Min throughput per process = 8961490.00 kB/sec 2024-05-09T04:53:18.334 INFO:tasks.workunit.client.0.smithi046.stdout: Max throughput per process = 8961490.00 kB/sec 2024-05-09T04:53:18.334 INFO:tasks.workunit.client.0.smithi046.stdout: Avg throughput per process = 8961490.00 kB/sec 2024-05-09T04:53:18.334 INFO:tasks.workunit.client.0.smithi046.stdout: Min xfer = 1048576.00 kB 2024-05-09T04:53:18.758 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:53:18.758 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:53:18.931 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:18 smithi072 ceph-mon[34088]: pgmap v69: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 171 MiB/s wr, 78 op/s 2024-05-09T04:53:18.931 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:18 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:53:19.016 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:18 smithi110 ceph-mon[34904]: pgmap v69: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 171 MiB/s wr, 78 op/s 2024-05-09T04:53:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:18 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:53:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:18 smithi046 ceph-mon[30136]: pgmap v69: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 171 MiB/s wr, 78 op/s 2024-05-09T04:53:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:18 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:53:20.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:19 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:53:20.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:19 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:53:20.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:19 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:53:20.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:19 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:53:20.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:19 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:53:20.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:19 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:53:20.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:19 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:53:20.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:19 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:53:20.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:19 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:53:20.690 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:20.690 INFO:tasks.workunit.client.0.smithi046.stdout: Children see throughput for 1 re-readers = 8513796.00 kB/sec 2024-05-09T04:53:20.690 INFO:tasks.workunit.client.0.smithi046.stdout: Parent sees throughput for 1 re-readers = 8496689.96 kB/sec 2024-05-09T04:53:20.690 INFO:tasks.workunit.client.0.smithi046.stdout: Min throughput per process = 8513796.00 kB/sec 2024-05-09T04:53:20.690 INFO:tasks.workunit.client.0.smithi046.stdout: Max throughput per process = 8513796.00 kB/sec 2024-05-09T04:53:20.690 INFO:tasks.workunit.client.0.smithi046.stdout: Avg throughput per process = 8513796.00 kB/sec 2024-05-09T04:53:20.690 INFO:tasks.workunit.client.0.smithi046.stdout: Min xfer = 1048576.00 kB 2024-05-09T04:53:20.752 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:20 smithi046 ceph-mon[30136]: pgmap v70: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 171 MiB/s wr, 73 op/s 2024-05-09T04:53:20.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.751+0000 7f126dbee700 1 -- 172.21.15.46:0/2000154976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1268108f30 msgr2=0x7f1268109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:20.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.751+0000 7f126dbee700 1 --2- 172.21.15.46:0/2000154976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1268108f30 0x7f1268109340 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f12500099c0 tx=0x7f1250009cd0 comp rx=0 tx=0).stop 2024-05-09T04:53:20.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.751+0000 7f126dbee700 1 -- 172.21.15.46:0/2000154976 shutdown_connections 2024-05-09T04:53:20.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.751+0000 7f126dbee700 1 --2- 172.21.15.46:0/2000154976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268109880 0x7f12681111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:20.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.751+0000 7f126dbee700 1 --2- 172.21.15.46:0/2000154976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1268104650 0x7f1268104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:20.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.751+0000 7f126dbee700 1 --2- 172.21.15.46:0/2000154976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1268108f30 0x7f1268109340 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:20.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.751+0000 7f126dbee700 1 -- 172.21.15.46:0/2000154976 >> 172.21.15.46:0/2000154976 conn(0x7f1268079f00 msgr2=0x7f126807a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:20.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.751+0000 7f126dbee700 1 -- 172.21.15.46:0/2000154976 shutdown_connections 2024-05-09T04:53:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.751+0000 7f126dbee700 1 -- 172.21.15.46:0/2000154976 wait complete. 2024-05-09T04:53:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.752+0000 7f126dbee700 1 Processor -- start 2024-05-09T04:53:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.752+0000 7f126dbee700 1 -- start start 2024-05-09T04:53:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.752+0000 7f126dbee700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1268104650 0x7f1268075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.752+0000 7f126dbee700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268108f30 0x7f1268076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.752+0000 7f126dbee700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1268109880 0x7f1268076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.752+0000 7f126dbee700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1268114490 con 0x7f1268109880 2024-05-09T04:53:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.752+0000 7f126dbee700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1268114310 con 0x7f1268104650 2024-05-09T04:53:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.752+0000 7f126dbee700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1268114610 con 0x7f1268108f30 2024-05-09T04:53:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.752+0000 7f1267fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1268109880 0x7f1268076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.752+0000 7f1267fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1268109880 0x7f1268076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45778/0 (socket says 172.21.15.46:45778) 2024-05-09T04:53:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.752+0000 7f1267fff700 1 -- 172.21.15.46:0/3321828691 learned_addr learned my addr 172.21.15.46:0/3321828691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:20.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.752+0000 7f125effd700 1 --2- 172.21.15.46:0/3321828691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268108f30 0x7f1268076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.752+0000 7f12677fe700 1 --2- 172.21.15.46:0/3321828691 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1268104650 0x7f1268075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.753+0000 7f1267fff700 1 -- 172.21.15.46:0/3321828691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268108f30 msgr2=0x7f1268076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.753+0000 7f1267fff700 1 --2- 172.21.15.46:0/3321828691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268108f30 0x7f1268076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.753+0000 7f1267fff700 1 -- 172.21.15.46:0/3321828691 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1268104650 msgr2=0x7f1268075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.753+0000 7f1267fff700 1 --2- 172.21.15.46:0/3321828691 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1268104650 0x7f1268075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.753+0000 7f1267fff700 1 -- 172.21.15.46:0/3321828691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12681a91e0 con 0x7f1268109880 2024-05-09T04:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.753+0000 7f1267fff700 1 --2- 172.21.15.46:0/3321828691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1268109880 0x7f1268076540 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1260007ee0 tx=0x7f126000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.753+0000 7f12657fa700 1 -- 172.21.15.46:0/3321828691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f126000dd20 con 0x7f1268109880 2024-05-09T04:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.753+0000 7f126dbee700 1 -- 172.21.15.46:0/3321828691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12681a94d0 con 0x7f1268109880 2024-05-09T04:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.753+0000 7f12657fa700 1 -- 172.21.15.46:0/3321828691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1260004020 con 0x7f1268109880 2024-05-09T04:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.753+0000 7f12657fa700 1 -- 172.21.15.46:0/3321828691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1260015660 con 0x7f1268109880 2024-05-09T04:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.753+0000 7f126dbee700 1 -- 172.21.15.46:0/3321828691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12681a99e0 con 0x7f1268109880 2024-05-09T04:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.754+0000 7f126dbee700 1 -- 172.21.15.46:0/3321828691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12680512c0 con 0x7f1268109880 2024-05-09T04:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.756+0000 7f12657fa700 1 -- 172.21.15.46:0/3321828691 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f12600041d0 con 0x7f1268109880 2024-05-09T04:53:20.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.756+0000 7f12657fa700 1 --2- 172.21.15.46:0/3321828691 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f12540b29f0 0x7f12540b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.757+0000 7f12677fe700 1 --2- 172.21.15.46:0/3321828691 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f12540b29f0 0x7f12540b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.757+0000 7f12657fa700 1 -- 172.21.15.46:0/3321828691 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f12600e27c0 con 0x7f1268109880 2024-05-09T04:53:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.757+0000 7f12677fe700 1 --2- 172.21.15.46:0/3321828691 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f12540b29f0 0x7f12540b4f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1258005950 tx=0x7f125800b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:20.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:20.760+0000 7f12657fa700 1 -- 172.21.15.46:0/3321828691 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1260010070 con 0x7f1268109880 2024-05-09T04:53:21.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:21.026+0000 7f126dbee700 1 -- 172.21.15.46:0/3321828691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f126802e5f0 con 0x7f1268109880 2024-05-09T04:53:21.028 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:21.027+0000 7f12657fa700 1 -- 172.21.15.46:0/3321828691 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7f12600b2070 con 0x7f1268109880 2024-05-09T04:53:21.028 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:53:21.029 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:53:21.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:21.029+0000 7f126dbee700 1 -- 172.21.15.46:0/3321828691 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f12540b29f0 msgr2=0x7f12540b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:21.029+0000 7f126dbee700 1 --2- 172.21.15.46:0/3321828691 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f12540b29f0 0x7f12540b4f00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f1258005950 tx=0x7f125800b410 comp rx=0 tx=0).stop 2024-05-09T04:53:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:21.029+0000 7f126dbee700 1 -- 172.21.15.46:0/3321828691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1268109880 msgr2=0x7f1268076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:21.029+0000 7f126dbee700 1 --2- 172.21.15.46:0/3321828691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1268109880 0x7f1268076540 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1260007ee0 tx=0x7f126000d3b0 comp rx=0 tx=0).stop 2024-05-09T04:53:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:21.030+0000 7f126dbee700 1 -- 172.21.15.46:0/3321828691 shutdown_connections 2024-05-09T04:53:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:21.030+0000 7f126dbee700 1 --2- 172.21.15.46:0/3321828691 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f12540b29f0 0x7f12540b4f00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:21.030+0000 7f126dbee700 1 --2- 172.21.15.46:0/3321828691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1268109880 0x7f1268076540 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:21.030+0000 7f126dbee700 1 --2- 172.21.15.46:0/3321828691 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1268104650 0x7f1268075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:21.030+0000 7f126dbee700 1 --2- 172.21.15.46:0/3321828691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1268108f30 0x7f1268076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:21.030+0000 7f126dbee700 1 -- 172.21.15.46:0/3321828691 >> 172.21.15.46:0/3321828691 conn(0x7f1268079f00 msgr2=0x7f1268102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:21.030+0000 7f126dbee700 1 -- 172.21.15.46:0/3321828691 shutdown_connections 2024-05-09T04:53:21.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:21.030+0000 7f126dbee700 1 -- 172.21.15.46:0/3321828691 wait complete. 2024-05-09T04:53:21.032 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:53:21.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:20 smithi110 ceph-mon[34904]: pgmap v70: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 171 MiB/s wr, 73 op/s 2024-05-09T04:53:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:20 smithi072 ceph-mon[34088]: pgmap v70: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 171 MiB/s wr, 73 op/s 2024-05-09T04:53:21.860 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:53:22.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:21 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3321828691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:22.079 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:21 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3321828691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:22.118 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:22.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:21 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3321828691' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:22.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:22 smithi046 ceph-mon[30136]: pgmap v71: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 171 MiB/s wr, 79 op/s 2024-05-09T04:53:23.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:22 smithi110 ceph-mon[34904]: pgmap v71: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 171 MiB/s wr, 79 op/s 2024-05-09T04:53:23.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:22 smithi072 ceph-mon[34088]: pgmap v71: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 171 MiB/s wr, 79 op/s 2024-05-09T04:53:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.964+0000 7f0b2ca82700 1 -- 172.21.15.46:0/4069082348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b281034c0 msgr2=0x7f0b281038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.964+0000 7f0b2ca82700 1 --2- 172.21.15.46:0/4069082348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b281034c0 0x7f0b281038d0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800b0d0 tx=0x7f0b1800b3e0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.964+0000 7f0b2ca82700 1 -- 172.21.15.46:0/4069082348 shutdown_connections 2024-05-09T04:53:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.964+0000 7f0b2ca82700 1 --2- 172.21.15.46:0/4069082348 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b28108740 0x7f0b281100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.964+0000 7f0b2ca82700 1 --2- 172.21.15.46:0/4069082348 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b281034c0 0x7f0b281038d0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.964+0000 7f0b2ca82700 1 --2- 172.21.15.46:0/4069082348 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b28107df0 0x7f0b28108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.964+0000 7f0b2ca82700 1 -- 172.21.15.46:0/4069082348 >> 172.21.15.46:0/4069082348 conn(0x7f0b280fedc0 msgr2=0x7f0b281011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.964+0000 7f0b2ca82700 1 -- 172.21.15.46:0/4069082348 shutdown_connections 2024-05-09T04:53:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.964+0000 7f0b2ca82700 1 -- 172.21.15.46:0/4069082348 wait complete. 2024-05-09T04:53:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.965+0000 7f0b2ca82700 1 Processor -- start 2024-05-09T04:53:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.965+0000 7f0b2ca82700 1 -- start start 2024-05-09T04:53:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.965+0000 7f0b2ca82700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b281034c0 0x7f0b280ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.965+0000 7f0b2ca82700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b28107df0 0x7f0b280ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.965+0000 7f0b2ca82700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b28108740 0x7f0b280ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.965+0000 7f0b2ca82700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b28113370 con 0x7f0b281034c0 2024-05-09T04:53:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.965+0000 7f0b2ca82700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281131f0 con 0x7f0b28107df0 2024-05-09T04:53:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.965+0000 7f0b2ca82700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281134f0 con 0x7f0b28108740 2024-05-09T04:53:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.965+0000 7f0b2659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b281034c0 0x7f0b280ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.965+0000 7f0b2659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b281034c0 0x7f0b280ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35792/0 (socket says 172.21.15.46:35792) 2024-05-09T04:53:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.965+0000 7f0b2659c700 1 -- 172.21.15.46:0/3472937012 learned_addr learned my addr 172.21.15.46:0/3472937012 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.965+0000 7f0b25d9b700 1 --2- 172.21.15.46:0/3472937012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b28107df0 0x7f0b280ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.965+0000 7f0b26d9d700 1 --2- 172.21.15.46:0/3472937012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b28108740 0x7f0b280ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.966+0000 7f0b2659c700 1 -- 172.21.15.46:0/3472937012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b28108740 msgr2=0x7f0b280ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.966+0000 7f0b2659c700 1 --2- 172.21.15.46:0/3472937012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b28108740 0x7f0b280ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.966+0000 7f0b2659c700 1 -- 172.21.15.46:0/3472937012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b28107df0 msgr2=0x7f0b280ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.966+0000 7f0b2659c700 1 --2- 172.21.15.46:0/3472937012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b28107df0 0x7f0b280ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.966+0000 7f0b2659c700 1 -- 172.21.15.46:0/3472937012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b281a9150 con 0x7f0b281034c0 2024-05-09T04:53:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.966+0000 7f0b2659c700 1 --2- 172.21.15.46:0/3472937012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b281034c0 0x7f0b280ff2a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800d010 tx=0x7f0b18007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.966+0000 7f0b177fe700 1 -- 172.21.15.46:0/3472937012 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1801f070 con 0x7f0b281034c0 2024-05-09T04:53:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.966+0000 7f0b177fe700 1 -- 172.21.15.46:0/3472937012 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b18003c00 con 0x7f0b281034c0 2024-05-09T04:53:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.966+0000 7f0b2ca82700 1 -- 172.21.15.46:0/3472937012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b281a93e0 con 0x7f0b281034c0 2024-05-09T04:53:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.966+0000 7f0b177fe700 1 -- 172.21.15.46:0/3472937012 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1801b6a0 con 0x7f0b281034c0 2024-05-09T04:53:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.966+0000 7f0b2ca82700 1 -- 172.21.15.46:0/3472937012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b281a9830 con 0x7f0b281034c0 2024-05-09T04:53:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.969+0000 7f0b177fe700 1 -- 172.21.15.46:0/3472937012 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0b18022070 con 0x7f0b281034c0 2024-05-09T04:53:23.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.970+0000 7f0b177fe700 1 --2- 172.21.15.46:0/3472937012 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0b100b29f0 0x7f0b100b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.970+0000 7f0b25d9b700 1 --2- 172.21.15.46:0/3472937012 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0b100b29f0 0x7f0b100b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.970+0000 7f0b3b498b80 1 Processor -- start 2024-05-09T04:53:23.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.970+0000 7f0b177fe700 1 -- 172.21.15.46:0/3472937012 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f0b1800f020 con 0x7f0b281034c0 2024-05-09T04:53:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.970+0000 7f0b3b498b80 1 -- start start 2024-05-09T04:53:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b3b498b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c376c160 0x5590c38054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b3b498b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c36824a0 0x5590c3805a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b25d9b700 1 --2- 172.21.15.46:0/3472937012 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0b100b29f0 0x7f0b100b4f00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b28107cb0 tx=0x7f0b1c00a300 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b3b498b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c37ff6a0 0x5590c37ffad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b3b498b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5590c3721ff0 con 0x5590c36824a0 2024-05-09T04:53:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b3b498b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5590c36c07f0 con 0x5590c37ff6a0 2024-05-09T04:53:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b3b498b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5590c371dfb0 con 0x5590c376c160 2024-05-09T04:53:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b25d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c36824a0 0x5590c3805a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b25d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c36824a0 0x5590c3805a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35798/0 (socket says 172.21.15.46:35798) 2024-05-09T04:53:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b25d9b700 1 -- 172.21.15.46:0/2802286424 learned_addr learned my addr 172.21.15.46:0/2802286424 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b2659c700 1 --2- 172.21.15.46:0/2802286424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c376c160 0x5590c38054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b26d9d700 1 --2- 172.21.15.46:0/2802286424 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c37ff6a0 0x5590c37ffad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b25d9b700 1 -- 172.21.15.46:0/2802286424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c376c160 msgr2=0x5590c38054f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:23.973 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b25d9b700 1 --2- 172.21.15.46:0/2802286424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c376c160 0x5590c38054f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b25d9b700 1 -- 172.21.15.46:0/2802286424 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c37ff6a0 msgr2=0x5590c37ffad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:23.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b25d9b700 1 --2- 172.21.15.46:0/2802286424 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c37ff6a0 0x5590c37ffad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b25d9b700 1 -- 172.21.15.46:0/2802286424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590c36b8370 con 0x5590c36824a0 2024-05-09T04:53:23.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.971+0000 7f0b25d9b700 1 --2- 172.21.15.46:0/2802286424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c36824a0 0x5590c3805a30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00fd70 tx=0x7f0b1c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8960bd33b6ca63d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:23.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.972+0000 7f0b14ff9700 1 -- 172.21.15.46:0/2802286424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c016070 con 0x5590c36824a0 2024-05-09T04:53:23.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.972+0000 7f0b14ff9700 1 -- 172.21.15.46:0/2802286424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b1c010d70 con 0x5590c36824a0 2024-05-09T04:53:23.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.972+0000 7f0b14ff9700 1 -- 172.21.15.46:0/2802286424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c025da0 con 0x5590c36824a0 2024-05-09T04:53:23.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.972+0000 7f0b3b498b80 1 -- 172.21.15.46:0/2802286424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c36824a0 msgr2=0x5590c3805a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:23.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.972+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/2802286424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c36824a0 0x5590c3805a30 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00fd70 tx=0x7f0b1c012000 comp rx=0 tx=0).stop 2024-05-09T04:53:23.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.973+0000 7f0b3b498b80 1 -- 172.21.15.46:0/2802286424 shutdown_connections 2024-05-09T04:53:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.973+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/2802286424 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c37ff6a0 0x5590c37ffad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.973+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/2802286424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c376c160 0x5590c38054f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.973+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/2802286424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c36824a0 0x5590c3805a30 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.973+0000 7f0b3b498b80 1 -- 172.21.15.46:0/2802286424 >> 172.21.15.46:0/2802286424 conn(0x5590c36e26a0 msgr2=0x5590c37fa400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.973+0000 7f0b3b498b80 1 -- 172.21.15.46:0/2802286424 shutdown_connections 2024-05-09T04:53:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.973+0000 7f0b3b498b80 1 -- 172.21.15.46:0/2802286424 wait complete. 2024-05-09T04:53:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.973+0000 7f0b3b498b80 1 -- start start 2024-05-09T04:53:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.973+0000 7f0b3b498b80 1 Processor -- start 2024-05-09T04:53:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b3b498b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c36824a0 0x5590c3809280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b3b498b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c376c160 0x5590c38097c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b3b498b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c37ff6a0 0x5590c3809d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b3b498b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5590c359dba0 con 0x5590c376c160 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b3b498b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5590c36dc150 con 0x5590c36824a0 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b3b498b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5590c380c1d0 con 0x5590c37ff6a0 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b3b498b80 10 client.0 _ll_register_callbacks cb 0x5590c35936c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b25d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c376c160 0x5590c38097c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b25d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c376c160 0x5590c38097c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35806/0 (socket says 172.21.15.46:35806) 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b25d9b700 1 -- 172.21.15.46:0/2051292839 learned_addr learned my addr 172.21.15.46:0/2051292839 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b2659c700 1 --2- 172.21.15.46:0/2051292839 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c36824a0 0x5590c3809280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b26d9d700 1 --2- 172.21.15.46:0/2051292839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c37ff6a0 0x5590c3809d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b25d9b700 1 -- 172.21.15.46:0/2051292839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c37ff6a0 msgr2=0x5590c3809d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b25d9b700 1 --2- 172.21.15.46:0/2051292839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c37ff6a0 0x5590c3809d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b25d9b700 1 -- 172.21.15.46:0/2051292839 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c36824a0 msgr2=0x5590c3809280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b25d9b700 1 --2- 172.21.15.46:0/2051292839 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c36824a0 0x5590c3809280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b25d9b700 1 -- 172.21.15.46:0/2051292839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590c359d490 con 0x5590c376c160 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b26d9d700 1 --2- 172.21.15.46:0/2051292839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c37ff6a0 0x5590c3809d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b2659c700 1 --2- 172.21.15.46:0/2051292839 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c36824a0 0x5590c3809280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:23.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b25d9b700 1 --2- 172.21.15.46:0/2051292839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c376c160 0x5590c38097c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00a9d0 tx=0x7f0b1c0108c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6132db411ee99ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.974+0000 7f0b0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.975+0000 7f0b0dffb700 1 -- 172.21.15.46:0/2051292839 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b1c025ba0 con 0x5590c376c160 2024-05-09T04:53:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.975+0000 7f0b0dffb700 1 -- 172.21.15.46:0/2051292839 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f0b1c025d40 con 0x5590c376c160 2024-05-09T04:53:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.975+0000 7f0b3b498b80 1 -- 172.21.15.46:0/2051292839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b1c010d70 con 0x5590c376c160 2024-05-09T04:53:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.975+0000 7f0b0dffb700 1 -- 172.21.15.46:0/2051292839 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0b1c005030 con 0x5590c376c160 2024-05-09T04:53:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.975+0000 7f0b3b498b80 10 client.15918 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.975+0000 7f0b3b498b80 1 -- 172.21.15.46:0/2051292839 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590c3816c60 con 0x5590c376c160 2024-05-09T04:53:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.975+0000 7f0b0dffb700 1 -- 172.21.15.46:0/2051292839 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b1c010d70 con 0x5590c376c160 2024-05-09T04:53:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.975+0000 7f0b0dffb700 1 -- 172.21.15.46:0/2051292839 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f0b1c0314e0 con 0x5590c376c160 2024-05-09T04:53:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.975+0000 7f0b3b498b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.975+0000 7f0b3b498b80 10 client.15918 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:53:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.975+0000 7f0b3b498b80 20 client.15918 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:23.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.975+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/2051292839 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5590c3819130 0x5590c381b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.975+0000 7f0b3b498b80 4 client.15918 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.975+0000 7f0b3b498b80 1 -- 172.21.15.46:0/2051292839 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590c372f9d0 con 0x5590c3819130 2024-05-09T04:53:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.976+0000 7f0b2659c700 1 --2- 172.21.15.46:0/2051292839 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5590c3819130 0x5590c381b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.976+0000 7f0b2659c700 1 --2- 172.21.15.46:0/2051292839 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5590c3819130 0x5590c381b550 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b459555bd88386c7 server_cookie=ac1da22aea8f9f2b in_seq=0 out_seq=0 2024-05-09T04:53:23.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.976+0000 7f0b0dffb700 10 client.15918 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:53:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.977+0000 7f0b0dffb700 1 -- 172.21.15.46:0/2051292839 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5590c372f9d0 con 0x5590c3819130 2024-05-09T04:53:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.977+0000 7f0b0dffb700 10 client.15918 handle_command_reply: tid=0 2024-05-09T04:53:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.977+0000 7f0b3b498b80 1 client.15918 shutdown 2024-05-09T04:53:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.977+0000 7f0b3b498b80 1 -- 172.21.15.46:0/2051292839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c376c160 msgr2=0x5590c38097c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.977+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/2051292839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c376c160 0x5590c38097c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f0b1c00a9d0 tx=0x7f0b1c0108c0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.977+0000 7f0b3b498b80 1 -- 172.21.15.46:0/2051292839 shutdown_connections 2024-05-09T04:53:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.977+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/2051292839 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5590c3819130 0x5590c381b550 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.977+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/2051292839 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c37ff6a0 0x5590c3809d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.977+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/2051292839 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c36824a0 0x5590c3809280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.977+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/2051292839 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c376c160 0x5590c38097c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.977+0000 7f0b3b498b80 1 -- 172.21.15.46:0/2051292839 >> 172.21.15.46:0/2051292839 conn(0x5590c36e26a0 msgr2=0x5590c37fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.977+0000 7f0b0dffb700 0 client.15918 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:53:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.977+0000 7f0b3b498b80 1 -- 172.21.15.46:0/2051292839 shutdown_connections 2024-05-09T04:53:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.977+0000 7f0b3b498b80 1 -- 172.21.15.46:0/2051292839 wait complete. 2024-05-09T04:53:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.977+0000 7f0b3b498b80 20 client.15918 trim_cache size 0 max 16384 2024-05-09T04:53:23.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.992+0000 7f0b3b498b80 1 Processor -- start 2024-05-09T04:53:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b3b498b80 1 -- start start 2024-05-09T04:53:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b3b498b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c36824a0 0x5590c3808280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b3b498b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c376c160 0x5590c3820bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b3b498b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c37ff6a0 0x5590c38210f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b3b498b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281b3980 con 0x5590c37ff6a0 2024-05-09T04:53:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b3b498b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281b1210 con 0x5590c376c160 2024-05-09T04:53:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b3b498b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281af060 con 0x5590c36824a0 2024-05-09T04:53:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b26d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c37ff6a0 0x5590c38210f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b26d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c37ff6a0 0x5590c38210f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35814/0 (socket says 172.21.15.46:35814) 2024-05-09T04:53:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b26d9d700 1 -- 172.21.15.46:0/4236783809 learned_addr learned my addr 172.21.15.46:0/4236783809 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b2659c700 1 --2- 172.21.15.46:0/4236783809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c36824a0 0x5590c3808280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b25d9b700 1 --2- 172.21.15.46:0/4236783809 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c376c160 0x5590c3820bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b26d9d700 1 -- 172.21.15.46:0/4236783809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c36824a0 msgr2=0x5590c3808280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b26d9d700 1 --2- 172.21.15.46:0/4236783809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c36824a0 0x5590c3808280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b26d9d700 1 -- 172.21.15.46:0/4236783809 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c376c160 msgr2=0x5590c3820bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b26d9d700 1 --2- 172.21.15.46:0/4236783809 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c376c160 0x5590c3820bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.993+0000 7f0b26d9d700 1 -- 172.21.15.46:0/4236783809 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590c3809050 con 0x5590c37ff6a0 2024-05-09T04:53:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.994+0000 7f0b26d9d700 1 --2- 172.21.15.46:0/4236783809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c37ff6a0 0x5590c38210f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0b20007d50 tx=0x7f0b20011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c511158fb4fc049c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.994+0000 7f0b0ffff700 1 -- 172.21.15.46:0/4236783809 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b20011da0 con 0x5590c37ff6a0 2024-05-09T04:53:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.994+0000 7f0b0ffff700 1 -- 172.21.15.46:0/4236783809 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b200044d0 con 0x5590c37ff6a0 2024-05-09T04:53:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.994+0000 7f0b0ffff700 1 -- 172.21.15.46:0/4236783809 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2001b650 con 0x5590c37ff6a0 2024-05-09T04:53:24.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.995+0000 7f0b3b498b80 1 -- 172.21.15.46:0/4236783809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c37ff6a0 msgr2=0x5590c38210f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:24.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.995+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/4236783809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c37ff6a0 0x5590c38210f0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f0b20007d50 tx=0x7f0b20011720 comp rx=0 tx=0).stop 2024-05-09T04:53:24.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.995+0000 7f0b3b498b80 1 -- 172.21.15.46:0/4236783809 shutdown_connections 2024-05-09T04:53:24.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.995+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/4236783809 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c37ff6a0 0x5590c38210f0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:24.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.995+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/4236783809 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c36824a0 0x5590c3808280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:24.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.995+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/4236783809 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c376c160 0x5590c3820bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:24.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.995+0000 7f0b3b498b80 1 -- 172.21.15.46:0/4236783809 >> 172.21.15.46:0/4236783809 conn(0x5590c36e26a0 msgr2=0x5590c3807b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:24.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.995+0000 7f0b3b498b80 1 -- 172.21.15.46:0/4236783809 shutdown_connections 2024-05-09T04:53:24.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.995+0000 7f0b3b498b80 1 -- 172.21.15.46:0/4236783809 wait complete. 2024-05-09T04:53:24.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.995+0000 7f0b3b498b80 1 -- start start 2024-05-09T04:53:24.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.995+0000 7f0b3b498b80 1 Processor -- start 2024-05-09T04:53:24.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b3b498b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c36824a0 0x5590c3811090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:24.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b3b498b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c376c160 0x5590c38115d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:24.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b3b498b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c37ff6a0 0x5590c381fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:24.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b3b498b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281af6d0 con 0x5590c36824a0 2024-05-09T04:53:24.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b3b498b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281b0b40 con 0x5590c37ff6a0 2024-05-09T04:53:24.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b3b498b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0b281afda0 con 0x5590c376c160 2024-05-09T04:53:24.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b3b498b80 10 client.0 _ll_register_callbacks cb 0x5590c35936c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:24.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b2659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c36824a0 0x5590c3811090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:24.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b2659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c36824a0 0x5590c3811090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35818/0 (socket says 172.21.15.46:35818) 2024-05-09T04:53:24.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b2659c700 1 -- 172.21.15.46:0/1628837518 learned_addr learned my addr 172.21.15.46:0/1628837518 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:24.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b26d9d700 1 --2- 172.21.15.46:0/1628837518 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c37ff6a0 0x5590c381fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b25d9b700 1 --2- 172.21.15.46:0/1628837518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c376c160 0x5590c38115d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b2659c700 1 -- 172.21.15.46:0/1628837518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c376c160 msgr2=0x5590c38115d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b2659c700 1 --2- 172.21.15.46:0/1628837518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c376c160 0x5590c38115d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b2659c700 1 -- 172.21.15.46:0/1628837518 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c37ff6a0 msgr2=0x5590c381fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b2659c700 1 --2- 172.21.15.46:0/1628837518 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c37ff6a0 0x5590c381fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b2659c700 1 -- 172.21.15.46:0/1628837518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590c3811c50 con 0x5590c36824a0 2024-05-09T04:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b2659c700 1 --2- 172.21.15.46:0/1628837518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c36824a0 0x5590c3811090 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009fd0 tx=0x7f0b18092040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2988e3e50e31c3c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.996+0000 7f0b0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.997+0000 7f0b0dffb700 1 -- 172.21.15.46:0/1628837518 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b18005a90 con 0x5590c36824a0 2024-05-09T04:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.997+0000 7f0b0dffb700 1 -- 172.21.15.46:0/1628837518 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f0b180a7110 con 0x5590c36824a0 2024-05-09T04:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.997+0000 7f0b3b498b80 1 -- 172.21.15.46:0/1628837518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b200044d0 con 0x5590c36824a0 2024-05-09T04:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.997+0000 7f0b0dffb700 1 -- 172.21.15.46:0/1628837518 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0b180ab220 con 0x5590c36824a0 2024-05-09T04:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.997+0000 7f0b3b498b80 10 client.15924 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.997+0000 7f0b3b498b80 1 -- 172.21.15.46:0/1628837518 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5590c3882c60 con 0x5590c36824a0 2024-05-09T04:53:24.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.997+0000 7f0b0dffb700 1 -- 172.21.15.46:0/1628837518 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b200044d0 con 0x5590c36824a0 2024-05-09T04:53:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.997+0000 7f0b0dffb700 1 -- 172.21.15.46:0/1628837518 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f0b18011070 con 0x5590c36824a0 2024-05-09T04:53:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.997+0000 7f0b3b498b80 10 client.15924 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.997+0000 7f0b3b498b80 10 client.15924 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:53:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.997+0000 7f0b3b498b80 20 client.15924 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.997+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/1628837518 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5590c3883250 0x5590c3885670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.997+0000 7f0b3b498b80 4 client.15924 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:53:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.997+0000 7f0b3b498b80 1 -- 172.21.15.46:0/1628837518 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590c3729a70 con 0x5590c3883250 2024-05-09T04:53:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.997+0000 7f0b25d9b700 1 --2- 172.21.15.46:0/1628837518 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5590c3883250 0x5590c3885670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.998+0000 7f0b25d9b700 1 --2- 172.21.15.46:0/1628837518 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5590c3883250 0x5590c3885670 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=69959de4150d0474 server_cookie=5fbc4241825455e1 in_seq=0 out_seq=0 2024-05-09T04:53:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.998+0000 7f0b0dffb700 10 client.15924 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:53:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.998+0000 7f0b0dffb700 1 -- 172.21.15.46:0/1628837518 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590c3729a70 con 0x5590c3883250 2024-05-09T04:53:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.998+0000 7f0b0dffb700 10 client.15924 handle_command_reply: tid=0 2024-05-09T04:53:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.998+0000 7f0b3b498b80 1 client.15924 shutdown 2024-05-09T04:53:24.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.998+0000 7f0b3b498b80 1 -- 172.21.15.46:0/1628837518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c36824a0 msgr2=0x5590c3811090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:24.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.998+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/1628837518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c36824a0 0x5590c3811090 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f0b18009fd0 tx=0x7f0b18092040 comp rx=0 tx=0).stop 2024-05-09T04:53:24.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.998+0000 7f0b3b498b80 1 -- 172.21.15.46:0/1628837518 shutdown_connections 2024-05-09T04:53:24.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.998+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/1628837518 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5590c37ff6a0 0x5590c381fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:24.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.998+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/1628837518 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5590c36824a0 0x5590c3811090 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:24.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.998+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/1628837518 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5590c3883250 0x5590c3885670 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:24.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.998+0000 7f0b3b498b80 1 --2- 172.21.15.46:0/1628837518 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5590c376c160 0x5590c38115d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:24.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.998+0000 7f0b3b498b80 1 -- 172.21.15.46:0/1628837518 >> 172.21.15.46:0/1628837518 conn(0x5590c36e26a0 msgr2=0x5590c3802e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:24.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.998+0000 7f0b0dffb700 0 client.15924 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:53:24.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.998+0000 7f0b3b498b80 1 -- 172.21.15.46:0/1628837518 shutdown_connections 2024-05-09T04:53:24.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.999+0000 7f0b3b498b80 1 -- 172.21.15.46:0/1628837518 wait complete. 2024-05-09T04:53:24.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.999+0000 7f0b3b498b80 20 client.15924 trim_cache size 0 max 16384 2024-05-09T04:53:24.005 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:53:24.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.999+0000 7f0b2ca82700 1 -- 172.21.15.46:0/3472937012 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0b100b29f0 msgr2=0x7f0b100b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:24.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.999+0000 7f0b2ca82700 1 --2- 172.21.15.46:0/3472937012 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0b100b29f0 0x7f0b100b4f00 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0b28107cb0 tx=0x7f0b1c00a300 comp rx=0 tx=0).stop 2024-05-09T04:53:24.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.999+0000 7f0b2ca82700 1 -- 172.21.15.46:0/3472937012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b281034c0 msgr2=0x7f0b280ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:24.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.999+0000 7f0b2ca82700 1 --2- 172.21.15.46:0/3472937012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b281034c0 0x7f0b280ff2a0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800d010 tx=0x7f0b18007a20 comp rx=0 tx=0).stop 2024-05-09T04:53:24.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.999+0000 7f0b2ca82700 1 -- 172.21.15.46:0/3472937012 shutdown_connections 2024-05-09T04:53:24.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.999+0000 7f0b2ca82700 1 --2- 172.21.15.46:0/3472937012 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0b100b29f0 0x7f0b100b4f00 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:24.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.999+0000 7f0b2ca82700 1 --2- 172.21.15.46:0/3472937012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0b28108740 0x7f0b280ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:24.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.999+0000 7f0b2ca82700 1 --2- 172.21.15.46:0/3472937012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0b281034c0 0x7f0b280ff2a0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:24.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.999+0000 7f0b2ca82700 1 --2- 172.21.15.46:0/3472937012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0b28107df0 0x7f0b280ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:24.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.999+0000 7f0b2ca82700 1 -- 172.21.15.46:0/3472937012 >> 172.21.15.46:0/3472937012 conn(0x7f0b280fedc0 msgr2=0x7f0b2806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:24.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.999+0000 7f0b2ca82700 1 -- 172.21.15.46:0/3472937012 shutdown_connections 2024-05-09T04:53:24.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:23.999+0000 7f0b2ca82700 1 -- 172.21.15.46:0/3472937012 wait complete. 2024-05-09T04:53:24.840 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:53:25.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:24 smithi110 ceph-mon[34904]: pgmap v72: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 85 MiB/s wr, 31 op/s 2024-05-09T04:53:25.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:24 smithi046 ceph-mon[30136]: pgmap v72: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 85 MiB/s wr, 31 op/s 2024-05-09T04:53:25.098 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:25.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:24 smithi072 ceph-mon[34088]: pgmap v72: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 85 MiB/s wr, 31 op/s 2024-05-09T04:53:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.882+0000 7faf94f97700 1 -- 172.21.15.46:0/2103560382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf901045d0 msgr2=0x7faf901049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:26.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.882+0000 7faf94f97700 1 --2- 172.21.15.46:0/2103560382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf901045d0 0x7faf901049e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7faf8000b0d0 tx=0x7faf8000b3e0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.884 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:26 smithi046 ceph-mon[30136]: pgmap v73: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 85 MiB/s wr, 31 op/s 2024-05-09T04:53:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.882+0000 7faf94f97700 1 -- 172.21.15.46:0/2103560382 shutdown_connections 2024-05-09T04:53:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.882+0000 7faf94f97700 1 --2- 172.21.15.46:0/2103560382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf90109850 0x7faf901111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.882+0000 7faf94f97700 1 --2- 172.21.15.46:0/2103560382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf901045d0 0x7faf901049e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.882+0000 7faf94f97700 1 --2- 172.21.15.46:0/2103560382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf90108f00 0x7faf90109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.882+0000 7faf94f97700 1 -- 172.21.15.46:0/2103560382 >> 172.21.15.46:0/2103560382 conn(0x7faf900ffea0 msgr2=0x7faf901022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.883+0000 7faf94f97700 1 -- 172.21.15.46:0/2103560382 shutdown_connections 2024-05-09T04:53:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.883+0000 7faf94f97700 1 -- 172.21.15.46:0/2103560382 wait complete. 2024-05-09T04:53:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.884+0000 7faf94f97700 1 Processor -- start 2024-05-09T04:53:26.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.884+0000 7faf94f97700 1 -- start start 2024-05-09T04:53:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.884+0000 7faf94f97700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf901045d0 0x7faf9019adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf94f97700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf90108f00 0x7faf9019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf94f97700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf90109850 0x7faf9019f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf94f97700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf90114480 con 0x7faf901045d0 2024-05-09T04:53:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf94f97700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7faf90114300 con 0x7faf90108f00 2024-05-09T04:53:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf94f97700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faf90114600 con 0x7faf90109850 2024-05-09T04:53:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf8e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf901045d0 0x7faf9019adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf8e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf901045d0 0x7faf9019adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35838/0 (socket says 172.21.15.46:35838) 2024-05-09T04:53:26.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf8e59c700 1 -- 172.21.15.46:0/3621959947 learned_addr learned my addr 172.21.15.46:0/3621959947 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf8dd9b700 1 --2- 172.21.15.46:0/3621959947 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf90108f00 0x7faf9019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf8ed9d700 1 --2- 172.21.15.46:0/3621959947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf90109850 0x7faf9019f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf8e59c700 1 -- 172.21.15.46:0/3621959947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf90109850 msgr2=0x7faf9019f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf8e59c700 1 --2- 172.21.15.46:0/3621959947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf90109850 0x7faf9019f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf8e59c700 1 -- 172.21.15.46:0/3621959947 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf90108f00 msgr2=0x7faf9019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf8e59c700 1 --2- 172.21.15.46:0/3621959947 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf90108f00 0x7faf9019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf8e59c700 1 -- 172.21.15.46:0/3621959947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf9019fc80 con 0x7faf901045d0 2024-05-09T04:53:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.885+0000 7faf8e59c700 1 --2- 172.21.15.46:0/3621959947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf901045d0 0x7faf9019adb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faf8000d010 tx=0x7faf80007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.886+0000 7faf877fe700 1 -- 172.21.15.46:0/3621959947 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8001f070 con 0x7faf901045d0 2024-05-09T04:53:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.886+0000 7faf94f97700 1 -- 172.21.15.46:0/3621959947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf9019ff10 con 0x7faf901045d0 2024-05-09T04:53:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.886+0000 7faf877fe700 1 -- 172.21.15.46:0/3621959947 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf80007e50 con 0x7faf901045d0 2024-05-09T04:53:26.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.886+0000 7faf877fe700 1 -- 172.21.15.46:0/3621959947 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8001b4a0 con 0x7faf901045d0 2024-05-09T04:53:26.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.886+0000 7faf94f97700 1 -- 172.21.15.46:0/3621959947 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf901a03f0 con 0x7faf901045d0 2024-05-09T04:53:26.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.888+0000 7faf877fe700 1 -- 172.21.15.46:0/3621959947 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faf80022070 con 0x7faf901045d0 2024-05-09T04:53:26.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.889+0000 7faf877fe700 1 --2- 172.21.15.46:0/3621959947 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7faf7c0b29f0 0x7faf7c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.889+0000 7faf8dd9b700 1 --2- 172.21.15.46:0/3621959947 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7faf7c0b29f0 0x7faf7c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.889+0000 7faf877fe700 1 -- 172.21.15.46:0/3621959947 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7faf8000f020 con 0x7faf901045d0 2024-05-09T04:53:26.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.890+0000 7fafa39adb80 1 Processor -- start 2024-05-09T04:53:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.890+0000 7fafa39adb80 1 -- start start 2024-05-09T04:53:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.890+0000 7faf8dd9b700 1 --2- 172.21.15.46:0/3621959947 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7faf7c0b29f0 0x7faf7c0b4f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf901034b0 tx=0x7faf7800b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.890+0000 7fafa39adb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45a15160 0x563a45ab2900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.890+0000 7fafa39adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ab2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.890+0000 7fafa39adb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45aacab0 0x563a45aacee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.890+0000 7fafa39adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563a459caff0 con 0x563a4592b4a0 2024-05-09T04:53:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.890+0000 7fafa39adb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x563a459697f0 con 0x563a45aacab0 2024-05-09T04:53:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.890+0000 7fafa39adb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563a459c6fb0 con 0x563a45a15160 2024-05-09T04:53:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.890+0000 7faf8dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ab2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.890+0000 7faf8dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ab2e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35840/0 (socket says 172.21.15.46:35840) 2024-05-09T04:53:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.890+0000 7faf8dd9b700 1 -- 172.21.15.46:0/4051163273 learned_addr learned my addr 172.21.15.46:0/4051163273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.890+0000 7faf8e59c700 1 --2- 172.21.15.46:0/4051163273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45a15160 0x563a45ab2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.891+0000 7faf8ed9d700 1 --2- 172.21.15.46:0/4051163273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45aacab0 0x563a45aacee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.891+0000 7faf8e59c700 1 -- 172.21.15.46:0/4051163273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45aacab0 msgr2=0x563a45aacee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.891+0000 7faf8e59c700 1 --2- 172.21.15.46:0/4051163273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45aacab0 0x563a45aacee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.891+0000 7faf8e59c700 1 -- 172.21.15.46:0/4051163273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 msgr2=0x563a45ab2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.891+0000 7faf8e59c700 1 --2- 172.21.15.46:0/4051163273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ab2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.891+0000 7faf8e59c700 1 -- 172.21.15.46:0/4051163273 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a45961370 con 0x563a45a15160 2024-05-09T04:53:26.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.891+0000 7faf8dd9b700 1 --2- 172.21.15.46:0/4051163273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ab2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:26.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.891+0000 7faf8e59c700 1 --2- 172.21.15.46:0/4051163273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45a15160 0x563a45ab2900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faf800e0f00 tx=0x7faf80005700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1624bd1eb365e054 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:26.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.891+0000 7faf84ff9700 1 -- 172.21.15.46:0/4051163273 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf800b3240 con 0x563a45a15160 2024-05-09T04:53:26.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.891+0000 7faf84ff9700 1 -- 172.21.15.46:0/4051163273 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf800abe50 con 0x563a45a15160 2024-05-09T04:53:26.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.891+0000 7faf84ff9700 1 -- 172.21.15.46:0/4051163273 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf800aaf80 con 0x563a45a15160 2024-05-09T04:53:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.892+0000 7fafa39adb80 1 -- 172.21.15.46:0/4051163273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45a15160 msgr2=0x563a45ab2900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.892+0000 7fafa39adb80 1 --2- 172.21.15.46:0/4051163273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45a15160 0x563a45ab2900 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7faf800e0f00 tx=0x7faf80005700 comp rx=0 tx=0).stop 2024-05-09T04:53:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.892+0000 7fafa39adb80 1 -- 172.21.15.46:0/4051163273 shutdown_connections 2024-05-09T04:53:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.892+0000 7fafa39adb80 1 --2- 172.21.15.46:0/4051163273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45aacab0 0x563a45aacee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.892+0000 7fafa39adb80 1 --2- 172.21.15.46:0/4051163273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45a15160 0x563a45ab2900 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.892+0000 7fafa39adb80 1 --2- 172.21.15.46:0/4051163273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ab2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.892+0000 7fafa39adb80 1 -- 172.21.15.46:0/4051163273 >> 172.21.15.46:0/4051163273 conn(0x563a4598b6a0 msgr2=0x563a45aa3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:26.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.892+0000 7fafa39adb80 1 -- 172.21.15.46:0/4051163273 shutdown_connections 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.892+0000 7fafa39adb80 1 -- 172.21.15.46:0/4051163273 wait complete. 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.893+0000 7fafa39adb80 1 -- start start 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.893+0000 7fafa39adb80 1 Processor -- start 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.893+0000 7fafa39adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45abc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.893+0000 7fafa39adb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45a15160 0x563a45abc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.893+0000 7fafa39adb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45aacab0 0x563a45ab6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.893+0000 7fafa39adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563a45846ba0 con 0x563a4592b4a0 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.893+0000 7fafa39adb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x563a45985150 con 0x563a45a15160 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.893+0000 7fafa39adb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563a45aad770 con 0x563a45aacab0 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.893+0000 7fafa39adb80 10 client.0 _ll_register_callbacks cb 0x563a4583c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.893+0000 7faf8e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45abc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.893+0000 7faf8e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45abc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35842/0 (socket says 172.21.15.46:35842) 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.893+0000 7faf8e59c700 1 -- 172.21.15.46:0/198827976 learned_addr learned my addr 172.21.15.46:0/198827976 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.894+0000 7faf8dd9b700 1 --2- 172.21.15.46:0/198827976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45a15160 0x563a45abc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.894+0000 7faf8ed9d700 1 --2- 172.21.15.46:0/198827976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45aacab0 0x563a45ab6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.894+0000 7faf8e59c700 1 -- 172.21.15.46:0/198827976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45aacab0 msgr2=0x563a45ab6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.894+0000 7faf8e59c700 1 --2- 172.21.15.46:0/198827976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45aacab0 0x563a45ab6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.894+0000 7faf8e59c700 1 -- 172.21.15.46:0/198827976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45a15160 msgr2=0x563a45abc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.894+0000 7faf8e59c700 1 --2- 172.21.15.46:0/198827976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45a15160 0x563a45abc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.894+0000 7faf8e59c700 1 -- 172.21.15.46:0/198827976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a45846490 con 0x563a4592b4a0 2024-05-09T04:53:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.894+0000 7faf8e59c700 1 --2- 172.21.15.46:0/198827976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45abc130 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7faf800095c0 tx=0x7faf800b3690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c489cff1554e17e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.894+0000 7faf75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.894+0000 7faf75ffb700 1 -- 172.21.15.46:0/198827976 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8006f060 con 0x563a4592b4a0 2024-05-09T04:53:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.894+0000 7faf75ffb700 1 -- 172.21.15.46:0/198827976 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7faf800cd320 con 0x563a4592b4a0 2024-05-09T04:53:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.894+0000 7fafa39adb80 1 -- 172.21.15.46:0/198827976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf800abe50 con 0x563a4592b4a0 2024-05-09T04:53:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.894+0000 7faf75ffb700 1 -- 172.21.15.46:0/198827976 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faf800b3240 con 0x563a4592b4a0 2024-05-09T04:53:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.894+0000 7fafa39adb80 10 client.15936 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.894+0000 7fafa39adb80 1 -- 172.21.15.46:0/198827976 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563a45ac3d50 con 0x563a4592b4a0 2024-05-09T04:53:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.895+0000 7faf75ffb700 1 -- 172.21.15.46:0/198827976 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf800abe50 con 0x563a4592b4a0 2024-05-09T04:53:26.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.895+0000 7faf75ffb700 1 -- 172.21.15.46:0/198827976 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7faf80011070 con 0x563a4592b4a0 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.895+0000 7fafa39adb80 10 client.15936 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.895+0000 7fafa39adb80 10 client.15936 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.895+0000 7fafa39adb80 20 client.15936 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.895+0000 7fafa39adb80 1 --2- 172.21.15.46:0/198827976 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x563a45ac6220 0x563a45ac8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.895+0000 7fafa39adb80 4 client.15936 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.895+0000 7fafa39adb80 1 -- 172.21.15.46:0/198827976 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a459d89d0 con 0x563a45ac6220 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.895+0000 7faf8dd9b700 1 --2- 172.21.15.46:0/198827976 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x563a45ac6220 0x563a45ac8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.896+0000 7faf8dd9b700 1 --2- 172.21.15.46:0/198827976 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x563a45ac6220 0x563a45ac8640 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de0720c5d3dad00 server_cookie=e7a9b2a1cd7a06f2 in_seq=0 out_seq=0 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.896+0000 7faf75ffb700 10 client.15936 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.896+0000 7faf75ffb700 1 -- 172.21.15.46:0/198827976 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563a459d89d0 con 0x563a45ac6220 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.896+0000 7faf75ffb700 10 client.15936 handle_command_reply: tid=0 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.896+0000 7fafa39adb80 1 client.15936 shutdown 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.897+0000 7fafa39adb80 1 -- 172.21.15.46:0/198827976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 msgr2=0x563a45abc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.897+0000 7fafa39adb80 1 --2- 172.21.15.46:0/198827976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45abc130 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7faf800095c0 tx=0x7faf800b3690 comp rx=0 tx=0).stop 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.897+0000 7fafa39adb80 1 -- 172.21.15.46:0/198827976 shutdown_connections 2024-05-09T04:53:26.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.897+0000 7fafa39adb80 1 --2- 172.21.15.46:0/198827976 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x563a45ac6220 0x563a45ac8640 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.897+0000 7fafa39adb80 1 --2- 172.21.15.46:0/198827976 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45aacab0 0x563a45ab6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.897+0000 7fafa39adb80 1 --2- 172.21.15.46:0/198827976 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45abc130 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.897+0000 7faf75ffb700 0 client.15936 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:53:26.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.897+0000 7fafa39adb80 1 --2- 172.21.15.46:0/198827976 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45a15160 0x563a45abc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.897+0000 7fafa39adb80 1 -- 172.21.15.46:0/198827976 >> 172.21.15.46:0/198827976 conn(0x563a4598b6a0 msgr2=0x563a45aa31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:26.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.897+0000 7fafa39adb80 1 -- 172.21.15.46:0/198827976 shutdown_connections 2024-05-09T04:53:26.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.897+0000 7fafa39adb80 1 -- 172.21.15.46:0/198827976 wait complete. 2024-05-09T04:53:26.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.897+0000 7fafa39adb80 20 client.15936 trim_cache size 0 max 16384 2024-05-09T04:53:26.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.912+0000 7fafa39adb80 1 Processor -- start 2024-05-09T04:53:26.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.912+0000 7fafa39adb80 1 -- start start 2024-05-09T04:53:26.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.912+0000 7fafa39adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ab0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.912+0000 7fafa39adb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45a15160 0x563a45aaf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.912+0000 7fafa39adb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45aacab0 0x563a45acddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.912+0000 7fafa39adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf901af520 con 0x563a4592b4a0 2024-05-09T04:53:26.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.912+0000 7fafa39adb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7faf901acdb0 con 0x563a45a15160 2024-05-09T04:53:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.912+0000 7fafa39adb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faf901aac20 con 0x563a45aacab0 2024-05-09T04:53:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf8e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ab0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf8e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ab0ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35848/0 (socket says 172.21.15.46:35848) 2024-05-09T04:53:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf8e59c700 1 -- 172.21.15.46:0/2504182212 learned_addr learned my addr 172.21.15.46:0/2504182212 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf8ed9d700 1 --2- 172.21.15.46:0/2504182212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45aacab0 0x563a45acddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf8dd9b700 1 --2- 172.21.15.46:0/2504182212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45a15160 0x563a45aaf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf8e59c700 1 -- 172.21.15.46:0/2504182212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45aacab0 msgr2=0x563a45acddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf8e59c700 1 --2- 172.21.15.46:0/2504182212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45aacab0 0x563a45acddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf8e59c700 1 -- 172.21.15.46:0/2504182212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45a15160 msgr2=0x563a45aaf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf8e59c700 1 --2- 172.21.15.46:0/2504182212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45a15160 0x563a45aaf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf8e59c700 1 -- 172.21.15.46:0/2504182212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a45aa4d00 con 0x563a4592b4a0 2024-05-09T04:53:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf8dd9b700 1 --2- 172.21.15.46:0/2504182212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45a15160 0x563a45aaf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:53:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf8ed9d700 1 --2- 172.21.15.46:0/2504182212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45aacab0 0x563a45acddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:26.916 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf8e59c700 1 --2- 172.21.15.46:0/2504182212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ab0ad0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7faf800a7d90 tx=0x7faf800a6a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a3322a17d9c11a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf77fff700 1 -- 172.21.15.46:0/2504182212 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf800c3070 con 0x563a4592b4a0 2024-05-09T04:53:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf77fff700 1 -- 172.21.15.46:0/2504182212 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf800b3240 con 0x563a4592b4a0 2024-05-09T04:53:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.913+0000 7faf77fff700 1 -- 172.21.15.46:0/2504182212 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf800b11f0 con 0x563a4592b4a0 2024-05-09T04:53:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.914+0000 7fafa39adb80 1 -- 172.21.15.46:0/2504182212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 msgr2=0x563a45ab0ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.914+0000 7fafa39adb80 1 --2- 172.21.15.46:0/2504182212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ab0ad0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7faf800a7d90 tx=0x7faf800a6a60 comp rx=0 tx=0).stop 2024-05-09T04:53:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.914+0000 7fafa39adb80 1 -- 172.21.15.46:0/2504182212 shutdown_connections 2024-05-09T04:53:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.914+0000 7fafa39adb80 1 --2- 172.21.15.46:0/2504182212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45aacab0 0x563a45acddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.914+0000 7fafa39adb80 1 --2- 172.21.15.46:0/2504182212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ab0ad0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.914+0000 7fafa39adb80 1 --2- 172.21.15.46:0/2504182212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45a15160 0x563a45aaf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.914+0000 7fafa39adb80 1 -- 172.21.15.46:0/2504182212 >> 172.21.15.46:0/2504182212 conn(0x563a4598b6a0 msgr2=0x563a45ab2900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.914+0000 7fafa39adb80 1 -- 172.21.15.46:0/2504182212 shutdown_connections 2024-05-09T04:53:26.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.914+0000 7fafa39adb80 1 -- 172.21.15.46:0/2504182212 wait complete. 2024-05-09T04:53:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.914+0000 7fafa39adb80 1 -- start start 2024-05-09T04:53:26.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7fafa39adb80 1 Processor -- start 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7fafa39adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ac2420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7fafa39adb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45a15160 0x563a45ac2960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7fafa39adb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45aacab0 0x563a45abd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7fafa39adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7faf901ab270 con 0x563a4592b4a0 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7fafa39adb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7faf901ac6e0 con 0x563a45aacab0 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7fafa39adb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7faf901ab940 con 0x563a45a15160 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7fafa39adb80 10 client.0 _ll_register_callbacks cb 0x563a4583c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7faf8e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ac2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7faf8e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ac2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35858/0 (socket says 172.21.15.46:35858) 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7faf8dd9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45a15160 0x563a45ac2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7faf8e59c700 1 -- 172.21.15.46:0/3791104549 learned_addr learned my addr 172.21.15.46:0/3791104549 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7faf8ed9d700 1 --2- 172.21.15.46:0/3791104549 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45aacab0 0x563a45abd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7faf8e59c700 1 -- 172.21.15.46:0/3791104549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45a15160 msgr2=0x563a45ac2960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7faf8e59c700 1 --2- 172.21.15.46:0/3791104549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45a15160 0x563a45ac2960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7faf8e59c700 1 -- 172.21.15.46:0/3791104549 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45aacab0 msgr2=0x563a45abd880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:26.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7faf8e59c700 1 --2- 172.21.15.46:0/3791104549 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45aacab0 0x563a45abd880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.915+0000 7faf8e59c700 1 -- 172.21.15.46:0/3791104549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a45abdf30 con 0x563a4592b4a0 2024-05-09T04:53:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7faf8e59c700 1 --2- 172.21.15.46:0/3791104549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ac2420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7faf800aef20 tx=0x7faf80003c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=557632de3033ee14 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7faf75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7faf75ffb700 1 -- 172.21.15.46:0/3791104549 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8008e060 con 0x563a4592b4a0 2024-05-09T04:53:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7faf75ffb700 1 -- 172.21.15.46:0/3791104549 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7faf80005c00 con 0x563a4592b4a0 2024-05-09T04:53:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7fafa39adb80 1 -- 172.21.15.46:0/3791104549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf800b3240 con 0x563a4592b4a0 2024-05-09T04:53:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7faf75ffb700 1 -- 172.21.15.46:0/3791104549 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7faf800b8050 con 0x563a4592b4a0 2024-05-09T04:53:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7fafa39adb80 10 client.15942 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7fafa39adb80 1 -- 172.21.15.46:0/3791104549 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563a45abdf30 con 0x563a4592b4a0 2024-05-09T04:53:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7faf75ffb700 1 -- 172.21.15.46:0/3791104549 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf800b3240 con 0x563a4592b4a0 2024-05-09T04:53:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7faf75ffb700 1 -- 172.21.15.46:0/3791104549 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7faf80011050 con 0x563a4592b4a0 2024-05-09T04:53:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7fafa39adb80 10 client.15942 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7fafa39adb80 10 client.15942 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:53:26.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7fafa39adb80 20 client.15942 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7fafa39adb80 1 --2- 172.21.15.46:0/3791104549 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x563a45b30520 0x563a45b32940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7fafa39adb80 4 client.15942 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:53:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.916+0000 7fafa39adb80 1 -- 172.21.15.46:0/3791104549 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563a459d2a70 con 0x563a45b30520 2024-05-09T04:53:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.917+0000 7faf8dd9b700 1 --2- 172.21.15.46:0/3791104549 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x563a45b30520 0x563a45b32940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.917+0000 7faf8dd9b700 1 --2- 172.21.15.46:0/3791104549 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x563a45b30520 0x563a45b32940 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=53d00355abd713c8 server_cookie=e74d56f86fe00cba in_seq=0 out_seq=0 2024-05-09T04:53:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.917+0000 7faf75ffb700 10 client.15942 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:53:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.917+0000 7faf75ffb700 1 -- 172.21.15.46:0/3791104549 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563a459d2a70 con 0x563a45b30520 2024-05-09T04:53:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.917+0000 7faf75ffb700 10 client.15942 handle_command_reply: tid=0 2024-05-09T04:53:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.917+0000 7fafa39adb80 1 client.15942 shutdown 2024-05-09T04:53:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.917+0000 7fafa39adb80 1 -- 172.21.15.46:0/3791104549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 msgr2=0x563a45ac2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.917+0000 7fafa39adb80 1 --2- 172.21.15.46:0/3791104549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ac2420 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7faf800aef20 tx=0x7faf80003c90 comp rx=0 tx=0).stop 2024-05-09T04:53:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.917+0000 7fafa39adb80 1 -- 172.21.15.46:0/3791104549 shutdown_connections 2024-05-09T04:53:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.917+0000 7fafa39adb80 1 --2- 172.21.15.46:0/3791104549 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x563a45b30520 0x563a45b32940 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7fafa39adb80 1 --2- 172.21.15.46:0/3791104549 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563a45aacab0 0x563a45abd880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7fafa39adb80 1 --2- 172.21.15.46:0/3791104549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563a4592b4a0 0x563a45ac2420 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7fafa39adb80 1 --2- 172.21.15.46:0/3791104549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563a45a15160 0x563a45ac2960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7fafa39adb80 1 -- 172.21.15.46:0/3791104549 >> 172.21.15.46:0/3791104549 conn(0x563a4598b6a0 msgr2=0x563a45ab5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7faf75ffb700 0 client.15942 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:53:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7fafa39adb80 1 -- 172.21.15.46:0/3791104549 shutdown_connections 2024-05-09T04:53:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7fafa39adb80 1 -- 172.21.15.46:0/3791104549 wait complete. 2024-05-09T04:53:26.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7fafa39adb80 20 client.15942 trim_cache size 0 max 16384 2024-05-09T04:53:26.924 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:53:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7faf94f97700 1 -- 172.21.15.46:0/3621959947 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7faf7c0b29f0 msgr2=0x7faf7c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7faf94f97700 1 --2- 172.21.15.46:0/3621959947 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7faf7c0b29f0 0x7faf7c0b4f00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf901034b0 tx=0x7faf7800b410 comp rx=0 tx=0).stop 2024-05-09T04:53:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7faf94f97700 1 -- 172.21.15.46:0/3621959947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf901045d0 msgr2=0x7faf9019adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7faf94f97700 1 --2- 172.21.15.46:0/3621959947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf901045d0 0x7faf9019adb0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7faf8000d010 tx=0x7faf80007a20 comp rx=0 tx=0).stop 2024-05-09T04:53:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7faf94f97700 1 -- 172.21.15.46:0/3621959947 shutdown_connections 2024-05-09T04:53:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7faf94f97700 1 --2- 172.21.15.46:0/3621959947 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7faf7c0b29f0 0x7faf7c0b4f00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7faf94f97700 1 --2- 172.21.15.46:0/3621959947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7faf90109850 0x7faf9019f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7faf94f97700 1 --2- 172.21.15.46:0/3621959947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7faf901045d0 0x7faf9019adb0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7faf94f97700 1 --2- 172.21.15.46:0/3621959947 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7faf90108f00 0x7faf9019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7faf94f97700 1 -- 172.21.15.46:0/3621959947 >> 172.21.15.46:0/3621959947 conn(0x7faf900ffea0 msgr2=0x7faf9010a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7faf94f97700 1 -- 172.21.15.46:0/3621959947 shutdown_connections 2024-05-09T04:53:26.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:26.918+0000 7faf94f97700 1 -- 172.21.15.46:0/3621959947 wait complete. 2024-05-09T04:53:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:26 smithi110 ceph-mon[34904]: pgmap v73: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 85 MiB/s wr, 31 op/s 2024-05-09T04:53:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:26 smithi072 ceph-mon[34088]: pgmap v73: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 85 MiB/s wr, 31 op/s 2024-05-09T04:53:27.346 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:27.347 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:27.347 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:27.347 INFO:tasks.workunit.client.0.smithi046.stdout:iozone test complete. 2024-05-09T04:53:27.347 INFO:tasks.workunit.client.0.smithi046.stderr:+ iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-05-09T04:53:27.413 INFO:tasks.workunit.client.0.smithi046.stdout: Iozone: Performance Test of File I/O 2024-05-09T04:53:27.413 INFO:tasks.workunit.client.0.smithi046.stdout: Version $Revision: 3.494 $ 2024-05-09T04:53:27.413 INFO:tasks.workunit.client.0.smithi046.stdout: Compiled for 64 bit mode. 2024-05-09T04:53:27.413 INFO:tasks.workunit.client.0.smithi046.stdout: Build: linux-AMD64 2024-05-09T04:53:27.413 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:27.413 INFO:tasks.workunit.client.0.smithi046.stdout: Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins 2024-05-09T04:53:27.413 INFO:tasks.workunit.client.0.smithi046.stdout: Al Slater, Scott Rhine, Mike Wisner, Ken Goss 2024-05-09T04:53:27.413 INFO:tasks.workunit.client.0.smithi046.stdout: Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, 2024-05-09T04:53:27.413 INFO:tasks.workunit.client.0.smithi046.stdout: Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, 2024-05-09T04:53:27.414 INFO:tasks.workunit.client.0.smithi046.stdout: Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, 2024-05-09T04:53:27.414 INFO:tasks.workunit.client.0.smithi046.stdout: Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, 2024-05-09T04:53:27.414 INFO:tasks.workunit.client.0.smithi046.stdout: Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, 2024-05-09T04:53:27.414 INFO:tasks.workunit.client.0.smithi046.stdout: Vangel Bojaxhi, Ben England, Vikentsi Lapa, 2024-05-09T04:53:27.414 INFO:tasks.workunit.client.0.smithi046.stdout: Alexey Skidanov, Sudhir Kumar. 2024-05-09T04:53:27.414 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:27.414 INFO:tasks.workunit.client.0.smithi046.stdout: Run began: Thu May 9 04:53:27 2024 2024-05-09T04:53:27.414 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:27.414 INFO:tasks.workunit.client.0.smithi046.stdout: Include close in write timing 2024-05-09T04:53:27.414 INFO:tasks.workunit.client.0.smithi046.stdout: Include fsync in write timing 2024-05-09T04:53:27.414 INFO:tasks.workunit.client.0.smithi046.stdout: File size set to 10485760 kB 2024-05-09T04:53:27.414 INFO:tasks.workunit.client.0.smithi046.stdout: Record Size 1024 kB 2024-05-09T04:53:27.414 INFO:tasks.workunit.client.0.smithi046.stdout: Command line used: iozone -c -e -s 10240M -r 1M -t 1 -F f3 -i 0 -i 1 2024-05-09T04:53:27.414 INFO:tasks.workunit.client.0.smithi046.stdout: Output is in kBytes/sec 2024-05-09T04:53:27.415 INFO:tasks.workunit.client.0.smithi046.stdout: Time Resolution = 0.000001 seconds. 2024-05-09T04:53:27.415 INFO:tasks.workunit.client.0.smithi046.stdout: Processor cache size set to 1024 kBytes. 2024-05-09T04:53:27.415 INFO:tasks.workunit.client.0.smithi046.stdout: Processor cache line size set to 32 bytes. 2024-05-09T04:53:27.415 INFO:tasks.workunit.client.0.smithi046.stdout: File stride size set to 17 * record size. 2024-05-09T04:53:27.415 INFO:tasks.workunit.client.0.smithi046.stdout: Throughput test with 1 process 2024-05-09T04:53:27.415 INFO:tasks.workunit.client.0.smithi046.stdout: Each process writes a 10485760 kByte file in 1024 kByte records 2024-05-09T04:53:28.015 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:53:28.376 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:28.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:28 smithi046 ceph-mon[30136]: pgmap v74: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-05-09T04:53:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:28 smithi110 ceph-mon[34904]: pgmap v74: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-05-09T04:53:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:28 smithi072 ceph-mon[34088]: pgmap v74: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 85 MiB/s wr, 34 op/s 2024-05-09T04:53:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.991+0000 7fea334b8640 1 -- 172.21.15.46:0/2304787281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea2c108bb0 msgr2=0x7fea2c102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.991+0000 7fea334b8640 1 --2- 172.21.15.46:0/2304787281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea2c108bb0 0x7fea2c102750 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fea28009900 tx=0x7fea2802a410 comp rx=0 tx=0).stop 2024-05-09T04:53:29.992 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.991+0000 7fea334b8640 1 -- 172.21.15.46:0/2304787281 shutdown_connections 2024-05-09T04:53:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.991+0000 7fea334b8640 1 --2- 172.21.15.46:0/2304787281 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea2c109050 0x7fea2c102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.991+0000 7fea334b8640 1 --2- 172.21.15.46:0/2304787281 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea2c103300 0x7fea2c10f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.991+0000 7fea334b8640 1 --2- 172.21.15.46:0/2304787281 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea2c108bb0 0x7fea2c102750 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.991+0000 7fea334b8640 1 -- 172.21.15.46:0/2304787281 >> 172.21.15.46:0/2304787281 conn(0x7fea2c078280 msgr2=0x7fea2c0786c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.991+0000 7fea334b8640 1 -- 172.21.15.46:0/2304787281 shutdown_connections 2024-05-09T04:53:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.991+0000 7fea334b8640 1 -- 172.21.15.46:0/2304787281 wait complete. 2024-05-09T04:53:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.992+0000 7fea334b8640 1 Processor -- start 2024-05-09T04:53:29.993 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.992+0000 7fea334b8640 1 -- start start 2024-05-09T04:53:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.993+0000 7fea334b8640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea2c103300 0x7fea2c199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:29.995 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.993+0000 7fea334b8640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea2c108bb0 0x7fea2c199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.993+0000 7fea334b8640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea2c109050 0x7fea2c199fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.994+0000 7fea334b8640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c112e70 con 0x7fea2c103300 2024-05-09T04:53:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.994+0000 7fea334b8640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c112cf0 con 0x7fea2c109050 2024-05-09T04:53:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.994+0000 7fea334b8640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c112ff0 con 0x7fea2c108bb0 2024-05-09T04:53:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.994+0000 7fea31254640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea2c103300 0x7fea2c199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.994+0000 7fea31254640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea2c103300 0x7fea2c199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35876/0 (socket says 172.21.15.46:35876) 2024-05-09T04:53:29.996 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.994+0000 7fea31254640 1 -- 172.21.15.46:0/1635915107 learned_addr learned my addr 172.21.15.46:0/1635915107 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.994+0000 7fea31254640 1 -- 172.21.15.46:0/1635915107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea2c108bb0 msgr2=0x7fea2c199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.994+0000 7fea31254640 1 --2- 172.21.15.46:0/1635915107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea2c108bb0 0x7fea2c199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.994+0000 7fea31254640 1 -- 172.21.15.46:0/1635915107 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea2c109050 msgr2=0x7fea2c199fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.994+0000 7fea31254640 1 --2- 172.21.15.46:0/1635915107 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea2c109050 0x7fea2c199fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:29.997 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.995+0000 7fea31254640 1 -- 172.21.15.46:0/1635915107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea2c1a8c30 con 0x7fea2c103300 2024-05-09T04:53:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.995+0000 7fea31254640 1 --2- 172.21.15.46:0/1635915107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea2c103300 0x7fea2c199560 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fea28031440 tx=0x7fea28003d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.995+0000 7fea227fc640 1 -- 172.21.15.46:0/1635915107 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2803f070 con 0x7fea2c103300 2024-05-09T04:53:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.996+0000 7fea334b8640 1 -- 172.21.15.46:0/1635915107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea2c1a8ec0 con 0x7fea2c103300 2024-05-09T04:53:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.996+0000 7fea227fc640 1 -- 172.21.15.46:0/1635915107 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea28031d60 con 0x7fea2c103300 2024-05-09T04:53:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.996+0000 7fea227fc640 1 -- 172.21.15.46:0/1635915107 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea28035030 con 0x7fea2c103300 2024-05-09T04:53:29.998 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:29.996+0000 7fea334b8640 1 -- 172.21.15.46:0/1635915107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea2c1a9270 con 0x7fea2c103300 2024-05-09T04:53:30.003 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.002+0000 7fea3a36b740 1 Processor -- start 2024-05-09T04:53:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.003+0000 7fea3a36b740 1 -- start start 2024-05-09T04:53:30.004 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.003+0000 7fea227fc640 1 -- 172.21.15.46:0/1635915107 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fea28042070 con 0x7fea2c103300 2024-05-09T04:53:30.005 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.004+0000 7fea3a36b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514aed1b0 0x562514e90680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.004+0000 7fea3a36b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514ac42f0 0x562514e90bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.004+0000 7fea31a55640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514aed1b0 0x562514e90680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.004+0000 7fea31a55640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514aed1b0 0x562514e90680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35884/0 (socket says 172.21.15.46:35884) 2024-05-09T04:53:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.004+0000 7fea227fc640 1 --2- 172.21.15.46:0/1635915107 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fea180b2aa0 0x7fea180b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.005+0000 7fea31a55640 1 -- 172.21.15.46:0/2900591965 learned_addr learned my addr 172.21.15.46:0/2900591965 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.005+0000 7fea3a36b740 1 --2- 172.21.15.46:0/2900591965 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514acc4d0 0x562514e91100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:30.006 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.005+0000 7fea3a36b740 1 -- 172.21.15.46:0/2900591965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562514df57c0 con 0x562514aed1b0 2024-05-09T04:53:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.005+0000 7fea30a53640 1 --2- 172.21.15.46:0/1635915107 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fea180b2aa0 0x7fea180b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.005+0000 7fea3a36b740 1 -- 172.21.15.46:0/2900591965 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562514d910e0 con 0x562514acc4d0 2024-05-09T04:53:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.005+0000 7fea3a36b740 1 -- 172.21.15.46:0/2900591965 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562514df1780 con 0x562514ac42f0 2024-05-09T04:53:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.005+0000 7fea227fc640 1 -- 172.21.15.46:0/1635915107 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fea281030a0 con 0x7fea2c103300 2024-05-09T04:53:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.006+0000 7fea30a53640 1 --2- 172.21.15.46:0/1635915107 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fea180b2aa0 0x7fea180b4fb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fea2c10e9a0 tx=0x7fea1c005c10 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.006+0000 7fea31a55640 1 -- 172.21.15.46:0/2900591965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514ac42f0 msgr2=0x562514e90bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.006+0000 7fea31a55640 1 --2- 172.21.15.46:0/2900591965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514ac42f0 0x562514e90bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.007 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.006+0000 7fea31a55640 1 -- 172.21.15.46:0/2900591965 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514acc4d0 msgr2=0x562514e91100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:53:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.006+0000 7fea31a55640 1 --2- 172.21.15.46:0/2900591965 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514acc4d0 0x562514e91100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.008 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.006+0000 7fea31a55640 1 -- 172.21.15.46:0/2900591965 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562514dba7c0 con 0x562514aed1b0 2024-05-09T04:53:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.007+0000 7fea31a55640 1 --2- 172.21.15.46:0/2900591965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514aed1b0 0x562514e90680 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fea24009d90 tx=0x7fea2400b420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d54642c19a7bbbb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.007+0000 7fea177fe640 1 -- 172.21.15.46:0/2900591965 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2400bcc0 con 0x562514aed1b0 2024-05-09T04:53:30.009 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.007+0000 7fea177fe640 1 -- 172.21.15.46:0/2900591965 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea24017070 con 0x562514aed1b0 2024-05-09T04:53:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.008+0000 7fea177fe640 1 -- 172.21.15.46:0/2900591965 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea240116a0 con 0x562514aed1b0 2024-05-09T04:53:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.008+0000 7fea3a36b740 1 -- 172.21.15.46:0/2900591965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514aed1b0 msgr2=0x562514e90680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:30.010 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.008+0000 7fea3a36b740 1 --2- 172.21.15.46:0/2900591965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514aed1b0 0x562514e90680 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fea24009d90 tx=0x7fea2400b420 comp rx=0 tx=0).stop 2024-05-09T04:53:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.009+0000 7fea3a36b740 1 -- 172.21.15.46:0/2900591965 shutdown_connections 2024-05-09T04:53:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.009+0000 7fea3a36b740 1 --2- 172.21.15.46:0/2900591965 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514ac42f0 0x562514e90bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.009+0000 7fea3a36b740 1 --2- 172.21.15.46:0/2900591965 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514acc4d0 0x562514e91100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.009+0000 7fea3a36b740 1 --2- 172.21.15.46:0/2900591965 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514aed1b0 0x562514e90680 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.009+0000 7fea3a36b740 1 -- 172.21.15.46:0/2900591965 >> 172.21.15.46:0/2900591965 conn(0x562514ae1300 msgr2=0x562514e8ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:30.011 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.009+0000 7fea3a36b740 1 -- 172.21.15.46:0/2900591965 shutdown_connections 2024-05-09T04:53:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.009+0000 7fea3a36b740 1 -- 172.21.15.46:0/2900591965 wait complete. 2024-05-09T04:53:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.009+0000 7fea3a36b740 1 -- start start 2024-05-09T04:53:30.012 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.009+0000 7fea3a36b740 1 Processor -- start 2024-05-09T04:53:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.010+0000 7fea3a36b740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514ac42f0 0x562514e9c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.010+0000 7fea3a36b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514e9cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.010+0000 7fea3a36b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514aed1b0 0x562514e9abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.010+0000 7fea3a36b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562514e9d850 con 0x562514acc4d0 2024-05-09T04:53:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.010+0000 7fea3a36b740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562514d7c9c0 con 0x562514ac42f0 2024-05-09T04:53:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.010+0000 7fea3a36b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562514e9d9d0 con 0x562514aed1b0 2024-05-09T04:53:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.010+0000 7fea30a53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514e9cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.010+0000 7fea30a53640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514e9cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35894/0 (socket says 172.21.15.46:35894) 2024-05-09T04:53:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.010+0000 7fea30a53640 1 -- 172.21.15.46:0/4205827402 learned_addr learned my addr 172.21.15.46:0/4205827402 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.010+0000 7fea3a36b740 10 client.0 _ll_register_callbacks cb 0x562514da9020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:30.015 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.010+0000 7fea31a55640 1 --2- 172.21.15.46:0/4205827402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514aed1b0 0x562514e9abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.011+0000 7fea30a53640 1 -- 172.21.15.46:0/4205827402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514aed1b0 msgr2=0x562514e9abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.011+0000 7fea30a53640 1 --2- 172.21.15.46:0/4205827402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514aed1b0 0x562514e9abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.011+0000 7fea30a53640 1 -- 172.21.15.46:0/4205827402 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514ac42f0 msgr2=0x562514e9c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.011+0000 7fea30a53640 1 --2- 172.21.15.46:0/4205827402 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514ac42f0 0x562514e9c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.011+0000 7fea30a53640 1 -- 172.21.15.46:0/4205827402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562514da31f0 con 0x562514acc4d0 2024-05-09T04:53:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.011+0000 7fea30a53640 1 --2- 172.21.15.46:0/4205827402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514e9cc10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00f6b0 tx=0x7fea1c00fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e0a51f9d6f82bb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.011+0000 7fea14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.011+0000 7fea14ff9640 1 -- 172.21.15.46:0/4205827402 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea1c020070 con 0x562514acc4d0 2024-05-09T04:53:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.011+0000 7fea14ff9640 1 -- 172.21.15.46:0/4205827402 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fea1c00e900 con 0x562514acc4d0 2024-05-09T04:53:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.011+0000 7fea3a36b740 1 -- 172.21.15.46:0/4205827402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea24017070 con 0x562514acc4d0 2024-05-09T04:53:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.011+0000 7fea14ff9640 1 -- 172.21.15.46:0/4205827402 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea1c00eb00 con 0x562514acc4d0 2024-05-09T04:53:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.011+0000 7fea3a36b740 10 client.15954 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.011+0000 7fea3a36b740 1 -- 172.21.15.46:0/4205827402 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562514b58790 con 0x562514acc4d0 2024-05-09T04:53:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.012+0000 7fea14ff9640 1 -- 172.21.15.46:0/4205827402 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea24017070 con 0x562514acc4d0 2024-05-09T04:53:30.016 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.012+0000 7fea14ff9640 1 -- 172.21.15.46:0/4205827402 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fea1c01b070 con 0x562514acc4d0 2024-05-09T04:53:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.012+0000 7fea3a36b740 10 client.15954 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.012+0000 7fea3a36b740 10 client.15954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.012+0000 7fea3a36b740 20 client.15954 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.012+0000 7fea3a36b740 1 --2- 172.21.15.46:0/4205827402 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562514aabd80 0x562514ead130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.012+0000 7fea3a36b740 4 client.15954 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.013+0000 7fea3a36b740 1 -- 172.21.15.46:0/4205827402 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562514e033d0 con 0x562514aabd80 2024-05-09T04:53:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.014+0000 7fea31254640 1 --2- 172.21.15.46:0/4205827402 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562514aabd80 0x562514ead130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:30.017 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.016+0000 7fea14ff9640 10 client.15954 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:30.018 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.016+0000 7fea31254640 1 --2- 172.21.15.46:0/4205827402 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562514aabd80 0x562514ead130 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f22be72ef801ab21 server_cookie=62199f56f7f34105 in_seq=0 out_seq=0 2024-05-09T04:53:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.019+0000 7fea14ff9640 1 -- 172.21.15.46:0/4205827402 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562514e033d0 con 0x562514aabd80 2024-05-09T04:53:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.019+0000 7fea14ff9640 10 client.15954 handle_command_reply: tid=0 2024-05-09T04:53:30.020 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.019+0000 7fea3a36b740 1 client.15954 shutdown 2024-05-09T04:53:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.019+0000 7fea3a36b740 1 -- 172.21.15.46:0/4205827402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 msgr2=0x562514e9cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.019+0000 7fea3a36b740 1 --2- 172.21.15.46:0/4205827402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514e9cc10 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fea1c00f6b0 tx=0x7fea1c00fab0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.019+0000 7fea3a36b740 1 -- 172.21.15.46:0/4205827402 shutdown_connections 2024-05-09T04:53:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.019+0000 7fea3a36b740 1 --2- 172.21.15.46:0/4205827402 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562514aabd80 0x562514ead130 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.021 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.019+0000 7fea3a36b740 1 --2- 172.21.15.46:0/4205827402 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514aed1b0 0x562514e9abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.019+0000 7fea3a36b740 1 --2- 172.21.15.46:0/4205827402 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514e9cc10 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.019+0000 7fea14ff9640 0 client.15954 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.019+0000 7fea3a36b740 1 --2- 172.21.15.46:0/4205827402 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514ac42f0 0x562514e9c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.019+0000 7fea3a36b740 1 -- 172.21.15.46:0/4205827402 >> 172.21.15.46:0/4205827402 conn(0x562514ae1300 msgr2=0x562514e8b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.020+0000 7fea3a36b740 1 -- 172.21.15.46:0/4205827402 shutdown_connections 2024-05-09T04:53:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.020+0000 7fea3a36b740 1 -- 172.21.15.46:0/4205827402 wait complete. 2024-05-09T04:53:30.022 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.020+0000 7fea3a36b740 20 client.15954 trim_cache size 0 max 16384 2024-05-09T04:53:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.040+0000 7fea3a36b740 1 Processor -- start 2024-05-09T04:53:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.040+0000 7fea3a36b740 1 -- start start 2024-05-09T04:53:30.042 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.040+0000 7fea3a36b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514aabd80 0x562514e8be10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.040+0000 7fea3a36b740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514ac42f0 0x562514eb2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.040+0000 7fea3a36b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514eb2e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.040+0000 7fea3a36b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1b4810 con 0x562514acc4d0 2024-05-09T04:53:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.040+0000 7fea3a36b740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1b20e0 con 0x562514ac42f0 2024-05-09T04:53:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.040+0000 7fea3a36b740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1aff30 con 0x562514aabd80 2024-05-09T04:53:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.041+0000 7fea31a55640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514eb2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.041+0000 7fea31a55640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514eb2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35900/0 (socket says 172.21.15.46:35900) 2024-05-09T04:53:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.041+0000 7fea31a55640 1 -- 172.21.15.46:0/252642156 learned_addr learned my addr 172.21.15.46:0/252642156 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.041+0000 7fea31a55640 1 -- 172.21.15.46:0/252642156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514aabd80 msgr2=0x562514e8be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.041+0000 7fea31a55640 1 --2- 172.21.15.46:0/252642156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514aabd80 0x562514e8be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.041+0000 7fea31a55640 1 -- 172.21.15.46:0/252642156 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514ac42f0 msgr2=0x562514eb2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.041+0000 7fea31a55640 1 --2- 172.21.15.46:0/252642156 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514ac42f0 0x562514eb2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.041+0000 7fea31a55640 1 -- 172.21.15.46:0/252642156 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562514bcf5f0 con 0x562514acc4d0 2024-05-09T04:53:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.041+0000 7fea31a55640 1 --2- 172.21.15.46:0/252642156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514eb2e70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fea24009d90 tx=0x7fea24009b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6890a47641624a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:30.043 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.041+0000 7fea16ffd640 1 -- 172.21.15.46:0/252642156 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea24022020 con 0x562514acc4d0 2024-05-09T04:53:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.042+0000 7fea16ffd640 1 -- 172.21.15.46:0/252642156 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea24017040 con 0x562514acc4d0 2024-05-09T04:53:30.044 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.042+0000 7fea16ffd640 1 -- 172.21.15.46:0/252642156 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2401bad0 con 0x562514acc4d0 2024-05-09T04:53:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.043+0000 7fea3a36b740 1 -- 172.21.15.46:0/252642156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 msgr2=0x562514eb2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.043+0000 7fea3a36b740 1 --2- 172.21.15.46:0/252642156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514eb2e70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fea24009d90 tx=0x7fea24009b00 comp rx=0 tx=0).stop 2024-05-09T04:53:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.043+0000 7fea3a36b740 1 -- 172.21.15.46:0/252642156 shutdown_connections 2024-05-09T04:53:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.043+0000 7fea3a36b740 1 --2- 172.21.15.46:0/252642156 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514eb2e70 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.043+0000 7fea3a36b740 1 --2- 172.21.15.46:0/252642156 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514ac42f0 0x562514eb2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.043+0000 7fea3a36b740 1 --2- 172.21.15.46:0/252642156 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514aabd80 0x562514e8be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.043+0000 7fea3a36b740 1 -- 172.21.15.46:0/252642156 >> 172.21.15.46:0/252642156 conn(0x562514ae1300 msgr2=0x562514e8b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.043+0000 7fea3a36b740 1 -- 172.21.15.46:0/252642156 shutdown_connections 2024-05-09T04:53:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.043+0000 7fea3a36b740 1 -- 172.21.15.46:0/252642156 wait complete. 2024-05-09T04:53:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.044+0000 7fea3a36b740 1 -- start start 2024-05-09T04:53:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.044+0000 7fea3a36b740 1 Processor -- start 2024-05-09T04:53:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.045+0000 7fea3a36b740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514aabd80 0x562514ea2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.045+0000 7fea3a36b740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514ac42f0 0x562514ea0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.045+0000 7fea3a36b740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514ea0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.045+0000 7fea3a36b740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1b05a0 con 0x562514acc4d0 2024-05-09T04:53:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.045+0000 7fea3a36b740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1b1a10 con 0x562514aabd80 2024-05-09T04:53:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.045+0000 7fea31a55640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514ea0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.045+0000 7fea31a55640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514ea0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35916/0 (socket says 172.21.15.46:35916) 2024-05-09T04:53:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.045+0000 7fea31a55640 1 -- 172.21.15.46:0/323831769 learned_addr learned my addr 172.21.15.46:0/323831769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.045+0000 7fea3a36b740 1 -- 172.21.15.46:0/323831769 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fea2c1b0c70 con 0x562514ac42f0 2024-05-09T04:53:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.045+0000 7fea3a36b740 10 client.0 _ll_register_callbacks cb 0x7fea1c00aec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.046+0000 7fea30a53640 1 --2- 172.21.15.46:0/323831769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514ac42f0 0x562514ea0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.046+0000 7fea31a55640 1 -- 172.21.15.46:0/323831769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514ac42f0 msgr2=0x562514ea0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.046+0000 7fea31a55640 1 --2- 172.21.15.46:0/323831769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514ac42f0 0x562514ea0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.046+0000 7fea31a55640 1 -- 172.21.15.46:0/323831769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514aabd80 msgr2=0x562514ea2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.046+0000 7fea31a55640 1 --2- 172.21.15.46:0/323831769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514aabd80 0x562514ea2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.046+0000 7fea31a55640 1 -- 172.21.15.46:0/323831769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562514db9ed0 con 0x562514acc4d0 2024-05-09T04:53:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.046+0000 7fea30a53640 1 --2- 172.21.15.46:0/323831769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514ac42f0 0x562514ea0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:53:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.046+0000 7fea31a55640 1 --2- 172.21.15.46:0/323831769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514ea0870 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fea2400b460 tx=0x7fea24012cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c426020a26211c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.046+0000 7fea14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.047+0000 7fea14ff9640 1 -- 172.21.15.46:0/323831769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2402d020 con 0x562514acc4d0 2024-05-09T04:53:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.047+0000 7fea14ff9640 1 -- 172.21.15.46:0/323831769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fea2401b4e0 con 0x562514acc4d0 2024-05-09T04:53:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.047+0000 7fea3a36b740 1 -- 172.21.15.46:0/323831769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea24017040 con 0x562514acc4d0 2024-05-09T04:53:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.047+0000 7fea14ff9640 1 -- 172.21.15.46:0/323831769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fea2401f040 con 0x562514acc4d0 2024-05-09T04:53:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.047+0000 7fea3a36b740 10 client.15960 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.047+0000 7fea3a36b740 1 -- 172.21.15.46:0/323831769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562514f1d7f0 con 0x562514acc4d0 2024-05-09T04:53:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.048+0000 7fea14ff9640 1 -- 172.21.15.46:0/323831769 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea24017040 con 0x562514acc4d0 2024-05-09T04:53:30.050 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.048+0000 7fea14ff9640 1 -- 172.21.15.46:0/323831769 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fea24011070 con 0x562514acc4d0 2024-05-09T04:53:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.048+0000 7fea3a36b740 10 client.15960 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.048+0000 7fea3a36b740 10 client.15960 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.048+0000 7fea3a36b740 20 client.15960 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.048+0000 7fea3a36b740 1 --2- 172.21.15.46:0/323831769 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562514aed1b0 0x562514f1fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.048+0000 7fea3a36b740 4 client.15960 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T04:53:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.048+0000 7fea3a36b740 1 -- 172.21.15.46:0/323831769 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562514ea7e80 con 0x562514aed1b0 2024-05-09T04:53:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.050+0000 7fea31254640 1 --2- 172.21.15.46:0/323831769 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562514aed1b0 0x562514f1fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.056+0000 7fea14ff9640 10 client.15960 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.056+0000 7fea31254640 1 --2- 172.21.15.46:0/323831769 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562514aed1b0 0x562514f1fd00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b41230addc2de232 server_cookie=21e2b17acc59a2f7 in_seq=0 out_seq=0 2024-05-09T04:53:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.128+0000 7fea14ff9640 1 -- 172.21.15.46:0/323831769 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x562514ea7e80 con 0x562514aed1b0 2024-05-09T04:53:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.128+0000 7fea14ff9640 10 client.15960 handle_command_reply: tid=0 2024-05-09T04:53:30.129 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.128+0000 7fea3a36b740 1 client.15960 shutdown 2024-05-09T04:53:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.128+0000 7fea3a36b740 1 -- 172.21.15.46:0/323831769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 msgr2=0x562514ea0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:30.130 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.128+0000 7fea3a36b740 1 --2- 172.21.15.46:0/323831769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514ea0870 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fea2400b460 tx=0x7fea24012cf0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.129+0000 7fea3a36b740 1 -- 172.21.15.46:0/323831769 shutdown_connections 2024-05-09T04:53:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.129+0000 7fea3a36b740 1 --2- 172.21.15.46:0/323831769 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562514aed1b0 0x562514f1fd00 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.131 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.129+0000 7fea3a36b740 1 --2- 172.21.15.46:0/323831769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562514ac42f0 0x562514ea0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.129+0000 7fea3a36b740 1 --2- 172.21.15.46:0/323831769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562514acc4d0 0x562514ea0870 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.129+0000 7fea14ff9640 0 client.15960 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.129+0000 7fea3a36b740 1 --2- 172.21.15.46:0/323831769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562514aabd80 0x562514ea2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.129+0000 7fea3a36b740 1 -- 172.21.15.46:0/323831769 >> 172.21.15.46:0/323831769 conn(0x562514ae1300 msgr2=0x562514e8c7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.129+0000 7fea3a36b740 1 -- 172.21.15.46:0/323831769 shutdown_connections 2024-05-09T04:53:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.130+0000 7fea3a36b740 1 -- 172.21.15.46:0/323831769 wait complete. 2024-05-09T04:53:30.132 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.130+0000 7fea3a36b740 20 client.15960 trim_cache size 0 max 16384 2024-05-09T04:53:30.133 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-05-09T04:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-05-09T04:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-05-09T04:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.131+0000 7fea334b8640 1 -- 172.21.15.46:0/1635915107 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fea180b2aa0 msgr2=0x7fea180b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.131+0000 7fea334b8640 1 --2- 172.21.15.46:0/1635915107 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fea180b2aa0 0x7fea180b4fb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fea2c10e9a0 tx=0x7fea1c005c10 comp rx=0 tx=0).stop 2024-05-09T04:53:30.134 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.131+0000 7fea334b8640 1 -- 172.21.15.46:0/1635915107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea2c103300 msgr2=0x7fea2c199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.131+0000 7fea334b8640 1 --2- 172.21.15.46:0/1635915107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea2c103300 0x7fea2c199560 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fea28031440 tx=0x7fea28003d60 comp rx=0 tx=0).stop 2024-05-09T04:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.132+0000 7fea334b8640 1 -- 172.21.15.46:0/1635915107 shutdown_connections 2024-05-09T04:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.132+0000 7fea334b8640 1 --2- 172.21.15.46:0/1635915107 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fea180b2aa0 0x7fea180b4fb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.132+0000 7fea334b8640 1 --2- 172.21.15.46:0/1635915107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fea2c108bb0 0x7fea2c199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.132+0000 7fea334b8640 1 --2- 172.21.15.46:0/1635915107 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fea2c109050 0x7fea2c199fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.132+0000 7fea334b8640 1 --2- 172.21.15.46:0/1635915107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fea2c103300 0x7fea2c199560 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.132+0000 7fea334b8640 1 -- 172.21.15.46:0/1635915107 >> 172.21.15.46:0/1635915107 conn(0x7fea2c078280 msgr2=0x7fea2c104250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:30.135 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.132+0000 7fea334b8640 1 -- 172.21.15.46:0/1635915107 shutdown_connections 2024-05-09T04:53:30.136 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:30.132+0000 7fea334b8640 1 -- 172.21.15.46:0/1635915107 wait complete. 2024-05-09T04:53:31.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:30 smithi110 ceph-mon[34904]: pgmap v75: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-05-09T04:53:31.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:30 smithi046 ceph-mon[30136]: pgmap v75: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-05-09T04:53:31.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:30 smithi072 ceph-mon[34088]: pgmap v75: 97 pgs: 97 active+clean; 2.0 GiB data, 745 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.0 KiB/s wr, 8 op/s 2024-05-09T04:53:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.649+0000 7ff84a50f700 1 -- 172.21.15.46:0/4165527261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8441034e0 msgr2=0x7ff8441038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.649+0000 7ff84a50f700 1 --2- 172.21.15.46:0/4165527261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8441034e0 0x7ff8441038f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff83400b0d0 tx=0x7ff83400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.649+0000 7ff84a50f700 1 -- 172.21.15.46:0/4165527261 shutdown_connections 2024-05-09T04:53:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.649+0000 7ff84a50f700 1 --2- 172.21.15.46:0/4165527261 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff844108760 0x7ff8441100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.649+0000 7ff84a50f700 1 --2- 172.21.15.46:0/4165527261 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8441034e0 0x7ff8441038f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.649+0000 7ff84a50f700 1 --2- 172.21.15.46:0/4165527261 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff844107e10 0x7ff844108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.649+0000 7ff84a50f700 1 -- 172.21.15.46:0/4165527261 >> 172.21.15.46:0/4165527261 conn(0x7ff8440fef30 msgr2=0x7ff844101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.649+0000 7ff84a50f700 1 -- 172.21.15.46:0/4165527261 shutdown_connections 2024-05-09T04:53:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.649+0000 7ff84a50f700 1 -- 172.21.15.46:0/4165527261 wait complete. 2024-05-09T04:53:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.650+0000 7ff84a50f700 1 Processor -- start 2024-05-09T04:53:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.650+0000 7ff84a50f700 1 -- start start 2024-05-09T04:53:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.651+0000 7ff84a50f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8441034e0 0x7ff8441a1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.651+0000 7ff84a50f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff844107e10 0x7ff8441a1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.651+0000 7ff84a50f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff844108760 0x7ff8441a5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.651+0000 7ff84a50f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff844112390 con 0x7ff844108760 2024-05-09T04:53:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.651+0000 7ff84a50f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff844112210 con 0x7ff844107e10 2024-05-09T04:53:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.651+0000 7ff84a50f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff844112510 con 0x7ff8441034e0 2024-05-09T04:53:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.651+0000 7ff848aac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff844108760 0x7ff8441a5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.652+0000 7ff848aac700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff844108760 0x7ff8441a5dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35942/0 (socket says 172.21.15.46:35942) 2024-05-09T04:53:32.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.652+0000 7ff848aac700 1 -- 172.21.15.46:0/1749862304 learned_addr learned my addr 172.21.15.46:0/1749862304 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.652+0000 7ff848aac700 1 -- 172.21.15.46:0/1749862304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8441034e0 msgr2=0x7ff8441a1590 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T04:53:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.652+0000 7ff848aac700 1 --2- 172.21.15.46:0/1749862304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8441034e0 0x7ff8441a1590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.652+0000 7ff848aac700 1 -- 172.21.15.46:0/1749862304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff844107e10 msgr2=0x7ff8441a1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.652+0000 7ff848aac700 1 --2- 172.21.15.46:0/1749862304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff844107e10 0x7ff8441a1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.652+0000 7ff848aac700 1 -- 172.21.15.46:0/1749862304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8441a6460 con 0x7ff844108760 2024-05-09T04:53:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.653+0000 7ff848aac700 1 --2- 172.21.15.46:0/1749862304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff844108760 0x7ff8441a5dc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00b4a0 tx=0x7ff83c00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.653+0000 7ff841ffb700 1 -- 172.21.15.46:0/1749862304 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c015020 con 0x7ff844108760 2024-05-09T04:53:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.653+0000 7ff84a50f700 1 -- 172.21.15.46:0/1749862304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8441a6750 con 0x7ff844108760 2024-05-09T04:53:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.653+0000 7ff841ffb700 1 -- 172.21.15.46:0/1749862304 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff83c011ce0 con 0x7ff844108760 2024-05-09T04:53:32.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.653+0000 7ff841ffb700 1 -- 172.21.15.46:0/1749862304 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c010cb0 con 0x7ff844108760 2024-05-09T04:53:32.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.654+0000 7ff84a50f700 1 -- 172.21.15.46:0/1749862304 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8441ae070 con 0x7ff844108760 2024-05-09T04:53:32.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.659+0000 7ff841ffb700 1 -- 172.21.15.46:0/1749862304 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff83c018020 con 0x7ff844108760 2024-05-09T04:53:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.660+0000 7ff858f25b80 1 Processor -- start 2024-05-09T04:53:32.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.660+0000 7ff858f25b80 1 -- start start 2024-05-09T04:53:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.661+0000 7ff858f25b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700db1160 0x55e700e4a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.661+0000 7ff858f25b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700cc74a0 0x55e700e4aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.661+0000 7ff841ffb700 1 --2- 172.21.15.46:0/1749862304 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff8300b2a60 0x7ff8300b4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.661+0000 7ff858f25b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700e446a0 0x55e700e44ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.661+0000 7ff858f25b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e700d66ff0 con 0x55e700e446a0 2024-05-09T04:53:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.661+0000 7ff858f25b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e700d057f0 con 0x55e700cc74a0 2024-05-09T04:53:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.661+0000 7ff858f25b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e700d62fb0 con 0x55e700db1160 2024-05-09T04:53:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.661+0000 7ff83b7fe700 1 --2- 172.21.15.46:0/1749862304 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff8300b2a60 0x7ff8300b4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:32.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.661+0000 7ff841ffb700 1 -- 172.21.15.46:0/1749862304 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7ff83c0e1d90 con 0x7ff844108760 2024-05-09T04:53:32.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.661+0000 7ff843fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700cc74a0 0x55e700e4aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:32.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.661+0000 7ff843fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700cc74a0 0x55e700e4aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:55006/0 (socket says 172.21.15.46:55006) 2024-05-09T04:53:32.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.661+0000 7ff843fff700 1 -- 172.21.15.46:0/753896181 learned_addr learned my addr 172.21.15.46:0/753896181 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:32.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.661+0000 7ff848aac700 1 --2- 172.21.15.46:0/753896181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700e446a0 0x55e700e44ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:32.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.662+0000 7ff848aac700 1 -- 172.21.15.46:0/753896181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700db1160 msgr2=0x55e700e4a4f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:53:32.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.662+0000 7ff848aac700 1 --2- 172.21.15.46:0/753896181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700db1160 0x55e700e4a4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.662+0000 7ff848aac700 1 -- 172.21.15.46:0/753896181 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700cc74a0 msgr2=0x55e700e4aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:32.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.662+0000 7ff848aac700 1 --2- 172.21.15.46:0/753896181 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700cc74a0 0x55e700e4aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.662+0000 7ff848aac700 1 -- 172.21.15.46:0/753896181 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e700cfd370 con 0x55e700e446a0 2024-05-09T04:53:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.662+0000 7ff83b7fe700 1 --2- 172.21.15.46:0/1749862304 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff8300b2a60 0x7ff8300b4f70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff844106550 tx=0x7ff82c007b70 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.662+0000 7ff848aac700 1 --2- 172.21.15.46:0/753896181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700e446a0 0x55e700e44ad0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0df570 tx=0x7ff83c010ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a8c18151cd2b612 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.662+0000 7ff83a7fc700 1 -- 172.21.15.46:0/753896181 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c0a9100 con 0x55e700e446a0 2024-05-09T04:53:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.663+0000 7ff83a7fc700 1 -- 172.21.15.46:0/753896181 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff83c0ad4b0 con 0x55e700e446a0 2024-05-09T04:53:32.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.663+0000 7ff83a7fc700 1 -- 172.21.15.46:0/753896181 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c0a95c0 con 0x55e700e446a0 2024-05-09T04:53:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.664+0000 7ff858f25b80 1 -- 172.21.15.46:0/753896181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700e446a0 msgr2=0x55e700e44ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.664+0000 7ff858f25b80 1 --2- 172.21.15.46:0/753896181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700e446a0 0x55e700e44ad0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7ff83c0df570 tx=0x7ff83c010ed0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.664+0000 7ff858f25b80 1 -- 172.21.15.46:0/753896181 shutdown_connections 2024-05-09T04:53:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.664+0000 7ff858f25b80 1 --2- 172.21.15.46:0/753896181 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700e446a0 0x55e700e44ad0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.664+0000 7ff858f25b80 1 --2- 172.21.15.46:0/753896181 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700db1160 0x55e700e4a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.664+0000 7ff858f25b80 1 --2- 172.21.15.46:0/753896181 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700cc74a0 0x55e700e4aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.664+0000 7ff858f25b80 1 -- 172.21.15.46:0/753896181 >> 172.21.15.46:0/753896181 conn(0x55e700d276a0 msgr2=0x55e700e3f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.664+0000 7ff858f25b80 1 -- 172.21.15.46:0/753896181 shutdown_connections 2024-05-09T04:53:32.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.665+0000 7ff858f25b80 1 -- 172.21.15.46:0/753896181 wait complete. 2024-05-09T04:53:32.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.665+0000 7ff858f25b80 1 -- start start 2024-05-09T04:53:32.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.666+0000 7ff858f25b80 1 Processor -- start 2024-05-09T04:53:32.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.666+0000 7ff858f25b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700cc74a0 0x55e700e4e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.666+0000 7ff858f25b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700db1160 0x55e700e4e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.666+0000 7ff858f25b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700e446a0 0x55e700e4ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.666+0000 7ff858f25b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e700be2ba0 con 0x55e700cc74a0 2024-05-09T04:53:32.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.666+0000 7ff858f25b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e700d21150 con 0x55e700e446a0 2024-05-09T04:53:32.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.666+0000 7ff858f25b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e700e511d0 con 0x55e700db1160 2024-05-09T04:53:32.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.666+0000 7ff858f25b80 10 client.0 _ll_register_callbacks cb 0x55e700bd86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:32.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.666+0000 7ff843fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700cc74a0 0x55e700e4e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:32.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.666+0000 7ff843fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700cc74a0 0x55e700e4e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35952/0 (socket says 172.21.15.46:35952) 2024-05-09T04:53:32.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.666+0000 7ff843fff700 1 -- 172.21.15.46:0/2507478309 learned_addr learned my addr 172.21.15.46:0/2507478309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:32.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.667+0000 7ff843fff700 1 -- 172.21.15.46:0/2507478309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700db1160 msgr2=0x55e700e4e750 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:53:32.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.667+0000 7ff843fff700 1 --2- 172.21.15.46:0/2507478309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700db1160 0x55e700e4e750 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.667+0000 7ff843fff700 1 -- 172.21.15.46:0/2507478309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700e446a0 msgr2=0x55e700e4ec90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:53:32.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.667+0000 7ff843fff700 1 --2- 172.21.15.46:0/2507478309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700e446a0 0x55e700e4ec90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.667+0000 7ff843fff700 1 -- 172.21.15.46:0/2507478309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e700be2490 con 0x55e700cc74a0 2024-05-09T04:53:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.667+0000 7ff843fff700 1 --2- 172.21.15.46:0/2507478309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700cc74a0 0x55e700e4e210 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff834004a10 tx=0x7ff834004af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28cbbaab0b6a2393 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.667+0000 7ff8237fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.667+0000 7ff8237fe700 1 -- 172.21.15.46:0/2507478309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff834020030 con 0x55e700cc74a0 2024-05-09T04:53:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.668+0000 7ff8237fe700 1 -- 172.21.15.46:0/2507478309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7ff8340075c0 con 0x55e700cc74a0 2024-05-09T04:53:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.668+0000 7ff858f25b80 1 -- 172.21.15.46:0/2507478309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff83c0ad4b0 con 0x55e700cc74a0 2024-05-09T04:53:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.668+0000 7ff8237fe700 1 -- 172.21.15.46:0/2507478309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff8340077c0 con 0x55e700cc74a0 2024-05-09T04:53:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.668+0000 7ff858f25b80 10 client.15972 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.668+0000 7ff858f25b80 1 -- 172.21.15.46:0/2507478309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e700e5ba20 con 0x55e700cc74a0 2024-05-09T04:53:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.668+0000 7ff8237fe700 1 -- 172.21.15.46:0/2507478309 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff83c0ad4b0 con 0x55e700cc74a0 2024-05-09T04:53:32.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.669+0000 7ff8237fe700 1 -- 172.21.15.46:0/2507478309 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff834011070 con 0x55e700cc74a0 2024-05-09T04:53:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.669+0000 7ff858f25b80 10 client.15972 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.669+0000 7ff858f25b80 10 client.15972 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.669+0000 7ff858f25b80 20 client.15972 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.669+0000 7ff858f25b80 1 --2- 172.21.15.46:0/2507478309 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e700e5e000 0x55e700e604d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.669+0000 7ff858f25b80 4 client.15972 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.669+0000 7ff858f25b80 1 -- 172.21.15.46:0/2507478309 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e700d749d0 con 0x55e700e5e000 2024-05-09T04:53:32.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.672+0000 7ff83b7fe700 1 --2- 172.21.15.46:0/2507478309 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e700e5e000 0x55e700e604d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.684+0000 7ff8237fe700 10 client.15972 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:32.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.684+0000 7ff83b7fe700 1 --2- 172.21.15.46:0/2507478309 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e700e5e000 0x55e700e604d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=329133a16b65485a server_cookie=914a90a3a793da56 in_seq=0 out_seq=0 2024-05-09T04:53:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.689+0000 7ff8237fe700 1 -- 172.21.15.46:0/2507478309 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e700d749d0 con 0x55e700e5e000 2024-05-09T04:53:32.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.689+0000 7ff8237fe700 10 client.15972 handle_command_reply: tid=0 2024-05-09T04:53:32.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.689+0000 7ff858f25b80 1 client.15972 shutdown 2024-05-09T04:53:32.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.690+0000 7ff858f25b80 1 -- 172.21.15.46:0/2507478309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700cc74a0 msgr2=0x55e700e4e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.690+0000 7ff858f25b80 1 --2- 172.21.15.46:0/2507478309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700cc74a0 0x55e700e4e210 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7ff834004a10 tx=0x7ff834004af0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.691+0000 7ff858f25b80 1 -- 172.21.15.46:0/2507478309 shutdown_connections 2024-05-09T04:53:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.691+0000 7ff858f25b80 1 --2- 172.21.15.46:0/2507478309 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e700e5e000 0x55e700e604d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.691+0000 7ff858f25b80 1 --2- 172.21.15.46:0/2507478309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700e446a0 0x55e700e4ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.691+0000 7ff858f25b80 1 --2- 172.21.15.46:0/2507478309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700cc74a0 0x55e700e4e210 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.691+0000 7ff858f25b80 1 --2- 172.21.15.46:0/2507478309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700db1160 0x55e700e4e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.691+0000 7ff858f25b80 1 -- 172.21.15.46:0/2507478309 >> 172.21.15.46:0/2507478309 conn(0x55e700d276a0 msgr2=0x55e700e3f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:32.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.691+0000 7ff8237fe700 0 client.15972 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:32.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.691+0000 7ff858f25b80 1 -- 172.21.15.46:0/2507478309 shutdown_connections 2024-05-09T04:53:32.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.691+0000 7ff858f25b80 1 -- 172.21.15.46:0/2507478309 wait complete. 2024-05-09T04:53:32.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.692+0000 7ff858f25b80 20 client.15972 trim_cache size 0 max 16384 2024-05-09T04:53:32.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.719+0000 7ff858f25b80 1 Processor -- start 2024-05-09T04:53:32.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.719+0000 7ff858f25b80 1 -- start start 2024-05-09T04:53:32.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.720+0000 7ff858f25b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700cc74a0 0x55e700e440c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.720+0000 7ff858f25b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700db1160 0x55e700e49990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.720+0000 7ff858f25b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700e446a0 0x55e700e4ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.720+0000 7ff858f25b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff8441b7e90 con 0x55e700db1160 2024-05-09T04:53:32.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.720+0000 7ff858f25b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff8441b5720 con 0x55e700e446a0 2024-05-09T04:53:32.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.720+0000 7ff858f25b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff8441b3570 con 0x55e700cc74a0 2024-05-09T04:53:32.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.720+0000 7ff83b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700db1160 0x55e700e49990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:32.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.720+0000 7ff83b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700db1160 0x55e700e49990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35966/0 (socket says 172.21.15.46:35966) 2024-05-09T04:53:32.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.720+0000 7ff83b7fe700 1 -- 172.21.15.46:0/2796423409 learned_addr learned my addr 172.21.15.46:0/2796423409 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:32.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.721+0000 7ff848aac700 1 --2- 172.21.15.46:0/2796423409 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700e446a0 0x55e700e4ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:32.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.721+0000 7ff843fff700 1 --2- 172.21.15.46:0/2796423409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700cc74a0 0x55e700e440c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:32.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.721+0000 7ff83b7fe700 1 -- 172.21.15.46:0/2796423409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700cc74a0 msgr2=0x55e700e440c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:32.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.721+0000 7ff83b7fe700 1 --2- 172.21.15.46:0/2796423409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700cc74a0 0x55e700e440c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.721+0000 7ff83b7fe700 1 -- 172.21.15.46:0/2796423409 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700e446a0 msgr2=0x55e700e4ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:32.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.721+0000 7ff83b7fe700 1 --2- 172.21.15.46:0/2796423409 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700e446a0 0x55e700e4ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.721+0000 7ff83b7fe700 1 -- 172.21.15.46:0/2796423409 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e700e4a050 con 0x55e700db1160 2024-05-09T04:53:32.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.721+0000 7ff83b7fe700 1 --2- 172.21.15.46:0/2796423409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700db1160 0x55e700e49990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x55e700e40580 tx=0x7ff82c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4a3076d841a75e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:32.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.722+0000 7ff839ffb700 1 -- 172.21.15.46:0/2796423409 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0169e0 con 0x55e700db1160 2024-05-09T04:53:32.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.722+0000 7ff839ffb700 1 -- 172.21.15.46:0/2796423409 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff82c0207f0 con 0x55e700db1160 2024-05-09T04:53:32.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.722+0000 7ff839ffb700 1 -- 172.21.15.46:0/2796423409 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c0072a0 con 0x55e700db1160 2024-05-09T04:53:32.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.723+0000 7ff858f25b80 1 -- 172.21.15.46:0/2796423409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700db1160 msgr2=0x55e700e49990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.723+0000 7ff858f25b80 1 --2- 172.21.15.46:0/2796423409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700db1160 0x55e700e49990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x55e700e40580 tx=0x7ff82c00f000 comp rx=0 tx=0).stop 2024-05-09T04:53:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.723+0000 7ff858f25b80 1 -- 172.21.15.46:0/2796423409 shutdown_connections 2024-05-09T04:53:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.723+0000 7ff858f25b80 1 --2- 172.21.15.46:0/2796423409 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700e446a0 0x55e700e4ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.723+0000 7ff858f25b80 1 --2- 172.21.15.46:0/2796423409 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700cc74a0 0x55e700e440c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.723+0000 7ff858f25b80 1 --2- 172.21.15.46:0/2796423409 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700db1160 0x55e700e49990 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.723+0000 7ff858f25b80 1 -- 172.21.15.46:0/2796423409 >> 172.21.15.46:0/2796423409 conn(0x55e700d276a0 msgr2=0x55e700e47e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.723+0000 7ff858f25b80 1 -- 172.21.15.46:0/2796423409 shutdown_connections 2024-05-09T04:53:32.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.723+0000 7ff858f25b80 1 -- 172.21.15.46:0/2796423409 wait complete. 2024-05-09T04:53:32.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.724+0000 7ff858f25b80 1 -- start start 2024-05-09T04:53:32.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.724+0000 7ff858f25b80 1 Processor -- start 2024-05-09T04:53:32.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.725+0000 7ff858f25b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700cc74a0 0x55e700e568a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.725+0000 7ff858f25b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700db1160 0x55e700e65260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.725+0000 7ff858f25b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700e446a0 0x55e700e657a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.725+0000 7ff858f25b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff8441b3be0 con 0x55e700db1160 2024-05-09T04:53:32.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.725+0000 7ff858f25b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff8441b5050 con 0x55e700e446a0 2024-05-09T04:53:32.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.725+0000 7ff858f25b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff8441b42b0 con 0x55e700cc74a0 2024-05-09T04:53:32.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.725+0000 7ff858f25b80 10 client.0 _ll_register_callbacks cb 0x55e700bd86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:32.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.725+0000 7ff83b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700db1160 0x55e700e65260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:32.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.725+0000 7ff83b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700db1160 0x55e700e65260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35974/0 (socket says 172.21.15.46:35974) 2024-05-09T04:53:32.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.725+0000 7ff83b7fe700 1 -- 172.21.15.46:0/1325208615 learned_addr learned my addr 172.21.15.46:0/1325208615 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:32.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.725+0000 7ff848aac700 1 --2- 172.21.15.46:0/1325208615 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700e446a0 0x55e700e657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:32.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.726+0000 7ff83b7fe700 1 -- 172.21.15.46:0/1325208615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700cc74a0 msgr2=0x55e700e568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:32.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.726+0000 7ff83b7fe700 1 --2- 172.21.15.46:0/1325208615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700cc74a0 0x55e700e568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.726+0000 7ff83b7fe700 1 -- 172.21.15.46:0/1325208615 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700e446a0 msgr2=0x55e700e657a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:32.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.726+0000 7ff83b7fe700 1 --2- 172.21.15.46:0/1325208615 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700e446a0 0x55e700e657a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.726+0000 7ff83b7fe700 1 -- 172.21.15.46:0/1325208615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e700e65e50 con 0x55e700db1160 2024-05-09T04:53:32.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.727+0000 7ff83b7fe700 1 --2- 172.21.15.46:0/1325208615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700db1160 0x55e700e65260 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff82c016650 tx=0x7ff82c020570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944a2e0f5305a2a2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:32.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.727+0000 7ff8237fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:32.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.727+0000 7ff8237fe700 1 -- 172.21.15.46:0/1325208615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff82c006fe0 con 0x55e700db1160 2024-05-09T04:53:32.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.728+0000 7ff858f25b80 1 -- 172.21.15.46:0/1325208615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff82c0207f0 con 0x55e700db1160 2024-05-09T04:53:32.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.729+0000 7ff8237fe700 1 -- 172.21.15.46:0/1325208615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7ff82c007180 con 0x55e700db1160 2024-05-09T04:53:32.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.729+0000 7ff8237fe700 1 -- 172.21.15.46:0/1325208615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff82c0073c0 con 0x55e700db1160 2024-05-09T04:53:32.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.729+0000 7ff858f25b80 10 client.15978 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:32.732 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.729+0000 7ff858f25b80 1 -- 172.21.15.46:0/1325208615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e700ec8620 con 0x55e700db1160 2024-05-09T04:53:32.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.730+0000 7ff8237fe700 1 -- 172.21.15.46:0/1325208615 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff82c014420 con 0x55e700db1160 2024-05-09T04:53:32.733 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.730+0000 7ff8237fe700 1 -- 172.21.15.46:0/1325208615 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff82c030070 con 0x55e700db1160 2024-05-09T04:53:32.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.730+0000 7ff858f25b80 10 client.15978 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:32.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.730+0000 7ff858f25b80 10 client.15978 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:32.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.730+0000 7ff858f25b80 20 client.15978 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:32.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.730+0000 7ff858f25b80 1 --2- 172.21.15.46:0/1325208615 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e700ec8ce0 0x55e700ecb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:32.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.730+0000 7ff858f25b80 4 client.15978 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:53:32.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.730+0000 7ff858f25b80 1 -- 172.21.15.46:0/1325208615 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7ff82c0207f0 con 0x55e700ec8ce0 2024-05-09T04:53:32.734 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.731+0000 7ff843fff700 1 --2- 172.21.15.46:0/1325208615 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e700ec8ce0 0x55e700ecb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:32.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.733+0000 7ff843fff700 1 --2- 172.21.15.46:0/1325208615 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e700ec8ce0 0x55e700ecb100 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85e42224716ce0ef server_cookie=b68e3ad50d3ac904 in_seq=0 out_seq=0 2024-05-09T04:53:32.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.733+0000 7ff8237fe700 10 client.15978 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:32.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.733+0000 7ff8237fe700 1 -- 172.21.15.46:0/1325208615 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7ff82c0207f0 con 0x55e700ec8ce0 2024-05-09T04:53:32.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.733+0000 7ff8237fe700 10 client.15978 handle_command_reply: tid=0 2024-05-09T04:53:32.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.734+0000 7ff858f25b80 1 client.15978 shutdown 2024-05-09T04:53:32.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.734+0000 7ff858f25b80 1 -- 172.21.15.46:0/1325208615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700db1160 msgr2=0x55e700e65260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:32.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.734+0000 7ff858f25b80 1 --2- 172.21.15.46:0/1325208615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700db1160 0x55e700e65260 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7ff82c016650 tx=0x7ff82c020570 comp rx=0 tx=0).stop 2024-05-09T04:53:32.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.734+0000 7ff858f25b80 1 -- 172.21.15.46:0/1325208615 shutdown_connections 2024-05-09T04:53:32.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.734+0000 7ff858f25b80 1 --2- 172.21.15.46:0/1325208615 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e700ec8ce0 0x55e700ecb100 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.734+0000 7ff858f25b80 1 --2- 172.21.15.46:0/1325208615 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e700e446a0 0x55e700e657a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.734+0000 7ff858f25b80 1 --2- 172.21.15.46:0/1325208615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e700cc74a0 0x55e700e568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.734+0000 7ff858f25b80 1 --2- 172.21.15.46:0/1325208615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e700db1160 0x55e700e65260 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.734+0000 7ff858f25b80 1 -- 172.21.15.46:0/1325208615 >> 172.21.15.46:0/1325208615 conn(0x55e700d276a0 msgr2=0x55e700e43e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:32.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.734+0000 7ff8237fe700 0 client.15978 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:32.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.734+0000 7ff858f25b80 1 -- 172.21.15.46:0/1325208615 shutdown_connections 2024-05-09T04:53:32.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.734+0000 7ff858f25b80 1 -- 172.21.15.46:0/1325208615 wait complete. 2024-05-09T04:53:32.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.734+0000 7ff858f25b80 20 client.15978 trim_cache size 0 max 16384 2024-05-09T04:53:32.738 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:53:32.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.735+0000 7ff84a50f700 1 -- 172.21.15.46:0/1749862304 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff8300b2a60 msgr2=0x7ff8300b4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:32.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.736+0000 7ff84a50f700 1 --2- 172.21.15.46:0/1749862304 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff8300b2a60 0x7ff8300b4f70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff844106550 tx=0x7ff82c007b70 comp rx=0 tx=0).stop 2024-05-09T04:53:32.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.736+0000 7ff84a50f700 1 -- 172.21.15.46:0/1749862304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff844108760 msgr2=0x7ff8441a5dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:32.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.736+0000 7ff84a50f700 1 --2- 172.21.15.46:0/1749862304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff844108760 0x7ff8441a5dc0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00b4a0 tx=0x7ff83c00b860 comp rx=0 tx=0).stop 2024-05-09T04:53:32.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.736+0000 7ff84a50f700 1 -- 172.21.15.46:0/1749862304 shutdown_connections 2024-05-09T04:53:32.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.736+0000 7ff84a50f700 1 --2- 172.21.15.46:0/1749862304 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff8300b2a60 0x7ff8300b4f70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.737+0000 7ff84a50f700 1 --2- 172.21.15.46:0/1749862304 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff844108760 0x7ff8441a5dc0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.737+0000 7ff84a50f700 1 --2- 172.21.15.46:0/1749862304 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8441034e0 0x7ff8441a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.737+0000 7ff84a50f700 1 --2- 172.21.15.46:0/1749862304 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff844107e10 0x7ff8441a1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:32.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.737+0000 7ff84a50f700 1 -- 172.21.15.46:0/1749862304 >> 172.21.15.46:0/1749862304 conn(0x7ff8440fef30 msgr2=0x7ff844101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:32.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.737+0000 7ff84a50f700 1 -- 172.21.15.46:0/1749862304 shutdown_connections 2024-05-09T04:53:32.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:32.737+0000 7ff84a50f700 1 -- 172.21.15.46:0/1749862304 wait complete. 2024-05-09T04:53:33.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:32 smithi110 ceph-mon[34904]: pgmap v76: 97 pgs: 97 active+clean; 4.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 241 MiB/s wr, 201 op/s 2024-05-09T04:53:33.082 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:32 smithi046 ceph-mon[30136]: pgmap v76: 97 pgs: 97 active+clean; 4.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 241 MiB/s wr, 201 op/s 2024-05-09T04:53:33.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:32 smithi072 ceph-mon[34088]: pgmap v76: 97 pgs: 97 active+clean; 4.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 304 KiB/s rd, 241 MiB/s wr, 201 op/s 2024-05-09T04:53:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:34 smithi110 ceph-mon[34904]: pgmap v77: 97 pgs: 97 active+clean; 4.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 241 MiB/s wr, 194 op/s 2024-05-09T04:53:35.068 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:53:35.068 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:53:35.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:34 smithi046 ceph-mon[30136]: pgmap v77: 97 pgs: 97 active+clean; 4.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 241 MiB/s wr, 194 op/s 2024-05-09T04:53:35.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:34 smithi072 ceph-mon[34088]: pgmap v77: 97 pgs: 97 active+clean; 4.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 241 MiB/s wr, 194 op/s 2024-05-09T04:53:35.907 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:37.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:36 smithi110 ceph-mon[34904]: pgmap v78: 97 pgs: 97 active+clean; 4.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 241 MiB/s wr, 194 op/s 2024-05-09T04:53:37.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:36 smithi046 ceph-mon[30136]: pgmap v78: 97 pgs: 97 active+clean; 4.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 241 MiB/s wr, 194 op/s 2024-05-09T04:53:37.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:36 smithi072 ceph-mon[34088]: pgmap v78: 97 pgs: 97 active+clean; 4.8 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 241 MiB/s wr, 194 op/s 2024-05-09T04:53:37.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.939+0000 7f815bee5700 1 -- 172.21.15.46:0/2745072466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8154107e10 msgr2=0x7f8154108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:37.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.939+0000 7f815bee5700 1 --2- 172.21.15.46:0/2745072466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8154107e10 0x7f8154108220 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f81440099c0 tx=0x7f8144009cd0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.939+0000 7f815bee5700 1 -- 172.21.15.46:0/2745072466 shutdown_connections 2024-05-09T04:53:37.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.939+0000 7f815bee5700 1 --2- 172.21.15.46:0/2745072466 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8154108760 0x7f8154110150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.939+0000 7f815bee5700 1 --2- 172.21.15.46:0/2745072466 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8154103ca0 0x7f81541040b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.939+0000 7f815bee5700 1 --2- 172.21.15.46:0/2745072466 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8154107e10 0x7f8154108220 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.939+0000 7f815bee5700 1 -- 172.21.15.46:0/2745072466 >> 172.21.15.46:0/2745072466 conn(0x7f81540fe6f0 msgr2=0x7f8154100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:37.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.940+0000 7f815bee5700 1 -- 172.21.15.46:0/2745072466 shutdown_connections 2024-05-09T04:53:37.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.940+0000 7f815bee5700 1 -- 172.21.15.46:0/2745072466 wait complete. 2024-05-09T04:53:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.941+0000 7f815bee5700 1 Processor -- start 2024-05-09T04:53:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.941+0000 7f815bee5700 1 -- start start 2024-05-09T04:53:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.941+0000 7f815bee5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8154103ca0 0x7f815419d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.941+0000 7f815bee5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8154107e10 0x7f815419d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.941+0000 7f815bee5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8154108760 0x7f81541a19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.941+0000 7f815bee5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8154114570 con 0x7f8154103ca0 2024-05-09T04:53:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.941+0000 7f815bee5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f81541143f0 con 0x7f8154108760 2024-05-09T04:53:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.941+0000 7f815bee5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f81541146f0 con 0x7f8154107e10 2024-05-09T04:53:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.941+0000 7f8159c81700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8154103ca0 0x7f815419d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.941+0000 7f8159c81700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8154103ca0 0x7f815419d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46708/0 (socket says 172.21.15.46:46708) 2024-05-09T04:53:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.941+0000 7f8159c81700 1 -- 172.21.15.46:0/1785433236 learned_addr learned my addr 172.21.15.46:0/1785433236 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.942+0000 7f815a482700 1 --2- 172.21.15.46:0/1785433236 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8154108760 0x7f81541a19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.942+0000 7f8159c81700 1 -- 172.21.15.46:0/1785433236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8154107e10 msgr2=0x7f815419d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:37.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.942+0000 7f8159c81700 1 --2- 172.21.15.46:0/1785433236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8154107e10 0x7f815419d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.942+0000 7f8159c81700 1 -- 172.21.15.46:0/1785433236 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8154108760 msgr2=0x7f81541a19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.942+0000 7f8159c81700 1 --2- 172.21.15.46:0/1785433236 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8154108760 0x7f81541a19b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.942+0000 7f8159c81700 1 -- 172.21.15.46:0/1785433236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81541a2050 con 0x7f8154103ca0 2024-05-09T04:53:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.942+0000 7f8159c81700 1 --2- 172.21.15.46:0/1785433236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8154103ca0 0x7f815419d180 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f815000b550 tx=0x7f815000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.942+0000 7f814affd700 1 -- 172.21.15.46:0/1785433236 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150015020 con 0x7f8154103ca0 2024-05-09T04:53:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.942+0000 7f814affd700 1 -- 172.21.15.46:0/1785433236 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8150011ce0 con 0x7f8154103ca0 2024-05-09T04:53:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.942+0000 7f815bee5700 1 -- 172.21.15.46:0/1785433236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81541a2340 con 0x7f8154103ca0 2024-05-09T04:53:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.942+0000 7f814affd700 1 -- 172.21.15.46:0/1785433236 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8150010d20 con 0x7f8154103ca0 2024-05-09T04:53:37.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.942+0000 7f815bee5700 1 -- 172.21.15.46:0/1785433236 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81541a9c60 con 0x7f8154103ca0 2024-05-09T04:53:37.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.946+0000 7f814affd700 1 -- 172.21.15.46:0/1785433236 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8150018020 con 0x7f8154103ca0 2024-05-09T04:53:37.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.947+0000 7f814affd700 1 --2- 172.21.15.46:0/1785433236 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f81400b2a70 0x7f81400b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:37.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.947+0000 7f8159480700 1 --2- 172.21.15.46:0/1785433236 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f81400b2a70 0x7f81400b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:37.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.947+0000 7f816a8fbb80 1 Processor -- start 2024-05-09T04:53:37.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.948+0000 7f814affd700 1 -- 172.21.15.46:0/1785433236 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f81500e1ce0 con 0x7f8154103ca0 2024-05-09T04:53:37.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.948+0000 7f816a8fbb80 1 -- start start 2024-05-09T04:53:37.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.948+0000 7f8159480700 1 --2- 172.21.15.46:0/1785433236 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f81400b2a70 0x7f81400b4f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8144003800 tx=0x7f8144000bc0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:37.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.948+0000 7f816a8fbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d6974b3d80 0x55d6975bc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:37.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.948+0000 7f816a8fbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d697439520 0x55d6975bc800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:37.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.948+0000 7f816a8fbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d6975b6480 0x55d6975b6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:37.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.948+0000 7f816a8fbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d6974d9120 con 0x55d6975b6480 2024-05-09T04:53:37.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.948+0000 7f816a8fbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d69747ea20 con 0x55d6974b3d80 2024-05-09T04:53:37.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.948+0000 7f816a8fbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d6974d50e0 con 0x55d697439520 2024-05-09T04:53:37.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.949+0000 7f815a482700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d6975b6480 0x55d6975b6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:37.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.949+0000 7f815a482700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d6975b6480 0x55d6975b6890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46722/0 (socket says 172.21.15.46:46722) 2024-05-09T04:53:37.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.949+0000 7f815a482700 1 -- 172.21.15.46:0/878743330 learned_addr learned my addr 172.21.15.46:0/878743330 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:37.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.949+0000 7f8159c81700 1 --2- 172.21.15.46:0/878743330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d6974b3d80 0x55d6975bc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:37.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.949+0000 7f815a482700 1 -- 172.21.15.46:0/878743330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d697439520 msgr2=0x55d6975bc800 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:53:37.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.949+0000 7f815a482700 1 --2- 172.21.15.46:0/878743330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d697439520 0x55d6975bc800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.949+0000 7f815a482700 1 -- 172.21.15.46:0/878743330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d6974b3d80 msgr2=0x55d6975bc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:37.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.949+0000 7f815a482700 1 --2- 172.21.15.46:0/878743330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d6974b3d80 0x55d6975bc2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.949+0000 7f815a482700 1 -- 172.21.15.46:0/878743330 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d69746f580 con 0x55d6975b6480 2024-05-09T04:53:37.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.950+0000 7f815a482700 1 --2- 172.21.15.46:0/878743330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d6975b6480 0x55d6975b6890 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f814c009f20 tx=0x7f814c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4673c4059e22a1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:37.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.950+0000 7f813ffff700 1 -- 172.21.15.46:0/878743330 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c01a070 con 0x55d6975b6480 2024-05-09T04:53:37.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.950+0000 7f813ffff700 1 -- 172.21.15.46:0/878743330 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f814c015410 con 0x55d6975b6480 2024-05-09T04:53:37.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.950+0000 7f813ffff700 1 -- 172.21.15.46:0/878743330 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c00a660 con 0x55d6975b6480 2024-05-09T04:53:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.951+0000 7f816a8fbb80 1 -- 172.21.15.46:0/878743330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d6975b6480 msgr2=0x55d6975b6890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.951+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/878743330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d6975b6480 0x55d6975b6890 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f814c009f20 tx=0x7f814c009f50 comp rx=0 tx=0).stop 2024-05-09T04:53:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.951+0000 7f816a8fbb80 1 -- 172.21.15.46:0/878743330 shutdown_connections 2024-05-09T04:53:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.951+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/878743330 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d6975b6480 0x55d6975b6890 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.951+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/878743330 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d6974b3d80 0x55d6975bc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.951+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/878743330 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d697439520 0x55d6975bc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.951+0000 7f816a8fbb80 1 -- 172.21.15.46:0/878743330 >> 172.21.15.46:0/878743330 conn(0x55d69743cff0 msgr2=0x55d6975b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.951+0000 7f816a8fbb80 1 -- 172.21.15.46:0/878743330 shutdown_connections 2024-05-09T04:53:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.951+0000 7f816a8fbb80 1 -- 172.21.15.46:0/878743330 wait complete. 2024-05-09T04:53:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.952+0000 7f816a8fbb80 1 -- start start 2024-05-09T04:53:37.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.952+0000 7f816a8fbb80 1 Processor -- start 2024-05-09T04:53:37.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.953+0000 7f816a8fbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d697439520 0x55d6975c00a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:37.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.953+0000 7f816a8fbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d6974b3d80 0x55d6975c05e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:37.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.953+0000 7f816a8fbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d6975b6480 0x55d6975c0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:37.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.953+0000 7f816a8fbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d6975b71e0 con 0x55d6975b6480 2024-05-09T04:53:37.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.953+0000 7f816a8fbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d697473ba0 con 0x55d697439520 2024-05-09T04:53:37.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.953+0000 7f816a8fbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d6975c30a0 con 0x55d6974b3d80 2024-05-09T04:53:37.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.953+0000 7f816a8fbb80 10 client.0 _ll_register_callbacks cb 0x55d69734a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:37.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.954+0000 7f815a482700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d6975b6480 0x55d6975c0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.954+0000 7f815a482700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d6975b6480 0x55d6975c0b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46730/0 (socket says 172.21.15.46:46730) 2024-05-09T04:53:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.954+0000 7f815a482700 1 -- 172.21.15.46:0/934870899 learned_addr learned my addr 172.21.15.46:0/934870899 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.954+0000 7f815a482700 1 -- 172.21.15.46:0/934870899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d6974b3d80 msgr2=0x55d6975c05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.954+0000 7f815a482700 1 --2- 172.21.15.46:0/934870899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d6974b3d80 0x55d6975c05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.954+0000 7f815a482700 1 -- 172.21.15.46:0/934870899 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d697439520 msgr2=0x55d6975c00a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:53:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.954+0000 7f815a482700 1 --2- 172.21.15.46:0/934870899 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d697439520 0x55d6975c00a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.954+0000 7f815a482700 1 -- 172.21.15.46:0/934870899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d697354510 con 0x55d6975b6480 2024-05-09T04:53:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.954+0000 7f815a482700 1 --2- 172.21.15.46:0/934870899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d6975b6480 0x55d6975c0b20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f814c018040 tx=0x7f814c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a3dd6271a91573 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.954+0000 7f813d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.955+0000 7f813d7fa700 1 -- 172.21.15.46:0/934870899 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814c01a040 con 0x55d6975b6480 2024-05-09T04:53:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.955+0000 7f813d7fa700 1 -- 172.21.15.46:0/934870899 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f814c020450 con 0x55d6975b6480 2024-05-09T04:53:37.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.955+0000 7f816a8fbb80 1 -- 172.21.15.46:0/934870899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f814c015410 con 0x55d6975b6480 2024-05-09T04:53:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.955+0000 7f813d7fa700 1 -- 172.21.15.46:0/934870899 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f814c020650 con 0x55d6975b6480 2024-05-09T04:53:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.955+0000 7f816a8fbb80 10 client.15990 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.955+0000 7f816a8fbb80 1 -- 172.21.15.46:0/934870899 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d6975cda70 con 0x55d6975b6480 2024-05-09T04:53:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.955+0000 7f813d7fa700 1 -- 172.21.15.46:0/934870899 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f814c015410 con 0x55d6975b6480 2024-05-09T04:53:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.956+0000 7f813d7fa700 1 -- 172.21.15.46:0/934870899 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f814c00a400 con 0x55d6975b6480 2024-05-09T04:53:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.956+0000 7f816a8fbb80 10 client.15990 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.956+0000 7f816a8fbb80 10 client.15990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.956+0000 7f816a8fbb80 20 client.15990 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.956+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/934870899 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d6975cfff0 0x55d6975d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.956+0000 7f816a8fbb80 4 client.15990 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.956+0000 7f816a8fbb80 1 -- 172.21.15.46:0/934870899 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d6974e6b00 con 0x55d6975cfff0 2024-05-09T04:53:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.957+0000 7f8159c81700 1 --2- 172.21.15.46:0/934870899 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d6975cfff0 0x55d6975d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:37.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.958+0000 7f813d7fa700 10 client.15990 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:37.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.958+0000 7f8159c81700 1 --2- 172.21.15.46:0/934870899 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d6975cfff0 0x55d6975d2410 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea92bc1ce4b3cc5 server_cookie=c9294feb49d9b9c2 in_seq=0 out_seq=0 2024-05-09T04:53:37.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.959+0000 7f813d7fa700 1 -- 172.21.15.46:0/934870899 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d6974e6b00 con 0x55d6975cfff0 2024-05-09T04:53:37.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.959+0000 7f813d7fa700 10 client.15990 handle_command_reply: tid=0 2024-05-09T04:53:37.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.959+0000 7f816a8fbb80 1 client.15990 shutdown 2024-05-09T04:53:37.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.959+0000 7f816a8fbb80 1 -- 172.21.15.46:0/934870899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d6975b6480 msgr2=0x55d6975c0b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:37.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.959+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/934870899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d6975b6480 0x55d6975c0b20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f814c018040 tx=0x7f814c00cc60 comp rx=0 tx=0).stop 2024-05-09T04:53:37.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.959+0000 7f816a8fbb80 1 -- 172.21.15.46:0/934870899 shutdown_connections 2024-05-09T04:53:37.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.959+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/934870899 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d6975b6480 0x55d6975c0b20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.959+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/934870899 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d6975cfff0 0x55d6975d2410 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.959+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/934870899 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d697439520 0x55d6975c00a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.959+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/934870899 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d6974b3d80 0x55d6975c05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:37.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.960+0000 7f813d7fa700 0 client.15990 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:37.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.960+0000 7f816a8fbb80 1 -- 172.21.15.46:0/934870899 >> 172.21.15.46:0/934870899 conn(0x55d69743cff0 msgr2=0x55d6975ab070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:37.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.960+0000 7f816a8fbb80 1 -- 172.21.15.46:0/934870899 shutdown_connections 2024-05-09T04:53:37.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.960+0000 7f816a8fbb80 1 -- 172.21.15.46:0/934870899 wait complete. 2024-05-09T04:53:37.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.960+0000 7f816a8fbb80 20 client.15990 trim_cache size 0 max 16384 2024-05-09T04:53:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.996+0000 7f816a8fbb80 1 Processor -- start 2024-05-09T04:53:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.997+0000 7f816a8fbb80 1 -- start start 2024-05-09T04:53:37.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.997+0000 7f816a8fbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d69743cff0 0x55d6975abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.997+0000 7f816a8fbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697439520 0x55d6975cc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.997+0000 7f816a8fbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d6974b3d80 0x55d6975b5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.997+0000 7f8159480700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697439520 0x55d6975cc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.997+0000 7f816a8fbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81541b3c20 con 0x55d697439520 2024-05-09T04:53:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.997+0000 7f816a8fbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f81541b14b0 con 0x55d6974b3d80 2024-05-09T04:53:37.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.997+0000 7f816a8fbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f81541af300 con 0x55d69743cff0 2024-05-09T04:53:38.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.998+0000 7f8159480700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697439520 0x55d6975cc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46746/0 (socket says 172.21.15.46:46746) 2024-05-09T04:53:38.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.998+0000 7f8159480700 1 -- 172.21.15.46:0/2874288163 learned_addr learned my addr 172.21.15.46:0/2874288163 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.998+0000 7f8159480700 1 -- 172.21.15.46:0/2874288163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d69743cff0 msgr2=0x55d6975abaa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:53:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.998+0000 7f8159480700 1 --2- 172.21.15.46:0/2874288163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d69743cff0 0x55d6975abaa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.998+0000 7f8159480700 1 -- 172.21.15.46:0/2874288163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d6974b3d80 msgr2=0x55d6975b5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.998+0000 7f8159480700 1 --2- 172.21.15.46:0/2874288163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d6974b3d80 0x55d6975b5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.998+0000 7f8159480700 1 -- 172.21.15.46:0/2874288163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d6975bb8d0 con 0x55d697439520 2024-05-09T04:53:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.999+0000 7f8159480700 1 --2- 172.21.15.46:0/2874288163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697439520 0x55d6975cc190 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f81440099c0 tx=0x7f814400b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b79bdc89d59882 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.999+0000 7f813f7fe700 1 -- 172.21.15.46:0/2874288163 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144005450 con 0x55d697439520 2024-05-09T04:53:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.999+0000 7f813f7fe700 1 -- 172.21.15.46:0/2874288163 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81440055f0 con 0x55d697439520 2024-05-09T04:53:38.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:37.999+0000 7f813f7fe700 1 -- 172.21.15.46:0/2874288163 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f814400be50 con 0x55d697439520 2024-05-09T04:53:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.001+0000 7f816a8fbb80 1 -- 172.21.15.46:0/2874288163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697439520 msgr2=0x55d6975cc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.001+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/2874288163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697439520 0x55d6975cc190 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f81440099c0 tx=0x7f814400b540 comp rx=0 tx=0).stop 2024-05-09T04:53:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.001+0000 7f816a8fbb80 1 -- 172.21.15.46:0/2874288163 shutdown_connections 2024-05-09T04:53:38.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.001+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/2874288163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d6974b3d80 0x55d6975b5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.001+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/2874288163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d69743cff0 0x55d6975abaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.001+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/2874288163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d697439520 0x55d6975cc190 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.001+0000 7f816a8fbb80 1 -- 172.21.15.46:0/2874288163 >> 172.21.15.46:0/2874288163 conn(0x55d6975b6b60 msgr2=0x55d6975bf000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.001+0000 7f816a8fbb80 1 -- 172.21.15.46:0/2874288163 shutdown_connections 2024-05-09T04:53:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.001+0000 7f816a8fbb80 1 -- 172.21.15.46:0/2874288163 wait complete. 2024-05-09T04:53:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.002+0000 7f816a8fbb80 1 -- start start 2024-05-09T04:53:38.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.002+0000 7f816a8fbb80 1 Processor -- start 2024-05-09T04:53:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.002+0000 7f816a8fbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d697439520 0x55d6975d5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.002+0000 7f816a8fbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d69743cff0 0x55d6975d3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.002+0000 7f816a8fbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d6974b3d80 0x55d6975d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.002+0000 7f816a8fbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f81541af970 con 0x55d69743cff0 2024-05-09T04:53:38.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.002+0000 7f816a8fbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f81541b0de0 con 0x55d697439520 2024-05-09T04:53:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.002+0000 7f816a8fbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f81541b0040 con 0x55d6974b3d80 2024-05-09T04:53:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.003+0000 7f816a8fbb80 10 client.0 _ll_register_callbacks cb 0x55d6975bbe20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.003+0000 7f8159480700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d69743cff0 0x55d6975d3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.003+0000 7f8159480700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d69743cff0 0x55d6975d3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46748/0 (socket says 172.21.15.46:46748) 2024-05-09T04:53:38.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.003+0000 7f8159480700 1 -- 172.21.15.46:0/4142349844 learned_addr learned my addr 172.21.15.46:0/4142349844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.004+0000 7f8159480700 1 -- 172.21.15.46:0/4142349844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d6974b3d80 msgr2=0x55d6975d4130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:53:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.004+0000 7f8159480700 1 --2- 172.21.15.46:0/4142349844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d6974b3d80 0x55d6975d4130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.004+0000 7f8159480700 1 -- 172.21.15.46:0/4142349844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d697439520 msgr2=0x55d6975d5860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:53:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.004+0000 7f8159480700 1 --2- 172.21.15.46:0/4142349844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d697439520 0x55d6975d5860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.004+0000 7f8159480700 1 -- 172.21.15.46:0/4142349844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6975d47d0 con 0x55d69743cff0 2024-05-09T04:53:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.004+0000 7f8159480700 1 --2- 172.21.15.46:0/4142349844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d69743cff0 0x55d6975d3bf0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8144037b00 tx=0x7f8144037be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16082a040bdc8968 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:38.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.004+0000 7f813d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.005+0000 7f813d7fa700 1 -- 172.21.15.46:0/4142349844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8144005450 con 0x55d69743cff0 2024-05-09T04:53:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.005+0000 7f813d7fa700 1 -- 172.21.15.46:0/4142349844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f814401e040 con 0x55d69743cff0 2024-05-09T04:53:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.005+0000 7f816a8fbb80 1 -- 172.21.15.46:0/4142349844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f81440055f0 con 0x55d69743cff0 2024-05-09T04:53:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.005+0000 7f813d7fa700 1 -- 172.21.15.46:0/4142349844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8144047050 con 0x55d69743cff0 2024-05-09T04:53:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.005+0000 7f816a8fbb80 10 client.15996 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.005+0000 7f816a8fbb80 1 -- 172.21.15.46:0/4142349844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d69763b3a0 con 0x55d69743cff0 2024-05-09T04:53:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.005+0000 7f813d7fa700 1 -- 172.21.15.46:0/4142349844 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8144004000 con 0x55d69743cff0 2024-05-09T04:53:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.006+0000 7f813d7fa700 1 -- 172.21.15.46:0/4142349844 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f814402b020 con 0x55d69743cff0 2024-05-09T04:53:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.006+0000 7f816a8fbb80 10 client.15996 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.006+0000 7f816a8fbb80 10 client.15996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.006+0000 7f816a8fbb80 20 client.15996 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:38.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.006+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/4142349844 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d6975b6480 0x55d69763d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.006+0000 7f816a8fbb80 4 client.15996 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:53:38.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:38.006+0000 7f816a8fbb80 1 -- 172.21.15.46:0/4142349844 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f81440055f0 con 0x55d6975b6480 2024-05-09T04:53:39.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.031+0000 7f8159c81700 1 --2- 172.21.15.46:0/4142349844 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d6975b6480 0x55d69763d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:39.032 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:38 smithi046 ceph-mon[30136]: pgmap v79: 97 pgs: 97 active+clean; 8.6 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 646 MiB/s wr, 400 op/s 2024-05-09T04:53:39.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.033+0000 7f8159c81700 1 --2- 172.21.15.46:0/4142349844 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d6975b6480 0x55d69763d950 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99ea3054546f9f59 server_cookie=c249a713d9cc9dea in_seq=0 out_seq=0 2024-05-09T04:53:39.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.033+0000 7f813d7fa700 10 client.15996 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:39.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.034+0000 7f813d7fa700 1 -- 172.21.15.46:0/4142349844 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f81440055f0 con 0x55d6975b6480 2024-05-09T04:53:39.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.034+0000 7f813d7fa700 10 client.15996 handle_command_reply: tid=0 2024-05-09T04:53:39.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.034+0000 7f816a8fbb80 1 client.15996 shutdown 2024-05-09T04:53:39.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.035+0000 7f816a8fbb80 1 -- 172.21.15.46:0/4142349844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d69743cff0 msgr2=0x55d6975d3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:39.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.035+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/4142349844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d69743cff0 0x55d6975d3bf0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8144037b00 tx=0x7f8144037be0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.035+0000 7f816a8fbb80 1 -- 172.21.15.46:0/4142349844 shutdown_connections 2024-05-09T04:53:39.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.035+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/4142349844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d6974b3d80 0x55d6975d4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.035+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/4142349844 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d6975b6480 0x55d69763d950 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.035+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/4142349844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d697439520 0x55d6975d5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.035+0000 7f816a8fbb80 1 --2- 172.21.15.46:0/4142349844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d69743cff0 0x55d6975d3bf0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.035+0000 7f816a8fbb80 1 -- 172.21.15.46:0/4142349844 >> 172.21.15.46:0/4142349844 conn(0x55d6975b6b60 msgr2=0x55d6975b5470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:39.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.035+0000 7f813d7fa700 0 client.15996 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:39.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.035+0000 7f816a8fbb80 1 -- 172.21.15.46:0/4142349844 shutdown_connections 2024-05-09T04:53:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.035+0000 7f816a8fbb80 1 -- 172.21.15.46:0/4142349844 wait complete. 2024-05-09T04:53:39.037 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.036+0000 7f816a8fbb80 20 client.15996 trim_cache size 0 max 16384 2024-05-09T04:53:39.037 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:53:39.037 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:53:39.037 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "37253760-a82b-4bcb-a6dd-ab05841515e3", 2024-05-09T04:53:39.037 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:53:39.037 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:53:39.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.037+0000 7f815bee5700 1 -- 172.21.15.46:0/1785433236 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f81400b2a70 msgr2=0x7f81400b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:39.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.037+0000 7f815bee5700 1 --2- 172.21.15.46:0/1785433236 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f81400b2a70 0x7f81400b4f80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8144003800 tx=0x7f8144000bc0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.037+0000 7f815bee5700 1 -- 172.21.15.46:0/1785433236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8154103ca0 msgr2=0x7f815419d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:39.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.037+0000 7f815bee5700 1 --2- 172.21.15.46:0/1785433236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8154103ca0 0x7f815419d180 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f815000b550 tx=0x7f815000b910 comp rx=0 tx=0).stop 2024-05-09T04:53:39.038 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.037+0000 7f815bee5700 1 -- 172.21.15.46:0/1785433236 shutdown_connections 2024-05-09T04:53:39.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.037+0000 7f815bee5700 1 --2- 172.21.15.46:0/1785433236 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f81400b2a70 0x7f81400b4f80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.037+0000 7f815bee5700 1 --2- 172.21.15.46:0/1785433236 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8154108760 0x7f81541a19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.037+0000 7f815bee5700 1 --2- 172.21.15.46:0/1785433236 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8154103ca0 0x7f815419d180 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.037+0000 7f815bee5700 1 --2- 172.21.15.46:0/1785433236 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8154107e10 0x7f815419d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:39.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.037+0000 7f815bee5700 1 -- 172.21.15.46:0/1785433236 >> 172.21.15.46:0/1785433236 conn(0x7f81540fe6f0 msgr2=0x7f815410fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:39.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.037+0000 7f815bee5700 1 -- 172.21.15.46:0/1785433236 shutdown_connections 2024-05-09T04:53:39.039 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:39.037+0000 7f815bee5700 1 -- 172.21.15.46:0/1785433236 wait complete. 2024-05-09T04:53:39.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:38 smithi110 ceph-mon[34904]: pgmap v79: 97 pgs: 97 active+clean; 8.6 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 646 MiB/s wr, 400 op/s 2024-05-09T04:53:39.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:38 smithi072 ceph-mon[34088]: pgmap v79: 97 pgs: 97 active+clean; 8.6 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 646 MiB/s wr, 400 op/s 2024-05-09T04:53:39.852 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:53:39.955 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:39.955 INFO:tasks.workunit.client.0.smithi046.stdout: Children see throughput for 1 initial writers = 838492.94 kB/sec 2024-05-09T04:53:39.955 INFO:tasks.workunit.client.0.smithi046.stdout: Parent sees throughput for 1 initial writers = 838477.92 kB/sec 2024-05-09T04:53:39.955 INFO:tasks.workunit.client.0.smithi046.stdout: Min throughput per process = 838492.94 kB/sec 2024-05-09T04:53:39.955 INFO:tasks.workunit.client.0.smithi046.stdout: Max throughput per process = 838492.94 kB/sec 2024-05-09T04:53:39.955 INFO:tasks.workunit.client.0.smithi046.stdout: Avg throughput per process = 838492.94 kB/sec 2024-05-09T04:53:39.955 INFO:tasks.workunit.client.0.smithi046.stdout: Min xfer = 10485760.00 kB 2024-05-09T04:53:40.160 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:40 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:53:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:40 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:53:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:40 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:53:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:40 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:53:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:40 smithi110 ceph-mon[34904]: pgmap v80: 97 pgs: 97 active+clean; 8.6 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 646 MiB/s wr, 398 op/s 2024-05-09T04:53:41.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:40 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:53:41.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:40 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:53:41.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:40 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:53:41.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:40 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:53:41.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:40 smithi046 ceph-mon[30136]: pgmap v80: 97 pgs: 97 active+clean; 8.6 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 646 MiB/s wr, 398 op/s 2024-05-09T04:53:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:40 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:53:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:40 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:53:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:40 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:53:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:40 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:53:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:40 smithi072 ceph-mon[34088]: pgmap v80: 97 pgs: 97 active+clean; 8.6 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 536 KiB/s rd, 646 MiB/s wr, 398 op/s 2024-05-09T04:53:41.986 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:41 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:53:42.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:41 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:53:42.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:41 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:53:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.903+0000 7f4869443700 1 -- 172.21.15.46:0/895046190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48641030e0 msgr2=0x7f4864111170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.903+0000 7f4869443700 1 --2- 172.21.15.46:0/895046190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48641030e0 0x7f4864111170 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f485c00b0a0 tx=0x7f485c00b3b0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.903+0000 7f4869443700 1 -- 172.21.15.46:0/895046190 shutdown_connections 2024-05-09T04:53:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.903+0000 7f4869443700 1 --2- 172.21.15.46:0/895046190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f48641030e0 0x7f4864111170 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.903+0000 7f4869443700 1 --2- 172.21.15.46:0/895046190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4864109300 0x7f4864109710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.903+0000 7f4869443700 1 --2- 172.21.15.46:0/895046190 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4864102790 0x7f4864102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.903+0000 7f4869443700 1 -- 172.21.15.46:0/895046190 >> 172.21.15.46:0/895046190 conn(0x7f48640fe520 msgr2=0x7f4864100970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.903+0000 7f4869443700 1 -- 172.21.15.46:0/895046190 shutdown_connections 2024-05-09T04:53:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.903+0000 7f4869443700 1 -- 172.21.15.46:0/895046190 wait complete. 2024-05-09T04:53:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.904+0000 7f4869443700 1 Processor -- start 2024-05-09T04:53:42.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.904+0000 7f4869443700 1 -- start start 2024-05-09T04:53:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f4869443700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4864102790 0x7f486419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f4869443700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48641030e0 0x7f486419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f4869443700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4864109300 0x7f486419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f4869443700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48641145a0 con 0x7f4864109300 2024-05-09T04:53:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f4869443700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4864114420 con 0x7f4864102790 2024-05-09T04:53:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f4869443700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4864114720 con 0x7f48641030e0 2024-05-09T04:53:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f48637fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4864109300 0x7f486419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f48637fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4864109300 0x7f486419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46766/0 (socket says 172.21.15.46:46766) 2024-05-09T04:53:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f48637fe700 1 -- 172.21.15.46:0/565166776 learned_addr learned my addr 172.21.15.46:0/565166776 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f4862ffd700 1 --2- 172.21.15.46:0/565166776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4864102790 0x7f486419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f48627fc700 1 --2- 172.21.15.46:0/565166776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48641030e0 0x7f486419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f48637fe700 1 -- 172.21.15.46:0/565166776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48641030e0 msgr2=0x7f486419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f48637fe700 1 --2- 172.21.15.46:0/565166776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48641030e0 0x7f486419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f48637fe700 1 -- 172.21.15.46:0/565166776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4864102790 msgr2=0x7f486419ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f48637fe700 1 --2- 172.21.15.46:0/565166776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4864102790 0x7f486419ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f48637fe700 1 -- 172.21.15.46:0/565166776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f486419fd00 con 0x7f4864109300 2024-05-09T04:53:42.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.905+0000 7f48637fe700 1 --2- 172.21.15.46:0/565166776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4864109300 0x7f486419f660 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f485c00b830 tx=0x7f485c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.906+0000 7f484bfff700 1 -- 172.21.15.46:0/565166776 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c021070 con 0x7f4864109300 2024-05-09T04:53:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.906+0000 7f484bfff700 1 -- 172.21.15.46:0/565166776 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f485c007e30 con 0x7f4864109300 2024-05-09T04:53:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.906+0000 7f4869443700 1 -- 172.21.15.46:0/565166776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f486419ff90 con 0x7f4864109300 2024-05-09T04:53:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.906+0000 7f484bfff700 1 -- 172.21.15.46:0/565166776 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f485c0047d0 con 0x7f4864109300 2024-05-09T04:53:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.906+0000 7f4869443700 1 -- 172.21.15.46:0/565166776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48641a7910 con 0x7f4864109300 2024-05-09T04:53:42.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.909+0000 7f484bfff700 1 -- 172.21.15.46:0/565166776 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f485c01d450 con 0x7f4864109300 2024-05-09T04:53:42.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.909+0000 7f484bfff700 1 --2- 172.21.15.46:0/565166776 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f484c0b29f0 0x7f484c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.910 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:42 smithi046 ceph-mon[30136]: pgmap v81: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 852 MiB/s wr, 453 op/s 2024-05-09T04:53:42.912 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.909+0000 7f4877e59b80 1 Processor -- start 2024-05-09T04:53:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.909+0000 7f4862ffd700 1 --2- 172.21.15.46:0/565166776 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f484c0b29f0 0x7f484c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.910+0000 7f484bfff700 1 -- 172.21.15.46:0/565166776 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f485c00f020 con 0x7f4864109300 2024-05-09T04:53:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.910+0000 7f4877e59b80 1 -- start start 2024-05-09T04:53:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.910+0000 7f4877e59b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586d4ed00 0x562586e5b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.910+0000 7f4877e59b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586cd44a0 0x562586e5bb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.910+0000 7f4877e59b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586e55810 0x562586e55c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.910+0000 7f4877e59b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562586d740a0 con 0x562586d4ed00 2024-05-09T04:53:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.910+0000 7f4877e59b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562586d199a0 con 0x562586cd44a0 2024-05-09T04:53:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.910+0000 7f4877e59b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562586d70060 con 0x562586e55810 2024-05-09T04:53:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.910+0000 7f4862ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586d4ed00 0x562586e5b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.910+0000 7f4862ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586d4ed00 0x562586e5b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46770/0 (socket says 172.21.15.46:46770) 2024-05-09T04:53:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.910+0000 7f4862ffd700 1 -- 172.21.15.46:0/1685211568 learned_addr learned my addr 172.21.15.46:0/1685211568 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.910+0000 7f4862ffd700 1 --2- 172.21.15.46:0/565166776 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f484c0b29f0 0x7f484c0b4f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f486406ca70 tx=0x7f4854007400 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:42.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.910+0000 7f48627fc700 1 --2- 172.21.15.46:0/1685211568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586cd44a0 0x562586e5bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.910+0000 7f48637fe700 1 --2- 172.21.15.46:0/1685211568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586e55810 0x562586e55c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.911+0000 7f4862ffd700 1 -- 172.21.15.46:0/1685211568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586e55810 msgr2=0x562586e55c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.911+0000 7f4862ffd700 1 --2- 172.21.15.46:0/1685211568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586e55810 0x562586e55c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.911+0000 7f4862ffd700 1 -- 172.21.15.46:0/1685211568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586cd44a0 msgr2=0x562586e5bb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.911+0000 7f4862ffd700 1 --2- 172.21.15.46:0/1685211568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586cd44a0 0x562586e5bb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.911+0000 7f4862ffd700 1 -- 172.21.15.46:0/1685211568 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562586d0a500 con 0x562586d4ed00 2024-05-09T04:53:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.911+0000 7f48637fe700 1 --2- 172.21.15.46:0/1685211568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586e55810 0x562586e55c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.911+0000 7f48627fc700 1 --2- 172.21.15.46:0/1685211568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586cd44a0 0x562586e5bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.911+0000 7f4862ffd700 1 --2- 172.21.15.46:0/1685211568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586d4ed00 0x562586e5b650 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4854010db0 tx=0x7f4854013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adec80334bfa28fb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.911+0000 7f48497fa700 1 -- 172.21.15.46:0/1685211568 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854010ea0 con 0x562586d4ed00 2024-05-09T04:53:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.911+0000 7f48497fa700 1 -- 172.21.15.46:0/1685211568 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4854011db0 con 0x562586d4ed00 2024-05-09T04:53:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.911+0000 7f48497fa700 1 -- 172.21.15.46:0/1685211568 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854026780 con 0x562586d4ed00 2024-05-09T04:53:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.912+0000 7f4877e59b80 1 -- 172.21.15.46:0/1685211568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586d4ed00 msgr2=0x562586e5b650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:42.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.912+0000 7f4877e59b80 1 --2- 172.21.15.46:0/1685211568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586d4ed00 0x562586e5b650 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4854010db0 tx=0x7f4854013000 comp rx=0 tx=0).stop 2024-05-09T04:53:42.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.912+0000 7f4877e59b80 1 -- 172.21.15.46:0/1685211568 shutdown_connections 2024-05-09T04:53:42.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.912+0000 7f4877e59b80 1 --2- 172.21.15.46:0/1685211568 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586e55810 0x562586e55c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.912+0000 7f4877e59b80 1 --2- 172.21.15.46:0/1685211568 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586d4ed00 0x562586e5b650 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.912+0000 7f4877e59b80 1 --2- 172.21.15.46:0/1685211568 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586cd44a0 0x562586e5bb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.912+0000 7f4877e59b80 1 -- 172.21.15.46:0/1685211568 >> 172.21.15.46:0/1685211568 conn(0x562586cd7f70 msgr2=0x562586e4c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:42.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.912+0000 7f4877e59b80 1 -- 172.21.15.46:0/1685211568 shutdown_connections 2024-05-09T04:53:42.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.912+0000 7f4877e59b80 1 -- 172.21.15.46:0/1685211568 wait complete. 2024-05-09T04:53:42.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.913+0000 7f4877e59b80 1 -- start start 2024-05-09T04:53:42.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.913+0000 7f4877e59b80 1 Processor -- start 2024-05-09T04:53:42.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.913+0000 7f4877e59b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e5f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.913+0000 7f4877e59b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586d4ed00 0x562586e5f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.913+0000 7f4877e59b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586e55810 0x562586e5feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.913+0000 7f4877e59b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x562586e56570 con 0x562586cd44a0 2024-05-09T04:53:42.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.913+0000 7f4877e59b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x562586d0eb20 con 0x562586e55810 2024-05-09T04:53:42.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.913+0000 7f4877e59b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x562586e62430 con 0x562586d4ed00 2024-05-09T04:53:42.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.913+0000 7f4877e59b80 10 client.0 _ll_register_callbacks cb 0x562586be56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:42.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.913+0000 7f4862ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e5f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.913+0000 7f4862ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e5f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46776/0 (socket says 172.21.15.46:46776) 2024-05-09T04:53:42.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.913+0000 7f4862ffd700 1 -- 172.21.15.46:0/487398554 learned_addr learned my addr 172.21.15.46:0/487398554 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:42.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.913+0000 7f48627fc700 1 --2- 172.21.15.46:0/487398554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586d4ed00 0x562586e5f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.914+0000 7f48637fe700 1 --2- 172.21.15.46:0/487398554 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586e55810 0x562586e5feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.914+0000 7f4862ffd700 1 -- 172.21.15.46:0/487398554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586d4ed00 msgr2=0x562586e5f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:42.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.914+0000 7f4862ffd700 1 --2- 172.21.15.46:0/487398554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586d4ed00 0x562586e5f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.914+0000 7f4862ffd700 1 -- 172.21.15.46:0/487398554 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586e55810 msgr2=0x562586e5feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:42.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.914+0000 7f4862ffd700 1 --2- 172.21.15.46:0/487398554 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586e55810 0x562586e5feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.914+0000 7f4862ffd700 1 -- 172.21.15.46:0/487398554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562586bef490 con 0x562586cd44a0 2024-05-09T04:53:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.914+0000 7f4862ffd700 1 --2- 172.21.15.46:0/487398554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e5f430 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f485401ef40 tx=0x7f485401e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f557ac30c884b47 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.914+0000 7f4846ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.914+0000 7f4846ff5700 1 -- 172.21.15.46:0/487398554 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854034060 con 0x562586cd44a0 2024-05-09T04:53:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.914+0000 7f4877e59b80 1 -- 172.21.15.46:0/487398554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4854011db0 con 0x562586cd44a0 2024-05-09T04:53:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.914+0000 7f4846ff5700 1 -- 172.21.15.46:0/487398554 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f4854011bb0 con 0x562586cd44a0 2024-05-09T04:53:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.914+0000 7f4846ff5700 1 -- 172.21.15.46:0/487398554 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4854010ea0 con 0x562586cd44a0 2024-05-09T04:53:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.914+0000 7f4877e59b80 10 client.16008 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.914+0000 7f4877e59b80 1 -- 172.21.15.46:0/487398554 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562586e6ce90 con 0x562586cd44a0 2024-05-09T04:53:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.915+0000 7f4846ff5700 1 -- 172.21.15.46:0/487398554 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4854011db0 con 0x562586cd44a0 2024-05-09T04:53:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.915+0000 7f4846ff5700 1 -- 172.21.15.46:0/487398554 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f4854003bb0 con 0x562586cd44a0 2024-05-09T04:53:42.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.915+0000 7f4877e59b80 10 client.16008 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.915+0000 7f4877e59b80 10 client.16008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.915+0000 7f4877e59b80 20 client.16008 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.915+0000 7f4877e59b80 1 --2- 172.21.15.46:0/487398554 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562586e6f470 0x562586e71890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.915+0000 7f4877e59b80 4 client.16008 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.915+0000 7f4877e59b80 1 -- 172.21.15.46:0/487398554 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562586d81a80 con 0x562586e6f470 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.915+0000 7f48627fc700 1 --2- 172.21.15.46:0/487398554 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562586e6f470 0x562586e71890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.916+0000 7f48627fc700 1 --2- 172.21.15.46:0/487398554 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562586e6f470 0x562586e71890 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=639218370aa0d347 server_cookie=e204a2ba349ff1bd in_seq=0 out_seq=0 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.916+0000 7f4846ff5700 10 client.16008 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.917+0000 7f4846ff5700 1 -- 172.21.15.46:0/487398554 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562586d81a80 con 0x562586e6f470 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.917+0000 7f4846ff5700 10 client.16008 handle_command_reply: tid=0 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.917+0000 7f4877e59b80 1 client.16008 shutdown 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.917+0000 7f4877e59b80 1 -- 172.21.15.46:0/487398554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 msgr2=0x562586e5f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.917+0000 7f4877e59b80 1 --2- 172.21.15.46:0/487398554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e5f430 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f485401ef40 tx=0x7f485401e650 comp rx=0 tx=0).stop 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.917+0000 7f4877e59b80 1 -- 172.21.15.46:0/487398554 shutdown_connections 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.917+0000 7f4877e59b80 1 --2- 172.21.15.46:0/487398554 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562586e6f470 0x562586e71890 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.917+0000 7f4877e59b80 1 --2- 172.21.15.46:0/487398554 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586e55810 0x562586e5feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.917+0000 7f4877e59b80 1 --2- 172.21.15.46:0/487398554 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e5f430 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.917+0000 7f4877e59b80 1 --2- 172.21.15.46:0/487398554 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586d4ed00 0x562586e5f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.917+0000 7f4877e59b80 1 -- 172.21.15.46:0/487398554 >> 172.21.15.46:0/487398554 conn(0x562586cd7f70 msgr2=0x562586e45ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:42.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.917+0000 7f4846ff5700 0 client.16008 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:42.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.917+0000 7f4877e59b80 1 -- 172.21.15.46:0/487398554 shutdown_connections 2024-05-09T04:53:42.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.917+0000 7f4877e59b80 1 -- 172.21.15.46:0/487398554 wait complete. 2024-05-09T04:53:42.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.917+0000 7f4877e59b80 20 client.16008 trim_cache size 0 max 16384 2024-05-09T04:53:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.938+0000 7f4877e59b80 1 Processor -- start 2024-05-09T04:53:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.938+0000 7f4877e59b80 1 -- start start 2024-05-09T04:53:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.938+0000 7f4877e59b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e46a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.938+0000 7f4877e59b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586d4ed00 0x562586e618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.938+0000 7f4877e59b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586e55810 0x562586e5fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.938+0000 7f4877e59b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48641af470 con 0x562586cd44a0 2024-05-09T04:53:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.938+0000 7f4877e59b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f48641acd00 con 0x562586e55810 2024-05-09T04:53:42.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.938+0000 7f4877e59b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f48641aab70 con 0x562586d4ed00 2024-05-09T04:53:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.938+0000 7f4862ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e46a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.938+0000 7f4862ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e46a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46790/0 (socket says 172.21.15.46:46790) 2024-05-09T04:53:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.938+0000 7f4862ffd700 1 -- 172.21.15.46:0/1405606844 learned_addr learned my addr 172.21.15.46:0/1405606844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.938+0000 7f48627fc700 1 --2- 172.21.15.46:0/1405606844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586d4ed00 0x562586e618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.939+0000 7f48637fe700 1 --2- 172.21.15.46:0/1405606844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586e55810 0x562586e5fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.939+0000 7f4862ffd700 1 -- 172.21.15.46:0/1405606844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586d4ed00 msgr2=0x562586e618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.939+0000 7f4862ffd700 1 --2- 172.21.15.46:0/1405606844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586d4ed00 0x562586e618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.939+0000 7f4862ffd700 1 -- 172.21.15.46:0/1405606844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586e55810 msgr2=0x562586e5fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.939+0000 7f4862ffd700 1 --2- 172.21.15.46:0/1405606844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586e55810 0x562586e5fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.939+0000 7f4862ffd700 1 -- 172.21.15.46:0/1405606844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562586e4ddd0 con 0x562586cd44a0 2024-05-09T04:53:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.939+0000 7f48627fc700 1 --2- 172.21.15.46:0/1405606844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586d4ed00 0x562586e618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.939+0000 7f4862ffd700 1 --2- 172.21.15.46:0/1405606844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e46a20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4854005810 tx=0x7f4854010940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71cc559df4e256a5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.939+0000 7f4848ff9700 1 -- 172.21.15.46:0/1405606844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854034070 con 0x562586cd44a0 2024-05-09T04:53:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.939+0000 7f4848ff9700 1 -- 172.21.15.46:0/1405606844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4854005520 con 0x562586cd44a0 2024-05-09T04:53:42.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.939+0000 7f4848ff9700 1 -- 172.21.15.46:0/1405606844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854003c10 con 0x562586cd44a0 2024-05-09T04:53:42.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.940+0000 7f4877e59b80 1 -- 172.21.15.46:0/1405606844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 msgr2=0x562586e46a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:42.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.940+0000 7f4877e59b80 1 --2- 172.21.15.46:0/1405606844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e46a20 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f4854005810 tx=0x7f4854010940 comp rx=0 tx=0).stop 2024-05-09T04:53:42.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.940+0000 7f4877e59b80 1 -- 172.21.15.46:0/1405606844 shutdown_connections 2024-05-09T04:53:42.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.940+0000 7f4877e59b80 1 --2- 172.21.15.46:0/1405606844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586e55810 0x562586e5fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.940+0000 7f4877e59b80 1 --2- 172.21.15.46:0/1405606844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e46a20 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.940+0000 7f4877e59b80 1 --2- 172.21.15.46:0/1405606844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586d4ed00 0x562586e618d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.940+0000 7f4877e59b80 1 -- 172.21.15.46:0/1405606844 >> 172.21.15.46:0/1405606844 conn(0x562586cd7f70 msgr2=0x562586e5e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:42.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.940+0000 7f4877e59b80 1 -- 172.21.15.46:0/1405606844 shutdown_connections 2024-05-09T04:53:42.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.940+0000 7f4877e59b80 1 -- 172.21.15.46:0/1405606844 wait complete. 2024-05-09T04:53:42.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.940+0000 7f4877e59b80 1 -- start start 2024-05-09T04:53:42.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.940+0000 7f4877e59b80 1 Processor -- start 2024-05-09T04:53:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4877e59b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e74d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4877e59b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586d4ed00 0x562586e72090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4877e59b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586e55810 0x562586e725d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4877e59b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f48641ab1c0 con 0x562586cd44a0 2024-05-09T04:53:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4877e59b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f48641ac630 con 0x562586d4ed00 2024-05-09T04:53:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4877e59b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f48641ab890 con 0x562586e55810 2024-05-09T04:53:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4877e59b80 10 client.0 _ll_register_callbacks cb 0x562586be56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4862ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e74d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4862ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e74d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46802/0 (socket says 172.21.15.46:46802) 2024-05-09T04:53:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4862ffd700 1 -- 172.21.15.46:0/3664749303 learned_addr learned my addr 172.21.15.46:0/3664749303 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f48627fc700 1 --2- 172.21.15.46:0/3664749303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586d4ed00 0x562586e72090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f48637fe700 1 --2- 172.21.15.46:0/3664749303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586e55810 0x562586e725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4862ffd700 1 -- 172.21.15.46:0/3664749303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586e55810 msgr2=0x562586e725d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:42.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4862ffd700 1 --2- 172.21.15.46:0/3664749303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586e55810 0x562586e725d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4862ffd700 1 -- 172.21.15.46:0/3664749303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586d4ed00 msgr2=0x562586e72090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4862ffd700 1 --2- 172.21.15.46:0/3664749303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586d4ed00 0x562586e72090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4862ffd700 1 -- 172.21.15.46:0/3664749303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562586e489d0 con 0x562586cd44a0 2024-05-09T04:53:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f48627fc700 1 --2- 172.21.15.46:0/3664749303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586d4ed00 0x562586e72090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4862ffd700 1 --2- 172.21.15.46:0/3664749303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e74d10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f485401ef60 tx=0x7f4854011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913b5d305bdb2209 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.941+0000 7f4846ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.942+0000 7f4846ff5700 1 -- 172.21.15.46:0/3664749303 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4854045070 con 0x562586cd44a0 2024-05-09T04:53:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.942+0000 7f4846ff5700 1 -- 172.21.15.46:0/3664749303 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f48540052d0 con 0x562586cd44a0 2024-05-09T04:53:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.942+0000 7f4877e59b80 1 -- 172.21.15.46:0/3664749303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4854005520 con 0x562586cd44a0 2024-05-09T04:53:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.942+0000 7f4846ff5700 1 -- 172.21.15.46:0/3664749303 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4854034040 con 0x562586cd44a0 2024-05-09T04:53:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.942+0000 7f4877e59b80 10 client.16014 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.942+0000 7f4877e59b80 1 -- 172.21.15.46:0/3664749303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x562586ed95f0 con 0x562586cd44a0 2024-05-09T04:53:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.942+0000 7f4846ff5700 1 -- 172.21.15.46:0/3664749303 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4854005520 con 0x562586cd44a0 2024-05-09T04:53:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.942+0000 7f4846ff5700 1 -- 172.21.15.46:0/3664749303 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f4854026a30 con 0x562586cd44a0 2024-05-09T04:53:42.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.942+0000 7f4877e59b80 10 client.16014 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.942+0000 7f4877e59b80 10 client.16014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.942+0000 7f4877e59b80 20 client.16014 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.942+0000 7f4877e59b80 1 --2- 172.21.15.46:0/3664749303 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562586ed9be0 0x562586edc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.942+0000 7f4877e59b80 4 client.16014 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:53:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.942+0000 7f4877e59b80 1 -- 172.21.15.46:0/3664749303 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562586d7bb20 con 0x562586ed9be0 2024-05-09T04:53:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f48627fc700 1 --2- 172.21.15.46:0/3664749303 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562586ed9be0 0x562586edc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f48627fc700 1 --2- 172.21.15.46:0/3664749303 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562586ed9be0 0x562586edc000 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=add5e1f43275ec08 server_cookie=d3c349fda553857b in_seq=0 out_seq=0 2024-05-09T04:53:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f4846ff5700 10 client.16014 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f4846ff5700 1 -- 172.21.15.46:0/3664749303 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x562586d7bb20 con 0x562586ed9be0 2024-05-09T04:53:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f4846ff5700 10 client.16014 handle_command_reply: tid=0 2024-05-09T04:53:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f4877e59b80 1 client.16014 shutdown 2024-05-09T04:53:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f4877e59b80 1 -- 172.21.15.46:0/3664749303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 msgr2=0x562586e74d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f4877e59b80 1 --2- 172.21.15.46:0/3664749303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e74d10 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f485401ef60 tx=0x7f4854011d60 comp rx=0 tx=0).stop 2024-05-09T04:53:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f4877e59b80 1 -- 172.21.15.46:0/3664749303 shutdown_connections 2024-05-09T04:53:42.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f4877e59b80 1 --2- 172.21.15.46:0/3664749303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x562586e55810 0x562586e725d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f4877e59b80 1 --2- 172.21.15.46:0/3664749303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x562586cd44a0 0x562586e74d10 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f4877e59b80 1 --2- 172.21.15.46:0/3664749303 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x562586ed9be0 0x562586edc000 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f4877e59b80 1 --2- 172.21.15.46:0/3664749303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x562586d4ed00 0x562586e72090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f4877e59b80 1 -- 172.21.15.46:0/3664749303 >> 172.21.15.46:0/3664749303 conn(0x562586cd7f70 msgr2=0x562586e46780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:42.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f4846ff5700 0 client.16014 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:42.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f4877e59b80 1 -- 172.21.15.46:0/3664749303 shutdown_connections 2024-05-09T04:53:42.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.943+0000 7f4877e59b80 1 -- 172.21.15.46:0/3664749303 wait complete. 2024-05-09T04:53:42.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.944+0000 7f4877e59b80 20 client.16014 trim_cache size 0 max 16384 2024-05-09T04:53:42.948 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:53:42.948 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:53:42.948 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:53:42.949 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:53:42.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.944+0000 7f4869443700 1 -- 172.21.15.46:0/565166776 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f484c0b29f0 msgr2=0x7f484c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:42.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.944+0000 7f4869443700 1 --2- 172.21.15.46:0/565166776 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f484c0b29f0 0x7f484c0b4f00 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f486406ca70 tx=0x7f4854007400 comp rx=0 tx=0).stop 2024-05-09T04:53:42.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.944+0000 7f4869443700 1 -- 172.21.15.46:0/565166776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4864109300 msgr2=0x7f486419f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:42.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.944+0000 7f4869443700 1 --2- 172.21.15.46:0/565166776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4864109300 0x7f486419f660 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f485c00b830 tx=0x7f485c004ce0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.944+0000 7f4869443700 1 -- 172.21.15.46:0/565166776 shutdown_connections 2024-05-09T04:53:42.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.944+0000 7f4869443700 1 --2- 172.21.15.46:0/565166776 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f484c0b29f0 0x7f484c0b4f00 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.944+0000 7f4869443700 1 --2- 172.21.15.46:0/565166776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4864109300 0x7f486419f660 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.944+0000 7f4869443700 1 --2- 172.21.15.46:0/565166776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4864102790 0x7f486419ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.944+0000 7f4869443700 1 --2- 172.21.15.46:0/565166776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f48641030e0 0x7f486419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:42.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.944+0000 7f4869443700 1 -- 172.21.15.46:0/565166776 >> 172.21.15.46:0/565166776 conn(0x7f48640fe520 msgr2=0x7f4864107050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:42.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.944+0000 7f4869443700 1 -- 172.21.15.46:0/565166776 shutdown_connections 2024-05-09T04:53:42.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:42.944+0000 7f4869443700 1 -- 172.21.15.46:0/565166776 wait complete. 2024-05-09T04:53:43.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:42 smithi110 ceph-mon[34904]: pgmap v81: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 852 MiB/s wr, 453 op/s 2024-05-09T04:53:43.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:42 smithi072 ceph-mon[34088]: pgmap v81: 97 pgs: 97 active+clean; 11 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 539 KiB/s rd, 852 MiB/s wr, 453 op/s 2024-05-09T04:53:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.257+0000 7f770fa05640 1 -- 172.21.15.46:0/378401653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77081073e0 msgr2=0x7f77081077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.257+0000 7f770fa05640 1 --2- 172.21.15.46:0/378401653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77081073e0 0x7f77081077f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f76f8009900 tx=0x7f76f802a410 comp rx=0 tx=0).stop 2024-05-09T04:53:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.257+0000 7f770fa05640 1 -- 172.21.15.46:0/378401653 shutdown_connections 2024-05-09T04:53:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.257+0000 7f770fa05640 1 --2- 172.21.15.46:0/378401653 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7708107e60 0x7f770810f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.257+0000 7f770fa05640 1 --2- 172.21.15.46:0/378401653 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f77081073e0 0x7f77081077f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.257+0000 7f770fa05640 1 --2- 172.21.15.46:0/378401653 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7708103520 0x7f7708103930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.257+0000 7f770fa05640 1 -- 172.21.15.46:0/378401653 >> 172.21.15.46:0/378401653 conn(0x7f77080779e0 msgr2=0x7f7708077e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.258+0000 7f770fa05640 1 -- 172.21.15.46:0/378401653 shutdown_connections 2024-05-09T04:53:43.258 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.258+0000 7f770fa05640 1 -- 172.21.15.46:0/378401653 wait complete. 2024-05-09T04:53:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.259+0000 7f770fa05640 1 Processor -- start 2024-05-09T04:53:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.259+0000 7f770fa05640 1 -- start start 2024-05-09T04:53:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.259+0000 7f770fa05640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7708103520 0x7f77081a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.259+0000 7f770fa05640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77081073e0 0x7f77081a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.259+0000 7f770fa05640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7708107e60 0x7f770819a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.259+0000 7f770fa05640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7708112d60 con 0x7f7708107e60 2024-05-09T04:53:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.259+0000 7f770fa05640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7708112be0 con 0x7f7708103520 2024-05-09T04:53:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.259+0000 7f770fa05640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7708112ee0 con 0x7f77081073e0 2024-05-09T04:53:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.259+0000 7f770d7a1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7708103520 0x7f77081a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.259+0000 7f770dfa2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7708107e60 0x7f770819a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.259+0000 7f770cfa0640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77081073e0 0x7f77081a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.261 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.259+0000 7f770d7a1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7708103520 0x7f77081a0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:54300/0 (socket says 172.21.15.46:54300) 2024-05-09T04:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.259+0000 7f770dfa2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7708107e60 0x7f770819a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46830/0 (socket says 172.21.15.46:46830) 2024-05-09T04:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.259+0000 7f770d7a1640 1 -- 172.21.15.46:0/763910726 learned_addr learned my addr 172.21.15.46:0/763910726 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.260+0000 7f770dfa2640 1 -- 172.21.15.46:0/763910726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77081073e0 msgr2=0x7f77081a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.260+0000 7f770dfa2640 1 --2- 172.21.15.46:0/763910726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77081073e0 0x7f77081a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.260+0000 7f770dfa2640 1 -- 172.21.15.46:0/763910726 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7708103520 msgr2=0x7f77081a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.260+0000 7f770dfa2640 1 --2- 172.21.15.46:0/763910726 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7708103520 0x7f77081a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.260+0000 7f770dfa2640 1 -- 172.21.15.46:0/763910726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f770819ae00 con 0x7f7708107e60 2024-05-09T04:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.260+0000 7f770dfa2640 1 --2- 172.21.15.46:0/763910726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7708107e60 0x7f770819a6d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f770000e940 tx=0x7f770000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.260+0000 7f76fe7fc640 1 -- 172.21.15.46:0/763910726 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700004d20 con 0x7f7708107e60 2024-05-09T04:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.260+0000 7f770fa05640 1 -- 172.21.15.46:0/763910726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f770819b090 con 0x7f7708107e60 2024-05-09T04:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.260+0000 7f770fa05640 1 -- 172.21.15.46:0/763910726 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77081a72f0 con 0x7f7708107e60 2024-05-09T04:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.260+0000 7f76fe7fc640 1 -- 172.21.15.46:0/763910726 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7700005020 con 0x7f7708107e60 2024-05-09T04:53:43.262 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.260+0000 7f76fe7fc640 1 -- 172.21.15.46:0/763910726 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77000116d0 con 0x7f7708107e60 2024-05-09T04:53:43.263 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.263+0000 7f76fe7fc640 1 -- 172.21.15.46:0/763910726 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7700005460 con 0x7f7708107e60 2024-05-09T04:53:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.264+0000 7f76fe7fc640 1 --2- 172.21.15.46:0/763910726 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f76f40b2950 0x7f76f40b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.264+0000 7f76fe7fc640 1 -- 172.21.15.46:0/763910726 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f77000e26f0 con 0x7f7708107e60 2024-05-09T04:53:43.264 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.264+0000 7f770d7a1640 1 --2- 172.21.15.46:0/763910726 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f76f40b2950 0x7f76f40b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.265+0000 7f770d7a1640 1 --2- 172.21.15.46:0/763910726 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f76f40b2950 0x7f76f40b4e60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7704009c70 tx=0x7f7704005f90 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.265+0000 7f7716992740 1 Processor -- start 2024-05-09T04:53:43.265 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.265+0000 7f7716992740 1 -- start start 2024-05-09T04:53:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.265+0000 7f7716992740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2cc1b0 0x55dccf66f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.265+0000 7f7716992740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2a32f0 0x55dccf66fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.265+0000 7f7716992740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf670100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.265+0000 7f7716992740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dccf5d47c0 con 0x55dccf2ab4d0 2024-05-09T04:53:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.265+0000 7f7716992740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55dccf5700e0 con 0x55dccf2cc1b0 2024-05-09T04:53:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.265+0000 7f7716992740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dccf5d0780 con 0x55dccf2a32f0 2024-05-09T04:53:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.266+0000 7f770dfa2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf670100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.266+0000 7f770dfa2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf670100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46834/0 (socket says 172.21.15.46:46834) 2024-05-09T04:53:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.266+0000 7f770dfa2640 1 -- 172.21.15.46:0/3927704303 learned_addr learned my addr 172.21.15.46:0/3927704303 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.266+0000 7f770cfa0640 1 --2- 172.21.15.46:0/3927704303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2a32f0 0x55dccf66fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.267 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.266+0000 7f770d7a1640 1 --2- 172.21.15.46:0/3927704303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2cc1b0 0x55dccf66f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.266+0000 7f770dfa2640 1 -- 172.21.15.46:0/3927704303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2a32f0 msgr2=0x55dccf66fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.266+0000 7f770dfa2640 1 --2- 172.21.15.46:0/3927704303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2a32f0 0x55dccf66fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.266+0000 7f770dfa2640 1 -- 172.21.15.46:0/3927704303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2cc1b0 msgr2=0x55dccf66f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.266+0000 7f770dfa2640 1 --2- 172.21.15.46:0/3927704303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2cc1b0 0x55dccf66f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.266+0000 7f770dfa2640 1 -- 172.21.15.46:0/3927704303 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dccf5997c0 con 0x55dccf2ab4d0 2024-05-09T04:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.266+0000 7f770d7a1640 1 --2- 172.21.15.46:0/3927704303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2cc1b0 0x55dccf66f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.266+0000 7f770dfa2640 1 --2- 172.21.15.46:0/3927704303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf670100 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f77000a5a80 tx=0x7f77000a5250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcad5cd5c0a47c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.266+0000 7f76f37fe640 1 -- 172.21.15.46:0/3927704303 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770001f040 con 0x55dccf2ab4d0 2024-05-09T04:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.266+0000 7f76f37fe640 1 -- 172.21.15.46:0/3927704303 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77000b2100 con 0x55dccf2ab4d0 2024-05-09T04:53:43.268 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.266+0000 7f76f37fe640 1 -- 172.21.15.46:0/3927704303 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77000cc0d0 con 0x55dccf2ab4d0 2024-05-09T04:53:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.268+0000 7f7716992740 1 -- 172.21.15.46:0/3927704303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 msgr2=0x55dccf670100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.268+0000 7f7716992740 1 --2- 172.21.15.46:0/3927704303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf670100 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f77000a5a80 tx=0x7f77000a5250 comp rx=0 tx=0).stop 2024-05-09T04:53:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.268+0000 7f7716992740 1 -- 172.21.15.46:0/3927704303 shutdown_connections 2024-05-09T04:53:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.268+0000 7f7716992740 1 --2- 172.21.15.46:0/3927704303 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2a32f0 0x55dccf66fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.268+0000 7f7716992740 1 --2- 172.21.15.46:0/3927704303 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf670100 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.268+0000 7f7716992740 1 --2- 172.21.15.46:0/3927704303 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2cc1b0 0x55dccf66f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.268+0000 7f7716992740 1 -- 172.21.15.46:0/3927704303 >> 172.21.15.46:0/3927704303 conn(0x55dccf2c0300 msgr2=0x55dccf66be30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:43.273 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.268+0000 7f7716992740 1 -- 172.21.15.46:0/3927704303 shutdown_connections 2024-05-09T04:53:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.268+0000 7f7716992740 1 -- 172.21.15.46:0/3927704303 wait complete. 2024-05-09T04:53:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.268+0000 7f7716992740 1 -- start start 2024-05-09T04:53:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.268+0000 7f7716992740 1 Processor -- start 2024-05-09T04:53:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.269+0000 7f7716992740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2a32f0 0x55dccf67b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.269+0000 7f7716992740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf67bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.269+0000 7f7716992740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2cc1b0 0x55dccf679c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.269+0000 7f7716992740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dccf67c750 con 0x55dccf2ab4d0 2024-05-09T04:53:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.269+0000 7f7716992740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55dccf55b9c0 con 0x55dccf2a32f0 2024-05-09T04:53:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.269+0000 7f7716992740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dccf67c8d0 con 0x55dccf2cc1b0 2024-05-09T04:53:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.269+0000 7f7716992740 10 client.0 _ll_register_callbacks cb 0x55dccf588020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.269+0000 7f770cfa0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf67bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.269+0000 7f770cfa0640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf67bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46840/0 (socket says 172.21.15.46:46840) 2024-05-09T04:53:43.274 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.269+0000 7f770d7a1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2a32f0 0x55dccf67b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.269+0000 7f770dfa2640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2cc1b0 0x55dccf679c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.269+0000 7f770cfa0640 1 -- 172.21.15.46:0/45212225 learned_addr learned my addr 172.21.15.46:0/45212225 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.270+0000 7f770dfa2640 1 -- 172.21.15.46:0/45212225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2a32f0 msgr2=0x55dccf67b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.270+0000 7f770dfa2640 1 --2- 172.21.15.46:0/45212225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2a32f0 0x55dccf67b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.270+0000 7f770dfa2640 1 -- 172.21.15.46:0/45212225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 msgr2=0x55dccf67bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.270+0000 7f770dfa2640 1 --2- 172.21.15.46:0/45212225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf67bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.270+0000 7f770dfa2640 1 -- 172.21.15.46:0/45212225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dccf5821f0 con 0x55dccf2cc1b0 2024-05-09T04:53:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.270+0000 7f770cfa0640 1 --2- 172.21.15.46:0/45212225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf67bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.270+0000 7f770dfa2640 1 --2- 172.21.15.46:0/45212225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2cc1b0 0x55dccf679c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f77000a9990 tx=0x7f77000a97d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ded1ac2e33bb05fd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.270+0000 7f76f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:53:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.270+0000 7f76f0ff9640 1 -- 172.21.15.46:0/45212225 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770001f040 con 0x55dccf2cc1b0 2024-05-09T04:53:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.270+0000 7f76f0ff9640 1 -- 172.21.15.46:0/45212225 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f770000cb20 con 0x55dccf2cc1b0 2024-05-09T04:53:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.270+0000 7f7716992740 1 -- 172.21.15.46:0/45212225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77000b2100 con 0x55dccf2cc1b0 2024-05-09T04:53:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.270+0000 7f76f0ff9640 1 -- 172.21.15.46:0/45212225 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f770000cd20 con 0x55dccf2cc1b0 2024-05-09T04:53:43.275 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.270+0000 7f7716992740 10 client.25609 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.270+0000 7f7716992740 1 -- 172.21.15.46:0/45212225 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dccf337790 con 0x55dccf2cc1b0 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.270+0000 7f76f0ff9640 1 -- 172.21.15.46:0/45212225 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77000b2100 con 0x55dccf2cc1b0 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.271+0000 7f76f0ff9640 1 -- 172.21.15.46:0/45212225 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f770000f040 con 0x55dccf2cc1b0 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.271+0000 7f7716992740 10 client.25609 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.271+0000 7f7716992740 10 client.25609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.271+0000 7f7716992740 20 client.25609 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.271+0000 7f7716992740 1 --2- 172.21.15.46:0/45212225 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dccf28ad80 0x55dccf68be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.271+0000 7f7716992740 4 client.25609 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.271+0000 7f7716992740 1 -- 172.21.15.46:0/45212225 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dccf5e23d0 con 0x55dccf28ad80 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.271+0000 7f770d7a1640 1 --2- 172.21.15.46:0/45212225 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dccf28ad80 0x55dccf68be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.272+0000 7f76f0ff9640 10 client.25609 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.272+0000 7f770d7a1640 1 --2- 172.21.15.46:0/45212225 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dccf28ad80 0x55dccf68be40 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2915b6d47cd6445e server_cookie=e85fde6aba8a8ac7 in_seq=0 out_seq=0 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.272+0000 7f76f0ff9640 1 -- 172.21.15.46:0/45212225 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dccf5e23d0 con 0x55dccf28ad80 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.272+0000 7f76f0ff9640 10 client.25609 handle_command_reply: tid=0 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.273+0000 7f7716992740 1 client.25609 shutdown 2024-05-09T04:53:43.276 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.273+0000 7f7716992740 1 -- 172.21.15.46:0/45212225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2cc1b0 msgr2=0x55dccf679c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.273+0000 7f7716992740 1 --2- 172.21.15.46:0/45212225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2cc1b0 0x55dccf679c20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f77000a9990 tx=0x7f77000a97d0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.273+0000 7f7716992740 1 -- 172.21.15.46:0/45212225 shutdown_connections 2024-05-09T04:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.273+0000 7f7716992740 1 --2- 172.21.15.46:0/45212225 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dccf28ad80 0x55dccf68be40 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.273+0000 7f7716992740 1 --2- 172.21.15.46:0/45212225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2cc1b0 0x55dccf679c20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.273+0000 7f76f0ff9640 0 client.25609 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.273+0000 7f7716992740 1 --2- 172.21.15.46:0/45212225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf67bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.273+0000 7f7716992740 1 --2- 172.21.15.46:0/45212225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2a32f0 0x55dccf67b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.273+0000 7f7716992740 1 -- 172.21.15.46:0/45212225 >> 172.21.15.46:0/45212225 conn(0x55dccf2c0300 msgr2=0x55dccf66a700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.273+0000 7f7716992740 1 -- 172.21.15.46:0/45212225 shutdown_connections 2024-05-09T04:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.273+0000 7f7716992740 1 -- 172.21.15.46:0/45212225 wait complete. 2024-05-09T04:53:43.277 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.273+0000 7f7716992740 20 client.25609 trim_cache size 0 max 16384 2024-05-09T04:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.308+0000 7f7716992740 1 Processor -- start 2024-05-09T04:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.308+0000 7f7716992740 1 -- start start 2024-05-09T04:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f7716992740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf28ad80 0x55dccf66be90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f7716992740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2a32f0 0x55dccf68ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.311 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f7716992740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2ab4d0 0x55dccf68f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f7716992740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77081b24e0 con 0x55dccf28ad80 2024-05-09T04:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f7716992740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f77081afdb0 con 0x55dccf2ab4d0 2024-05-09T04:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f7716992740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f77081adc00 con 0x55dccf2a32f0 2024-05-09T04:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f770d7a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf28ad80 0x55dccf66be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f770d7a1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf28ad80 0x55dccf66be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46852/0 (socket says 172.21.15.46:46852) 2024-05-09T04:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f770d7a1640 1 -- 172.21.15.46:0/892499092 learned_addr learned my addr 172.21.15.46:0/892499092 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f770cfa0640 1 --2- 172.21.15.46:0/892499092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2a32f0 0x55dccf68ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f770dfa2640 1 --2- 172.21.15.46:0/892499092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2ab4d0 0x55dccf68f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f770d7a1640 1 -- 172.21.15.46:0/892499092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2a32f0 msgr2=0x55dccf68ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:43.312 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f770d7a1640 1 --2- 172.21.15.46:0/892499092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2a32f0 0x55dccf68ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f770d7a1640 1 -- 172.21.15.46:0/892499092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2ab4d0 msgr2=0x55dccf68f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f770d7a1640 1 --2- 172.21.15.46:0/892499092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2ab4d0 0x55dccf68f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f770d7a1640 1 -- 172.21.15.46:0/892499092 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dccf6754e0 con 0x55dccf28ad80 2024-05-09T04:53:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f770cfa0640 1 --2- 172.21.15.46:0/892499092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2a32f0 0x55dccf68ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.309+0000 7f770dfa2640 1 --2- 172.21.15.46:0/892499092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2ab4d0 0x55dccf68f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:43.313 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.310+0000 7f770d7a1640 1 --2- 172.21.15.46:0/892499092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf28ad80 0x55dccf66be90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x55dccf66c7f0 tx=0x7f7704014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a899020b0b98bd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.310+0000 7f76f2ffd640 1 -- 172.21.15.46:0/892499092 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704030070 con 0x55dccf28ad80 2024-05-09T04:53:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.310+0000 7f76f2ffd640 1 -- 172.21.15.46:0/892499092 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f770402c8e0 con 0x55dccf28ad80 2024-05-09T04:53:43.314 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.310+0000 7f76f2ffd640 1 -- 172.21.15.46:0/892499092 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7704029400 con 0x55dccf28ad80 2024-05-09T04:53:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.311+0000 7f7716992740 1 -- 172.21.15.46:0/892499092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf28ad80 msgr2=0x55dccf66be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.311+0000 7f7716992740 1 --2- 172.21.15.46:0/892499092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf28ad80 0x55dccf66be90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x55dccf66c7f0 tx=0x7f7704014000 comp rx=0 tx=0).stop 2024-05-09T04:53:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.311+0000 7f7716992740 1 -- 172.21.15.46:0/892499092 shutdown_connections 2024-05-09T04:53:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.311+0000 7f7716992740 1 --2- 172.21.15.46:0/892499092 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf2a32f0 0x55dccf68ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.311+0000 7f7716992740 1 --2- 172.21.15.46:0/892499092 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2ab4d0 0x55dccf68f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.311+0000 7f7716992740 1 --2- 172.21.15.46:0/892499092 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf28ad80 0x55dccf66be90 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.311+0000 7f7716992740 1 -- 172.21.15.46:0/892499092 >> 172.21.15.46:0/892499092 conn(0x55dccf2c0300 msgr2=0x55dccf66b780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.311+0000 7f7716992740 1 -- 172.21.15.46:0/892499092 shutdown_connections 2024-05-09T04:53:43.318 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.311+0000 7f7716992740 1 -- 172.21.15.46:0/892499092 wait complete. 2024-05-09T04:53:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.312+0000 7f7716992740 1 -- start start 2024-05-09T04:53:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.312+0000 7f7716992740 1 Processor -- start 2024-05-09T04:53:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f7716992740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf28ad80 0x55dccf68efd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f7716992740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2a32f0 0x55dccf68f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f7716992740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf68c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f7716992740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f77081ae270 con 0x55dccf2ab4d0 2024-05-09T04:53:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f7716992740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f77081af6e0 con 0x55dccf2a32f0 2024-05-09T04:53:43.319 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f7716992740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f77081ae940 con 0x55dccf28ad80 2024-05-09T04:53:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f7716992740 10 client.0 _ll_register_callbacks cb 0x7f77000b2050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f770dfa2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf68c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f770d7a1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf28ad80 0x55dccf68efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f770dfa2640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf68c450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46854/0 (socket says 172.21.15.46:46854) 2024-05-09T04:53:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f770dfa2640 1 -- 172.21.15.46:0/225361718 learned_addr learned my addr 172.21.15.46:0/225361718 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f770cfa0640 1 --2- 172.21.15.46:0/225361718 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2a32f0 0x55dccf68f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f770d7a1640 1 -- 172.21.15.46:0/225361718 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2a32f0 msgr2=0x55dccf68f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f770d7a1640 1 --2- 172.21.15.46:0/225361718 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2a32f0 0x55dccf68f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f770d7a1640 1 -- 172.21.15.46:0/225361718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 msgr2=0x55dccf68c450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.313+0000 7f770d7a1640 1 --2- 172.21.15.46:0/225361718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf68c450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.314+0000 7f770d7a1640 1 -- 172.21.15.46:0/225361718 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dccf67bb80 con 0x55dccf28ad80 2024-05-09T04:53:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.314+0000 7f770d7a1640 1 --2- 172.21.15.46:0/225361718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf28ad80 0x55dccf68efd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55dccf66c7f0 tx=0x7f7704011e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ead0d1893b996f99 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.314+0000 7f76f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:53:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.314+0000 7f76f0ff9640 1 -- 172.21.15.46:0/225361718 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770403a030 con 0x55dccf28ad80 2024-05-09T04:53:43.320 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.314+0000 7f76f0ff9640 1 -- 172.21.15.46:0/225361718 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f770400e4f0 con 0x55dccf28ad80 2024-05-09T04:53:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.314+0000 7f7716992740 1 -- 172.21.15.46:0/225361718 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f770402c8e0 con 0x55dccf28ad80 2024-05-09T04:53:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.314+0000 7f76f0ff9640 1 -- 172.21.15.46:0/225361718 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7704044050 con 0x55dccf28ad80 2024-05-09T04:53:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.314+0000 7f7716992740 10 client.25615 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.314+0000 7f7716992740 1 -- 172.21.15.46:0/225361718 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dccf3ae5f0 con 0x55dccf28ad80 2024-05-09T04:53:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.315+0000 7f76f0ff9640 1 -- 172.21.15.46:0/225361718 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f770402c8e0 con 0x55dccf28ad80 2024-05-09T04:53:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.315+0000 7f76f0ff9640 1 -- 172.21.15.46:0/225361718 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7704005040 con 0x55dccf28ad80 2024-05-09T04:53:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.315+0000 7f7716992740 10 client.25615 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.315+0000 7f7716992740 10 client.25615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.315+0000 7f7716992740 20 client.25615 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:43.321 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.315+0000 7f7716992740 1 --2- 172.21.15.46:0/225361718 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dccf2cc1b0 0x55dccf6fa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.315+0000 7f7716992740 4 client.25615 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T04:53:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.315+0000 7f7716992740 1 -- 172.21.15.46:0/225361718 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55dccf5dc240 con 0x55dccf2cc1b0 2024-05-09T04:53:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.315+0000 7f770cfa0640 1 --2- 172.21.15.46:0/225361718 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dccf2cc1b0 0x55dccf6fa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.316+0000 7f76f0ff9640 10 client.25615 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:43.322 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.316+0000 7f770cfa0640 1 --2- 172.21.15.46:0/225361718 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dccf2cc1b0 0x55dccf6fa0a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b920d2cccf5ea158 server_cookie=145c269f8a671f24 in_seq=0 out_seq=0 2024-05-09T04:53:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.329+0000 7f76f0ff9640 1 -- 172.21.15.46:0/225361718 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x55dccf5dc240 con 0x55dccf2cc1b0 2024-05-09T04:53:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.329+0000 7f76f0ff9640 10 client.25615 handle_command_reply: tid=0 2024-05-09T04:53:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.329+0000 7f7716992740 1 client.25615 shutdown 2024-05-09T04:53:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.329+0000 7f7716992740 1 -- 172.21.15.46:0/225361718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf28ad80 msgr2=0x55dccf68efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.329+0000 7f7716992740 1 --2- 172.21.15.46:0/225361718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf28ad80 0x55dccf68efd0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x55dccf66c7f0 tx=0x7f7704011e70 comp rx=0 tx=0).stop 2024-05-09T04:53:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.329+0000 7f7716992740 1 -- 172.21.15.46:0/225361718 shutdown_connections 2024-05-09T04:53:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.329+0000 7f7716992740 1 --2- 172.21.15.46:0/225361718 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dccf2cc1b0 0x55dccf6fa0a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.329+0000 7f7716992740 1 --2- 172.21.15.46:0/225361718 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dccf2ab4d0 0x55dccf68c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.329+0000 7f76f0ff9640 0 client.25615 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.329+0000 7f7716992740 1 --2- 172.21.15.46:0/225361718 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dccf2a32f0 0x55dccf68f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.329+0000 7f7716992740 1 --2- 172.21.15.46:0/225361718 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dccf28ad80 0x55dccf68efd0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.329+0000 7f7716992740 1 -- 172.21.15.46:0/225361718 >> 172.21.15.46:0/225361718 conn(0x55dccf2c0300 msgr2=0x55dccf6712c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.329+0000 7f7716992740 1 -- 172.21.15.46:0/225361718 shutdown_connections 2024-05-09T04:53:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.329+0000 7f7716992740 1 -- 172.21.15.46:0/225361718 wait complete. 2024-05-09T04:53:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.329+0000 7f7716992740 20 client.25615 trim_cache size 0 max 16384 2024-05-09T04:53:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-05-09T04:53:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-05-09T04:53:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-05-09T04:53:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.330+0000 7f770fa05640 1 -- 172.21.15.46:0/763910726 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f76f40b2950 msgr2=0x7f76f40b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.330+0000 7f770fa05640 1 --2- 172.21.15.46:0/763910726 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f76f40b2950 0x7f76f40b4e60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7704009c70 tx=0x7f7704005f90 comp rx=0 tx=0).stop 2024-05-09T04:53:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.330+0000 7f770fa05640 1 -- 172.21.15.46:0/763910726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7708107e60 msgr2=0x7f770819a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.330+0000 7f770fa05640 1 --2- 172.21.15.46:0/763910726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7708107e60 0x7f770819a6d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f770000e940 tx=0x7f770000ee00 comp rx=0 tx=0).stop 2024-05-09T04:53:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.330+0000 7f770fa05640 1 -- 172.21.15.46:0/763910726 shutdown_connections 2024-05-09T04:53:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.330+0000 7f770fa05640 1 --2- 172.21.15.46:0/763910726 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f76f40b2950 0x7f76f40b4e60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.330+0000 7f770fa05640 1 --2- 172.21.15.46:0/763910726 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f77081073e0 0x7f77081a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.330+0000 7f770fa05640 1 --2- 172.21.15.46:0/763910726 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7708107e60 0x7f770819a6d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.330+0000 7f770fa05640 1 --2- 172.21.15.46:0/763910726 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7708103520 0x7f77081a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.330+0000 7f770fa05640 1 -- 172.21.15.46:0/763910726 >> 172.21.15.46:0/763910726 conn(0x7f77080779e0 msgr2=0x7f7708102230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.330+0000 7f770fa05640 1 -- 172.21.15.46:0/763910726 shutdown_connections 2024-05-09T04:53:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:43.330+0000 7f770fa05640 1 -- 172.21.15.46:0/763910726 wait complete. 2024-05-09T04:53:43.688 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:53:43.688 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:53:44.248 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:45.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:44 smithi110 ceph-mon[34904]: pgmap v82: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 613 MiB/s wr, 262 op/s 2024-05-09T04:53:45.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:44 smithi046 ceph-mon[30136]: pgmap v82: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 613 MiB/s wr, 262 op/s 2024-05-09T04:53:45.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:44 smithi072 ceph-mon[34088]: pgmap v82: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 613 MiB/s wr, 262 op/s 2024-05-09T04:53:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:46 smithi110 ceph-mon[34904]: pgmap v83: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 613 MiB/s wr, 262 op/s 2024-05-09T04:53:47.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:46 smithi046 ceph-mon[30136]: pgmap v83: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 613 MiB/s wr, 262 op/s 2024-05-09T04:53:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:46 smithi072 ceph-mon[34088]: pgmap v83: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 613 MiB/s wr, 262 op/s 2024-05-09T04:53:48.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:48 smithi046 ceph-mon[30136]: pgmap v84: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 823 MiB/s wr, 385 op/s 2024-05-09T04:53:49.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:48 smithi110 ceph-mon[34904]: pgmap v84: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 823 MiB/s wr, 385 op/s 2024-05-09T04:53:49.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:48 smithi072 ceph-mon[34088]: pgmap v84: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 412 KiB/s rd, 823 MiB/s wr, 385 op/s 2024-05-09T04:53:49.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.637+0000 7f9822f66700 1 -- 172.21.15.46:0/1093488222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c107df0 msgr2=0x7f981c108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:49.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.637+0000 7f9822f66700 1 --2- 172.21.15.46:0/1093488222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c107df0 0x7f981c108200 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f98080099c0 tx=0x7f9808009cd0 comp rx=0 tx=0).stop 2024-05-09T04:53:49.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.637+0000 7f9822f66700 1 -- 172.21.15.46:0/1093488222 shutdown_connections 2024-05-09T04:53:49.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.637+0000 7f9822f66700 1 --2- 172.21.15.46:0/1093488222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981c108740 0x7f981c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:49.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.637+0000 7f9822f66700 1 --2- 172.21.15.46:0/1093488222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f981c1034c0 0x7f981c1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:49.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.638+0000 7f9822f66700 1 --2- 172.21.15.46:0/1093488222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c107df0 0x7f981c108200 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:49.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.638+0000 7f9822f66700 1 -- 172.21.15.46:0/1093488222 >> 172.21.15.46:0/1093488222 conn(0x7f981c0fedc0 msgr2=0x7f981c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:49.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.638+0000 7f9822f66700 1 -- 172.21.15.46:0/1093488222 shutdown_connections 2024-05-09T04:53:49.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.638+0000 7f9822f66700 1 -- 172.21.15.46:0/1093488222 wait complete. 2024-05-09T04:53:49.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.639+0000 7f9822f66700 1 Processor -- start 2024-05-09T04:53:49.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.639+0000 7f9822f66700 1 -- start start 2024-05-09T04:53:49.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.639+0000 7f9822f66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c1034c0 0x7f981c1a2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:49.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.639+0000 7f9822f66700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981c107df0 0x7f981c1a2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:49.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.639+0000 7f9822f66700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f981c108740 0x7f981c1ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:49.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.639+0000 7f9822f66700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f981c1145a0 con 0x7f981c1034c0 2024-05-09T04:53:49.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.639+0000 7f9822f66700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f981c114420 con 0x7f981c107df0 2024-05-09T04:53:49.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.639+0000 7f9822f66700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f981c114720 con 0x7f981c108740 2024-05-09T04:53:49.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.639+0000 7f9820d02700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c1034c0 0x7f981c1a2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:49.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.640+0000 7f9820d02700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c1034c0 0x7f981c1a2430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57330/0 (socket says 172.21.15.46:57330) 2024-05-09T04:53:49.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.640+0000 7f9820d02700 1 -- 172.21.15.46:0/1647446570 learned_addr learned my addr 172.21.15.46:0/1647446570 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:49.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.640+0000 7f9820d02700 1 -- 172.21.15.46:0/1647446570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f981c108740 msgr2=0x7f981c1ad2e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T04:53:49.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.640+0000 7f9820d02700 1 --2- 172.21.15.46:0/1647446570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f981c108740 0x7f981c1ad2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:49.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.640+0000 7f9820d02700 1 -- 172.21.15.46:0/1647446570 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981c107df0 msgr2=0x7f981c1a2970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-09T04:53:49.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.640+0000 7f9820d02700 1 --2- 172.21.15.46:0/1647446570 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981c107df0 0x7f981c1a2970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:49.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.640+0000 7f9820d02700 1 -- 172.21.15.46:0/1647446570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f981c1ad820 con 0x7f981c1034c0 2024-05-09T04:53:49.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.640+0000 7f9820d02700 1 --2- 172.21.15.46:0/1647446570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c1034c0 0x7f981c1a2430 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f981000b550 tx=0x7f981000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:49.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.641+0000 7f9819ffb700 1 -- 172.21.15.46:0/1647446570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9810015020 con 0x7f981c1034c0 2024-05-09T04:53:49.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.641+0000 7f9822f66700 1 -- 172.21.15.46:0/1647446570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f981c1adab0 con 0x7f981c1034c0 2024-05-09T04:53:49.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.641+0000 7f9822f66700 1 -- 172.21.15.46:0/1647446570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f981c1adff0 con 0x7f981c1034c0 2024-05-09T04:53:49.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.641+0000 7f9819ffb700 1 -- 172.21.15.46:0/1647446570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9810011ce0 con 0x7f981c1034c0 2024-05-09T04:53:49.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.641+0000 7f9819ffb700 1 -- 172.21.15.46:0/1647446570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9810010d20 con 0x7f981c1034c0 2024-05-09T04:53:49.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.643+0000 7f98077fe700 1 -- 172.21.15.46:0/1647446570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97fc005330 con 0x7f981c1034c0 2024-05-09T04:53:49.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.645+0000 7f9819ffb700 1 -- 172.21.15.46:0/1647446570 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9810018020 con 0x7f981c1034c0 2024-05-09T04:53:49.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.646+0000 7f9819ffb700 1 --2- 172.21.15.46:0/1647446570 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f980c0b2af0 0x7f980c0b5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:49.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.646+0000 7f9819ffb700 1 -- 172.21.15.46:0/1647446570 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f98100e2360 con 0x7f981c1034c0 2024-05-09T04:53:49.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.646+0000 7f981bfff700 1 --2- 172.21.15.46:0/1647446570 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f980c0b2af0 0x7f980c0b5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:49.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.647+0000 7f981bfff700 1 --2- 172.21.15.46:0/1647446570 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f980c0b2af0 0x7f980c0b5000 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9808009750 tx=0x7f9808005fd0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:49.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:49.652+0000 7f9819ffb700 1 -- 172.21.15.46:0/1647446570 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f981001f020 con 0x7f981c1034c0 2024-05-09T04:53:50.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:50.100+0000 7f98077fe700 1 -- 172.21.15.46:0/1647446570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97fc005620 con 0x7f981c1034c0 2024-05-09T04:53:50.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:50.101+0000 7f9819ffb700 1 -- 172.21.15.46:0/1647446570 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7f98100a9dd0 con 0x7f981c1034c0 2024-05-09T04:53:50.102 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:53:50.103 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:53:50.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:50.105+0000 7f98077fe700 1 -- 172.21.15.46:0/1647446570 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f980c0b2af0 msgr2=0x7f980c0b5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:50.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:50.105+0000 7f98077fe700 1 --2- 172.21.15.46:0/1647446570 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f980c0b2af0 0x7f980c0b5000 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f9808009750 tx=0x7f9808005fd0 comp rx=0 tx=0).stop 2024-05-09T04:53:50.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:50.105+0000 7f98077fe700 1 -- 172.21.15.46:0/1647446570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c1034c0 msgr2=0x7f981c1a2430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:50.106 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:50.105+0000 7f98077fe700 1 --2- 172.21.15.46:0/1647446570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c1034c0 0x7f981c1a2430 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f981000b550 tx=0x7f981000b910 comp rx=0 tx=0).stop 2024-05-09T04:53:50.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:50.106+0000 7f98077fe700 1 -- 172.21.15.46:0/1647446570 shutdown_connections 2024-05-09T04:53:50.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:50.106+0000 7f98077fe700 1 --2- 172.21.15.46:0/1647446570 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f980c0b2af0 0x7f980c0b5000 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:50.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:50.106+0000 7f98077fe700 1 --2- 172.21.15.46:0/1647446570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f981c108740 0x7f981c1ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:50.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:50.106+0000 7f98077fe700 1 --2- 172.21.15.46:0/1647446570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f981c1034c0 0x7f981c1a2430 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:50.107 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:50.106+0000 7f98077fe700 1 --2- 172.21.15.46:0/1647446570 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f981c107df0 0x7f981c1a2970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:50.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:50.106+0000 7f98077fe700 1 -- 172.21.15.46:0/1647446570 >> 172.21.15.46:0/1647446570 conn(0x7f981c0fedc0 msgr2=0x7f981c109740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:50.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:50.106+0000 7f98077fe700 1 -- 172.21.15.46:0/1647446570 shutdown_connections 2024-05-09T04:53:50.108 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:50.106+0000 7f98077fe700 1 -- 172.21.15.46:0/1647446570 wait complete. 2024-05-09T04:53:50.108 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:53:50.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:50 smithi046 ceph-mon[30136]: pgmap v85: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 416 MiB/s wr, 179 op/s 2024-05-09T04:53:50.879 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:50 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1647446570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:51.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:50 smithi110 ceph-mon[34904]: pgmap v85: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 416 MiB/s wr, 179 op/s 2024-05-09T04:53:51.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:50 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1647446570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:51.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:50 smithi072 ceph-mon[34088]: pgmap v85: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 174 KiB/s rd, 416 MiB/s wr, 179 op/s 2024-05-09T04:53:51.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:50 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1647446570' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:53:51.429 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:53:52.164 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:52.995 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:52 smithi046 ceph-mon[30136]: pgmap v86: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 615 MiB/s wr, 295 op/s 2024-05-09T04:53:53.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:52 smithi110 ceph-mon[34904]: pgmap v86: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 615 MiB/s wr, 295 op/s 2024-05-09T04:53:53.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:52 smithi072 ceph-mon[34088]: pgmap v86: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 615 MiB/s wr, 295 op/s 2024-05-09T04:53:54.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.393+0000 7f43eafbe700 1 -- 172.21.15.46:0/3507218755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e41027e0 msgr2=0x7f43e4102bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:54.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.393+0000 7f43eafbe700 1 --2- 172.21.15.46:0/3507218755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e41027e0 0x7f43e4102bf0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009990 tx=0x7f43d0009ca0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.394+0000 7f43eafbe700 1 -- 172.21.15.46:0/3507218755 shutdown_connections 2024-05-09T04:53:54.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.394+0000 7f43eafbe700 1 --2- 172.21.15.46:0/3507218755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e4103130 0x7f43e4111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.394+0000 7f43eafbe700 1 --2- 172.21.15.46:0/3507218755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43e410a3e0 0x7f43e410a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.394+0000 7f43eafbe700 1 --2- 172.21.15.46:0/3507218755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e41027e0 0x7f43e4102bf0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.394+0000 7f43eafbe700 1 -- 172.21.15.46:0/3507218755 >> 172.21.15.46:0/3507218755 conn(0x7f43e40fe6f0 msgr2=0x7f43e4100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:54.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.394+0000 7f43eafbe700 1 -- 172.21.15.46:0/3507218755 shutdown_connections 2024-05-09T04:53:54.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.394+0000 7f43eafbe700 1 -- 172.21.15.46:0/3507218755 wait complete. 2024-05-09T04:53:54.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.394+0000 7f43eafbe700 1 Processor -- start 2024-05-09T04:53:54.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.395+0000 7f43eafbe700 1 -- start start 2024-05-09T04:53:54.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.395+0000 7f43eafbe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e41027e0 0x7f43e41039c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.395+0000 7f43eafbe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43e4103130 0x7f43e4103f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.395+0000 7f43eafbe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e410a3e0 0x7f43e4104440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.395+0000 7f43eafbe700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4114590 con 0x7f43e41027e0 2024-05-09T04:53:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.395+0000 7f43eafbe700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4114410 con 0x7f43e410a3e0 2024-05-09T04:53:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.395+0000 7f43eafbe700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f43e4114710 con 0x7f43e4103130 2024-05-09T04:53:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.395+0000 7f43e8d5a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e41027e0 0x7f43e41039c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.395+0000 7f43e8d5a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e41027e0 0x7f43e41039c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46762/0 (socket says 172.21.15.46:46762) 2024-05-09T04:53:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.395+0000 7f43e8d5a700 1 -- 172.21.15.46:0/195423265 learned_addr learned my addr 172.21.15.46:0/195423265 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.396+0000 7f43e3fff700 1 --2- 172.21.15.46:0/195423265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43e4103130 0x7f43e4103f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.396+0000 7f43e8d5a700 1 -- 172.21.15.46:0/195423265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43e4103130 msgr2=0x7f43e4103f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:54.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.396+0000 7f43e8d5a700 1 --2- 172.21.15.46:0/195423265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43e4103130 0x7f43e4103f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.396+0000 7f43e8d5a700 1 -- 172.21.15.46:0/195423265 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e410a3e0 msgr2=0x7f43e4104440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.396+0000 7f43e8d5a700 1 --2- 172.21.15.46:0/195423265 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e410a3e0 0x7f43e4104440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.396+0000 7f43e8d5a700 1 -- 172.21.15.46:0/195423265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43e41a9360 con 0x7f43e41027e0 2024-05-09T04:53:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.396+0000 7f43e8d5a700 1 --2- 172.21.15.46:0/195423265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e41027e0 0x7f43e41039c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f43d800b550 tx=0x7f43d800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.396+0000 7f43e1ffb700 1 -- 172.21.15.46:0/195423265 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8015020 con 0x7f43e41027e0 2024-05-09T04:53:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.396+0000 7f43e1ffb700 1 -- 172.21.15.46:0/195423265 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d8011ce0 con 0x7f43e41027e0 2024-05-09T04:53:54.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.396+0000 7f43e1ffb700 1 -- 172.21.15.46:0/195423265 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d8010d20 con 0x7f43e41027e0 2024-05-09T04:53:54.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.396+0000 7f43eafbe700 1 -- 172.21.15.46:0/195423265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43e41a9650 con 0x7f43e41027e0 2024-05-09T04:53:54.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.397+0000 7f43eafbe700 1 -- 172.21.15.46:0/195423265 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43e41a9be0 con 0x7f43e41027e0 2024-05-09T04:53:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.401+0000 7f43f99d4b80 1 Processor -- start 2024-05-09T04:53:54.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.402+0000 7f43f99d4b80 1 -- start start 2024-05-09T04:53:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.402+0000 7f43e1ffb700 1 -- 172.21.15.46:0/195423265 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f43d8018020 con 0x7f43e41027e0 2024-05-09T04:53:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.402+0000 7f43f99d4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf76e8160 0x55ccf77814f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.402+0000 7f43f99d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf75fe4a0 0x55ccf7781a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.402+0000 7f43f99d4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf777b6a0 0x55ccf777bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.402+0000 7f43f99d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ccf769dff0 con 0x55ccf75fe4a0 2024-05-09T04:53:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.402+0000 7f43f99d4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ccf763c7f0 con 0x55ccf777b6a0 2024-05-09T04:53:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.402+0000 7f43f99d4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ccf7699fb0 con 0x55ccf76e8160 2024-05-09T04:53:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.402+0000 7f43e8d5a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf75fe4a0 0x55ccf7781a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.402+0000 7f43e8d5a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf75fe4a0 0x55ccf7781a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46774/0 (socket says 172.21.15.46:46774) 2024-05-09T04:53:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.402+0000 7f43e8d5a700 1 -- 172.21.15.46:0/151280001 learned_addr learned my addr 172.21.15.46:0/151280001 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:54.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.403+0000 7f43e955b700 1 --2- 172.21.15.46:0/151280001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf76e8160 0x55ccf77814f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:54.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.403+0000 7f43e8d5a700 1 -- 172.21.15.46:0/151280001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf76e8160 msgr2=0x55ccf77814f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:54.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.403+0000 7f43e8d5a700 1 --2- 172.21.15.46:0/151280001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf76e8160 0x55ccf77814f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.403+0000 7f43e8d5a700 1 -- 172.21.15.46:0/151280001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf777b6a0 msgr2=0x55ccf777bad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:53:54.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.403+0000 7f43e8d5a700 1 --2- 172.21.15.46:0/151280001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf777b6a0 0x55ccf777bad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.403+0000 7f43e8d5a700 1 -- 172.21.15.46:0/151280001 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccf7634370 con 0x55ccf75fe4a0 2024-05-09T04:53:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.404+0000 7f43e8d5a700 1 --2- 172.21.15.46:0/151280001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf75fe4a0 0x55ccf7781a30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f43d80df4a0 tx=0x7f43d80ea740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4f749855a47e99 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.404+0000 7f43ceffd700 1 -- 172.21.15.46:0/151280001 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d80eae60 con 0x55ccf75fe4a0 2024-05-09T04:53:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.404+0000 7f43ceffd700 1 -- 172.21.15.46:0/151280001 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d80e7d00 con 0x55ccf75fe4a0 2024-05-09T04:53:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.404+0000 7f43ceffd700 1 -- 172.21.15.46:0/151280001 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d80f2cf0 con 0x55ccf75fe4a0 2024-05-09T04:53:54.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.404+0000 7f43e1ffb700 1 --2- 172.21.15.46:0/195423265 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f43d40b2a70 0x7f43d40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.405+0000 7f43e955b700 1 --2- 172.21.15.46:0/195423265 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f43d40b2a70 0x7f43d40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:54.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.405+0000 7f43e1ffb700 1 -- 172.21.15.46:0/195423265 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f43d80e1cf0 con 0x7f43e41027e0 2024-05-09T04:53:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.405+0000 7f43f99d4b80 1 -- 172.21.15.46:0/151280001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf75fe4a0 msgr2=0x55ccf7781a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.405+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/151280001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf75fe4a0 0x55ccf7781a30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f43d80df4a0 tx=0x7f43d80ea740 comp rx=0 tx=0).stop 2024-05-09T04:53:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.405+0000 7f43e955b700 1 --2- 172.21.15.46:0/195423265 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f43d40b2a70 0x7f43d40b4f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009e60 tx=0x7f43dc009450 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.406+0000 7f43f99d4b80 1 -- 172.21.15.46:0/151280001 shutdown_connections 2024-05-09T04:53:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.406+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/151280001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf777b6a0 0x55ccf777bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.406+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/151280001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf76e8160 0x55ccf77814f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.406+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/151280001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf75fe4a0 0x55ccf7781a30 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.406+0000 7f43f99d4b80 1 -- 172.21.15.46:0/151280001 >> 172.21.15.46:0/151280001 conn(0x55ccf765e6a0 msgr2=0x55ccf7776400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:54.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.406+0000 7f43f99d4b80 1 -- 172.21.15.46:0/151280001 shutdown_connections 2024-05-09T04:53:54.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.406+0000 7f43f99d4b80 1 -- 172.21.15.46:0/151280001 wait complete. 2024-05-09T04:53:54.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.406+0000 7f43f99d4b80 1 -- start start 2024-05-09T04:53:54.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.406+0000 7f43f99d4b80 1 Processor -- start 2024-05-09T04:53:54.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.407+0000 7f43f99d4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf75fe4a0 0x55ccf7785110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.407+0000 7f43f99d4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf76e8160 0x55ccf7785650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.407+0000 7f43f99d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf777b6a0 0x55ccf7785b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.407+0000 7f43f99d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ccf7519ba0 con 0x55ccf777b6a0 2024-05-09T04:53:54.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.407+0000 7f43f99d4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ccf7658150 con 0x55ccf76e8160 2024-05-09T04:53:54.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.407+0000 7f43f99d4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ccf777c420 con 0x55ccf75fe4a0 2024-05-09T04:53:54.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.407+0000 7f43f99d4b80 10 client.0 _ll_register_callbacks cb 0x55ccf750f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:54.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.407+0000 7f43e955b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf777b6a0 0x55ccf7785b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:54.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.407+0000 7f43e955b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf777b6a0 0x55ccf7785b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46780/0 (socket says 172.21.15.46:46780) 2024-05-09T04:53:54.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.407+0000 7f43e955b700 1 -- 172.21.15.46:0/2355407417 learned_addr learned my addr 172.21.15.46:0/2355407417 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:54.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.408+0000 7f43e955b700 1 -- 172.21.15.46:0/2355407417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf75fe4a0 msgr2=0x55ccf7785110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:54.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.408+0000 7f43e955b700 1 --2- 172.21.15.46:0/2355407417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf75fe4a0 0x55ccf7785110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.408+0000 7f43e955b700 1 -- 172.21.15.46:0/2355407417 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf76e8160 msgr2=0x55ccf7785650 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:53:54.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.408+0000 7f43e955b700 1 --2- 172.21.15.46:0/2355407417 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf76e8160 0x55ccf7785650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.408+0000 7f43e955b700 1 -- 172.21.15.46:0/2355407417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccf7519490 con 0x55ccf777b6a0 2024-05-09T04:53:54.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.408+0000 7f43e955b700 1 --2- 172.21.15.46:0/2355407417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf777b6a0 0x55ccf7785b90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f43dc017690 tx=0x7f43dc004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b45a8260f2e4193a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:54.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.408+0000 7f43c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:54.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.408+0000 7f43c3fff700 1 -- 172.21.15.46:0/2355407417 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43dc027060 con 0x55ccf777b6a0 2024-05-09T04:53:54.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.409+0000 7f43f99d4b80 1 -- 172.21.15.46:0/2355407417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43d80e7d00 con 0x55ccf777b6a0 2024-05-09T04:53:54.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.409+0000 7f43c3fff700 1 -- 172.21.15.46:0/2355407417 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f43dc012030 con 0x55ccf777b6a0 2024-05-09T04:53:54.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.409+0000 7f43c3fff700 1 -- 172.21.15.46:0/2355407417 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f43dc006e60 con 0x55ccf777b6a0 2024-05-09T04:53:54.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.409+0000 7f43f99d4b80 10 client.16050 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:54.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.409+0000 7f43f99d4b80 1 -- 172.21.15.46:0/2355407417 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ccf7792a90 con 0x55ccf777b6a0 2024-05-09T04:53:54.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.409+0000 7f43c3fff700 1 -- 172.21.15.46:0/2355407417 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d80e7d00 con 0x55ccf777b6a0 2024-05-09T04:53:54.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.409+0000 7f43c3fff700 1 -- 172.21.15.46:0/2355407417 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f43dc016430 con 0x55ccf777b6a0 2024-05-09T04:53:54.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.409+0000 7f43f99d4b80 10 client.16050 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:54.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.410+0000 7f43f99d4b80 10 client.16050 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:53:54.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.410+0000 7f43f99d4b80 20 client.16050 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:54.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.410+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/2355407417 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ccf77950d0 0x55ccf77975a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.410+0000 7f43f99d4b80 4 client.16050 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:54.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.410+0000 7f43f99d4b80 1 -- 172.21.15.46:0/2355407417 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ccf76ab9d0 con 0x55ccf77950d0 2024-05-09T04:53:54.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.410+0000 7f43e8d5a700 1 --2- 172.21.15.46:0/2355407417 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ccf77950d0 0x55ccf77975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:54.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.412+0000 7f43c3fff700 10 client.16050 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:53:54.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.412+0000 7f43e8d5a700 1 --2- 172.21.15.46:0/2355407417 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ccf77950d0 0x55ccf77975a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8d09a66f34f7454c server_cookie=db6e9ca5882569e9 in_seq=0 out_seq=0 2024-05-09T04:53:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.414+0000 7f43c3fff700 1 -- 172.21.15.46:0/2355407417 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ccf76ab9d0 con 0x55ccf77950d0 2024-05-09T04:53:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.414+0000 7f43c3fff700 10 client.16050 handle_command_reply: tid=0 2024-05-09T04:53:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.414+0000 7f43f99d4b80 1 client.16050 shutdown 2024-05-09T04:53:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.414+0000 7f43f99d4b80 1 -- 172.21.15.46:0/2355407417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf777b6a0 msgr2=0x55ccf7785b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.414+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/2355407417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf777b6a0 0x55ccf7785b90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f43dc017690 tx=0x7f43dc004bd0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.415+0000 7f43f99d4b80 1 -- 172.21.15.46:0/2355407417 shutdown_connections 2024-05-09T04:53:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.415+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/2355407417 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf777b6a0 0x55ccf7785b90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.415+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/2355407417 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ccf77950d0 0x55ccf77975a0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.415+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/2355407417 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf75fe4a0 0x55ccf7785110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.415+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/2355407417 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf76e8160 0x55ccf7785650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.415+0000 7f43f99d4b80 1 -- 172.21.15.46:0/2355407417 >> 172.21.15.46:0/2355407417 conn(0x55ccf765e6a0 msgr2=0x55ccf77761c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.415+0000 7f43c3fff700 0 client.16050 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:53:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.415+0000 7f43f99d4b80 1 -- 172.21.15.46:0/2355407417 shutdown_connections 2024-05-09T04:53:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.415+0000 7f43f99d4b80 1 -- 172.21.15.46:0/2355407417 wait complete. 2024-05-09T04:53:54.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.415+0000 7f43f99d4b80 20 client.16050 trim_cache size 0 max 16384 2024-05-09T04:53:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43f99d4b80 1 Processor -- start 2024-05-09T04:53:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43f99d4b80 1 -- start start 2024-05-09T04:53:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43f99d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf75fe4a0 0x55ccf77859d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43f99d4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf76e8160 0x55ccf777f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43f99d4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf777b6a0 0x55ccf7790f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43f99d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41b3c10 con 0x55ccf75fe4a0 2024-05-09T04:53:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43f99d4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41b14a0 con 0x55ccf777b6a0 2024-05-09T04:53:54.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43f99d4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41af2f0 con 0x55ccf76e8160 2024-05-09T04:53:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43e8d5a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf75fe4a0 0x55ccf77859d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43e8d5a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf75fe4a0 0x55ccf77859d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46788/0 (socket says 172.21.15.46:46788) 2024-05-09T04:53:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43e8d5a700 1 -- 172.21.15.46:0/4200563966 learned_addr learned my addr 172.21.15.46:0/4200563966 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43e8d5a700 1 -- 172.21.15.46:0/4200563966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf76e8160 msgr2=0x55ccf777f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43e8d5a700 1 --2- 172.21.15.46:0/4200563966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf76e8160 0x55ccf777f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43e8d5a700 1 -- 172.21.15.46:0/4200563966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf777b6a0 msgr2=0x55ccf7790f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:53:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43e8d5a700 1 --2- 172.21.15.46:0/4200563966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf777b6a0 0x55ccf7790f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.441+0000 7f43e8d5a700 1 -- 172.21.15.46:0/4200563966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ccf77772e0 con 0x55ccf75fe4a0 2024-05-09T04:53:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.442+0000 7f43e8d5a700 1 --2- 172.21.15.46:0/4200563966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf75fe4a0 0x55ccf77859d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f43d800b520 tx=0x7f43d80f27b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72d8345e554d3cdd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.442+0000 7f43ce7fc700 1 -- 172.21.15.46:0/4200563966 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d80eae60 con 0x55ccf75fe4a0 2024-05-09T04:53:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.442+0000 7f43ce7fc700 1 -- 172.21.15.46:0/4200563966 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d80a8bd0 con 0x55ccf75fe4a0 2024-05-09T04:53:54.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.442+0000 7f43ce7fc700 1 -- 172.21.15.46:0/4200563966 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d80a8830 con 0x55ccf75fe4a0 2024-05-09T04:53:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.443+0000 7f43f99d4b80 1 -- 172.21.15.46:0/4200563966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf75fe4a0 msgr2=0x55ccf77859d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.443+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/4200563966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf75fe4a0 0x55ccf77859d0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f43d800b520 tx=0x7f43d80f27b0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.443+0000 7f43f99d4b80 1 -- 172.21.15.46:0/4200563966 shutdown_connections 2024-05-09T04:53:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.443+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/4200563966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf777b6a0 0x55ccf7790f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.443+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/4200563966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf75fe4a0 0x55ccf77859d0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.443+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/4200563966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf76e8160 0x55ccf777f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.443+0000 7f43f99d4b80 1 -- 172.21.15.46:0/4200563966 >> 172.21.15.46:0/4200563966 conn(0x55ccf765e6a0 msgr2=0x55ccf777ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:54.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.443+0000 7f43f99d4b80 1 -- 172.21.15.46:0/4200563966 shutdown_connections 2024-05-09T04:53:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.443+0000 7f43f99d4b80 1 -- 172.21.15.46:0/4200563966 wait complete. 2024-05-09T04:53:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.444+0000 7f43f99d4b80 1 -- start start 2024-05-09T04:53:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.444+0000 7f43f99d4b80 1 Processor -- start 2024-05-09T04:53:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.445+0000 7f43f99d4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf75fe4a0 0x55ccf778d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.445+0000 7f43f99d4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf76e8160 0x55ccf778d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.445+0000 7f43f99d4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf777b6a0 0x55ccf77814f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.445+0000 7f43f99d4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41af960 con 0x55ccf76e8160 2024-05-09T04:53:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.445+0000 7f43f99d4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41b0dd0 con 0x55ccf75fe4a0 2024-05-09T04:53:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.445+0000 7f43f99d4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f43e41b0030 con 0x55ccf777b6a0 2024-05-09T04:53:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.445+0000 7f43f99d4b80 10 client.0 _ll_register_callbacks cb 0x55ccf750f6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.445+0000 7f43e3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf76e8160 0x55ccf778d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:54.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.445+0000 7f43e3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf76e8160 0x55ccf778d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46804/0 (socket says 172.21.15.46:46804) 2024-05-09T04:53:54.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.445+0000 7f43e3fff700 1 -- 172.21.15.46:0/1805156392 learned_addr learned my addr 172.21.15.46:0/1805156392 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:54.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.445+0000 7f43e3fff700 1 -- 172.21.15.46:0/1805156392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf777b6a0 msgr2=0x55ccf77814f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T04:53:54.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.445+0000 7f43e3fff700 1 --2- 172.21.15.46:0/1805156392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf777b6a0 0x55ccf77814f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.445+0000 7f43e3fff700 1 -- 172.21.15.46:0/1805156392 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf75fe4a0 msgr2=0x55ccf778d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:54.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.445+0000 7f43e3fff700 1 --2- 172.21.15.46:0/1805156392 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf75fe4a0 0x55ccf778d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.445+0000 7f43e3fff700 1 -- 172.21.15.46:0/1805156392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ccf77830e0 con 0x55ccf76e8160 2024-05-09T04:53:54.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.446+0000 7f43e3fff700 1 --2- 172.21.15.46:0/1805156392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf76e8160 0x55ccf778d7e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009990 tx=0x7f43d000bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1cd21b13d8c0352 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:54.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.446+0000 7f43c3fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:54.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.446+0000 7f43c3fff700 1 -- 172.21.15.46:0/1805156392 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43d0003c90 con 0x55ccf76e8160 2024-05-09T04:53:54.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.446+0000 7f43c3fff700 1 -- 172.21.15.46:0/1805156392 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f43d0005d40 con 0x55ccf76e8160 2024-05-09T04:53:54.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.446+0000 7f43f99d4b80 1 -- 172.21.15.46:0/1805156392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43d80a8bd0 con 0x55ccf76e8160 2024-05-09T04:53:54.449 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.446+0000 7f43c3fff700 1 -- 172.21.15.46:0/1805156392 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f43d0003e30 con 0x55ccf76e8160 2024-05-09T04:53:54.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.446+0000 7f43f99d4b80 10 client.16056 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:54.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.446+0000 7f43f99d4b80 1 -- 172.21.15.46:0/1805156392 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ccf77fee60 con 0x55ccf76e8160 2024-05-09T04:53:54.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.447+0000 7f43c3fff700 1 -- 172.21.15.46:0/1805156392 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43d80a8bd0 con 0x55ccf76e8160 2024-05-09T04:53:54.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.447+0000 7f43c3fff700 1 -- 172.21.15.46:0/1805156392 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f43d000f8e0 con 0x55ccf76e8160 2024-05-09T04:53:54.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.447+0000 7f43f99d4b80 10 client.16056 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:54.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.447+0000 7f43f99d4b80 10 client.16056 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:53:54.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.447+0000 7f43f99d4b80 20 client.16056 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:54.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.447+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/1805156392 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ccf77ff5f0 0x55ccf7801a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:54.451 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.447+0000 7f43f99d4b80 4 client.16056 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:53:54.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.447+0000 7f43f99d4b80 1 -- 172.21.15.46:0/1805156392 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ccf76a5a70 con 0x55ccf77ff5f0 2024-05-09T04:53:54.452 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.448+0000 7f43e8d5a700 1 --2- 172.21.15.46:0/1805156392 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ccf77ff5f0 0x55ccf7801a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:54.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.451+0000 7f43c3fff700 10 client.16056 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:53:54.453 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.451+0000 7f43e8d5a700 1 --2- 172.21.15.46:0/1805156392 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ccf77ff5f0 0x55ccf7801a10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=49385b2500f01cc server_cookie=a5b4b0775c8a9e8f in_seq=0 out_seq=0 2024-05-09T04:53:54.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.452+0000 7f43c3fff700 1 -- 172.21.15.46:0/1805156392 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ccf76a5a70 con 0x55ccf77ff5f0 2024-05-09T04:53:54.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.452+0000 7f43c3fff700 10 client.16056 handle_command_reply: tid=0 2024-05-09T04:53:54.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.452+0000 7f43f99d4b80 1 client.16056 shutdown 2024-05-09T04:53:54.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.452+0000 7f43f99d4b80 1 -- 172.21.15.46:0/1805156392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf76e8160 msgr2=0x55ccf778d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:54.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.452+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/1805156392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf76e8160 0x55ccf778d7e0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f43d0009990 tx=0x7f43d000bf90 comp rx=0 tx=0).stop 2024-05-09T04:53:54.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.452+0000 7f43f99d4b80 1 -- 172.21.15.46:0/1805156392 shutdown_connections 2024-05-09T04:53:54.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.452+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/1805156392 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55ccf77ff5f0 0x55ccf7801a10 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.452+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/1805156392 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ccf777b6a0 0x55ccf77814f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.452+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/1805156392 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ccf75fe4a0 0x55ccf778d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.452+0000 7f43f99d4b80 1 --2- 172.21.15.46:0/1805156392 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ccf76e8160 0x55ccf778d7e0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.454 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.452+0000 7f43f99d4b80 1 -- 172.21.15.46:0/1805156392 >> 172.21.15.46:0/1805156392 conn(0x55ccf765e6a0 msgr2=0x55ccf777f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.452+0000 7f43c3fff700 0 client.16056 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:53:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.452+0000 7f43f99d4b80 1 -- 172.21.15.46:0/1805156392 shutdown_connections 2024-05-09T04:53:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.452+0000 7f43f99d4b80 1 -- 172.21.15.46:0/1805156392 wait complete. 2024-05-09T04:53:54.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.453+0000 7f43f99d4b80 20 client.16056 trim_cache size 0 max 16384 2024-05-09T04:53:54.455 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:53:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.454+0000 7f43eafbe700 1 -- 172.21.15.46:0/195423265 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f43d40b2a70 msgr2=0x7f43d40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.454+0000 7f43eafbe700 1 --2- 172.21.15.46:0/195423265 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f43d40b2a70 0x7f43d40b4f80 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f43dc009e60 tx=0x7f43dc009450 comp rx=0 tx=0).stop 2024-05-09T04:53:54.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.454+0000 7f43eafbe700 1 -- 172.21.15.46:0/195423265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e41027e0 msgr2=0x7f43e41039c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.454+0000 7f43eafbe700 1 --2- 172.21.15.46:0/195423265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e41027e0 0x7f43e41039c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f43d800b550 tx=0x7f43d800b910 comp rx=0 tx=0).stop 2024-05-09T04:53:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.454+0000 7f43eafbe700 1 -- 172.21.15.46:0/195423265 shutdown_connections 2024-05-09T04:53:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.454+0000 7f43eafbe700 1 --2- 172.21.15.46:0/195423265 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f43d40b2a70 0x7f43d40b4f80 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.454+0000 7f43eafbe700 1 --2- 172.21.15.46:0/195423265 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f43e410a3e0 0x7f43e4104440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.454+0000 7f43eafbe700 1 --2- 172.21.15.46:0/195423265 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f43e41027e0 0x7f43e41039c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.454+0000 7f43eafbe700 1 --2- 172.21.15.46:0/195423265 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f43e4103130 0x7f43e4103f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.454+0000 7f43eafbe700 1 -- 172.21.15.46:0/195423265 >> 172.21.15.46:0/195423265 conn(0x7f43e40fe6f0 msgr2=0x7f43e4100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.454+0000 7f43eafbe700 1 -- 172.21.15.46:0/195423265 shutdown_connections 2024-05-09T04:53:54.457 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:54.454+0000 7f43eafbe700 1 -- 172.21.15.46:0/195423265 wait complete. 2024-05-09T04:53:55.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:54 smithi110 ceph-mon[34904]: pgmap v87: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 573 MiB/s wr, 280 op/s 2024-05-09T04:53:55.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:54 smithi046 ceph-mon[30136]: pgmap v87: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 573 MiB/s wr, 280 op/s 2024-05-09T04:53:55.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:54 smithi072 ceph-mon[34088]: pgmap v87: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 573 MiB/s wr, 280 op/s 2024-05-09T04:53:55.294 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:53:55.294 INFO:tasks.workunit.client.0.smithi046.stdout: Children see throughput for 1 rewriters = 862997.81 kB/sec 2024-05-09T04:53:55.294 INFO:tasks.workunit.client.0.smithi046.stdout: Parent sees throughput for 1 rewriters = 862982.33 kB/sec 2024-05-09T04:53:55.294 INFO:tasks.workunit.client.0.smithi046.stdout: Min throughput per process = 862997.81 kB/sec 2024-05-09T04:53:55.294 INFO:tasks.workunit.client.0.smithi046.stdout: Max throughput per process = 862997.81 kB/sec 2024-05-09T04:53:55.294 INFO:tasks.workunit.client.0.smithi046.stdout: Avg throughput per process = 862997.81 kB/sec 2024-05-09T04:53:55.294 INFO:tasks.workunit.client.0.smithi046.stdout: Min xfer = 10485760.00 kB 2024-05-09T04:53:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.451+0000 7f298a566640 1 -- 172.21.15.46:0/71457929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298410b800 msgr2=0x7f298410dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.451+0000 7f298a566640 1 --2- 172.21.15.46:0/71457929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298410b800 0x7f298410dc20 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f2978009900 tx=0x7f297802a410 comp rx=0 tx=0).stop 2024-05-09T04:53:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.452+0000 7f298a566640 1 -- 172.21.15.46:0/71457929 shutdown_connections 2024-05-09T04:53:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.452+0000 7f298a566640 1 --2- 172.21.15.46:0/71457929 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f298410e1f0 0x7f29841106e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.452+0000 7f298a566640 1 --2- 172.21.15.46:0/71457929 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298410b800 0x7f298410dc20 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.452+0000 7f298a566640 1 --2- 172.21.15.46:0/71457929 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2984104da0 0x7f29841051b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.452+0000 7f298a566640 1 -- 172.21.15.46:0/71457929 >> 172.21.15.46:0/71457929 conn(0x7f29840782e0 msgr2=0x7f2984078720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.452+0000 7f298a566640 1 -- 172.21.15.46:0/71457929 shutdown_connections 2024-05-09T04:53:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.452+0000 7f298a566640 1 -- 172.21.15.46:0/71457929 wait complete. 2024-05-09T04:53:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.452+0000 7f298a566640 1 Processor -- start 2024-05-09T04:53:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.452+0000 7f298a566640 1 -- start start 2024-05-09T04:53:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.453+0000 7f298a566640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2984104da0 0x7f29841a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.453+0000 7f298a566640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f298410b800 0x7f29841a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.453+0000 7f298a566640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298410e1f0 0x7f298419a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.453+0000 7f298a566640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2984113e50 con 0x7f298410e1f0 2024-05-09T04:53:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.453+0000 7f298a566640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2984113cd0 con 0x7f2984104da0 2024-05-09T04:53:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.453+0000 7f298a566640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2984113fd0 con 0x7f298410b800 2024-05-09T04:53:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.453+0000 7f2988b03640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298410e1f0 0x7f298419a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.453+0000 7f2988b03640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298410e1f0 0x7f298419a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46808/0 (socket says 172.21.15.46:46808) 2024-05-09T04:53:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.453+0000 7f2988b03640 1 -- 172.21.15.46:0/3671830873 learned_addr learned my addr 172.21.15.46:0/3671830873 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.453+0000 7f29837fe640 1 --2- 172.21.15.46:0/3671830873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f298410b800 0x7f29841a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.453+0000 7f2983fff640 1 --2- 172.21.15.46:0/3671830873 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2984104da0 0x7f29841a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.455 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.453+0000 7f2988b03640 1 -- 172.21.15.46:0/3671830873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f298410b800 msgr2=0x7f29841a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.453+0000 7f2988b03640 1 --2- 172.21.15.46:0/3671830873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f298410b800 0x7f29841a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.454+0000 7f2988b03640 1 -- 172.21.15.46:0/3671830873 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2984104da0 msgr2=0x7f29841a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.454+0000 7f2988b03640 1 --2- 172.21.15.46:0/3671830873 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2984104da0 0x7f29841a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.454+0000 7f2988b03640 1 -- 172.21.15.46:0/3671830873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f298419ae00 con 0x7f298410e1f0 2024-05-09T04:53:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.454+0000 7f2983fff640 1 --2- 172.21.15.46:0/3671830873 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2984104da0 0x7f29841a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.454+0000 7f2988b03640 1 --2- 172.21.15.46:0/3671830873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298410e1f0 0x7f298419a6d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f297c00eb10 tx=0x7f297c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.454+0000 7f29817fa640 1 -- 172.21.15.46:0/3671830873 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c004b20 con 0x7f298410e1f0 2024-05-09T04:53:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.454+0000 7f29817fa640 1 -- 172.21.15.46:0/3671830873 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f297c004cc0 con 0x7f298410e1f0 2024-05-09T04:53:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.454+0000 7f298a566640 1 -- 172.21.15.46:0/3671830873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f298419b090 con 0x7f298410e1f0 2024-05-09T04:53:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.454+0000 7f29817fa640 1 -- 172.21.15.46:0/3671830873 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c010670 con 0x7f298410e1f0 2024-05-09T04:53:56.456 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.454+0000 7f298a566640 1 -- 172.21.15.46:0/3671830873 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29841a72f0 con 0x7f298410e1f0 2024-05-09T04:53:56.458 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.457+0000 7f29817fa640 1 -- 172.21.15.46:0/3671830873 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f297c005020 con 0x7f298410e1f0 2024-05-09T04:53:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.457+0000 7f2991416740 1 Processor -- start 2024-05-09T04:53:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.457+0000 7f2991416740 1 -- start start 2024-05-09T04:53:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.457+0000 7f2991416740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7806d1b0 0x556c78410680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.457+0000 7f2991416740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c780442f0 0x556c78410bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.457+0000 7f2991416740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7804c4d0 0x556c78411100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.457+0000 7f2991416740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556c783757c0 con 0x556c7804c4d0 2024-05-09T04:53:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.457+0000 7f2991416740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556c783110e0 con 0x556c780442f0 2024-05-09T04:53:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.457+0000 7f2991416740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556c78371780 con 0x556c7806d1b0 2024-05-09T04:53:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.458+0000 7f29837fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7804c4d0 0x556c78411100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.458+0000 7f29837fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7804c4d0 0x556c78411100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46812/0 (socket says 172.21.15.46:46812) 2024-05-09T04:53:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.458+0000 7f29837fe640 1 -- 172.21.15.46:0/204534777 learned_addr learned my addr 172.21.15.46:0/204534777 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.458+0000 7f2983fff640 1 --2- 172.21.15.46:0/204534777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c780442f0 0x556c78410bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.459 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.458+0000 7f29837fe640 1 --2- 172.21.15.46:0/204534777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7806d1b0 0x556c78410680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.458+0000 7f29817fa640 1 --2- 172.21.15.46:0/3671830873 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f296c0b2950 0x7f296c0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.458+0000 7f2988b03640 1 --2- 172.21.15.46:0/3671830873 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f296c0b2950 0x7f296c0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.458+0000 7f29837fe640 1 -- 172.21.15.46:0/204534777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7806d1b0 msgr2=0x556c78410680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.458+0000 7f29837fe640 1 --2- 172.21.15.46:0/204534777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7806d1b0 0x556c78410680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.458+0000 7f29837fe640 1 -- 172.21.15.46:0/204534777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c780442f0 msgr2=0x556c78410bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.458+0000 7f29837fe640 1 --2- 172.21.15.46:0/204534777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c780442f0 0x556c78410bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.458+0000 7f29837fe640 1 -- 172.21.15.46:0/204534777 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c7833a7c0 con 0x556c7804c4d0 2024-05-09T04:53:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.458+0000 7f29817fa640 1 -- 172.21.15.46:0/3671830873 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f297c0e2bf0 con 0x7f298410e1f0 2024-05-09T04:53:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.458+0000 7f29837fe640 1 --2- 172.21.15.46:0/204534777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7804c4d0 0x556c78411100 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f297802a930 tx=0x7f2978009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbfa6f117d6419d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.458+0000 7f296a7fc640 1 -- 172.21.15.46:0/204534777 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978035070 con 0x556c7804c4d0 2024-05-09T04:53:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.459+0000 7f2988b03640 1 --2- 172.21.15.46:0/3671830873 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f296c0b2950 0x7f296c0b4e60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f297c0e03d0 tx=0x7f297c0767e0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.459+0000 7f296a7fc640 1 -- 172.21.15.46:0/204534777 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29780318c0 con 0x556c7804c4d0 2024-05-09T04:53:56.460 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.459+0000 7f296a7fc640 1 -- 172.21.15.46:0/204534777 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978003e20 con 0x556c7804c4d0 2024-05-09T04:53:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.459+0000 7f2991416740 1 -- 172.21.15.46:0/204534777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7804c4d0 msgr2=0x556c78411100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.459+0000 7f2991416740 1 --2- 172.21.15.46:0/204534777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7804c4d0 0x556c78411100 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f297802a930 tx=0x7f2978009560 comp rx=0 tx=0).stop 2024-05-09T04:53:56.465 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.459+0000 7f2991416740 1 -- 172.21.15.46:0/204534777 shutdown_connections 2024-05-09T04:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.459+0000 7f2991416740 1 --2- 172.21.15.46:0/204534777 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7804c4d0 0x556c78411100 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.459+0000 7f2991416740 1 --2- 172.21.15.46:0/204534777 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c780442f0 0x556c78410bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.460+0000 7f2991416740 1 --2- 172.21.15.46:0/204534777 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7806d1b0 0x556c78410680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.460+0000 7f2991416740 1 -- 172.21.15.46:0/204534777 >> 172.21.15.46:0/204534777 conn(0x556c78061300 msgr2=0x556c7840ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.460+0000 7f2991416740 1 -- 172.21.15.46:0/204534777 shutdown_connections 2024-05-09T04:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.460+0000 7f2991416740 1 -- 172.21.15.46:0/204534777 wait complete. 2024-05-09T04:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.460+0000 7f2991416740 1 -- start start 2024-05-09T04:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.460+0000 7f2991416740 1 Processor -- start 2024-05-09T04:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f2991416740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c780442f0 0x556c7841c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f2991416740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7804c4d0 0x556c7841cc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f2991416740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c7806d1b0 0x556c7841abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f2991416740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556c7841d750 con 0x556c780442f0 2024-05-09T04:53:56.466 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f2991416740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556c782fc9c0 con 0x556c7806d1b0 2024-05-09T04:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f2991416740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556c7841d8d0 con 0x556c7804c4d0 2024-05-09T04:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f29837fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c780442f0 0x556c7841c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f29837fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c780442f0 0x556c7841c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46824/0 (socket says 172.21.15.46:46824) 2024-05-09T04:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f29837fe640 1 -- 172.21.15.46:0/2737454045 learned_addr learned my addr 172.21.15.46:0/2737454045 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f2991416740 10 client.0 _ll_register_callbacks cb 0x556c78329020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f29837fe640 1 --2- 172.21.15.46:0/2737454045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c7806d1b0 0x556c7841abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f2983fff640 1 --2- 172.21.15.46:0/2737454045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7804c4d0 0x556c7841cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f29837fe640 1 -- 172.21.15.46:0/2737454045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7804c4d0 msgr2=0x556c7841cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f29837fe640 1 --2- 172.21.15.46:0/2737454045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7804c4d0 0x556c7841cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f29837fe640 1 -- 172.21.15.46:0/2737454045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c7806d1b0 msgr2=0x556c7841abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f29837fe640 1 --2- 172.21.15.46:0/2737454045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c7806d1b0 0x556c7841abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f29837fe640 1 -- 172.21.15.46:0/2737454045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c783231f0 con 0x556c780442f0 2024-05-09T04:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f29837fe640 1 --2- 172.21.15.46:0/2737454045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c780442f0 0x556c7841c6c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2978006010 tx=0x7f2978005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb575fdfbd5d90b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.461+0000 7f29637fe640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:56.467 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.462+0000 7f29637fe640 1 -- 172.21.15.46:0/2737454045 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978035040 con 0x556c780442f0 2024-05-09T04:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.462+0000 7f2991416740 1 -- 172.21.15.46:0/2737454045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f29780318c0 con 0x556c780442f0 2024-05-09T04:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.462+0000 7f29637fe640 1 -- 172.21.15.46:0/2737454045 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f2978005260 con 0x556c780442f0 2024-05-09T04:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.462+0000 7f29637fe640 1 -- 172.21.15.46:0/2737454045 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2978005460 con 0x556c780442f0 2024-05-09T04:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.462+0000 7f2991416740 10 client.16068 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.462+0000 7f2991416740 1 -- 172.21.15.46:0/2737454045 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556c780d8790 con 0x556c780442f0 2024-05-09T04:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.462+0000 7f29637fe640 1 -- 172.21.15.46:0/2737454045 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29780318c0 con 0x556c780442f0 2024-05-09T04:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.462+0000 7f29637fe640 1 -- 172.21.15.46:0/2737454045 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2978003c10 con 0x556c780442f0 2024-05-09T04:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.462+0000 7f2991416740 10 client.16068 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.462+0000 7f2991416740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.462+0000 7f2991416740 20 client.16068 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.462+0000 7f2991416740 1 --2- 172.21.15.46:0/2737454045 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556c7802bd80 0x556c7842cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.462+0000 7f2991416740 4 client.16068 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.462+0000 7f2991416740 1 -- 172.21.15.46:0/2737454045 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c783833d0 con 0x556c7802bd80 2024-05-09T04:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.463+0000 7f2983fff640 1 --2- 172.21.15.46:0/2737454045 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556c7802bd80 0x556c7842cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.468 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.463+0000 7f2983fff640 1 --2- 172.21.15.46:0/2737454045 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556c7802bd80 0x556c7842cdf0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac47f20c9dc063e server_cookie=57d291d2a37e80ad in_seq=0 out_seq=0 2024-05-09T04:53:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.463+0000 7f29637fe640 10 client.16068 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.464+0000 7f29637fe640 1 -- 172.21.15.46:0/2737454045 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556c783833d0 con 0x556c7802bd80 2024-05-09T04:53:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.464+0000 7f29637fe640 10 client.16068 handle_command_reply: tid=0 2024-05-09T04:53:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.464+0000 7f2991416740 1 client.16068 shutdown 2024-05-09T04:53:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.464+0000 7f2991416740 1 -- 172.21.15.46:0/2737454045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c780442f0 msgr2=0x556c7841c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.464+0000 7f2991416740 1 --2- 172.21.15.46:0/2737454045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c780442f0 0x556c7841c6c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f2978006010 tx=0x7f2978005c90 comp rx=0 tx=0).stop 2024-05-09T04:53:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.464+0000 7f2991416740 1 -- 172.21.15.46:0/2737454045 shutdown_connections 2024-05-09T04:53:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.464+0000 7f2991416740 1 --2- 172.21.15.46:0/2737454045 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556c7802bd80 0x556c7842cdf0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.464+0000 7f2991416740 1 --2- 172.21.15.46:0/2737454045 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7804c4d0 0x556c7841cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.464+0000 7f2991416740 1 --2- 172.21.15.46:0/2737454045 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c7806d1b0 0x556c7841abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.464+0000 7f2991416740 1 --2- 172.21.15.46:0/2737454045 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c780442f0 0x556c7841c6c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.464+0000 7f2991416740 1 -- 172.21.15.46:0/2737454045 >> 172.21.15.46:0/2737454045 conn(0x556c78061300 msgr2=0x556c784123a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.464+0000 7f29637fe640 0 client.16068 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.464+0000 7f2991416740 1 -- 172.21.15.46:0/2737454045 shutdown_connections 2024-05-09T04:53:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.464+0000 7f2991416740 1 -- 172.21.15.46:0/2737454045 wait complete. 2024-05-09T04:53:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.464+0000 7f2991416740 20 client.16068 trim_cache size 0 max 16384 2024-05-09T04:53:56.486 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.480+0000 7f2991416740 1 Processor -- start 2024-05-09T04:53:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.480+0000 7f2991416740 1 -- start start 2024-05-09T04:53:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.480+0000 7f2991416740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c780442f0 0x556c7841ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.480+0000 7f2991416740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7804c4d0 0x556c78414330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.480+0000 7f2991416740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7806d1b0 0x556c78414870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.480+0000 7f2991416740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29841b24b0 con 0x556c7804c4d0 2024-05-09T04:53:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.480+0000 7f2991416740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f29841afd80 con 0x556c780442f0 2024-05-09T04:53:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.480+0000 7f2991416740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f29841adbd0 con 0x556c7806d1b0 2024-05-09T04:53:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.480+0000 7f2983fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7804c4d0 0x556c78414330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.480+0000 7f2983fff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7804c4d0 0x556c78414330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46836/0 (socket says 172.21.15.46:46836) 2024-05-09T04:53:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.480+0000 7f2983fff640 1 -- 172.21.15.46:0/809783308 learned_addr learned my addr 172.21.15.46:0/809783308 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:56.487 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.480+0000 7f29837fe640 1 --2- 172.21.15.46:0/809783308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c780442f0 0x556c7841ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.480+0000 7f29837fe640 1 --2- 172.21.15.46:0/809783308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7806d1b0 0x556c78414870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.481+0000 7f2983fff640 1 -- 172.21.15.46:0/809783308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7806d1b0 msgr2=0x556c78414870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.481+0000 7f2983fff640 1 --2- 172.21.15.46:0/809783308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7806d1b0 0x556c78414870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.481+0000 7f2983fff640 1 -- 172.21.15.46:0/809783308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c780442f0 msgr2=0x556c7841ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.481+0000 7f2983fff640 1 --2- 172.21.15.46:0/809783308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c780442f0 0x556c7841ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.481+0000 7f2983fff640 1 -- 172.21.15.46:0/809783308 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c7814f5f0 con 0x556c7804c4d0 2024-05-09T04:53:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.481+0000 7f29837fe640 1 --2- 172.21.15.46:0/809783308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7806d1b0 0x556c78414870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:53:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.481+0000 7f2983fff640 1 --2- 172.21.15.46:0/809783308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7804c4d0 0x556c78414330 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x556c7840e170 tx=0x7f297400bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b15bcfc1bfdd1732 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.481+0000 7f2969ffb640 1 -- 172.21.15.46:0/809783308 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974023070 con 0x556c7804c4d0 2024-05-09T04:53:56.488 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.481+0000 7f2969ffb640 1 -- 172.21.15.46:0/809783308 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f297401c910 con 0x556c7804c4d0 2024-05-09T04:53:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.481+0000 7f2969ffb640 1 -- 172.21.15.46:0/809783308 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2974011a00 con 0x556c7804c4d0 2024-05-09T04:53:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.482+0000 7f2991416740 1 -- 172.21.15.46:0/809783308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7804c4d0 msgr2=0x556c78414330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.482+0000 7f2991416740 1 --2- 172.21.15.46:0/809783308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7804c4d0 0x556c78414330 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x556c7840e170 tx=0x7f297400bdd0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.482+0000 7f2991416740 1 -- 172.21.15.46:0/809783308 shutdown_connections 2024-05-09T04:53:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.482+0000 7f2991416740 1 --2- 172.21.15.46:0/809783308 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7806d1b0 0x556c78414870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.482+0000 7f2991416740 1 --2- 172.21.15.46:0/809783308 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7804c4d0 0x556c78414330 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.482+0000 7f2991416740 1 --2- 172.21.15.46:0/809783308 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c780442f0 0x556c7841ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.482+0000 7f2991416740 1 -- 172.21.15.46:0/809783308 >> 172.21.15.46:0/809783308 conn(0x556c78061300 msgr2=0x556c7841c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:56.489 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.482+0000 7f2991416740 1 -- 172.21.15.46:0/809783308 shutdown_connections 2024-05-09T04:53:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.482+0000 7f2991416740 1 -- 172.21.15.46:0/809783308 wait complete. 2024-05-09T04:53:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.482+0000 7f2991416740 1 -- start start 2024-05-09T04:53:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.482+0000 7f2991416740 1 Processor -- start 2024-05-09T04:53:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f2991416740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c780442f0 0x556c7848d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f2991416740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7804c4d0 0x556c7848da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f2991416740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7806d1b0 0x556c7842f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f2991416740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f29841ae240 con 0x556c7806d1b0 2024-05-09T04:53:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f2991416740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f29841af6b0 con 0x556c780442f0 2024-05-09T04:53:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f2991416740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f29841ae910 con 0x556c7804c4d0 2024-05-09T04:53:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f2991416740 10 client.0 _ll_register_callbacks cb 0x7f297400b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f29837fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7806d1b0 0x556c7842f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f29837fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7806d1b0 0x556c7842f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46844/0 (socket says 172.21.15.46:46844) 2024-05-09T04:53:56.490 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f29837fe640 1 -- 172.21.15.46:0/1655764270 learned_addr learned my addr 172.21.15.46:0/1655764270 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f2983fff640 1 --2- 172.21.15.46:0/1655764270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7804c4d0 0x556c7848da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f29837fe640 1 --2- 172.21.15.46:0/1655764270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c780442f0 0x556c7848d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f29837fe640 1 -- 172.21.15.46:0/1655764270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7804c4d0 msgr2=0x556c7848da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f29837fe640 1 --2- 172.21.15.46:0/1655764270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7804c4d0 0x556c7848da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f29837fe640 1 -- 172.21.15.46:0/1655764270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c780442f0 msgr2=0x556c7848d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f29837fe640 1 --2- 172.21.15.46:0/1655764270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c780442f0 0x556c7848d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f29837fe640 1 -- 172.21.15.46:0/1655764270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c78339ed0 con 0x556c7806d1b0 2024-05-09T04:53:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f29837fe640 1 --2- 172.21.15.46:0/1655764270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7806d1b0 0x556c7842f430 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2978001eb0 tx=0x7f297804d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c40083acecbda5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.483+0000 7f29637fe640 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.484+0000 7f29637fe640 1 -- 172.21.15.46:0/1655764270 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978035040 con 0x556c7806d1b0 2024-05-09T04:53:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.484+0000 7f29637fe640 1 -- 172.21.15.46:0/1655764270 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f2978005260 con 0x556c7806d1b0 2024-05-09T04:53:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.484+0000 7f2991416740 1 -- 172.21.15.46:0/1655764270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f297401c910 con 0x556c7806d1b0 2024-05-09T04:53:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.484+0000 7f29637fe640 1 -- 172.21.15.46:0/1655764270 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f29780035f0 con 0x556c7806d1b0 2024-05-09T04:53:56.491 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.484+0000 7f2991416740 10 client.16074 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.484+0000 7f2991416740 1 -- 172.21.15.46:0/1655764270 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556c78494940 con 0x556c7806d1b0 2024-05-09T04:53:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.484+0000 7f29637fe640 1 -- 172.21.15.46:0/1655764270 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f297401c910 con 0x556c7806d1b0 2024-05-09T04:53:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.484+0000 7f29637fe640 1 -- 172.21.15.46:0/1655764270 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f29780046c0 con 0x556c7806d1b0 2024-05-09T04:53:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.484+0000 7f2991416740 10 client.16074 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.484+0000 7f2991416740 10 client.16074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:53:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.484+0000 7f2991416740 20 client.16074 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.484+0000 7f2991416740 1 --2- 172.21.15.46:0/1655764270 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556c780011c0 0x556c78497150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.484+0000 7f2991416740 4 client.16074 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T04:53:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.484+0000 7f2991416740 1 -- 172.21.15.46:0/1655764270 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c7837d240 con 0x556c780011c0 2024-05-09T04:53:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.485+0000 7f2983fff640 1 --2- 172.21.15.46:0/1655764270 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556c780011c0 0x556c78497150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.485+0000 7f29637fe640 10 client.16074 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:56.492 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.485+0000 7f2983fff640 1 --2- 172.21.15.46:0/1655764270 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556c780011c0 0x556c78497150 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec22838f6b503a55 server_cookie=8d17ac8ce6266407 in_seq=0 out_seq=0 2024-05-09T04:53:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.497+0000 7f29637fe640 1 -- 172.21.15.46:0/1655764270 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x556c7837d240 con 0x556c780011c0 2024-05-09T04:53:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.497+0000 7f29637fe640 10 client.16074 handle_command_reply: tid=0 2024-05-09T04:53:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.497+0000 7f2991416740 1 client.16074 shutdown 2024-05-09T04:53:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.497+0000 7f2991416740 1 -- 172.21.15.46:0/1655764270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7806d1b0 msgr2=0x556c7842f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.497+0000 7f2991416740 1 --2- 172.21.15.46:0/1655764270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7806d1b0 0x556c7842f430 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f2978001eb0 tx=0x7f297804d040 comp rx=0 tx=0).stop 2024-05-09T04:53:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.497+0000 7f2991416740 1 -- 172.21.15.46:0/1655764270 shutdown_connections 2024-05-09T04:53:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.497+0000 7f2991416740 1 --2- 172.21.15.46:0/1655764270 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556c780011c0 0x556c78497150 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.497+0000 7f2991416740 1 --2- 172.21.15.46:0/1655764270 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556c7804c4d0 0x556c7848da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.497+0000 7f2991416740 1 --2- 172.21.15.46:0/1655764270 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556c7806d1b0 0x556c7842f430 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.498 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.497+0000 7f2991416740 1 --2- 172.21.15.46:0/1655764270 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556c780442f0 0x556c7848d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.497+0000 7f29637fe640 0 client.16074 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:53:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.497+0000 7f2991416740 1 -- 172.21.15.46:0/1655764270 >> 172.21.15.46:0/1655764270 conn(0x556c78061300 msgr2=0x556c7840b700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.498+0000 7f2991416740 1 -- 172.21.15.46:0/1655764270 shutdown_connections 2024-05-09T04:53:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.498+0000 7f2991416740 1 -- 172.21.15.46:0/1655764270 wait complete. 2024-05-09T04:53:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.498+0000 7f2991416740 20 client.16074 trim_cache size 0 max 16384 2024-05-09T04:53:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-05-09T04:53:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-05-09T04:53:56.499 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-05-09T04:53:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.498+0000 7f298a566640 1 -- 172.21.15.46:0/3671830873 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f296c0b2950 msgr2=0x7f296c0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:56.500 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.498+0000 7f298a566640 1 --2- 172.21.15.46:0/3671830873 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f296c0b2950 0x7f296c0b4e60 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f297c0e03d0 tx=0x7f297c0767e0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.498+0000 7f298a566640 1 -- 172.21.15.46:0/3671830873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298410e1f0 msgr2=0x7f298419a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.498+0000 7f298a566640 1 --2- 172.21.15.46:0/3671830873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298410e1f0 0x7f298419a6d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f297c00eb10 tx=0x7f297c00c2d0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.498+0000 7f298a566640 1 -- 172.21.15.46:0/3671830873 shutdown_connections 2024-05-09T04:53:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.498+0000 7f298a566640 1 --2- 172.21.15.46:0/3671830873 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f296c0b2950 0x7f296c0b4e60 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.498+0000 7f298a566640 1 --2- 172.21.15.46:0/3671830873 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f298410b800 0x7f29841a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.498+0000 7f298a566640 1 --2- 172.21.15.46:0/3671830873 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f298410e1f0 0x7f298419a6d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.501 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.498+0000 7f298a566640 1 --2- 172.21.15.46:0/3671830873 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2984104da0 0x7f29841a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.498+0000 7f298a566640 1 -- 172.21.15.46:0/3671830873 >> 172.21.15.46:0/3671830873 conn(0x7f29840782e0 msgr2=0x7f29841012c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.498+0000 7f298a566640 1 -- 172.21.15.46:0/3671830873 shutdown_connections 2024-05-09T04:53:56.502 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:53:56.498+0000 7f298a566640 1 -- 172.21.15.46:0/3671830873 wait complete. 2024-05-09T04:53:56.642 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:53:56.898 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:53:57.020 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:56 smithi046 ceph-mon[30136]: pgmap v88: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 573 MiB/s wr, 280 op/s 2024-05-09T04:53:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:56 smithi110 ceph-mon[34904]: pgmap v88: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 573 MiB/s wr, 280 op/s 2024-05-09T04:53:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:56 smithi072 ceph-mon[34088]: pgmap v88: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 338 KiB/s rd, 573 MiB/s wr, 280 op/s 2024-05-09T04:53:58.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.827+0000 7fac2c6ca700 1 -- 172.21.15.46:0/3751144779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac241034c0 msgr2=0x7fac241038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:58.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.827+0000 7fac2c6ca700 1 --2- 172.21.15.46:0/3751144779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac241034c0 0x7fac241038d0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fac200099c0 tx=0x7fac20009cd0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.829 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.827+0000 7fac2c6ca700 1 -- 172.21.15.46:0/3751144779 shutdown_connections 2024-05-09T04:53:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.827+0000 7fac2c6ca700 1 --2- 172.21.15.46:0/3751144779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac24108740 0x7fac241100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.827+0000 7fac2c6ca700 1 --2- 172.21.15.46:0/3751144779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac241034c0 0x7fac241038d0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.827+0000 7fac2c6ca700 1 --2- 172.21.15.46:0/3751144779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac24107df0 0x7fac24108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.827+0000 7fac2c6ca700 1 -- 172.21.15.46:0/3751144779 >> 172.21.15.46:0/3751144779 conn(0x7fac240fedc0 msgr2=0x7fac241011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.828+0000 7fac2c6ca700 1 -- 172.21.15.46:0/3751144779 shutdown_connections 2024-05-09T04:53:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.828+0000 7fac2c6ca700 1 -- 172.21.15.46:0/3751144779 wait complete. 2024-05-09T04:53:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.828+0000 7fac2c6ca700 1 Processor -- start 2024-05-09T04:53:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.828+0000 7fac2c6ca700 1 -- start start 2024-05-09T04:53:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2c6ca700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac241034c0 0x7fac240ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2c6ca700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac24107df0 0x7fac240ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2c6ca700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac24108740 0x7fac240ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2c6ca700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fac24113400 con 0x7fac241034c0 2024-05-09T04:53:58.830 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2c6ca700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fac24113280 con 0x7fac24108740 2024-05-09T04:53:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2c6ca700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fac24113580 con 0x7fac24107df0 2024-05-09T04:53:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2a466700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac241034c0 0x7fac240ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2a466700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac241034c0 0x7fac240ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46854/0 (socket says 172.21.15.46:46854) 2024-05-09T04:53:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2a466700 1 -- 172.21.15.46:0/626354706 learned_addr learned my addr 172.21.15.46:0/626354706 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac29c65700 1 --2- 172.21.15.46:0/626354706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac24107df0 0x7fac240ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2ac67700 1 --2- 172.21.15.46:0/626354706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac24108740 0x7fac240ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2a466700 1 -- 172.21.15.46:0/626354706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac24107df0 msgr2=0x7fac240ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2a466700 1 --2- 172.21.15.46:0/626354706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac24107df0 0x7fac240ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2a466700 1 -- 172.21.15.46:0/626354706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac24108740 msgr2=0x7fac240ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2a466700 1 --2- 172.21.15.46:0/626354706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac24108740 0x7fac240ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2a466700 1 -- 172.21.15.46:0/626354706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac241a9200 con 0x7fac241034c0 2024-05-09T04:53:58.831 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.829+0000 7fac2a466700 1 --2- 172.21.15.46:0/626354706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac241034c0 0x7fac240ff2a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fac2000b580 tx=0x7fac200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:58.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.830+0000 7fac1b7fe700 1 -- 172.21.15.46:0/626354706 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20027070 con 0x7fac241034c0 2024-05-09T04:53:58.832 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.830+0000 7fac1b7fe700 1 -- 172.21.15.46:0/626354706 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac20003f20 con 0x7fac241034c0 2024-05-09T04:53:58.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.830+0000 7fac1b7fe700 1 -- 172.21.15.46:0/626354706 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac20022d70 con 0x7fac241034c0 2024-05-09T04:53:58.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.830+0000 7fac2c6ca700 1 -- 172.21.15.46:0/626354706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac241a9490 con 0x7fac241034c0 2024-05-09T04:53:58.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.830+0000 7fac2c6ca700 1 -- 172.21.15.46:0/626354706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac241a98e0 con 0x7fac241034c0 2024-05-09T04:53:58.833 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.832+0000 7fac1b7fe700 1 -- 172.21.15.46:0/626354706 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fac20022500 con 0x7fac241034c0 2024-05-09T04:53:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.833+0000 7fac1b7fe700 1 --2- 172.21.15.46:0/626354706 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fac100b29f0 0x7fac100b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.833+0000 7fac1b7fe700 1 -- 172.21.15.46:0/626354706 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fac200e3350 con 0x7fac241034c0 2024-05-09T04:53:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.833+0000 7fac29c65700 1 --2- 172.21.15.46:0/626354706 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fac100b29f0 0x7fac100b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.834+0000 7fac3b0e0b80 1 Processor -- start 2024-05-09T04:53:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.834+0000 7fac29c65700 1 --2- 172.21.15.46:0/626354706 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fac100b29f0 0x7fac100b4f00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fac24107cd0 tx=0x7fac14009190 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.834+0000 7fac3b0e0b80 1 -- start start 2024-05-09T04:53:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.834+0000 7fac3b0e0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a286a160 0x55c7a29034f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.834+0000 7fac3b0e0b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a27804a0 0x55c7a2903a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.835 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.834+0000 7fac3b0e0b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a28fd6a0 0x55c7a28fdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.834+0000 7fac3b0e0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c7a281fff0 con 0x55c7a286a160 2024-05-09T04:53:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.834+0000 7fac3b0e0b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c7a27be7f0 con 0x55c7a28fd6a0 2024-05-09T04:53:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.834+0000 7fac3b0e0b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c7a281bfb0 con 0x55c7a27804a0 2024-05-09T04:53:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.834+0000 7fac2a466700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a286a160 0x55c7a29034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.834+0000 7fac2a466700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a286a160 0x55c7a29034f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46866/0 (socket says 172.21.15.46:46866) 2024-05-09T04:53:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.834+0000 7fac2a466700 1 -- 172.21.15.46:0/2598480442 learned_addr learned my addr 172.21.15.46:0/2598480442 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.834+0000 7fac29c65700 1 --2- 172.21.15.46:0/2598480442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a27804a0 0x55c7a2903a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.834+0000 7fac2ac67700 1 --2- 172.21.15.46:0/2598480442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a28fd6a0 0x55c7a28fdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.835+0000 7fac2a466700 1 -- 172.21.15.46:0/2598480442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a27804a0 msgr2=0x55c7a2903a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.835+0000 7fac2a466700 1 --2- 172.21.15.46:0/2598480442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a27804a0 0x55c7a2903a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.835+0000 7fac2a466700 1 -- 172.21.15.46:0/2598480442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a28fd6a0 msgr2=0x55c7a28fdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.835+0000 7fac2a466700 1 --2- 172.21.15.46:0/2598480442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a28fd6a0 0x55c7a28fdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.835+0000 7fac2a466700 1 -- 172.21.15.46:0/2598480442 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7a27b6370 con 0x55c7a286a160 2024-05-09T04:53:58.836 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.835+0000 7fac2a466700 1 --2- 172.21.15.46:0/2598480442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a286a160 0x55c7a29034f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fac20022850 tx=0x7fac200228a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e484a1196cca18c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:58.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.835+0000 7fac18ff9700 1 -- 172.21.15.46:0/2598480442 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac200e2070 con 0x55c7a286a160 2024-05-09T04:53:58.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.835+0000 7fac18ff9700 1 -- 172.21.15.46:0/2598480442 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac200ae9f0 con 0x55c7a286a160 2024-05-09T04:53:58.837 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.835+0000 7fac18ff9700 1 -- 172.21.15.46:0/2598480442 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac200b2f80 con 0x55c7a286a160 2024-05-09T04:53:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.836+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/2598480442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a286a160 msgr2=0x55c7a29034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.836+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/2598480442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a286a160 0x55c7a29034f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fac20022850 tx=0x7fac200228a0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.836+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/2598480442 shutdown_connections 2024-05-09T04:53:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.836+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/2598480442 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a28fd6a0 0x55c7a28fdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.836+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/2598480442 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a286a160 0x55c7a29034f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.836+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/2598480442 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a27804a0 0x55c7a2903a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.836+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/2598480442 >> 172.21.15.46:0/2598480442 conn(0x55c7a27e06a0 msgr2=0x55c7a28f8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.836+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/2598480442 shutdown_connections 2024-05-09T04:53:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.836+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/2598480442 wait complete. 2024-05-09T04:53:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.836+0000 7fac3b0e0b80 1 -- start start 2024-05-09T04:53:58.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac3b0e0b80 1 Processor -- start 2024-05-09T04:53:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac3b0e0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a27804a0 0x55c7a290cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac3b0e0b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a286a160 0x55c7a290d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac3b0e0b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a28fd6a0 0x55c7a2907150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac3b0e0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c7a269bba0 con 0x55c7a27804a0 2024-05-09T04:53:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac3b0e0b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c7a27da150 con 0x55c7a28fd6a0 2024-05-09T04:53:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac3b0e0b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c7a28fe360 con 0x55c7a286a160 2024-05-09T04:53:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac3b0e0b80 10 client.0 _ll_register_callbacks cb 0x55c7a26916c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac2a466700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a27804a0 0x55c7a290cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac2a466700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a27804a0 0x55c7a290cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46872/0 (socket says 172.21.15.46:46872) 2024-05-09T04:53:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac2a466700 1 -- 172.21.15.46:0/1830666635 learned_addr learned my addr 172.21.15.46:0/1830666635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac2ac67700 1 --2- 172.21.15.46:0/1830666635 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a28fd6a0 0x55c7a2907150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac29c65700 1 --2- 172.21.15.46:0/1830666635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a286a160 0x55c7a290d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac2a466700 1 -- 172.21.15.46:0/1830666635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a286a160 msgr2=0x55c7a290d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:58.842 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac2a466700 1 --2- 172.21.15.46:0/1830666635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a286a160 0x55c7a290d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac2a466700 1 -- 172.21.15.46:0/1830666635 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a28fd6a0 msgr2=0x55c7a2907150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac2a466700 1 --2- 172.21.15.46:0/1830666635 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a28fd6a0 0x55c7a2907150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.837+0000 7fac2a466700 1 -- 172.21.15.46:0/1830666635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7a269b490 con 0x55c7a27804a0 2024-05-09T04:53:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.838+0000 7fac2a466700 1 --2- 172.21.15.46:0/1830666635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a27804a0 0x55c7a290cd50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fac200e32c0 tx=0x7fac20009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79218e0f2f69d75 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.838+0000 7fac0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.838+0000 7fac0dffb700 1 -- 172.21.15.46:0/1830666635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac200e2070 con 0x55c7a27804a0 2024-05-09T04:53:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.838+0000 7fac0dffb700 1 -- 172.21.15.46:0/1830666635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fac200b3190 con 0x55c7a27804a0 2024-05-09T04:53:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.838+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/1830666635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac200ae9f0 con 0x55c7a27804a0 2024-05-09T04:53:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.838+0000 7fac0dffb700 1 -- 172.21.15.46:0/1830666635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fac200b3410 con 0x55c7a27804a0 2024-05-09T04:53:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.838+0000 7fac3b0e0b80 10 client.16086 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.838+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/1830666635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c7a2914970 con 0x55c7a27804a0 2024-05-09T04:53:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.838+0000 7fac0dffb700 1 -- 172.21.15.46:0/1830666635 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac200ae9f0 con 0x55c7a27804a0 2024-05-09T04:53:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.838+0000 7fac0dffb700 1 -- 172.21.15.46:0/1830666635 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fac200b1ce0 con 0x55c7a27804a0 2024-05-09T04:53:58.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.838+0000 7fac3b0e0b80 10 client.16086 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.838+0000 7fac3b0e0b80 10 client.16086 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.838+0000 7fac3b0e0b80 20 client.16086 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.839+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/1830666635 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55c7a2916e40 0x55c7a2919260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.839+0000 7fac3b0e0b80 4 client.16086 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.839+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/1830666635 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7a282d9d0 con 0x55c7a2916e40 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.839+0000 7fac29c65700 1 --2- 172.21.15.46:0/1830666635 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55c7a2916e40 0x55c7a2919260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.839+0000 7fac0dffb700 10 client.16086 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.839+0000 7fac29c65700 1 --2- 172.21.15.46:0/1830666635 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55c7a2916e40 0x55c7a2919260 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fdb1975235c8cbb6 server_cookie=184eac69c8f86f3a in_seq=0 out_seq=0 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.840+0000 7fac0dffb700 1 -- 172.21.15.46:0/1830666635 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c7a282d9d0 con 0x55c7a2916e40 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.840+0000 7fac0dffb700 10 client.16086 handle_command_reply: tid=0 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.840+0000 7fac3b0e0b80 1 client.16086 shutdown 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.840+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/1830666635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a27804a0 msgr2=0x55c7a290cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.840+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/1830666635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a27804a0 0x55c7a290cd50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fac200e32c0 tx=0x7fac20009310 comp rx=0 tx=0).stop 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.840+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/1830666635 shutdown_connections 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.840+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/1830666635 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55c7a2916e40 0x55c7a2919260 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.840+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/1830666635 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a28fd6a0 0x55c7a2907150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.840+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/1830666635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a27804a0 0x55c7a290cd50 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.840+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/1830666635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a286a160 0x55c7a290d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.840+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/1830666635 >> 172.21.15.46:0/1830666635 conn(0x55c7a27e06a0 msgr2=0x55c7a28f81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.840+0000 7fac0dffb700 0 client.16086 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:53:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.841+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/1830666635 shutdown_connections 2024-05-09T04:53:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.841+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/1830666635 wait complete. 2024-05-09T04:53:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.841+0000 7fac3b0e0b80 20 client.16086 trim_cache size 0 max 16384 2024-05-09T04:53:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.856+0000 7fac3b0e0b80 1 Processor -- start 2024-05-09T04:53:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.856+0000 7fac3b0e0b80 1 -- start start 2024-05-09T04:53:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.856+0000 7fac3b0e0b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a27804a0 0x55c7a2901860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.856+0000 7fac3b0e0b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a286a160 0x55c7a28ffe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.856+0000 7fac3b0e0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a28fd6a0 0x55c7a291e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.856+0000 7fac3b0e0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fac241b3860 con 0x55c7a28fd6a0 2024-05-09T04:53:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.856+0000 7fac3b0e0b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fac241b10f0 con 0x55c7a286a160 2024-05-09T04:53:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.856+0000 7fac3b0e0b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fac241aef40 con 0x55c7a27804a0 2024-05-09T04:53:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.856+0000 7fac2ac67700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a28fd6a0 0x55c7a291e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.856+0000 7fac2ac67700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a28fd6a0 0x55c7a291e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46886/0 (socket says 172.21.15.46:46886) 2024-05-09T04:53:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.856+0000 7fac2ac67700 1 -- 172.21.15.46:0/3801957437 learned_addr learned my addr 172.21.15.46:0/3801957437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.856+0000 7fac29c65700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a286a160 0x55c7a28ffe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.856+0000 7fac2a466700 1 --2- 172.21.15.46:0/3801957437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a27804a0 0x55c7a2901860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.856+0000 7fac2ac67700 1 -- 172.21.15.46:0/3801957437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a27804a0 msgr2=0x55c7a2901860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.857+0000 7fac2ac67700 1 --2- 172.21.15.46:0/3801957437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a27804a0 0x55c7a2901860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.857+0000 7fac2ac67700 1 -- 172.21.15.46:0/3801957437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a286a160 msgr2=0x55c7a28ffe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.857+0000 7fac2ac67700 1 --2- 172.21.15.46:0/3801957437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a286a160 0x55c7a28ffe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.857+0000 7fac2ac67700 1 -- 172.21.15.46:0/3801957437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7a28f9d00 con 0x55c7a28fd6a0 2024-05-09T04:53:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.857+0000 7fac29c65700 1 --2- 172.21.15.46:0/3801957437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a286a160 0x55c7a28ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:53:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.857+0000 7fac2ac67700 1 --2- 172.21.15.46:0/3801957437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a28fd6a0 0x55c7a291e960 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009e50 tx=0x7fac1c0077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a40e7fa6d78ec8d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.857+0000 7fac0ffff700 1 -- 172.21.15.46:0/3801957437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c01b070 con 0x55c7a28fd6a0 2024-05-09T04:53:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.857+0000 7fac0ffff700 1 -- 172.21.15.46:0/3801957437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac1c0178f0 con 0x55c7a28fd6a0 2024-05-09T04:53:58.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.857+0000 7fac0ffff700 1 -- 172.21.15.46:0/3801957437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1c016900 con 0x55c7a28fd6a0 2024-05-09T04:53:58.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.858+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/3801957437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a28fd6a0 msgr2=0x55c7a291e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:58.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.858+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/3801957437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a28fd6a0 0x55c7a291e960 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009e50 tx=0x7fac1c0077e0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.858+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/3801957437 shutdown_connections 2024-05-09T04:53:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.858+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/3801957437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a28fd6a0 0x55c7a291e960 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.858+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/3801957437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a27804a0 0x55c7a2901860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.858+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/3801957437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a286a160 0x55c7a28ffe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.858+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/3801957437 >> 172.21.15.46:0/3801957437 conn(0x55c7a27e06a0 msgr2=0x55c7a29034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.858+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/3801957437 shutdown_connections 2024-05-09T04:53:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.858+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/3801957437 wait complete. 2024-05-09T04:53:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.858+0000 7fac3b0e0b80 1 -- start start 2024-05-09T04:53:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.858+0000 7fac3b0e0b80 1 Processor -- start 2024-05-09T04:53:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.858+0000 7fac3b0e0b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a27804a0 0x55c7a290e360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.858+0000 7fac3b0e0b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a286a160 0x55c7a290e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac3b0e0b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a28fd6a0 0x55c7a2987d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac3b0e0b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fac241af5b0 con 0x55c7a286a160 2024-05-09T04:53:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac3b0e0b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fac241b0a20 con 0x55c7a27804a0 2024-05-09T04:53:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac3b0e0b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fac241afc80 con 0x55c7a28fd6a0 2024-05-09T04:53:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac3b0e0b80 10 client.0 _ll_register_callbacks cb 0x55c7a26916c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:53:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac29c65700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a286a160 0x55c7a290e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac29c65700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a286a160 0x55c7a290e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46888/0 (socket says 172.21.15.46:46888) 2024-05-09T04:53:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac29c65700 1 -- 172.21.15.46:0/4009245645 learned_addr learned my addr 172.21.15.46:0/4009245645 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:53:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac2a466700 1 --2- 172.21.15.46:0/4009245645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a27804a0 0x55c7a290e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac29c65700 1 -- 172.21.15.46:0/4009245645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a28fd6a0 msgr2=0x55c7a2987d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac29c65700 1 --2- 172.21.15.46:0/4009245645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a28fd6a0 0x55c7a2987d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac29c65700 1 -- 172.21.15.46:0/4009245645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a27804a0 msgr2=0x55c7a290e360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:53:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac29c65700 1 --2- 172.21.15.46:0/4009245645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a27804a0 0x55c7a290e360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac29c65700 1 -- 172.21.15.46:0/4009245645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7a28f1f60 con 0x55c7a286a160 2024-05-09T04:53:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac2a466700 1 --2- 172.21.15.46:0/4009245645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a27804a0 0x55c7a290e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:53:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac29c65700 1 --2- 172.21.15.46:0/4009245645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a286a160 0x55c7a290e8a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fac1400e600 tx=0x7fac14004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e28e1220307d731 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:53:58.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac0dffb700 1 -- 172.21.15.46:0/4009245645 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1401b040 con 0x55c7a286a160 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/4009245645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac1c0178f0 con 0x55c7a286a160 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.859+0000 7fac0dffb700 1 -- 172.21.15.46:0/4009245645 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fac14028130 con 0x55c7a286a160 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.860+0000 7fac0dffb700 1 -- 172.21.15.46:0/4009245645 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fac14028330 con 0x55c7a286a160 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.860+0000 7fac3b0e0b80 10 client.16092 fetch_fsmap learned FSMap version 25 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.860+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/4009245645 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c7a2989240 con 0x55c7a286a160 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.860+0000 7fac0dffb700 1 -- 172.21.15.46:0/4009245645 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac1c0178f0 con 0x55c7a286a160 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.860+0000 7fac0dffb700 1 -- 172.21.15.46:0/4009245645 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fac14015ad0 con 0x55c7a286a160 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.860+0000 7fac3b0e0b80 10 client.16092 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.860+0000 7fac3b0e0b80 10 client.16092 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.860+0000 7fac3b0e0b80 20 client.16092 populate_metadata read hostname 'smithi046' 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.860+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/4009245645 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55c7a29898e0 0x55c7a298bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.860+0000 7fac3b0e0b80 4 client.16092 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.860+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/4009245645 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c7a2827a70 con 0x55c7a29898e0 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.860+0000 7fac2a466700 1 --2- 172.21.15.46:0/4009245645 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55c7a29898e0 0x55c7a298bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:53:58.865 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac0dffb700 10 client.16092 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:53:58.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac2a466700 1 --2- 172.21.15.46:0/4009245645 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55c7a29898e0 0x55c7a298bd00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b27988e1c82ffb97 server_cookie=390716e21cc342c5 in_seq=0 out_seq=0 2024-05-09T04:53:58.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac0dffb700 1 -- 172.21.15.46:0/4009245645 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c7a2827a70 con 0x55c7a29898e0 2024-05-09T04:53:58.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac0dffb700 10 client.16092 handle_command_reply: tid=0 2024-05-09T04:53:58.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac3b0e0b80 1 client.16092 shutdown 2024-05-09T04:53:58.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/4009245645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a286a160 msgr2=0x55c7a290e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:58.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/4009245645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a286a160 0x55c7a290e8a0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fac1400e600 tx=0x7fac14004750 comp rx=0 tx=0).stop 2024-05-09T04:53:58.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/4009245645 shutdown_connections 2024-05-09T04:53:58.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/4009245645 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55c7a29898e0 0x55c7a298bd00 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac0dffb700 0 client.16092 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:53:58.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/4009245645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7a28fd6a0 0x55c7a2987d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/4009245645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7a27804a0 0x55c7a290e360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac3b0e0b80 1 --2- 172.21.15.46:0/4009245645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7a286a160 0x55c7a290e8a0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/4009245645 >> 172.21.15.46:0/4009245645 conn(0x55c7a27e06a0 msgr2=0x55c7a29034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:58.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/4009245645 shutdown_connections 2024-05-09T04:53:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac3b0e0b80 1 -- 172.21.15.46:0/4009245645 wait complete. 2024-05-09T04:53:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.861+0000 7fac3b0e0b80 20 client.16092 trim_cache size 0 max 16384 2024-05-09T04:53:58.867 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:53:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.862+0000 7fac2c6ca700 1 -- 172.21.15.46:0/626354706 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fac100b29f0 msgr2=0x7fac100b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.862+0000 7fac2c6ca700 1 --2- 172.21.15.46:0/626354706 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fac100b29f0 0x7fac100b4f00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fac24107cd0 tx=0x7fac14009190 comp rx=0 tx=0).stop 2024-05-09T04:53:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.862+0000 7fac2c6ca700 1 -- 172.21.15.46:0/626354706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac241034c0 msgr2=0x7fac240ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:53:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.862+0000 7fac2c6ca700 1 --2- 172.21.15.46:0/626354706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac241034c0 0x7fac240ff2a0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fac2000b580 tx=0x7fac200047b0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.862+0000 7fac2c6ca700 1 -- 172.21.15.46:0/626354706 shutdown_connections 2024-05-09T04:53:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.862+0000 7fac2c6ca700 1 --2- 172.21.15.46:0/626354706 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fac100b29f0 0x7fac100b4f00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.862+0000 7fac2c6ca700 1 --2- 172.21.15.46:0/626354706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fac24108740 0x7fac240ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.862+0000 7fac2c6ca700 1 --2- 172.21.15.46:0/626354706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fac241034c0 0x7fac240ff2a0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.862+0000 7fac2c6ca700 1 --2- 172.21.15.46:0/626354706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fac24107df0 0x7fac240ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:53:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.862+0000 7fac2c6ca700 1 -- 172.21.15.46:0/626354706 >> 172.21.15.46:0/626354706 conn(0x7fac240fedc0 msgr2=0x7fac2406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:53:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.862+0000 7fac2c6ca700 1 -- 172.21.15.46:0/626354706 shutdown_connections 2024-05-09T04:53:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:53:58.862+0000 7fac2c6ca700 1 -- 172.21.15.46:0/626354706 wait complete. 2024-05-09T04:53:59.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:53:58 smithi110 ceph-mon[34904]: pgmap v89: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 853 MiB/s wr, 356 op/s 2024-05-09T04:53:59.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:53:58 smithi046 ceph-mon[30136]: pgmap v89: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 853 MiB/s wr, 356 op/s 2024-05-09T04:53:59.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:53:58 smithi072 ceph-mon[34088]: pgmap v89: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 340 KiB/s rd, 853 MiB/s wr, 356 op/s 2024-05-09T04:53:59.634 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:53:59.926 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:00.201 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:54:00.201 INFO:tasks.workunit.client.0.smithi046.stdout: Children see throughput for 1 readers = 8929538.00 kB/sec 2024-05-09T04:54:00.201 INFO:tasks.workunit.client.0.smithi046.stdout: Parent sees throughput for 1 readers = 8927895.90 kB/sec 2024-05-09T04:54:00.201 INFO:tasks.workunit.client.0.smithi046.stdout: Min throughput per process = 8929538.00 kB/sec 2024-05-09T04:54:00.201 INFO:tasks.workunit.client.0.smithi046.stdout: Max throughput per process = 8929538.00 kB/sec 2024-05-09T04:54:00.201 INFO:tasks.workunit.client.0.smithi046.stdout: Avg throughput per process = 8929538.00 kB/sec 2024-05-09T04:54:00.201 INFO:tasks.workunit.client.0.smithi046.stdout: Min xfer = 10485760.00 kB 2024-05-09T04:54:00.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:00 smithi046 ceph-mon[30136]: pgmap v90: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 643 MiB/s wr, 232 op/s 2024-05-09T04:54:01.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:00 smithi110 ceph-mon[34904]: pgmap v90: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 643 MiB/s wr, 232 op/s 2024-05-09T04:54:01.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:00 smithi072 ceph-mon[34088]: pgmap v90: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 643 MiB/s wr, 232 op/s 2024-05-09T04:54:02.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.017+0000 7f851a3be700 1 -- 172.21.15.46:0/4151534467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8514109850 msgr2=0x7f85141111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:02.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.017+0000 7f851a3be700 1 --2- 172.21.15.46:0/4151534467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8514109850 0x7f85141111b0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f850c00b0d0 tx=0x7f850c00b3e0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.017+0000 7f851a3be700 1 -- 172.21.15.46:0/4151534467 shutdown_connections 2024-05-09T04:54:02.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.017+0000 7f851a3be700 1 --2- 172.21.15.46:0/4151534467 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8514109850 0x7f85141111b0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.017+0000 7f851a3be700 1 --2- 172.21.15.46:0/4151534467 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f85141045d0 0x7f85141049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.017+0000 7f851a3be700 1 --2- 172.21.15.46:0/4151534467 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8514108f00 0x7f8514109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.017+0000 7f851a3be700 1 -- 172.21.15.46:0/4151534467 >> 172.21.15.46:0/4151534467 conn(0x7f85140ffea0 msgr2=0x7f85141022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:02.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.017+0000 7f851a3be700 1 -- 172.21.15.46:0/4151534467 shutdown_connections 2024-05-09T04:54:02.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.017+0000 7f851a3be700 1 -- 172.21.15.46:0/4151534467 wait complete. 2024-05-09T04:54:02.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.017+0000 7f851a3be700 1 Processor -- start 2024-05-09T04:54:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f851a3be700 1 -- start start 2024-05-09T04:54:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f851a3be700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85141045d0 0x7f8514076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f851a3be700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8514108f00 0x7f8514077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f851a3be700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8514109850 0x7f8514077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f851a3be700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8514114480 con 0x7f8514108f00 2024-05-09T04:54:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f851a3be700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8514114300 con 0x7f8514109850 2024-05-09T04:54:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f851a3be700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8514114600 con 0x7f85141045d0 2024-05-09T04:54:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f850bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8514108f00 0x7f8514077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f850bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8514108f00 0x7f8514077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46892/0 (socket says 172.21.15.46:46892) 2024-05-09T04:54:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f850bfff700 1 -- 172.21.15.46:0/3117363263 learned_addr learned my addr 172.21.15.46:0/3117363263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f8513fff700 1 --2- 172.21.15.46:0/3117363263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85141045d0 0x7f8514076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f851895b700 1 --2- 172.21.15.46:0/3117363263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8514109850 0x7f8514077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f850bfff700 1 -- 172.21.15.46:0/3117363263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85141045d0 msgr2=0x7f8514076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:02.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f850bfff700 1 --2- 172.21.15.46:0/3117363263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85141045d0 0x7f8514076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f850bfff700 1 -- 172.21.15.46:0/3117363263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8514109850 msgr2=0x7f8514077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:02.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f850bfff700 1 --2- 172.21.15.46:0/3117363263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8514109850 0x7f8514077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.018+0000 7f850bfff700 1 -- 172.21.15.46:0/3117363263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85141a9080 con 0x7f8514108f00 2024-05-09T04:54:02.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.019+0000 7f850bfff700 1 --2- 172.21.15.46:0/3117363263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8514108f00 0x7f8514077110 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c9e0 tx=0x7f84fc00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:02.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.019+0000 7f8511ffb700 1 -- 172.21.15.46:0/3117363263 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc007aa0 con 0x7f8514108f00 2024-05-09T04:54:02.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.019+0000 7f8511ffb700 1 -- 172.21.15.46:0/3117363263 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84fc007c40 con 0x7f8514108f00 2024-05-09T04:54:02.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.019+0000 7f851a3be700 1 -- 172.21.15.46:0/3117363263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85141a9370 con 0x7f8514108f00 2024-05-09T04:54:02.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.019+0000 7f851a3be700 1 -- 172.21.15.46:0/3117363263 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85141a98b0 con 0x7f8514108f00 2024-05-09T04:54:02.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.019+0000 7f8511ffb700 1 -- 172.21.15.46:0/3117363263 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc00fe60 con 0x7f8514108f00 2024-05-09T04:54:02.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.021+0000 7f8511ffb700 1 -- 172.21.15.46:0/3117363263 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f84fc00f450 con 0x7f8514108f00 2024-05-09T04:54:02.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.022+0000 7f8511ffb700 1 --2- 172.21.15.46:0/3117363263 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f85000b29f0 0x7f85000b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.022+0000 7f8511ffb700 1 -- 172.21.15.46:0/3117363263 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f84fc0e24c0 con 0x7f8514108f00 2024-05-09T04:54:02.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.022+0000 7f8513fff700 1 --2- 172.21.15.46:0/3117363263 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f85000b29f0 0x7f85000b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.023+0000 7f8528dd4b80 1 Processor -- start 2024-05-09T04:54:02.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.023+0000 7f8528dd4b80 1 -- start start 2024-05-09T04:54:02.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.023+0000 7f8513fff700 1 --2- 172.21.15.46:0/3117363263 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f85000b29f0 0x7f85000b4f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8504005950 tx=0x7f850400a300 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:02.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.023+0000 7f8528dd4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a06ad00 0x56016a173240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.023+0000 7f8528dd4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560169ff04a0 0x56016a173780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.023+0000 7f8528dd4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a16d400 0x56016a16d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.023+0000 7f8528dd4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56016a0900a0 con 0x560169ff04a0 2024-05-09T04:54:02.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.023+0000 7f8528dd4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56016a0359a0 con 0x56016a16d400 2024-05-09T04:54:02.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.023+0000 7f8528dd4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56016a08c060 con 0x56016a06ad00 2024-05-09T04:54:02.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.023+0000 7f850bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560169ff04a0 0x56016a173780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.023+0000 7f850bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560169ff04a0 0x56016a173780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46906/0 (socket says 172.21.15.46:46906) 2024-05-09T04:54:02.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.023+0000 7f850bfff700 1 -- 172.21.15.46:0/3716356023 learned_addr learned my addr 172.21.15.46:0/3716356023 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:02.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.023+0000 7f8513fff700 1 --2- 172.21.15.46:0/3716356023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a06ad00 0x56016a173240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.023+0000 7f851895b700 1 --2- 172.21.15.46:0/3716356023 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a16d400 0x56016a16d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.024+0000 7f8513fff700 1 -- 172.21.15.46:0/3716356023 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a16d400 msgr2=0x56016a16d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:02.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.024+0000 7f8513fff700 1 --2- 172.21.15.46:0/3716356023 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a16d400 0x56016a16d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.024+0000 7f8513fff700 1 -- 172.21.15.46:0/3716356023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560169ff04a0 msgr2=0x56016a173780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.024+0000 7f8513fff700 1 --2- 172.21.15.46:0/3716356023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560169ff04a0 0x56016a173780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.024+0000 7f8513fff700 1 -- 172.21.15.46:0/3716356023 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56016a026500 con 0x56016a06ad00 2024-05-09T04:54:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.024+0000 7f850bfff700 1 --2- 172.21.15.46:0/3716356023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560169ff04a0 0x56016a173780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.024+0000 7f8513fff700 1 --2- 172.21.15.46:0/3716356023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a06ad00 0x56016a173240 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f850400fd00 tx=0x7f8504012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c148db2827382850 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.024+0000 7f850a7fc700 1 -- 172.21.15.46:0/3716356023 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504016070 con 0x56016a06ad00 2024-05-09T04:54:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.024+0000 7f850a7fc700 1 -- 172.21.15.46:0/3716356023 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8504010d20 con 0x56016a06ad00 2024-05-09T04:54:02.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.024+0000 7f850a7fc700 1 -- 172.21.15.46:0/3716356023 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850401bdf0 con 0x56016a06ad00 2024-05-09T04:54:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.025+0000 7f8528dd4b80 1 -- 172.21.15.46:0/3716356023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a06ad00 msgr2=0x56016a173240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.025+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/3716356023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a06ad00 0x56016a173240 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f850400fd00 tx=0x7f8504012000 comp rx=0 tx=0).stop 2024-05-09T04:54:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.025+0000 7f8528dd4b80 1 -- 172.21.15.46:0/3716356023 shutdown_connections 2024-05-09T04:54:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.025+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/3716356023 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a16d400 0x56016a16d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.025+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/3716356023 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a06ad00 0x56016a173240 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.025+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/3716356023 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560169ff04a0 0x56016a173780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.025+0000 7f8528dd4b80 1 -- 172.21.15.46:0/3716356023 >> 172.21.15.46:0/3716356023 conn(0x560169ff3f70 msgr2=0x56016a168220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.025+0000 7f8528dd4b80 1 -- 172.21.15.46:0/3716356023 shutdown_connections 2024-05-09T04:54:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.025+0000 7f8528dd4b80 1 -- 172.21.15.46:0/3716356023 wait complete. 2024-05-09T04:54:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.025+0000 7f8528dd4b80 1 -- start start 2024-05-09T04:54:02.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.026+0000 7f8528dd4b80 1 Processor -- start 2024-05-09T04:54:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.026+0000 7f8528dd4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560169ff04a0 0x56016a177020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.026+0000 7f8528dd4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56016a06ad00 0x56016a177560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.026+0000 7f8528dd4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a16d400 0x56016a177aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.026+0000 7f8528dd4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56016a16e160 con 0x56016a06ad00 2024-05-09T04:54:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.026+0000 7f8528dd4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56016a02ab20 con 0x560169ff04a0 2024-05-09T04:54:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.026+0000 7f8528dd4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56016a17a020 con 0x56016a16d400 2024-05-09T04:54:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.026+0000 7f8528dd4b80 10 client.0 _ll_register_callbacks cb 0x560169f016c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.026+0000 7f850bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56016a06ad00 0x56016a177560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.026+0000 7f850bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56016a06ad00 0x56016a177560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46910/0 (socket says 172.21.15.46:46910) 2024-05-09T04:54:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.026+0000 7f850bfff700 1 -- 172.21.15.46:0/2277797607 learned_addr learned my addr 172.21.15.46:0/2277797607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.026+0000 7f8513fff700 1 --2- 172.21.15.46:0/2277797607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560169ff04a0 0x56016a177020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.026+0000 7f851895b700 1 --2- 172.21.15.46:0/2277797607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a16d400 0x56016a177aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.027+0000 7f850bfff700 1 -- 172.21.15.46:0/2277797607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a16d400 msgr2=0x56016a177aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:02.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.027+0000 7f850bfff700 1 --2- 172.21.15.46:0/2277797607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a16d400 0x56016a177aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.027+0000 7f850bfff700 1 -- 172.21.15.46:0/2277797607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560169ff04a0 msgr2=0x56016a177020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.027+0000 7f850bfff700 1 --2- 172.21.15.46:0/2277797607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560169ff04a0 0x56016a177020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.027+0000 7f850bfff700 1 -- 172.21.15.46:0/2277797607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560169f0b490 con 0x56016a06ad00 2024-05-09T04:54:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.027+0000 7f850bfff700 1 --2- 172.21.15.46:0/2277797607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56016a06ad00 0x56016a177560 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0dfca0 tx=0x7f84fc0a9310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b709156e915af7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.027+0000 7f84f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.027+0000 7f84f37fe700 1 -- 172.21.15.46:0/2277797607 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84fc0a9780 con 0x56016a06ad00 2024-05-09T04:54:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.027+0000 7f84f37fe700 1 -- 172.21.15.46:0/2277797607 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f84fc0a5d70 con 0x56016a06ad00 2024-05-09T04:54:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.027+0000 7f8528dd4b80 1 -- 172.21.15.46:0/2277797607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8504010d20 con 0x56016a06ad00 2024-05-09T04:54:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.027+0000 7f84f37fe700 1 -- 172.21.15.46:0/2277797607 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f84fc0adb30 con 0x56016a06ad00 2024-05-09T04:54:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.027+0000 7f8528dd4b80 10 client.16104 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.027+0000 7f8528dd4b80 1 -- 172.21.15.46:0/2277797607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56016a1849f0 con 0x56016a06ad00 2024-05-09T04:54:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.027+0000 7f84f37fe700 1 -- 172.21.15.46:0/2277797607 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8504010d20 con 0x56016a06ad00 2024-05-09T04:54:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.028+0000 7f84f37fe700 1 -- 172.21.15.46:0/2277797607 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f84fc0b2150 con 0x56016a06ad00 2024-05-09T04:54:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.028+0000 7f8528dd4b80 10 client.16104 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:02.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.028+0000 7f8528dd4b80 10 client.16104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.028+0000 7f8528dd4b80 20 client.16104 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.028+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/2277797607 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56016a187030 0x56016a189450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.028+0000 7f8528dd4b80 4 client.16104 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.028+0000 7f8528dd4b80 1 -- 172.21.15.46:0/2277797607 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56016a09da80 con 0x56016a187030 2024-05-09T04:54:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.028+0000 7f8513fff700 1 --2- 172.21.15.46:0/2277797607 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56016a187030 0x56016a189450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.028+0000 7f8513fff700 1 --2- 172.21.15.46:0/2277797607 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56016a187030 0x56016a189450 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4efd6e2284a0a0b server_cookie=36ea1447a80b7071 in_seq=0 out_seq=0 2024-05-09T04:54:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.028+0000 7f84f37fe700 10 client.16104 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.029+0000 7f84f37fe700 1 -- 172.21.15.46:0/2277797607 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56016a09da80 con 0x56016a187030 2024-05-09T04:54:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.029+0000 7f84f37fe700 10 client.16104 handle_command_reply: tid=0 2024-05-09T04:54:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.029+0000 7f8528dd4b80 1 client.16104 shutdown 2024-05-09T04:54:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.029+0000 7f8528dd4b80 1 -- 172.21.15.46:0/2277797607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56016a06ad00 msgr2=0x56016a177560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.029+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/2277797607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56016a06ad00 0x56016a177560 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f84fc0dfca0 tx=0x7f84fc0a9310 comp rx=0 tx=0).stop 2024-05-09T04:54:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.029+0000 7f8528dd4b80 1 -- 172.21.15.46:0/2277797607 shutdown_connections 2024-05-09T04:54:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.029+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/2277797607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a16d400 0x56016a177aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.029+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/2277797607 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56016a187030 0x56016a189450 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.029+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/2277797607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560169ff04a0 0x56016a177020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.029+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/2277797607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56016a06ad00 0x56016a177560 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.029+0000 7f8528dd4b80 1 -- 172.21.15.46:0/2277797607 >> 172.21.15.46:0/2277797607 conn(0x560169ff3f70 msgr2=0x56016a161ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.030+0000 7f84f37fe700 0 client.16104 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.030+0000 7f8528dd4b80 1 -- 172.21.15.46:0/2277797607 shutdown_connections 2024-05-09T04:54:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.030+0000 7f8528dd4b80 1 -- 172.21.15.46:0/2277797607 wait complete. 2024-05-09T04:54:02.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.030+0000 7f8528dd4b80 20 client.16104 trim_cache size 0 max 16384 2024-05-09T04:54:02.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.045+0000 7f8528dd4b80 1 Processor -- start 2024-05-09T04:54:02.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.045+0000 7f8528dd4b80 1 -- start start 2024-05-09T04:54:02.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.045+0000 7f8528dd4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560169ff04a0 0x56016a172880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.045+0000 7f8528dd4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a06ad00 0x56016a161ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.045+0000 7f8528dd4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56016a16d400 0x56016a162530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.045+0000 7f8528dd4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85141b3950 con 0x56016a16d400 2024-05-09T04:54:02.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.045+0000 7f8528dd4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f85141b11e0 con 0x56016a06ad00 2024-05-09T04:54:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.045+0000 7f8528dd4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85141af030 con 0x560169ff04a0 2024-05-09T04:54:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.045+0000 7f851895b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56016a16d400 0x56016a162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.045+0000 7f851895b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56016a16d400 0x56016a162530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46912/0 (socket says 172.21.15.46:46912) 2024-05-09T04:54:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.045+0000 7f851895b700 1 -- 172.21.15.46:0/855194229 learned_addr learned my addr 172.21.15.46:0/855194229 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.045+0000 7f8513fff700 1 --2- 172.21.15.46:0/855194229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560169ff04a0 0x56016a172880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.045+0000 7f850bfff700 1 --2- 172.21.15.46:0/855194229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a06ad00 0x56016a161ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.045+0000 7f851895b700 1 -- 172.21.15.46:0/855194229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560169ff04a0 msgr2=0x56016a172880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.045+0000 7f851895b700 1 --2- 172.21.15.46:0/855194229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560169ff04a0 0x56016a172880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.046+0000 7f851895b700 1 -- 172.21.15.46:0/855194229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a06ad00 msgr2=0x56016a161ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.046+0000 7f851895b700 1 --2- 172.21.15.46:0/855194229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a06ad00 0x56016a161ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.046+0000 7f851895b700 1 -- 172.21.15.46:0/855194229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56016a1649d0 con 0x56016a16d400 2024-05-09T04:54:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.046+0000 7f851895b700 1 --2- 172.21.15.46:0/855194229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56016a16d400 0x56016a162530 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f850c015000 tx=0x7f850c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d699b3102f12a0e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.046+0000 7f8509ffb700 1 -- 172.21.15.46:0/855194229 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c003c00 con 0x56016a16d400 2024-05-09T04:54:02.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.046+0000 7f8509ffb700 1 -- 172.21.15.46:0/855194229 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850c003da0 con 0x56016a16d400 2024-05-09T04:54:02.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.046+0000 7f8509ffb700 1 -- 172.21.15.46:0/855194229 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f850c005610 con 0x56016a16d400 2024-05-09T04:54:02.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.047+0000 7f8528dd4b80 1 -- 172.21.15.46:0/855194229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56016a16d400 msgr2=0x56016a162530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:02.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.047+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/855194229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56016a16d400 0x56016a162530 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f850c015000 tx=0x7f850c00b040 comp rx=0 tx=0).stop 2024-05-09T04:54:02.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.047+0000 7f8528dd4b80 1 -- 172.21.15.46:0/855194229 shutdown_connections 2024-05-09T04:54:02.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.047+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/855194229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56016a16d400 0x56016a162530 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.047+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/855194229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560169ff04a0 0x56016a172880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.047+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/855194229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a06ad00 0x56016a161ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.047+0000 7f8528dd4b80 1 -- 172.21.15.46:0/855194229 >> 172.21.15.46:0/855194229 conn(0x560169ff3f70 msgr2=0x56016a172170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:02.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.047+0000 7f8528dd4b80 1 -- 172.21.15.46:0/855194229 shutdown_connections 2024-05-09T04:54:02.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.047+0000 7f8528dd4b80 1 -- 172.21.15.46:0/855194229 wait complete. 2024-05-09T04:54:02.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.047+0000 7f8528dd4b80 1 -- start start 2024-05-09T04:54:02.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.047+0000 7f8528dd4b80 1 Processor -- start 2024-05-09T04:54:02.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8528dd4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560169ff04a0 0x56016a17f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8528dd4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a06ad00 0x56016a18e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8528dd4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a16d400 0x56016a18e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8528dd4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f85141af6a0 con 0x560169ff04a0 2024-05-09T04:54:02.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8528dd4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f85141b0b10 con 0x56016a16d400 2024-05-09T04:54:02.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8528dd4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f85141afd70 con 0x56016a06ad00 2024-05-09T04:54:02.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8528dd4b80 10 client.0 _ll_register_callbacks cb 0x560169f016c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:02.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8513fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560169ff04a0 0x56016a17f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8513fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560169ff04a0 0x56016a17f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46918/0 (socket says 172.21.15.46:46918) 2024-05-09T04:54:02.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8513fff700 1 -- 172.21.15.46:0/782550600 learned_addr learned my addr 172.21.15.46:0/782550600 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:02.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f850bfff700 1 --2- 172.21.15.46:0/782550600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a06ad00 0x56016a18e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f851895b700 1 --2- 172.21.15.46:0/782550600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a16d400 0x56016a18e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8513fff700 1 -- 172.21.15.46:0/782550600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a06ad00 msgr2=0x56016a18e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:02.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8513fff700 1 --2- 172.21.15.46:0/782550600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a06ad00 0x56016a18e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8513fff700 1 -- 172.21.15.46:0/782550600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a16d400 msgr2=0x56016a18e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:02.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8513fff700 1 --2- 172.21.15.46:0/782550600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a16d400 0x56016a18e730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8513fff700 1 -- 172.21.15.46:0/782550600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56016a173af0 con 0x560169ff04a0 2024-05-09T04:54:02.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f851895b700 1 --2- 172.21.15.46:0/782550600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a16d400 0x56016a18e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:02.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f8513fff700 1 --2- 172.21.15.46:0/782550600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560169ff04a0 0x56016a17f6c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f850400fcd0 tx=0x7f850401b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c07ca2c76bdb92 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:02.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.048+0000 7f84f37fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:02.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.049+0000 7f84f37fe700 1 -- 172.21.15.46:0/782550600 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8504035020 con 0x560169ff04a0 2024-05-09T04:54:02.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.049+0000 7f84f37fe700 1 -- 172.21.15.46:0/782550600 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f850401bc20 con 0x560169ff04a0 2024-05-09T04:54:02.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.049+0000 7f8528dd4b80 1 -- 172.21.15.46:0/782550600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f850c003da0 con 0x560169ff04a0 2024-05-09T04:54:02.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.049+0000 7f84f37fe700 1 -- 172.21.15.46:0/782550600 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8504040050 con 0x560169ff04a0 2024-05-09T04:54:02.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.049+0000 7f8528dd4b80 10 client.16110 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:02.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.049+0000 7f8528dd4b80 1 -- 172.21.15.46:0/782550600 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56016a1f1400 con 0x560169ff04a0 2024-05-09T04:54:02.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.049+0000 7f84f37fe700 1 -- 172.21.15.46:0/782550600 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f850c003da0 con 0x560169ff04a0 2024-05-09T04:54:02.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.049+0000 7f84f37fe700 1 -- 172.21.15.46:0/782550600 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f85040315b0 con 0x560169ff04a0 2024-05-09T04:54:02.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.049+0000 7f8528dd4b80 10 client.16110 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:02.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.049+0000 7f8528dd4b80 10 client.16110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:02.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.049+0000 7f8528dd4b80 20 client.16110 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:02.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.049+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/782550600 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56016a1f1830 0x56016a1f3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:02.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.049+0000 7f8528dd4b80 4 client.16110 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:54:02.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.049+0000 7f8528dd4b80 1 -- 172.21.15.46:0/782550600 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56016a097b20 con 0x56016a1f1830 2024-05-09T04:54:02.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.050+0000 7f850bfff700 1 --2- 172.21.15.46:0/782550600 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56016a1f1830 0x56016a1f3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:02.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.050+0000 7f850bfff700 1 --2- 172.21.15.46:0/782550600 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56016a1f1830 0x56016a1f3c50 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9248e5d916b2dd52 server_cookie=1bb999e7fd689327 in_seq=0 out_seq=0 2024-05-09T04:54:02.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.050+0000 7f84f37fe700 10 client.16110 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:02.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.050+0000 7f84f37fe700 1 -- 172.21.15.46:0/782550600 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56016a097b20 con 0x56016a1f1830 2024-05-09T04:54:02.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.050+0000 7f84f37fe700 10 client.16110 handle_command_reply: tid=0 2024-05-09T04:54:02.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.050+0000 7f8528dd4b80 1 client.16110 shutdown 2024-05-09T04:54:02.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f8528dd4b80 1 -- 172.21.15.46:0/782550600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560169ff04a0 msgr2=0x56016a17f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:02.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/782550600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560169ff04a0 0x56016a17f6c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f850400fcd0 tx=0x7f850401b860 comp rx=0 tx=0).stop 2024-05-09T04:54:02.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f8528dd4b80 1 -- 172.21.15.46:0/782550600 shutdown_connections 2024-05-09T04:54:02.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/782550600 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56016a1f1830 0x56016a1f3c50 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/782550600 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56016a16d400 0x56016a18e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/782550600 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560169ff04a0 0x56016a17f6c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f8528dd4b80 1 --2- 172.21.15.46:0/782550600 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56016a06ad00 0x56016a18e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f84f37fe700 0 client.16110 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:02.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f8528dd4b80 1 -- 172.21.15.46:0/782550600 >> 172.21.15.46:0/782550600 conn(0x560169ff3f70 msgr2=0x56016a161ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:02.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f8528dd4b80 1 -- 172.21.15.46:0/782550600 shutdown_connections 2024-05-09T04:54:02.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f8528dd4b80 1 -- 172.21.15.46:0/782550600 wait complete. 2024-05-09T04:54:02.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f8528dd4b80 20 client.16110 trim_cache size 0 max 16384 2024-05-09T04:54:02.056 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:54:02.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f851a3be700 1 -- 172.21.15.46:0/3117363263 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f85000b29f0 msgr2=0x7f85000b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:02.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f851a3be700 1 --2- 172.21.15.46:0/3117363263 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f85000b29f0 0x7f85000b4f00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8504005950 tx=0x7f850400a300 comp rx=0 tx=0).stop 2024-05-09T04:54:02.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f851a3be700 1 -- 172.21.15.46:0/3117363263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8514108f00 msgr2=0x7f8514077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:02.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f851a3be700 1 --2- 172.21.15.46:0/3117363263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8514108f00 0x7f8514077110 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f84fc00c9e0 tx=0x7f84fc00ccf0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f851a3be700 1 -- 172.21.15.46:0/3117363263 shutdown_connections 2024-05-09T04:54:02.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f851a3be700 1 --2- 172.21.15.46:0/3117363263 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f85000b29f0 0x7f85000b4f00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f851a3be700 1 --2- 172.21.15.46:0/3117363263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8514109850 0x7f8514077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f851a3be700 1 --2- 172.21.15.46:0/3117363263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f85141045d0 0x7f8514076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f851a3be700 1 --2- 172.21.15.46:0/3117363263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8514108f00 0x7f8514077110 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:02.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.051+0000 7f851a3be700 1 -- 172.21.15.46:0/3117363263 >> 172.21.15.46:0/3117363263 conn(0x7f85140ffea0 msgr2=0x7f8514108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:02.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.052+0000 7f851a3be700 1 -- 172.21.15.46:0/3117363263 shutdown_connections 2024-05-09T04:54:02.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:02.052+0000 7f851a3be700 1 -- 172.21.15.46:0/3117363263 wait complete. 2024-05-09T04:54:02.896 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:02 smithi046 ceph-mon[30136]: pgmap v91: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 643 MiB/s wr, 236 op/s 2024-05-09T04:54:03.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:02 smithi110 ceph-mon[34904]: pgmap v91: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 643 MiB/s wr, 236 op/s 2024-05-09T04:54:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:02 smithi072 ceph-mon[34088]: pgmap v91: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 172 KiB/s rd, 643 MiB/s wr, 236 op/s 2024-05-09T04:54:03.542 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:54:03.542 INFO:tasks.workunit.client.0.smithi046.stdout: Children see throughput for 1 re-readers = 9050199.00 kB/sec 2024-05-09T04:54:03.542 INFO:tasks.workunit.client.0.smithi046.stdout: Parent sees throughput for 1 re-readers = 9048402.65 kB/sec 2024-05-09T04:54:03.542 INFO:tasks.workunit.client.0.smithi046.stdout: Min throughput per process = 9050199.00 kB/sec 2024-05-09T04:54:03.542 INFO:tasks.workunit.client.0.smithi046.stdout: Max throughput per process = 9050199.00 kB/sec 2024-05-09T04:54:03.542 INFO:tasks.workunit.client.0.smithi046.stdout: Avg throughput per process = 9050199.00 kB/sec 2024-05-09T04:54:03.542 INFO:tasks.workunit.client.0.smithi046.stdout: Min xfer = 10485760.00 kB 2024-05-09T04:54:03.939 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:54:03.939 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:54:04.193 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:05.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:04 smithi110 ceph-mon[34904]: pgmap v92: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 445 MiB/s wr, 134 op/s 2024-05-09T04:54:05.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:04 smithi046 ceph-mon[30136]: pgmap v92: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 445 MiB/s wr, 134 op/s 2024-05-09T04:54:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:04 smithi072 ceph-mon[34088]: pgmap v92: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 445 MiB/s wr, 134 op/s 2024-05-09T04:54:05.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.880+0000 7fbc1d150700 1 -- 172.21.15.46:0/2054126979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18103550 msgr2=0x7fbc18103960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:05.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.880+0000 7fbc1d150700 1 --2- 172.21.15.46:0/2054126979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18103550 0x7fbc18103960 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fbc080099c0 tx=0x7fbc08009cd0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.880+0000 7fbc1d150700 1 -- 172.21.15.46:0/2054126979 shutdown_connections 2024-05-09T04:54:05.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.880+0000 7fbc1d150700 1 --2- 172.21.15.46:0/2054126979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18108780 0x7fbc181100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.880+0000 7fbc1d150700 1 --2- 172.21.15.46:0/2054126979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18103550 0x7fbc18103960 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.880+0000 7fbc1d150700 1 --2- 172.21.15.46:0/2054126979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc18107e30 0x7fbc18108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.881 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.880+0000 7fbc1d150700 1 -- 172.21.15.46:0/2054126979 >> 172.21.15.46:0/2054126979 conn(0x7fbc18078e00 msgr2=0x7fbc18079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:05.881 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:05 smithi046 ceph-mon[30136]: pgmap v93: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 280 MiB/s wr, 92 op/s 2024-05-09T04:54:05.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.880+0000 7fbc1d150700 1 -- 172.21.15.46:0/2054126979 shutdown_connections 2024-05-09T04:54:05.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.880+0000 7fbc1d150700 1 -- 172.21.15.46:0/2054126979 wait complete. 2024-05-09T04:54:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.881+0000 7fbc1d150700 1 Processor -- start 2024-05-09T04:54:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1d150700 1 -- start start 2024-05-09T04:54:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1d150700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc18103550 0x7fbc1819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1d150700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18107e30 0x7fbc1819c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1d150700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18108780 0x7fbc1819c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1d150700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbc181133b0 con 0x7fbc18107e30 2024-05-09T04:54:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1d150700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbc18113230 con 0x7fbc18103550 2024-05-09T04:54:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1d150700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbc18113530 con 0x7fbc18108780 2024-05-09T04:54:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18107e30 0x7fbc1819c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18107e30 0x7fbc1819c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57072/0 (socket says 172.21.15.46:57072) 2024-05-09T04:54:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1659c700 1 -- 172.21.15.46:0/2616345358 learned_addr learned my addr 172.21.15.46:0/2616345358 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc16d9d700 1 --2- 172.21.15.46:0/2616345358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc18103550 0x7fbc1819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1759e700 1 --2- 172.21.15.46:0/2616345358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18108780 0x7fbc1819c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1659c700 1 -- 172.21.15.46:0/2616345358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18108780 msgr2=0x7fbc1819c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:05.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1659c700 1 --2- 172.21.15.46:0/2616345358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18108780 0x7fbc1819c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1659c700 1 -- 172.21.15.46:0/2616345358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc18103550 msgr2=0x7fbc1819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1659c700 1 --2- 172.21.15.46:0/2616345358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc18103550 0x7fbc1819bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1659c700 1 -- 172.21.15.46:0/2616345358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc181a6fe0 con 0x7fbc18107e30 2024-05-09T04:54:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.882+0000 7fbc1759e700 1 --2- 172.21.15.46:0/2616345358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18108780 0x7fbc1819c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.883+0000 7fbc1659c700 1 --2- 172.21.15.46:0/2616345358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18107e30 0x7fbc1819c1d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00d980 tx=0x7fbc0c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.883+0000 7fbbfffff700 1 -- 172.21.15.46:0/2616345358 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c017070 con 0x7fbc18107e30 2024-05-09T04:54:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.883+0000 7fbc1d150700 1 -- 172.21.15.46:0/2616345358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc181a72d0 con 0x7fbc18107e30 2024-05-09T04:54:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.883+0000 7fbbfffff700 1 -- 172.21.15.46:0/2616345358 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc0c009de0 con 0x7fbc18107e30 2024-05-09T04:54:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.883+0000 7fbbfffff700 1 -- 172.21.15.46:0/2616345358 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c00f3c0 con 0x7fbc18107e30 2024-05-09T04:54:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.883+0000 7fbc1d150700 1 -- 172.21.15.46:0/2616345358 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc181a7860 con 0x7fbc18107e30 2024-05-09T04:54:05.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.886+0000 7fbbfffff700 1 -- 172.21.15.46:0/2616345358 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbc0c00f560 con 0x7fbc18107e30 2024-05-09T04:54:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.886+0000 7fbbfffff700 1 --2- 172.21.15.46:0/2616345358 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbc000b29f0 0x7fbc000b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.886+0000 7fbc16d9d700 1 --2- 172.21.15.46:0/2616345358 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbc000b29f0 0x7fbc000b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.887+0000 7fbbfffff700 1 -- 172.21.15.46:0/2616345358 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fbc0c0e2250 con 0x7fbc18107e30 2024-05-09T04:54:05.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.887+0000 7fbc2bb66b80 1 Processor -- start 2024-05-09T04:54:05.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.887+0000 7fbc16d9d700 1 --2- 172.21.15.46:0/2616345358 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbc000b29f0 0x7fbc000b4f00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbc08000c00 tx=0x7fbc0801a040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:05.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.887+0000 7fbc2bb66b80 1 -- start start 2024-05-09T04:54:05.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.887+0000 7fbc2bb66b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269130d80 0x55c2692392c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.887+0000 7fbc2bb66b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c2690b6520 0x55c269239800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.887+0000 7fbc2bb66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c269233480 0x55c269233890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.887+0000 7fbc2bb66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c269156120 con 0x55c269233480 2024-05-09T04:54:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.887+0000 7fbc2bb66b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c2690fba20 con 0x55c269130d80 2024-05-09T04:54:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.887+0000 7fbc2bb66b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c2691520e0 con 0x55c2690b6520 2024-05-09T04:54:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.887+0000 7fbc1759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c269233480 0x55c269233890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.888+0000 7fbc1759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c269233480 0x55c269233890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57086/0 (socket says 172.21.15.46:57086) 2024-05-09T04:54:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.888+0000 7fbc1759e700 1 -- 172.21.15.46:0/2008847648 learned_addr learned my addr 172.21.15.46:0/2008847648 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.888+0000 7fbc16d9d700 1 --2- 172.21.15.46:0/2008847648 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269130d80 0x55c2692392c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.888+0000 7fbc1659c700 1 --2- 172.21.15.46:0/2008847648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c2690b6520 0x55c269239800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.888+0000 7fbc1759e700 1 -- 172.21.15.46:0/2008847648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c2690b6520 msgr2=0x55c269239800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.888+0000 7fbc1759e700 1 --2- 172.21.15.46:0/2008847648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c2690b6520 0x55c269239800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.888+0000 7fbc1759e700 1 -- 172.21.15.46:0/2008847648 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269130d80 msgr2=0x55c2692392c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.888+0000 7fbc1759e700 1 --2- 172.21.15.46:0/2008847648 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269130d80 0x55c2692392c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.888+0000 7fbc1759e700 1 -- 172.21.15.46:0/2008847648 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2690ec580 con 0x55c269233480 2024-05-09T04:54:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.888+0000 7fbc1759e700 1 --2- 172.21.15.46:0/2008847648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c269233480 0x55c269233890 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbc100093b0 tx=0x7fbc10009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6bf1bc8de794fde server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.888+0000 7fbbfd7fa700 1 -- 172.21.15.46:0/2008847648 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc10019030 con 0x55c269233480 2024-05-09T04:54:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.888+0000 7fbbfd7fa700 1 -- 172.21.15.46:0/2008847648 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc100158f0 con 0x55c269233480 2024-05-09T04:54:05.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.888+0000 7fbbfd7fa700 1 -- 172.21.15.46:0/2008847648 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc100147c0 con 0x55c269233480 2024-05-09T04:54:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.889+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/2008847648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c269233480 msgr2=0x55c269233890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.889+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/2008847648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c269233480 0x55c269233890 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fbc100093b0 tx=0x7fbc10009920 comp rx=0 tx=0).stop 2024-05-09T04:54:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.889+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/2008847648 shutdown_connections 2024-05-09T04:54:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.889+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/2008847648 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c269233480 0x55c269233890 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.889+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/2008847648 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269130d80 0x55c2692392c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.889+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/2008847648 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c2690b6520 0x55c269239800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.889+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/2008847648 >> 172.21.15.46:0/2008847648 conn(0x55c2690b9ff0 msgr2=0x55c26922e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:05.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.889+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/2008847648 shutdown_connections 2024-05-09T04:54:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.889+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/2008847648 wait complete. 2024-05-09T04:54:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.890+0000 7fbc2bb66b80 1 -- start start 2024-05-09T04:54:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.890+0000 7fbc2bb66b80 1 Processor -- start 2024-05-09T04:54:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.890+0000 7fbc2bb66b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2690b6520 0x55c26923d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.890+0000 7fbc2bb66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c269130d80 0x55c26923d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.890+0000 7fbc2bb66b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c269233480 0x55c26923db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.890+0000 7fbc2bb66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c2692341e0 con 0x55c269130d80 2024-05-09T04:54:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.890+0000 7fbc2bb66b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c2690f0ba0 con 0x55c2690b6520 2024-05-09T04:54:05.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.890+0000 7fbc2bb66b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c2692400a0 con 0x55c269233480 2024-05-09T04:54:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.890+0000 7fbc2bb66b80 10 client.0 _ll_register_callbacks cb 0x55c268fc7740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.890+0000 7fbc1659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c269130d80 0x55c26923d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.890+0000 7fbc1659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c269130d80 0x55c26923d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57100/0 (socket says 172.21.15.46:57100) 2024-05-09T04:54:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.890+0000 7fbc1659c700 1 -- 172.21.15.46:0/2187292512 learned_addr learned my addr 172.21.15.46:0/2187292512 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.890+0000 7fbc16d9d700 1 --2- 172.21.15.46:0/2187292512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2690b6520 0x55c26923d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.890+0000 7fbc1759e700 1 --2- 172.21.15.46:0/2187292512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c269233480 0x55c26923db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.891+0000 7fbc1659c700 1 -- 172.21.15.46:0/2187292512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c269233480 msgr2=0x55c26923db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.891+0000 7fbc1659c700 1 --2- 172.21.15.46:0/2187292512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c269233480 0x55c26923db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.891+0000 7fbc1659c700 1 -- 172.21.15.46:0/2187292512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2690b6520 msgr2=0x55c26923d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.891+0000 7fbc1659c700 1 --2- 172.21.15.46:0/2187292512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2690b6520 0x55c26923d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.891+0000 7fbc1659c700 1 -- 172.21.15.46:0/2187292512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c268fd1510 con 0x55c269130d80 2024-05-09T04:54:05.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.891+0000 7fbc16d9d700 1 --2- 172.21.15.46:0/2187292512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2690b6520 0x55c26923d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.891+0000 7fbc1659c700 1 --2- 172.21.15.46:0/2187292512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c269130d80 0x55c26923d640 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0df980 tx=0x7fbc0c0108d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bcfa7394c9b2e7d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.891+0000 7fbbfaff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.891+0000 7fbbfaff5700 1 -- 172.21.15.46:0/2187292512 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0c020040 con 0x55c269130d80 2024-05-09T04:54:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.891+0000 7fbbfaff5700 1 -- 172.21.15.46:0/2187292512 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fbc0c0b0df0 con 0x55c269130d80 2024-05-09T04:54:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.891+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/2187292512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc100158f0 con 0x55c269130d80 2024-05-09T04:54:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.891+0000 7fbbfaff5700 1 -- 172.21.15.46:0/2187292512 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc0c0b2310 con 0x55c269130d80 2024-05-09T04:54:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.891+0000 7fbc2bb66b80 10 client.16122 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.891+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/2187292512 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c26924ef40 con 0x55c269130d80 2024-05-09T04:54:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.891+0000 7fbbfaff5700 1 -- 172.21.15.46:0/2187292512 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc100158f0 con 0x55c269130d80 2024-05-09T04:54:05.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.892+0000 7fbbfaff5700 1 -- 172.21.15.46:0/2187292512 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fbc0c0b1e80 con 0x55c269130d80 2024-05-09T04:54:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.892+0000 7fbc2bb66b80 10 client.16122 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.892+0000 7fbc2bb66b80 10 client.16122 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.892+0000 7fbc2bb66b80 20 client.16122 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.892+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/2187292512 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c269251580 0x55c2692539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.892+0000 7fbc2bb66b80 4 client.16122 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.892+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/2187292512 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c269163b00 con 0x55c269251580 2024-05-09T04:54:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.892+0000 7fbc16d9d700 1 --2- 172.21.15.46:0/2187292512 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c269251580 0x55c2692539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.892+0000 7fbc16d9d700 1 --2- 172.21.15.46:0/2187292512 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c269251580 0x55c2692539a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fda2e9ddaedbc97 server_cookie=bde228479dff4cc7 in_seq=0 out_seq=0 2024-05-09T04:54:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.892+0000 7fbbfaff5700 10 client.16122 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.893+0000 7fbbfaff5700 1 -- 172.21.15.46:0/2187292512 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c269163b00 con 0x55c269251580 2024-05-09T04:54:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.893+0000 7fbbfaff5700 10 client.16122 handle_command_reply: tid=0 2024-05-09T04:54:05.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.893+0000 7fbc2bb66b80 1 client.16122 shutdown 2024-05-09T04:54:05.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.893+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/2187292512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c269130d80 msgr2=0x55c26923d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:05.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.893+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/2187292512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c269130d80 0x55c26923d640 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c0df980 tx=0x7fbc0c0108d0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.893+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/2187292512 shutdown_connections 2024-05-09T04:54:05.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.893+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/2187292512 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c269233480 0x55c26923db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.893+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/2187292512 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c269251580 0x55c2692539a0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.894+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/2187292512 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c2690b6520 0x55c26923d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.894+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/2187292512 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c269130d80 0x55c26923d640 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.894+0000 7fbbfaff5700 0 client.16122 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.894+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/2187292512 >> 172.21.15.46:0/2187292512 conn(0x55c2690b9ff0 msgr2=0x55c269228070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.894+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/2187292512 shutdown_connections 2024-05-09T04:54:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.894+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/2187292512 wait complete. 2024-05-09T04:54:05.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.894+0000 7fbc2bb66b80 20 client.16122 trim_cache size 0 max 16384 2024-05-09T04:54:05.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.914+0000 7fbc2bb66b80 1 Processor -- start 2024-05-09T04:54:05.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.914+0000 7fbc2bb66b80 1 -- start start 2024-05-09T04:54:05.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc2bb66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2690b6520 0x55c269228aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc2bb66b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c269130d80 0x55c269232b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc2bb66b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269233480 0x55c269259090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc2bb66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbc181af710 con 0x55c2690b6520 2024-05-09T04:54:05.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc2bb66b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbc181acfa0 con 0x55c269233480 2024-05-09T04:54:05.917 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc2bb66b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbc181aadf0 con 0x55c269130d80 2024-05-09T04:54:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc16d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2690b6520 0x55c269228aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc16d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2690b6520 0x55c269228aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57106/0 (socket says 172.21.15.46:57106) 2024-05-09T04:54:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc16d9d700 1 -- 172.21.15.46:0/4145639339 learned_addr learned my addr 172.21.15.46:0/4145639339 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc1759e700 1 --2- 172.21.15.46:0/4145639339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269233480 0x55c269259090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc1659c700 1 --2- 172.21.15.46:0/4145639339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c269130d80 0x55c269232b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc16d9d700 1 -- 172.21.15.46:0/4145639339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c269130d80 msgr2=0x55c269232b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc16d9d700 1 --2- 172.21.15.46:0/4145639339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c269130d80 0x55c269232b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc16d9d700 1 -- 172.21.15.46:0/4145639339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269233480 msgr2=0x55c269259090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc16d9d700 1 --2- 172.21.15.46:0/4145639339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269233480 0x55c269259090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc16d9d700 1 -- 172.21.15.46:0/4145639339 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c26923f9a0 con 0x55c2690b6520 2024-05-09T04:54:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc1759e700 1 --2- 172.21.15.46:0/4145639339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269233480 0x55c269259090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:54:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbc16d9d700 1 --2- 172.21.15.46:0/4145639339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2690b6520 0x55c269228aa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbc08005f50 tx=0x7fbc08009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae58941bb96d052 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.915+0000 7fbbfcff9700 1 -- 172.21.15.46:0/4145639339 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc0800bad0 con 0x55c2690b6520 2024-05-09T04:54:05.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.916+0000 7fbbfcff9700 1 -- 172.21.15.46:0/4145639339 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc08038070 con 0x55c2690b6520 2024-05-09T04:54:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.916+0000 7fbbfcff9700 1 -- 172.21.15.46:0/4145639339 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc080316c0 con 0x55c2690b6520 2024-05-09T04:54:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.916+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/4145639339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2690b6520 msgr2=0x55c269228aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.916+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/4145639339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2690b6520 0x55c269228aa0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fbc08005f50 tx=0x7fbc08009cd0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.916+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/4145639339 shutdown_connections 2024-05-09T04:54:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.916+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/4145639339 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269233480 0x55c269259090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.916+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/4145639339 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2690b6520 0x55c269228aa0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.916+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/4145639339 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c269130d80 0x55c269232b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.916+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/4145639339 >> 172.21.15.46:0/4145639339 conn(0x55c2690b9ff0 msgr2=0x55c269232470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.916+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/4145639339 shutdown_connections 2024-05-09T04:54:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.916+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/4145639339 wait complete. 2024-05-09T04:54:05.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.917+0000 7fbc2bb66b80 1 -- start start 2024-05-09T04:54:05.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.917+0000 7fbc2bb66b80 1 Processor -- start 2024-05-09T04:54:05.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.917+0000 7fbc2bb66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2690b6520 0x55c2692424a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.917+0000 7fbc2bb66b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269130d80 0x55c2692429e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.917+0000 7fbc2bb66b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c269233480 0x55c2692bbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.917+0000 7fbc2bb66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbc181ab460 con 0x55c2690b6520 2024-05-09T04:54:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.917+0000 7fbc2bb66b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbc181ac8d0 con 0x55c269130d80 2024-05-09T04:54:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.917+0000 7fbc2bb66b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbc181abb30 con 0x55c269233480 2024-05-09T04:54:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.917+0000 7fbc2bb66b80 10 client.0 _ll_register_callbacks cb 0x55c268fc7740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.917+0000 7fbc16d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2690b6520 0x55c2692424a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.917+0000 7fbc16d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2690b6520 0x55c2692424a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57118/0 (socket says 172.21.15.46:57118) 2024-05-09T04:54:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.917+0000 7fbc16d9d700 1 -- 172.21.15.46:0/3016477939 learned_addr learned my addr 172.21.15.46:0/3016477939 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.918+0000 7fbc1659c700 1 --2- 172.21.15.46:0/3016477939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269130d80 0x55c2692429e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.918+0000 7fbc1759e700 1 --2- 172.21.15.46:0/3016477939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c269233480 0x55c2692bbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.918+0000 7fbc16d9d700 1 -- 172.21.15.46:0/3016477939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c269233480 msgr2=0x55c2692bbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.918+0000 7fbc16d9d700 1 --2- 172.21.15.46:0/3016477939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c269233480 0x55c2692bbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.918+0000 7fbc16d9d700 1 -- 172.21.15.46:0/3016477939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269130d80 msgr2=0x55c2692429e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.918+0000 7fbc16d9d700 1 --2- 172.21.15.46:0/3016477939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269130d80 0x55c2692429e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.918+0000 7fbc16d9d700 1 -- 172.21.15.46:0/3016477939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2692bc190 con 0x55c2690b6520 2024-05-09T04:54:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.918+0000 7fbc16d9d700 1 --2- 172.21.15.46:0/3016477939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2690b6520 0x55c2692424a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800a010 tx=0x7fbc080325e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33505f73761469d2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.918+0000 7fbbfaff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.918+0000 7fbbfaff5700 1 -- 172.21.15.46:0/3016477939 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbc08040070 con 0x55c2690b6520 2024-05-09T04:54:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.918+0000 7fbbfaff5700 1 -- 172.21.15.46:0/3016477939 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fbc0800bce0 con 0x55c2690b6520 2024-05-09T04:54:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.918+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/3016477939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbc08038070 con 0x55c2690b6520 2024-05-09T04:54:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.918+0000 7fbbfaff5700 1 -- 172.21.15.46:0/3016477939 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbc08004040 con 0x55c2690b6520 2024-05-09T04:54:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.918+0000 7fbc2bb66b80 10 client.16128 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.918+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/3016477939 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c2692bd320 con 0x55c2690b6520 2024-05-09T04:54:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.919+0000 7fbbfaff5700 1 -- 172.21.15.46:0/3016477939 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc08038070 con 0x55c2690b6520 2024-05-09T04:54:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.919+0000 7fbbfaff5700 1 -- 172.21.15.46:0/3016477939 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fbc08021070 con 0x55c2690b6520 2024-05-09T04:54:05.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.919+0000 7fbc2bb66b80 10 client.16128 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.919+0000 7fbc2bb66b80 10 client.16128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.919+0000 7fbc2bb66b80 20 client.16128 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.919+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/3016477939 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c269251580 0x55c2692bf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.919+0000 7fbc2bb66b80 4 client.16128 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:54:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.919+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/3016477939 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c26915dba0 con 0x55c269251580 2024-05-09T04:54:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.919+0000 7fbc1659c700 1 --2- 172.21.15.46:0/3016477939 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c269251580 0x55c2692bf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.919+0000 7fbc1659c700 1 --2- 172.21.15.46:0/3016477939 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c269251580 0x55c2692bf900 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9b81164601f46d6 server_cookie=5955065054d3393 in_seq=0 out_seq=0 2024-05-09T04:54:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.919+0000 7fbbfaff5700 10 client.16128 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.920+0000 7fbbfaff5700 1 -- 172.21.15.46:0/3016477939 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c26915dba0 con 0x55c269251580 2024-05-09T04:54:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.920+0000 7fbbfaff5700 10 client.16128 handle_command_reply: tid=0 2024-05-09T04:54:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.920+0000 7fbc2bb66b80 1 client.16128 shutdown 2024-05-09T04:54:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.921+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/3016477939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2690b6520 msgr2=0x55c2692424a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:05.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.921+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/3016477939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2690b6520 0x55c2692424a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fbc0800a010 tx=0x7fbc080325e0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.921+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/3016477939 shutdown_connections 2024-05-09T04:54:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.921+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/3016477939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c269233480 0x55c2692bbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.921+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/3016477939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c2690b6520 0x55c2692424a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.921+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/3016477939 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c269251580 0x55c2692bf900 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.921+0000 7fbc2bb66b80 1 --2- 172.21.15.46:0/3016477939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c269130d80 0x55c2692429e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.921+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/3016477939 >> 172.21.15.46:0/3016477939 conn(0x55c2690b9ff0 msgr2=0x55c269228800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.921+0000 7fbbfaff5700 0 client.16128 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.921+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/3016477939 shutdown_connections 2024-05-09T04:54:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.921+0000 7fbc2bb66b80 1 -- 172.21.15.46:0/3016477939 wait complete. 2024-05-09T04:54:05.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.921+0000 7fbc2bb66b80 20 client.16128 trim_cache size 0 max 16384 2024-05-09T04:54:05.926 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:54:05.927 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:54:05.927 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "3a1e3a0e-c490-4895-a00f-f8c05852edbb", 2024-05-09T04:54:05.927 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:54:05.927 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:54:05.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.922+0000 7fbc1d150700 1 -- 172.21.15.46:0/2616345358 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbc000b29f0 msgr2=0x7fbc000b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:05.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.922+0000 7fbc1d150700 1 --2- 172.21.15.46:0/2616345358 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbc000b29f0 0x7fbc000b4f00 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbc08000c00 tx=0x7fbc0801a040 comp rx=0 tx=0).stop 2024-05-09T04:54:05.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.922+0000 7fbc1d150700 1 -- 172.21.15.46:0/2616345358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18107e30 msgr2=0x7fbc1819c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:05.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.922+0000 7fbc1d150700 1 --2- 172.21.15.46:0/2616345358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18107e30 0x7fbc1819c1d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbc0c00d980 tx=0x7fbc0c00dc90 comp rx=0 tx=0).stop 2024-05-09T04:54:05.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.922+0000 7fbc1d150700 1 -- 172.21.15.46:0/2616345358 shutdown_connections 2024-05-09T04:54:05.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.922+0000 7fbc1d150700 1 --2- 172.21.15.46:0/2616345358 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbc000b29f0 0x7fbc000b4f00 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.922+0000 7fbc1d150700 1 --2- 172.21.15.46:0/2616345358 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbc18108780 0x7fbc1819c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.922+0000 7fbc1d150700 1 --2- 172.21.15.46:0/2616345358 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbc18103550 0x7fbc1819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.922+0000 7fbc1d150700 1 --2- 172.21.15.46:0/2616345358 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbc18107e30 0x7fbc1819c1d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:05.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.922+0000 7fbc1d150700 1 -- 172.21.15.46:0/2616345358 >> 172.21.15.46:0/2616345358 conn(0x7fbc18078e00 msgr2=0x7fbc18100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:05.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.922+0000 7fbc1d150700 1 -- 172.21.15.46:0/2616345358 shutdown_connections 2024-05-09T04:54:05.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:05.922+0000 7fbc1d150700 1 -- 172.21.15.46:0/2616345358 wait complete. 2024-05-09T04:54:06.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:05 smithi110 ceph-mon[34904]: pgmap v93: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 280 MiB/s wr, 92 op/s 2024-05-09T04:54:06.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:05 smithi072 ceph-mon[34088]: pgmap v93: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 280 MiB/s wr, 92 op/s 2024-05-09T04:54:06.668 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:54:06.922 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:07.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:06 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:54:07.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:06 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:07.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:06 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:54:07.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:06 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:07.046 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:06 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:54:07.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:06 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:54:07.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:06 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:07.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:06 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:54:07.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:06 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:07.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:06 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:54:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:06 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:54:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:06 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:06 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:54:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:06 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:06 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:54:08.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:07 smithi046 ceph-mon[30136]: pgmap v94: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 280 MiB/s wr, 96 op/s 2024-05-09T04:54:08.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:07 smithi072 ceph-mon[34088]: pgmap v94: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 280 MiB/s wr, 96 op/s 2024-05-09T04:54:08.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:07 smithi110 ceph-mon[34904]: pgmap v94: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 280 MiB/s wr, 96 op/s 2024-05-09T04:54:09.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.045+0000 7f1c7ef32700 1 -- 172.21.15.46:0/1112655067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c78107df0 msgr2=0x7f1c78108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.045+0000 7f1c7ef32700 1 --2- 172.21.15.46:0/1112655067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c78107df0 0x7f1c78108200 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1c640099c0 tx=0x7f1c64009cd0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.045+0000 7f1c7ef32700 1 -- 172.21.15.46:0/1112655067 shutdown_connections 2024-05-09T04:54:09.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.045+0000 7f1c7ef32700 1 --2- 172.21.15.46:0/1112655067 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c78108740 0x7f1c781100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.045+0000 7f1c7ef32700 1 --2- 172.21.15.46:0/1112655067 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c781034c0 0x7f1c781038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.045+0000 7f1c7ef32700 1 --2- 172.21.15.46:0/1112655067 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c78107df0 0x7f1c78108200 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.045+0000 7f1c7ef32700 1 -- 172.21.15.46:0/1112655067 >> 172.21.15.46:0/1112655067 conn(0x7f1c780fedf0 msgr2=0x7f1c78101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:09.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.045+0000 7f1c7ef32700 1 -- 172.21.15.46:0/1112655067 shutdown_connections 2024-05-09T04:54:09.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.045+0000 7f1c7ef32700 1 -- 172.21.15.46:0/1112655067 wait complete. 2024-05-09T04:54:09.046 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.046+0000 7f1c7ef32700 1 Processor -- start 2024-05-09T04:54:09.047 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.046+0000 7f1c7ef32700 1 -- start start 2024-05-09T04:54:09.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.046+0000 7f1c7ef32700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c781034c0 0x7f1c7819d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.046+0000 7f1c7ef32700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c78107df0 0x7f1c7819d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.046+0000 7f1c7ef32700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c78108740 0x7f1c781a1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.046+0000 7f1c7ef32700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1c78113350 con 0x7f1c78108740 2024-05-09T04:54:09.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.046+0000 7f1c7ef32700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781131d0 con 0x7f1c781034c0 2024-05-09T04:54:09.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.046+0000 7f1c7ef32700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781134d0 con 0x7f1c78107df0 2024-05-09T04:54:09.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.046+0000 7f1c7d4cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c78108740 0x7f1c781a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.047+0000 7f1c7d4cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c78108740 0x7f1c781a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57132/0 (socket says 172.21.15.46:57132) 2024-05-09T04:54:09.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.047+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/2369177035 learned_addr learned my addr 172.21.15.46:0/2369177035 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:09.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.047+0000 7f1c77fff700 1 --2- 172.21.15.46:0/2369177035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c78107df0 0x7f1c7819d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.047+0000 7f1c7ccce700 1 --2- 172.21.15.46:0/2369177035 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c781034c0 0x7f1c7819d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.047+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/2369177035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c78107df0 msgr2=0x7f1c7819d5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.047+0000 7f1c7d4cf700 1 --2- 172.21.15.46:0/2369177035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c78107df0 0x7f1c7819d5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.047+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/2369177035 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c781034c0 msgr2=0x7f1c7819d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.047+0000 7f1c7d4cf700 1 --2- 172.21.15.46:0/2369177035 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c781034c0 0x7f1c7819d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.047+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/2369177035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c781a1ed0 con 0x7f1c78108740 2024-05-09T04:54:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.047+0000 7f1c7d4cf700 1 --2- 172.21.15.46:0/2369177035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c78108740 0x7f1c781a1830 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1c70007ee0 tx=0x7f1c7000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.047+0000 7f1c75ffb700 1 -- 172.21.15.46:0/2369177035 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70004bb0 con 0x7f1c78108740 2024-05-09T04:54:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.047+0000 7f1c75ffb700 1 -- 172.21.15.46:0/2369177035 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c70004d50 con 0x7f1c78108740 2024-05-09T04:54:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.047+0000 7f1c75ffb700 1 -- 172.21.15.46:0/2369177035 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c70005920 con 0x7f1c78108740 2024-05-09T04:54:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.047+0000 7f1c7ef32700 1 -- 172.21.15.46:0/2369177035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c781a21c0 con 0x7f1c78108740 2024-05-09T04:54:09.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.048+0000 7f1c7ef32700 1 -- 172.21.15.46:0/2369177035 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c781a9ae0 con 0x7f1c78108740 2024-05-09T04:54:09.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.050+0000 7f1c75ffb700 1 -- 172.21.15.46:0/2369177035 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1c7001a020 con 0x7f1c78108740 2024-05-09T04:54:09.051 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.051+0000 7f1c75ffb700 1 --2- 172.21.15.46:0/2369177035 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f1c680b2a70 0x7f1c680b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.051+0000 7f1c7ccce700 1 --2- 172.21.15.46:0/2369177035 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f1c680b2a70 0x7f1c680b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.051+0000 7f1c75ffb700 1 -- 172.21.15.46:0/2369177035 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f1c700e1c50 con 0x7f1c78108740 2024-05-09T04:54:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.051+0000 7f1c8d948b80 1 Processor -- start 2024-05-09T04:54:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c8d948b80 1 -- start start 2024-05-09T04:54:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c7ccce700 1 --2- 172.21.15.46:0/2369177035 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f1c680b2a70 0x7f1c680b4f80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c005950 tx=0x7f1c6c00a300 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c8d948b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb13e160 0x555abb1d74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c8d948b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb0544a0 0x555abb1d7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c8d948b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1d1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c8d948b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555abb0f3ff0 con 0x555abb1d16a0 2024-05-09T04:54:09.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c8d948b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x555abb0927f0 con 0x555abb13e160 2024-05-09T04:54:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c8d948b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555abb0effb0 con 0x555abb0544a0 2024-05-09T04:54:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c7d4cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1d1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c7d4cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1d1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57144/0 (socket says 172.21.15.46:57144) 2024-05-09T04:54:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/3188378636 learned_addr learned my addr 172.21.15.46:0/3188378636 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c77fff700 1 --2- 172.21.15.46:0/3188378636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb0544a0 0x555abb1d7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c7ccce700 1 --2- 172.21.15.46:0/3188378636 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb13e160 0x555abb1d74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/3188378636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb0544a0 msgr2=0x555abb1d7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c7d4cf700 1 --2- 172.21.15.46:0/3188378636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb0544a0 0x555abb1d7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/3188378636 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb13e160 msgr2=0x555abb1d74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c7d4cf700 1 --2- 172.21.15.46:0/3188378636 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb13e160 0x555abb1d74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.052+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/3188378636 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555abb08a370 con 0x555abb1d16a0 2024-05-09T04:54:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.053+0000 7f1c7d4cf700 1 --2- 172.21.15.46:0/3188378636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1d1ad0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1c700a8e40 tx=0x7f1c700a8930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44bb2bfba5d662a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.053+0000 7f1c62ffd700 1 -- 172.21.15.46:0/3188378636 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c700a89c0 con 0x555abb1d16a0 2024-05-09T04:54:09.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.053+0000 7f1c62ffd700 1 -- 172.21.15.46:0/3188378636 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c700a95b0 con 0x555abb1d16a0 2024-05-09T04:54:09.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.053+0000 7f1c62ffd700 1 -- 172.21.15.46:0/3188378636 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c700a9110 con 0x555abb1d16a0 2024-05-09T04:54:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.054+0000 7f1c8d948b80 1 -- 172.21.15.46:0/3188378636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 msgr2=0x555abb1d1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.054+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/3188378636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1d1ad0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f1c700a8e40 tx=0x7f1c700a8930 comp rx=0 tx=0).stop 2024-05-09T04:54:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.054+0000 7f1c8d948b80 1 -- 172.21.15.46:0/3188378636 shutdown_connections 2024-05-09T04:54:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.054+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/3188378636 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1d1ad0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.054+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/3188378636 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb13e160 0x555abb1d74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.054+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/3188378636 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb0544a0 0x555abb1d7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.054+0000 7f1c8d948b80 1 -- 172.21.15.46:0/3188378636 >> 172.21.15.46:0/3188378636 conn(0x555abb0b46a0 msgr2=0x555abb1cc400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.054+0000 7f1c8d948b80 1 -- 172.21.15.46:0/3188378636 shutdown_connections 2024-05-09T04:54:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.054+0000 7f1c8d948b80 1 -- 172.21.15.46:0/3188378636 wait complete. 2024-05-09T04:54:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.054+0000 7f1c8d948b80 1 -- start start 2024-05-09T04:54:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.054+0000 7f1c8d948b80 1 Processor -- start 2024-05-09T04:54:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c8d948b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb0544a0 0x555abb1db1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c8d948b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb13e160 0x555abb1db6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c8d948b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb1d16a0 0x555abb1dbc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c8d948b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555abaf6fba0 con 0x555abb13e160 2024-05-09T04:54:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c8d948b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x555abb0ae150 con 0x555abb1d16a0 2024-05-09T04:54:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c8d948b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555abb1de1d0 con 0x555abb0544a0 2024-05-09T04:54:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c8d948b80 10 client.0 _ll_register_callbacks cb 0x555abaf656c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c77fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb13e160 0x555abb1db6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c77fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb13e160 0x555abb1db6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57156/0 (socket says 172.21.15.46:57156) 2024-05-09T04:54:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c77fff700 1 -- 172.21.15.46:0/995492611 learned_addr learned my addr 172.21.15.46:0/995492611 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c7ccce700 1 --2- 172.21.15.46:0/995492611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb0544a0 0x555abb1db1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c7d4cf700 1 --2- 172.21.15.46:0/995492611 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb1d16a0 0x555abb1dbc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c77fff700 1 -- 172.21.15.46:0/995492611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb0544a0 msgr2=0x555abb1db1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c77fff700 1 --2- 172.21.15.46:0/995492611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb0544a0 0x555abb1db1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c77fff700 1 -- 172.21.15.46:0/995492611 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb1d16a0 msgr2=0x555abb1dbc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c77fff700 1 --2- 172.21.15.46:0/995492611 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb1d16a0 0x555abb1dbc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.060 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.055+0000 7f1c77fff700 1 -- 172.21.15.46:0/995492611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555abaf6f490 con 0x555abb13e160 2024-05-09T04:54:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.056+0000 7f1c77fff700 1 --2- 172.21.15.46:0/995492611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb13e160 0x555abb1db6f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1c78076b40 tx=0x7f1c64003e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c613594f286216d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.056+0000 7f1c57fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.056+0000 7f1c57fff700 1 -- 172.21.15.46:0/995492611 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c640054c0 con 0x555abb13e160 2024-05-09T04:54:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.056+0000 7f1c57fff700 1 -- 172.21.15.46:0/995492611 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f1c64005660 con 0x555abb13e160 2024-05-09T04:54:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.056+0000 7f1c8d948b80 1 -- 172.21.15.46:0/995492611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c700a95b0 con 0x555abb13e160 2024-05-09T04:54:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.056+0000 7f1c57fff700 1 -- 172.21.15.46:0/995492611 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c64031050 con 0x555abb13e160 2024-05-09T04:54:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.056+0000 7f1c8d948b80 10 client.16140 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.056+0000 7f1c8d948b80 1 -- 172.21.15.46:0/995492611 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555abb1e8be0 con 0x555abb13e160 2024-05-09T04:54:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.056+0000 7f1c57fff700 1 -- 172.21.15.46:0/995492611 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c700a95b0 con 0x555abb13e160 2024-05-09T04:54:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.056+0000 7f1c57fff700 1 -- 172.21.15.46:0/995492611 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f1c640238f0 con 0x555abb13e160 2024-05-09T04:54:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.056+0000 7f1c8d948b80 10 client.16140 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.056+0000 7f1c8d948b80 10 client.16140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.057+0000 7f1c8d948b80 20 client.16140 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.057+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/995492611 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x555abb1eb270 0x555abb1ed6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.061 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.057+0000 7f1c8d948b80 4 client.16140 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.057+0000 7f1c8d948b80 1 -- 172.21.15.46:0/995492611 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555abb1019d0 con 0x555abb1eb270 2024-05-09T04:54:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.057+0000 7f1c7ccce700 1 --2- 172.21.15.46:0/995492611 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x555abb1eb270 0x555abb1ed6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.057+0000 7f1c7ccce700 1 --2- 172.21.15.46:0/995492611 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x555abb1eb270 0x555abb1ed6c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71a66bea8355ee1b server_cookie=9a5bd7539dbea429 in_seq=0 out_seq=0 2024-05-09T04:54:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.057+0000 7f1c57fff700 10 client.16140 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.058+0000 7f1c57fff700 1 -- 172.21.15.46:0/995492611 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555abb1019d0 con 0x555abb1eb270 2024-05-09T04:54:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.058+0000 7f1c57fff700 10 client.16140 handle_command_reply: tid=0 2024-05-09T04:54:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.058+0000 7f1c8d948b80 1 client.16140 shutdown 2024-05-09T04:54:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.058+0000 7f1c8d948b80 1 -- 172.21.15.46:0/995492611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb13e160 msgr2=0x555abb1db6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.058+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/995492611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb13e160 0x555abb1db6f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f1c78076b40 tx=0x7f1c64003e10 comp rx=0 tx=0).stop 2024-05-09T04:54:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.058+0000 7f1c8d948b80 1 -- 172.21.15.46:0/995492611 shutdown_connections 2024-05-09T04:54:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.058+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/995492611 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x555abb1eb270 0x555abb1ed6c0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.058+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/995492611 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb1d16a0 0x555abb1dbc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.058+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/995492611 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb0544a0 0x555abb1db1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.058+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/995492611 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb13e160 0x555abb1db6f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.058+0000 7f1c8d948b80 1 -- 172.21.15.46:0/995492611 >> 172.21.15.46:0/995492611 conn(0x555abb0b46a0 msgr2=0x555abb1cc1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.058+0000 7f1c57fff700 0 client.16140 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.058+0000 7f1c8d948b80 1 -- 172.21.15.46:0/995492611 shutdown_connections 2024-05-09T04:54:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.058+0000 7f1c8d948b80 1 -- 172.21.15.46:0/995492611 wait complete. 2024-05-09T04:54:09.063 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.059+0000 7f1c8d948b80 20 client.16140 trim_cache size 0 max 16384 2024-05-09T04:54:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.079+0000 7f1c8d948b80 1 Processor -- start 2024-05-09T04:54:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.079+0000 7f1c8d948b80 1 -- start start 2024-05-09T04:54:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.079+0000 7f1c8d948b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb0544a0 0x555abb1da170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.079+0000 7f1c8d948b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb13e160 0x555abb1d5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.079+0000 7f1c8d948b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.079+0000 7f1c8d948b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781b3820 con 0x555abb1d16a0 2024-05-09T04:54:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.079+0000 7f1c8d948b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781b10b0 con 0x555abb0544a0 2024-05-09T04:54:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.079+0000 7f1c8d948b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781aef00 con 0x555abb13e160 2024-05-09T04:54:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.079+0000 7f1c7d4cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.079+0000 7f1c7d4cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1e6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57160/0 (socket says 172.21.15.46:57160) 2024-05-09T04:54:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.079+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/346094363 learned_addr learned my addr 172.21.15.46:0/346094363 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.079+0000 7f1c77fff700 1 --2- 172.21.15.46:0/346094363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb13e160 0x555abb1d5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.080+0000 7f1c7ccce700 1 --2- 172.21.15.46:0/346094363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb0544a0 0x555abb1da170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.080+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/346094363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb13e160 msgr2=0x555abb1d5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.080+0000 7f1c7d4cf700 1 --2- 172.21.15.46:0/346094363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb13e160 0x555abb1d5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.080+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/346094363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb0544a0 msgr2=0x555abb1da170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.080+0000 7f1c7d4cf700 1 --2- 172.21.15.46:0/346094363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb0544a0 0x555abb1da170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.080+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/346094363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555abb1c5f60 con 0x555abb1d16a0 2024-05-09T04:54:09.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.080+0000 7f1c7ccce700 1 --2- 172.21.15.46:0/346094363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb0544a0 0x555abb1da170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.080+0000 7f1c7d4cf700 1 --2- 172.21.15.46:0/346094363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1e6fd0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1c700a9760 tx=0x7f1c700053c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb58ca811775193 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.080+0000 7f1c627fc700 1 -- 172.21.15.46:0/346094363 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c700a89c0 con 0x555abb1d16a0 2024-05-09T04:54:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.080+0000 7f1c627fc700 1 -- 172.21.15.46:0/346094363 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c700a95b0 con 0x555abb1d16a0 2024-05-09T04:54:09.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.080+0000 7f1c627fc700 1 -- 172.21.15.46:0/346094363 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c700a9110 con 0x555abb1d16a0 2024-05-09T04:54:09.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.081+0000 7f1c8d948b80 1 -- 172.21.15.46:0/346094363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 msgr2=0x555abb1e6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.081+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/346094363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1e6fd0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f1c700a9760 tx=0x7f1c700053c0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.081+0000 7f1c8d948b80 1 -- 172.21.15.46:0/346094363 shutdown_connections 2024-05-09T04:54:09.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.081+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/346094363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1e6fd0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.081+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/346094363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb0544a0 0x555abb1da170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.081+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/346094363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb13e160 0x555abb1d5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.081+0000 7f1c8d948b80 1 -- 172.21.15.46:0/346094363 >> 172.21.15.46:0/346094363 conn(0x555abb0b46a0 msgr2=0x555abb1d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.081+0000 7f1c8d948b80 1 -- 172.21.15.46:0/346094363 shutdown_connections 2024-05-09T04:54:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.081+0000 7f1c8d948b80 1 -- 172.21.15.46:0/346094363 wait complete. 2024-05-09T04:54:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.081+0000 7f1c8d948b80 1 -- start start 2024-05-09T04:54:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.081+0000 7f1c8d948b80 1 Processor -- start 2024-05-09T04:54:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c8d948b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb0544a0 0x555abb1f2850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c8d948b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb13e160 0x555abb1f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c8d948b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c8d948b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781af570 con 0x555abb1d16a0 2024-05-09T04:54:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c8d948b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781b09e0 con 0x555abb0544a0 2024-05-09T04:54:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c8d948b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1c781afc40 con 0x555abb13e160 2024-05-09T04:54:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c8d948b80 10 client.0 _ll_register_callbacks cb 0x555abaf656c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:09.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c7d4cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c7ccce700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb0544a0 0x555abb1f2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c7d4cf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1f32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57172/0 (socket says 172.21.15.46:57172) 2024-05-09T04:54:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/831597615 learned_addr learned my addr 172.21.15.46:0/831597615 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c77fff700 1 --2- 172.21.15.46:0/831597615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb13e160 0x555abb1f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/831597615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb13e160 msgr2=0x555abb1f2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c7d4cf700 1 --2- 172.21.15.46:0/831597615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb13e160 0x555abb1f2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/831597615 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb0544a0 msgr2=0x555abb1f2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c7d4cf700 1 --2- 172.21.15.46:0/831597615 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb0544a0 0x555abb1f2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c7d4cf700 1 -- 172.21.15.46:0/831597615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555abb254ac0 con 0x555abb1d16a0 2024-05-09T04:54:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.082+0000 7f1c7ccce700 1 --2- 172.21.15.46:0/831597615 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb0544a0 0x555abb1f2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c7d4cf700 1 --2- 172.21.15.46:0/831597615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1f32d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1c700a6120 tx=0x7f1c700a8930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea0a2a00da88b8f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c57fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c57fff700 1 -- 172.21.15.46:0/831597615 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c700a8eb0 con 0x555abb1d16a0 2024-05-09T04:54:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c57fff700 1 -- 172.21.15.46:0/831597615 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f1c700b1a90 con 0x555abb1d16a0 2024-05-09T04:54:09.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c8d948b80 1 -- 172.21.15.46:0/831597615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1c700a95b0 con 0x555abb1d16a0 2024-05-09T04:54:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c57fff700 1 -- 172.21.15.46:0/831597615 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1c700b1d10 con 0x555abb1d16a0 2024-05-09T04:54:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c8d948b80 10 client.16146 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c8d948b80 1 -- 172.21.15.46:0/831597615 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555abb255d50 con 0x555abb1d16a0 2024-05-09T04:54:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c57fff700 1 -- 172.21.15.46:0/831597615 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c700a95b0 con 0x555abb1d16a0 2024-05-09T04:54:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c57fff700 1 -- 172.21.15.46:0/831597615 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f1c700a82b0 con 0x555abb1d16a0 2024-05-09T04:54:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c8d948b80 10 client.16146 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c8d948b80 10 client.16146 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c8d948b80 20 client.16146 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/831597615 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x555abb255f70 0x555abb258390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c8d948b80 4 client.16146 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:54:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.083+0000 7f1c8d948b80 1 -- 172.21.15.46:0/831597615 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555abb0fba70 con 0x555abb255f70 2024-05-09T04:54:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.084+0000 7f1c7ccce700 1 --2- 172.21.15.46:0/831597615 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x555abb255f70 0x555abb258390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.084+0000 7f1c7ccce700 1 --2- 172.21.15.46:0/831597615 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x555abb255f70 0x555abb258390 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5ca0d18d120171 server_cookie=c0b73466f4c8ef4b in_seq=0 out_seq=0 2024-05-09T04:54:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.084+0000 7f1c57fff700 10 client.16146 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:09.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.084+0000 7f1c57fff700 1 -- 172.21.15.46:0/831597615 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555abb0fba70 con 0x555abb255f70 2024-05-09T04:54:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.084+0000 7f1c57fff700 10 client.16146 handle_command_reply: tid=0 2024-05-09T04:54:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.084+0000 7f1c8d948b80 1 client.16146 shutdown 2024-05-09T04:54:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.084+0000 7f1c8d948b80 1 -- 172.21.15.46:0/831597615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 msgr2=0x555abb1f32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.084+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/831597615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1f32d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f1c700a6120 tx=0x7f1c700a8930 comp rx=0 tx=0).stop 2024-05-09T04:54:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.085+0000 7f1c8d948b80 1 -- 172.21.15.46:0/831597615 shutdown_connections 2024-05-09T04:54:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.085+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/831597615 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555abb1d16a0 0x555abb1f32d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.085+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/831597615 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x555abb255f70 0x555abb258390 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.085+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/831597615 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555abb0544a0 0x555abb1f2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.085+0000 7f1c8d948b80 1 --2- 172.21.15.46:0/831597615 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555abb13e160 0x555abb1f2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.085+0000 7f1c8d948b80 1 -- 172.21.15.46:0/831597615 >> 172.21.15.46:0/831597615 conn(0x555abb0b46a0 msgr2=0x555abb1cb8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.085+0000 7f1c57fff700 0 client.16146 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.085+0000 7f1c8d948b80 1 -- 172.21.15.46:0/831597615 shutdown_connections 2024-05-09T04:54:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.085+0000 7f1c8d948b80 1 -- 172.21.15.46:0/831597615 wait complete. 2024-05-09T04:54:09.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.085+0000 7f1c8d948b80 20 client.16146 trim_cache size 0 max 16384 2024-05-09T04:54:09.090 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:54:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:54:09.090 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:54:09.090 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:54:09.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.085+0000 7f1c7ef32700 1 -- 172.21.15.46:0/2369177035 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f1c680b2a70 msgr2=0x7f1c680b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.085+0000 7f1c7ef32700 1 --2- 172.21.15.46:0/2369177035 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f1c680b2a70 0x7f1c680b4f80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1c6c005950 tx=0x7f1c6c00a300 comp rx=0 tx=0).stop 2024-05-09T04:54:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.085+0000 7f1c7ef32700 1 -- 172.21.15.46:0/2369177035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c78108740 msgr2=0x7f1c781a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.085+0000 7f1c7ef32700 1 --2- 172.21.15.46:0/2369177035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c78108740 0x7f1c781a1830 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f1c70007ee0 tx=0x7f1c7000d3b0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.086+0000 7f1c7ef32700 1 -- 172.21.15.46:0/2369177035 shutdown_connections 2024-05-09T04:54:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.086+0000 7f1c7ef32700 1 --2- 172.21.15.46:0/2369177035 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f1c680b2a70 0x7f1c680b4f80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.086+0000 7f1c7ef32700 1 --2- 172.21.15.46:0/2369177035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1c78108740 0x7f1c781a1830 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.086+0000 7f1c7ef32700 1 --2- 172.21.15.46:0/2369177035 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1c781034c0 0x7f1c7819d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.086+0000 7f1c7ef32700 1 --2- 172.21.15.46:0/2369177035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1c78107df0 0x7f1c7819d5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.086+0000 7f1c7ef32700 1 -- 172.21.15.46:0/2369177035 >> 172.21.15.46:0/2369177035 conn(0x7f1c780fedf0 msgr2=0x7f1c7810ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.086+0000 7f1c7ef32700 1 -- 172.21.15.46:0/2369177035 shutdown_connections 2024-05-09T04:54:09.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:09.086+0000 7f1c7ef32700 1 -- 172.21.15.46:0/2369177035 wait complete. 2024-05-09T04:54:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.610+0000 7f0341bff640 1 -- 172.21.15.46:0/1010526503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03340af680 msgr2=0x7f03340afb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.610+0000 7f0341bff640 1 --2- 172.21.15.46:0/1010526503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03340af680 0x7f03340afb10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0328009900 tx=0x7f032802a400 comp rx=0 tx=0).stop 2024-05-09T04:54:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.611+0000 7f0341bff640 1 -- 172.21.15.46:0/1010526503 shutdown_connections 2024-05-09T04:54:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.611+0000 7f0341bff640 1 --2- 172.21.15.46:0/1010526503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03340b0050 0x7f03340bae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.611+0000 7f0341bff640 1 --2- 172.21.15.46:0/1010526503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03340af680 0x7f03340afb10 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.611+0000 7f0341bff640 1 --2- 172.21.15.46:0/1010526503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03340a5aa0 0x7f03340a5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.611+0000 7f0341bff640 1 -- 172.21.15.46:0/1010526503 >> 172.21.15.46:0/1010526503 conn(0x7f033401aee0 msgr2=0x7f033401b320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.611+0000 7f0341bff640 1 -- 172.21.15.46:0/1010526503 shutdown_connections 2024-05-09T04:54:09.612 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.611+0000 7f0341bff640 1 -- 172.21.15.46:0/1010526503 wait complete. 2024-05-09T04:54:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.612+0000 7f0341bff640 1 Processor -- start 2024-05-09T04:54:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.612+0000 7f0341bff640 1 -- start start 2024-05-09T04:54:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.613+0000 7f0341bff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03340a5aa0 0x7f033414a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.613+0000 7f0341bff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03340af680 0x7f033414ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.613+0000 7f0341bff640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03340b0050 0x7f0334144a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.613 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.613+0000 7f0341bff640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03340be2f0 con 0x7f03340af680 2024-05-09T04:54:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.613+0000 7f0341bff640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f03340be170 con 0x7f03340a5aa0 2024-05-09T04:54:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.613+0000 7f0341bff640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03340be470 con 0x7f03340b0050 2024-05-09T04:54:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.613+0000 7f033bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03340af680 0x7f033414ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.613+0000 7f033bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03340af680 0x7f033414ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57188/0 (socket says 172.21.15.46:57188) 2024-05-09T04:54:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.613+0000 7f033bfff640 1 -- 172.21.15.46:0/756565535 learned_addr learned my addr 172.21.15.46:0/756565535 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.613+0000 7f03413fe640 1 --2- 172.21.15.46:0/756565535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03340b0050 0x7f0334144a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.614 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.613+0000 7f0340bfd640 1 --2- 172.21.15.46:0/756565535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03340a5aa0 0x7f033414a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.613+0000 7f033bfff640 1 -- 172.21.15.46:0/756565535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03340b0050 msgr2=0x7f0334144a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.614+0000 7f033bfff640 1 --2- 172.21.15.46:0/756565535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03340b0050 0x7f0334144a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.614+0000 7f033bfff640 1 -- 172.21.15.46:0/756565535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03340a5aa0 msgr2=0x7f033414a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.614+0000 7f033bfff640 1 --2- 172.21.15.46:0/756565535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03340a5aa0 0x7f033414a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.614+0000 7f033bfff640 1 -- 172.21.15.46:0/756565535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03341450f0 con 0x7f03340af680 2024-05-09T04:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.614+0000 7f033bfff640 1 --2- 172.21.15.46:0/756565535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03340af680 0x7f033414ad80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0328005ab0 tx=0x7f0328003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.614+0000 7f0339ffb640 1 -- 172.21.15.46:0/756565535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032803f070 con 0x7f03340af680 2024-05-09T04:54:09.615 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.614+0000 7f0339ffb640 1 -- 172.21.15.46:0/756565535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0328031b60 con 0x7f03340af680 2024-05-09T04:54:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.614+0000 7f0341bff640 1 -- 172.21.15.46:0/756565535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0334145380 con 0x7f03340af680 2024-05-09T04:54:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.614+0000 7f0339ffb640 1 -- 172.21.15.46:0/756565535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328035030 con 0x7f03340af680 2024-05-09T04:54:09.616 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.614+0000 7f0341bff640 1 -- 172.21.15.46:0/756565535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0334145870 con 0x7f03340af680 2024-05-09T04:54:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.618+0000 7f0349df8740 1 Processor -- start 2024-05-09T04:54:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.618+0000 7f0349df8740 1 -- start start 2024-05-09T04:54:09.619 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.619+0000 7f0349df8740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8ea061b0 0x557e8eda9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.619+0000 7f0349df8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 0x557e8eda9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.619+0000 7f0349df8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9e54d0 0x557e8edaa2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.619+0000 7f0349df8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557e8ed0e7c0 con 0x557e8e9dd2f0 2024-05-09T04:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.619+0000 7f0349df8740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557e8ecaa0e0 con 0x557e8ea061b0 2024-05-09T04:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.619+0000 7f0349df8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557e8ed0a780 con 0x557e8e9e54d0 2024-05-09T04:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.619+0000 7f03413fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8ea061b0 0x557e8eda9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.619+0000 7f03413fe640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8ea061b0 0x557e8eda9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:38748/0 (socket says 172.21.15.46:38748) 2024-05-09T04:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.619+0000 7f03413fe640 1 -- 172.21.15.46:0/476485627 learned_addr learned my addr 172.21.15.46:0/476485627 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:09.620 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.619+0000 7f0340bfd640 1 --2- 172.21.15.46:0/476485627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 0x557e8eda9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.620+0000 7f03413fe640 1 -- 172.21.15.46:0/476485627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9e54d0 msgr2=0x557e8edaa2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-09T04:54:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.620+0000 7f03413fe640 1 --2- 172.21.15.46:0/476485627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9e54d0 0x557e8edaa2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.620+0000 7f03413fe640 1 -- 172.21.15.46:0/476485627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 msgr2=0x557e8eda9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.620+0000 7f03413fe640 1 --2- 172.21.15.46:0/476485627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 0x557e8eda9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.620+0000 7f03413fe640 1 -- 172.21.15.46:0/476485627 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e8ecd37c0 con 0x557e8ea061b0 2024-05-09T04:54:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.620+0000 7f03413fe640 1 --2- 172.21.15.46:0/476485627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8ea061b0 0x557e8eda9840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f033c02e1f0 tx=0x7f033c068f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f480865af1eb343d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:09.621 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.620+0000 7f0339ffb640 1 -- 172.21.15.46:0/756565535 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0328042070 con 0x7f03340af680 2024-05-09T04:54:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.621+0000 7f0339ffb640 1 --2- 172.21.15.46:0/756565535 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f032c0b29e0 0x7f032c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.622+0000 7f0339ffb640 1 -- 172.21.15.46:0/756565535 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f0328102d20 con 0x7f03340af680 2024-05-09T04:54:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.622+0000 7f0326ffd640 1 -- 172.21.15.46:0/476485627 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c077cd0 con 0x557e8ea061b0 2024-05-09T04:54:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.622+0000 7f0326ffd640 1 -- 172.21.15.46:0/476485627 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f033c074890 con 0x557e8ea061b0 2024-05-09T04:54:09.622 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.622+0000 7f0326ffd640 1 -- 172.21.15.46:0/476485627 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c075780 con 0x557e8ea061b0 2024-05-09T04:54:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.623+0000 7f0349df8740 1 -- 172.21.15.46:0/476485627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8ea061b0 msgr2=0x557e8eda9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.623+0000 7f0349df8740 1 --2- 172.21.15.46:0/476485627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8ea061b0 0x557e8eda9840 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f033c02e1f0 tx=0x7f033c068f90 comp rx=0 tx=0).stop 2024-05-09T04:54:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.623+0000 7f0340bfd640 1 --2- 172.21.15.46:0/756565535 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f032c0b29e0 0x7f032c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.623+0000 7f0349df8740 1 -- 172.21.15.46:0/476485627 shutdown_connections 2024-05-09T04:54:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.623+0000 7f0349df8740 1 --2- 172.21.15.46:0/476485627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9e54d0 0x557e8edaa2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.623 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.623+0000 7f0349df8740 1 --2- 172.21.15.46:0/476485627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 0x557e8eda9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.623+0000 7f0349df8740 1 --2- 172.21.15.46:0/476485627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8ea061b0 0x557e8eda9840 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.623+0000 7f0349df8740 1 -- 172.21.15.46:0/476485627 >> 172.21.15.46:0/476485627 conn(0x557e8e9fa300 msgr2=0x557e8eda5ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.623+0000 7f0349df8740 1 -- 172.21.15.46:0/476485627 shutdown_connections 2024-05-09T04:54:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.623+0000 7f0349df8740 1 -- 172.21.15.46:0/476485627 wait complete. 2024-05-09T04:54:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.624+0000 7f0349df8740 1 -- start start 2024-05-09T04:54:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.624+0000 7f0340bfd640 1 --2- 172.21.15.46:0/756565535 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f032c0b29e0 0x7f032c0b4ef0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x557e8edaabf0 tx=0x7f0330005f90 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.624+0000 7f0349df8740 1 Processor -- start 2024-05-09T04:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.625+0000 7f0349df8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9dd2f0 0x557e8edb58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.625+0000 7f0349df8740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9e54d0 0x557e8edb5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.625+0000 7f0349df8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8ea061b0 0x557e8edb3d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.625+0000 7f0349df8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557e8edb6970 con 0x557e8ea061b0 2024-05-09T04:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.625+0000 7f0349df8740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557e8ec959c0 con 0x557e8e9e54d0 2024-05-09T04:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.625+0000 7f0349df8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557e8edb6af0 con 0x557e8e9dd2f0 2024-05-09T04:54:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.625+0000 7f0349df8740 10 client.0 _ll_register_callbacks cb 0x557e8ecc2020 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.628+0000 7f033bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9e54d0 0x557e8edb5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.629+0000 7f033bfff640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9e54d0 0x557e8edb5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:38750/0 (socket says 172.21.15.46:38750) 2024-05-09T04:54:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.629+0000 7f033bfff640 1 -- 172.21.15.46:0/1513232916 learned_addr learned my addr 172.21.15.46:0/1513232916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.630+0000 7f033bfff640 1 -- 172.21.15.46:0/1513232916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9dd2f0 msgr2=0x557e8edb58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.630+0000 7f033bfff640 1 --2- 172.21.15.46:0/1513232916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9dd2f0 0x557e8edb58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.630+0000 7f033bfff640 1 -- 172.21.15.46:0/1513232916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8ea061b0 msgr2=0x557e8edb3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.630+0000 7f033bfff640 1 --2- 172.21.15.46:0/1513232916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8ea061b0 0x557e8edb3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.630+0000 7f033bfff640 1 -- 172.21.15.46:0/1513232916 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e8ecbc1f0 con 0x557e8e9e54d0 2024-05-09T04:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.630+0000 7f033bfff640 1 --2- 172.21.15.46:0/1513232916 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9e54d0 0x557e8edb5e50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0328031440 tx=0x7f03280c9a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb7aa77175274f66 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.630+0000 7f031bfff640 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:54:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.631+0000 7f031bfff640 1 -- 172.21.15.46:0/1513232916 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328049070 con 0x557e8e9e54d0 2024-05-09T04:54:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.631+0000 7f0349df8740 1 -- 172.21.15.46:0/1513232916 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f033c074890 con 0x557e8e9e54d0 2024-05-09T04:54:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.631+0000 7f031bfff640 1 -- 172.21.15.46:0/1513232916 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f0328100670 con 0x557e8e9e54d0 2024-05-09T04:54:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.632+0000 7f031bfff640 1 -- 172.21.15.46:0/1513232916 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f03280ca030 con 0x557e8e9e54d0 2024-05-09T04:54:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.632+0000 7f0349df8740 10 client.25784 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.632+0000 7f0349df8740 1 -- 172.21.15.46:0/1513232916 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557e8ea71790 con 0x557e8e9e54d0 2024-05-09T04:54:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.632+0000 7f031bfff640 1 -- 172.21.15.46:0/1513232916 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f033c074890 con 0x557e8e9e54d0 2024-05-09T04:54:09.632 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.632+0000 7f031bfff640 1 -- 172.21.15.46:0/1513232916 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f03280d2950 con 0x557e8e9e54d0 2024-05-09T04:54:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.633+0000 7f0349df8740 10 client.25784 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.633+0000 7f0349df8740 10 client.25784 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.633+0000 7f0349df8740 20 client.25784 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.633+0000 7f0349df8740 1 --2- 172.21.15.46:0/1513232916 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e8e9c4d80 0x557e8edc66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.633+0000 7f0349df8740 4 client.25784 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.633+0000 7f0349df8740 1 -- 172.21.15.46:0/1513232916 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557e8ed1c3d0 con 0x557e8e9c4d80 2024-05-09T04:54:09.633 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.633+0000 7f0340bfd640 1 --2- 172.21.15.46:0/1513232916 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e8e9c4d80 0x557e8edc66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.634+0000 7f0340bfd640 1 --2- 172.21.15.46:0/1513232916 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e8e9c4d80 0x557e8edc66d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a30093f069e147c3 server_cookie=4cfa1a98bbe90f23 in_seq=0 out_seq=0 2024-05-09T04:54:09.634 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.634+0000 7f031bfff640 10 client.25784 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.635+0000 7f031bfff640 1 -- 172.21.15.46:0/1513232916 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557e8ed1c3d0 con 0x557e8e9c4d80 2024-05-09T04:54:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.635+0000 7f031bfff640 10 client.25784 handle_command_reply: tid=0 2024-05-09T04:54:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.635+0000 7f0349df8740 1 client.25784 shutdown 2024-05-09T04:54:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.636+0000 7f0349df8740 1 -- 172.21.15.46:0/1513232916 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9e54d0 msgr2=0x557e8edb5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.636+0000 7f0349df8740 1 --2- 172.21.15.46:0/1513232916 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9e54d0 0x557e8edb5e50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0328031440 tx=0x7f03280c9a40 comp rx=0 tx=0).stop 2024-05-09T04:54:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.636+0000 7f0349df8740 1 -- 172.21.15.46:0/1513232916 shutdown_connections 2024-05-09T04:54:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.636+0000 7f0349df8740 1 --2- 172.21.15.46:0/1513232916 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e8e9c4d80 0x557e8edc66d0 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.636+0000 7f0349df8740 1 --2- 172.21.15.46:0/1513232916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8ea061b0 0x557e8edb3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.636+0000 7f0349df8740 1 --2- 172.21.15.46:0/1513232916 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9e54d0 0x557e8edb5e50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.636+0000 7f0349df8740 1 --2- 172.21.15.46:0/1513232916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9dd2f0 0x557e8edb58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.636+0000 7f0349df8740 1 -- 172.21.15.46:0/1513232916 >> 172.21.15.46:0/1513232916 conn(0x557e8e9fa300 msgr2=0x557e8eda9670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.636+0000 7f031bfff640 0 client.25784 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.636+0000 7f0349df8740 1 -- 172.21.15.46:0/1513232916 shutdown_connections 2024-05-09T04:54:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.636+0000 7f0349df8740 1 -- 172.21.15.46:0/1513232916 wait complete. 2024-05-09T04:54:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.636+0000 7f0349df8740 20 client.25784 trim_cache size 0 max 16384 2024-05-09T04:54:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.660+0000 7f0349df8740 1 Processor -- start 2024-05-09T04:54:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.660+0000 7f0349df8740 1 -- start start 2024-05-09T04:54:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.660+0000 7f0349df8740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9fa300 0x557e8edb3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.660+0000 7f0349df8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 0x557e8edad490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.660+0000 7f0349df8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9e54d0 0x557e8edad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.660+0000 7f0349df8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f033415c7c0 con 0x557e8e9dd2f0 2024-05-09T04:54:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.660+0000 7f0349df8740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f033415a090 con 0x557e8e9fa300 2024-05-09T04:54:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.660+0000 7f0349df8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0334157ee0 con 0x557e8e9e54d0 2024-05-09T04:54:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.660+0000 7f0340bfd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9fa300 0x557e8edb3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.661+0000 7f033bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 0x557e8edad490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.661+0000 7f0340bfd640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9fa300 0x557e8edb3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:38752/0 (socket says 172.21.15.46:38752) 2024-05-09T04:54:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.661+0000 7f0340bfd640 1 -- 172.21.15.46:0/1634390535 learned_addr learned my addr 172.21.15.46:0/1634390535 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.661+0000 7f03413fe640 1 --2- 172.21.15.46:0/1634390535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9e54d0 0x557e8edad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.661+0000 7f033bfff640 1 -- 172.21.15.46:0/1634390535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9e54d0 msgr2=0x557e8edad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.661+0000 7f033bfff640 1 --2- 172.21.15.46:0/1634390535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9e54d0 0x557e8edad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.661+0000 7f033bfff640 1 -- 172.21.15.46:0/1634390535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9fa300 msgr2=0x557e8edb3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.661+0000 7f033bfff640 1 --2- 172.21.15.46:0/1634390535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9fa300 0x557e8edb3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.661+0000 7f033bfff640 1 -- 172.21.15.46:0/1634390535 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557e8edaf630 con 0x557e8e9dd2f0 2024-05-09T04:54:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.661+0000 7f0340bfd640 1 --2- 172.21.15.46:0/1634390535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9fa300 0x557e8edb3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.661+0000 7f033bfff640 1 --2- 172.21.15.46:0/1634390535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 0x557e8edad490 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0328005f70 tx=0x7f03280045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5248772d5df016a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.662+0000 7f03267fc640 1 -- 172.21.15.46:0/1634390535 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328049070 con 0x557e8e9dd2f0 2024-05-09T04:54:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.662+0000 7f03267fc640 1 -- 172.21.15.46:0/1634390535 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03280ce330 con 0x557e8e9dd2f0 2024-05-09T04:54:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.662+0000 7f03267fc640 1 -- 172.21.15.46:0/1634390535 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0328100670 con 0x557e8e9dd2f0 2024-05-09T04:54:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.663+0000 7f0349df8740 1 -- 172.21.15.46:0/1634390535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 msgr2=0x557e8edad490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.663+0000 7f0349df8740 1 --2- 172.21.15.46:0/1634390535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 0x557e8edad490 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0328005f70 tx=0x7f03280045e0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.665+0000 7f0349df8740 1 -- 172.21.15.46:0/1634390535 shutdown_connections 2024-05-09T04:54:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.665+0000 7f0349df8740 1 --2- 172.21.15.46:0/1634390535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9e54d0 0x557e8edad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.665+0000 7f0349df8740 1 --2- 172.21.15.46:0/1634390535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 0x557e8edad490 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.665+0000 7f0349df8740 1 --2- 172.21.15.46:0/1634390535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9fa300 0x557e8edb3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.665 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.665+0000 7f0349df8740 1 -- 172.21.15.46:0/1634390535 >> 172.21.15.46:0/1634390535 conn(0x557e8edc0dd0 msgr2=0x557e8edb3600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.665+0000 7f0349df8740 1 -- 172.21.15.46:0/1634390535 shutdown_connections 2024-05-09T04:54:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.665+0000 7f0349df8740 1 -- 172.21.15.46:0/1634390535 wait complete. 2024-05-09T04:54:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.666+0000 7f0349df8740 1 -- start start 2024-05-09T04:54:09.666 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.666+0000 7f0349df8740 1 Processor -- start 2024-05-09T04:54:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.667+0000 7f0349df8740 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 0x557e8edb8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.667+0000 7f0349df8740 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9e54d0 0x557e8edb90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.667 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.667+0000 7f0349df8740 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9fa300 0x557e8edb95f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.667+0000 7f0349df8740 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0334158550 con 0x557e8e9dd2f0 2024-05-09T04:54:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.667+0000 7f0349df8740 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f03341599c0 con 0x557e8e9e54d0 2024-05-09T04:54:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.668+0000 7f0349df8740 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0334158c20 con 0x557e8e9fa300 2024-05-09T04:54:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.668+0000 7f0349df8740 10 client.0 _ll_register_callbacks cb 0x7f0328005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.668+0000 7f0340bfd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 0x557e8edb8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.668 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.668+0000 7f0340bfd640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 0x557e8edb8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57212/0 (socket says 172.21.15.46:57212) 2024-05-09T04:54:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.668+0000 7f0340bfd640 1 -- 172.21.15.46:0/1029781101 learned_addr learned my addr 172.21.15.46:0/1029781101 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.668+0000 7f03413fe640 1 --2- 172.21.15.46:0/1029781101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9fa300 0x557e8edb95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.668+0000 7f033bfff640 1 --2- 172.21.15.46:0/1029781101 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9e54d0 0x557e8edb90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.669+0000 7f03413fe640 1 -- 172.21.15.46:0/1029781101 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9e54d0 msgr2=0x557e8edb90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.669+0000 7f03413fe640 1 --2- 172.21.15.46:0/1029781101 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9e54d0 0x557e8edb90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.669+0000 7f03413fe640 1 -- 172.21.15.46:0/1029781101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 msgr2=0x557e8edb8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.669+0000 7f03413fe640 1 --2- 172.21.15.46:0/1029781101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 0x557e8edb8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.669+0000 7f03413fe640 1 -- 172.21.15.46:0/1029781101 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557e8eae85f0 con 0x557e8e9fa300 2024-05-09T04:54:09.669 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.669+0000 7f03413fe640 1 --2- 172.21.15.46:0/1029781101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9fa300 0x557e8edb95f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f033c052220 tx=0x7f033c0699f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b4ff434c713c6076 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.669+0000 7f031bfff640 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:54:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.670+0000 7f031bfff640 1 -- 172.21.15.46:0/1029781101 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f033c06a040 con 0x557e8e9fa300 2024-05-09T04:54:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.670+0000 7f031bfff640 1 -- 172.21.15.46:0/1029781101 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f033c06a1e0 con 0x557e8e9fa300 2024-05-09T04:54:09.670 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.670+0000 7f0349df8740 1 -- 172.21.15.46:0/1029781101 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03280ce330 con 0x557e8e9fa300 2024-05-09T04:54:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.676+0000 7f031bfff640 1 -- 172.21.15.46:0/1029781101 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f033c06a3e0 con 0x557e8e9fa300 2024-05-09T04:54:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.677+0000 7f0349df8740 10 client.25711 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:09.677 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.677+0000 7f0349df8740 1 -- 172.21.15.46:0/1029781101 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557e8ecd2ed0 con 0x557e8e9fa300 2024-05-09T04:54:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.678+0000 7f031bfff640 1 -- 172.21.15.46:0/1029781101 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03280ce330 con 0x557e8e9fa300 2024-05-09T04:54:09.678 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.679+0000 7f031bfff640 1 -- 172.21.15.46:0/1029781101 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f033c07e020 con 0x557e8e9fa300 2024-05-09T04:54:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.679+0000 7f0349df8740 10 client.25711 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.679+0000 7f0349df8740 10 client.25711 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:09.679 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.679+0000 7f0349df8740 20 client.25711 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.679+0000 7f0349df8740 1 --2- 172.21.15.46:0/1029781101 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e8ea061b0 0x557e8ee30080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.679+0000 7f0349df8740 4 client.25711 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-09T04:54:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.679+0000 7f0349df8740 1 -- 172.21.15.46:0/1029781101 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "flush journal"}) -- 0x557e8ed16240 con 0x557e8ea061b0 2024-05-09T04:54:09.680 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.680+0000 7f0340bfd640 1 --2- 172.21.15.46:0/1029781101 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e8ea061b0 0x557e8ee30080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.682+0000 7f0340bfd640 1 --2- 172.21.15.46:0/1029781101 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e8ea061b0 0x557e8ee30080 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49abb7ed2ac2c3ac server_cookie=787be46bb9b50f0d in_seq=0 out_seq=0 2024-05-09T04:54:09.681 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.682+0000 7f031bfff640 10 client.25711 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.716+0000 7f031bfff640 1 -- 172.21.15.46:0/1029781101 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+25 (crc 0 0 0) 0x557e8ed16240 con 0x557e8ea061b0 2024-05-09T04:54:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.716+0000 7f031bfff640 10 client.25711 handle_command_reply: tid=0 2024-05-09T04:54:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.716+0000 7f0349df8740 1 client.25711 shutdown 2024-05-09T04:54:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.717+0000 7f0349df8740 1 -- 172.21.15.46:0/1029781101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9fa300 msgr2=0x557e8edb95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.717+0000 7f0349df8740 1 --2- 172.21.15.46:0/1029781101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9fa300 0x557e8edb95f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f033c052220 tx=0x7f033c0699f0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.717+0000 7f0349df8740 1 -- 172.21.15.46:0/1029781101 shutdown_connections 2024-05-09T04:54:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.717+0000 7f0349df8740 1 --2- 172.21.15.46:0/1029781101 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557e8ea061b0 0x557e8ee30080 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.717+0000 7f031bfff640 0 client.25711 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.717+0000 7f0349df8740 1 --2- 172.21.15.46:0/1029781101 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557e8e9fa300 0x557e8edb95f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.717+0000 7f0349df8740 1 --2- 172.21.15.46:0/1029781101 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557e8e9e54d0 0x557e8edb90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.717+0000 7f0349df8740 1 --2- 172.21.15.46:0/1029781101 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557e8e9dd2f0 0x557e8edb8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.717+0000 7f0349df8740 1 -- 172.21.15.46:0/1029781101 >> 172.21.15.46:0/1029781101 conn(0x557e8edc0dd0 msgr2=0x557e8edad490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.717+0000 7f0349df8740 1 -- 172.21.15.46:0/1029781101 shutdown_connections 2024-05-09T04:54:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.717+0000 7f0349df8740 1 -- 172.21.15.46:0/1029781101 wait complete. 2024-05-09T04:54:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.718+0000 7f0349df8740 20 client.25711 trim_cache size 0 max 16384 2024-05-09T04:54:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:{ 2024-05-09T04:54:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout: "return_code": 0 2024-05-09T04:54:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stdout:} 2024-05-09T04:54:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.720+0000 7f0341bff640 1 -- 172.21.15.46:0/756565535 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f032c0b29e0 msgr2=0x7f032c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.720+0000 7f0341bff640 1 --2- 172.21.15.46:0/756565535 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f032c0b29e0 0x7f032c0b4ef0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x557e8edaabf0 tx=0x7f0330005f90 comp rx=0 tx=0).stop 2024-05-09T04:54:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.720+0000 7f0341bff640 1 -- 172.21.15.46:0/756565535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03340af680 msgr2=0x7f033414ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.720+0000 7f0341bff640 1 --2- 172.21.15.46:0/756565535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03340af680 0x7f033414ad80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0328005ab0 tx=0x7f0328003c10 comp rx=0 tx=0).stop 2024-05-09T04:54:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.720+0000 7f0341bff640 1 -- 172.21.15.46:0/756565535 shutdown_connections 2024-05-09T04:54:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.720+0000 7f0341bff640 1 --2- 172.21.15.46:0/756565535 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f032c0b29e0 0x7f032c0b4ef0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.720+0000 7f0341bff640 1 --2- 172.21.15.46:0/756565535 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03340b0050 0x7f0334144a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.720+0000 7f0341bff640 1 --2- 172.21.15.46:0/756565535 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03340af680 0x7f033414ad80 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.720+0000 7f0341bff640 1 --2- 172.21.15.46:0/756565535 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03340a5aa0 0x7f033414a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.720+0000 7f0341bff640 1 -- 172.21.15.46:0/756565535 >> 172.21.15.46:0/756565535 conn(0x7f033401aee0 msgr2=0x7f03340ad560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.721+0000 7f0341bff640 1 -- 172.21.15.46:0/756565535 shutdown_connections 2024-05-09T04:54:09.721 INFO:teuthology.task.background_exec.ubuntu@smithi046.front.sepia.ceph.com.smithi046.stderr:2024-05-09T04:54:09.721+0000 7f0341bff640 1 -- 172.21.15.46:0/756565535 wait complete. 2024-05-09T04:54:09.950 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:54:09.951 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:54:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:10.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:10 smithi046 ceph-mon[30136]: pgmap v95: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-05-09T04:54:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:10 smithi110 ceph-mon[34904]: pgmap v95: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-05-09T04:54:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:10 smithi072 ceph-mon[34088]: pgmap v95: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-05-09T04:54:13.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:12 smithi110 ceph-mon[34904]: pgmap v96: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 111 op/s 2024-05-09T04:54:13.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:12 smithi046 ceph-mon[30136]: pgmap v96: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 111 op/s 2024-05-09T04:54:13.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:12 smithi072 ceph-mon[34088]: pgmap v96: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 111 op/s 2024-05-09T04:54:15.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:14 smithi110 ceph-mon[34904]: pgmap v97: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 178 op/s 2024-05-09T04:54:15.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:14 smithi046 ceph-mon[30136]: pgmap v97: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 178 op/s 2024-05-09T04:54:15.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:14 smithi072 ceph-mon[34088]: pgmap v97: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 15 KiB/s wr, 178 op/s 2024-05-09T04:54:16.244 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:54:16.244 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:54:16.244 INFO:tasks.workunit.client.0.smithi046.stdout: 2024-05-09T04:54:16.244 INFO:tasks.workunit.client.0.smithi046.stdout:iozone test complete. 2024-05-09T04:54:16.245 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-09T04:54:16.245 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-09T04:54:16.282 INFO:tasks.workunit:Stopping ['suites/iozone.sh'] on client.0... 2024-05-09T04:54:16.282 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-09T04:54:16.639 DEBUG:teuthology.parallel:result is None 2024-05-09T04:54:16.640 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T04:54:16.674 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-09T04:54:16.675 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-09T04:54:16.685 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T04:54:16.723 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-09T04:54:16.736 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-09T04:54:16.736 INFO:teuthology.task.exec:Executing custom commands... 2024-05-09T04:54:16.736 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T04:54:16.736 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-09T04:54:16.791 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-09T04:54:16.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:16 smithi046 ceph-mon[30136]: pgmap v98: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 165 op/s 2024-05-09T04:54:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:16 smithi110 ceph-mon[34904]: pgmap v98: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 165 op/s 2024-05-09T04:54:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:16 smithi072 ceph-mon[34088]: pgmap v98: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 4.2 KiB/s wr, 165 op/s 2024-05-09T04:54:17.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.545+0000 7f35b28cd700 1 -- 172.21.15.46:0/2772682890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35ac108740 msgr2=0x7f35ac1100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:17.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.545+0000 7f35b28cd700 1 --2- 172.21.15.46:0/2772682890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35ac108740 0x7f35ac1100a0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f35a400b0d0 tx=0x7f35a400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:54:17.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.545+0000 7f35b28cd700 1 -- 172.21.15.46:0/2772682890 shutdown_connections 2024-05-09T04:54:17.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.545+0000 7f35b28cd700 1 --2- 172.21.15.46:0/2772682890 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35ac108740 0x7f35ac1100a0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:17.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.545+0000 7f35b28cd700 1 --2- 172.21.15.46:0/2772682890 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35ac1034c0 0x7f35ac1038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:17.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.545+0000 7f35b28cd700 1 --2- 172.21.15.46:0/2772682890 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35ac107df0 0x7f35ac108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:17.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.545+0000 7f35b28cd700 1 -- 172.21.15.46:0/2772682890 >> 172.21.15.46:0/2772682890 conn(0x7f35ac0fedf0 msgr2=0x7f35ac101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:17.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.545+0000 7f35b28cd700 1 -- 172.21.15.46:0/2772682890 shutdown_connections 2024-05-09T04:54:17.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.545+0000 7f35b28cd700 1 -- 172.21.15.46:0/2772682890 wait complete. 2024-05-09T04:54:17.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.546+0000 7f35b28cd700 1 Processor -- start 2024-05-09T04:54:17.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.546+0000 7f35b28cd700 1 -- start start 2024-05-09T04:54:17.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.546+0000 7f35b28cd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35ac1034c0 0x7f35ac19bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:17.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.546+0000 7f35b28cd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35ac107df0 0x7f35ac19c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:17.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.546+0000 7f35b28cd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35ac108740 0x7f35ac19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:17.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.546+0000 7f35b28cd700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac113370 con 0x7f35ac107df0 2024-05-09T04:54:17.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.546+0000 7f35b28cd700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac1131f0 con 0x7f35ac108740 2024-05-09T04:54:17.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.546+0000 7f35b28cd700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f35ac1134f0 con 0x7f35ac1034c0 2024-05-09T04:54:17.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.546+0000 7f35a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35ac107df0 0x7f35ac19c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:17.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.546+0000 7f35a3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35ac107df0 0x7f35ac19c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44610/0 (socket says 172.21.15.46:44610) 2024-05-09T04:54:17.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.546+0000 7f35a3fff700 1 -- 172.21.15.46:0/2526063346 learned_addr learned my addr 172.21.15.46:0/2526063346 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:17.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.546+0000 7f35abfff700 1 --2- 172.21.15.46:0/2526063346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35ac1034c0 0x7f35ac19bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:17.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.546+0000 7f35b0e6a700 1 --2- 172.21.15.46:0/2526063346 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35ac108740 0x7f35ac19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:17.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.547+0000 7f35a3fff700 1 -- 172.21.15.46:0/2526063346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35ac1034c0 msgr2=0x7f35ac19bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:17.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.547+0000 7f35a3fff700 1 --2- 172.21.15.46:0/2526063346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35ac1034c0 0x7f35ac19bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:17.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.547+0000 7f35a3fff700 1 -- 172.21.15.46:0/2526063346 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35ac108740 msgr2=0x7f35ac19c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:17.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.547+0000 7f35a3fff700 1 --2- 172.21.15.46:0/2526063346 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35ac108740 0x7f35ac19c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:17.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.547+0000 7f35a3fff700 1 -- 172.21.15.46:0/2526063346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35ac1a6f10 con 0x7f35ac107df0 2024-05-09T04:54:17.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.547+0000 7f35a3fff700 1 --2- 172.21.15.46:0/2526063346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35ac107df0 0x7f35ac19c190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f359400d950 tx=0x7f359400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:17.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.547+0000 7f35a9ffb700 1 -- 172.21.15.46:0/2526063346 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594016070 con 0x7f35ac107df0 2024-05-09T04:54:17.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.547+0000 7f35a9ffb700 1 -- 172.21.15.46:0/2526063346 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3594004830 con 0x7f35ac107df0 2024-05-09T04:54:17.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.547+0000 7f35b28cd700 1 -- 172.21.15.46:0/2526063346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35ac1a7200 con 0x7f35ac107df0 2024-05-09T04:54:17.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.547+0000 7f35a9ffb700 1 -- 172.21.15.46:0/2526063346 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3594005520 con 0x7f35ac107df0 2024-05-09T04:54:17.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.547+0000 7f35b28cd700 1 -- 172.21.15.46:0/2526063346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35ac1a7740 con 0x7f35ac107df0 2024-05-09T04:54:17.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.548+0000 7f35b28cd700 1 -- 172.21.15.46:0/2526063346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35ac0512c0 con 0x7f35ac107df0 2024-05-09T04:54:17.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.550+0000 7f35a9ffb700 1 -- 172.21.15.46:0/2526063346 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3594004220 con 0x7f35ac107df0 2024-05-09T04:54:17.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.551+0000 7f35a9ffb700 1 --2- 172.21.15.46:0/2526063346 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f35980b29f0 0x7f35980b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:17.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.551+0000 7f35a9ffb700 1 -- 172.21.15.46:0/2526063346 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f35940e27d0 con 0x7f35ac107df0 2024-05-09T04:54:17.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.551+0000 7f35abfff700 1 --2- 172.21.15.46:0/2526063346 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f35980b29f0 0x7f35980b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:17.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.551+0000 7f35abfff700 1 --2- 172.21.15.46:0/2526063346 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f35980b29f0 0x7f35980b4f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f359c00ac50 tx=0x7f359c00a380 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:17.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.554+0000 7f35a9ffb700 1 -- 172.21.15.46:0/2526063346 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f359401f020 con 0x7f35ac107df0 2024-05-09T04:54:17.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.821+0000 7f35b28cd700 1 -- 172.21.15.46:0/2526063346 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35ac075fe0 con 0x7f35ac107df0 2024-05-09T04:54:17.821 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.821+0000 7f35a9ffb700 1 -- 172.21.15.46:0/2526063346 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7f35940a5020 con 0x7f35ac107df0 2024-05-09T04:54:17.822 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:54:17.822 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:54:17.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.824+0000 7f35b28cd700 1 -- 172.21.15.46:0/2526063346 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f35980b29f0 msgr2=0x7f35980b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:17.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.824+0000 7f35b28cd700 1 --2- 172.21.15.46:0/2526063346 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f35980b29f0 0x7f35980b4f00 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f359c00ac50 tx=0x7f359c00a380 comp rx=0 tx=0).stop 2024-05-09T04:54:17.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.824+0000 7f35b28cd700 1 -- 172.21.15.46:0/2526063346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35ac107df0 msgr2=0x7f35ac19c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:17.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.824+0000 7f35b28cd700 1 --2- 172.21.15.46:0/2526063346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35ac107df0 0x7f35ac19c190 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f359400d950 tx=0x7f359400dc60 comp rx=0 tx=0).stop 2024-05-09T04:54:17.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.824+0000 7f35b28cd700 1 -- 172.21.15.46:0/2526063346 shutdown_connections 2024-05-09T04:54:17.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.824+0000 7f35b28cd700 1 --2- 172.21.15.46:0/2526063346 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f35980b29f0 0x7f35980b4f00 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:17.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.824+0000 7f35b28cd700 1 --2- 172.21.15.46:0/2526063346 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f35ac108740 0x7f35ac19c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:17.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.824+0000 7f35b28cd700 1 --2- 172.21.15.46:0/2526063346 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f35ac1034c0 0x7f35ac19bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:17.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.824+0000 7f35b28cd700 1 --2- 172.21.15.46:0/2526063346 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f35ac107df0 0x7f35ac19c190 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:17.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.824+0000 7f35b28cd700 1 -- 172.21.15.46:0/2526063346 >> 172.21.15.46:0/2526063346 conn(0x7f35ac0fedf0 msgr2=0x7f35ac107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:17.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.824+0000 7f35b28cd700 1 -- 172.21.15.46:0/2526063346 shutdown_connections 2024-05-09T04:54:17.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:17.824+0000 7f35b28cd700 1 -- 172.21.15.46:0/2526063346 wait complete. 2024-05-09T04:54:17.825 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:54:18.609 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:54:18.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:18 smithi046 ceph-mon[30136]: pgmap v99: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 226 op/s 2024-05-09T04:54:18.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:18 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2526063346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:54:18.870 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:18 smithi110 ceph-mon[34904]: pgmap v99: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 226 op/s 2024-05-09T04:54:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:18 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2526063346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:54:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:18 smithi072 ceph-mon[34088]: pgmap v99: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.2 KiB/s wr, 226 op/s 2024-05-09T04:54:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:18 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2526063346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:54:20.013 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:19 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:54:20.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:19 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:54:20.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:19 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:54:20.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.706+0000 7f9285dce700 1 -- 172.21.15.46:0/109655419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9280107df0 msgr2=0x7f9280108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:20.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.706+0000 7f9285dce700 1 --2- 172.21.15.46:0/109655419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9280107df0 0x7f9280108200 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f92740099c0 tx=0x7f9274009cd0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.706+0000 7f9285dce700 1 -- 172.21.15.46:0/109655419 shutdown_connections 2024-05-09T04:54:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.706+0000 7f9285dce700 1 --2- 172.21.15.46:0/109655419 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9280108740 0x7f92801100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.706+0000 7f9285dce700 1 --2- 172.21.15.46:0/109655419 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f92801034c0 0x7f92801038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.706+0000 7f9285dce700 1 --2- 172.21.15.46:0/109655419 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9280107df0 0x7f9280108200 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.706+0000 7f9285dce700 1 -- 172.21.15.46:0/109655419 >> 172.21.15.46:0/109655419 conn(0x7f92800fedf0 msgr2=0x7f9280101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.706+0000 7f9285dce700 1 -- 172.21.15.46:0/109655419 shutdown_connections 2024-05-09T04:54:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.706+0000 7f9285dce700 1 -- 172.21.15.46:0/109655419 wait complete. 2024-05-09T04:54:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.707+0000 7f9285dce700 1 Processor -- start 2024-05-09T04:54:20.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.707+0000 7f9285dce700 1 -- start start 2024-05-09T04:54:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f9285dce700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92801034c0 0x7f928019aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f9285dce700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9280107df0 0x7f928019b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f9285dce700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9280108740 0x7f928019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f9285dce700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9280113350 con 0x7f9280107df0 2024-05-09T04:54:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f9285dce700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f92801131d0 con 0x7f92801034c0 2024-05-09T04:54:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f9285dce700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f92801134d0 con 0x7f9280108740 2024-05-09T04:54:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f927effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9280107df0 0x7f928019b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f927effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9280107df0 0x7f928019b470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44616/0 (socket says 172.21.15.46:44616) 2024-05-09T04:54:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f927effd700 1 -- 172.21.15.46:0/3013947072 learned_addr learned my addr 172.21.15.46:0/3013947072 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f927f7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92801034c0 0x7f928019aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f927ffff700 1 --2- 172.21.15.46:0/3013947072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9280108740 0x7f928019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f927effd700 1 -- 172.21.15.46:0/3013947072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9280108740 msgr2=0x7f928019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:20.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f927effd700 1 --2- 172.21.15.46:0/3013947072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9280108740 0x7f928019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f927effd700 1 -- 172.21.15.46:0/3013947072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92801034c0 msgr2=0x7f928019aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:20.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f927effd700 1 --2- 172.21.15.46:0/3013947072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92801034c0 0x7f928019aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f927effd700 1 -- 172.21.15.46:0/3013947072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f928019fda0 con 0x7f9280107df0 2024-05-09T04:54:20.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.708+0000 7f927effd700 1 --2- 172.21.15.46:0/3013947072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9280107df0 0x7f928019b470 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9274006010 tx=0x7f92740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:20.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.709+0000 7f927cff9700 1 -- 172.21.15.46:0/3013947072 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274027070 con 0x7f9280107df0 2024-05-09T04:54:20.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.709+0000 7f9285dce700 1 -- 172.21.15.46:0/3013947072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92801a0030 con 0x7f9280107df0 2024-05-09T04:54:20.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.709+0000 7f927cff9700 1 -- 172.21.15.46:0/3013947072 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9274003f20 con 0x7f9280107df0 2024-05-09T04:54:20.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.709+0000 7f927cff9700 1 -- 172.21.15.46:0/3013947072 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9274022c60 con 0x7f9280107df0 2024-05-09T04:54:20.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.709+0000 7f9285dce700 1 -- 172.21.15.46:0/3013947072 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92801a0510 con 0x7f9280107df0 2024-05-09T04:54:20.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.712+0000 7f927cff9700 1 -- 172.21.15.46:0/3013947072 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f927402a070 con 0x7f9280107df0 2024-05-09T04:54:20.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.712+0000 7f927cff9700 1 --2- 172.21.15.46:0/3013947072 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f92680b29f0 0x7f92680b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f927f7fe700 1 --2- 172.21.15.46:0/3013947072 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f92680b29f0 0x7f92680b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f92947e4b80 1 Processor -- start 2024-05-09T04:54:20.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f927cff9700 1 -- 172.21.15.46:0/3013947072 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f9274022e00 con 0x7f9280107df0 2024-05-09T04:54:20.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f92947e4b80 1 -- start start 2024-05-09T04:54:20.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f92947e4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d05166160 0x558d051ff4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f92947e4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 0x558d051ffa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f927f7fe700 1 --2- 172.21.15.46:0/3013947072 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f92680b29f0 0x7f92680b4f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9270005950 tx=0x7f927000b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f92947e4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d051f96a0 0x558d051f9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f92947e4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558d0511bff0 con 0x558d05166160 2024-05-09T04:54:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f92947e4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558d050ba7f0 con 0x558d051f96a0 2024-05-09T04:54:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f92947e4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558d05117fb0 con 0x558d0507c4a0 2024-05-09T04:54:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f927f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d05166160 0x558d051ff4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f927effd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 0x558d051ffa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f927f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d05166160 0x558d051ff4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44626/0 (socket says 172.21.15.46:44626) 2024-05-09T04:54:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f927f7fe700 1 -- 172.21.15.46:0/2050971103 learned_addr learned my addr 172.21.15.46:0/2050971103 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.713+0000 7f927ffff700 1 --2- 172.21.15.46:0/2050971103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d051f96a0 0x558d051f9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.714+0000 7f927effd700 1 -- 172.21.15.46:0/2050971103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d051f96a0 msgr2=0x558d051f9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.714+0000 7f927effd700 1 --2- 172.21.15.46:0/2050971103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d051f96a0 0x558d051f9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.714+0000 7f927effd700 1 -- 172.21.15.46:0/2050971103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d05166160 msgr2=0x558d051ff4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.714+0000 7f927effd700 1 --2- 172.21.15.46:0/2050971103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d05166160 0x558d051ff4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.714+0000 7f927effd700 1 -- 172.21.15.46:0/2050971103 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d050b2370 con 0x558d0507c4a0 2024-05-09T04:54:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.714+0000 7f927f7fe700 1 --2- 172.21.15.46:0/2050971103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d05166160 0x558d051ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.714+0000 7f927effd700 1 --2- 172.21.15.46:0/2050971103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 0x558d051ffa30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f92740e0b90 tx=0x7f92740095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2395a708ddbc49c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.714+0000 7f9265ffb700 1 -- 172.21.15.46:0/2050971103 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92740aaeb0 con 0x558d0507c4a0 2024-05-09T04:54:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.714+0000 7f9265ffb700 1 -- 172.21.15.46:0/2050971103 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92740b1e50 con 0x558d0507c4a0 2024-05-09T04:54:20.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.714+0000 7f9265ffb700 1 -- 172.21.15.46:0/2050971103 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92740aaa10 con 0x558d0507c4a0 2024-05-09T04:54:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.715+0000 7f92947e4b80 1 -- 172.21.15.46:0/2050971103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 msgr2=0x558d051ffa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.715+0000 7f92947e4b80 1 --2- 172.21.15.46:0/2050971103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 0x558d051ffa30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f92740e0b90 tx=0x7f92740095c0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.715+0000 7f92947e4b80 1 -- 172.21.15.46:0/2050971103 shutdown_connections 2024-05-09T04:54:20.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.715+0000 7f92947e4b80 1 --2- 172.21.15.46:0/2050971103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d051f96a0 0x558d051f9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.715+0000 7f92947e4b80 1 --2- 172.21.15.46:0/2050971103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d05166160 0x558d051ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.715+0000 7f92947e4b80 1 --2- 172.21.15.46:0/2050971103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 0x558d051ffa30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.715+0000 7f92947e4b80 1 -- 172.21.15.46:0/2050971103 >> 172.21.15.46:0/2050971103 conn(0x558d050dc6a0 msgr2=0x558d051f4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.715+0000 7f92947e4b80 1 -- 172.21.15.46:0/2050971103 shutdown_connections 2024-05-09T04:54:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.715+0000 7f92947e4b80 1 -- 172.21.15.46:0/2050971103 wait complete. 2024-05-09T04:54:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.715+0000 7f92947e4b80 1 -- start start 2024-05-09T04:54:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.716+0000 7f92947e4b80 1 Processor -- start 2024-05-09T04:54:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.716+0000 7f92947e4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 0x558d05208d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.716+0000 7f92947e4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d05166160 0x558d05209250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.716+0000 7f92947e4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d05203110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.716+0000 7f92947e4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558d04f97ba0 con 0x558d051f96a0 2024-05-09T04:54:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.716+0000 7f92947e4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558d050d6150 con 0x558d05166160 2024-05-09T04:54:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.716+0000 7f92947e4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558d051fa360 con 0x558d0507c4a0 2024-05-09T04:54:20.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.716+0000 7f927f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 0x558d05208d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.716+0000 7f927f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 0x558d05208d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:54292/0 (socket says 172.21.15.46:54292) 2024-05-09T04:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.716+0000 7f927f7fe700 1 -- 172.21.15.46:0/3926033289 learned_addr learned my addr 172.21.15.46:0/3926033289 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.716+0000 7f92947e4b80 10 client.0 _ll_register_callbacks cb 0x558d04f8d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.716+0000 7f927ffff700 1 --2- 172.21.15.46:0/3926033289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d05203110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.716+0000 7f927effd700 1 --2- 172.21.15.46:0/3926033289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d05166160 0x558d05209250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.717+0000 7f927f7fe700 1 -- 172.21.15.46:0/3926033289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d05166160 msgr2=0x558d05209250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.717+0000 7f927f7fe700 1 --2- 172.21.15.46:0/3926033289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d05166160 0x558d05209250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.717+0000 7f927f7fe700 1 -- 172.21.15.46:0/3926033289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 msgr2=0x558d05203110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.717+0000 7f927f7fe700 1 --2- 172.21.15.46:0/3926033289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d05203110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.717+0000 7f927f7fe700 1 -- 172.21.15.46:0/3926033289 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d04f97490 con 0x558d0507c4a0 2024-05-09T04:54:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.717+0000 7f927effd700 1 --2- 172.21.15.46:0/3926033289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d05166160 0x558d05209250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:54:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.717+0000 7f927f7fe700 1 --2- 172.21.15.46:0/3926033289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 0x558d05208d10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9270011790 tx=0x7f92700047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5400763c04a43e1b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.717+0000 7f925effd700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:54:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.717+0000 7f925effd700 1 -- 172.21.15.46:0/3926033289 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9270004a30 con 0x558d0507c4a0 2024-05-09T04:54:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.717+0000 7f925effd700 1 -- 172.21.15.46:0/3926033289 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f9270004bd0 con 0x558d0507c4a0 2024-05-09T04:54:20.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.717+0000 7f92947e4b80 1 -- 172.21.15.46:0/3926033289 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92740b1e50 con 0x558d0507c4a0 2024-05-09T04:54:20.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.721+0000 7f925effd700 1 -- 172.21.15.46:0/3926033289 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9270004dd0 con 0x558d0507c4a0 2024-05-09T04:54:20.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.721+0000 7f92947e4b80 10 client.25726 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:20.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.721+0000 7f92947e4b80 1 -- 172.21.15.46:0/3926033289 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d05210970 con 0x558d0507c4a0 2024-05-09T04:54:20.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.721+0000 7f925effd700 1 -- 172.21.15.46:0/3926033289 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9270023070 con 0x558d0507c4a0 2024-05-09T04:54:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.722+0000 7f925effd700 1 -- 172.21.15.46:0/3926033289 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f927001f8a0 con 0x558d0507c4a0 2024-05-09T04:54:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.722+0000 7f92947e4b80 10 client.25726 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.722+0000 7f92947e4b80 10 client.25726 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:54:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.722+0000 7f92947e4b80 20 client.25726 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:20.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.722+0000 7f92947e4b80 1 --2- 172.21.15.46:0/3926033289 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x558d05212fb0 0x558d052153d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.722+0000 7f92947e4b80 4 client.25726 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.722+0000 7f92947e4b80 1 -- 172.21.15.46:0/3926033289 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f92740b1e50 con 0x558d05212fb0 2024-05-09T04:54:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.722+0000 7f927effd700 1 --2- 172.21.15.46:0/3926033289 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x558d05212fb0 0x558d052153d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.722+0000 7f925effd700 10 client.25726 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:54:20.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.722+0000 7f927effd700 1 --2- 172.21.15.46:0/3926033289 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x558d05212fb0 0x558d052153d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=88c2c652b3cbe32c server_cookie=5bbce5f12e467102 in_seq=0 out_seq=0 2024-05-09T04:54:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.723+0000 7f925effd700 1 -- 172.21.15.46:0/3926033289 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f92740b1e50 con 0x558d05212fb0 2024-05-09T04:54:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.723+0000 7f925effd700 10 client.25726 handle_command_reply: tid=0 2024-05-09T04:54:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.723+0000 7f92947e4b80 1 client.25726 shutdown 2024-05-09T04:54:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.723+0000 7f92947e4b80 1 -- 172.21.15.46:0/3926033289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 msgr2=0x558d05208d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.723+0000 7f92947e4b80 1 --2- 172.21.15.46:0/3926033289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 0x558d05208d10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9270011790 tx=0x7f92700047c0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.723+0000 7f92947e4b80 1 -- 172.21.15.46:0/3926033289 shutdown_connections 2024-05-09T04:54:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.723+0000 7f92947e4b80 1 --2- 172.21.15.46:0/3926033289 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d05203110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.723+0000 7f92947e4b80 1 --2- 172.21.15.46:0/3926033289 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x558d05212fb0 0x558d052153d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.723+0000 7f92947e4b80 1 --2- 172.21.15.46:0/3926033289 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 0x558d05208d10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.723+0000 7f92947e4b80 1 --2- 172.21.15.46:0/3926033289 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d05166160 0x558d05209250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.723+0000 7f925effd700 0 client.25726 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:54:20.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.723+0000 7f92947e4b80 1 -- 172.21.15.46:0/3926033289 >> 172.21.15.46:0/3926033289 conn(0x558d050dc6a0 msgr2=0x558d051f41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.723+0000 7f92947e4b80 1 -- 172.21.15.46:0/3926033289 shutdown_connections 2024-05-09T04:54:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.723+0000 7f92947e4b80 1 -- 172.21.15.46:0/3926033289 wait complete. 2024-05-09T04:54:20.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.724+0000 7f92947e4b80 20 client.25726 trim_cache size 0 max 16384 2024-05-09T04:54:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.739+0000 7f92947e4b80 1 Processor -- start 2024-05-09T04:54:20.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.739+0000 7f92947e4b80 1 -- start start 2024-05-09T04:54:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.739+0000 7f92947e4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 0x558d051fd720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.739+0000 7f92947e4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d05166160 0x558d052030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.739+0000 7f92947e4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d051fbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.739+0000 7f92947e4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f92801b1720 con 0x558d051f96a0 2024-05-09T04:54:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.739+0000 7f92947e4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f92801aefb0 con 0x558d05166160 2024-05-09T04:54:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.739+0000 7f92947e4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f92801ace00 con 0x558d0507c4a0 2024-05-09T04:54:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.739+0000 7f927ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d051fbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.739+0000 7f927ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d051fbe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44658/0 (socket says 172.21.15.46:44658) 2024-05-09T04:54:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.739+0000 7f927ffff700 1 -- 172.21.15.46:0/2659504793 learned_addr learned my addr 172.21.15.46:0/2659504793 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.739+0000 7f927f7fe700 1 --2- 172.21.15.46:0/2659504793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 0x558d051fd720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.739+0000 7f927effd700 1 --2- 172.21.15.46:0/2659504793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d05166160 0x558d052030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.740+0000 7f927ffff700 1 -- 172.21.15.46:0/2659504793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 msgr2=0x558d051fd720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.740+0000 7f927ffff700 1 --2- 172.21.15.46:0/2659504793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 0x558d051fd720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.740+0000 7f927ffff700 1 -- 172.21.15.46:0/2659504793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d05166160 msgr2=0x558d052030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.740+0000 7f927ffff700 1 --2- 172.21.15.46:0/2659504793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d05166160 0x558d052030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.740+0000 7f927ffff700 1 -- 172.21.15.46:0/2659504793 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d051f6150 con 0x558d051f96a0 2024-05-09T04:54:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.740+0000 7f927effd700 1 --2- 172.21.15.46:0/2659504793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d05166160 0x558d052030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:54:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.740+0000 7f927ffff700 1 --2- 172.21.15.46:0/2659504793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d051fbe00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f927800a830 tx=0x7f92780078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4519fcddc5e9be01 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.740+0000 7f92657fa700 1 -- 172.21.15.46:0/2659504793 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278003c00 con 0x558d051f96a0 2024-05-09T04:54:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.740+0000 7f92657fa700 1 -- 172.21.15.46:0/2659504793 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f927800a2e0 con 0x558d051f96a0 2024-05-09T04:54:20.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.740+0000 7f92657fa700 1 -- 172.21.15.46:0/2659504793 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92780178a0 con 0x558d051f96a0 2024-05-09T04:54:20.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.741+0000 7f92947e4b80 1 -- 172.21.15.46:0/2659504793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 msgr2=0x558d051fbe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:20.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.741+0000 7f92947e4b80 1 --2- 172.21.15.46:0/2659504793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d051fbe00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f927800a830 tx=0x7f92780078d0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.741+0000 7f92947e4b80 1 -- 172.21.15.46:0/2659504793 shutdown_connections 2024-05-09T04:54:20.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.741+0000 7f92947e4b80 1 --2- 172.21.15.46:0/2659504793 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d051fbe00 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.741+0000 7f92947e4b80 1 --2- 172.21.15.46:0/2659504793 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d0507c4a0 0x558d051fd720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.741+0000 7f92947e4b80 1 --2- 172.21.15.46:0/2659504793 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d05166160 0x558d052030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.741+0000 7f92947e4b80 1 -- 172.21.15.46:0/2659504793 >> 172.21.15.46:0/2659504793 conn(0x558d050dc6a0 msgr2=0x558d052029c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:20.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.741+0000 7f92947e4b80 1 -- 172.21.15.46:0/2659504793 shutdown_connections 2024-05-09T04:54:20.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.741+0000 7f92947e4b80 1 -- 172.21.15.46:0/2659504793 wait complete. 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.741+0000 7f92947e4b80 1 -- start start 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f92947e4b80 1 Processor -- start 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f92947e4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d0507c4a0 0x558d0520a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f92947e4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d05166160 0x558d0520a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f92947e4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d0527b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f92947e4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f92801ad470 con 0x558d051f96a0 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f92947e4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f92801ae8e0 con 0x558d0507c4a0 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f92947e4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f92801adb40 con 0x558d05166160 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f92947e4b80 10 client.0 _ll_register_callbacks cb 0x558d04f8d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f927ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d0527b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f927ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d0527b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44666/0 (socket says 172.21.15.46:44666) 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f927f7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d0507c4a0 0x558d0520a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f927ffff700 1 -- 172.21.15.46:0/3660950438 learned_addr learned my addr 172.21.15.46:0/3660950438 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f927effd700 1 --2- 172.21.15.46:0/3660950438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d05166160 0x558d0520a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f927ffff700 1 -- 172.21.15.46:0/3660950438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d05166160 msgr2=0x558d0520a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:20.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f927ffff700 1 --2- 172.21.15.46:0/3660950438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d05166160 0x558d0520a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f927ffff700 1 -- 172.21.15.46:0/3660950438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d0507c4a0 msgr2=0x558d0520a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f927ffff700 1 --2- 172.21.15.46:0/3660950438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d0507c4a0 0x558d0520a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f927ffff700 1 -- 172.21.15.46:0/3660950438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d0527bc70 con 0x558d051f96a0 2024-05-09T04:54:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.742+0000 7f927f7fe700 1 --2- 172.21.15.46:0/3660950438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d0507c4a0 0x558d0520a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.743+0000 7f927ffff700 1 --2- 172.21.15.46:0/3660950438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d0527b730 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9278000f80 tx=0x7f9278004250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c02443ea1950a21 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.743+0000 7f925effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.743+0000 7f925effd700 1 -- 172.21.15.46:0/3660950438 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9278004cd0 con 0x558d051f96a0 2024-05-09T04:54:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.743+0000 7f925effd700 1 -- 172.21.15.46:0/3660950438 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f92780174c0 con 0x558d051f96a0 2024-05-09T04:54:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.743+0000 7f92947e4b80 1 -- 172.21.15.46:0/3660950438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f927800a2e0 con 0x558d051f96a0 2024-05-09T04:54:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.743+0000 7f925effd700 1 -- 172.21.15.46:0/3660950438 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f92780176c0 con 0x558d051f96a0 2024-05-09T04:54:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.743+0000 7f92947e4b80 10 client.16185 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.743+0000 7f92947e4b80 1 -- 172.21.15.46:0/3660950438 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558d0527cd90 con 0x558d051f96a0 2024-05-09T04:54:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.743+0000 7f925effd700 1 -- 172.21.15.46:0/3660950438 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f927800a2e0 con 0x558d051f96a0 2024-05-09T04:54:20.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.743+0000 7f925effd700 1 -- 172.21.15.46:0/3660950438 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f927801f850 con 0x558d051f96a0 2024-05-09T04:54:20.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.743+0000 7f92947e4b80 10 client.16185 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:20.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.743+0000 7f92947e4b80 10 client.16185 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:54:20.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.744+0000 7f92947e4b80 20 client.16185 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:20.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.744+0000 7f92947e4b80 1 --2- 172.21.15.46:0/3660950438 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x558d0527d3b0 0x558d0527f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:20.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.744+0000 7f92947e4b80 4 client.16185 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:54:20.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.744+0000 7f92947e4b80 1 -- 172.21.15.46:0/3660950438 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d051299d0 con 0x558d0527d3b0 2024-05-09T04:54:20.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.744+0000 7f927f7fe700 1 --2- 172.21.15.46:0/3660950438 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x558d0527d3b0 0x558d0527f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:20.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.744+0000 7f927f7fe700 1 --2- 172.21.15.46:0/3660950438 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x558d0527d3b0 0x558d0527f7d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=24e5c5e63eb08fa1 server_cookie=e40055c99402a4c1 in_seq=0 out_seq=0 2024-05-09T04:54:20.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.744+0000 7f925effd700 10 client.16185 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:54:20.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.744+0000 7f925effd700 1 -- 172.21.15.46:0/3660950438 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d051299d0 con 0x558d0527d3b0 2024-05-09T04:54:20.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.744+0000 7f925effd700 10 client.16185 handle_command_reply: tid=0 2024-05-09T04:54:20.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.744+0000 7f92947e4b80 1 client.16185 shutdown 2024-05-09T04:54:20.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.745+0000 7f92947e4b80 1 -- 172.21.15.46:0/3660950438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 msgr2=0x558d0527b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.745+0000 7f92947e4b80 1 --2- 172.21.15.46:0/3660950438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d0527b730 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f9278000f80 tx=0x7f9278004250 comp rx=0 tx=0).stop 2024-05-09T04:54:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.745+0000 7f92947e4b80 1 -- 172.21.15.46:0/3660950438 shutdown_connections 2024-05-09T04:54:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.745+0000 7f92947e4b80 1 --2- 172.21.15.46:0/3660950438 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558d051f96a0 0x558d0527b730 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.745+0000 7f92947e4b80 1 --2- 172.21.15.46:0/3660950438 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558d0507c4a0 0x558d0520a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.745+0000 7f92947e4b80 1 --2- 172.21.15.46:0/3660950438 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x558d0527d3b0 0x558d0527f7d0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.745+0000 7f92947e4b80 1 --2- 172.21.15.46:0/3660950438 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558d05166160 0x558d0520a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.745+0000 7f925effd700 0 client.16185 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:54:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.745+0000 7f92947e4b80 1 -- 172.21.15.46:0/3660950438 >> 172.21.15.46:0/3660950438 conn(0x558d050dc6a0 msgr2=0x558d051fe100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.745+0000 7f92947e4b80 1 -- 172.21.15.46:0/3660950438 shutdown_connections 2024-05-09T04:54:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.745+0000 7f92947e4b80 1 -- 172.21.15.46:0/3660950438 wait complete. 2024-05-09T04:54:20.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.745+0000 7f92947e4b80 20 client.16185 trim_cache size 0 max 16384 2024-05-09T04:54:20.751 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:54:20.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.745+0000 7f9285dce700 1 -- 172.21.15.46:0/3013947072 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f92680b29f0 msgr2=0x7f92680b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:20.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.746+0000 7f9285dce700 1 --2- 172.21.15.46:0/3013947072 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f92680b29f0 0x7f92680b4f00 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f9270005950 tx=0x7f927000b410 comp rx=0 tx=0).stop 2024-05-09T04:54:20.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.746+0000 7f9285dce700 1 -- 172.21.15.46:0/3013947072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9280107df0 msgr2=0x7f928019b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:20.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.746+0000 7f9285dce700 1 --2- 172.21.15.46:0/3013947072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9280107df0 0x7f928019b470 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f9274006010 tx=0x7f92740047b0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.746+0000 7f9285dce700 1 -- 172.21.15.46:0/3013947072 shutdown_connections 2024-05-09T04:54:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.746+0000 7f9285dce700 1 --2- 172.21.15.46:0/3013947072 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f92680b29f0 0x7f92680b4f00 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.746+0000 7f9285dce700 1 --2- 172.21.15.46:0/3013947072 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9280108740 0x7f928019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.746+0000 7f9285dce700 1 --2- 172.21.15.46:0/3013947072 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f92801034c0 0x7f928019aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.746+0000 7f9285dce700 1 --2- 172.21.15.46:0/3013947072 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9280107df0 0x7f928019b470 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.746+0000 7f9285dce700 1 -- 172.21.15.46:0/3013947072 >> 172.21.15.46:0/3013947072 conn(0x7f92800fedf0 msgr2=0x7f9280075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.746+0000 7f9285dce700 1 -- 172.21.15.46:0/3013947072 shutdown_connections 2024-05-09T04:54:20.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:20.746+0000 7f9285dce700 1 -- 172.21.15.46:0/3013947072 wait complete. 2024-05-09T04:54:21.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:20 smithi110 ceph-mon[34904]: pgmap v100: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.2 KiB/s wr, 222 op/s 2024-05-09T04:54:21.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:20 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:54:21.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:20 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:54:21.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:20 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:54:21.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:20 smithi046 ceph-mon[30136]: pgmap v100: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.2 KiB/s wr, 222 op/s 2024-05-09T04:54:21.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:20 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:54:21.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:20 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:54:21.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:20 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:54:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:20 smithi072 ceph-mon[34088]: pgmap v100: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 4.2 KiB/s wr, 222 op/s 2024-05-09T04:54:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:20 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:54:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:20 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:54:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:20 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:54:21.553 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:54:21.821 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:22.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:22 smithi046 ceph-mon[30136]: pgmap v101: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.3 KiB/s wr, 224 op/s 2024-05-09T04:54:23.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:22 smithi110 ceph-mon[34904]: pgmap v101: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.3 KiB/s wr, 224 op/s 2024-05-09T04:54:23.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:22 smithi072 ceph-mon[34088]: pgmap v101: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.3 KiB/s wr, 224 op/s 2024-05-09T04:54:23.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.399+0000 7f15ee4d5700 1 -- 172.21.15.46:0/2977100252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e81054a0 msgr2=0x7f15e81058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:23.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.399+0000 7f15ee4d5700 1 --2- 172.21.15.46:0/2977100252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e81054a0 0x7f15e81058b0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f15d8009990 tx=0x7f15d8009ca0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.399+0000 7f15ee4d5700 1 -- 172.21.15.46:0/2977100252 shutdown_connections 2024-05-09T04:54:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.399+0000 7f15ee4d5700 1 --2- 172.21.15.46:0/2977100252 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e810e910 0x7f15e8110e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.399+0000 7f15ee4d5700 1 --2- 172.21.15.46:0/2977100252 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e81054a0 0x7f15e81058b0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.399+0000 7f15ee4d5700 1 --2- 172.21.15.46:0/2977100252 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15e8105df0 0x7f15e810e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.399+0000 7f15ee4d5700 1 -- 172.21.15.46:0/2977100252 >> 172.21.15.46:0/2977100252 conn(0x7f15e8078e50 msgr2=0x7f15e8079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.399+0000 7f15ee4d5700 1 -- 172.21.15.46:0/2977100252 shutdown_connections 2024-05-09T04:54:23.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.400+0000 7f15ee4d5700 1 -- 172.21.15.46:0/2977100252 wait complete. 2024-05-09T04:54:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.401+0000 7f15ee4d5700 1 Processor -- start 2024-05-09T04:54:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.401+0000 7f15ee4d5700 1 -- start start 2024-05-09T04:54:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.401+0000 7f15ee4d5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15e81054a0 0x7f15e819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.401+0000 7f15ee4d5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e8105df0 0x7f15e819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.401+0000 7f15ee4d5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e810e910 0x7f15e819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.401+0000 7f15ee4d5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81144c0 con 0x7f15e810e910 2024-05-09T04:54:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.401+0000 7f15ee4d5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8114340 con 0x7f15e81054a0 2024-05-09T04:54:23.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.401+0000 7f15ee4d5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15e8114640 con 0x7f15e8105df0 2024-05-09T04:54:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.401+0000 7f15eca72700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e810e910 0x7f15e819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.401+0000 7f15eca72700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e810e910 0x7f15e819f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44680/0 (socket says 172.21.15.46:44680) 2024-05-09T04:54:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.401+0000 7f15eca72700 1 -- 172.21.15.46:0/3833372055 learned_addr learned my addr 172.21.15.46:0/3833372055 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.401+0000 7f15e7fff700 1 --2- 172.21.15.46:0/3833372055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15e81054a0 0x7f15e819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.401+0000 7f15dffff700 1 --2- 172.21.15.46:0/3833372055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e8105df0 0x7f15e819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.402+0000 7f15eca72700 1 -- 172.21.15.46:0/3833372055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e8105df0 msgr2=0x7f15e819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.402+0000 7f15eca72700 1 --2- 172.21.15.46:0/3833372055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e8105df0 0x7f15e819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.402+0000 7f15eca72700 1 -- 172.21.15.46:0/3833372055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15e81054a0 msgr2=0x7f15e819aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:23.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.402+0000 7f15eca72700 1 --2- 172.21.15.46:0/3833372055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15e81054a0 0x7f15e819aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.402+0000 7f15eca72700 1 -- 172.21.15.46:0/3833372055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15e819fcd0 con 0x7f15e810e910 2024-05-09T04:54:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.402+0000 7f15eca72700 1 --2- 172.21.15.46:0/3833372055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e810e910 0x7f15e819f6e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f15e0007ee0 tx=0x7f15e000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.402+0000 7f15e5ffb700 1 -- 172.21.15.46:0/3833372055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0004bb0 con 0x7f15e810e910 2024-05-09T04:54:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.402+0000 7f15ee4d5700 1 -- 172.21.15.46:0/3833372055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15e819ffc0 con 0x7f15e810e910 2024-05-09T04:54:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.402+0000 7f15e5ffb700 1 -- 172.21.15.46:0/3833372055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e0004d50 con 0x7f15e810e910 2024-05-09T04:54:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.402+0000 7f15e5ffb700 1 -- 172.21.15.46:0/3833372055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e0005920 con 0x7f15e810e910 2024-05-09T04:54:23.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.402+0000 7f15ee4d5700 1 -- 172.21.15.46:0/3833372055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15e81a0500 con 0x7f15e810e910 2024-05-09T04:54:23.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.405+0000 7f15e5ffb700 1 -- 172.21.15.46:0/3833372055 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f15e001a020 con 0x7f15e810e910 2024-05-09T04:54:23.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.405+0000 7f15e5ffb700 1 --2- 172.21.15.46:0/3833372055 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f15d40b29f0 0x7f15d40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.406+0000 7f15e5ffb700 1 -- 172.21.15.46:0/3833372055 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f15e00e1c60 con 0x7f15e810e910 2024-05-09T04:54:23.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.406+0000 7f15e7fff700 1 --2- 172.21.15.46:0/3833372055 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f15d40b29f0 0x7f15d40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.406+0000 7f15fceebb80 1 Processor -- start 2024-05-09T04:54:23.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.406+0000 7f15fceebb80 1 -- start start 2024-05-09T04:54:23.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.406+0000 7f15e7fff700 1 --2- 172.21.15.46:0/3833372055 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f15d40b29f0 0x7f15d40b4f00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f15d8006010 tx=0x7f15d8005960 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:23.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.406+0000 7f15fceebb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134ebbd00 0x557134fc4240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.406+0000 7f15fceebb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134e414a0 0x557134fc4780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.406+0000 7f15fceebb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 0x557134fbe810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.406+0000 7f15fceebb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557134ee10a0 con 0x557134ebbd00 2024-05-09T04:54:23.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.406+0000 7f15fceebb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557134e869a0 con 0x557134e414a0 2024-05-09T04:54:23.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.406+0000 7f15fceebb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557134edd060 con 0x557134fbe400 2024-05-09T04:54:23.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.406+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134ebbd00 0x557134fc4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.406+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134ebbd00 0x557134fc4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44684/0 (socket says 172.21.15.46:44684) 2024-05-09T04:54:23.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.406+0000 7f15e7fff700 1 -- 172.21.15.46:0/3717280507 learned_addr learned my addr 172.21.15.46:0/3717280507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:23.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.407+0000 7f15dffff700 1 --2- 172.21.15.46:0/3717280507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134e414a0 0x557134fc4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.407+0000 7f15eca72700 1 --2- 172.21.15.46:0/3717280507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 0x557134fbe810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.407+0000 7f15e7fff700 1 -- 172.21.15.46:0/3717280507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 msgr2=0x557134fbe810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:23.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.407+0000 7f15e7fff700 1 --2- 172.21.15.46:0/3717280507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 0x557134fbe810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.407+0000 7f15e7fff700 1 -- 172.21.15.46:0/3717280507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134e414a0 msgr2=0x557134fc4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:23.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.407+0000 7f15e7fff700 1 --2- 172.21.15.46:0/3717280507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134e414a0 0x557134fc4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.407+0000 7f15e7fff700 1 -- 172.21.15.46:0/3717280507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557134e77500 con 0x557134ebbd00 2024-05-09T04:54:23.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.407+0000 7f15e7fff700 1 --2- 172.21.15.46:0/3717280507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134ebbd00 0x557134fc4240 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f15e806d130 tx=0x7f15d801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff72db291d5d6ea9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:23.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.407+0000 7f15de7fc700 1 -- 172.21.15.46:0/3717280507 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d8005430 con 0x557134ebbd00 2024-05-09T04:54:23.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.407+0000 7f15de7fc700 1 -- 172.21.15.46:0/3717280507 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15d80055d0 con 0x557134ebbd00 2024-05-09T04:54:23.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.407+0000 7f15de7fc700 1 -- 172.21.15.46:0/3717280507 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d800fae0 con 0x557134ebbd00 2024-05-09T04:54:23.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.408+0000 7f15fceebb80 1 -- 172.21.15.46:0/3717280507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134ebbd00 msgr2=0x557134fc4240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:23.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.408+0000 7f15fceebb80 1 --2- 172.21.15.46:0/3717280507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134ebbd00 0x557134fc4240 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f15e806d130 tx=0x7f15d801e000 comp rx=0 tx=0).stop 2024-05-09T04:54:23.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.408+0000 7f15fceebb80 1 -- 172.21.15.46:0/3717280507 shutdown_connections 2024-05-09T04:54:23.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.408+0000 7f15fceebb80 1 --2- 172.21.15.46:0/3717280507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 0x557134fbe810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.408+0000 7f15fceebb80 1 --2- 172.21.15.46:0/3717280507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134ebbd00 0x557134fc4240 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.408+0000 7f15fceebb80 1 --2- 172.21.15.46:0/3717280507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134e414a0 0x557134fc4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.408+0000 7f15fceebb80 1 -- 172.21.15.46:0/3717280507 >> 172.21.15.46:0/3717280507 conn(0x557134e44f70 msgr2=0x557134fb9220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:23.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.408+0000 7f15fceebb80 1 -- 172.21.15.46:0/3717280507 shutdown_connections 2024-05-09T04:54:23.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.408+0000 7f15fceebb80 1 -- 172.21.15.46:0/3717280507 wait complete. 2024-05-09T04:54:23.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.409+0000 7f15fceebb80 1 -- start start 2024-05-09T04:54:23.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.409+0000 7f15fceebb80 1 Processor -- start 2024-05-09T04:54:23.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.409+0000 7f15fceebb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134e414a0 0x557134fc8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.409+0000 7f15fceebb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134ebbd00 0x557134fc8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.409+0000 7f15fceebb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 0x557134fc8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.409+0000 7f15fceebb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557134fbf0a0 con 0x557134e414a0 2024-05-09T04:54:23.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.409+0000 7f15fceebb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557134e7bb20 con 0x557134ebbd00 2024-05-09T04:54:23.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.409+0000 7f15fceebb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557134fbf220 con 0x557134fbe400 2024-05-09T04:54:23.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.409+0000 7f15fceebb80 10 client.0 _ll_register_callbacks cb 0x557134d526c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:23.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.409+0000 7f15dffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134ebbd00 0x557134fc8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.409+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134e414a0 0x557134fc8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.409+0000 7f15dffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134ebbd00 0x557134fc8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:35834/0 (socket says 172.21.15.46:35834) 2024-05-09T04:54:23.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.409+0000 7f15dffff700 1 -- 172.21.15.46:0/1881672361 learned_addr learned my addr 172.21.15.46:0/1881672361 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:23.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.410+0000 7f15eca72700 1 --2- 172.21.15.46:0/1881672361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 0x557134fc8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.410+0000 7f15dffff700 1 -- 172.21.15.46:0/1881672361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 msgr2=0x557134fc8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:23.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.410+0000 7f15dffff700 1 --2- 172.21.15.46:0/1881672361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 0x557134fc8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.410+0000 7f15dffff700 1 -- 172.21.15.46:0/1881672361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134e414a0 msgr2=0x557134fc8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:23.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.410+0000 7f15dffff700 1 --2- 172.21.15.46:0/1881672361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134e414a0 0x557134fc8020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.410+0000 7f15dffff700 1 -- 172.21.15.46:0/1881672361 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557134d5c490 con 0x557134ebbd00 2024-05-09T04:54:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.410+0000 7f15dffff700 1 --2- 172.21.15.46:0/1881672361 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134ebbd00 0x557134fc8560 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f15d0010a50 tx=0x7f15d0010d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82c980286ad5d542 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.410+0000 7f15c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:54:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.410+0000 7f15c77fe700 1 -- 172.21.15.46:0/1881672361 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d0004320 con 0x557134ebbd00 2024-05-09T04:54:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.410+0000 7f15fceebb80 1 -- 172.21.15.46:0/1881672361 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15d80055d0 con 0x557134ebbd00 2024-05-09T04:54:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.410+0000 7f15c77fe700 1 -- 172.21.15.46:0/1881672361 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f15d00044c0 con 0x557134ebbd00 2024-05-09T04:54:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.410+0000 7f15c77fe700 1 -- 172.21.15.46:0/1881672361 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f15d000f890 con 0x557134ebbd00 2024-05-09T04:54:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.410+0000 7f15fceebb80 10 client.25814 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.410+0000 7f15fceebb80 1 -- 172.21.15.46:0/1881672361 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557134fd5a20 con 0x557134ebbd00 2024-05-09T04:54:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.411+0000 7f15c77fe700 1 -- 172.21.15.46:0/1881672361 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15d80055d0 con 0x557134ebbd00 2024-05-09T04:54:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.411+0000 7f15c77fe700 1 -- 172.21.15.46:0/1881672361 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f15d0022420 con 0x557134ebbd00 2024-05-09T04:54:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.411+0000 7f15fceebb80 10 client.25814 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.411+0000 7f15fceebb80 10 client.25814 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:54:23.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.411+0000 7f15fceebb80 20 client.25814 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.411+0000 7f15fceebb80 1 --2- 172.21.15.46:0/1881672361 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x557134fd8060 0x557134fda480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.411+0000 7f15fceebb80 4 client.25814 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.411+0000 7f15fceebb80 1 -- 172.21.15.46:0/1881672361 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557134eeea80 con 0x557134fd8060 2024-05-09T04:54:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.411+0000 7f15e7fff700 1 --2- 172.21.15.46:0/1881672361 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x557134fd8060 0x557134fda480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.412+0000 7f15c77fe700 10 client.25814 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:54:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.412+0000 7f15e7fff700 1 --2- 172.21.15.46:0/1881672361 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x557134fd8060 0x557134fda480 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1f6a10aadb30b44e server_cookie=f21ae71fc00ff9ba in_seq=0 out_seq=0 2024-05-09T04:54:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.412+0000 7f15c77fe700 1 -- 172.21.15.46:0/1881672361 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557134eeea80 con 0x557134fd8060 2024-05-09T04:54:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.412+0000 7f15c77fe700 10 client.25814 handle_command_reply: tid=0 2024-05-09T04:54:23.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.413+0000 7f15fceebb80 1 client.25814 shutdown 2024-05-09T04:54:23.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.413+0000 7f15fceebb80 1 -- 172.21.15.46:0/1881672361 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134ebbd00 msgr2=0x557134fc8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:23.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.413+0000 7f15fceebb80 1 --2- 172.21.15.46:0/1881672361 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134ebbd00 0x557134fc8560 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f15d0010a50 tx=0x7f15d0010d60 comp rx=0 tx=0).stop 2024-05-09T04:54:23.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.413+0000 7f15fceebb80 1 -- 172.21.15.46:0/1881672361 shutdown_connections 2024-05-09T04:54:23.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.413+0000 7f15fceebb80 1 --2- 172.21.15.46:0/1881672361 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x557134fd8060 0x557134fda480 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.413+0000 7f15fceebb80 1 --2- 172.21.15.46:0/1881672361 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 0x557134fc8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.413+0000 7f15fceebb80 1 --2- 172.21.15.46:0/1881672361 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134e414a0 0x557134fc8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.413+0000 7f15fceebb80 1 --2- 172.21.15.46:0/1881672361 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134ebbd00 0x557134fc8560 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.413+0000 7f15c77fe700 0 client.25814 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:54:23.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.413+0000 7f15fceebb80 1 -- 172.21.15.46:0/1881672361 >> 172.21.15.46:0/1881672361 conn(0x557134e44f70 msgr2=0x557134fb2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:23.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.413+0000 7f15fceebb80 1 -- 172.21.15.46:0/1881672361 shutdown_connections 2024-05-09T04:54:23.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.413+0000 7f15fceebb80 1 -- 172.21.15.46:0/1881672361 wait complete. 2024-05-09T04:54:23.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.413+0000 7f15fceebb80 20 client.25814 trim_cache size 0 max 16384 2024-05-09T04:54:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.428+0000 7f15fceebb80 1 Processor -- start 2024-05-09T04:54:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.428+0000 7f15fceebb80 1 -- start start 2024-05-09T04:54:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15fceebb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134e414a0 0x557134fc3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15fceebb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134ebbd00 0x557134fd3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15fceebb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134fbe400 0x557134fdfc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15fceebb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81b16f0 con 0x557134e414a0 2024-05-09T04:54:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15fceebb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81aef80 con 0x557134fbe400 2024-05-09T04:54:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15fceebb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81acdd0 con 0x557134ebbd00 2024-05-09T04:54:23.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134e414a0 0x557134fc3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134e414a0 0x557134fc3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44698/0 (socket says 172.21.15.46:44698) 2024-05-09T04:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15e7fff700 1 -- 172.21.15.46:0/91999608 learned_addr learned my addr 172.21.15.46:0/91999608 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15eca72700 1 --2- 172.21.15.46:0/91999608 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134fbe400 0x557134fdfc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15dffff700 1 --2- 172.21.15.46:0/91999608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134ebbd00 0x557134fd3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15e7fff700 1 -- 172.21.15.46:0/91999608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134ebbd00 msgr2=0x557134fd3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15e7fff700 1 --2- 172.21.15.46:0/91999608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134ebbd00 0x557134fd3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15e7fff700 1 -- 172.21.15.46:0/91999608 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134fbe400 msgr2=0x557134fdfc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15e7fff700 1 --2- 172.21.15.46:0/91999608 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134fbe400 0x557134fdfc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15e7fff700 1 -- 172.21.15.46:0/91999608 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557134fca8c0 con 0x557134e414a0 2024-05-09T04:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.429+0000 7f15e7fff700 1 --2- 172.21.15.46:0/91999608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134e414a0 0x557134fc3820 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f15d800a010 tx=0x7f15d8021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49f4f349b94e4234 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.430+0000 7f15ddffb700 1 -- 172.21.15.46:0/91999608 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d800b7d0 con 0x557134e414a0 2024-05-09T04:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.430+0000 7f15ddffb700 1 -- 172.21.15.46:0/91999608 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15d800be20 con 0x557134e414a0 2024-05-09T04:54:23.431 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.430+0000 7f15ddffb700 1 -- 172.21.15.46:0/91999608 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d803da40 con 0x557134e414a0 2024-05-09T04:54:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.430+0000 7f15fceebb80 1 -- 172.21.15.46:0/91999608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134e414a0 msgr2=0x557134fc3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.430+0000 7f15fceebb80 1 --2- 172.21.15.46:0/91999608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134e414a0 0x557134fc3820 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f15d800a010 tx=0x7f15d8021000 comp rx=0 tx=0).stop 2024-05-09T04:54:23.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.430+0000 7f15fceebb80 1 -- 172.21.15.46:0/91999608 shutdown_connections 2024-05-09T04:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.430+0000 7f15fceebb80 1 --2- 172.21.15.46:0/91999608 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134fbe400 0x557134fdfc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.430+0000 7f15fceebb80 1 --2- 172.21.15.46:0/91999608 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134e414a0 0x557134fc3820 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.430+0000 7f15fceebb80 1 --2- 172.21.15.46:0/91999608 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134ebbd00 0x557134fd3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.430+0000 7f15fceebb80 1 -- 172.21.15.46:0/91999608 >> 172.21.15.46:0/91999608 conn(0x557134e44f70 msgr2=0x557134fc3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.431+0000 7f15fceebb80 1 -- 172.21.15.46:0/91999608 shutdown_connections 2024-05-09T04:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.431+0000 7f15fceebb80 1 -- 172.21.15.46:0/91999608 wait complete. 2024-05-09T04:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.431+0000 7f15fceebb80 1 -- start start 2024-05-09T04:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.431+0000 7f15fceebb80 1 Processor -- start 2024-05-09T04:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.431+0000 7f15fceebb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134e414a0 0x557134fd0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.431+0000 7f15fceebb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134ebbd00 0x557134fdecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.431+0000 7f15fceebb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 0x557134fdf210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.431+0000 7f15fceebb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81ad440 con 0x557134ebbd00 2024-05-09T04:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.431+0000 7f15fceebb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81ae8b0 con 0x557134e414a0 2024-05-09T04:54:23.436 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.431+0000 7f15fceebb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15e81adb10 con 0x557134fbe400 2024-05-09T04:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15fceebb80 10 client.0 _ll_register_callbacks cb 0x557134d526c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15dffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134ebbd00 0x557134fdecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15e7fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134e414a0 0x557134fd0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15dffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134ebbd00 0x557134fdecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:44708/0 (socket says 172.21.15.46:44708) 2024-05-09T04:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15dffff700 1 -- 172.21.15.46:0/4103894572 learned_addr learned my addr 172.21.15.46:0/4103894572 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15eca72700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 0x557134fdf210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15e7fff700 1 -- 172.21.15.46:0/4103894572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 msgr2=0x557134fdf210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15e7fff700 1 --2- 172.21.15.46:0/4103894572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 0x557134fdf210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15e7fff700 1 -- 172.21.15.46:0/4103894572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134ebbd00 msgr2=0x557134fdecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15e7fff700 1 --2- 172.21.15.46:0/4103894572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134ebbd00 0x557134fdecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15e7fff700 1 -- 172.21.15.46:0/4103894572 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557134fd0b20 con 0x557134e414a0 2024-05-09T04:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15dffff700 1 --2- 172.21.15.46:0/4103894572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134ebbd00 0x557134fdecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15eca72700 1 --2- 172.21.15.46:0/4103894572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 0x557134fdf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15e7fff700 1 --2- 172.21.15.46:0/4103894572 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134e414a0 0x557134fd0320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f15d8000c00 tx=0x7f15d800bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e482e9118b1f1017 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:23.437 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15c77fe700 1 -- 172.21.15.46:0/4103894572 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15d804a070 con 0x557134e414a0 2024-05-09T04:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.432+0000 7f15fceebb80 1 -- 172.21.15.46:0/4103894572 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15d800be20 con 0x557134e414a0 2024-05-09T04:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.433+0000 7f15c77fe700 1 -- 172.21.15.46:0/4103894572 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f15d803d1e0 con 0x557134e414a0 2024-05-09T04:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.433+0000 7f15c77fe700 1 -- 172.21.15.46:0/4103894572 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f15d803d3e0 con 0x557134e414a0 2024-05-09T04:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.433+0000 7f15fceebb80 10 client.25820 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.433+0000 7f15fceebb80 1 -- 172.21.15.46:0/4103894572 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5571350422b0 con 0x557134e414a0 2024-05-09T04:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.433+0000 7f15c77fe700 1 -- 172.21.15.46:0/4103894572 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15d800be20 con 0x557134e414a0 2024-05-09T04:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.433+0000 7f15c77fe700 1 -- 172.21.15.46:0/4103894572 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f15d8022070 con 0x557134e414a0 2024-05-09T04:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.433+0000 7f15fceebb80 10 client.25820 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.433+0000 7f15fceebb80 10 client.25820 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.433+0000 7f15fceebb80 20 client.25820 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.433+0000 7f15fceebb80 1 --2- 172.21.15.46:0/4103894572 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5571350424e0 0x557135044900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.433+0000 7f15fceebb80 4 client.25820 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.433+0000 7f15fceebb80 1 -- 172.21.15.46:0/4103894572 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557134ee8b20 con 0x5571350424e0 2024-05-09T04:54:23.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.433+0000 7f15dffff700 1 --2- 172.21.15.46:0/4103894572 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5571350424e0 0x557135044900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15c77fe700 10 client.25820 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15dffff700 1 --2- 172.21.15.46:0/4103894572 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5571350424e0 0x557135044900 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f6219b7458475bc8 server_cookie=620f03b98740acf3 in_seq=0 out_seq=0 2024-05-09T04:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15c77fe700 1 -- 172.21.15.46:0/4103894572 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557134ee8b20 con 0x5571350424e0 2024-05-09T04:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15c77fe700 10 client.25820 handle_command_reply: tid=0 2024-05-09T04:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15fceebb80 1 client.25820 shutdown 2024-05-09T04:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15fceebb80 1 -- 172.21.15.46:0/4103894572 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134e414a0 msgr2=0x557134fd0320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15fceebb80 1 --2- 172.21.15.46:0/4103894572 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134e414a0 0x557134fd0320 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f15d8000c00 tx=0x7f15d800bfd0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15fceebb80 1 -- 172.21.15.46:0/4103894572 shutdown_connections 2024-05-09T04:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15fceebb80 1 --2- 172.21.15.46:0/4103894572 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5571350424e0 0x557135044900 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15fceebb80 1 --2- 172.21.15.46:0/4103894572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557134fbe400 0x557134fdf210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15c77fe700 0 client.25820 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15fceebb80 1 --2- 172.21.15.46:0/4103894572 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557134e414a0 0x557134fd0320 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15fceebb80 1 --2- 172.21.15.46:0/4103894572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557134ebbd00 0x557134fdecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15fceebb80 1 -- 172.21.15.46:0/4103894572 >> 172.21.15.46:0/4103894572 conn(0x557134e44f70 msgr2=0x557134fb2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:23.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15fceebb80 1 -- 172.21.15.46:0/4103894572 shutdown_connections 2024-05-09T04:54:23.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15fceebb80 1 -- 172.21.15.46:0/4103894572 wait complete. 2024-05-09T04:54:23.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.434+0000 7f15fceebb80 20 client.25820 trim_cache size 0 max 16384 2024-05-09T04:54:23.440 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:54:23.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.435+0000 7f15ee4d5700 1 -- 172.21.15.46:0/3833372055 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f15d40b29f0 msgr2=0x7f15d40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:23.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.435+0000 7f15ee4d5700 1 --2- 172.21.15.46:0/3833372055 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f15d40b29f0 0x7f15d40b4f00 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f15d8006010 tx=0x7f15d8005960 comp rx=0 tx=0).stop 2024-05-09T04:54:23.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.435+0000 7f15ee4d5700 1 -- 172.21.15.46:0/3833372055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e810e910 msgr2=0x7f15e819f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:23.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.435+0000 7f15ee4d5700 1 --2- 172.21.15.46:0/3833372055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e810e910 0x7f15e819f6e0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f15e0007ee0 tx=0x7f15e000d3b0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.435+0000 7f15ee4d5700 1 -- 172.21.15.46:0/3833372055 shutdown_connections 2024-05-09T04:54:23.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.435+0000 7f15ee4d5700 1 --2- 172.21.15.46:0/3833372055 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f15d40b29f0 0x7f15d40b4f00 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.435+0000 7f15ee4d5700 1 --2- 172.21.15.46:0/3833372055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15e810e910 0x7f15e819f6e0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.435+0000 7f15ee4d5700 1 --2- 172.21.15.46:0/3833372055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15e81054a0 0x7f15e819aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.435+0000 7f15ee4d5700 1 --2- 172.21.15.46:0/3833372055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15e8105df0 0x7f15e819b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:23.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.435+0000 7f15ee4d5700 1 -- 172.21.15.46:0/3833372055 >> 172.21.15.46:0/3833372055 conn(0x7f15e8078e50 msgr2=0x7f15e8101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:23.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.435+0000 7f15ee4d5700 1 -- 172.21.15.46:0/3833372055 shutdown_connections 2024-05-09T04:54:23.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:23.435+0000 7f15ee4d5700 1 -- 172.21.15.46:0/3833372055 wait complete. 2024-05-09T04:54:24.259 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:54:24.512 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:24.889 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:24 smithi046 ceph-mon[30136]: pgmap v102: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.2 KiB/s wr, 135 op/s 2024-05-09T04:54:25.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:24 smithi110 ceph-mon[34904]: pgmap v102: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.2 KiB/s wr, 135 op/s 2024-05-09T04:54:25.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:24 smithi072 ceph-mon[34088]: pgmap v102: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 3.2 KiB/s wr, 135 op/s 2024-05-09T04:54:26.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.353+0000 7fbed70a0700 1 -- 172.21.15.46:0/345819337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0108760 msgr2=0x7fbed01100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:26.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.353+0000 7fbed70a0700 1 --2- 172.21.15.46:0/345819337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0108760 0x7fbed01100c0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fbec800b0d0 tx=0x7fbec800b3e0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.353+0000 7fbed70a0700 1 -- 172.21.15.46:0/345819337 shutdown_connections 2024-05-09T04:54:26.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.353+0000 7fbed70a0700 1 --2- 172.21.15.46:0/345819337 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0108760 0x7fbed01100c0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.353+0000 7fbed70a0700 1 --2- 172.21.15.46:0/345819337 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed01034e0 0x7fbed01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.353+0000 7fbed70a0700 1 --2- 172.21.15.46:0/345819337 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbed0107e10 0x7fbed0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.353+0000 7fbed70a0700 1 -- 172.21.15.46:0/345819337 >> 172.21.15.46:0/345819337 conn(0x7fbed00fef30 msgr2=0x7fbed0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:26.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.353+0000 7fbed70a0700 1 -- 172.21.15.46:0/345819337 shutdown_connections 2024-05-09T04:54:26.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.353+0000 7fbed70a0700 1 -- 172.21.15.46:0/345819337 wait complete. 2024-05-09T04:54:26.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.354+0000 7fbed70a0700 1 Processor -- start 2024-05-09T04:54:26.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.354+0000 7fbed70a0700 1 -- start start 2024-05-09T04:54:26.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.354+0000 7fbed70a0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbed01034e0 0x7fbed019af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.354+0000 7fbed70a0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0107e10 0x7fbed019b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.354+0000 7fbed70a0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed0108760 0x7fbed019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.354+0000 7fbed70a0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0113390 con 0x7fbed0107e10 2024-05-09T04:54:26.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.354+0000 7fbed70a0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0113210 con 0x7fbed01034e0 2024-05-09T04:54:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.354+0000 7fbed70a0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbed0113510 con 0x7fbed0108760 2024-05-09T04:54:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.355+0000 7fbecffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0107e10 0x7fbed019b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.355+0000 7fbecffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0107e10 0x7fbed019b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48044/0 (socket says 172.21.15.46:48044) 2024-05-09T04:54:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.355+0000 7fbecffff700 1 -- 172.21.15.46:0/522910631 learned_addr learned my addr 172.21.15.46:0/522910631 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.355+0000 7fbed4e3c700 1 --2- 172.21.15.46:0/522910631 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbed01034e0 0x7fbed019af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.355+0000 7fbed563d700 1 --2- 172.21.15.46:0/522910631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed0108760 0x7fbed019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.355+0000 7fbecffff700 1 -- 172.21.15.46:0/522910631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed0108760 msgr2=0x7fbed019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.355+0000 7fbecffff700 1 --2- 172.21.15.46:0/522910631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed0108760 0x7fbed019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.355+0000 7fbecffff700 1 -- 172.21.15.46:0/522910631 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbed01034e0 msgr2=0x7fbed019af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.355+0000 7fbecffff700 1 --2- 172.21.15.46:0/522910631 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbed01034e0 0x7fbed019af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.355+0000 7fbecffff700 1 -- 172.21.15.46:0/522910631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbed019fde0 con 0x7fbed0107e10 2024-05-09T04:54:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.355+0000 7fbecffff700 1 --2- 172.21.15.46:0/522910631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0107e10 0x7fbed019b4b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00d950 tx=0x7fbebc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.355+0000 7fbecdffb700 1 -- 172.21.15.46:0/522910631 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc016070 con 0x7fbed0107e10 2024-05-09T04:54:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.355+0000 7fbecdffb700 1 -- 172.21.15.46:0/522910631 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbebc004830 con 0x7fbed0107e10 2024-05-09T04:54:26.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.356+0000 7fbed70a0700 1 -- 172.21.15.46:0/522910631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbed01a00d0 con 0x7fbed0107e10 2024-05-09T04:54:26.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.356+0000 7fbecdffb700 1 -- 172.21.15.46:0/522910631 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc005440 con 0x7fbed0107e10 2024-05-09T04:54:26.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.356+0000 7fbed70a0700 1 -- 172.21.15.46:0/522910631 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbed01a79f0 con 0x7fbed0107e10 2024-05-09T04:54:26.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.358+0000 7fbecdffb700 1 -- 172.21.15.46:0/522910631 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbebc004220 con 0x7fbed0107e10 2024-05-09T04:54:26.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.359+0000 7fbecdffb700 1 --2- 172.21.15.46:0/522910631 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbec00b2a70 0x7fbec00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.359+0000 7fbed4e3c700 1 --2- 172.21.15.46:0/522910631 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbec00b2a70 0x7fbec00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.359+0000 7fbecdffb700 1 -- 172.21.15.46:0/522910631 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fbebc0e2300 con 0x7fbed0107e10 2024-05-09T04:54:26.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.360+0000 7fbed4e3c700 1 --2- 172.21.15.46:0/522910631 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbec00b2a70 0x7fbec00b4f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009c60 tx=0x7fbec4009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:26.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.360+0000 7fbee5ab6b80 1 Processor -- start 2024-05-09T04:54:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.360+0000 7fbee5ab6b80 1 -- start start 2024-05-09T04:54:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.360+0000 7fbee5ab6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031f5ed00 0x558032067240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.360+0000 7fbee5ab6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee44a0 0x558032067780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.360+0000 7fbee5ab6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558032061400 0x558032061810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.360+0000 7fbee5ab6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558031f840a0 con 0x558032061400 2024-05-09T04:54:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.360+0000 7fbee5ab6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558031f299a0 con 0x558031f5ed00 2024-05-09T04:54:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.360+0000 7fbee5ab6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558031f80060 con 0x558031ee44a0 2024-05-09T04:54:26.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbed4e3c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031f5ed00 0x558032067240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbed563d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558032061400 0x558032061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbed563d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558032061400 0x558032061810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48060/0 (socket says 172.21.15.46:48060) 2024-05-09T04:54:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbed4e3c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031f5ed00 0x558032067240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:44568/0 (socket says 172.21.15.46:44568) 2024-05-09T04:54:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbed563d700 1 -- 172.21.15.46:0/552761541 learned_addr learned my addr 172.21.15.46:0/552761541 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbecffff700 1 --2- 172.21.15.46:0/552761541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee44a0 0x558032067780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbed563d700 1 -- 172.21.15.46:0/552761541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee44a0 msgr2=0x558032067780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbed563d700 1 --2- 172.21.15.46:0/552761541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee44a0 0x558032067780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbed563d700 1 -- 172.21.15.46:0/552761541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031f5ed00 msgr2=0x558032067240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbed563d700 1 --2- 172.21.15.46:0/552761541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031f5ed00 0x558032067240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbed563d700 1 -- 172.21.15.46:0/552761541 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558031f1a500 con 0x558032061400 2024-05-09T04:54:26.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbed4e3c700 1 --2- 172.21.15.46:0/552761541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031f5ed00 0x558032067240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:26.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbed563d700 1 --2- 172.21.15.46:0/552761541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558032061400 0x558032061810 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009bb0 tx=0x7fbec8009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b51a8152e9efafc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbebaffd700 1 -- 172.21.15.46:0/552761541 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec8020030 con 0x558032061400 2024-05-09T04:54:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbebaffd700 1 -- 172.21.15.46:0/552761541 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbec80092e0 con 0x558032061400 2024-05-09T04:54:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.361+0000 7fbebaffd700 1 -- 172.21.15.46:0/552761541 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec80079e0 con 0x558032061400 2024-05-09T04:54:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.362+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/552761541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558032061400 msgr2=0x558032061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.362+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/552761541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558032061400 0x558032061810 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fbec8009bb0 tx=0x7fbec8009be0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.362+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/552761541 shutdown_connections 2024-05-09T04:54:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.362+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/552761541 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558032061400 0x558032061810 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.362+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/552761541 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031f5ed00 0x558032067240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.362+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/552761541 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee44a0 0x558032067780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.362+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/552761541 >> 172.21.15.46:0/552761541 conn(0x558031ee7f70 msgr2=0x55803205c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:26.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.362+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/552761541 shutdown_connections 2024-05-09T04:54:26.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.362+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/552761541 wait complete. 2024-05-09T04:54:26.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.363+0000 7fbee5ab6b80 1 -- start start 2024-05-09T04:54:26.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.363+0000 7fbee5ab6b80 1 Processor -- start 2024-05-09T04:54:26.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.363+0000 7fbee5ab6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee44a0 0x55803206b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.363+0000 7fbee5ab6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031f5ed00 0x55803206b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.363+0000 7fbee5ab6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558032061400 0x55803206bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.363+0000 7fbee5ab6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558032062100 con 0x558032061400 2024-05-09T04:54:26.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.363+0000 7fbee5ab6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558031f1eb20 con 0x558031f5ed00 2024-05-09T04:54:26.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.363+0000 7fbee5ab6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558032077200 con 0x558031ee44a0 2024-05-09T04:54:26.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbee5ab6b80 10 client.0 _ll_register_callbacks cb 0x558031df56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:26.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbed563d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558032061400 0x55803206bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbed4e3c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee44a0 0x55803206b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbed563d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558032061400 0x55803206bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48062/0 (socket says 172.21.15.46:48062) 2024-05-09T04:54:26.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbed563d700 1 -- 172.21.15.46:0/1476442667 learned_addr learned my addr 172.21.15.46:0/1476442667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:26.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbed4e3c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee44a0 0x55803206b0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:38530/0 (socket says 172.21.15.46:38530) 2024-05-09T04:54:26.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbecffff700 1 --2- 172.21.15.46:0/1476442667 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031f5ed00 0x55803206b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbed563d700 1 -- 172.21.15.46:0/1476442667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee44a0 msgr2=0x55803206b0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:26.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbed563d700 1 --2- 172.21.15.46:0/1476442667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee44a0 0x55803206b0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbed563d700 1 -- 172.21.15.46:0/1476442667 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031f5ed00 msgr2=0x55803206b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:26.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbed563d700 1 --2- 172.21.15.46:0/1476442667 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031f5ed00 0x55803206b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbed563d700 1 -- 172.21.15.46:0/1476442667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558031dff490 con 0x558032061400 2024-05-09T04:54:26.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbed563d700 1 --2- 172.21.15.46:0/1476442667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558032061400 0x55803206bb30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbec8004f50 tx=0x7fbec800cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9f15ef1435ed8d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:26.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbeaffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:26.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbeaffff700 1 -- 172.21.15.46:0/1476442667 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec802b070 con 0x558032061400 2024-05-09T04:54:26.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbeaffff700 1 -- 172.21.15.46:0/1476442667 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fbec8007660 con 0x558032061400 2024-05-09T04:54:26.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.364+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/1476442667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbec80092e0 con 0x558032061400 2024-05-09T04:54:26.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.365+0000 7fbeaffff700 1 -- 172.21.15.46:0/1476442667 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbec8037050 con 0x558032061400 2024-05-09T04:54:26.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.365+0000 7fbee5ab6b80 10 client.16215 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:26.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.365+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/1476442667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558032078ba0 con 0x558032061400 2024-05-09T04:54:26.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.365+0000 7fbeaffff700 1 -- 172.21.15.46:0/1476442667 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbec80092e0 con 0x558032061400 2024-05-09T04:54:26.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.365+0000 7fbeaffff700 1 -- 172.21.15.46:0/1476442667 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fbec8011070 con 0x558032061400 2024-05-09T04:54:26.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.365+0000 7fbee5ab6b80 10 client.16215 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.365+0000 7fbee5ab6b80 10 client.16215 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.365+0000 7fbee5ab6b80 20 client.16215 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.365+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/1476442667 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55803207b070 0x55803207d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.365+0000 7fbee5ab6b80 4 client.16215 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.365+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/1476442667 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558031f91a80 con 0x55803207b070 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.365+0000 7fbed4e3c700 1 --2- 172.21.15.46:0/1476442667 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55803207b070 0x55803207d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.366+0000 7fbeaffff700 10 client.16215 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.366+0000 7fbed4e3c700 1 --2- 172.21.15.46:0/1476442667 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55803207b070 0x55803207d490 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d057ca6925f8aa3 server_cookie=f4862ec9d8beb159 in_seq=0 out_seq=0 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.366+0000 7fbeaffff700 1 -- 172.21.15.46:0/1476442667 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558031f91a80 con 0x55803207b070 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.366+0000 7fbeaffff700 10 client.16215 handle_command_reply: tid=0 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.367+0000 7fbee5ab6b80 1 client.16215 shutdown 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.367+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/1476442667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558032061400 msgr2=0x55803206bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.367+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/1476442667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558032061400 0x55803206bb30 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fbec8004f50 tx=0x7fbec800cd60 comp rx=0 tx=0).stop 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.367+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/1476442667 shutdown_connections 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.367+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/1476442667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558032061400 0x55803206bb30 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.367+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/1476442667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee44a0 0x55803206b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.367+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/1476442667 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55803207b070 0x55803207d490 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.367+0000 7fbeaffff700 0 client.16215 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:26.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.367+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/1476442667 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031f5ed00 0x55803206b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.367+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/1476442667 >> 172.21.15.46:0/1476442667 conn(0x558031ee7f70 msgr2=0x558032055ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:26.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.367+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/1476442667 shutdown_connections 2024-05-09T04:54:26.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.367+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/1476442667 wait complete. 2024-05-09T04:54:26.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.367+0000 7fbee5ab6b80 20 client.16215 trim_cache size 0 max 16384 2024-05-09T04:54:26.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.382+0000 7fbee5ab6b80 1 Processor -- start 2024-05-09T04:54:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.382+0000 7fbee5ab6b80 1 -- start start 2024-05-09T04:54:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.382+0000 7fbee5ab6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee7f70 0x558032056a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.382+0000 7fbee5ab6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031ee44a0 0x55803206d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.382+0000 7fbee5ab6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558031f5ed00 0x5580320603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.382+0000 7fbee5ab6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbed01b17a0 con 0x558031f5ed00 2024-05-09T04:54:26.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.382+0000 7fbee5ab6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbed01af030 con 0x558031ee44a0 2024-05-09T04:54:26.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.382+0000 7fbee5ab6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbed01ace80 con 0x558031ee7f70 2024-05-09T04:54:26.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.383+0000 7fbed563d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558031f5ed00 0x5580320603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.383+0000 7fbed563d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558031f5ed00 0x5580320603f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48078/0 (socket says 172.21.15.46:48078) 2024-05-09T04:54:26.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.383+0000 7fbed563d700 1 -- 172.21.15.46:0/3269661196 learned_addr learned my addr 172.21.15.46:0/3269661196 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.383+0000 7fbecffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031ee44a0 0x55803206d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.383+0000 7fbed4e3c700 1 --2- 172.21.15.46:0/3269661196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee7f70 0x558032056a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.383+0000 7fbed563d700 1 -- 172.21.15.46:0/3269661196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee7f70 msgr2=0x558032056a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.383+0000 7fbed563d700 1 --2- 172.21.15.46:0/3269661196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee7f70 0x558032056a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.383+0000 7fbed563d700 1 -- 172.21.15.46:0/3269661196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031ee44a0 msgr2=0x55803206d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:26.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.383+0000 7fbed563d700 1 --2- 172.21.15.46:0/3269661196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031ee44a0 0x55803206d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.383+0000 7fbed563d700 1 -- 172.21.15.46:0/3269661196 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5580320589d0 con 0x558031f5ed00 2024-05-09T04:54:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.383+0000 7fbecffff700 1 --2- 172.21.15.46:0/3269661196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031ee44a0 0x55803206d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.383+0000 7fbed563d700 1 --2- 172.21.15.46:0/3269661196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558031f5ed00 0x5580320603f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbec800b770 tx=0x7fbec8009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4b6445e01c15d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.383+0000 7fbeba7fc700 1 -- 172.21.15.46:0/3269661196 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec802b030 con 0x558031f5ed00 2024-05-09T04:54:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.383+0000 7fbeba7fc700 1 -- 172.21.15.46:0/3269661196 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbec80058f0 con 0x558031f5ed00 2024-05-09T04:54:26.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.383+0000 7fbeba7fc700 1 -- 172.21.15.46:0/3269661196 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec8020040 con 0x558031f5ed00 2024-05-09T04:54:26.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.384+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/3269661196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558031f5ed00 msgr2=0x5580320603f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:26.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.384+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/3269661196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558031f5ed00 0x5580320603f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fbec800b770 tx=0x7fbec8009b80 comp rx=0 tx=0).stop 2024-05-09T04:54:26.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.384+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/3269661196 shutdown_connections 2024-05-09T04:54:26.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.384+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/3269661196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558031f5ed00 0x5580320603f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.384+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/3269661196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031ee7f70 0x558032056a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.384+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/3269661196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031ee44a0 0x55803206d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.384+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/3269661196 >> 172.21.15.46:0/3269661196 conn(0x5580320775c0 msgr2=0x558032066190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.384+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/3269661196 shutdown_connections 2024-05-09T04:54:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.384+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/3269661196 wait complete. 2024-05-09T04:54:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.385+0000 7fbee5ab6b80 1 -- start start 2024-05-09T04:54:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.385+0000 7fbee5ab6b80 1 Processor -- start 2024-05-09T04:54:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.385+0000 7fbee5ab6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558031ee44a0 0x5580320805d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.385+0000 7fbee5ab6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031ee7f70 0x558032080b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.385+0000 7fbee5ab6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031f5ed00 0x55803207db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.385+0000 7fbee5ab6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbed01ad4f0 con 0x558031ee44a0 2024-05-09T04:54:26.394 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.385+0000 7fbee5ab6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbed01ae960 con 0x558031ee7f70 2024-05-09T04:54:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.385+0000 7fbee5ab6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbed01adbc0 con 0x558031f5ed00 2024-05-09T04:54:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.385+0000 7fbee5ab6b80 10 client.0 _ll_register_callbacks cb 0x558031df56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.385+0000 7fbed4e3c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558031ee44a0 0x5580320805d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.385+0000 7fbed4e3c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558031ee44a0 0x5580320805d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48088/0 (socket says 172.21.15.46:48088) 2024-05-09T04:54:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.385+0000 7fbed4e3c700 1 -- 172.21.15.46:0/2063661012 learned_addr learned my addr 172.21.15.46:0/2063661012 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.385+0000 7fbecffff700 1 --2- 172.21.15.46:0/2063661012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031ee7f70 0x558032080b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.386+0000 7fbed563d700 1 --2- 172.21.15.46:0/2063661012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031f5ed00 0x55803207db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.386+0000 7fbed4e3c700 1 -- 172.21.15.46:0/2063661012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031f5ed00 msgr2=0x55803207db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.386+0000 7fbed4e3c700 1 --2- 172.21.15.46:0/2063661012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031f5ed00 0x55803207db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.386+0000 7fbed4e3c700 1 -- 172.21.15.46:0/2063661012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031ee7f70 msgr2=0x558032080b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.386+0000 7fbed4e3c700 1 --2- 172.21.15.46:0/2063661012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031ee7f70 0x558032080b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.386+0000 7fbed4e3c700 1 -- 172.21.15.46:0/2063661012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55803207e320 con 0x558031ee44a0 2024-05-09T04:54:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.386+0000 7fbed4e3c700 1 --2- 172.21.15.46:0/2063661012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558031ee44a0 0x5580320805d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbec40097c0 tx=0x7fbec40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b337e3f967d49ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:26.395 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.386+0000 7fbeaffff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.386+0000 7fbeaffff700 1 -- 172.21.15.46:0/2063661012 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbec401de50 con 0x558031ee44a0 2024-05-09T04:54:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.386+0000 7fbeaffff700 1 -- 172.21.15.46:0/2063661012 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fbec4003680 con 0x558031ee44a0 2024-05-09T04:54:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.386+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/2063661012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbec80058f0 con 0x558031ee44a0 2024-05-09T04:54:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.386+0000 7fbeaffff700 1 -- 172.21.15.46:0/2063661012 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbec403c050 con 0x558031ee44a0 2024-05-09T04:54:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.386+0000 7fbee5ab6b80 10 client.16221 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.386+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/2063661012 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5580320e9530 con 0x558031ee44a0 2024-05-09T04:54:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.387+0000 7fbeaffff700 1 -- 172.21.15.46:0/2063661012 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbec80058f0 con 0x558031ee44a0 2024-05-09T04:54:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.387+0000 7fbeaffff700 1 -- 172.21.15.46:0/2063661012 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fbec402c6d0 con 0x558031ee44a0 2024-05-09T04:54:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.387+0000 7fbee5ab6b80 10 client.16221 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.387+0000 7fbee5ab6b80 10 client.16221 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.387+0000 7fbee5ab6b80 20 client.16221 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.387+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/2063661012 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x558032061400 0x5580320ebc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.387+0000 7fbee5ab6b80 4 client.16221 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:54:26.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.387+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/2063661012 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558031f8bb20 con 0x558032061400 2024-05-09T04:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.387+0000 7fbecffff700 1 --2- 172.21.15.46:0/2063661012 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x558032061400 0x5580320ebc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbecffff700 1 --2- 172.21.15.46:0/2063661012 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x558032061400 0x5580320ebc60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54acb31459909ee1 server_cookie=6c491d0736feb057 in_seq=0 out_seq=0 2024-05-09T04:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbeaffff700 10 client.16221 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbeaffff700 1 -- 172.21.15.46:0/2063661012 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558031f8bb20 con 0x558032061400 2024-05-09T04:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbeaffff700 10 client.16221 handle_command_reply: tid=0 2024-05-09T04:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbee5ab6b80 1 client.16221 shutdown 2024-05-09T04:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/2063661012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558031ee44a0 msgr2=0x5580320805d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/2063661012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558031ee44a0 0x5580320805d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbec40097c0 tx=0x7fbec40047c0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/2063661012 shutdown_connections 2024-05-09T04:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/2063661012 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558031f5ed00 0x55803207db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/2063661012 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558031ee44a0 0x5580320805d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/2063661012 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x558032061400 0x5580320ebc60 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbee5ab6b80 1 --2- 172.21.15.46:0/2063661012 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558031ee7f70 0x558032080b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbeaffff700 0 client.16221 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/2063661012 >> 172.21.15.46:0/2063661012 conn(0x5580320775c0 msgr2=0x558032064cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/2063661012 shutdown_connections 2024-05-09T04:54:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbee5ab6b80 1 -- 172.21.15.46:0/2063661012 wait complete. 2024-05-09T04:54:26.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.388+0000 7fbee5ab6b80 20 client.16221 trim_cache size 0 max 16384 2024-05-09T04:54:26.398 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.389+0000 7fbed70a0700 1 -- 172.21.15.46:0/522910631 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbec00b2a70 msgr2=0x7fbec00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.389+0000 7fbed70a0700 1 --2- 172.21.15.46:0/522910631 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbec00b2a70 0x7fbec00b4f80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fbec4009c60 tx=0x7fbec4009260 comp rx=0 tx=0).stop 2024-05-09T04:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.389+0000 7fbed70a0700 1 -- 172.21.15.46:0/522910631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0107e10 msgr2=0x7fbed019b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.389+0000 7fbed70a0700 1 --2- 172.21.15.46:0/522910631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0107e10 0x7fbed019b4b0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fbebc00d950 tx=0x7fbebc00dc60 comp rx=0 tx=0).stop 2024-05-09T04:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.389+0000 7fbed70a0700 1 -- 172.21.15.46:0/522910631 shutdown_connections 2024-05-09T04:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.389+0000 7fbed70a0700 1 --2- 172.21.15.46:0/522910631 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbec00b2a70 0x7fbec00b4f80 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.389+0000 7fbed70a0700 1 --2- 172.21.15.46:0/522910631 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbed0108760 0x7fbed019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.389+0000 7fbed70a0700 1 --2- 172.21.15.46:0/522910631 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbed01034e0 0x7fbed019af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.389+0000 7fbed70a0700 1 --2- 172.21.15.46:0/522910631 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbed0107e10 0x7fbed019b4b0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.389+0000 7fbed70a0700 1 -- 172.21.15.46:0/522910631 >> 172.21.15.46:0/522910631 conn(0x7fbed00fef30 msgr2=0x7fbed0107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.389+0000 7fbed70a0700 1 -- 172.21.15.46:0/522910631 shutdown_connections 2024-05-09T04:54:26.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:26.389+0000 7fbed70a0700 1 -- 172.21.15.46:0/522910631 wait complete. 2024-05-09T04:54:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:26 smithi110 ceph-mon[34904]: pgmap v103: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.4 KiB/s wr, 64 op/s 2024-05-09T04:54:27.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:26 smithi046 ceph-mon[30136]: pgmap v103: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.4 KiB/s wr, 64 op/s 2024-05-09T04:54:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:26 smithi072 ceph-mon[34088]: pgmap v103: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 2.4 KiB/s wr, 64 op/s 2024-05-09T04:54:28.136 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:54:28.136 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:54:28.404 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:28.801 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:28 smithi046 ceph-mon[30136]: pgmap v104: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.5 KiB/s wr, 67 op/s 2024-05-09T04:54:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:28 smithi110 ceph-mon[34904]: pgmap v104: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.5 KiB/s wr, 67 op/s 2024-05-09T04:54:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:28 smithi072 ceph-mon[34088]: pgmap v104: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.5 KiB/s wr, 67 op/s 2024-05-09T04:54:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.250+0000 7fdbfd9ee700 1 -- 172.21.15.46:0/2305771932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf8108780 msgr2=0x7fdbf81100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.250+0000 7fdbfd9ee700 1 --2- 172.21.15.46:0/2305771932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf8108780 0x7fdbf81100e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000b0a0 tx=0x7fdbf000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.250+0000 7fdbfd9ee700 1 -- 172.21.15.46:0/2305771932 shutdown_connections 2024-05-09T04:54:30.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.250+0000 7fdbfd9ee700 1 --2- 172.21.15.46:0/2305771932 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf8108780 0x7fdbf81100e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.250+0000 7fdbfd9ee700 1 --2- 172.21.15.46:0/2305771932 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf8103550 0x7fdbf8103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.250+0000 7fdbfd9ee700 1 --2- 172.21.15.46:0/2305771932 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdbf8107e30 0x7fdbf8108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.250+0000 7fdbfd9ee700 1 -- 172.21.15.46:0/2305771932 >> 172.21.15.46:0/2305771932 conn(0x7fdbf8078e00 msgr2=0x7fdbf8079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.250+0000 7fdbfd9ee700 1 -- 172.21.15.46:0/2305771932 shutdown_connections 2024-05-09T04:54:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.250+0000 7fdbfd9ee700 1 -- 172.21.15.46:0/2305771932 wait complete. 2024-05-09T04:54:30.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.251+0000 7fdbfd9ee700 1 Processor -- start 2024-05-09T04:54:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.251+0000 7fdbfd9ee700 1 -- start start 2024-05-09T04:54:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbfd9ee700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdbf8103550 0x7fdbf819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbfd9ee700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf8107e30 0x7fdbf819b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbfd9ee700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf8108780 0x7fdbf819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbfd9ee700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf8114510 con 0x7fdbf8108780 2024-05-09T04:54:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbfd9ee700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf8114390 con 0x7fdbf8103550 2024-05-09T04:54:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbfd9ee700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf8114690 con 0x7fdbf8107e30 2024-05-09T04:54:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbf77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf8108780 0x7fdbf819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbf77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf8108780 0x7fdbf819f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48110/0 (socket says 172.21.15.46:48110) 2024-05-09T04:54:30.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbf77fe700 1 -- 172.21.15.46:0/163023229 learned_addr learned my addr 172.21.15.46:0/163023229 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:30.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbf6ffd700 1 --2- 172.21.15.46:0/163023229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdbf8103550 0x7fdbf819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbf67fc700 1 --2- 172.21.15.46:0/163023229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf8107e30 0x7fdbf819b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbf77fe700 1 -- 172.21.15.46:0/163023229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf8107e30 msgr2=0x7fdbf819b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:30.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbf77fe700 1 --2- 172.21.15.46:0/163023229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf8107e30 0x7fdbf819b340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbf77fe700 1 -- 172.21.15.46:0/163023229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdbf8103550 msgr2=0x7fdbf819ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbf77fe700 1 --2- 172.21.15.46:0/163023229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdbf8103550 0x7fdbf819ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.252+0000 7fdbf77fe700 1 -- 172.21.15.46:0/163023229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbf819fc70 con 0x7fdbf8108780 2024-05-09T04:54:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.253+0000 7fdbf77fe700 1 --2- 172.21.15.46:0/163023229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf8108780 0x7fdbf819f5d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000b830 tx=0x7fdbf0004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.253+0000 7fdbfc9ec700 1 -- 172.21.15.46:0/163023229 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0021070 con 0x7fdbf8108780 2024-05-09T04:54:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.253+0000 7fdbfc9ec700 1 -- 172.21.15.46:0/163023229 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbf0007e30 con 0x7fdbf8108780 2024-05-09T04:54:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.253+0000 7fdbfd9ee700 1 -- 172.21.15.46:0/163023229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbf819ff00 con 0x7fdbf8108780 2024-05-09T04:54:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.253+0000 7fdbfc9ec700 1 -- 172.21.15.46:0/163023229 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbf0004700 con 0x7fdbf8108780 2024-05-09T04:54:30.258 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.253+0000 7fdbfd9ee700 1 -- 172.21.15.46:0/163023229 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbf81a7880 con 0x7fdbf8108780 2024-05-09T04:54:30.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.256+0000 7fdbfc9ec700 1 -- 172.21.15.46:0/163023229 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdbf001d450 con 0x7fdbf8108780 2024-05-09T04:54:30.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.256+0000 7fdbfc9ec700 1 --2- 172.21.15.46:0/163023229 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdbe00b29f0 0x7fdbe00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.256+0000 7fdbf6ffd700 1 --2- 172.21.15.46:0/163023229 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdbe00b29f0 0x7fdbe00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdbfc9ec700 1 -- 172.21.15.46:0/163023229 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fdbf000f020 con 0x7fdbf8108780 2024-05-09T04:54:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdc0c404b80 1 Processor -- start 2024-05-09T04:54:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdc0c404b80 1 -- start start 2024-05-09T04:54:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdc0c404b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c036031e0 0x564c0369c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdc0c404b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c0369cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdc0c404b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03696720 0x564c03696b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdc0c404b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564c035b9070 con 0x564c03519520 2024-05-09T04:54:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdc0c404b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x564c03557870 con 0x564c03696720 2024-05-09T04:54:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdc0c404b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564c035b5030 con 0x564c036031e0 2024-05-09T04:54:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdbf6ffd700 1 --2- 172.21.15.46:0/163023229 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdbe00b29f0 0x7fdbe00b4f00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8005950 tx=0x7fdbe800b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdbf67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c0369cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdbf67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c0369cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48120/0 (socket says 172.21.15.46:48120) 2024-05-09T04:54:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdbf67fc700 1 -- 172.21.15.46:0/2002694163 learned_addr learned my addr 172.21.15.46:0/2002694163 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:30.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdbf6ffd700 1 --2- 172.21.15.46:0/2002694163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c036031e0 0x564c0369c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdbf77fe700 1 --2- 172.21.15.46:0/2002694163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03696720 0x564c03696b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdbf67fc700 1 -- 172.21.15.46:0/2002694163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c036031e0 msgr2=0x564c0369c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdbf67fc700 1 --2- 172.21.15.46:0/2002694163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c036031e0 0x564c0369c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdbf67fc700 1 -- 172.21.15.46:0/2002694163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03696720 msgr2=0x564c03696b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdbf67fc700 1 --2- 172.21.15.46:0/2002694163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03696720 0x564c03696b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.257+0000 7fdbf67fc700 1 -- 172.21.15.46:0/2002694163 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c0354f3f0 con 0x564c03519520 2024-05-09T04:54:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.258+0000 7fdbf6ffd700 1 --2- 172.21.15.46:0/2002694163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c036031e0 0x564c0369c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.258+0000 7fdbf67fc700 1 --2- 172.21.15.46:0/2002694163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c0369cab0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00bc70 tx=0x7fdbec00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0cb3949192ace3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.258+0000 7fdbddffb700 1 -- 172.21.15.46:0/2002694163 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec010c90 con 0x564c03519520 2024-05-09T04:54:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.258+0000 7fdbddffb700 1 -- 172.21.15.46:0/2002694163 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbec010e30 con 0x564c03519520 2024-05-09T04:54:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.258+0000 7fdbddffb700 1 -- 172.21.15.46:0/2002694163 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec005280 con 0x564c03519520 2024-05-09T04:54:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.259+0000 7fdc0c404b80 1 -- 172.21.15.46:0/2002694163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 msgr2=0x564c0369cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.259+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/2002694163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c0369cab0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fdbec00bc70 tx=0x7fdbec00bf80 comp rx=0 tx=0).stop 2024-05-09T04:54:30.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.259+0000 7fdc0c404b80 1 -- 172.21.15.46:0/2002694163 shutdown_connections 2024-05-09T04:54:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.259+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/2002694163 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03696720 0x564c03696b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.259+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/2002694163 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c036031e0 0x564c0369c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.259+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/2002694163 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c0369cab0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.259+0000 7fdc0c404b80 1 -- 172.21.15.46:0/2002694163 >> 172.21.15.46:0/2002694163 conn(0x564c03579720 msgr2=0x564c03691480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.259+0000 7fdc0c404b80 1 -- 172.21.15.46:0/2002694163 shutdown_connections 2024-05-09T04:54:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.259+0000 7fdc0c404b80 1 -- 172.21.15.46:0/2002694163 wait complete. 2024-05-09T04:54:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.259+0000 7fdc0c404b80 1 -- start start 2024-05-09T04:54:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdc0c404b80 1 Processor -- start 2024-05-09T04:54:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdc0c404b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03519520 0x564c036a0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdc0c404b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c036031e0 0x564c036a06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdc0c404b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c03696720 0x564c036a0c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdc0c404b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564c03434c20 con 0x564c036031e0 2024-05-09T04:54:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdc0c404b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x564c035731d0 con 0x564c03519520 2024-05-09T04:54:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdc0c404b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564c036a3250 con 0x564c03696720 2024-05-09T04:54:30.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdc0c404b80 10 client.0 _ll_register_callbacks cb 0x564c0342a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdbf67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c036031e0 0x564c036a06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdbf67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c036031e0 0x564c036a06d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48136/0 (socket says 172.21.15.46:48136) 2024-05-09T04:54:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdbf67fc700 1 -- 172.21.15.46:0/4204016126 learned_addr learned my addr 172.21.15.46:0/4204016126 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdbf6ffd700 1 --2- 172.21.15.46:0/4204016126 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03519520 0x564c036a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdbf77fe700 1 --2- 172.21.15.46:0/4204016126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c03696720 0x564c036a0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdbf67fc700 1 -- 172.21.15.46:0/4204016126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c03696720 msgr2=0x564c036a0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdbf67fc700 1 --2- 172.21.15.46:0/4204016126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c03696720 0x564c036a0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdbf67fc700 1 -- 172.21.15.46:0/4204016126 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03519520 msgr2=0x564c036a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdbf67fc700 1 --2- 172.21.15.46:0/4204016126 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03519520 0x564c036a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdbf67fc700 1 -- 172.21.15.46:0/4204016126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c03434510 con 0x564c036031e0 2024-05-09T04:54:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.260+0000 7fdbf6ffd700 1 --2- 172.21.15.46:0/4204016126 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03519520 0x564c036a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.261+0000 7fdbf67fc700 1 --2- 172.21.15.46:0/4204016126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c036031e0 0x564c036a06d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdbec010e00 tx=0x7fdbec004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4756422942cb6a9f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.261+0000 7fdbd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.261+0000 7fdbd6ffd700 1 -- 172.21.15.46:0/4204016126 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbec004250 con 0x564c036031e0 2024-05-09T04:54:30.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.261+0000 7fdbd6ffd700 1 -- 172.21.15.46:0/4204016126 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fdbec0043f0 con 0x564c036031e0 2024-05-09T04:54:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.261+0000 7fdc0c404b80 1 -- 172.21.15.46:0/4204016126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbec010e30 con 0x564c036031e0 2024-05-09T04:54:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.261+0000 7fdbd6ffd700 1 -- 172.21.15.46:0/4204016126 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdbec003680 con 0x564c036031e0 2024-05-09T04:54:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.261+0000 7fdc0c404b80 10 client.16233 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.261+0000 7fdc0c404b80 1 -- 172.21.15.46:0/4204016126 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564c036adc30 con 0x564c036031e0 2024-05-09T04:54:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.261+0000 7fdbd6ffd700 1 -- 172.21.15.46:0/4204016126 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbec010e30 con 0x564c036031e0 2024-05-09T04:54:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.261+0000 7fdbd6ffd700 1 -- 172.21.15.46:0/4204016126 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fdbec0050e0 con 0x564c036031e0 2024-05-09T04:54:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.262+0000 7fdc0c404b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.262+0000 7fdc0c404b80 10 client.16233 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.262+0000 7fdc0c404b80 20 client.16233 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.262+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/4204016126 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564c036b01b0 0x564c036b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.262+0000 7fdc0c404b80 4 client.16233 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.262+0000 7fdc0c404b80 1 -- 172.21.15.46:0/4204016126 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c035c6a50 con 0x564c036b01b0 2024-05-09T04:54:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.262+0000 7fdbf6ffd700 1 --2- 172.21.15.46:0/4204016126 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564c036b01b0 0x564c036b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.262+0000 7fdbf6ffd700 1 --2- 172.21.15.46:0/4204016126 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564c036b01b0 0x564c036b25d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=167452e8840d4964 server_cookie=cd18b78fef17227e in_seq=0 out_seq=0 2024-05-09T04:54:30.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.262+0000 7fdbd6ffd700 10 client.16233 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.263+0000 7fdbd6ffd700 1 -- 172.21.15.46:0/4204016126 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564c035c6a50 con 0x564c036b01b0 2024-05-09T04:54:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.263+0000 7fdbd6ffd700 10 client.16233 handle_command_reply: tid=0 2024-05-09T04:54:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.263+0000 7fdc0c404b80 1 client.16233 shutdown 2024-05-09T04:54:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.263+0000 7fdc0c404b80 1 -- 172.21.15.46:0/4204016126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c036031e0 msgr2=0x564c036a06d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.263+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/4204016126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c036031e0 0x564c036a06d0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdbec010e00 tx=0x7fdbec004800 comp rx=0 tx=0).stop 2024-05-09T04:54:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.263+0000 7fdc0c404b80 1 -- 172.21.15.46:0/4204016126 shutdown_connections 2024-05-09T04:54:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.263+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/4204016126 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c03696720 0x564c036a0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.263+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/4204016126 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564c036b01b0 0x564c036b25d0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.263+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/4204016126 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03519520 0x564c036a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.263+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/4204016126 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c036031e0 0x564c036a06d0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.263+0000 7fdc0c404b80 1 -- 172.21.15.46:0/4204016126 >> 172.21.15.46:0/4204016126 conn(0x564c03579720 msgr2=0x564c03691240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.263+0000 7fdbd6ffd700 0 client.16233 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.264+0000 7fdc0c404b80 1 -- 172.21.15.46:0/4204016126 shutdown_connections 2024-05-09T04:54:30.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.264+0000 7fdc0c404b80 1 -- 172.21.15.46:0/4204016126 wait complete. 2024-05-09T04:54:30.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.264+0000 7fdc0c404b80 20 client.16233 trim_cache size 0 max 16384 2024-05-09T04:54:30.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.284+0000 7fdc0c404b80 1 Processor -- start 2024-05-09T04:54:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.284+0000 7fdc0c404b80 1 -- start start 2024-05-09T04:54:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.284+0000 7fdc0c404b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c0369f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.284+0000 7fdc0c404b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c036031e0 0x564c0369a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.284+0000 7fdc0c404b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c03696720 0x564c036ac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.284+0000 7fdc0c404b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81b17b0 con 0x564c03519520 2024-05-09T04:54:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.284+0000 7fdc0c404b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81af040 con 0x564c036031e0 2024-05-09T04:54:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.284+0000 7fdc0c404b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81ace90 con 0x564c03696720 2024-05-09T04:54:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.285+0000 7fdbf6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c0369f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.285+0000 7fdbf6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c0369f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48142/0 (socket says 172.21.15.46:48142) 2024-05-09T04:54:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.285+0000 7fdbf6ffd700 1 -- 172.21.15.46:0/2383939110 learned_addr learned my addr 172.21.15.46:0/2383939110 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.285+0000 7fdbf67fc700 1 --2- 172.21.15.46:0/2383939110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c036031e0 0x564c0369a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.285+0000 7fdbf77fe700 1 --2- 172.21.15.46:0/2383939110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c03696720 0x564c036ac120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.285+0000 7fdbf6ffd700 1 -- 172.21.15.46:0/2383939110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c03696720 msgr2=0x564c036ac120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.285+0000 7fdbf6ffd700 1 --2- 172.21.15.46:0/2383939110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c03696720 0x564c036ac120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.285+0000 7fdbf6ffd700 1 -- 172.21.15.46:0/2383939110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c036031e0 msgr2=0x564c0369a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.285+0000 7fdbf6ffd700 1 --2- 172.21.15.46:0/2383939110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c036031e0 0x564c0369a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.285+0000 7fdbf6ffd700 1 -- 172.21.15.46:0/2383939110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c0368afe0 con 0x564c03519520 2024-05-09T04:54:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.285+0000 7fdbf77fe700 1 --2- 172.21.15.46:0/2383939110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c03696720 0x564c036ac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:54:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.285+0000 7fdbf6ffd700 1 --2- 172.21.15.46:0/2383939110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c0369f4b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8005d90 tx=0x7fdbe8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bdbd5fb6c0de834 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.285+0000 7fdbdd7fa700 1 -- 172.21.15.46:0/2383939110 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8021d20 con 0x564c03519520 2024-05-09T04:54:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.285+0000 7fdbdd7fa700 1 -- 172.21.15.46:0/2383939110 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbe802c030 con 0x564c03519520 2024-05-09T04:54:30.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.285+0000 7fdbdd7fa700 1 -- 172.21.15.46:0/2383939110 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe80282d0 con 0x564c03519520 2024-05-09T04:54:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.286+0000 7fdc0c404b80 1 -- 172.21.15.46:0/2383939110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 msgr2=0x564c0369f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.286+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/2383939110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c0369f4b0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8005d90 tx=0x7fdbe8013000 comp rx=0 tx=0).stop 2024-05-09T04:54:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.286+0000 7fdc0c404b80 1 -- 172.21.15.46:0/2383939110 shutdown_connections 2024-05-09T04:54:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.286+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/2383939110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c03696720 0x564c036ac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.286+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/2383939110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c0369f4b0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.286+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/2383939110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c036031e0 0x564c0369a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.286+0000 7fdc0c404b80 1 -- 172.21.15.46:0/2383939110 >> 172.21.15.46:0/2383939110 conn(0x564c03579720 msgr2=0x564c03699e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.286+0000 7fdc0c404b80 1 -- 172.21.15.46:0/2383939110 shutdown_connections 2024-05-09T04:54:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.286+0000 7fdc0c404b80 1 -- 172.21.15.46:0/2383939110 wait complete. 2024-05-09T04:54:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.287+0000 7fdc0c404b80 1 -- start start 2024-05-09T04:54:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.287+0000 7fdc0c404b80 1 Processor -- start 2024-05-09T04:54:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.287+0000 7fdc0c404b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c036a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.287+0000 7fdc0c404b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c036031e0 0x564c036a89e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.287+0000 7fdc0c404b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03696720 0x564c036a5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.287+0000 7fdc0c404b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81ad500 con 0x564c03519520 2024-05-09T04:54:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.287+0000 7fdc0c404b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81ae970 con 0x564c03696720 2024-05-09T04:54:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.287+0000 7fdc0c404b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdbf81adbd0 con 0x564c036031e0 2024-05-09T04:54:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.287+0000 7fdbf6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c036a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.287+0000 7fdc0c404b80 10 client.0 _ll_register_callbacks cb 0x564c0342a740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.287+0000 7fdbf6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c036a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48156/0 (socket says 172.21.15.46:48156) 2024-05-09T04:54:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.287+0000 7fdbf6ffd700 1 -- 172.21.15.46:0/1559967025 learned_addr learned my addr 172.21.15.46:0/1559967025 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:30.291 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.287+0000 7fdbf67fc700 1 --2- 172.21.15.46:0/1559967025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c036031e0 0x564c036a89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.288+0000 7fdbf77fe700 1 --2- 172.21.15.46:0/1559967025 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03696720 0x564c036a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.288+0000 7fdbf6ffd700 1 -- 172.21.15.46:0/1559967025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c036031e0 msgr2=0x564c036a89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.288+0000 7fdbf6ffd700 1 --2- 172.21.15.46:0/1559967025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c036031e0 0x564c036a89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.288+0000 7fdbf6ffd700 1 -- 172.21.15.46:0/1559967025 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03696720 msgr2=0x564c036a5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.288+0000 7fdbf6ffd700 1 --2- 172.21.15.46:0/1559967025 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03696720 0x564c036a5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.288+0000 7fdbf6ffd700 1 -- 172.21.15.46:0/1559967025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c036a5ba0 con 0x564c03519520 2024-05-09T04:54:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.288+0000 7fdbf77fe700 1 --2- 172.21.15.46:0/1559967025 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03696720 0x564c036a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.288+0000 7fdbf6ffd700 1 --2- 172.21.15.46:0/1559967025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c036a84a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8005340 tx=0x7fdbe8010ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe93e95b472f1e38 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.288+0000 7fdbd6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.288+0000 7fdbd6ffd700 1 -- 172.21.15.46:0/1559967025 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbe8005560 con 0x564c03519520 2024-05-09T04:54:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.288+0000 7fdbd6ffd700 1 -- 172.21.15.46:0/1559967025 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fdbe8005700 con 0x564c03519520 2024-05-09T04:54:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.288+0000 7fdc0c404b80 1 -- 172.21.15.46:0/1559967025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdbe802c030 con 0x564c03519520 2024-05-09T04:54:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.288+0000 7fdbd6ffd700 1 -- 172.21.15.46:0/1559967025 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdbe8004750 con 0x564c03519520 2024-05-09T04:54:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.288+0000 7fdc0c404b80 10 client.16239 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:30.292 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.288+0000 7fdc0c404b80 1 -- 172.21.15.46:0/1559967025 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564c0371b120 con 0x564c03519520 2024-05-09T04:54:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.289+0000 7fdbd6ffd700 1 -- 172.21.15.46:0/1559967025 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbe802c030 con 0x564c03519520 2024-05-09T04:54:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.289+0000 7fdbd6ffd700 1 -- 172.21.15.46:0/1559967025 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fdbe8028ce0 con 0x564c03519520 2024-05-09T04:54:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.289+0000 7fdc0c404b80 10 client.16239 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.289+0000 7fdc0c404b80 10 client.16239 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.289+0000 7fdc0c404b80 20 client.16239 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.289+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/1559967025 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564c0371b7c0 0x564c0371dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.289+0000 7fdc0c404b80 4 client.16239 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:54:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.289+0000 7fdc0c404b80 1 -- 172.21.15.46:0/1559967025 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564c035c0af0 con 0x564c0371b7c0 2024-05-09T04:54:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.289+0000 7fdbf67fc700 1 --2- 172.21.15.46:0/1559967025 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564c0371b7c0 0x564c0371dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.289+0000 7fdbd6ffd700 10 client.16239 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:30.293 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.289+0000 7fdbf67fc700 1 --2- 172.21.15.46:0/1559967025 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564c0371b7c0 0x564c0371dbe0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2cdc3fe5ac8559b server_cookie=6bc7914647088708 in_seq=0 out_seq=0 2024-05-09T04:54:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.291+0000 7fdbd6ffd700 1 -- 172.21.15.46:0/1559967025 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564c035c0af0 con 0x564c0371b7c0 2024-05-09T04:54:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.291+0000 7fdbd6ffd700 10 client.16239 handle_command_reply: tid=0 2024-05-09T04:54:30.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.291+0000 7fdc0c404b80 1 client.16239 shutdown 2024-05-09T04:54:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.291+0000 7fdc0c404b80 1 -- 172.21.15.46:0/1559967025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 msgr2=0x564c036a84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.291+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/1559967025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c036a84a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8005340 tx=0x7fdbe8010ad0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.291+0000 7fdc0c404b80 1 -- 172.21.15.46:0/1559967025 shutdown_connections 2024-05-09T04:54:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.291+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/1559967025 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564c0371b7c0 0x564c0371dbe0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.291+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/1559967025 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564c03696720 0x564c036a5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.291+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/1559967025 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564c03519520 0x564c036a84a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.291+0000 7fdc0c404b80 1 --2- 172.21.15.46:0/1559967025 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564c036031e0 0x564c036a89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.291+0000 7fdc0c404b80 1 -- 172.21.15.46:0/1559967025 >> 172.21.15.46:0/1559967025 conn(0x564c03579720 msgr2=0x564c0369f210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.291+0000 7fdbd6ffd700 0 client.16239 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.291+0000 7fdc0c404b80 1 -- 172.21.15.46:0/1559967025 shutdown_connections 2024-05-09T04:54:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.291+0000 7fdc0c404b80 1 -- 172.21.15.46:0/1559967025 wait complete. 2024-05-09T04:54:30.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.291+0000 7fdc0c404b80 20 client.16239 trim_cache size 0 max 16384 2024-05-09T04:54:30.296 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:54:30.296 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:54:30.296 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "79dc4f12-5e2b-4856-bb87-2ed54b0ec50a", 2024-05-09T04:54:30.296 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:54:30.296 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:54:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.292+0000 7fdbfd9ee700 1 -- 172.21.15.46:0/163023229 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdbe00b29f0 msgr2=0x7fdbe00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.292+0000 7fdbfd9ee700 1 --2- 172.21.15.46:0/163023229 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdbe00b29f0 0x7fdbe00b4f00 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fdbe8005950 tx=0x7fdbe800b410 comp rx=0 tx=0).stop 2024-05-09T04:54:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.292+0000 7fdbfd9ee700 1 -- 172.21.15.46:0/163023229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf8108780 msgr2=0x7fdbf819f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.292+0000 7fdbfd9ee700 1 --2- 172.21.15.46:0/163023229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf8108780 0x7fdbf819f5d0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fdbf000b830 tx=0x7fdbf0004ce0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.292+0000 7fdbfd9ee700 1 -- 172.21.15.46:0/163023229 shutdown_connections 2024-05-09T04:54:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.292+0000 7fdbfd9ee700 1 --2- 172.21.15.46:0/163023229 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdbe00b29f0 0x7fdbe00b4f00 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.292+0000 7fdbfd9ee700 1 --2- 172.21.15.46:0/163023229 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdbf8108780 0x7fdbf819f5d0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.292+0000 7fdbfd9ee700 1 --2- 172.21.15.46:0/163023229 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdbf8103550 0x7fdbf819ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.292+0000 7fdbfd9ee700 1 --2- 172.21.15.46:0/163023229 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdbf8107e30 0x7fdbf819b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.292+0000 7fdbfd9ee700 1 -- 172.21.15.46:0/163023229 >> 172.21.15.46:0/163023229 conn(0x7fdbf8078e00 msgr2=0x7fdbf8100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.292+0000 7fdbfd9ee700 1 -- 172.21.15.46:0/163023229 shutdown_connections 2024-05-09T04:54:30.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:30.292+0000 7fdbfd9ee700 1 -- 172.21.15.46:0/163023229 wait complete. 2024-05-09T04:54:30.999 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:30 smithi046 ceph-mon[30136]: pgmap v105: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-05-09T04:54:31.032 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:54:31.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:30 smithi110 ceph-mon[34904]: pgmap v105: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-05-09T04:54:31.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:30 smithi072 ceph-mon[34088]: pgmap v105: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 3.4 KiB/s wr, 6 op/s 2024-05-09T04:54:31.287 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:31 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:54:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:31 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:31 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:54:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:31 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:31 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:54:32.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:31 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:54:32.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:31 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:32.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:31 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:54:32.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:31 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:32.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:31 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:54:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:31 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:54:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:31 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:31 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:54:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:31 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:31 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:54:32.933 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:32 smithi046 ceph-mon[30136]: pgmap v106: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-05-09T04:54:33.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:32 smithi110 ceph-mon[34904]: pgmap v106: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-05-09T04:54:33.153 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:32 smithi072 ceph-mon[34088]: pgmap v106: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 3.4 KiB/s wr, 9 op/s 2024-05-09T04:54:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.201+0000 7fb4c6823700 1 -- 172.21.15.46:0/734129104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4c0107e10 msgr2=0x7fb4c0108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.201+0000 7fb4c6823700 1 --2- 172.21.15.46:0/734129104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4c0107e10 0x7fb4c0108220 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fb4a80099c0 tx=0x7fb4a8009cd0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.202+0000 7fb4c6823700 1 -- 172.21.15.46:0/734129104 shutdown_connections 2024-05-09T04:54:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.202+0000 7fb4c6823700 1 --2- 172.21.15.46:0/734129104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c0108760 0x7fb4c01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.202+0000 7fb4c6823700 1 --2- 172.21.15.46:0/734129104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4c01034e0 0x7fb4c01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.202+0000 7fb4c6823700 1 --2- 172.21.15.46:0/734129104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4c0107e10 0x7fb4c0108220 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.202+0000 7fb4c6823700 1 -- 172.21.15.46:0/734129104 >> 172.21.15.46:0/734129104 conn(0x7fb4c00fef30 msgr2=0x7fb4c0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:33.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.202+0000 7fb4c6823700 1 -- 172.21.15.46:0/734129104 shutdown_connections 2024-05-09T04:54:33.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.202+0000 7fb4c6823700 1 -- 172.21.15.46:0/734129104 wait complete. 2024-05-09T04:54:33.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.202+0000 7fb4c6823700 1 Processor -- start 2024-05-09T04:54:33.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.202+0000 7fb4c6823700 1 -- start start 2024-05-09T04:54:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4c6823700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c01034e0 0x7fb4c00ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4c6823700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4c0107e10 0x7fb4c00ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4c6823700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4c0108760 0x7fb4c00ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4c6823700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c01133e0 con 0x7fb4c0108760 2024-05-09T04:54:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4c6823700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c0113260 con 0x7fb4c01034e0 2024-05-09T04:54:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4c6823700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c0113560 con 0x7fb4c0107e10 2024-05-09T04:54:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4c4dc0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4c0108760 0x7fb4c00ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4c4dc0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4c0108760 0x7fb4c00ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48178/0 (socket says 172.21.15.46:48178) 2024-05-09T04:54:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4c4dc0700 1 -- 172.21.15.46:0/1394705616 learned_addr learned my addr 172.21.15.46:0/1394705616 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4bffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c01034e0 0x7fb4c00ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4b7fff700 1 --2- 172.21.15.46:0/1394705616 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4c0107e10 0x7fb4c00ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4c4dc0700 1 -- 172.21.15.46:0/1394705616 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4c0107e10 msgr2=0x7fb4c00ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:33.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4c4dc0700 1 --2- 172.21.15.46:0/1394705616 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4c0107e10 0x7fb4c00ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4c4dc0700 1 -- 172.21.15.46:0/1394705616 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c01034e0 msgr2=0x7fb4c00ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4c4dc0700 1 --2- 172.21.15.46:0/1394705616 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c01034e0 0x7fb4c00ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4c4dc0700 1 -- 172.21.15.46:0/1394705616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4c01a9220 con 0x7fb4c0108760 2024-05-09T04:54:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.203+0000 7fb4c4dc0700 1 --2- 172.21.15.46:0/1394705616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4c0108760 0x7fb4c00ffe90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8007ee0 tx=0x7fb4b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.204+0000 7fb4bdffb700 1 -- 172.21.15.46:0/1394705616 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8004bb0 con 0x7fb4c0108760 2024-05-09T04:54:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.204+0000 7fb4c6823700 1 -- 172.21.15.46:0/1394705616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4c01a9510 con 0x7fb4c0108760 2024-05-09T04:54:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.204+0000 7fb4c6823700 1 -- 172.21.15.46:0/1394705616 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4c01a9a20 con 0x7fb4c0108760 2024-05-09T04:54:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.204+0000 7fb4bdffb700 1 -- 172.21.15.46:0/1394705616 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b8004d50 con 0x7fb4c0108760 2024-05-09T04:54:33.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.204+0000 7fb4bdffb700 1 -- 172.21.15.46:0/1394705616 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b8005920 con 0x7fb4c0108760 2024-05-09T04:54:33.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.206+0000 7fb4bdffb700 1 -- 172.21.15.46:0/1394705616 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb4b801a020 con 0x7fb4c0108760 2024-05-09T04:54:33.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.207+0000 7fb4bdffb700 1 --2- 172.21.15.46:0/1394705616 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb4ac0b29f0 0x7fb4ac0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.207+0000 7fb4bffff700 1 --2- 172.21.15.46:0/1394705616 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb4ac0b29f0 0x7fb4ac0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.207+0000 7fb4bdffb700 1 -- 172.21.15.46:0/1394705616 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fb4b80e1d20 con 0x7fb4c0108760 2024-05-09T04:54:33.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.208+0000 7fb4d5239b80 1 Processor -- start 2024-05-09T04:54:33.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.208+0000 7fb4d5239b80 1 -- start start 2024-05-09T04:54:33.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.208+0000 7fb4bffff700 1 --2- 172.21.15.46:0/1394705616 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb4ac0b29f0 0x7fb4ac0b4f00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009a10 tx=0x7fb4b0009190 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:33.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.208+0000 7fb4d5239b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 0x5629f41204f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.208+0000 7fb4d5239b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f3f9d4a0 0x5629f4120a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.208+0000 7fb4d5239b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f411a6a0 0x5629f411aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.208+0000 7fb4d5239b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5629f403cff0 con 0x5629f411a6a0 2024-05-09T04:54:33.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.208+0000 7fb4d5239b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5629f3fdb7f0 con 0x5629f4087160 2024-05-09T04:54:33.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.208+0000 7fb4d5239b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5629f4038fb0 con 0x5629f3f9d4a0 2024-05-09T04:54:33.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.208+0000 7fb4c4dc0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f411a6a0 0x5629f411aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.208+0000 7fb4c4dc0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f411a6a0 0x5629f411aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48194/0 (socket says 172.21.15.46:48194) 2024-05-09T04:54:33.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.208+0000 7fb4c4dc0700 1 -- 172.21.15.46:0/2922760628 learned_addr learned my addr 172.21.15.46:0/2922760628 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:33.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.208+0000 7fb4bffff700 1 --2- 172.21.15.46:0/2922760628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 0x5629f41204f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.208+0000 7fb4b7fff700 1 --2- 172.21.15.46:0/2922760628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f3f9d4a0 0x5629f4120a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.209+0000 7fb4c4dc0700 1 -- 172.21.15.46:0/2922760628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f3f9d4a0 msgr2=0x5629f4120a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:33.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.209+0000 7fb4c4dc0700 1 --2- 172.21.15.46:0/2922760628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f3f9d4a0 0x5629f4120a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.209+0000 7fb4c4dc0700 1 -- 172.21.15.46:0/2922760628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 msgr2=0x5629f41204f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:33.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.209+0000 7fb4c4dc0700 1 --2- 172.21.15.46:0/2922760628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 0x5629f41204f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.209+0000 7fb4c4dc0700 1 -- 172.21.15.46:0/2922760628 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f3fd3370 con 0x5629f411a6a0 2024-05-09T04:54:33.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.209+0000 7fb4bffff700 1 --2- 172.21.15.46:0/2922760628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 0x5629f41204f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.209+0000 7fb4c4dc0700 1 --2- 172.21.15.46:0/2922760628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f411a6a0 0x5629f411aad0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80df4b0 tx=0x7fb4b8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b05eb74376643e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.209+0000 7fb4b67fc700 1 -- 172.21.15.46:0/2922760628 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80a96e0 con 0x5629f411a6a0 2024-05-09T04:54:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.209+0000 7fb4b67fc700 1 -- 172.21.15.46:0/2922760628 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b80a8fe0 con 0x5629f411a6a0 2024-05-09T04:54:33.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.209+0000 7fb4b67fc700 1 -- 172.21.15.46:0/2922760628 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80a9240 con 0x5629f411a6a0 2024-05-09T04:54:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.210+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2922760628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f411a6a0 msgr2=0x5629f411aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.210+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2922760628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f411a6a0 0x5629f411aad0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80df4b0 tx=0x7fb4b8007f90 comp rx=0 tx=0).stop 2024-05-09T04:54:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.210+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2922760628 shutdown_connections 2024-05-09T04:54:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.210+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2922760628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f411a6a0 0x5629f411aad0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.210+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2922760628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 0x5629f41204f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.216 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.210+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2922760628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f3f9d4a0 0x5629f4120a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.210+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2922760628 >> 172.21.15.46:0/2922760628 conn(0x5629f3ffd6a0 msgr2=0x5629f4115400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.211+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2922760628 shutdown_connections 2024-05-09T04:54:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.211+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2922760628 wait complete. 2024-05-09T04:54:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.211+0000 7fb4d5239b80 1 -- start start 2024-05-09T04:54:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.211+0000 7fb4d5239b80 1 Processor -- start 2024-05-09T04:54:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.211+0000 7fb4d5239b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f3f9d4a0 0x5629f41242d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4d5239b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f4087160 0x5629f4124810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4d5239b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f411a6a0 0x5629f4124d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4d5239b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5629f3eb8ba0 con 0x5629f411a6a0 2024-05-09T04:54:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4d5239b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5629f3ff7150 con 0x5629f3f9d4a0 2024-05-09T04:54:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4d5239b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5629f41271d0 con 0x5629f4087160 2024-05-09T04:54:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4d5239b80 10 client.0 _ll_register_callbacks cb 0x5629f3eae6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:33.217 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4c4dc0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f411a6a0 0x5629f4124d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4c4dc0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f411a6a0 0x5629f4124d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48208/0 (socket says 172.21.15.46:48208) 2024-05-09T04:54:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4c4dc0700 1 -- 172.21.15.46:0/2091767350 learned_addr learned my addr 172.21.15.46:0/2091767350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4bffff700 1 --2- 172.21.15.46:0/2091767350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f3f9d4a0 0x5629f41242d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4b7fff700 1 --2- 172.21.15.46:0/2091767350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f4087160 0x5629f4124810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4c4dc0700 1 -- 172.21.15.46:0/2091767350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f4087160 msgr2=0x5629f4124810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4c4dc0700 1 --2- 172.21.15.46:0/2091767350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f4087160 0x5629f4124810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4c4dc0700 1 -- 172.21.15.46:0/2091767350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f3f9d4a0 msgr2=0x5629f41242d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4c4dc0700 1 --2- 172.21.15.46:0/2091767350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f3f9d4a0 0x5629f41242d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4c4dc0700 1 -- 172.21.15.46:0/2091767350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f3eb8490 con 0x5629f411a6a0 2024-05-09T04:54:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4c4dc0700 1 --2- 172.21.15.46:0/2091767350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f411a6a0 0x5629f4124d50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80a5060 tx=0x7fb4b80a5480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e8c8f5661fb658d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:33.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb49f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb49f7fe700 1 -- 172.21.15.46:0/2091767350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b80ad390 con 0x5629f411a6a0 2024-05-09T04:54:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb49f7fe700 1 -- 172.21.15.46:0/2091767350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fb4b80b1bc0 con 0x5629f411a6a0 2024-05-09T04:54:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.212+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2091767350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b80a8fe0 con 0x5629f411a6a0 2024-05-09T04:54:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.213+0000 7fb49f7fe700 1 -- 172.21.15.46:0/2091767350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb4b80a96e0 con 0x5629f411a6a0 2024-05-09T04:54:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.213+0000 7fb4d5239b80 10 client.16251 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.213+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2091767350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5629f4131ca0 con 0x5629f411a6a0 2024-05-09T04:54:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.213+0000 7fb49f7fe700 1 -- 172.21.15.46:0/2091767350 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b80a8fe0 con 0x5629f411a6a0 2024-05-09T04:54:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.213+0000 7fb49f7fe700 1 -- 172.21.15.46:0/2091767350 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fb4b80a8240 con 0x5629f411a6a0 2024-05-09T04:54:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.213+0000 7fb4d5239b80 10 client.16251 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.213+0000 7fb4d5239b80 10 client.16251 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.213+0000 7fb4d5239b80 20 client.16251 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.213+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2091767350 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5629f4134170 0x5629f4136590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.213+0000 7fb4d5239b80 4 client.16251 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:33.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.213+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2091767350 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629f404a9d0 con 0x5629f4134170 2024-05-09T04:54:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.213+0000 7fb4bffff700 1 --2- 172.21.15.46:0/2091767350 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5629f4134170 0x5629f4136590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.214+0000 7fb4bffff700 1 --2- 172.21.15.46:0/2091767350 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5629f4134170 0x5629f4136590 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4b1a35d21f59d03 server_cookie=ae7efc64ebdf6460 in_seq=0 out_seq=0 2024-05-09T04:54:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.214+0000 7fb49f7fe700 10 client.16251 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.214+0000 7fb49f7fe700 1 -- 172.21.15.46:0/2091767350 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5629f404a9d0 con 0x5629f4134170 2024-05-09T04:54:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.214+0000 7fb49f7fe700 10 client.16251 handle_command_reply: tid=0 2024-05-09T04:54:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.215+0000 7fb4d5239b80 1 client.16251 shutdown 2024-05-09T04:54:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.215+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2091767350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f411a6a0 msgr2=0x5629f4124d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.215+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2091767350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f411a6a0 0x5629f4124d50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fb4b80a5060 tx=0x7fb4b80a5480 comp rx=0 tx=0).stop 2024-05-09T04:54:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.215+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2091767350 shutdown_connections 2024-05-09T04:54:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.215+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2091767350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f411a6a0 0x5629f4124d50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.215+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2091767350 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5629f4134170 0x5629f4136590 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.215+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2091767350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f3f9d4a0 0x5629f41242d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.215+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2091767350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f4087160 0x5629f4124810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.215+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2091767350 >> 172.21.15.46:0/2091767350 conn(0x5629f3ffd6a0 msgr2=0x5629f41151c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:33.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.215+0000 7fb49f7fe700 0 client.16251 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.215+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2091767350 shutdown_connections 2024-05-09T04:54:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.215+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2091767350 wait complete. 2024-05-09T04:54:33.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.215+0000 7fb4d5239b80 20 client.16251 trim_cache size 0 max 16384 2024-05-09T04:54:33.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.235+0000 7fb4d5239b80 1 Processor -- start 2024-05-09T04:54:33.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.235+0000 7fb4d5239b80 1 -- start start 2024-05-09T04:54:33.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4d5239b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f3f9d4a0 0x5629f4114100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4d5239b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 0x5629f41232f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4d5239b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f411a6a0 0x5629f41395b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4d5239b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c01b3a40 con 0x5629f3f9d4a0 2024-05-09T04:54:33.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4d5239b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c01b12d0 con 0x5629f4087160 2024-05-09T04:54:33.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4d5239b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c01af120 con 0x5629f411a6a0 2024-05-09T04:54:33.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4bffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f3f9d4a0 0x5629f4114100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4bffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f3f9d4a0 0x5629f4114100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48218/0 (socket says 172.21.15.46:48218) 2024-05-09T04:54:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4bffff700 1 -- 172.21.15.46:0/401546952 learned_addr learned my addr 172.21.15.46:0/401546952 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4b7fff700 1 --2- 172.21.15.46:0/401546952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 0x5629f41232f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4bffff700 1 -- 172.21.15.46:0/401546952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f411a6a0 msgr2=0x5629f41395b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4c4dc0700 1 --2- 172.21.15.46:0/401546952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f411a6a0 0x5629f41395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4bffff700 1 --2- 172.21.15.46:0/401546952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f411a6a0 0x5629f41395b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4bffff700 1 -- 172.21.15.46:0/401546952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 msgr2=0x5629f41232f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4bffff700 1 --2- 172.21.15.46:0/401546952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 0x5629f41232f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4bffff700 1 -- 172.21.15.46:0/401546952 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f411fd00 con 0x5629f3f9d4a0 2024-05-09T04:54:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4b7fff700 1 --2- 172.21.15.46:0/401546952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 0x5629f41232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:54:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.236+0000 7fb4bffff700 1 --2- 172.21.15.46:0/401546952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f3f9d4a0 0x5629f4114100 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000e930 tx=0x7fb4b0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29156d5f5729e7e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.237+0000 7fb4b5ffb700 1 -- 172.21.15.46:0/401546952 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b001b040 con 0x5629f3f9d4a0 2024-05-09T04:54:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.237+0000 7fb4b5ffb700 1 -- 172.21.15.46:0/401546952 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b002f070 con 0x5629f3f9d4a0 2024-05-09T04:54:33.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.237+0000 7fb4b5ffb700 1 -- 172.21.15.46:0/401546952 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0015790 con 0x5629f3f9d4a0 2024-05-09T04:54:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.237+0000 7fb4d5239b80 1 -- 172.21.15.46:0/401546952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f3f9d4a0 msgr2=0x5629f4114100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.237+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/401546952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f3f9d4a0 0x5629f4114100 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fb4b000e930 tx=0x7fb4b0015000 comp rx=0 tx=0).stop 2024-05-09T04:54:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.237+0000 7fb4d5239b80 1 -- 172.21.15.46:0/401546952 shutdown_connections 2024-05-09T04:54:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.237+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/401546952 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f411a6a0 0x5629f41395b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.237+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/401546952 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f3f9d4a0 0x5629f4114100 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.237+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/401546952 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 0x5629f41232f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.237+0000 7fb4d5239b80 1 -- 172.21.15.46:0/401546952 >> 172.21.15.46:0/401546952 conn(0x5629f3ffd6a0 msgr2=0x5629f4122be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.237+0000 7fb4d5239b80 1 -- 172.21.15.46:0/401546952 shutdown_connections 2024-05-09T04:54:33.244 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.238+0000 7fb4d5239b80 1 -- 172.21.15.46:0/401546952 wait complete. 2024-05-09T04:54:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.238+0000 7fb4d5239b80 1 -- start start 2024-05-09T04:54:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.238+0000 7fb4d5239b80 1 Processor -- start 2024-05-09T04:54:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.238+0000 7fb4d5239b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f3f9d4a0 0x5629f4137b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.238+0000 7fb4d5239b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 0x5629f41380b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.238+0000 7fb4d5239b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f411a6a0 0x5629f41385f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.238+0000 7fb4d5239b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c01af790 con 0x5629f3f9d4a0 2024-05-09T04:54:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.238+0000 7fb4d5239b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c01b0c00 con 0x5629f4087160 2024-05-09T04:54:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.238+0000 7fb4d5239b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb4c01afe60 con 0x5629f411a6a0 2024-05-09T04:54:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb4d5239b80 10 client.0 _ll_register_callbacks cb 0x5629f3eae6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb4bffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f3f9d4a0 0x5629f4137b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb4bffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f3f9d4a0 0x5629f4137b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48220/0 (socket says 172.21.15.46:48220) 2024-05-09T04:54:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb4bffff700 1 -- 172.21.15.46:0/2763356670 learned_addr learned my addr 172.21.15.46:0/2763356670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:33.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb4b7fff700 1 --2- 172.21.15.46:0/2763356670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 0x5629f41380b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb4c4dc0700 1 --2- 172.21.15.46:0/2763356670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f411a6a0 0x5629f41385f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb4bffff700 1 -- 172.21.15.46:0/2763356670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f411a6a0 msgr2=0x5629f41385f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb4bffff700 1 --2- 172.21.15.46:0/2763356670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f411a6a0 0x5629f41385f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb4bffff700 1 -- 172.21.15.46:0/2763356670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 msgr2=0x5629f41380b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb4bffff700 1 --2- 172.21.15.46:0/2763356670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 0x5629f41380b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb4bffff700 1 -- 172.21.15.46:0/2763356670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f419d6c0 con 0x5629f3f9d4a0 2024-05-09T04:54:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb4bffff700 1 --2- 172.21.15.46:0/2763356670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f3f9d4a0 0x5629f4137b70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009270 tx=0x7fb4b0004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6dece3fdff62ddd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb49f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb49f7fe700 1 -- 172.21.15.46:0/2763356670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b0029a60 con 0x5629f3f9d4a0 2024-05-09T04:54:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb49f7fe700 1 -- 172.21.15.46:0/2763356670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fb4b001b040 con 0x5629f3f9d4a0 2024-05-09T04:54:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.239+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2763356670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4b002f070 con 0x5629f3f9d4a0 2024-05-09T04:54:33.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.240+0000 7fb49f7fe700 1 -- 172.21.15.46:0/2763356670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fb4b0041050 con 0x5629f3f9d4a0 2024-05-09T04:54:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.240+0000 7fb4d5239b80 10 client.16257 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.240+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2763356670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5629f419e6d0 con 0x5629f3f9d4a0 2024-05-09T04:54:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.240+0000 7fb49f7fe700 1 -- 172.21.15.46:0/2763356670 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4b002f070 con 0x5629f3f9d4a0 2024-05-09T04:54:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.240+0000 7fb49f7fe700 1 -- 172.21.15.46:0/2763356670 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fb4b0029ce0 con 0x5629f3f9d4a0 2024-05-09T04:54:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.240+0000 7fb4d5239b80 10 client.16257 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.240+0000 7fb4d5239b80 10 client.16257 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.240+0000 7fb4d5239b80 20 client.16257 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.240+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2763356670 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5629f419edd0 0x5629f41a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.240+0000 7fb4d5239b80 4 client.16257 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:54:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.240+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2763356670 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5629f4044a70 con 0x5629f419edd0 2024-05-09T04:54:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.240+0000 7fb4b7fff700 1 --2- 172.21.15.46:0/2763356670 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5629f419edd0 0x5629f41a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:33.247 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb4b7fff700 1 --2- 172.21.15.46:0/2763356670 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5629f419edd0 0x5629f41a11f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17a55dcef4ce64be server_cookie=ce2323799152d18e in_seq=0 out_seq=0 2024-05-09T04:54:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb49f7fe700 10 client.16257 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb49f7fe700 1 -- 172.21.15.46:0/2763356670 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5629f4044a70 con 0x5629f419edd0 2024-05-09T04:54:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb49f7fe700 10 client.16257 handle_command_reply: tid=0 2024-05-09T04:54:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb4d5239b80 1 client.16257 shutdown 2024-05-09T04:54:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2763356670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f3f9d4a0 msgr2=0x5629f4137b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:33.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2763356670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f3f9d4a0 0x5629f4137b70 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009270 tx=0x7fb4b0004860 comp rx=0 tx=0).stop 2024-05-09T04:54:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2763356670 shutdown_connections 2024-05-09T04:54:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2763356670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5629f411a6a0 0x5629f41385f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2763356670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5629f3f9d4a0 0x5629f4137b70 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2763356670 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5629f419edd0 0x5629f41a11f0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb4d5239b80 1 --2- 172.21.15.46:0/2763356670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5629f4087160 0x5629f41380b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb49f7fe700 0 client.16257 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2763356670 >> 172.21.15.46:0/2763356670 conn(0x5629f3ffd6a0 msgr2=0x5629f4119690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2763356670 shutdown_connections 2024-05-09T04:54:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb4d5239b80 1 -- 172.21.15.46:0/2763356670 wait complete. 2024-05-09T04:54:33.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.241+0000 7fb4d5239b80 20 client.16257 trim_cache size 0 max 16384 2024-05-09T04:54:33.251 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:54:33.251 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:54:33.251 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:54:33.251 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:54:33.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.242+0000 7fb4c6823700 1 -- 172.21.15.46:0/1394705616 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb4ac0b29f0 msgr2=0x7fb4ac0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:33.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.242+0000 7fb4c6823700 1 --2- 172.21.15.46:0/1394705616 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb4ac0b29f0 0x7fb4ac0b4f00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0009a10 tx=0x7fb4b0009190 comp rx=0 tx=0).stop 2024-05-09T04:54:33.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.242+0000 7fb4c6823700 1 -- 172.21.15.46:0/1394705616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4c0108760 msgr2=0x7fb4c00ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:33.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.242+0000 7fb4c6823700 1 --2- 172.21.15.46:0/1394705616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4c0108760 0x7fb4c00ffe90 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fb4b8007ee0 tx=0x7fb4b800d3b0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.242+0000 7fb4c6823700 1 -- 172.21.15.46:0/1394705616 shutdown_connections 2024-05-09T04:54:33.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.242+0000 7fb4c6823700 1 --2- 172.21.15.46:0/1394705616 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb4ac0b29f0 0x7fb4ac0b4f00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.242+0000 7fb4c6823700 1 --2- 172.21.15.46:0/1394705616 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb4c0108760 0x7fb4c00ffe90 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.242+0000 7fb4c6823700 1 --2- 172.21.15.46:0/1394705616 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb4c01034e0 0x7fb4c00ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.242+0000 7fb4c6823700 1 --2- 172.21.15.46:0/1394705616 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb4c0107e10 0x7fb4c00ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:33.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.242+0000 7fb4c6823700 1 -- 172.21.15.46:0/1394705616 >> 172.21.15.46:0/1394705616 conn(0x7fb4c00fef30 msgr2=0x7fb4c006c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:33.252 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.242+0000 7fb4c6823700 1 -- 172.21.15.46:0/1394705616 shutdown_connections 2024-05-09T04:54:33.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:33.242+0000 7fb4c6823700 1 -- 172.21.15.46:0/1394705616 wait complete. 2024-05-09T04:54:34.026 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:54:34.026 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:54:34.282 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:34 smithi110 ceph-mon[34904]: pgmap v107: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-09T04:54:35.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:34 smithi046 ceph-mon[30136]: pgmap v107: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-09T04:54:35.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:34 smithi072 ceph-mon[34088]: pgmap v107: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 1.2 KiB/s wr, 7 op/s 2024-05-09T04:54:36.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.083+0000 7f84d4614700 1 -- 172.21.15.46:0/3807686515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84cc1045f0 msgr2=0x7f84cc104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:36.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.083+0000 7f84d4614700 1 --2- 172.21.15.46:0/3807686515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84cc1045f0 0x7f84cc104a00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f84bc00b0d0 tx=0x7f84bc00b3e0 comp rx=0 tx=0).stop 2024-05-09T04:54:36.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.083+0000 7f84d4614700 1 -- 172.21.15.46:0/3807686515 shutdown_connections 2024-05-09T04:54:36.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.083+0000 7f84d4614700 1 --2- 172.21.15.46:0/3807686515 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f84cc109870 0x7f84cc1111d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:36.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.083+0000 7f84d4614700 1 --2- 172.21.15.46:0/3807686515 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84cc1045f0 0x7f84cc104a00 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:36.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.083+0000 7f84d4614700 1 --2- 172.21.15.46:0/3807686515 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84cc108f20 0x7f84cc109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:36.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.083+0000 7f84d4614700 1 -- 172.21.15.46:0/3807686515 >> 172.21.15.46:0/3807686515 conn(0x7f84cc100040 msgr2=0x7f84cc102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:36.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.083+0000 7f84d4614700 1 -- 172.21.15.46:0/3807686515 shutdown_connections 2024-05-09T04:54:36.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.083+0000 7f84d4614700 1 -- 172.21.15.46:0/3807686515 wait complete. 2024-05-09T04:54:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.084+0000 7f84d4614700 1 Processor -- start 2024-05-09T04:54:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.084+0000 7f84d4614700 1 -- start start 2024-05-09T04:54:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.084+0000 7f84d4614700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84cc1045f0 0x7f84cc075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.084+0000 7f84d4614700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f84cc108f20 0x7f84cc076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.084+0000 7f84d4614700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84cc109870 0x7f84cc076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.084+0000 7f84d4614700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc1144f0 con 0x7f84cc109870 2024-05-09T04:54:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.084+0000 7f84d4614700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc114370 con 0x7f84cc1045f0 2024-05-09T04:54:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.084+0000 7f84d4614700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f84cc114670 con 0x7f84cc108f20 2024-05-09T04:54:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.084+0000 7f84d2bb1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84cc109870 0x7f84cc076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.084+0000 7f84d2bb1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84cc109870 0x7f84cc076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53382/0 (socket says 172.21.15.46:53382) 2024-05-09T04:54:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.084+0000 7f84d2bb1700 1 -- 172.21.15.46:0/4116714040 learned_addr learned my addr 172.21.15.46:0/4116714040 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.084+0000 7f84d1baf700 1 --2- 172.21.15.46:0/4116714040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f84cc108f20 0x7f84cc076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:36.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.085+0000 7f84d23b0700 1 --2- 172.21.15.46:0/4116714040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84cc1045f0 0x7f84cc075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:36.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.085+0000 7f84d2bb1700 1 -- 172.21.15.46:0/4116714040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f84cc108f20 msgr2=0x7f84cc076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:36.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.085+0000 7f84d2bb1700 1 --2- 172.21.15.46:0/4116714040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f84cc108f20 0x7f84cc076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:36.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.085+0000 7f84d2bb1700 1 -- 172.21.15.46:0/4116714040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84cc1045f0 msgr2=0x7f84cc075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:36.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.085+0000 7f84d2bb1700 1 --2- 172.21.15.46:0/4116714040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84cc1045f0 0x7f84cc075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:36.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.085+0000 7f84d2bb1700 1 -- 172.21.15.46:0/4116714040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84cc1a91d0 con 0x7f84cc109870 2024-05-09T04:54:36.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.085+0000 7f84d23b0700 1 --2- 172.21.15.46:0/4116714040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84cc1045f0 0x7f84cc075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:36.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.085+0000 7f84d2bb1700 1 --2- 172.21.15.46:0/4116714040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84cc109870 0x7f84cc076540 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f84c400e940 tx=0x7f84c400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:36.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.085+0000 7f84cb7fe700 1 -- 172.21.15.46:0/4116714040 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c400cbd0 con 0x7f84cc109870 2024-05-09T04:54:36.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.085+0000 7f84cb7fe700 1 -- 172.21.15.46:0/4116714040 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84c400ee90 con 0x7f84cc109870 2024-05-09T04:54:36.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.085+0000 7f84d4614700 1 -- 172.21.15.46:0/4116714040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84cc1a94c0 con 0x7f84cc109870 2024-05-09T04:54:36.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.085+0000 7f84d4614700 1 -- 172.21.15.46:0/4116714040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84cc1a99b0 con 0x7f84cc109870 2024-05-09T04:54:36.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.085+0000 7f84cb7fe700 1 -- 172.21.15.46:0/4116714040 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4005590 con 0x7f84cc109870 2024-05-09T04:54:36.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.086+0000 7f84d4614700 1 -- 172.21.15.46:0/4116714040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84cc0512c0 con 0x7f84cc109870 2024-05-09T04:54:36.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.088+0000 7f84cb7fe700 1 -- 172.21.15.46:0/4116714040 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f84c4004750 con 0x7f84cc109870 2024-05-09T04:54:36.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.089+0000 7f84cb7fe700 1 --2- 172.21.15.46:0/4116714040 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f84b40b29f0 0x7f84b40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:36.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.089+0000 7f84d23b0700 1 --2- 172.21.15.46:0/4116714040 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f84b40b29f0 0x7f84b40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:36.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.089+0000 7f84cb7fe700 1 -- 172.21.15.46:0/4116714040 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f84c40e2ad0 con 0x7f84cc109870 2024-05-09T04:54:36.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.089+0000 7f84d23b0700 1 --2- 172.21.15.46:0/4116714040 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f84b40b29f0 0x7f84b40b4f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009fd0 tx=0x7f84bc00b040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:36.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.093+0000 7f84cb7fe700 1 -- 172.21.15.46:0/4116714040 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f84c4054df0 con 0x7f84cc109870 2024-05-09T04:54:36.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.354+0000 7f84d4614700 1 -- 172.21.15.46:0/4116714040 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84cc02e5f0 con 0x7f84cc109870 2024-05-09T04:54:36.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.355+0000 7f84cb7fe700 1 -- 172.21.15.46:0/4116714040 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7f84c40aa430 con 0x7f84cc109870 2024-05-09T04:54:36.355 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:54:36.355 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:54:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.357+0000 7f84d4614700 1 -- 172.21.15.46:0/4116714040 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f84b40b29f0 msgr2=0x7f84b40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:36.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.357+0000 7f84d4614700 1 --2- 172.21.15.46:0/4116714040 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f84b40b29f0 0x7f84b40b4f00 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009fd0 tx=0x7f84bc00b040 comp rx=0 tx=0).stop 2024-05-09T04:54:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.357+0000 7f84d4614700 1 -- 172.21.15.46:0/4116714040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84cc109870 msgr2=0x7f84cc076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.357+0000 7f84d4614700 1 --2- 172.21.15.46:0/4116714040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84cc109870 0x7f84cc076540 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f84c400e940 tx=0x7f84c400ec50 comp rx=0 tx=0).stop 2024-05-09T04:54:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.357+0000 7f84d4614700 1 -- 172.21.15.46:0/4116714040 shutdown_connections 2024-05-09T04:54:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.357+0000 7f84d4614700 1 --2- 172.21.15.46:0/4116714040 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f84b40b29f0 0x7f84b40b4f00 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.357+0000 7f84d4614700 1 --2- 172.21.15.46:0/4116714040 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f84cc109870 0x7f84cc076540 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.357+0000 7f84d4614700 1 --2- 172.21.15.46:0/4116714040 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f84cc1045f0 0x7f84cc075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.357+0000 7f84d4614700 1 --2- 172.21.15.46:0/4116714040 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f84cc108f20 0x7f84cc076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.357+0000 7f84d4614700 1 -- 172.21.15.46:0/4116714040 >> 172.21.15.46:0/4116714040 conn(0x7f84cc100040 msgr2=0x7f84cc108690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.357+0000 7f84d4614700 1 -- 172.21.15.46:0/4116714040 shutdown_connections 2024-05-09T04:54:36.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:36.357+0000 7f84d4614700 1 -- 172.21.15.46:0/4116714040 wait complete. 2024-05-09T04:54:36.359 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:54:37.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:36 smithi110 ceph-mon[34904]: pgmap v108: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-09T04:54:37.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:36 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/4116714040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:54:37.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:36 smithi046 ceph-mon[30136]: pgmap v108: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-09T04:54:37.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:36 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/4116714040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:54:37.111 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:54:37.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:36 smithi072 ceph-mon[34088]: pgmap v108: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-05-09T04:54:37.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:36 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/4116714040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:54:37.363 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:38.921 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:38 smithi046 ceph-mon[30136]: pgmap v109: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-09T04:54:39.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:38 smithi110 ceph-mon[34904]: pgmap v109: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-09T04:54:39.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:38 smithi072 ceph-mon[34088]: pgmap v109: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-05-09T04:54:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.190+0000 7f759336c700 1 -- 172.21.15.46:0/835647448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f758c1045d0 msgr2=0x7f758c1049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.190+0000 7f759336c700 1 --2- 172.21.15.46:0/835647448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f758c1045d0 0x7f758c1049e0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f75880099c0 tx=0x7f7588009cd0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.190+0000 7f759336c700 1 -- 172.21.15.46:0/835647448 shutdown_connections 2024-05-09T04:54:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.190+0000 7f759336c700 1 --2- 172.21.15.46:0/835647448 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f758c109850 0x7f758c1111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.190+0000 7f759336c700 1 --2- 172.21.15.46:0/835647448 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f758c1045d0 0x7f758c1049e0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.190+0000 7f759336c700 1 --2- 172.21.15.46:0/835647448 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f758c108f00 0x7f758c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.190+0000 7f759336c700 1 -- 172.21.15.46:0/835647448 >> 172.21.15.46:0/835647448 conn(0x7f758c0ffea0 msgr2=0x7f758c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.190+0000 7f759336c700 1 -- 172.21.15.46:0/835647448 shutdown_connections 2024-05-09T04:54:39.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.190+0000 7f759336c700 1 -- 172.21.15.46:0/835647448 wait complete. 2024-05-09T04:54:39.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.191+0000 7f759336c700 1 Processor -- start 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.191+0000 7f759336c700 1 -- start start 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.191+0000 7f759336c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f758c1045d0 0x7f758c19aec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.191+0000 7f759336c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f758c108f00 0x7f758c19b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.191+0000 7f759336c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f758c109850 0x7f758c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.191+0000 7f759336c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f758c1144d0 con 0x7f758c108f00 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.191+0000 7f759336c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f758c114350 con 0x7f758c1045d0 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.191+0000 7f759336c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f758c114650 con 0x7f758c109850 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.191+0000 7f7590907700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f758c108f00 0x7f758c19b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.191+0000 7f7590907700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f758c108f00 0x7f758c19b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53402/0 (socket says 172.21.15.46:53402) 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.191+0000 7f7590907700 1 -- 172.21.15.46:0/4028333005 learned_addr learned my addr 172.21.15.46:0/4028333005 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.192+0000 7f7591108700 1 --2- 172.21.15.46:0/4028333005 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f758c1045d0 0x7f758c19aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.192+0000 7f7591909700 1 --2- 172.21.15.46:0/4028333005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f758c109850 0x7f758c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.192+0000 7f7590907700 1 -- 172.21.15.46:0/4028333005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f758c109850 msgr2=0x7f758c19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.192+0000 7f7590907700 1 --2- 172.21.15.46:0/4028333005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f758c109850 0x7f758c19f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.192+0000 7f7590907700 1 -- 172.21.15.46:0/4028333005 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f758c1045d0 msgr2=0x7f758c19aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.192+0000 7f7590907700 1 --2- 172.21.15.46:0/4028333005 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f758c1045d0 0x7f758c19aec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.192+0000 7f7590907700 1 -- 172.21.15.46:0/4028333005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f758c19fd90 con 0x7f758c108f00 2024-05-09T04:54:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.192+0000 7f7591909700 1 --2- 172.21.15.46:0/4028333005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f758c109850 0x7f758c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.192+0000 7f7590907700 1 --2- 172.21.15.46:0/4028333005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f758c108f00 0x7f758c19b460 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f757c00e970 tx=0x7f757c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.192+0000 7f75827fc700 1 -- 172.21.15.46:0/4028333005 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c00cbd0 con 0x7f758c108f00 2024-05-09T04:54:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.192+0000 7f759336c700 1 -- 172.21.15.46:0/4028333005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f758c1a0080 con 0x7f758c108f00 2024-05-09T04:54:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.192+0000 7f75827fc700 1 -- 172.21.15.46:0/4028333005 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f757c00cd70 con 0x7f758c108f00 2024-05-09T04:54:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.192+0000 7f75827fc700 1 -- 172.21.15.46:0/4028333005 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f757c005680 con 0x7f758c108f00 2024-05-09T04:54:39.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.192+0000 7f759336c700 1 -- 172.21.15.46:0/4028333005 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f758c076b00 con 0x7f758c108f00 2024-05-09T04:54:39.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.195+0000 7f75827fc700 1 -- 172.21.15.46:0/4028333005 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f757c004750 con 0x7f758c108f00 2024-05-09T04:54:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.196+0000 7f75827fc700 1 --2- 172.21.15.46:0/4028333005 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f75780b29f0 0x7f75780b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.196+0000 7f7591108700 1 --2- 172.21.15.46:0/4028333005 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f75780b29f0 0x7f75780b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.196+0000 7f75827fc700 1 -- 172.21.15.46:0/4028333005 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f757c0e28b0 con 0x7f758c108f00 2024-05-09T04:54:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.197+0000 7f7591108700 1 --2- 172.21.15.46:0/4028333005 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f75780b29f0 0x7f75780b4f00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7588000c00 tx=0x7f758801e040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.197+0000 7f75a1d82b80 1 Processor -- start 2024-05-09T04:54:39.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.197+0000 7f75a1d82b80 1 -- start start 2024-05-09T04:54:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.197+0000 7f75a1d82b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845ed240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.197+0000 7f75a1d82b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 0x55dc845ed780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.197+0000 7f75a1d82b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845e7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.197+0000 7f75a1d82b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dc8450a0a0 con 0x55dc845e7400 2024-05-09T04:54:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.197+0000 7f75a1d82b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55dc844af9a0 con 0x55dc8446a4a0 2024-05-09T04:54:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.197+0000 7f75a1d82b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dc84506060 con 0x55dc844e4d00 2024-05-09T04:54:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.198+0000 7f7591909700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845e7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.198+0000 7f7591909700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845e7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53406/0 (socket says 172.21.15.46:53406) 2024-05-09T04:54:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.198+0000 7f7591909700 1 -- 172.21.15.46:0/3584869138 learned_addr learned my addr 172.21.15.46:0/3584869138 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.198+0000 7f7590907700 1 --2- 172.21.15.46:0/3584869138 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 0x55dc845ed780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.198+0000 7f7591108700 1 --2- 172.21.15.46:0/3584869138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845ed240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.198+0000 7f7591909700 1 -- 172.21.15.46:0/3584869138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 msgr2=0x55dc845ed240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.198+0000 7f7591909700 1 --2- 172.21.15.46:0/3584869138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845ed240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.198+0000 7f7591909700 1 -- 172.21.15.46:0/3584869138 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 msgr2=0x55dc845ed780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.198+0000 7f7591909700 1 --2- 172.21.15.46:0/3584869138 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 0x55dc845ed780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.198+0000 7f7591909700 1 -- 172.21.15.46:0/3584869138 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc844a0500 con 0x55dc845e7400 2024-05-09T04:54:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.198+0000 7f7590907700 1 --2- 172.21.15.46:0/3584869138 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 0x55dc845ed780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.198+0000 7f7591909700 1 --2- 172.21.15.46:0/3584869138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845e7810 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7584009dd0 tx=0x7f75840077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=969ee3a4586d6118 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.198+0000 7f75777fe700 1 -- 172.21.15.46:0/3584869138 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584004bb0 con 0x55dc845e7400 2024-05-09T04:54:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.198+0000 7f75777fe700 1 -- 172.21.15.46:0/3584869138 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7584004d50 con 0x55dc845e7400 2024-05-09T04:54:39.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.198+0000 7f75777fe700 1 -- 172.21.15.46:0/3584869138 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584014630 con 0x55dc845e7400 2024-05-09T04:54:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.199+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3584869138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 msgr2=0x55dc845e7810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.199+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3584869138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845e7810 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7584009dd0 tx=0x7f75840077e0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.199+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3584869138 shutdown_connections 2024-05-09T04:54:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.199+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3584869138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845e7810 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.199+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3584869138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845ed240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.199+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3584869138 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 0x55dc845ed780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.200+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3584869138 >> 172.21.15.46:0/3584869138 conn(0x55dc8446df70 msgr2=0x55dc845e2220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.200+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3584869138 shutdown_connections 2024-05-09T04:54:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.200+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3584869138 wait complete. 2024-05-09T04:54:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.200+0000 7f75a1d82b80 1 -- start start 2024-05-09T04:54:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.200+0000 7f75a1d82b80 1 Processor -- start 2024-05-09T04:54:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f75a1d82b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc8446a4a0 0x55dc845f0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f75a1d82b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845f14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f75a1d82b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc845e7400 0x55dc845f19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f75a1d82b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dc845e8160 con 0x55dc8446a4a0 2024-05-09T04:54:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f75a1d82b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55dc844a4b20 con 0x55dc845e7400 2024-05-09T04:54:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f75a1d82b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dc845f4020 con 0x55dc844e4d00 2024-05-09T04:54:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f75a1d82b80 10 client.0 _ll_register_callbacks cb 0x55dc8437b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f7591108700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc8446a4a0 0x55dc845f0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f7591108700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc8446a4a0 0x55dc845f0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53416/0 (socket says 172.21.15.46:53416) 2024-05-09T04:54:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f7591108700 1 -- 172.21.15.46:0/3819740055 learned_addr learned my addr 172.21.15.46:0/3819740055 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f7591909700 1 --2- 172.21.15.46:0/3819740055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc845e7400 0x55dc845f19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f7590907700 1 --2- 172.21.15.46:0/3819740055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845f14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f7591108700 1 -- 172.21.15.46:0/3819740055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 msgr2=0x55dc845f14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f7591108700 1 --2- 172.21.15.46:0/3819740055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845f14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f7591108700 1 -- 172.21.15.46:0/3819740055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc845e7400 msgr2=0x55dc845f19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f7591108700 1 --2- 172.21.15.46:0/3819740055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc845e7400 0x55dc845f19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f7591108700 1 -- 172.21.15.46:0/3819740055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc84385490 con 0x55dc8446a4a0 2024-05-09T04:54:39.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f7591108700 1 --2- 172.21.15.46:0/3819740055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc8446a4a0 0x55dc845f0f60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f75880098e0 tx=0x7f7588009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=746c20bcfc03ed4c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.201+0000 7f7574ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.202+0000 7f7574ff9700 1 -- 172.21.15.46:0/3819740055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7588003c90 con 0x55dc8446a4a0 2024-05-09T04:54:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.202+0000 7f7574ff9700 1 -- 172.21.15.46:0/3819740055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f758800b730 con 0x55dc8446a4a0 2024-05-09T04:54:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.202+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3819740055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7584004d50 con 0x55dc8446a4a0 2024-05-09T04:54:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.202+0000 7f7574ff9700 1 -- 172.21.15.46:0/3819740055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7588003e30 con 0x55dc8446a4a0 2024-05-09T04:54:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.202+0000 7f75a1d82b80 10 client.16275 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.202+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3819740055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dc845fe930 con 0x55dc8446a4a0 2024-05-09T04:54:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.202+0000 7f7574ff9700 1 -- 172.21.15.46:0/3819740055 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7584004d50 con 0x55dc8446a4a0 2024-05-09T04:54:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.202+0000 7f7574ff9700 1 -- 172.21.15.46:0/3819740055 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f75880348d0 con 0x55dc8446a4a0 2024-05-09T04:54:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.202+0000 7f75a1d82b80 10 client.16275 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.202+0000 7f75a1d82b80 10 client.16275 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:54:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.202+0000 7f75a1d82b80 20 client.16275 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.202+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3819740055 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55dc84600fc0 0x55dc84603400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.202+0000 7f75a1d82b80 4 client.16275 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:39.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.202+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3819740055 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc84517a80 con 0x55dc84600fc0 2024-05-09T04:54:39.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.203+0000 7f7590907700 1 --2- 172.21.15.46:0/3819740055 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55dc84600fc0 0x55dc84603400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.203+0000 7f7590907700 1 --2- 172.21.15.46:0/3819740055 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55dc84600fc0 0x55dc84603400 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=63d70f06e8dc5a08 server_cookie=21303d304741e537 in_seq=0 out_seq=0 2024-05-09T04:54:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.203+0000 7f7574ff9700 10 client.16275 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:54:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.204+0000 7f7574ff9700 1 -- 172.21.15.46:0/3819740055 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dc84517a80 con 0x55dc84600fc0 2024-05-09T04:54:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.204+0000 7f7574ff9700 10 client.16275 handle_command_reply: tid=0 2024-05-09T04:54:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.204+0000 7f75a1d82b80 1 client.16275 shutdown 2024-05-09T04:54:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.204+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3819740055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc8446a4a0 msgr2=0x55dc845f0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.204+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3819740055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc8446a4a0 0x55dc845f0f60 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f75880098e0 tx=0x7f7588009cd0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.204+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3819740055 shutdown_connections 2024-05-09T04:54:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.204+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3819740055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc845e7400 0x55dc845f19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.204+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3819740055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc8446a4a0 0x55dc845f0f60 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.204+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3819740055 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55dc84600fc0 0x55dc84603400 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.204+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3819740055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845f14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.204+0000 7f7574ff9700 0 client.16275 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:54:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.204+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3819740055 >> 172.21.15.46:0/3819740055 conn(0x55dc8446df70 msgr2=0x55dc845dbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.204+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3819740055 shutdown_connections 2024-05-09T04:54:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.204+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3819740055 wait complete. 2024-05-09T04:54:39.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.204+0000 7f75a1d82b80 20 client.16275 trim_cache size 0 max 16384 2024-05-09T04:54:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f75a1d82b80 1 Processor -- start 2024-05-09T04:54:39.220 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f75a1d82b80 1 -- start start 2024-05-09T04:54:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f75a1d82b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 0x55dc845f1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f75a1d82b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845dc700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f75a1d82b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845fcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f75a1d82b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f758c1af710 con 0x55dc845e7400 2024-05-09T04:54:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f75a1d82b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f758c1acfa0 con 0x55dc8446a4a0 2024-05-09T04:54:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f75a1d82b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f758c1aadf0 con 0x55dc844e4d00 2024-05-09T04:54:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f7591909700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845fcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f7590907700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845dc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f7591909700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845fcdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53426/0 (socket says 172.21.15.46:53426) 2024-05-09T04:54:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f7591909700 1 -- 172.21.15.46:0/777474414 learned_addr learned my addr 172.21.15.46:0/777474414 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f7591108700 1 --2- 172.21.15.46:0/777474414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 0x55dc845f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f7591909700 1 -- 172.21.15.46:0/777474414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 msgr2=0x55dc845dc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.221 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f7591909700 1 --2- 172.21.15.46:0/777474414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845dc700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f7591909700 1 -- 172.21.15.46:0/777474414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 msgr2=0x55dc845f1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f7591909700 1 --2- 172.21.15.46:0/777474414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 0x55dc845f1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.220+0000 7f7591909700 1 -- 172.21.15.46:0/777474414 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc845e8600 con 0x55dc845e7400 2024-05-09T04:54:39.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.221+0000 7f7591909700 1 --2- 172.21.15.46:0/777474414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845fcdd0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7584017000 tx=0x7f7584005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62a572e72da46eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.221+0000 7f7576ffd700 1 -- 172.21.15.46:0/777474414 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7584006000 con 0x55dc845e7400 2024-05-09T04:54:39.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.221+0000 7f7576ffd700 1 -- 172.21.15.46:0/777474414 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f758401e8d0 con 0x55dc845e7400 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.221+0000 7f7576ffd700 1 -- 172.21.15.46:0/777474414 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75840147e0 con 0x55dc845e7400 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.222+0000 7f75a1d82b80 1 -- 172.21.15.46:0/777474414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 msgr2=0x55dc845fcdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.222+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/777474414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845fcdd0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7584017000 tx=0x7f7584005a40 comp rx=0 tx=0).stop 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.222+0000 7f75a1d82b80 1 -- 172.21.15.46:0/777474414 shutdown_connections 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.222+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/777474414 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845fcdd0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.222+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/777474414 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 0x55dc845f1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.222+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/777474414 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845dc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.222+0000 7f75a1d82b80 1 -- 172.21.15.46:0/777474414 >> 172.21.15.46:0/777474414 conn(0x55dc8446df70 msgr2=0x55dc845dbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.222+0000 7f75a1d82b80 1 -- 172.21.15.46:0/777474414 shutdown_connections 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.222+0000 7f75a1d82b80 1 -- 172.21.15.46:0/777474414 wait complete. 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.222+0000 7f75a1d82b80 1 -- start start 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.222+0000 7f75a1d82b80 1 Processor -- start 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f75a1d82b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 0x55dc845f93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f75a1d82b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845ed800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f75a1d82b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845edd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f75a1d82b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f758c1ab460 con 0x55dc845e7400 2024-05-09T04:54:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f75a1d82b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f758c1ac8d0 con 0x55dc8446a4a0 2024-05-09T04:54:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f75a1d82b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f758c1abb30 con 0x55dc844e4d00 2024-05-09T04:54:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f75a1d82b80 10 client.0 _ll_register_callbacks cb 0x55dc8437b6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f7591909700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845edd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f7591909700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845edd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53436/0 (socket says 172.21.15.46:53436) 2024-05-09T04:54:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f7591909700 1 -- 172.21.15.46:0/3291687483 learned_addr learned my addr 172.21.15.46:0/3291687483 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f7590907700 1 --2- 172.21.15.46:0/3291687483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845ed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f7591108700 1 --2- 172.21.15.46:0/3291687483 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 0x55dc845f93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f7591909700 1 -- 172.21.15.46:0/3291687483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 msgr2=0x55dc845ed800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f7591909700 1 --2- 172.21.15.46:0/3291687483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845ed800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f7591909700 1 -- 172.21.15.46:0/3291687483 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 msgr2=0x55dc845f93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f7591909700 1 --2- 172.21.15.46:0/3291687483 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 0x55dc845f93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.223+0000 7f7591909700 1 -- 172.21.15.46:0/3291687483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc845ee280 con 0x55dc845e7400 2024-05-09T04:54:39.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.224+0000 7f7591909700 1 --2- 172.21.15.46:0/3291687483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845edd40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7584005ab0 tx=0x7f7584009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcf8389da4bd0d4b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.224+0000 7f7574ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.224+0000 7f7574ff9700 1 -- 172.21.15.46:0/3291687483 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f758401ee40 con 0x55dc845e7400 2024-05-09T04:54:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.224+0000 7f7574ff9700 1 -- 172.21.15.46:0/3291687483 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f7584033480 con 0x55dc845e7400 2024-05-09T04:54:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.224+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3291687483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f758401e8d0 con 0x55dc845e7400 2024-05-09T04:54:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.224+0000 7f7574ff9700 1 -- 172.21.15.46:0/3291687483 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7584033680 con 0x55dc845e7400 2024-05-09T04:54:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.224+0000 7f75a1d82b80 10 client.16281 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.224+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3291687483 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dc8466b3d0 con 0x55dc845e7400 2024-05-09T04:54:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.224+0000 7f7574ff9700 1 -- 172.21.15.46:0/3291687483 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f758401e8d0 con 0x55dc845e7400 2024-05-09T04:54:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.224+0000 7f7574ff9700 1 -- 172.21.15.46:0/3291687483 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7584014970 con 0x55dc845e7400 2024-05-09T04:54:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.225+0000 7f75a1d82b80 10 client.16281 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.225+0000 7f75a1d82b80 10 client.16281 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:54:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.225+0000 7f75a1d82b80 20 client.16281 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.225+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3291687483 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55dc8466b6e0 0x55dc8466db00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.225+0000 7f75a1d82b80 4 client.16281 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:54:39.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.225+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3291687483 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dc84511b20 con 0x55dc8466b6e0 2024-05-09T04:54:39.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.225+0000 7f7591108700 1 --2- 172.21.15.46:0/3291687483 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55dc8466b6e0 0x55dc8466db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:39.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.225+0000 7f7574ff9700 10 client.16281 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:54:39.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.225+0000 7f7591108700 1 --2- 172.21.15.46:0/3291687483 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55dc8466b6e0 0x55dc8466db00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b84017988e9ca5c4 server_cookie=c21b98303adde578 in_seq=0 out_seq=0 2024-05-09T04:54:39.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.226+0000 7f7574ff9700 1 -- 172.21.15.46:0/3291687483 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dc84511b20 con 0x55dc8466b6e0 2024-05-09T04:54:39.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.226+0000 7f7574ff9700 10 client.16281 handle_command_reply: tid=0 2024-05-09T04:54:39.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.226+0000 7f75a1d82b80 1 client.16281 shutdown 2024-05-09T04:54:39.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.226+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3291687483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 msgr2=0x55dc845edd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.226+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3291687483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845edd40 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7584005ab0 tx=0x7f7584009000 comp rx=0 tx=0).stop 2024-05-09T04:54:39.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.226+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3291687483 shutdown_connections 2024-05-09T04:54:39.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.226+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3291687483 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dc845e7400 0x55dc845edd40 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.226+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3291687483 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dc8446a4a0 0x55dc845f93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.226+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3291687483 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55dc8466b6e0 0x55dc8466db00 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.226+0000 7f75a1d82b80 1 --2- 172.21.15.46:0/3291687483 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dc844e4d00 0x55dc845ed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.226+0000 7f7574ff9700 0 client.16281 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:54:39.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.226+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3291687483 >> 172.21.15.46:0/3291687483 conn(0x55dc8446df70 msgr2=0x55dc845eac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:39.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.226+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3291687483 shutdown_connections 2024-05-09T04:54:39.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.226+0000 7f75a1d82b80 1 -- 172.21.15.46:0/3291687483 wait complete. 2024-05-09T04:54:39.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.226+0000 7f75a1d82b80 20 client.16281 trim_cache size 0 max 16384 2024-05-09T04:54:39.230 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:54:39.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.227+0000 7f759336c700 1 -- 172.21.15.46:0/4028333005 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f75780b29f0 msgr2=0x7f75780b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.227+0000 7f759336c700 1 --2- 172.21.15.46:0/4028333005 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f75780b29f0 0x7f75780b4f00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f7588000c00 tx=0x7f758801e040 comp rx=0 tx=0).stop 2024-05-09T04:54:39.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.227+0000 7f759336c700 1 -- 172.21.15.46:0/4028333005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f758c108f00 msgr2=0x7f758c19b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:39.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.227+0000 7f759336c700 1 --2- 172.21.15.46:0/4028333005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f758c108f00 0x7f758c19b460 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f757c00e970 tx=0x7f757c00ec80 comp rx=0 tx=0).stop 2024-05-09T04:54:39.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.227+0000 7f759336c700 1 -- 172.21.15.46:0/4028333005 shutdown_connections 2024-05-09T04:54:39.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.227+0000 7f759336c700 1 --2- 172.21.15.46:0/4028333005 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f75780b29f0 0x7f75780b4f00 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.227+0000 7f759336c700 1 --2- 172.21.15.46:0/4028333005 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f758c109850 0x7f758c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.227+0000 7f759336c700 1 --2- 172.21.15.46:0/4028333005 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f758c1045d0 0x7f758c19aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.227+0000 7f759336c700 1 --2- 172.21.15.46:0/4028333005 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f758c108f00 0x7f758c19b460 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:39.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.227+0000 7f759336c700 1 -- 172.21.15.46:0/4028333005 >> 172.21.15.46:0/4028333005 conn(0x7f758c0ffea0 msgr2=0x7f758c10a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:39.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.227+0000 7f759336c700 1 -- 172.21.15.46:0/4028333005 shutdown_connections 2024-05-09T04:54:39.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:39.227+0000 7f759336c700 1 -- 172.21.15.46:0/4028333005 wait complete. 2024-05-09T04:54:39.958 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:54:40.222 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:40 smithi110 ceph-mon[34904]: pgmap v110: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6 op/s 2024-05-09T04:54:41.075 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:40 smithi046 ceph-mon[30136]: pgmap v110: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6 op/s 2024-05-09T04:54:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:40 smithi072 ceph-mon[34088]: pgmap v110: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6 op/s 2024-05-09T04:54:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.018+0000 7f36e8bfc700 1 -- 172.21.15.46:0/2579132459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e41034c0 msgr2=0x7f36e41038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.018+0000 7f36e8bfc700 1 --2- 172.21.15.46:0/2579132459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e41034c0 0x7f36e41038d0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f36d40099c0 tx=0x7f36d4009cd0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.018+0000 7f36e8bfc700 1 -- 172.21.15.46:0/2579132459 shutdown_connections 2024-05-09T04:54:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.018+0000 7f36e8bfc700 1 --2- 172.21.15.46:0/2579132459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e4108740 0x7f36e41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.018+0000 7f36e8bfc700 1 --2- 172.21.15.46:0/2579132459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e41034c0 0x7f36e41038d0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.018+0000 7f36e8bfc700 1 --2- 172.21.15.46:0/2579132459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e4107df0 0x7f36e4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.018+0000 7f36e8bfc700 1 -- 172.21.15.46:0/2579132459 >> 172.21.15.46:0/2579132459 conn(0x7f36e40fedc0 msgr2=0x7f36e41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.018+0000 7f36e8bfc700 1 -- 172.21.15.46:0/2579132459 shutdown_connections 2024-05-09T04:54:42.019 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.018+0000 7f36e8bfc700 1 -- 172.21.15.46:0/2579132459 wait complete. 2024-05-09T04:54:42.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.019+0000 7f36e8bfc700 1 Processor -- start 2024-05-09T04:54:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.019+0000 7f36e8bfc700 1 -- start start 2024-05-09T04:54:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.019+0000 7f36e8bfc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e41034c0 0x7f36e40ff2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.019+0000 7f36e8bfc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e4107df0 0x7f36e40ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.019+0000 7f36e8bfc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e4108740 0x7f36e40ffd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.019+0000 7f36e8bfc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36e4113370 con 0x7f36e41034c0 2024-05-09T04:54:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.019+0000 7f36e8bfc700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41131f0 con 0x7f36e4108740 2024-05-09T04:54:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.019+0000 7f36e8bfc700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41134f0 con 0x7f36e4107df0 2024-05-09T04:54:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.019+0000 7f36e259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e41034c0 0x7f36e40ff2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.019+0000 7f36e259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e41034c0 0x7f36e40ff2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53448/0 (socket says 172.21.15.46:53448) 2024-05-09T04:54:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.019+0000 7f36e259c700 1 -- 172.21.15.46:0/492917746 learned_addr learned my addr 172.21.15.46:0/492917746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.020+0000 7f36e1d9b700 1 --2- 172.21.15.46:0/492917746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e4107df0 0x7f36e40ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.020+0000 7f36e2d9d700 1 --2- 172.21.15.46:0/492917746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e4108740 0x7f36e40ffd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.020+0000 7f36e259c700 1 -- 172.21.15.46:0/492917746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e4107df0 msgr2=0x7f36e40ff7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.020+0000 7f36e259c700 1 --2- 172.21.15.46:0/492917746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e4107df0 0x7f36e40ff7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.020+0000 7f36e259c700 1 -- 172.21.15.46:0/492917746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e4108740 msgr2=0x7f36e40ffd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.020+0000 7f36e259c700 1 --2- 172.21.15.46:0/492917746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e4108740 0x7f36e40ffd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.020+0000 7f36e259c700 1 -- 172.21.15.46:0/492917746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e41a9190 con 0x7f36e41034c0 2024-05-09T04:54:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.020+0000 7f36e2d9d700 1 --2- 172.21.15.46:0/492917746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e4108740 0x7f36e40ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:54:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.020+0000 7f36e259c700 1 --2- 172.21.15.46:0/492917746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e41034c0 0x7f36e40ff2a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f36d400b580 tx=0x7f36d40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.020+0000 7f36d37fe700 1 -- 172.21.15.46:0/492917746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4027070 con 0x7f36e41034c0 2024-05-09T04:54:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.020+0000 7f36e8bfc700 1 -- 172.21.15.46:0/492917746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e41a9420 con 0x7f36e41034c0 2024-05-09T04:54:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.020+0000 7f36d37fe700 1 -- 172.21.15.46:0/492917746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d4004020 con 0x7f36e41034c0 2024-05-09T04:54:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.020+0000 7f36d37fe700 1 -- 172.21.15.46:0/492917746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4022e30 con 0x7f36e41034c0 2024-05-09T04:54:42.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.020+0000 7f36e8bfc700 1 -- 172.21.15.46:0/492917746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e41a98d0 con 0x7f36e41034c0 2024-05-09T04:54:42.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.023+0000 7f36d37fe700 1 -- 172.21.15.46:0/492917746 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f36d400f460 con 0x7f36e41034c0 2024-05-09T04:54:42.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.024+0000 7f36d37fe700 1 --2- 172.21.15.46:0/492917746 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f36cc0b29f0 0x7f36cc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.024+0000 7f36e1d9b700 1 --2- 172.21.15.46:0/492917746 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f36cc0b29f0 0x7f36cc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.024+0000 7f36d37fe700 1 -- 172.21.15.46:0/492917746 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f36d40e3360 con 0x7f36e41034c0 2024-05-09T04:54:42.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.024+0000 7f36f7612b80 1 Processor -- start 2024-05-09T04:54:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36f7612b80 1 -- start start 2024-05-09T04:54:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36e1d9b700 1 --2- 172.21.15.46:0/492917746 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f36cc0b29f0 0x7f36cc0b4f00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f36d8005fd0 tx=0x7f36d800a560 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36f7612b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d92a6160 0x55e3d933f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36f7612b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d91bc4a0 0x55e3d933fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36f7612b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d93396a0 0x55e3d9339ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36f7612b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e3d925bff0 con 0x55e3d93396a0 2024-05-09T04:54:42.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36f7612b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e3d91fa7f0 con 0x55e3d92a6160 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36f7612b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e3d9257fb0 con 0x55e3d91bc4a0 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36e2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d93396a0 0x55e3d9339ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36e2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d93396a0 0x55e3d9339ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53462/0 (socket says 172.21.15.46:53462) 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36e2d9d700 1 -- 172.21.15.46:0/2456953459 learned_addr learned my addr 172.21.15.46:0/2456953459 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36e1d9b700 1 --2- 172.21.15.46:0/2456953459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d91bc4a0 0x55e3d933fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36e259c700 1 --2- 172.21.15.46:0/2456953459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d92a6160 0x55e3d933f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36e2d9d700 1 -- 172.21.15.46:0/2456953459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d91bc4a0 msgr2=0x55e3d933fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36e2d9d700 1 --2- 172.21.15.46:0/2456953459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d91bc4a0 0x55e3d933fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36e2d9d700 1 -- 172.21.15.46:0/2456953459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d92a6160 msgr2=0x55e3d933f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36e2d9d700 1 --2- 172.21.15.46:0/2456953459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d92a6160 0x55e3d933f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36e2d9d700 1 -- 172.21.15.46:0/2456953459 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3d91f2370 con 0x55e3d93396a0 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.025+0000 7f36e1d9b700 1 --2- 172.21.15.46:0/2456953459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d91bc4a0 0x55e3d933fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.026+0000 7f36e2d9d700 1 --2- 172.21.15.46:0/2456953459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d93396a0 0x55e3d9339ad0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009cd0 tx=0x7f36dc007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea02504bd3c59a7e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.026+0000 7f36d0ff9700 1 -- 172.21.15.46:0/2456953459 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc019070 con 0x55e3d93396a0 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.026+0000 7f36d0ff9700 1 -- 172.21.15.46:0/2456953459 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36dc0148b0 con 0x55e3d93396a0 2024-05-09T04:54:42.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.026+0000 7f36d0ff9700 1 -- 172.21.15.46:0/2456953459 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc013880 con 0x55e3d93396a0 2024-05-09T04:54:42.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.027+0000 7f36f7612b80 1 -- 172.21.15.46:0/2456953459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d93396a0 msgr2=0x55e3d9339ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.027+0000 7f36f7612b80 1 --2- 172.21.15.46:0/2456953459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d93396a0 0x55e3d9339ad0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f36dc009cd0 tx=0x7f36dc007500 comp rx=0 tx=0).stop 2024-05-09T04:54:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.027+0000 7f36f7612b80 1 -- 172.21.15.46:0/2456953459 shutdown_connections 2024-05-09T04:54:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.027+0000 7f36f7612b80 1 --2- 172.21.15.46:0/2456953459 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d93396a0 0x55e3d9339ad0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.027+0000 7f36f7612b80 1 --2- 172.21.15.46:0/2456953459 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d92a6160 0x55e3d933f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.027+0000 7f36f7612b80 1 --2- 172.21.15.46:0/2456953459 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d91bc4a0 0x55e3d933fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.027+0000 7f36f7612b80 1 -- 172.21.15.46:0/2456953459 >> 172.21.15.46:0/2456953459 conn(0x55e3d921c6a0 msgr2=0x55e3d9334400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.027+0000 7f36f7612b80 1 -- 172.21.15.46:0/2456953459 shutdown_connections 2024-05-09T04:54:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.027+0000 7f36f7612b80 1 -- 172.21.15.46:0/2456953459 wait complete. 2024-05-09T04:54:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.027+0000 7f36f7612b80 1 -- start start 2024-05-09T04:54:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.027+0000 7f36f7612b80 1 Processor -- start 2024-05-09T04:54:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36f7612b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d91bc4a0 0x55e3d9343090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36f7612b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d92a6160 0x55e3d93435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36f7612b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d93396a0 0x55e3d9343b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36f7612b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e3d90d7ba0 con 0x55e3d93396a0 2024-05-09T04:54:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36f7612b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e3d9216150 con 0x55e3d92a6160 2024-05-09T04:54:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36f7612b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e3d93461d0 con 0x55e3d91bc4a0 2024-05-09T04:54:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36f7612b80 10 client.0 _ll_register_callbacks cb 0x55e3d90cd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36e2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d93396a0 0x55e3d9343b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36e2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d93396a0 0x55e3d9343b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53476/0 (socket says 172.21.15.46:53476) 2024-05-09T04:54:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36e2d9d700 1 -- 172.21.15.46:0/1587163069 learned_addr learned my addr 172.21.15.46:0/1587163069 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36e1d9b700 1 --2- 172.21.15.46:0/1587163069 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d92a6160 0x55e3d93435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36e259c700 1 --2- 172.21.15.46:0/1587163069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d91bc4a0 0x55e3d9343090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36e2d9d700 1 -- 172.21.15.46:0/1587163069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d91bc4a0 msgr2=0x55e3d9343090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36e2d9d700 1 --2- 172.21.15.46:0/1587163069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d91bc4a0 0x55e3d9343090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36e2d9d700 1 -- 172.21.15.46:0/1587163069 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d92a6160 msgr2=0x55e3d93435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36e2d9d700 1 --2- 172.21.15.46:0/1587163069 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d92a6160 0x55e3d93435d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.028+0000 7f36e2d9d700 1 -- 172.21.15.46:0/1587163069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3d90d7490 con 0x55e3d93396a0 2024-05-09T04:54:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.029+0000 7f36e2d9d700 1 --2- 172.21.15.46:0/1587163069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d93396a0 0x55e3d9343b10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f36dc014fa0 tx=0x7f36dc0097e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb56c2ea0cd4e2fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:42.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.029+0000 7f36c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.029+0000 7f36c9ffb700 1 -- 172.21.15.46:0/1587163069 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36dc021020 con 0x55e3d93396a0 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.029+0000 7f36f7612b80 1 -- 172.21.15.46:0/1587163069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36dc0148b0 con 0x55e3d93396a0 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.029+0000 7f36c9ffb700 1 -- 172.21.15.46:0/1587163069 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f36dc004ad0 con 0x55e3d93396a0 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.029+0000 7f36c9ffb700 1 -- 172.21.15.46:0/1587163069 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36dc02c050 con 0x55e3d93396a0 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.029+0000 7f36f7612b80 10 client.16293 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.029+0000 7f36f7612b80 1 -- 172.21.15.46:0/1587163069 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e3d9350b80 con 0x55e3d93396a0 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.029+0000 7f36c9ffb700 1 -- 172.21.15.46:0/1587163069 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36dc0148b0 con 0x55e3d93396a0 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.029+0000 7f36c9ffb700 1 -- 172.21.15.46:0/1587163069 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f36dc01c050 con 0x55e3d93396a0 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.030+0000 7f36f7612b80 10 client.16293 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.030+0000 7f36f7612b80 10 client.16293 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.030+0000 7f36f7612b80 20 client.16293 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.030+0000 7f36f7612b80 1 --2- 172.21.15.46:0/1587163069 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e3d93530b0 0x55e3d93554d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.030+0000 7f36f7612b80 4 client.16293 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.030+0000 7f36f7612b80 1 -- 172.21.15.46:0/1587163069 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3d92699d0 con 0x55e3d93530b0 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.030+0000 7f36e259c700 1 --2- 172.21.15.46:0/1587163069 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e3d93530b0 0x55e3d93554d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.030+0000 7f36e259c700 1 --2- 172.21.15.46:0/1587163069 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e3d93530b0 0x55e3d93554d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=df027398b7e02a78 server_cookie=aab873063c300e0a in_seq=0 out_seq=0 2024-05-09T04:54:42.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.030+0000 7f36c9ffb700 10 client.16293 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:54:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.031+0000 7f36c9ffb700 1 -- 172.21.15.46:0/1587163069 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e3d92699d0 con 0x55e3d93530b0 2024-05-09T04:54:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.031+0000 7f36c9ffb700 10 client.16293 handle_command_reply: tid=0 2024-05-09T04:54:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.031+0000 7f36f7612b80 1 client.16293 shutdown 2024-05-09T04:54:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.031+0000 7f36f7612b80 1 -- 172.21.15.46:0/1587163069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d93396a0 msgr2=0x55e3d9343b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.031+0000 7f36f7612b80 1 --2- 172.21.15.46:0/1587163069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d93396a0 0x55e3d9343b10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f36dc014fa0 tx=0x7f36dc0097e0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.031+0000 7f36f7612b80 1 -- 172.21.15.46:0/1587163069 shutdown_connections 2024-05-09T04:54:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.031+0000 7f36f7612b80 1 --2- 172.21.15.46:0/1587163069 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e3d93530b0 0x55e3d93554d0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.032+0000 7f36f7612b80 1 --2- 172.21.15.46:0/1587163069 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d93396a0 0x55e3d9343b10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.032+0000 7f36c9ffb700 0 client.16293 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:54:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.032+0000 7f36f7612b80 1 --2- 172.21.15.46:0/1587163069 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d91bc4a0 0x55e3d9343090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.032+0000 7f36f7612b80 1 --2- 172.21.15.46:0/1587163069 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d92a6160 0x55e3d93435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.032+0000 7f36f7612b80 1 -- 172.21.15.46:0/1587163069 >> 172.21.15.46:0/1587163069 conn(0x55e3d921c6a0 msgr2=0x55e3d93341c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.032+0000 7f36f7612b80 1 -- 172.21.15.46:0/1587163069 shutdown_connections 2024-05-09T04:54:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.032+0000 7f36f7612b80 1 -- 172.21.15.46:0/1587163069 wait complete. 2024-05-09T04:54:42.036 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.032+0000 7f36f7612b80 20 client.16293 trim_cache size 0 max 16384 2024-05-09T04:54:42.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.047+0000 7f36f7612b80 1 Processor -- start 2024-05-09T04:54:42.048 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.047+0000 7f36f7612b80 1 -- start start 2024-05-09T04:54:42.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.047+0000 7f36f7612b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d91bc4a0 0x55e3d9342010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.047+0000 7f36f7612b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d92a6160 0x55e3d93390c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.047+0000 7f36f7612b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d93396a0 0x55e3d9343950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.047+0000 7f36f7612b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41b39c0 con 0x55e3d92a6160 2024-05-09T04:54:42.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.047+0000 7f36f7612b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41b1250 con 0x55e3d91bc4a0 2024-05-09T04:54:42.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.047+0000 7f36f7612b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41af0a0 con 0x55e3d93396a0 2024-05-09T04:54:42.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.048+0000 7f36e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d92a6160 0x55e3d93390c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.048+0000 7f36e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d92a6160 0x55e3d93390c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53480/0 (socket says 172.21.15.46:53480) 2024-05-09T04:54:42.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.048+0000 7f36e1d9b700 1 -- 172.21.15.46:0/1227822779 learned_addr learned my addr 172.21.15.46:0/1227822779 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:42.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.048+0000 7f36e259c700 1 --2- 172.21.15.46:0/1227822779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d91bc4a0 0x55e3d9342010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.048+0000 7f36e2d9d700 1 --2- 172.21.15.46:0/1227822779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d93396a0 0x55e3d9343950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.048+0000 7f36e1d9b700 1 -- 172.21.15.46:0/1227822779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d93396a0 msgr2=0x55e3d9343950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:42.049 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.048+0000 7f36e1d9b700 1 --2- 172.21.15.46:0/1227822779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d93396a0 0x55e3d9343950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.048+0000 7f36e1d9b700 1 -- 172.21.15.46:0/1227822779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d91bc4a0 msgr2=0x55e3d9342010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:42.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.048+0000 7f36e1d9b700 1 --2- 172.21.15.46:0/1227822779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d91bc4a0 0x55e3d9342010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.048+0000 7f36e1d9b700 1 -- 172.21.15.46:0/1227822779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3d9343e90 con 0x55e3d92a6160 2024-05-09T04:54:42.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.048+0000 7f36e1d9b700 1 --2- 172.21.15.46:0/1227822779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d92a6160 0x55e3d93390c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f36e41021e0 tx=0x7f36d8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52f0b286a4efca0d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:42.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.048+0000 7f36cbfff700 1 -- 172.21.15.46:0/1227822779 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8004a30 con 0x55e3d92a6160 2024-05-09T04:54:42.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.048+0000 7f36cbfff700 1 -- 172.21.15.46:0/1227822779 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d80054d0 con 0x55e3d92a6160 2024-05-09T04:54:42.050 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.048+0000 7f36cbfff700 1 -- 172.21.15.46:0/1227822779 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8010930 con 0x55e3d92a6160 2024-05-09T04:54:42.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.049+0000 7f36f7612b80 1 -- 172.21.15.46:0/1227822779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d92a6160 msgr2=0x55e3d93390c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:42.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.049+0000 7f36f7612b80 1 --2- 172.21.15.46:0/1227822779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d92a6160 0x55e3d93390c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f36e41021e0 tx=0x7f36d8011000 comp rx=0 tx=0).stop 2024-05-09T04:54:42.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.049+0000 7f36f7612b80 1 -- 172.21.15.46:0/1227822779 shutdown_connections 2024-05-09T04:54:42.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.049+0000 7f36f7612b80 1 --2- 172.21.15.46:0/1227822779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d93396a0 0x55e3d9343950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.049+0000 7f36f7612b80 1 --2- 172.21.15.46:0/1227822779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d91bc4a0 0x55e3d9342010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.049+0000 7f36f7612b80 1 --2- 172.21.15.46:0/1227822779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d92a6160 0x55e3d93390c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.049+0000 7f36f7612b80 1 -- 172.21.15.46:0/1227822779 >> 172.21.15.46:0/1227822779 conn(0x55e3d921c6a0 msgr2=0x55e3d9344740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:42.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.049+0000 7f36f7612b80 1 -- 172.21.15.46:0/1227822779 shutdown_connections 2024-05-09T04:54:42.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.049+0000 7f36f7612b80 1 -- 172.21.15.46:0/1227822779 wait complete. 2024-05-09T04:54:42.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36f7612b80 1 -- start start 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36f7612b80 1 Processor -- start 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36f7612b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d91bc4a0 0x55e3d934b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36f7612b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d92a6160 0x55e3d933faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36f7612b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d93396a0 0x55e3d9340030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36f7612b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41af710 con 0x55e3d92a6160 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36f7612b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41b0b80 con 0x55e3d91bc4a0 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36f7612b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36e41afde0 con 0x55e3d93396a0 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36f7612b80 10 client.0 _ll_register_callbacks cb 0x55e3d93559f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d92a6160 0x55e3d933faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36e1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d92a6160 0x55e3d933faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:53486/0 (socket says 172.21.15.46:53486) 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36e1d9b700 1 -- 172.21.15.46:0/4181177841 learned_addr learned my addr 172.21.15.46:0/4181177841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36e259c700 1 --2- 172.21.15.46:0/4181177841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d91bc4a0 0x55e3d934b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36e2d9d700 1 --2- 172.21.15.46:0/4181177841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d93396a0 0x55e3d9340030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36e1d9b700 1 -- 172.21.15.46:0/4181177841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d93396a0 msgr2=0x55e3d9340030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36e1d9b700 1 --2- 172.21.15.46:0/4181177841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d93396a0 0x55e3d9340030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36e1d9b700 1 -- 172.21.15.46:0/4181177841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d91bc4a0 msgr2=0x55e3d934b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36e1d9b700 1 --2- 172.21.15.46:0/4181177841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d91bc4a0 0x55e3d934b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36e1d9b700 1 -- 172.21.15.46:0/4181177841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e3d9340570 con 0x55e3d92a6160 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.050+0000 7f36e2d9d700 1 --2- 172.21.15.46:0/4181177841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d93396a0 0x55e3d9340030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.051+0000 7f36e1d9b700 1 --2- 172.21.15.46:0/4181177841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d92a6160 0x55e3d933faf0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f36d8005440 tx=0x7f36d8005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336fcfbc95a03a0c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.051+0000 7f36c9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.051+0000 7f36c9ffb700 1 -- 172.21.15.46:0/4181177841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d8034070 con 0x55e3d92a6160 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.051+0000 7f36c9ffb700 1 -- 172.21.15.46:0/4181177841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f36d8004a60 con 0x55e3d92a6160 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.051+0000 7f36f7612b80 1 -- 172.21.15.46:0/4181177841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36d80054d0 con 0x55e3d92a6160 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.051+0000 7f36c9ffb700 1 -- 172.21.15.46:0/4181177841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36d8040050 con 0x55e3d92a6160 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.051+0000 7f36f7612b80 10 client.16299 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.051+0000 7f36f7612b80 1 -- 172.21.15.46:0/4181177841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e3d934b110 con 0x55e3d92a6160 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.051+0000 7f36c9ffb700 1 -- 172.21.15.46:0/4181177841 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36d80054d0 con 0x55e3d92a6160 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.051+0000 7f36c9ffb700 1 -- 172.21.15.46:0/4181177841 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f36d8026070 con 0x55e3d92a6160 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.052+0000 7f36f7612b80 10 client.16299 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.052+0000 7f36f7612b80 10 client.16299 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:54:42.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.052+0000 7f36f7612b80 20 client.16299 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.052+0000 7f36f7612b80 1 --2- 172.21.15.46:0/4181177841 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e3d93530b0 0x55e3d93bfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.052+0000 7f36f7612b80 4 client.16299 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.052+0000 7f36f7612b80 1 -- 172.21.15.46:0/4181177841 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e3d9263a70 con 0x55e3d93530b0 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.052+0000 7f36e259c700 1 --2- 172.21.15.46:0/4181177841 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e3d93530b0 0x55e3d93bfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.052+0000 7f36c9ffb700 10 client.16299 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.052+0000 7f36e259c700 1 --2- 172.21.15.46:0/4181177841 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e3d93530b0 0x55e3d93bfa90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ee75a0111f0935a server_cookie=a0c98bfb1ce699ac in_seq=0 out_seq=0 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.052+0000 7f36c9ffb700 1 -- 172.21.15.46:0/4181177841 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e3d9263a70 con 0x55e3d93530b0 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.052+0000 7f36c9ffb700 10 client.16299 handle_command_reply: tid=0 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.052+0000 7f36f7612b80 1 client.16299 shutdown 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36f7612b80 1 -- 172.21.15.46:0/4181177841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d92a6160 msgr2=0x55e3d933faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36f7612b80 1 --2- 172.21.15.46:0/4181177841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d92a6160 0x55e3d933faf0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f36d8005440 tx=0x7f36d8005680 comp rx=0 tx=0).stop 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36f7612b80 1 -- 172.21.15.46:0/4181177841 shutdown_connections 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36f7612b80 1 --2- 172.21.15.46:0/4181177841 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e3d93530b0 0x55e3d93bfa90 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36f7612b80 1 --2- 172.21.15.46:0/4181177841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e3d93396a0 0x55e3d9340030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36f7612b80 1 --2- 172.21.15.46:0/4181177841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e3d91bc4a0 0x55e3d934b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36f7612b80 1 --2- 172.21.15.46:0/4181177841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e3d92a6160 0x55e3d933faf0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36f7612b80 1 -- 172.21.15.46:0/4181177841 >> 172.21.15.46:0/4181177841 conn(0x55e3d921c6a0 msgr2=0x55e3d9334310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36c9ffb700 0 client.16299 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:54:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36f7612b80 1 -- 172.21.15.46:0/4181177841 shutdown_connections 2024-05-09T04:54:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36f7612b80 1 -- 172.21.15.46:0/4181177841 wait complete. 2024-05-09T04:54:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36f7612b80 20 client.16299 trim_cache size 0 max 16384 2024-05-09T04:54:42.058 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:54:42.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36e8bfc700 1 -- 172.21.15.46:0/492917746 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f36cc0b29f0 msgr2=0x7f36cc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:42.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36e8bfc700 1 --2- 172.21.15.46:0/492917746 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f36cc0b29f0 0x7f36cc0b4f00 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f36d8005fd0 tx=0x7f36d800a560 comp rx=0 tx=0).stop 2024-05-09T04:54:42.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36e8bfc700 1 -- 172.21.15.46:0/492917746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e41034c0 msgr2=0x7f36e40ff2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:42.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.053+0000 7f36e8bfc700 1 --2- 172.21.15.46:0/492917746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e41034c0 0x7f36e40ff2a0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f36d400b580 tx=0x7f36d40047b0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.054+0000 7f36e8bfc700 1 -- 172.21.15.46:0/492917746 shutdown_connections 2024-05-09T04:54:42.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.054+0000 7f36e8bfc700 1 --2- 172.21.15.46:0/492917746 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f36cc0b29f0 0x7f36cc0b4f00 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.054+0000 7f36e8bfc700 1 --2- 172.21.15.46:0/492917746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36e4108740 0x7f36e40ffd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.054+0000 7f36e8bfc700 1 --2- 172.21.15.46:0/492917746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36e41034c0 0x7f36e40ff2a0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.054+0000 7f36e8bfc700 1 --2- 172.21.15.46:0/492917746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36e4107df0 0x7f36e40ff7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:42.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.054+0000 7f36e8bfc700 1 -- 172.21.15.46:0/492917746 >> 172.21.15.46:0/492917746 conn(0x7f36e40fedc0 msgr2=0x7f36e41063a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:42.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.054+0000 7f36e8bfc700 1 -- 172.21.15.46:0/492917746 shutdown_connections 2024-05-09T04:54:42.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:42.054+0000 7f36e8bfc700 1 -- 172.21.15.46:0/492917746 wait complete. 2024-05-09T04:54:42.787 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:54:43.045 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:43.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:42 smithi110 ceph-mon[34904]: pgmap v111: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 8 op/s 2024-05-09T04:54:43.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:42 smithi046 ceph-mon[30136]: pgmap v111: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 8 op/s 2024-05-09T04:54:43.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:42 smithi072 ceph-mon[34088]: pgmap v111: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 8 op/s 2024-05-09T04:54:44.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.868+0000 7fe6244f0700 1 -- 172.21.15.46:0/3991247785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe61c1034c0 msgr2=0x7fe61c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.868+0000 7fe6244f0700 1 --2- 172.21.15.46:0/3991247785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe61c1034c0 0x7fe61c1038d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fe6180099c0 tx=0x7fe618009cd0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.868+0000 7fe6244f0700 1 -- 172.21.15.46:0/3991247785 shutdown_connections 2024-05-09T04:54:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.868+0000 7fe6244f0700 1 --2- 172.21.15.46:0/3991247785 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe61c108740 0x7fe61c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.868+0000 7fe6244f0700 1 --2- 172.21.15.46:0/3991247785 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe61c1034c0 0x7fe61c1038d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.868+0000 7fe6244f0700 1 --2- 172.21.15.46:0/3991247785 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe61c107df0 0x7fe61c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.868+0000 7fe6244f0700 1 -- 172.21.15.46:0/3991247785 >> 172.21.15.46:0/3991247785 conn(0x7fe61c0fedf0 msgr2=0x7fe61c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.868+0000 7fe6244f0700 1 -- 172.21.15.46:0/3991247785 shutdown_connections 2024-05-09T04:54:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.868+0000 7fe6244f0700 1 -- 172.21.15.46:0/3991247785 wait complete. 2024-05-09T04:54:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.868+0000 7fe6244f0700 1 Processor -- start 2024-05-09T04:54:44.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe6244f0700 1 -- start start 2024-05-09T04:54:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe6244f0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe61c1034c0 0x7fe61c19ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe6244f0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe61c107df0 0x7fe61c19b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe6244f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe61c108740 0x7fe61c19f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe6244f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1144c0 con 0x7fe61c108740 2024-05-09T04:54:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe6244f0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c114340 con 0x7fe61c107df0 2024-05-09T04:54:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe6244f0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c114640 con 0x7fe61c1034c0 2024-05-09T04:54:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe622a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe61c108740 0x7fe61c19f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe622a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe61c108740 0x7fe61c19f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45156/0 (socket says 172.21.15.46:45156) 2024-05-09T04:54:44.872 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe622a8d700 1 -- 172.21.15.46:0/3173709917 learned_addr learned my addr 172.21.15.46:0/3173709917 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:44.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe62228c700 1 --2- 172.21.15.46:0/3173709917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe61c1034c0 0x7fe61c19ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe621a8b700 1 --2- 172.21.15.46:0/3173709917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe61c107df0 0x7fe61c19b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe622a8d700 1 -- 172.21.15.46:0/3173709917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe61c1034c0 msgr2=0x7fe61c19ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:44.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe622a8d700 1 --2- 172.21.15.46:0/3173709917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe61c1034c0 0x7fe61c19ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe622a8d700 1 -- 172.21.15.46:0/3173709917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe61c107df0 msgr2=0x7fe61c19b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:44.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe622a8d700 1 --2- 172.21.15.46:0/3173709917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe61c107df0 0x7fe61c19b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe622a8d700 1 -- 172.21.15.46:0/3173709917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe61c19fc60 con 0x7fe61c108740 2024-05-09T04:54:44.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.869+0000 7fe62228c700 1 --2- 172.21.15.46:0/3173709917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe61c1034c0 0x7fe61c19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:44.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.870+0000 7fe622a8d700 1 --2- 172.21.15.46:0/3173709917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe61c108740 0x7fe61c19f670 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe614007d70 tx=0x7fe61400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:44.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.870+0000 7fe6137fe700 1 -- 172.21.15.46:0/3173709917 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe614016020 con 0x7fe61c108740 2024-05-09T04:54:44.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.870+0000 7fe6137fe700 1 -- 172.21.15.46:0/3173709917 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe61400ac90 con 0x7fe61c108740 2024-05-09T04:54:44.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.870+0000 7fe6244f0700 1 -- 172.21.15.46:0/3173709917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe61c19ff50 con 0x7fe61c108740 2024-05-09T04:54:44.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.870+0000 7fe6137fe700 1 -- 172.21.15.46:0/3173709917 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6140115e0 con 0x7fe61c108740 2024-05-09T04:54:44.873 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.870+0000 7fe6244f0700 1 -- 172.21.15.46:0/3173709917 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe61c1a0440 con 0x7fe61c108740 2024-05-09T04:54:44.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.873+0000 7fe6137fe700 1 -- 172.21.15.46:0/3173709917 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe614019020 con 0x7fe61c108740 2024-05-09T04:54:44.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe632f06b80 1 Processor -- start 2024-05-09T04:54:44.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe6137fe700 1 --2- 172.21.15.46:0/3173709917 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe6080b29f0 0x7fe6080b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe621a8b700 1 --2- 172.21.15.46:0/3173709917 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe6080b29f0 0x7fe6080b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe6137fe700 1 -- 172.21.15.46:0/3173709917 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fe6140e24d0 con 0x7fe61c108740 2024-05-09T04:54:44.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe632f06b80 1 -- start start 2024-05-09T04:54:44.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe632f06b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabaf38d00 0x55dabb041240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe632f06b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb041780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe632f06b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabb03b400 0x55dabb03b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe632f06b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dabaf5e0a0 con 0x55dabb03b400 2024-05-09T04:54:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe632f06b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55dabaf039a0 con 0x55dabaf38d00 2024-05-09T04:54:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe632f06b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dabaf5a060 con 0x55dabaebe4a0 2024-05-09T04:54:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe621a8b700 1 --2- 172.21.15.46:0/3173709917 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe6080b29f0 0x7fe6080b4f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe61c1064c0 tx=0x7fe60c00b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe622a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabb03b400 0x55dabb03b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe622a8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabb03b400 0x55dabb03b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45162/0 (socket says 172.21.15.46:45162) 2024-05-09T04:54:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe622a8d700 1 -- 172.21.15.46:0/345991990 learned_addr learned my addr 172.21.15.46:0/345991990 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.874+0000 7fe62228c700 1 --2- 172.21.15.46:0/345991990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabaf38d00 0x55dabb041240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.875+0000 7fe622a8d700 1 -- 172.21.15.46:0/345991990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 msgr2=0x55dabb041780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.875+0000 7fe621a8b700 1 --2- 172.21.15.46:0/345991990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb041780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.875+0000 7fe622a8d700 1 --2- 172.21.15.46:0/345991990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb041780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.875+0000 7fe622a8d700 1 -- 172.21.15.46:0/345991990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabaf38d00 msgr2=0x55dabb041240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:44.877 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.875+0000 7fe622a8d700 1 --2- 172.21.15.46:0/345991990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabaf38d00 0x55dabb041240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.875+0000 7fe622a8d700 1 -- 172.21.15.46:0/345991990 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dabaef4500 con 0x55dabb03b400 2024-05-09T04:54:44.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.875+0000 7fe62228c700 1 --2- 172.21.15.46:0/345991990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabaf38d00 0x55dabb041240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:44.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.875+0000 7fe622a8d700 1 --2- 172.21.15.46:0/345991990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabb03b400 0x55dabb03b810 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe6140a58c0 tx=0x7fe6140a9190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60215a865af86795 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:44.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.875+0000 7fe610d47700 1 -- 172.21.15.46:0/345991990 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6140a9350 con 0x55dabb03b400 2024-05-09T04:54:44.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.875+0000 7fe610d47700 1 -- 172.21.15.46:0/345991990 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6140a9f40 con 0x55dabb03b400 2024-05-09T04:54:44.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.875+0000 7fe610d47700 1 -- 172.21.15.46:0/345991990 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6140b2210 con 0x55dabb03b400 2024-05-09T04:54:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.876+0000 7fe632f06b80 1 -- 172.21.15.46:0/345991990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabb03b400 msgr2=0x55dabb03b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.876+0000 7fe632f06b80 1 --2- 172.21.15.46:0/345991990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabb03b400 0x55dabb03b810 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fe6140a58c0 tx=0x7fe6140a9190 comp rx=0 tx=0).stop 2024-05-09T04:54:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.876+0000 7fe632f06b80 1 -- 172.21.15.46:0/345991990 shutdown_connections 2024-05-09T04:54:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.876+0000 7fe632f06b80 1 --2- 172.21.15.46:0/345991990 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabb03b400 0x55dabb03b810 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.876+0000 7fe632f06b80 1 --2- 172.21.15.46:0/345991990 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabaf38d00 0x55dabb041240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.882 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.876+0000 7fe632f06b80 1 --2- 172.21.15.46:0/345991990 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb041780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.876+0000 7fe632f06b80 1 -- 172.21.15.46:0/345991990 >> 172.21.15.46:0/345991990 conn(0x55dabaec1f70 msgr2=0x55dabb036220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.876+0000 7fe632f06b80 1 -- 172.21.15.46:0/345991990 shutdown_connections 2024-05-09T04:54:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.876+0000 7fe632f06b80 1 -- 172.21.15.46:0/345991990 wait complete. 2024-05-09T04:54:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.877+0000 7fe632f06b80 1 -- start start 2024-05-09T04:54:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.877+0000 7fe632f06b80 1 Processor -- start 2024-05-09T04:54:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.877+0000 7fe632f06b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb045040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.877+0000 7fe632f06b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb045580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.877+0000 7fe632f06b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 0x55dabb045ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.877+0000 7fe632f06b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55dabb03c160 con 0x55dabaf38d00 2024-05-09T04:54:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.877+0000 7fe632f06b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55dabaef8b20 con 0x55dabb03b400 2024-05-09T04:54:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.877+0000 7fe632f06b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55dabb048020 con 0x55dabaebe4a0 2024-05-09T04:54:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.877+0000 7fe632f06b80 10 client.0 _ll_register_callbacks cb 0x55dabadcf6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.877+0000 7fe621a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb045580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.877+0000 7fe621a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb045580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45172/0 (socket says 172.21.15.46:45172) 2024-05-09T04:54:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.877+0000 7fe621a8b700 1 -- 172.21.15.46:0/754553830 learned_addr learned my addr 172.21.15.46:0/754553830 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.877+0000 7fe62228c700 1 --2- 172.21.15.46:0/754553830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb045040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.877+0000 7fe622a8d700 1 --2- 172.21.15.46:0/754553830 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 0x55dabb045ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.878+0000 7fe621a8b700 1 -- 172.21.15.46:0/754553830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 msgr2=0x55dabb045040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.878+0000 7fe621a8b700 1 --2- 172.21.15.46:0/754553830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb045040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.878+0000 7fe621a8b700 1 -- 172.21.15.46:0/754553830 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 msgr2=0x55dabb045ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.878+0000 7fe621a8b700 1 --2- 172.21.15.46:0/754553830 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 0x55dabb045ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.878+0000 7fe621a8b700 1 -- 172.21.15.46:0/754553830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dabadd9490 con 0x55dabaf38d00 2024-05-09T04:54:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.878+0000 7fe622a8d700 1 --2- 172.21.15.46:0/754553830 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 0x55dabb045ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.878+0000 7fe621a8b700 1 --2- 172.21.15.46:0/754553830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb045580 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00bfc0 tx=0x7fe60c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48a88cb340b995de server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.878+0000 7fe605ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.878+0000 7fe605ffb700 1 -- 172.21.15.46:0/754553830 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c027670 con 0x55dabaf38d00 2024-05-09T04:54:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.878+0000 7fe632f06b80 1 -- 172.21.15.46:0/754553830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe6140a9f40 con 0x55dabaf38d00 2024-05-09T04:54:44.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.878+0000 7fe605ffb700 1 -- 172.21.15.46:0/754553830 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fe60c027810 con 0x55dabaf38d00 2024-05-09T04:54:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.878+0000 7fe605ffb700 1 -- 172.21.15.46:0/754553830 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe60c027a50 con 0x55dabaf38d00 2024-05-09T04:54:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.878+0000 7fe632f06b80 10 client.16311 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.878+0000 7fe632f06b80 1 -- 172.21.15.46:0/754553830 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dabb052a70 con 0x55dabaf38d00 2024-05-09T04:54:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.878+0000 7fe605ffb700 1 -- 172.21.15.46:0/754553830 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6140a9f40 con 0x55dabaf38d00 2024-05-09T04:54:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.879+0000 7fe605ffb700 1 -- 172.21.15.46:0/754553830 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fe60c01ec20 con 0x55dabaf38d00 2024-05-09T04:54:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.879+0000 7fe632f06b80 10 client.16311 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.879+0000 7fe632f06b80 10 client.16311 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.879+0000 7fe632f06b80 20 client.16311 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.879+0000 7fe632f06b80 1 --2- 172.21.15.46:0/754553830 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dabb0550b0 0x55dabb0574d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.879+0000 7fe632f06b80 4 client.16311 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.879+0000 7fe632f06b80 1 -- 172.21.15.46:0/754553830 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dabaf6ba80 con 0x55dabb0550b0 2024-05-09T04:54:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.879+0000 7fe62228c700 1 --2- 172.21.15.46:0/754553830 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dabb0550b0 0x55dabb0574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.879+0000 7fe62228c700 1 --2- 172.21.15.46:0/754553830 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dabb0550b0 0x55dabb0574d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5856cd7e74408347 server_cookie=5f40af91228b9cdd in_seq=0 out_seq=0 2024-05-09T04:54:44.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.879+0000 7fe605ffb700 10 client.16311 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.880+0000 7fe605ffb700 1 -- 172.21.15.46:0/754553830 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55dabaf6ba80 con 0x55dabb0550b0 2024-05-09T04:54:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.880+0000 7fe605ffb700 10 client.16311 handle_command_reply: tid=0 2024-05-09T04:54:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.880+0000 7fe632f06b80 1 client.16311 shutdown 2024-05-09T04:54:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.880+0000 7fe632f06b80 1 -- 172.21.15.46:0/754553830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 msgr2=0x55dabb045580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.880+0000 7fe632f06b80 1 --2- 172.21.15.46:0/754553830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb045580 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00bfc0 tx=0x7fe60c0047c0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.880+0000 7fe632f06b80 1 -- 172.21.15.46:0/754553830 shutdown_connections 2024-05-09T04:54:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.880+0000 7fe632f06b80 1 --2- 172.21.15.46:0/754553830 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dabb0550b0 0x55dabb0574d0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.880+0000 7fe632f06b80 1 --2- 172.21.15.46:0/754553830 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 0x55dabb045ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.880+0000 7fe632f06b80 1 --2- 172.21.15.46:0/754553830 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb045040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.880+0000 7fe605ffb700 0 client.16311 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.880+0000 7fe632f06b80 1 --2- 172.21.15.46:0/754553830 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb045580 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.880+0000 7fe632f06b80 1 -- 172.21.15.46:0/754553830 >> 172.21.15.46:0/754553830 conn(0x55dabaec1f70 msgr2=0x55dabb02fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:44.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.881+0000 7fe632f06b80 1 -- 172.21.15.46:0/754553830 shutdown_connections 2024-05-09T04:54:44.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.881+0000 7fe632f06b80 1 -- 172.21.15.46:0/754553830 wait complete. 2024-05-09T04:54:44.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.881+0000 7fe632f06b80 20 client.16311 trim_cache size 0 max 16384 2024-05-09T04:54:44.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.896+0000 7fe632f06b80 1 Processor -- start 2024-05-09T04:54:44.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.896+0000 7fe632f06b80 1 -- start start 2024-05-09T04:54:44.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.896+0000 7fe632f06b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb045900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.896+0000 7fe632f06b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb030700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.896+0000 7fe632f06b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 0x55dabb050e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.896+0000 7fe632f06b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1b1850 con 0x55dabaf38d00 2024-05-09T04:54:44.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.896+0000 7fe632f06b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1af0e0 con 0x55dabb03b400 2024-05-09T04:54:44.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.896+0000 7fe632f06b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1acf30 con 0x55dabaebe4a0 2024-05-09T04:54:44.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.896+0000 7fe621a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb030700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.896+0000 7fe621a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb030700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45176/0 (socket says 172.21.15.46:45176) 2024-05-09T04:54:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.896+0000 7fe621a8b700 1 -- 172.21.15.46:0/1162667140 learned_addr learned my addr 172.21.15.46:0/1162667140 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.896+0000 7fe62228c700 1 --2- 172.21.15.46:0/1162667140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb045900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.896+0000 7fe622a8d700 1 --2- 172.21.15.46:0/1162667140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 0x55dabb050e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.897+0000 7fe621a8b700 1 -- 172.21.15.46:0/1162667140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 msgr2=0x55dabb045900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.897+0000 7fe621a8b700 1 --2- 172.21.15.46:0/1162667140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb045900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.897+0000 7fe621a8b700 1 -- 172.21.15.46:0/1162667140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 msgr2=0x55dabb050e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.897+0000 7fe621a8b700 1 --2- 172.21.15.46:0/1162667140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 0x55dabb050e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.897+0000 7fe621a8b700 1 -- 172.21.15.46:0/1162667140 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dabb0329d0 con 0x55dabaf38d00 2024-05-09T04:54:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.897+0000 7fe621a8b700 1 --2- 172.21.15.46:0/1162667140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb030700 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00bf90 tx=0x7fe60c004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a6d2454bd310df server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.897+0000 7fe607fff700 1 -- 172.21.15.46:0/1162667140 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c034040 con 0x55dabaf38d00 2024-05-09T04:54:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.897+0000 7fe607fff700 1 -- 172.21.15.46:0/1162667140 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe60c022070 con 0x55dabaf38d00 2024-05-09T04:54:44.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.897+0000 7fe607fff700 1 -- 172.21.15.46:0/1162667140 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c0054e0 con 0x55dabaf38d00 2024-05-09T04:54:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.898+0000 7fe632f06b80 1 -- 172.21.15.46:0/1162667140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 msgr2=0x55dabb030700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.898+0000 7fe632f06b80 1 --2- 172.21.15.46:0/1162667140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb030700 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fe60c00bf90 tx=0x7fe60c004710 comp rx=0 tx=0).stop 2024-05-09T04:54:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.898+0000 7fe632f06b80 1 -- 172.21.15.46:0/1162667140 shutdown_connections 2024-05-09T04:54:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.898+0000 7fe632f06b80 1 --2- 172.21.15.46:0/1162667140 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 0x55dabb050e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.898+0000 7fe632f06b80 1 --2- 172.21.15.46:0/1162667140 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb045900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.898+0000 7fe632f06b80 1 --2- 172.21.15.46:0/1162667140 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb030700 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.898+0000 7fe632f06b80 1 -- 172.21.15.46:0/1162667140 >> 172.21.15.46:0/1162667140 conn(0x55dabaec1f70 msgr2=0x55dabb02fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.898+0000 7fe632f06b80 1 -- 172.21.15.46:0/1162667140 shutdown_connections 2024-05-09T04:54:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.898+0000 7fe632f06b80 1 -- 172.21.15.46:0/1162667140 wait complete. 2024-05-09T04:54:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.898+0000 7fe632f06b80 1 -- start start 2024-05-09T04:54:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.898+0000 7fe632f06b80 1 Processor -- start 2024-05-09T04:54:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe632f06b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb04d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe632f06b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb05c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe632f06b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 0x55dabb05c730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe632f06b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1ad5a0 con 0x55dabaf38d00 2024-05-09T04:54:44.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe632f06b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1aea10 con 0x55dabb03b400 2024-05-09T04:54:44.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe632f06b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe61c1adc70 con 0x55dabaebe4a0 2024-05-09T04:54:44.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe632f06b80 10 client.0 _ll_register_callbacks cb 0x55dabadcf6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:44.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe62228c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb04d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe621a8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb05c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe62228c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb04d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:57222/0 (socket says 172.21.15.46:57222) 2024-05-09T04:54:44.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe62228c700 1 -- 172.21.15.46:0/3033970599 learned_addr learned my addr 172.21.15.46:0/3033970599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:44.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe622a8d700 1 --2- 172.21.15.46:0/3033970599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 0x55dabb05c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe621a8b700 1 -- 172.21.15.46:0/3033970599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 msgr2=0x55dabb04d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:44.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe621a8b700 1 --2- 172.21.15.46:0/3033970599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb04d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe621a8b700 1 -- 172.21.15.46:0/3033970599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 msgr2=0x55dabb05c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:44.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe621a8b700 1 --2- 172.21.15.46:0/3033970599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 0x55dabb05c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe621a8b700 1 -- 172.21.15.46:0/3033970599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dabb05ce10 con 0x55dabaf38d00 2024-05-09T04:54:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe62228c700 1 --2- 172.21.15.46:0/3033970599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb04d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe621a8b700 1 --2- 172.21.15.46:0/3033970599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb05c1f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe60c005950 tx=0x7fe60c027820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585a922b62fbe44d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.899+0000 7fe605ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.900+0000 7fe605ffb700 1 -- 172.21.15.46:0/3033970599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60c046060 con 0x55dabaf38d00 2024-05-09T04:54:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.900+0000 7fe605ffb700 1 -- 172.21.15.46:0/3033970599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fe60c005670 con 0x55dabaf38d00 2024-05-09T04:54:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.900+0000 7fe632f06b80 1 -- 172.21.15.46:0/3033970599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe60c022070 con 0x55dabaf38d00 2024-05-09T04:54:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.900+0000 7fe605ffb700 1 -- 172.21.15.46:0/3033970599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe60c034030 con 0x55dabaf38d00 2024-05-09T04:54:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.900+0000 7fe632f06b80 10 client.16317 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.900+0000 7fe632f06b80 1 -- 172.21.15.46:0/3033970599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55dabb0bf620 con 0x55dabaf38d00 2024-05-09T04:54:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.900+0000 7fe605ffb700 1 -- 172.21.15.46:0/3033970599 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe60c022070 con 0x55dabaf38d00 2024-05-09T04:54:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.900+0000 7fe605ffb700 1 -- 172.21.15.46:0/3033970599 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fe60c037030 con 0x55dabaf38d00 2024-05-09T04:54:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.900+0000 7fe632f06b80 10 client.16317 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.900+0000 7fe632f06b80 10 client.16317 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.900+0000 7fe632f06b80 20 client.16317 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:44.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.900+0000 7fe632f06b80 1 --2- 172.21.15.46:0/3033970599 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dabb0bfbc0 0x55dabb0c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:44.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.900+0000 7fe632f06b80 4 client.16317 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:54:44.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.900+0000 7fe632f06b80 1 -- 172.21.15.46:0/3033970599 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dabaf65b20 con 0x55dabb0bfbc0 2024-05-09T04:54:44.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.901+0000 7fe62228c700 1 --2- 172.21.15.46:0/3033970599 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dabb0bfbc0 0x55dabb0c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:44.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.901+0000 7fe62228c700 1 --2- 172.21.15.46:0/3033970599 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dabb0bfbc0 0x55dabb0c1fe0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c66e079629b34f63 server_cookie=8e35a0cd8ccc0151 in_seq=0 out_seq=0 2024-05-09T04:54:44.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.901+0000 7fe605ffb700 10 client.16317 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:44.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.901+0000 7fe605ffb700 1 -- 172.21.15.46:0/3033970599 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dabaf65b20 con 0x55dabb0bfbc0 2024-05-09T04:54:44.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.901+0000 7fe605ffb700 10 client.16317 handle_command_reply: tid=0 2024-05-09T04:54:44.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.901+0000 7fe632f06b80 1 client.16317 shutdown 2024-05-09T04:54:44.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.901+0000 7fe632f06b80 1 -- 172.21.15.46:0/3033970599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 msgr2=0x55dabb05c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:44.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.901+0000 7fe632f06b80 1 --2- 172.21.15.46:0/3033970599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb05c1f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fe60c005950 tx=0x7fe60c027820 comp rx=0 tx=0).stop 2024-05-09T04:54:44.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.901+0000 7fe632f06b80 1 -- 172.21.15.46:0/3033970599 shutdown_connections 2024-05-09T04:54:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.901+0000 7fe632f06b80 1 --2- 172.21.15.46:0/3033970599 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55dabb0bfbc0 0x55dabb0c1fe0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.901+0000 7fe632f06b80 1 --2- 172.21.15.46:0/3033970599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55dabb03b400 0x55dabb05c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.901+0000 7fe632f06b80 1 --2- 172.21.15.46:0/3033970599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55dabaebe4a0 0x55dabb04d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.901+0000 7fe605ffb700 0 client.16317 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.901+0000 7fe632f06b80 1 --2- 172.21.15.46:0/3033970599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55dabaf38d00 0x55dabb05c1f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.902+0000 7fe632f06b80 1 -- 172.21.15.46:0/3033970599 >> 172.21.15.46:0/3033970599 conn(0x55dabaec1f70 msgr2=0x55dabb03a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.902+0000 7fe632f06b80 1 -- 172.21.15.46:0/3033970599 shutdown_connections 2024-05-09T04:54:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.902+0000 7fe632f06b80 1 -- 172.21.15.46:0/3033970599 wait complete. 2024-05-09T04:54:44.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.902+0000 7fe632f06b80 20 client.16317 trim_cache size 0 max 16384 2024-05-09T04:54:44.908 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:54:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.902+0000 7fe6244f0700 1 -- 172.21.15.46:0/3173709917 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe6080b29f0 msgr2=0x7fe6080b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.902+0000 7fe6244f0700 1 --2- 172.21.15.46:0/3173709917 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe6080b29f0 0x7fe6080b4f00 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe61c1064c0 tx=0x7fe60c00b410 comp rx=0 tx=0).stop 2024-05-09T04:54:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.902+0000 7fe6244f0700 1 -- 172.21.15.46:0/3173709917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe61c108740 msgr2=0x7fe61c19f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.902+0000 7fe6244f0700 1 --2- 172.21.15.46:0/3173709917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe61c108740 0x7fe61c19f670 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fe614007d70 tx=0x7fe61400a340 comp rx=0 tx=0).stop 2024-05-09T04:54:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.902+0000 7fe6244f0700 1 -- 172.21.15.46:0/3173709917 shutdown_connections 2024-05-09T04:54:44.909 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.902+0000 7fe6244f0700 1 --2- 172.21.15.46:0/3173709917 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe6080b29f0 0x7fe6080b4f00 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.902+0000 7fe6244f0700 1 --2- 172.21.15.46:0/3173709917 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe61c108740 0x7fe61c19f670 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.902+0000 7fe6244f0700 1 --2- 172.21.15.46:0/3173709917 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe61c1034c0 0x7fe61c19ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.902+0000 7fe6244f0700 1 --2- 172.21.15.46:0/3173709917 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe61c107df0 0x7fe61c19b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.902+0000 7fe6244f0700 1 -- 172.21.15.46:0/3173709917 >> 172.21.15.46:0/3173709917 conn(0x7fe61c0fedf0 msgr2=0x7fe61c101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.902+0000 7fe6244f0700 1 -- 172.21.15.46:0/3173709917 shutdown_connections 2024-05-09T04:54:44.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:44.903+0000 7fe6244f0700 1 -- 172.21.15.46:0/3173709917 wait complete. 2024-05-09T04:54:45.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:44 smithi110 ceph-mon[34904]: pgmap v112: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:54:45.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:44 smithi046 ceph-mon[30136]: pgmap v112: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:54:45.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:44 smithi072 ceph-mon[34088]: pgmap v112: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:54:46.624 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:54:46.624 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:54:46.890 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:47.013 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:46 smithi046 ceph-mon[30136]: pgmap v113: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:54:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:46 smithi110 ceph-mon[34904]: pgmap v113: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:54:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:46 smithi072 ceph-mon[34088]: pgmap v113: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:54:48.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.733+0000 7f13898de700 1 -- 172.21.15.46:0/3353745722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13841089f0 msgr2=0x7f1384108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:48.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.733+0000 7f13898de700 1 --2- 172.21.15.46:0/3353745722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13841089f0 0x7f1384108e00 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f1374009990 tx=0x7f1374009ca0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.733+0000 7f13898de700 1 -- 172.21.15.46:0/3353745722 shutdown_connections 2024-05-09T04:54:48.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.733+0000 7f13898de700 1 --2- 172.21.15.46:0/3353745722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1384103310 0x7f138410f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.733+0000 7f13898de700 1 --2- 172.21.15.46:0/3353745722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13841089f0 0x7f1384108e00 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.733+0000 7f13898de700 1 --2- 172.21.15.46:0/3353745722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13841029c0 0x7f1384102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.733+0000 7f13898de700 1 -- 172.21.15.46:0/3353745722 >> 172.21.15.46:0/3353745722 conn(0x7f13840fe6f0 msgr2=0x7f1384100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:48.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.733+0000 7f13898de700 1 -- 172.21.15.46:0/3353745722 shutdown_connections 2024-05-09T04:54:48.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.733+0000 7f13898de700 1 -- 172.21.15.46:0/3353745722 wait complete. 2024-05-09T04:54:48.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.733+0000 7f13898de700 1 Processor -- start 2024-05-09T04:54:48.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13898de700 1 -- start start 2024-05-09T04:54:48.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13898de700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13841029c0 0x7f1384104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13898de700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384103310 0x7f1384104ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13898de700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13841089f0 0x7f1384105010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13898de700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1384113390 con 0x7f1384103310 2024-05-09T04:54:48.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13898de700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1384113210 con 0x7f13841089f0 2024-05-09T04:54:48.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13898de700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1384113510 con 0x7f13841029c0 2024-05-09T04:54:48.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13827fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384103310 0x7f1384104ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13827fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384103310 0x7f1384104ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45210/0 (socket says 172.21.15.46:45210) 2024-05-09T04:54:48.737 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13827fc700 1 -- 172.21.15.46:0/3886092966 learned_addr learned my addr 172.21.15.46:0/3886092966 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:48.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f1382ffd700 1 --2- 172.21.15.46:0/3886092966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13841029c0 0x7f1384104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13837fe700 1 --2- 172.21.15.46:0/3886092966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13841089f0 0x7f1384105010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13827fc700 1 -- 172.21.15.46:0/3886092966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13841029c0 msgr2=0x7f1384104590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:48.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13827fc700 1 --2- 172.21.15.46:0/3886092966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13841029c0 0x7f1384104590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13827fc700 1 -- 172.21.15.46:0/3886092966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13841089f0 msgr2=0x7f1384105010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:48.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13827fc700 1 --2- 172.21.15.46:0/3886092966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13841089f0 0x7f1384105010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.734+0000 7f13827fc700 1 -- 172.21.15.46:0/3886092966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13841a9210 con 0x7f1384103310 2024-05-09T04:54:48.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.735+0000 7f13827fc700 1 --2- 172.21.15.46:0/3886092966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384103310 0x7f1384104ad0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f137800d980 tx=0x7f137800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:48.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.735+0000 7f13888dc700 1 -- 172.21.15.46:0/3886092966 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378017070 con 0x7f1384103310 2024-05-09T04:54:48.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.735+0000 7f13898de700 1 -- 172.21.15.46:0/3886092966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13841a9500 con 0x7f1384103310 2024-05-09T04:54:48.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.735+0000 7f13888dc700 1 -- 172.21.15.46:0/3886092966 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f137800fd40 con 0x7f1384103310 2024-05-09T04:54:48.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.735+0000 7f13888dc700 1 -- 172.21.15.46:0/3886092966 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137800e3b0 con 0x7f1384103310 2024-05-09T04:54:48.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.735+0000 7f13898de700 1 -- 172.21.15.46:0/3886092966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13841a9a40 con 0x7f1384103310 2024-05-09T04:54:48.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.738+0000 7f13888dc700 1 -- 172.21.15.46:0/3886092966 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f137800e550 con 0x7f1384103310 2024-05-09T04:54:48.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.738+0000 7f13888dc700 1 --2- 172.21.15.46:0/3886092966 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f136c0b29f0 0x7f136c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.739+0000 7f1382ffd700 1 --2- 172.21.15.46:0/3886092966 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f136c0b29f0 0x7f136c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.739+0000 7f13888dc700 1 -- 172.21.15.46:0/3886092966 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f13780e20f0 con 0x7f1384103310 2024-05-09T04:54:48.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.739+0000 7f13982f4b80 1 Processor -- start 2024-05-09T04:54:48.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.739+0000 7f13982f4b80 1 -- start start 2024-05-09T04:54:48.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.739+0000 7f1382ffd700 1 --2- 172.21.15.46:0/3886092966 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f136c0b29f0 0x7f136c0b4f00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1374006010 tx=0x7f1374005960 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:48.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.739+0000 7f13982f4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 0x55f172b232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.739+0000 7f13982f4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1729a0520 0x55f172b23800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.739+0000 7f13982f4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f172b1d480 0x55f172b1d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.739+0000 7f13982f4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f172a40120 con 0x55f172b1d480 2024-05-09T04:54:48.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.739+0000 7f13982f4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f1729e5a20 con 0x55f172a1ad80 2024-05-09T04:54:48.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.739+0000 7f13982f4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f172a3c0e0 con 0x55f1729a0520 2024-05-09T04:54:48.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.739+0000 7f13837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f172b1d480 0x55f172b1d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.739+0000 7f13837fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f172b1d480 0x55f172b1d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45226/0 (socket says 172.21.15.46:45226) 2024-05-09T04:54:48.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.739+0000 7f13837fe700 1 -- 172.21.15.46:0/1836794757 learned_addr learned my addr 172.21.15.46:0/1836794757 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:48.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.740+0000 7f13827fc700 1 --2- 172.21.15.46:0/1836794757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1729a0520 0x55f172b23800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.740+0000 7f1382ffd700 1 --2- 172.21.15.46:0/1836794757 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 0x55f172b232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.740+0000 7f13837fe700 1 -- 172.21.15.46:0/1836794757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1729a0520 msgr2=0x55f172b23800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.740+0000 7f13837fe700 1 --2- 172.21.15.46:0/1836794757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1729a0520 0x55f172b23800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.740+0000 7f13837fe700 1 -- 172.21.15.46:0/1836794757 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 msgr2=0x55f172b232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.740+0000 7f13837fe700 1 --2- 172.21.15.46:0/1836794757 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 0x55f172b232c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.740+0000 7f13837fe700 1 -- 172.21.15.46:0/1836794757 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1729d6580 con 0x55f172b1d480 2024-05-09T04:54:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.740+0000 7f13837fe700 1 --2- 172.21.15.46:0/1836794757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f172b1d480 0x55f172b1d890 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f137c011680 tx=0x7f137c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6286d0eef7e45f65 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.740+0000 7f1369ffb700 1 -- 172.21.15.46:0/1836794757 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c019030 con 0x55f172b1d480 2024-05-09T04:54:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.740+0000 7f1369ffb700 1 -- 172.21.15.46:0/1836794757 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f137c0092e0 con 0x55f172b1d480 2024-05-09T04:54:48.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.740+0000 7f1369ffb700 1 -- 172.21.15.46:0/1836794757 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137c007500 con 0x55f172b1d480 2024-05-09T04:54:48.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.741+0000 7f13982f4b80 1 -- 172.21.15.46:0/1836794757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f172b1d480 msgr2=0x55f172b1d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.741+0000 7f13982f4b80 1 --2- 172.21.15.46:0/1836794757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f172b1d480 0x55f172b1d890 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f137c011680 tx=0x7f137c009920 comp rx=0 tx=0).stop 2024-05-09T04:54:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.741+0000 7f13982f4b80 1 -- 172.21.15.46:0/1836794757 shutdown_connections 2024-05-09T04:54:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.741+0000 7f13982f4b80 1 --2- 172.21.15.46:0/1836794757 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f172b1d480 0x55f172b1d890 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.741+0000 7f13982f4b80 1 --2- 172.21.15.46:0/1836794757 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 0x55f172b232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.741+0000 7f13982f4b80 1 --2- 172.21.15.46:0/1836794757 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f1729a0520 0x55f172b23800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.741+0000 7f13982f4b80 1 -- 172.21.15.46:0/1836794757 >> 172.21.15.46:0/1836794757 conn(0x55f1729a3ff0 msgr2=0x55f172b182a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.741+0000 7f13982f4b80 1 -- 172.21.15.46:0/1836794757 shutdown_connections 2024-05-09T04:54:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.741+0000 7f13982f4b80 1 -- 172.21.15.46:0/1836794757 wait complete. 2024-05-09T04:54:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.742+0000 7f13982f4b80 1 -- start start 2024-05-09T04:54:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.742+0000 7f13982f4b80 1 Processor -- start 2024-05-09T04:54:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.742+0000 7f13982f4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1729a0520 0x55f172b27160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.742+0000 7f13982f4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 0x55f172b276a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.742+0000 7f13982f4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f172b1d480 0x55f172b27be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.742+0000 7f13982f4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f172b1e1e0 con 0x55f1729a0520 2024-05-09T04:54:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.742+0000 7f13982f4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f1729daba0 con 0x55f172a1ad80 2024-05-09T04:54:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.742+0000 7f13982f4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f172b2a0a0 con 0x55f172b1d480 2024-05-09T04:54:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.742+0000 7f13982f4b80 10 client.0 _ll_register_callbacks cb 0x55f1728b1740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.742+0000 7f1382ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1729a0520 0x55f172b27160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.742+0000 7f1382ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1729a0520 0x55f172b27160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45228/0 (socket says 172.21.15.46:45228) 2024-05-09T04:54:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.742+0000 7f1382ffd700 1 -- 172.21.15.46:0/3263331317 learned_addr learned my addr 172.21.15.46:0/3263331317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f13827fc700 1 --2- 172.21.15.46:0/3263331317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 0x55f172b276a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f13837fe700 1 --2- 172.21.15.46:0/3263331317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f172b1d480 0x55f172b27be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f1382ffd700 1 -- 172.21.15.46:0/3263331317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f172b1d480 msgr2=0x55f172b27be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f1382ffd700 1 --2- 172.21.15.46:0/3263331317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f172b1d480 0x55f172b27be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f1382ffd700 1 -- 172.21.15.46:0/3263331317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 msgr2=0x55f172b276a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f1382ffd700 1 --2- 172.21.15.46:0/3263331317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 0x55f172b276a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f1382ffd700 1 -- 172.21.15.46:0/3263331317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1728bb510 con 0x55f1729a0520 2024-05-09T04:54:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f13827fc700 1 --2- 172.21.15.46:0/3263331317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 0x55f172b276a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f1382ffd700 1 --2- 172.21.15.46:0/3263331317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1729a0520 0x55f172b27160 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1374015000 tx=0x7f1374009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71db4dea226ecd6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f1362ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f1362ffd700 1 -- 172.21.15.46:0/3263331317 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137400f660 con 0x55f1729a0520 2024-05-09T04:54:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f13982f4b80 1 -- 172.21.15.46:0/3263331317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f137c0092e0 con 0x55f1729a0520 2024-05-09T04:54:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f1362ffd700 1 -- 172.21.15.46:0/3263331317 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f137400f800 con 0x55f1729a0520 2024-05-09T04:54:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f1362ffd700 1 -- 172.21.15.46:0/3263331317 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1374005690 con 0x55f1729a0520 2024-05-09T04:54:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f13982f4b80 10 client.16329 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.743+0000 7f13982f4b80 1 -- 172.21.15.46:0/3263331317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f172b34b80 con 0x55f1729a0520 2024-05-09T04:54:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.744+0000 7f1362ffd700 1 -- 172.21.15.46:0/3263331317 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f137c0092e0 con 0x55f1729a0520 2024-05-09T04:54:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.744+0000 7f1362ffd700 1 -- 172.21.15.46:0/3263331317 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f137402e070 con 0x55f1729a0520 2024-05-09T04:54:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.744+0000 7f13982f4b80 10 client.16329 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:48.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.744+0000 7f13982f4b80 10 client.16329 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.744+0000 7f13982f4b80 20 client.16329 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.744+0000 7f13982f4b80 1 --2- 172.21.15.46:0/3263331317 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f172b37100 0x55f172b39520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.744+0000 7f13982f4b80 4 client.16329 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.744+0000 7f13982f4b80 1 -- 172.21.15.46:0/3263331317 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f172a4db00 con 0x55f172b37100 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.744+0000 7f13827fc700 1 --2- 172.21.15.46:0/3263331317 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f172b37100 0x55f172b39520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.745+0000 7f1362ffd700 10 client.16329 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.745+0000 7f13827fc700 1 --2- 172.21.15.46:0/3263331317 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f172b37100 0x55f172b39520 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=128151d0063f6ac2 server_cookie=545d0678fd35271c in_seq=0 out_seq=0 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.745+0000 7f1362ffd700 1 -- 172.21.15.46:0/3263331317 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f172a4db00 con 0x55f172b37100 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.745+0000 7f1362ffd700 10 client.16329 handle_command_reply: tid=0 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.746+0000 7f13982f4b80 1 client.16329 shutdown 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.746+0000 7f13982f4b80 1 -- 172.21.15.46:0/3263331317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1729a0520 msgr2=0x55f172b27160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.746+0000 7f13982f4b80 1 --2- 172.21.15.46:0/3263331317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1729a0520 0x55f172b27160 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1374015000 tx=0x7f1374009ca0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.746+0000 7f13982f4b80 1 -- 172.21.15.46:0/3263331317 shutdown_connections 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.746+0000 7f13982f4b80 1 --2- 172.21.15.46:0/3263331317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f172b1d480 0x55f172b27be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.746+0000 7f13982f4b80 1 --2- 172.21.15.46:0/3263331317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1729a0520 0x55f172b27160 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.746+0000 7f13982f4b80 1 --2- 172.21.15.46:0/3263331317 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f172b37100 0x55f172b39520 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.746+0000 7f13982f4b80 1 --2- 172.21.15.46:0/3263331317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 0x55f172b276a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.746+0000 7f13982f4b80 1 -- 172.21.15.46:0/3263331317 >> 172.21.15.46:0/3263331317 conn(0x55f1729a3ff0 msgr2=0x55f172b12070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.746+0000 7f1362ffd700 0 client.16329 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.746+0000 7f13982f4b80 1 -- 172.21.15.46:0/3263331317 shutdown_connections 2024-05-09T04:54:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.746+0000 7f13982f4b80 1 -- 172.21.15.46:0/3263331317 wait complete. 2024-05-09T04:54:48.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.746+0000 7f13982f4b80 20 client.16329 trim_cache size 0 max 16384 2024-05-09T04:54:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.767+0000 7f13982f4b80 1 Processor -- start 2024-05-09T04:54:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.767+0000 7f13982f4b80 1 -- start start 2024-05-09T04:54:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.767+0000 7f13982f4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f1729a0520 0x55f172b1cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.767+0000 7f13982f4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f172a1ad80 0x55f172b276d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.767+0000 7f13982f4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f172b1d480 0x55f172b32ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.767+0000 7f13982f4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13841b3a80 con 0x55f172a1ad80 2024-05-09T04:54:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.767+0000 7f13982f4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f13841b1310 con 0x55f1729a0520 2024-05-09T04:54:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.767+0000 7f13982f4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f13841af160 con 0x55f172b1d480 2024-05-09T04:54:48.769 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.767+0000 7f13827fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f172a1ad80 0x55f172b276d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.767+0000 7f1382ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f1729a0520 0x55f172b1cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.767+0000 7f13827fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f172a1ad80 0x55f172b276d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45230/0 (socket says 172.21.15.46:45230) 2024-05-09T04:54:48.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.767+0000 7f13827fc700 1 -- 172.21.15.46:0/2552296826 learned_addr learned my addr 172.21.15.46:0/2552296826 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:48.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.767+0000 7f13837fe700 1 --2- 172.21.15.46:0/2552296826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f172b1d480 0x55f172b32ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.768+0000 7f13827fc700 1 -- 172.21.15.46:0/2552296826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f172b1d480 msgr2=0x55f172b32ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:48.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.768+0000 7f13827fc700 1 --2- 172.21.15.46:0/2552296826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f172b1d480 0x55f172b32ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.768+0000 7f13827fc700 1 -- 172.21.15.46:0/2552296826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f1729a0520 msgr2=0x55f172b1cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:48.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.768+0000 7f13827fc700 1 --2- 172.21.15.46:0/2552296826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f1729a0520 0x55f172b1cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.768+0000 7f13827fc700 1 -- 172.21.15.46:0/2552296826 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f172b19140 con 0x55f172a1ad80 2024-05-09T04:54:48.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.768+0000 7f1382ffd700 1 --2- 172.21.15.46:0/2552296826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f1729a0520 0x55f172b1cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:48.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.768+0000 7f13827fc700 1 --2- 172.21.15.46:0/2552296826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f172a1ad80 0x55f172b276d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f13780a90b0 tx=0x7f13780109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a3bf73e4511b0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.769+0000 7f13697fa700 1 -- 172.21.15.46:0/2552296826 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13780c3070 con 0x55f172a1ad80 2024-05-09T04:54:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.769+0000 7f13697fa700 1 -- 172.21.15.46:0/2552296826 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13780ad760 con 0x55f172a1ad80 2024-05-09T04:54:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.769+0000 7f13697fa700 1 -- 172.21.15.46:0/2552296826 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1378020040 con 0x55f172a1ad80 2024-05-09T04:54:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.770+0000 7f13982f4b80 1 -- 172.21.15.46:0/2552296826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f172a1ad80 msgr2=0x55f172b276d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.770+0000 7f13982f4b80 1 --2- 172.21.15.46:0/2552296826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f172a1ad80 0x55f172b276d0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f13780a90b0 tx=0x7f13780109b0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.770+0000 7f13982f4b80 1 -- 172.21.15.46:0/2552296826 shutdown_connections 2024-05-09T04:54:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.770+0000 7f13982f4b80 1 --2- 172.21.15.46:0/2552296826 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f172b1d480 0x55f172b32ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.770+0000 7f13982f4b80 1 --2- 172.21.15.46:0/2552296826 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f1729a0520 0x55f172b1cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.770+0000 7f13982f4b80 1 --2- 172.21.15.46:0/2552296826 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f172a1ad80 0x55f172b276d0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.770+0000 7f13982f4b80 1 -- 172.21.15.46:0/2552296826 >> 172.21.15.46:0/2552296826 conn(0x55f1729a3ff0 msgr2=0x55f172b126e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.770+0000 7f13982f4b80 1 -- 172.21.15.46:0/2552296826 shutdown_connections 2024-05-09T04:54:48.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.770+0000 7f13982f4b80 1 -- 172.21.15.46:0/2552296826 wait complete. 2024-05-09T04:54:48.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.770+0000 7f13982f4b80 1 -- start start 2024-05-09T04:54:48.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.771+0000 7f13982f4b80 1 Processor -- start 2024-05-09T04:54:48.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.771+0000 7f13982f4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1729a0520 0x55f172b2d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.771+0000 7f13982f4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 0x55f172b2d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.771+0000 7f13982f4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f172b1d480 0x55f172b2db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.771+0000 7f13982f4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13841af7d0 con 0x55f1729a0520 2024-05-09T04:54:48.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.771+0000 7f13982f4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f13841b0c40 con 0x55f172a1ad80 2024-05-09T04:54:48.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.771+0000 7f13982f4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f13841afea0 con 0x55f172b1d480 2024-05-09T04:54:48.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.771+0000 7f13982f4b80 10 client.0 _ll_register_callbacks cb 0x55f1728b1740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:48.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.771+0000 7f1382ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1729a0520 0x55f172b2d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.771+0000 7f1382ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1729a0520 0x55f172b2d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45244/0 (socket says 172.21.15.46:45244) 2024-05-09T04:54:48.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.771+0000 7f1382ffd700 1 -- 172.21.15.46:0/2176386055 learned_addr learned my addr 172.21.15.46:0/2176386055 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:48.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.771+0000 7f13827fc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 0x55f172b2d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.771+0000 7f13837fe700 1 --2- 172.21.15.46:0/2176386055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f172b1d480 0x55f172b2db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.772+0000 7f1382ffd700 1 -- 172.21.15.46:0/2176386055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f172b1d480 msgr2=0x55f172b2db40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.772+0000 7f1382ffd700 1 --2- 172.21.15.46:0/2176386055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f172b1d480 0x55f172b2db40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.772+0000 7f1382ffd700 1 -- 172.21.15.46:0/2176386055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 msgr2=0x55f172b2d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.772+0000 7f1382ffd700 1 --2- 172.21.15.46:0/2176386055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 0x55f172b2d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.772+0000 7f1382ffd700 1 -- 172.21.15.46:0/2176386055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f172ba0b00 con 0x55f1729a0520 2024-05-09T04:54:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.772+0000 7f13827fc700 1 --2- 172.21.15.46:0/2176386055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 0x55f172b2d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.772+0000 7f1382ffd700 1 --2- 172.21.15.46:0/2176386055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1729a0520 0x55f172b2d0a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f137400b2e0 tx=0x7f137400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75f484da38d47c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.772+0000 7f1362ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.772+0000 7f1362ffd700 1 -- 172.21.15.46:0/2176386055 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f137402f7f0 con 0x55f1729a0520 2024-05-09T04:54:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.772+0000 7f1362ffd700 1 -- 172.21.15.46:0/2176386055 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f1374030440 con 0x55f1729a0520 2024-05-09T04:54:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.772+0000 7f13982f4b80 1 -- 172.21.15.46:0/2176386055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13780ad760 con 0x55f1729a0520 2024-05-09T04:54:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.772+0000 7f1362ffd700 1 -- 172.21.15.46:0/2176386055 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f137402f990 con 0x55f1729a0520 2024-05-09T04:54:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.772+0000 7f13982f4b80 10 client.16335 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.772+0000 7f13982f4b80 1 -- 172.21.15.46:0/2176386055 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f172ba1da0 con 0x55f1729a0520 2024-05-09T04:54:48.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.772+0000 7f1362ffd700 1 -- 172.21.15.46:0/2176386055 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13780ad760 con 0x55f1729a0520 2024-05-09T04:54:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.773+0000 7f1362ffd700 1 -- 172.21.15.46:0/2176386055 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f137403f020 con 0x55f1729a0520 2024-05-09T04:54:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.773+0000 7f13982f4b80 10 client.16335 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.773+0000 7f13982f4b80 10 client.16335 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.773+0000 7f13982f4b80 20 client.16335 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.773+0000 7f13982f4b80 1 --2- 172.21.15.46:0/2176386055 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f172ba22b0 0x55f172ba46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.773+0000 7f13982f4b80 4 client.16335 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:54:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.773+0000 7f13982f4b80 1 -- 172.21.15.46:0/2176386055 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f172a47ba0 con 0x55f172ba22b0 2024-05-09T04:54:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.773+0000 7f13827fc700 1 --2- 172.21.15.46:0/2176386055 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f172ba22b0 0x55f172ba46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.773+0000 7f1362ffd700 10 client.16335 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.773+0000 7f13827fc700 1 --2- 172.21.15.46:0/2176386055 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f172ba22b0 0x55f172ba46d0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7dc15f02cd85c91b server_cookie=84e0f60686eec539 in_seq=0 out_seq=0 2024-05-09T04:54:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.774+0000 7f1362ffd700 1 -- 172.21.15.46:0/2176386055 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f172a47ba0 con 0x55f172ba22b0 2024-05-09T04:54:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.774+0000 7f1362ffd700 10 client.16335 handle_command_reply: tid=0 2024-05-09T04:54:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.774+0000 7f13982f4b80 1 client.16335 shutdown 2024-05-09T04:54:48.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.775+0000 7f13982f4b80 1 -- 172.21.15.46:0/2176386055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1729a0520 msgr2=0x55f172b2d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.775+0000 7f13982f4b80 1 --2- 172.21.15.46:0/2176386055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1729a0520 0x55f172b2d0a0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f137400b2e0 tx=0x7f137400f710 comp rx=0 tx=0).stop 2024-05-09T04:54:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.775+0000 7f13982f4b80 1 -- 172.21.15.46:0/2176386055 shutdown_connections 2024-05-09T04:54:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.775+0000 7f13982f4b80 1 --2- 172.21.15.46:0/2176386055 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f172b1d480 0x55f172b2db40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.775+0000 7f13982f4b80 1 --2- 172.21.15.46:0/2176386055 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f1729a0520 0x55f172b2d0a0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.775+0000 7f13982f4b80 1 --2- 172.21.15.46:0/2176386055 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f172ba22b0 0x55f172ba46d0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.775+0000 7f13982f4b80 1 --2- 172.21.15.46:0/2176386055 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f172a1ad80 0x55f172b2d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.775+0000 7f13982f4b80 1 -- 172.21.15.46:0/2176386055 >> 172.21.15.46:0/2176386055 conn(0x55f1729a3ff0 msgr2=0x55f172b20d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.775+0000 7f1362ffd700 0 client.16335 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.775+0000 7f13982f4b80 1 -- 172.21.15.46:0/2176386055 shutdown_connections 2024-05-09T04:54:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.775+0000 7f13982f4b80 1 -- 172.21.15.46:0/2176386055 wait complete. 2024-05-09T04:54:48.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.775+0000 7f13982f4b80 20 client.16335 trim_cache size 0 max 16384 2024-05-09T04:54:48.780 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:54:48.781 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:54:48.781 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "39a5e7ec-14b2-4b52-be79-0aaff1d02568", 2024-05-09T04:54:48.781 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:54:48.781 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:54:48.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.776+0000 7f13898de700 1 -- 172.21.15.46:0/3886092966 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f136c0b29f0 msgr2=0x7f136c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:48.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.776+0000 7f13898de700 1 --2- 172.21.15.46:0/3886092966 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f136c0b29f0 0x7f136c0b4f00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1374006010 tx=0x7f1374005960 comp rx=0 tx=0).stop 2024-05-09T04:54:48.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.776+0000 7f13898de700 1 -- 172.21.15.46:0/3886092966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384103310 msgr2=0x7f1384104ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:48.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.776+0000 7f13898de700 1 --2- 172.21.15.46:0/3886092966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384103310 0x7f1384104ad0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f137800d980 tx=0x7f137800dc90 comp rx=0 tx=0).stop 2024-05-09T04:54:48.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.776+0000 7f13898de700 1 -- 172.21.15.46:0/3886092966 shutdown_connections 2024-05-09T04:54:48.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.776+0000 7f13898de700 1 --2- 172.21.15.46:0/3886092966 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f136c0b29f0 0x7f136c0b4f00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.776+0000 7f13898de700 1 --2- 172.21.15.46:0/3886092966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13841089f0 0x7f1384105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.776+0000 7f13898de700 1 --2- 172.21.15.46:0/3886092966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13841029c0 0x7f1384104590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.776+0000 7f13898de700 1 --2- 172.21.15.46:0/3886092966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1384103310 0x7f1384104ad0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:48.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.776+0000 7f13898de700 1 -- 172.21.15.46:0/3886092966 >> 172.21.15.46:0/3886092966 conn(0x7f13840fe6f0 msgr2=0x7f1384100f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:48.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.776+0000 7f13898de700 1 -- 172.21.15.46:0/3886092966 shutdown_connections 2024-05-09T04:54:48.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:48.776+0000 7f13898de700 1 -- 172.21.15.46:0/3886092966 wait complete. 2024-05-09T04:54:49.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:48 smithi110 ceph-mon[34904]: pgmap v114: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:54:49.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:48 smithi046 ceph-mon[30136]: pgmap v114: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:54:49.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:48 smithi072 ceph-mon[34088]: pgmap v114: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:54:49.613 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:54:49.872 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:49.996 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:49 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:54:49.996 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:49 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:49.996 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:49 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:54:49.996 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:49 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:50.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:49 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:54:50.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:49 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:50.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:49 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:54:50.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:49 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:50.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:49 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:54:50.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:49 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:50.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:49 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:54:50.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:49 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:54:50.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:50 smithi046 ceph-mon[30136]: pgmap v115: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:54:51.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:50 smithi110 ceph-mon[34904]: pgmap v115: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:54:51.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:50 smithi072 ceph-mon[34088]: pgmap v115: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:54:51.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.700+0000 7fab736cb700 1 -- 172.21.15.46:0/3354001196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab6c1089f0 msgr2=0x7fab6c108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:51.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.700+0000 7fab736cb700 1 --2- 172.21.15.46:0/3354001196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab6c1089f0 0x7fab6c108e00 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fab6800aaa0 tx=0x7fab6800adb0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.700+0000 7fab736cb700 1 -- 172.21.15.46:0/3354001196 shutdown_connections 2024-05-09T04:54:51.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.700+0000 7fab736cb700 1 --2- 172.21.15.46:0/3354001196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab6c103310 0x7fab6c10f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.700+0000 7fab736cb700 1 --2- 172.21.15.46:0/3354001196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab6c1089f0 0x7fab6c108e00 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.700+0000 7fab736cb700 1 --2- 172.21.15.46:0/3354001196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab6c1029c0 0x7fab6c102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.700+0000 7fab736cb700 1 -- 172.21.15.46:0/3354001196 >> 172.21.15.46:0/3354001196 conn(0x7fab6c0fe540 msgr2=0x7fab6c100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:51.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.700+0000 7fab736cb700 1 -- 172.21.15.46:0/3354001196 shutdown_connections 2024-05-09T04:54:51.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.700+0000 7fab736cb700 1 -- 172.21.15.46:0/3354001196 wait complete. 2024-05-09T04:54:51.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.701+0000 7fab736cb700 1 Processor -- start 2024-05-09T04:54:51.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.701+0000 7fab736cb700 1 -- start start 2024-05-09T04:54:51.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.702+0000 7fab736cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab6c1029c0 0x7fab6c19ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.702+0000 7fab736cb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab6c103310 0x7fab6c19b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.702+0000 7fab736cb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab6c1089f0 0x7fab6c19f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.702+0000 7fab736cb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c1122e0 con 0x7fab6c1029c0 2024-05-09T04:54:51.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.702+0000 7fab736cb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c112160 con 0x7fab6c1089f0 2024-05-09T04:54:51.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.702+0000 7fab736cb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c112460 con 0x7fab6c103310 2024-05-09T04:54:51.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.702+0000 7fab71467700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab6c1029c0 0x7fab6c19ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.702+0000 7fab71467700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab6c1029c0 0x7fab6c19ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45256/0 (socket says 172.21.15.46:45256) 2024-05-09T04:54:51.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.702+0000 7fab71467700 1 -- 172.21.15.46:0/170275759 learned_addr learned my addr 172.21.15.46:0/170275759 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:51.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.702+0000 7fab71c68700 1 --2- 172.21.15.46:0/170275759 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab6c1089f0 0x7fab6c19f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.702+0000 7fab70c66700 1 --2- 172.21.15.46:0/170275759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab6c103310 0x7fab6c19b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.703+0000 7fab71467700 1 -- 172.21.15.46:0/170275759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab6c103310 msgr2=0x7fab6c19b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:51.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.703+0000 7fab71467700 1 --2- 172.21.15.46:0/170275759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab6c103310 0x7fab6c19b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.703+0000 7fab71467700 1 -- 172.21.15.46:0/170275759 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab6c1089f0 msgr2=0x7fab6c19f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:51.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.703+0000 7fab71467700 1 --2- 172.21.15.46:0/170275759 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab6c1089f0 0x7fab6c19f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.703+0000 7fab71467700 1 -- 172.21.15.46:0/170275759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab6c19fc90 con 0x7fab6c1029c0 2024-05-09T04:54:51.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.703+0000 7fab71467700 1 --2- 172.21.15.46:0/170275759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab6c1029c0 0x7fab6c19ae70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fab68003a70 tx=0x7fab68003aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:51.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.703+0000 7fab627fc700 1 -- 172.21.15.46:0/170275759 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68023030 con 0x7fab6c1029c0 2024-05-09T04:54:51.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.703+0000 7fab627fc700 1 -- 172.21.15.46:0/170275759 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab68004830 con 0x7fab6c1029c0 2024-05-09T04:54:51.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.703+0000 7fab736cb700 1 -- 172.21.15.46:0/170275759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab6c19ff20 con 0x7fab6c1029c0 2024-05-09T04:54:51.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.703+0000 7fab627fc700 1 -- 172.21.15.46:0/170275759 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68005200 con 0x7fab6c1029c0 2024-05-09T04:54:51.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.703+0000 7fab736cb700 1 -- 172.21.15.46:0/170275759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab6c1a0450 con 0x7fab6c1029c0 2024-05-09T04:54:51.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.706+0000 7fab627fc700 1 -- 172.21.15.46:0/170275759 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fab68026070 con 0x7fab6c1029c0 2024-05-09T04:54:51.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.707+0000 7fab627fc700 1 --2- 172.21.15.46:0/170275759 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fab580b2a70 0x7fab580b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.707+0000 7fab70c66700 1 --2- 172.21.15.46:0/170275759 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fab580b2a70 0x7fab580b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.707+0000 7fab627fc700 1 -- 172.21.15.46:0/170275759 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fab680e3950 con 0x7fab6c1029c0 2024-05-09T04:54:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.707+0000 7fab70c66700 1 --2- 172.21.15.46:0/170275759 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fab580b2a70 0x7fab580b4f80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fab6c100200 tx=0x7fab5c00a300 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.707+0000 7fab820e1b80 1 Processor -- start 2024-05-09T04:54:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab820e1b80 1 -- start start 2024-05-09T04:54:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab820e1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b2f6d00 0x556e8b3ff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab820e1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b27c4a0 0x556e8b3ff780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab820e1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b3f9400 0x556e8b3f9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab820e1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556e8b31c0a0 con 0x556e8b2f6d00 2024-05-09T04:54:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab820e1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556e8b2c19a0 con 0x556e8b27c4a0 2024-05-09T04:54:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab820e1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556e8b318060 con 0x556e8b3f9400 2024-05-09T04:54:51.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab71467700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b2f6d00 0x556e8b3ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab71467700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b2f6d00 0x556e8b3ff240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45258/0 (socket says 172.21.15.46:45258) 2024-05-09T04:54:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab71467700 1 -- 172.21.15.46:0/4037862406 learned_addr learned my addr 172.21.15.46:0/4037862406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab70c66700 1 --2- 172.21.15.46:0/4037862406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b27c4a0 0x556e8b3ff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab71c68700 1 --2- 172.21.15.46:0/4037862406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b3f9400 0x556e8b3f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab71467700 1 -- 172.21.15.46:0/4037862406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b3f9400 msgr2=0x556e8b3f9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab71467700 1 --2- 172.21.15.46:0/4037862406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b3f9400 0x556e8b3f9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab71467700 1 -- 172.21.15.46:0/4037862406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b27c4a0 msgr2=0x556e8b3ff780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:51.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab71467700 1 --2- 172.21.15.46:0/4037862406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b27c4a0 0x556e8b3ff780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab71467700 1 -- 172.21.15.46:0/4037862406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e8b2b2500 con 0x556e8b2f6d00 2024-05-09T04:54:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.708+0000 7fab70c66700 1 --2- 172.21.15.46:0/4037862406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b27c4a0 0x556e8b3ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.709+0000 7fab71467700 1 --2- 172.21.15.46:0/4037862406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b2f6d00 0x556e8b3ff240 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fab680a6cc0 tx=0x7fab6800a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff0616f5d5afa64 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.709+0000 7fab577fe700 1 -- 172.21.15.46:0/4037862406 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab680cbe60 con 0x556e8b2f6d00 2024-05-09T04:54:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.709+0000 7fab577fe700 1 -- 172.21.15.46:0/4037862406 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab680b1070 con 0x556e8b2f6d00 2024-05-09T04:54:51.715 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.709+0000 7fab577fe700 1 -- 172.21.15.46:0/4037862406 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab680e12a0 con 0x556e8b2f6d00 2024-05-09T04:54:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.710+0000 7fab820e1b80 1 -- 172.21.15.46:0/4037862406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b2f6d00 msgr2=0x556e8b3ff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.710+0000 7fab820e1b80 1 --2- 172.21.15.46:0/4037862406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b2f6d00 0x556e8b3ff240 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fab680a6cc0 tx=0x7fab6800a420 comp rx=0 tx=0).stop 2024-05-09T04:54:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.710+0000 7fab820e1b80 1 -- 172.21.15.46:0/4037862406 shutdown_connections 2024-05-09T04:54:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.710+0000 7fab820e1b80 1 --2- 172.21.15.46:0/4037862406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b3f9400 0x556e8b3f9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.710+0000 7fab820e1b80 1 --2- 172.21.15.46:0/4037862406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b2f6d00 0x556e8b3ff240 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.710+0000 7fab820e1b80 1 --2- 172.21.15.46:0/4037862406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b27c4a0 0x556e8b3ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.710+0000 7fab820e1b80 1 -- 172.21.15.46:0/4037862406 >> 172.21.15.46:0/4037862406 conn(0x556e8b27ff70 msgr2=0x556e8b3f4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.710+0000 7fab820e1b80 1 -- 172.21.15.46:0/4037862406 shutdown_connections 2024-05-09T04:54:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.710+0000 7fab820e1b80 1 -- 172.21.15.46:0/4037862406 wait complete. 2024-05-09T04:54:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.710+0000 7fab820e1b80 1 -- start start 2024-05-09T04:54:51.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.710+0000 7fab820e1b80 1 Processor -- start 2024-05-09T04:54:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab820e1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b27c4a0 0x556e8b408a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab820e1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b2f6d00 0x556e8b408fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab820e1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b3f9400 0x556e8b402e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab820e1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556e8b31c0a0 con 0x556e8b2f6d00 2024-05-09T04:54:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab820e1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556e8b2b6b20 con 0x556e8b27c4a0 2024-05-09T04:54:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab820e1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556e8b3f9fb0 con 0x556e8b3f9400 2024-05-09T04:54:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab820e1b80 10 client.0 _ll_register_callbacks cb 0x556e8b18d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab70c66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b2f6d00 0x556e8b408fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab70c66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b2f6d00 0x556e8b408fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45264/0 (socket says 172.21.15.46:45264) 2024-05-09T04:54:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab70c66700 1 -- 172.21.15.46:0/1998796690 learned_addr learned my addr 172.21.15.46:0/1998796690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:51.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab71467700 1 --2- 172.21.15.46:0/1998796690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b27c4a0 0x556e8b408a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab71c68700 1 --2- 172.21.15.46:0/1998796690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b3f9400 0x556e8b402e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab70c66700 1 -- 172.21.15.46:0/1998796690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b3f9400 msgr2=0x556e8b402e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab70c66700 1 --2- 172.21.15.46:0/1998796690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b3f9400 0x556e8b402e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab70c66700 1 -- 172.21.15.46:0/1998796690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b27c4a0 msgr2=0x556e8b408a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab70c66700 1 --2- 172.21.15.46:0/1998796690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b27c4a0 0x556e8b408a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.711+0000 7fab70c66700 1 -- 172.21.15.46:0/1998796690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e8b197490 con 0x556e8b2f6d00 2024-05-09T04:54:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.712+0000 7fab70c66700 1 --2- 172.21.15.46:0/1998796690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b2f6d00 0x556e8b408fc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fab5c01daf0 tx=0x7fab5c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed95d1165750b9d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.712+0000 7fab54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.712+0000 7fab54ff9700 1 -- 172.21.15.46:0/1998796690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab5c016070 con 0x556e8b2f6d00 2024-05-09T04:54:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.712+0000 7fab820e1b80 1 -- 172.21.15.46:0/1998796690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab680b1070 con 0x556e8b2f6d00 2024-05-09T04:54:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.712+0000 7fab54ff9700 1 -- 172.21.15.46:0/1998796690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fab5c004a30 con 0x556e8b2f6d00 2024-05-09T04:54:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.712+0000 7fab54ff9700 1 -- 172.21.15.46:0/1998796690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab5c010700 con 0x556e8b2f6d00 2024-05-09T04:54:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.712+0000 7fab820e1b80 10 client.16347 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:51.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.712+0000 7fab820e1b80 1 -- 172.21.15.46:0/1998796690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556e8b4106c0 con 0x556e8b2f6d00 2024-05-09T04:54:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.712+0000 7fab54ff9700 1 -- 172.21.15.46:0/1998796690 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab680b1070 con 0x556e8b2f6d00 2024-05-09T04:54:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.712+0000 7fab54ff9700 1 -- 172.21.15.46:0/1998796690 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fab5c027a40 con 0x556e8b2f6d00 2024-05-09T04:54:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.713+0000 7fab820e1b80 10 client.16347 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.713+0000 7fab820e1b80 10 client.16347 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.713+0000 7fab820e1b80 20 client.16347 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.713+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1998796690 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556e8b412d00 0x556e8b415120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.713+0000 7fab820e1b80 4 client.16347 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.713+0000 7fab820e1b80 1 -- 172.21.15.46:0/1998796690 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e8b329a80 con 0x556e8b412d00 2024-05-09T04:54:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.713+0000 7fab71467700 1 --2- 172.21.15.46:0/1998796690 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556e8b412d00 0x556e8b415120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.713+0000 7fab54ff9700 10 client.16347 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.713+0000 7fab71467700 1 --2- 172.21.15.46:0/1998796690 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556e8b412d00 0x556e8b415120 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8037e84fac36b63 server_cookie=cd8cac0ae6b6bf83 in_seq=0 out_seq=0 2024-05-09T04:54:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.714+0000 7fab54ff9700 1 -- 172.21.15.46:0/1998796690 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556e8b329a80 con 0x556e8b412d00 2024-05-09T04:54:51.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.714+0000 7fab54ff9700 10 client.16347 handle_command_reply: tid=0 2024-05-09T04:54:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.714+0000 7fab820e1b80 1 client.16347 shutdown 2024-05-09T04:54:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.714+0000 7fab820e1b80 1 -- 172.21.15.46:0/1998796690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b2f6d00 msgr2=0x556e8b408fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.714+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1998796690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b2f6d00 0x556e8b408fc0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fab5c01daf0 tx=0x7fab5c0047c0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.714+0000 7fab820e1b80 1 -- 172.21.15.46:0/1998796690 shutdown_connections 2024-05-09T04:54:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.714+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1998796690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b3f9400 0x556e8b402e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.714+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1998796690 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556e8b412d00 0x556e8b415120 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.714+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1998796690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b27c4a0 0x556e8b408a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.714+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1998796690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b2f6d00 0x556e8b408fc0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.714+0000 7fab54ff9700 0 client.16347 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.714+0000 7fab820e1b80 1 -- 172.21.15.46:0/1998796690 >> 172.21.15.46:0/1998796690 conn(0x556e8b27ff70 msgr2=0x556e8b3edff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.714+0000 7fab820e1b80 1 -- 172.21.15.46:0/1998796690 shutdown_connections 2024-05-09T04:54:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.714+0000 7fab820e1b80 1 -- 172.21.15.46:0/1998796690 wait complete. 2024-05-09T04:54:51.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.715+0000 7fab820e1b80 20 client.16347 trim_cache size 0 max 16384 2024-05-09T04:54:51.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.735+0000 7fab820e1b80 1 Processor -- start 2024-05-09T04:54:51.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.735+0000 7fab820e1b80 1 -- start start 2024-05-09T04:54:51.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.735+0000 7fab820e1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b27c4a0 0x556e8b402160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.735+0000 7fab820e1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b2f6d00 0x556e8b3f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.735+0000 7fab820e1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b3f9400 0x556e8b41a900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.735+0000 7fab820e1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c1b3990 con 0x556e8b3f9400 2024-05-09T04:54:51.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.735+0000 7fab820e1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c1b1220 con 0x556e8b2f6d00 2024-05-09T04:54:51.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.735+0000 7fab820e1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c1af070 con 0x556e8b27c4a0 2024-05-09T04:54:51.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.735+0000 7fab71c68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b3f9400 0x556e8b41a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.735+0000 7fab71c68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b3f9400 0x556e8b41a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45274/0 (socket says 172.21.15.46:45274) 2024-05-09T04:54:51.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.735+0000 7fab71c68700 1 -- 172.21.15.46:0/1370432605 learned_addr learned my addr 172.21.15.46:0/1370432605 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:51.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.736+0000 7fab70c66700 1 --2- 172.21.15.46:0/1370432605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b2f6d00 0x556e8b3f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.736+0000 7fab71467700 1 --2- 172.21.15.46:0/1370432605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b27c4a0 0x556e8b402160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.736+0000 7fab71c68700 1 -- 172.21.15.46:0/1370432605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b27c4a0 msgr2=0x556e8b402160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:51.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.736+0000 7fab71c68700 1 --2- 172.21.15.46:0/1370432605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b27c4a0 0x556e8b402160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.736+0000 7fab71c68700 1 -- 172.21.15.46:0/1370432605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b2f6d00 msgr2=0x556e8b3f42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.736+0000 7fab71c68700 1 --2- 172.21.15.46:0/1370432605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b2f6d00 0x556e8b3f42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.736+0000 7fab71c68700 1 -- 172.21.15.46:0/1370432605 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e8b3f09d0 con 0x556e8b3f9400 2024-05-09T04:54:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.736+0000 7fab71467700 1 --2- 172.21.15.46:0/1370432605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b27c4a0 0x556e8b402160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:54:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.736+0000 7fab71c68700 1 --2- 172.21.15.46:0/1370432605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b3f9400 0x556e8b41a900 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fab64006090 tx=0x7fab640157f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9faeaf648c418ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.736+0000 7fab56ffd700 1 -- 172.21.15.46:0/1370432605 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64003a00 con 0x556e8b3f9400 2024-05-09T04:54:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.736+0000 7fab56ffd700 1 -- 172.21.15.46:0/1370432605 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab6401a070 con 0x556e8b3f9400 2024-05-09T04:54:51.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.736+0000 7fab56ffd700 1 -- 172.21.15.46:0/1370432605 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64004280 con 0x556e8b3f9400 2024-05-09T04:54:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.737+0000 7fab820e1b80 1 -- 172.21.15.46:0/1370432605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b3f9400 msgr2=0x556e8b41a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.737+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1370432605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b3f9400 0x556e8b41a900 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fab64006090 tx=0x7fab640157f0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.737+0000 7fab820e1b80 1 -- 172.21.15.46:0/1370432605 shutdown_connections 2024-05-09T04:54:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.737+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1370432605 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b3f9400 0x556e8b41a900 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.737+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1370432605 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b27c4a0 0x556e8b402160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.737+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1370432605 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b2f6d00 0x556e8b3f42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.737+0000 7fab820e1b80 1 -- 172.21.15.46:0/1370432605 >> 172.21.15.46:0/1370432605 conn(0x556e8b27ff70 msgr2=0x556e8b3fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:51.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.737+0000 7fab820e1b80 1 -- 172.21.15.46:0/1370432605 shutdown_connections 2024-05-09T04:54:51.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.737+0000 7fab820e1b80 1 -- 172.21.15.46:0/1370432605 wait complete. 2024-05-09T04:54:51.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.737+0000 7fab820e1b80 1 -- start start 2024-05-09T04:54:51.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.737+0000 7fab820e1b80 1 Processor -- start 2024-05-09T04:54:51.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab820e1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b27c4a0 0x556e8b40d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab820e1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b2f6d00 0x556e8b40d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab820e1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b3f9400 0x556e8b476500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab820e1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c1af6e0 con 0x556e8b3f9400 2024-05-09T04:54:51.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab820e1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c1b0b50 con 0x556e8b2f6d00 2024-05-09T04:54:51.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab820e1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fab6c1afdb0 con 0x556e8b27c4a0 2024-05-09T04:54:51.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab820e1b80 10 client.0 _ll_register_callbacks cb 0x556e8b18d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:51.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab71c68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b3f9400 0x556e8b476500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab71c68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b3f9400 0x556e8b476500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45290/0 (socket says 172.21.15.46:45290) 2024-05-09T04:54:51.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab71c68700 1 -- 172.21.15.46:0/1007447271 learned_addr learned my addr 172.21.15.46:0/1007447271 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:51.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab71467700 1 --2- 172.21.15.46:0/1007447271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b27c4a0 0x556e8b40d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab70c66700 1 --2- 172.21.15.46:0/1007447271 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b2f6d00 0x556e8b40d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab71c68700 1 -- 172.21.15.46:0/1007447271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b27c4a0 msgr2=0x556e8b40d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:51.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab71c68700 1 --2- 172.21.15.46:0/1007447271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b27c4a0 0x556e8b40d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab71c68700 1 -- 172.21.15.46:0/1007447271 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b2f6d00 msgr2=0x556e8b40d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:51.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.738+0000 7fab71c68700 1 --2- 172.21.15.46:0/1007447271 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b2f6d00 0x556e8b40d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.739+0000 7fab71c68700 1 -- 172.21.15.46:0/1007447271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e8b476a40 con 0x556e8b3f9400 2024-05-09T04:54:51.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.739+0000 7fab71467700 1 --2- 172.21.15.46:0/1007447271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b27c4a0 0x556e8b40d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:51.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.739+0000 7fab71c68700 1 --2- 172.21.15.46:0/1007447271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b3f9400 0x556e8b476500 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fab64016d60 tx=0x7fab64003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f213e5c4a857c2bf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:51.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.739+0000 7fab54ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:51.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.739+0000 7fab54ff9700 1 -- 172.21.15.46:0/1007447271 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab64004cd0 con 0x556e8b3f9400 2024-05-09T04:54:51.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.739+0000 7fab54ff9700 1 -- 172.21.15.46:0/1007447271 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fab64004020 con 0x556e8b3f9400 2024-05-09T04:54:51.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.739+0000 7fab820e1b80 1 -- 172.21.15.46:0/1007447271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab6401a070 con 0x556e8b3f9400 2024-05-09T04:54:51.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.739+0000 7fab54ff9700 1 -- 172.21.15.46:0/1007447271 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fab64004220 con 0x556e8b3f9400 2024-05-09T04:54:51.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.739+0000 7fab820e1b80 10 client.16353 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:51.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.739+0000 7fab820e1b80 1 -- 172.21.15.46:0/1007447271 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556e8b47dcb0 con 0x556e8b3f9400 2024-05-09T04:54:51.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.739+0000 7fab54ff9700 1 -- 172.21.15.46:0/1007447271 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab6401a070 con 0x556e8b3f9400 2024-05-09T04:54:51.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.740+0000 7fab54ff9700 1 -- 172.21.15.46:0/1007447271 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fab64025020 con 0x556e8b3f9400 2024-05-09T04:54:51.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.740+0000 7fab820e1b80 10 client.16353 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:51.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.740+0000 7fab820e1b80 10 client.16353 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:54:51.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.740+0000 7fab820e1b80 20 client.16353 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:51.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.740+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1007447271 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556e8b47e280 0x556e8b4806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:51.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.740+0000 7fab820e1b80 4 client.16353 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:54:51.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.740+0000 7fab820e1b80 1 -- 172.21.15.46:0/1007447271 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556e8b323b20 con 0x556e8b47e280 2024-05-09T04:54:51.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.740+0000 7fab71467700 1 --2- 172.21.15.46:0/1007447271 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556e8b47e280 0x556e8b4806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:51.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.740+0000 7fab54ff9700 10 client.16353 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:51.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.740+0000 7fab71467700 1 --2- 172.21.15.46:0/1007447271 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556e8b47e280 0x556e8b4806a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ef7c84ea72063313 server_cookie=5a990f4a0312aceb in_seq=0 out_seq=0 2024-05-09T04:54:51.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.741+0000 7fab54ff9700 1 -- 172.21.15.46:0/1007447271 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x556e8b323b20 con 0x556e8b47e280 2024-05-09T04:54:51.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.741+0000 7fab54ff9700 10 client.16353 handle_command_reply: tid=0 2024-05-09T04:54:51.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.741+0000 7fab820e1b80 1 client.16353 shutdown 2024-05-09T04:54:51.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.741+0000 7fab820e1b80 1 -- 172.21.15.46:0/1007447271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b3f9400 msgr2=0x556e8b476500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:51.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.741+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1007447271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b3f9400 0x556e8b476500 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fab64016d60 tx=0x7fab64003ee0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.741+0000 7fab820e1b80 1 -- 172.21.15.46:0/1007447271 shutdown_connections 2024-05-09T04:54:51.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.741+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1007447271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556e8b3f9400 0x556e8b476500 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.741+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1007447271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556e8b27c4a0 0x556e8b40d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.741+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1007447271 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556e8b47e280 0x556e8b4806a0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.741+0000 7fab820e1b80 1 --2- 172.21.15.46:0/1007447271 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556e8b2f6d00 0x556e8b40d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.741+0000 7fab820e1b80 1 -- 172.21.15.46:0/1007447271 >> 172.21.15.46:0/1007447271 conn(0x556e8b27ff70 msgr2=0x556e8b3fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:51.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.741+0000 7fab54ff9700 0 client.16353 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:54:51.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.741+0000 7fab820e1b80 1 -- 172.21.15.46:0/1007447271 shutdown_connections 2024-05-09T04:54:51.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.741+0000 7fab820e1b80 1 -- 172.21.15.46:0/1007447271 wait complete. 2024-05-09T04:54:51.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.741+0000 7fab820e1b80 20 client.16353 trim_cache size 0 max 16384 2024-05-09T04:54:51.755 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:54:51.756 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:54:51.756 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:54:51.756 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:54:51.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.742+0000 7fab736cb700 1 -- 172.21.15.46:0/170275759 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fab580b2a70 msgr2=0x7fab580b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:51.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.742+0000 7fab736cb700 1 --2- 172.21.15.46:0/170275759 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fab580b2a70 0x7fab580b4f80 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fab6c100200 tx=0x7fab5c00a300 comp rx=0 tx=0).stop 2024-05-09T04:54:51.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.742+0000 7fab736cb700 1 -- 172.21.15.46:0/170275759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab6c1029c0 msgr2=0x7fab6c19ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:51.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.742+0000 7fab736cb700 1 --2- 172.21.15.46:0/170275759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab6c1029c0 0x7fab6c19ae70 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fab68003a70 tx=0x7fab68003aa0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.742+0000 7fab736cb700 1 -- 172.21.15.46:0/170275759 shutdown_connections 2024-05-09T04:54:51.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.742+0000 7fab736cb700 1 --2- 172.21.15.46:0/170275759 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fab580b2a70 0x7fab580b4f80 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.742+0000 7fab736cb700 1 --2- 172.21.15.46:0/170275759 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fab6c1089f0 0x7fab6c19f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.742+0000 7fab736cb700 1 --2- 172.21.15.46:0/170275759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fab6c1029c0 0x7fab6c19ae70 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.742+0000 7fab736cb700 1 --2- 172.21.15.46:0/170275759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fab6c103310 0x7fab6c19b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:51.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.742+0000 7fab736cb700 1 -- 172.21.15.46:0/170275759 >> 172.21.15.46:0/170275759 conn(0x7fab6c0fe540 msgr2=0x7fab6c106a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:51.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.742+0000 7fab736cb700 1 -- 172.21.15.46:0/170275759 shutdown_connections 2024-05-09T04:54:51.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:51.742+0000 7fab736cb700 1 -- 172.21.15.46:0/170275759 wait complete. 2024-05-09T04:54:52.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:51 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:54:52.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:51 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:54:52.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:51 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:54:52.644 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:54:52.645 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:54:52.905 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:53.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:52 smithi046 ceph-mon[30136]: pgmap v116: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-05-09T04:54:53.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:52 smithi110 ceph-mon[34904]: pgmap v116: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-05-09T04:54:53.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:52 smithi072 ceph-mon[34088]: pgmap v116: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-05-09T04:54:54.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.739+0000 7f0cc063f700 1 -- 172.21.15.46:0/2996822861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb810a3e0 msgr2=0x7f0cb810a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:54.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.739+0000 7f0cc063f700 1 --2- 172.21.15.46:0/2996822861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb810a3e0 0x7f0cb810a7f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400b0a0 tx=0x7f0cb400b3b0 comp rx=0 tx=0).stop 2024-05-09T04:54:54.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.739+0000 7f0cc063f700 1 -- 172.21.15.46:0/2996822861 shutdown_connections 2024-05-09T04:54:54.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.739+0000 7f0cc063f700 1 --2- 172.21.15.46:0/2996822861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cb8103130 0x7f0cb8111120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:54.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.739+0000 7f0cc063f700 1 --2- 172.21.15.46:0/2996822861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb810a3e0 0x7f0cb810a7f0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:54.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.739+0000 7f0cc063f700 1 --2- 172.21.15.46:0/2996822861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb81027e0 0x7f0cb8102bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:54.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.739+0000 7f0cc063f700 1 -- 172.21.15.46:0/2996822861 >> 172.21.15.46:0/2996822861 conn(0x7f0cb80fe6f0 msgr2=0x7f0cb8100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:54.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.739+0000 7f0cc063f700 1 -- 172.21.15.46:0/2996822861 shutdown_connections 2024-05-09T04:54:54.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.739+0000 7f0cc063f700 1 -- 172.21.15.46:0/2996822861 wait complete. 2024-05-09T04:54:54.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cc063f700 1 Processor -- start 2024-05-09T04:54:54.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cc063f700 1 -- start start 2024-05-09T04:54:54.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cc063f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb81027e0 0x7f0cb819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:54.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cc063f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cb8103130 0x7f0cb819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:54.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cc063f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb810a3e0 0x7f0cb819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:54.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cc063f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb8114480 con 0x7f0cb81027e0 2024-05-09T04:54:54.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cc063f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb8114300 con 0x7f0cb810a3e0 2024-05-09T04:54:54.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cc063f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0cb8114600 con 0x7f0cb8103130 2024-05-09T04:54:54.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cbe3db700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb81027e0 0x7f0cb819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:54.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cbe3db700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb81027e0 0x7f0cb819ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51350/0 (socket says 172.21.15.46:51350) 2024-05-09T04:54:54.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cbe3db700 1 -- 172.21.15.46:0/2390721495 learned_addr learned my addr 172.21.15.46:0/2390721495 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:54.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cbdbda700 1 --2- 172.21.15.46:0/2390721495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cb8103130 0x7f0cb819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:54.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cbe3db700 1 -- 172.21.15.46:0/2390721495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cb8103130 msgr2=0x7f0cb819b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:54.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cbe3db700 1 --2- 172.21.15.46:0/2390721495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cb8103130 0x7f0cb819b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:54.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cbe3db700 1 -- 172.21.15.46:0/2390721495 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb810a3e0 msgr2=0x7f0cb819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:54.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cbe3db700 1 --2- 172.21.15.46:0/2390721495 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb810a3e0 0x7f0cb819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:54.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cbe3db700 1 -- 172.21.15.46:0/2390721495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cb819fce0 con 0x7f0cb81027e0 2024-05-09T04:54:54.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.740+0000 7f0cbdbda700 1 --2- 172.21.15.46:0/2390721495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cb8103130 0x7f0cb819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:54:54.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.741+0000 7f0cbe3db700 1 --2- 172.21.15.46:0/2390721495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb81027e0 0x7f0cb819ae10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400b070 tx=0x7f0cb4004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:54.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.741+0000 7f0caf7fe700 1 -- 172.21.15.46:0/2390721495 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4021070 con 0x7f0cb81027e0 2024-05-09T04:54:54.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.741+0000 7f0cc063f700 1 -- 172.21.15.46:0/2390721495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cb819ff70 con 0x7f0cb81027e0 2024-05-09T04:54:54.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.741+0000 7f0caf7fe700 1 -- 172.21.15.46:0/2390721495 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cb4003c00 con 0x7f0cb81027e0 2024-05-09T04:54:54.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.741+0000 7f0caf7fe700 1 -- 172.21.15.46:0/2390721495 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cb4004880 con 0x7f0cb81027e0 2024-05-09T04:54:54.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.741+0000 7f0cc063f700 1 -- 172.21.15.46:0/2390721495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cb81a78f0 con 0x7f0cb81027e0 2024-05-09T04:54:54.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.742+0000 7f0cc063f700 1 -- 172.21.15.46:0/2390721495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cb810c850 con 0x7f0cb81027e0 2024-05-09T04:54:54.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.744+0000 7f0caf7fe700 1 -- 172.21.15.46:0/2390721495 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0cb4004000 con 0x7f0cb81027e0 2024-05-09T04:54:54.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.744+0000 7f0caf7fe700 1 --2- 172.21.15.46:0/2390721495 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0ca40b29f0 0x7f0ca40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:54.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.744+0000 7f0cbdbda700 1 --2- 172.21.15.46:0/2390721495 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0ca40b29f0 0x7f0ca40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:54.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.744+0000 7f0caf7fe700 1 -- 172.21.15.46:0/2390721495 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f0cb400f020 con 0x7f0cb81027e0 2024-05-09T04:54:54.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.745+0000 7f0cbdbda700 1 --2- 172.21.15.46:0/2390721495 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0ca40b29f0 0x7f0ca40b4f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8100190 tx=0x7f0ca8009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:54.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:54.748+0000 7f0caf7fe700 1 -- 172.21.15.46:0/2390721495 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0cb402a020 con 0x7f0cb81027e0 2024-05-09T04:54:55.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:55.011+0000 7f0cc063f700 1 -- 172.21.15.46:0/2390721495 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0cb819c170 con 0x7f0cb81027e0 2024-05-09T04:54:55.012 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:54 smithi046 ceph-mon[30136]: pgmap v117: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T04:54:55.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:55.011+0000 7f0caf7fe700 1 -- 172.21.15.46:0/2390721495 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7f0cb40aba30 con 0x7f0cb81027e0 2024-05-09T04:54:55.013 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:54:55.013 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:54:55.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:55.014+0000 7f0cc063f700 1 -- 172.21.15.46:0/2390721495 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0ca40b29f0 msgr2=0x7f0ca40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:55.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:55.014+0000 7f0cc063f700 1 --2- 172.21.15.46:0/2390721495 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0ca40b29f0 0x7f0ca40b4f00 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8100190 tx=0x7f0ca8009260 comp rx=0 tx=0).stop 2024-05-09T04:54:55.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:55.014+0000 7f0cc063f700 1 -- 172.21.15.46:0/2390721495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb81027e0 msgr2=0x7f0cb819ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:55.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:55.014+0000 7f0cc063f700 1 --2- 172.21.15.46:0/2390721495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb81027e0 0x7f0cb819ae10 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0cb400b070 tx=0x7f0cb4004db0 comp rx=0 tx=0).stop 2024-05-09T04:54:55.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:55.014+0000 7f0cc063f700 1 -- 172.21.15.46:0/2390721495 shutdown_connections 2024-05-09T04:54:55.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:55.014+0000 7f0cc063f700 1 --2- 172.21.15.46:0/2390721495 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0ca40b29f0 0x7f0ca40b4f00 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:55.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:55.014+0000 7f0cc063f700 1 --2- 172.21.15.46:0/2390721495 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0cb810a3e0 0x7f0cb819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:55.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:55.014+0000 7f0cc063f700 1 --2- 172.21.15.46:0/2390721495 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0cb81027e0 0x7f0cb819ae10 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:55.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:55.014+0000 7f0cc063f700 1 --2- 172.21.15.46:0/2390721495 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0cb8103130 0x7f0cb819b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:55.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:55.014+0000 7f0cc063f700 1 -- 172.21.15.46:0/2390721495 >> 172.21.15.46:0/2390721495 conn(0x7f0cb80fe6f0 msgr2=0x7f0cb8100b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:55.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:55.014+0000 7f0cc063f700 1 -- 172.21.15.46:0/2390721495 shutdown_connections 2024-05-09T04:54:55.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:55.014+0000 7f0cc063f700 1 -- 172.21.15.46:0/2390721495 wait complete. 2024-05-09T04:54:55.016 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:54:55.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:54 smithi110 ceph-mon[34904]: pgmap v117: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T04:54:55.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:54 smithi072 ceph-mon[34088]: pgmap v117: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T04:54:55.789 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:54:56.048 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:56.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:55 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2390721495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:54:56.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:55 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2390721495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:54:56.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:55 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2390721495' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:54:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:56 smithi110 ceph-mon[34904]: pgmap v118: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:54:57.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:56 smithi046 ceph-mon[30136]: pgmap v118: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:54:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:56 smithi072 ceph-mon[34088]: pgmap v118: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:54:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.889+0000 7f25f5ced700 1 -- 172.21.15.46:0/1233266675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f0107e10 msgr2=0x7f25f0108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.889+0000 7f25f5ced700 1 --2- 172.21.15.46:0/1233266675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f0107e10 0x7f25f0108220 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f25e40099c0 tx=0x7f25e4009cd0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.889+0000 7f25f5ced700 1 -- 172.21.15.46:0/1233266675 shutdown_connections 2024-05-09T04:54:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.889+0000 7f25f5ced700 1 --2- 172.21.15.46:0/1233266675 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0108760 0x7f25f01100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.889+0000 7f25f5ced700 1 --2- 172.21.15.46:0/1233266675 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25f01034e0 0x7f25f01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.889+0000 7f25f5ced700 1 --2- 172.21.15.46:0/1233266675 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f0107e10 0x7f25f0108220 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.889+0000 7f25f5ced700 1 -- 172.21.15.46:0/1233266675 >> 172.21.15.46:0/1233266675 conn(0x7f25f00fef30 msgr2=0x7f25f0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.889+0000 7f25f5ced700 1 -- 172.21.15.46:0/1233266675 shutdown_connections 2024-05-09T04:54:57.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.889+0000 7f25f5ced700 1 -- 172.21.15.46:0/1233266675 wait complete. 2024-05-09T04:54:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.890+0000 7f25f5ced700 1 Processor -- start 2024-05-09T04:54:57.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.890+0000 7f25f5ced700 1 -- start start 2024-05-09T04:54:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25f5ced700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25f01034e0 0x7f25f019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25f5ced700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f0107e10 0x7f25f019c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25f5ced700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0108760 0x7f25f019c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25f5ced700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25f01133c0 con 0x7f25f0107e10 2024-05-09T04:54:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25f5ced700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f25f0113240 con 0x7f25f01034e0 2024-05-09T04:54:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25f5ced700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25f0113540 con 0x7f25f0108760 2024-05-09T04:54:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25eeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f0107e10 0x7f25f019c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25eeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f0107e10 0x7f25f019c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51368/0 (socket says 172.21.15.46:51368) 2024-05-09T04:54:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25eeffd700 1 -- 172.21.15.46:0/1202949493 learned_addr learned my addr 172.21.15.46:0/1202949493 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25ef7fe700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25f01034e0 0x7f25f019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25effff700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0108760 0x7f25f019c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25eeffd700 1 -- 172.21.15.46:0/1202949493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0108760 msgr2=0x7f25f019c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:57.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25eeffd700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0108760 0x7f25f019c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25eeffd700 1 -- 172.21.15.46:0/1202949493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25f01034e0 msgr2=0x7f25f019bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25eeffd700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25f01034e0 0x7f25f019bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25eeffd700 1 -- 172.21.15.46:0/1202949493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25f01a7050 con 0x7f25f0107e10 2024-05-09T04:54:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25effff700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0108760 0x7f25f019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.891+0000 7f25ef7fe700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25f01034e0 0x7f25f019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.892+0000 7f25eeffd700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f0107e10 0x7f25f019c240 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f25e4006010 tx=0x7f25e40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.892+0000 7f25ecff9700 1 -- 172.21.15.46:0/1202949493 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e4026070 con 0x7f25f0107e10 2024-05-09T04:54:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.892+0000 7f25ecff9700 1 -- 172.21.15.46:0/1202949493 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25e4003f20 con 0x7f25f0107e10 2024-05-09T04:54:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.892+0000 7f25ecff9700 1 -- 172.21.15.46:0/1202949493 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e400fcc0 con 0x7f25f0107e10 2024-05-09T04:54:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.892+0000 7f25f5ced700 1 -- 172.21.15.46:0/1202949493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25f01a72e0 con 0x7f25f0107e10 2024-05-09T04:54:57.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.892+0000 7f25f5ced700 1 -- 172.21.15.46:0/1202949493 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25f01a7730 con 0x7f25f0107e10 2024-05-09T04:54:57.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.895+0000 7f25ecff9700 1 -- 172.21.15.46:0/1202949493 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f25e4029070 con 0x7f25f0107e10 2024-05-09T04:54:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.896+0000 7f25ecff9700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f25d80b29f0 0x7f25d80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.896+0000 7f25ef7fe700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f25d80b29f0 0x7f25d80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.896+0000 7f25ecff9700 1 -- 172.21.15.46:0/1202949493 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f25e40e3130 con 0x7f25f0107e10 2024-05-09T04:54:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.896+0000 7f2604703b80 1 Processor -- start 2024-05-09T04:54:57.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.896+0000 7f2604703b80 1 -- start start 2024-05-09T04:54:57.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.896+0000 7f25ef7fe700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f25d80b29f0 0x7f25d80b4f00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f25e0007fd0 tx=0x7f25e00073d0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.896+0000 7f2604703b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556428f0ed00 0x556429017240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.896+0000 7f2604703b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 0x556429017780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.896+0000 7f2604703b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556429011400 0x556429011810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.896+0000 7f2604703b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556428f340a0 con 0x556429011400 2024-05-09T04:54:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.896+0000 7f2604703b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556428ed99a0 con 0x556428e944a0 2024-05-09T04:54:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.896+0000 7f2604703b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556428f30060 con 0x556428f0ed00 2024-05-09T04:54:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.897+0000 7f25effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556429011400 0x556429011810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.897+0000 7f25effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556429011400 0x556429011810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51380/0 (socket says 172.21.15.46:51380) 2024-05-09T04:54:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.897+0000 7f25effff700 1 -- 172.21.15.46:0/3202163107 learned_addr learned my addr 172.21.15.46:0/3202163107 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.897+0000 7f25eeffd700 1 --2- 172.21.15.46:0/3202163107 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 0x556429017780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.897+0000 7f25ef7fe700 1 --2- 172.21.15.46:0/3202163107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556428f0ed00 0x556429017240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.897+0000 7f25effff700 1 -- 172.21.15.46:0/3202163107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556428f0ed00 msgr2=0x556429017240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:57.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.897+0000 7f25effff700 1 --2- 172.21.15.46:0/3202163107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556428f0ed00 0x556429017240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.897+0000 7f25effff700 1 -- 172.21.15.46:0/3202163107 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 msgr2=0x556429017780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:57.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.897+0000 7f25effff700 1 --2- 172.21.15.46:0/3202163107 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 0x556429017780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.897+0000 7f25effff700 1 -- 172.21.15.46:0/3202163107 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556428eca500 con 0x556429011400 2024-05-09T04:54:57.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.897+0000 7f25effff700 1 --2- 172.21.15.46:0/3202163107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556429011400 0x556429011810 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f25e80085a0 tx=0x7f25e8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b66dc370e6c2aa8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:57.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.897+0000 7f25d5ffb700 1 -- 172.21.15.46:0/3202163107 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8019070 con 0x556429011400 2024-05-09T04:54:57.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.897+0000 7f25d5ffb700 1 -- 172.21.15.46:0/3202163107 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25e8012bb0 con 0x556429011400 2024-05-09T04:54:57.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.897+0000 7f25d5ffb700 1 -- 172.21.15.46:0/3202163107 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8014ba0 con 0x556429011400 2024-05-09T04:54:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.898+0000 7f2604703b80 1 -- 172.21.15.46:0/3202163107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556429011400 msgr2=0x556429011810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.898+0000 7f2604703b80 1 --2- 172.21.15.46:0/3202163107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556429011400 0x556429011810 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f25e80085a0 tx=0x7f25e8007f90 comp rx=0 tx=0).stop 2024-05-09T04:54:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.898+0000 7f2604703b80 1 -- 172.21.15.46:0/3202163107 shutdown_connections 2024-05-09T04:54:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.898+0000 7f2604703b80 1 --2- 172.21.15.46:0/3202163107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556429011400 0x556429011810 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.898+0000 7f2604703b80 1 --2- 172.21.15.46:0/3202163107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556428f0ed00 0x556429017240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.898+0000 7f2604703b80 1 --2- 172.21.15.46:0/3202163107 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 0x556429017780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.898+0000 7f2604703b80 1 -- 172.21.15.46:0/3202163107 >> 172.21.15.46:0/3202163107 conn(0x556428e97f70 msgr2=0x55642900c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.898+0000 7f2604703b80 1 -- 172.21.15.46:0/3202163107 shutdown_connections 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.898+0000 7f2604703b80 1 -- 172.21.15.46:0/3202163107 wait complete. 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.899+0000 7f2604703b80 1 -- start start 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.899+0000 7f2604703b80 1 Processor -- start 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.899+0000 7f2604703b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 0x55642901b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.899+0000 7f2604703b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556428f0ed00 0x55642901b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.899+0000 7f2604703b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556429011400 0x55642901bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.899+0000 7f2604703b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556429012160 con 0x556429011400 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.899+0000 7f2604703b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556428eceb20 con 0x556428e944a0 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.899+0000 7f2604703b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55642901e020 con 0x556428f0ed00 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.899+0000 7f2604703b80 10 client.0 _ll_register_callbacks cb 0x556428da56c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.900+0000 7f25effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556429011400 0x55642901bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.900+0000 7f25effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556429011400 0x55642901bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51390/0 (socket says 172.21.15.46:51390) 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.900+0000 7f25effff700 1 -- 172.21.15.46:0/2450152329 learned_addr learned my addr 172.21.15.46:0/2450152329 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:57.904 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.900+0000 7f25ef7fe700 1 --2- 172.21.15.46:0/2450152329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 0x55642901b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.900+0000 7f25eeffd700 1 --2- 172.21.15.46:0/2450152329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556428f0ed00 0x55642901b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.900+0000 7f25effff700 1 -- 172.21.15.46:0/2450152329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556428f0ed00 msgr2=0x55642901b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.900+0000 7f25effff700 1 --2- 172.21.15.46:0/2450152329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556428f0ed00 0x55642901b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.900+0000 7f25effff700 1 -- 172.21.15.46:0/2450152329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 msgr2=0x55642901b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.900+0000 7f25effff700 1 --2- 172.21.15.46:0/2450152329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 0x55642901b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.900+0000 7f25effff700 1 -- 172.21.15.46:0/2450152329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556428daf490 con 0x556429011400 2024-05-09T04:54:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.900+0000 7f25effff700 1 --2- 172.21.15.46:0/2450152329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556429011400 0x55642901bc20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f25e8007d40 tx=0x7f25e8014600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8130462c0bdf1cc7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.900+0000 7f25ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:54:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.900+0000 7f25ceffd700 1 -- 172.21.15.46:0/2450152329 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e8022060 con 0x556429011400 2024-05-09T04:54:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.900+0000 7f25ceffd700 1 -- 172.21.15.46:0/2450152329 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f25e8004dd0 con 0x556429011400 2024-05-09T04:54:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.900+0000 7f2604703b80 1 -- 172.21.15.46:0/2450152329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25e8012bb0 con 0x556429011400 2024-05-09T04:54:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.901+0000 7f25ceffd700 1 -- 172.21.15.46:0/2450152329 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f25e8019040 con 0x556429011400 2024-05-09T04:54:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.901+0000 7f2604703b80 10 client.16371 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:57.905 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.901+0000 7f2604703b80 1 -- 172.21.15.46:0/2450152329 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556429028b70 con 0x556429011400 2024-05-09T04:54:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.901+0000 7f25ceffd700 1 -- 172.21.15.46:0/2450152329 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25e8012bb0 con 0x556429011400 2024-05-09T04:54:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.901+0000 7f25ceffd700 1 -- 172.21.15.46:0/2450152329 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f25e8013070 con 0x556429011400 2024-05-09T04:54:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.901+0000 7f2604703b80 10 client.16371 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.901+0000 7f2604703b80 10 client.16371 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:54:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.901+0000 7f2604703b80 20 client.16371 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.901+0000 7f2604703b80 1 --2- 172.21.15.46:0/2450152329 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55642902b0f0 0x55642902d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.901+0000 7f2604703b80 4 client.16371 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:54:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.901+0000 7f2604703b80 1 -- 172.21.15.46:0/2450152329 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556428f41a80 con 0x55642902b0f0 2024-05-09T04:54:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.901+0000 7f25ef7fe700 1 --2- 172.21.15.46:0/2450152329 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55642902b0f0 0x55642902d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.902+0000 7f25ef7fe700 1 --2- 172.21.15.46:0/2450152329 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55642902b0f0 0x55642902d510 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e5e5e97f1b076ec server_cookie=9435dca2e77b93b0 in_seq=0 out_seq=0 2024-05-09T04:54:57.906 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.902+0000 7f25ceffd700 10 client.16371 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:54:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.903+0000 7f25ceffd700 1 -- 172.21.15.46:0/2450152329 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556428f41a80 con 0x55642902b0f0 2024-05-09T04:54:57.907 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.903+0000 7f25ceffd700 10 client.16371 handle_command_reply: tid=0 2024-05-09T04:54:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.903+0000 7f2604703b80 1 client.16371 shutdown 2024-05-09T04:54:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.903+0000 7f2604703b80 1 -- 172.21.15.46:0/2450152329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556429011400 msgr2=0x55642901bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.903+0000 7f2604703b80 1 --2- 172.21.15.46:0/2450152329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556429011400 0x55642901bc20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f25e8007d40 tx=0x7f25e8014600 comp rx=0 tx=0).stop 2024-05-09T04:54:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.903+0000 7f2604703b80 1 -- 172.21.15.46:0/2450152329 shutdown_connections 2024-05-09T04:54:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.903+0000 7f2604703b80 1 --2- 172.21.15.46:0/2450152329 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556429011400 0x55642901bc20 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.903+0000 7f2604703b80 1 --2- 172.21.15.46:0/2450152329 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 0x55642901b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.903+0000 7f2604703b80 1 --2- 172.21.15.46:0/2450152329 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55642902b0f0 0x55642902d510 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.903+0000 7f25ceffd700 0 client.16371 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:54:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.903+0000 7f2604703b80 1 --2- 172.21.15.46:0/2450152329 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556428f0ed00 0x55642901b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.903+0000 7f2604703b80 1 -- 172.21.15.46:0/2450152329 >> 172.21.15.46:0/2450152329 conn(0x556428e97f70 msgr2=0x556429005ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.903+0000 7f2604703b80 1 -- 172.21.15.46:0/2450152329 shutdown_connections 2024-05-09T04:54:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.903+0000 7f2604703b80 1 -- 172.21.15.46:0/2450152329 wait complete. 2024-05-09T04:54:57.908 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.903+0000 7f2604703b80 20 client.16371 trim_cache size 0 max 16384 2024-05-09T04:54:57.921 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.918+0000 7f2604703b80 1 Processor -- start 2024-05-09T04:54:57.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.918+0000 7f2604703b80 1 -- start start 2024-05-09T04:54:57.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f2604703b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556428e944a0 0x556429006a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f2604703b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428f0ed00 0x55642901a970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f2604703b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556429011400 0x556429010b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f2604703b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25f01b17a0 con 0x556428e944a0 2024-05-09T04:54:57.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f2604703b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f25f01af030 con 0x556428f0ed00 2024-05-09T04:54:57.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f2604703b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25f01ace80 con 0x556429011400 2024-05-09T04:54:57.922 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f25ef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556428e944a0 0x556429006a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f25ef7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556428e944a0 0x556429006a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51400/0 (socket says 172.21.15.46:51400) 2024-05-09T04:54:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f25ef7fe700 1 -- 172.21.15.46:0/1389156108 learned_addr learned my addr 172.21.15.46:0/1389156108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f25eeffd700 1 --2- 172.21.15.46:0/1389156108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428f0ed00 0x55642901a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f25effff700 1 --2- 172.21.15.46:0/1389156108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556429011400 0x556429010b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f25ef7fe700 1 -- 172.21.15.46:0/1389156108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556429011400 msgr2=0x556429010b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f25ef7fe700 1 --2- 172.21.15.46:0/1389156108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556429011400 0x556429010b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f25ef7fe700 1 -- 172.21.15.46:0/1389156108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428f0ed00 msgr2=0x55642901a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f25ef7fe700 1 --2- 172.21.15.46:0/1389156108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428f0ed00 0x55642901a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f25ef7fe700 1 -- 172.21.15.46:0/1389156108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556429019710 con 0x556428e944a0 2024-05-09T04:54:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f25eeffd700 1 --2- 172.21.15.46:0/1389156108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428f0ed00 0x55642901a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:54:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.919+0000 7f25ef7fe700 1 --2- 172.21.15.46:0/1389156108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556428e944a0 0x556429006a20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f25e0003680 tx=0x7f25e0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f70de35a055e9591 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.920+0000 7f25d57fa700 1 -- 172.21.15.46:0/1389156108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e002e070 con 0x556428e944a0 2024-05-09T04:54:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.920+0000 7f25d57fa700 1 -- 172.21.15.46:0/1389156108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25e0015070 con 0x556428e944a0 2024-05-09T04:54:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.920+0000 7f25d57fa700 1 -- 172.21.15.46:0/1389156108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0024070 con 0x556428e944a0 2024-05-09T04:54:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.920+0000 7f2604703b80 1 -- 172.21.15.46:0/1389156108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556428e944a0 msgr2=0x556429006a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.920+0000 7f2604703b80 1 --2- 172.21.15.46:0/1389156108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556428e944a0 0x556429006a20 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f25e0003680 tx=0x7f25e0013000 comp rx=0 tx=0).stop 2024-05-09T04:54:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.920+0000 7f2604703b80 1 -- 172.21.15.46:0/1389156108 shutdown_connections 2024-05-09T04:54:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.920+0000 7f2604703b80 1 --2- 172.21.15.46:0/1389156108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556429011400 0x556429010b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.920+0000 7f2604703b80 1 --2- 172.21.15.46:0/1389156108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556428e944a0 0x556429006a20 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.920+0000 7f2604703b80 1 --2- 172.21.15.46:0/1389156108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428f0ed00 0x55642901a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.920+0000 7f2604703b80 1 -- 172.21.15.46:0/1389156108 >> 172.21.15.46:0/1389156108 conn(0x556428e97f70 msgr2=0x5564290103f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.920+0000 7f2604703b80 1 -- 172.21.15.46:0/1389156108 shutdown_connections 2024-05-09T04:54:57.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.920+0000 7f2604703b80 1 -- 172.21.15.46:0/1389156108 wait complete. 2024-05-09T04:54:57.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.921+0000 7f2604703b80 1 -- start start 2024-05-09T04:54:57.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.921+0000 7f2604703b80 1 Processor -- start 2024-05-09T04:54:57.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.921+0000 7f2604703b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 0x556429030850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.921+0000 7f2604703b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556428f0ed00 0x55642902dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.921+0000 7f2604703b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556429011400 0x55642902e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.921+0000 7f2604703b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25f01ad4f0 con 0x556428f0ed00 2024-05-09T04:54:57.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.921+0000 7f2604703b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f25f01ae960 con 0x556428e944a0 2024-05-09T04:54:57.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.921+0000 7f2604703b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25f01adbc0 con 0x556429011400 2024-05-09T04:54:57.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f2604703b80 10 client.0 _ll_register_callbacks cb 0x556429011dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:54:57.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25eeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556428f0ed00 0x55642902dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25ef7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 0x556429030850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25ef7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 0x556429030850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:40366/0 (socket says 172.21.15.46:40366) 2024-05-09T04:54:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25ef7fe700 1 -- 172.21.15.46:0/2262004079 learned_addr learned my addr 172.21.15.46:0/2262004079 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:54:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25effff700 1 --2- 172.21.15.46:0/2262004079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556429011400 0x55642902e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25ef7fe700 1 -- 172.21.15.46:0/2262004079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556429011400 msgr2=0x55642902e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25ef7fe700 1 --2- 172.21.15.46:0/2262004079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556429011400 0x55642902e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25ef7fe700 1 -- 172.21.15.46:0/2262004079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556428f0ed00 msgr2=0x55642902dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:54:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25ef7fe700 1 --2- 172.21.15.46:0/2262004079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556428f0ed00 0x55642902dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25ef7fe700 1 -- 172.21.15.46:0/2262004079 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55642902e650 con 0x556428e944a0 2024-05-09T04:54:57.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25eeffd700 1 --2- 172.21.15.46:0/2262004079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556428f0ed00 0x55642902dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:54:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25ef7fe700 1 --2- 172.21.15.46:0/2262004079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 0x556429030850 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f25e00046e0 tx=0x7f25e0004450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ce3a44f76e7be3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:54:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25ceffd700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:54:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25ceffd700 1 -- 172.21.15.46:0/2262004079 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25e0039060 con 0x556428e944a0 2024-05-09T04:54:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f2604703b80 1 -- 172.21.15.46:0/2262004079 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25e0015070 con 0x556428e944a0 2024-05-09T04:54:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25ceffd700 1 -- 172.21.15.46:0/2262004079 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f25e00292d0 con 0x556428e944a0 2024-05-09T04:54:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f25ceffd700 1 -- 172.21.15.46:0/2262004079 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f25e0024070 con 0x556428e944a0 2024-05-09T04:54:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f2604703b80 10 client.25976 fetch_fsmap learned FSMap version 25 2024-05-09T04:54:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.922+0000 7f2604703b80 1 -- 172.21.15.46:0/2262004079 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5564290952f0 con 0x556428e944a0 2024-05-09T04:54:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.923+0000 7f25ceffd700 1 -- 172.21.15.46:0/2262004079 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25e0015070 con 0x556428e944a0 2024-05-09T04:54:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.923+0000 7f25ceffd700 1 -- 172.21.15.46:0/2262004079 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f25e0029ce0 con 0x556428e944a0 2024-05-09T04:54:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.923+0000 7f2604703b80 10 client.25976 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:54:57.930 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.923+0000 7f2604703b80 10 client.25976 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:54:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.923+0000 7f2604703b80 20 client.25976 populate_metadata read hostname 'smithi046' 2024-05-09T04:54:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.923+0000 7f2604703b80 1 --2- 172.21.15.46:0/2262004079 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55642902b0f0 0x556429097940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:54:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.923+0000 7f2604703b80 4 client.25976 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:54:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.923+0000 7f2604703b80 1 -- 172.21.15.46:0/2262004079 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556428f3bb20 con 0x55642902b0f0 2024-05-09T04:54:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.923+0000 7f25eeffd700 1 --2- 172.21.15.46:0/2262004079 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55642902b0f0 0x556429097940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:54:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f25eeffd700 1 --2- 172.21.15.46:0/2262004079 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55642902b0f0 0x556429097940 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2c3bbfe75c7d78e server_cookie=ecea79bdf8173013 in_seq=0 out_seq=0 2024-05-09T04:54:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f25ceffd700 10 client.25976 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:54:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f25ceffd700 1 -- 172.21.15.46:0/2262004079 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556428f3bb20 con 0x55642902b0f0 2024-05-09T04:54:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f25ceffd700 10 client.25976 handle_command_reply: tid=0 2024-05-09T04:54:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f2604703b80 1 client.25976 shutdown 2024-05-09T04:54:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f2604703b80 1 -- 172.21.15.46:0/2262004079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 msgr2=0x556429030850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f2604703b80 1 --2- 172.21.15.46:0/2262004079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 0x556429030850 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f25e00046e0 tx=0x7f25e0004450 comp rx=0 tx=0).stop 2024-05-09T04:54:57.931 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f2604703b80 1 -- 172.21.15.46:0/2262004079 shutdown_connections 2024-05-09T04:54:57.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f2604703b80 1 --2- 172.21.15.46:0/2262004079 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55642902b0f0 0x556429097940 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f2604703b80 1 --2- 172.21.15.46:0/2262004079 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556429011400 0x55642902e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f2604703b80 1 --2- 172.21.15.46:0/2262004079 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556428e944a0 0x556429030850 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f25ceffd700 0 client.25976 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:54:57.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f2604703b80 1 --2- 172.21.15.46:0/2262004079 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556428f0ed00 0x55642902dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f2604703b80 1 -- 172.21.15.46:0/2262004079 >> 172.21.15.46:0/2262004079 conn(0x556428e97f70 msgr2=0x556429006780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:57.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f2604703b80 1 -- 172.21.15.46:0/2262004079 shutdown_connections 2024-05-09T04:54:57.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f2604703b80 1 -- 172.21.15.46:0/2262004079 wait complete. 2024-05-09T04:54:57.932 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.924+0000 7f2604703b80 20 client.25976 trim_cache size 0 max 16384 2024-05-09T04:54:57.932 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:54:57.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.925+0000 7f25f5ced700 1 -- 172.21.15.46:0/1202949493 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f25d80b29f0 msgr2=0x7f25d80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:57.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.925+0000 7f25f5ced700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f25d80b29f0 0x7f25d80b4f00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f25e0007fd0 tx=0x7f25e00073d0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.925+0000 7f25f5ced700 1 -- 172.21.15.46:0/1202949493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f0107e10 msgr2=0x7f25f019c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:54:57.933 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.925+0000 7f25f5ced700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f0107e10 0x7f25f019c240 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f25e4006010 tx=0x7f25e40047b0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.925+0000 7f25f5ced700 1 -- 172.21.15.46:0/1202949493 shutdown_connections 2024-05-09T04:54:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.925+0000 7f25f5ced700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f25d80b29f0 0x7f25d80b4f00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.925+0000 7f25f5ced700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25f0108760 0x7f25f019c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.925+0000 7f25f5ced700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25f01034e0 0x7f25f019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.925+0000 7f25f5ced700 1 --2- 172.21.15.46:0/1202949493 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25f0107e10 0x7f25f019c240 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:54:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.925+0000 7f25f5ced700 1 -- 172.21.15.46:0/1202949493 >> 172.21.15.46:0/1202949493 conn(0x7f25f00fef30 msgr2=0x7f25f01096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:54:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.925+0000 7f25f5ced700 1 -- 172.21.15.46:0/1202949493 shutdown_connections 2024-05-09T04:54:57.934 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:54:57.925+0000 7f25f5ced700 1 -- 172.21.15.46:0/1202949493 wait complete. 2024-05-09T04:54:58.737 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:54:58.997 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:54:59.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:54:58 smithi110 ceph-mon[34904]: pgmap v119: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T04:54:59.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:54:58 smithi046 ceph-mon[30136]: pgmap v119: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T04:54:59.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:54:58 smithi072 ceph-mon[34088]: pgmap v119: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T04:55:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.687+0000 7f01e5c0e700 1 -- 172.21.15.46:0/3019587505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e0109870 msgr2=0x7f01e01111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.687+0000 7f01e5c0e700 1 --2- 172.21.15.46:0/3019587505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e0109870 0x7f01e01111d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f01d800b0d0 tx=0x7f01d800b3e0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.687+0000 7f01e5c0e700 1 -- 172.21.15.46:0/3019587505 shutdown_connections 2024-05-09T04:55:00.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.687+0000 7f01e5c0e700 1 --2- 172.21.15.46:0/3019587505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e0109870 0x7f01e01111d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.687+0000 7f01e5c0e700 1 --2- 172.21.15.46:0/3019587505 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01e01045f0 0x7f01e0104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.687+0000 7f01e5c0e700 1 --2- 172.21.15.46:0/3019587505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01e0108f20 0x7f01e0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.687+0000 7f01e5c0e700 1 -- 172.21.15.46:0/3019587505 >> 172.21.15.46:0/3019587505 conn(0x7f01e0100040 msgr2=0x7f01e0102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:00.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.687+0000 7f01e5c0e700 1 -- 172.21.15.46:0/3019587505 shutdown_connections 2024-05-09T04:55:00.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.687+0000 7f01e5c0e700 1 -- 172.21.15.46:0/3019587505 wait complete. 2024-05-09T04:55:00.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.688+0000 7f01e5c0e700 1 Processor -- start 2024-05-09T04:55:00.691 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01e5c0e700 1 -- start start 2024-05-09T04:55:00.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01e5c0e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01e01045f0 0x7f01e019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01e5c0e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01e0108f20 0x7f01e019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01e5c0e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e0109870 0x7f01e019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01e5c0e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01e0114510 con 0x7f01e0109870 2024-05-09T04:55:00.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01e5c0e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f01e0114390 con 0x7f01e01045f0 2024-05-09T04:55:00.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01e5c0e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f01e0114690 con 0x7f01e0108f20 2024-05-09T04:55:00.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01dffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e0109870 0x7f01e019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01dffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e0109870 0x7f01e019f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51430/0 (socket says 172.21.15.46:51430) 2024-05-09T04:55:00.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01dffff700 1 -- 172.21.15.46:0/3316482786 learned_addr learned my addr 172.21.15.46:0/3316482786 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:00.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01deffd700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01e0108f20 0x7f01e019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01df7fe700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01e01045f0 0x7f01e019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01dffff700 1 -- 172.21.15.46:0/3316482786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01e0108f20 msgr2=0x7f01e019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:00.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01dffff700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01e0108f20 0x7f01e019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01dffff700 1 -- 172.21.15.46:0/3316482786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01e01045f0 msgr2=0x7f01e019ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01dffff700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01e01045f0 0x7f01e019ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01dffff700 1 -- 172.21.15.46:0/3316482786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01e019fd30 con 0x7f01e0109870 2024-05-09T04:55:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01deffd700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01e0108f20 0x7f01e019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.689+0000 7f01df7fe700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01e01045f0 0x7f01e019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.690+0000 7f01dffff700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e0109870 0x7f01e019f690 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f01d80095c0 tx=0x7f01d8007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.690+0000 7f01dcff9700 1 -- 172.21.15.46:0/3316482786 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d801f070 con 0x7f01e0109870 2024-05-09T04:55:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.690+0000 7f01e5c0e700 1 -- 172.21.15.46:0/3316482786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01e019ffc0 con 0x7f01e0109870 2024-05-09T04:55:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.690+0000 7f01dcff9700 1 -- 172.21.15.46:0/3316482786 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01d8003c00 con 0x7f01e0109870 2024-05-09T04:55:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.690+0000 7f01dcff9700 1 -- 172.21.15.46:0/3316482786 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d801b6a0 con 0x7f01e0109870 2024-05-09T04:55:00.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.690+0000 7f01e5c0e700 1 -- 172.21.15.46:0/3316482786 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01e01a7940 con 0x7f01e0109870 2024-05-09T04:55:00.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.694+0000 7f01dcff9700 1 -- 172.21.15.46:0/3316482786 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f01d8022070 con 0x7f01e0109870 2024-05-09T04:55:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01f4624b80 1 Processor -- start 2024-05-09T04:55:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01f4624b80 1 -- start start 2024-05-09T04:55:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01dcff9700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f01cc0b2a70 0x7f01cc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01deffd700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f01cc0b2a70 0x7f01cc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01dcff9700 1 -- 172.21.15.46:0/3316482786 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f01d800f020 con 0x7f01e0109870 2024-05-09T04:55:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01f4624b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b276ed00 0x5598b2875ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01f4624b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b26f44a0 0x5598b2876030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01f4624b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b287a1e0 0x5598b287a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01f4624b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5598b27940a0 con 0x5598b26f44a0 2024-05-09T04:55:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01f4624b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5598b27399a0 con 0x5598b276ed00 2024-05-09T04:55:00.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01f4624b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5598b2790060 con 0x5598b287a1e0 2024-05-09T04:55:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01df7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b276ed00 0x5598b2875ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01deffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b26f44a0 0x5598b2876030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01df7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b276ed00 0x5598b2875ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:40386/0 (socket says 172.21.15.46:40386) 2024-05-09T04:55:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01df7fe700 1 -- 172.21.15.46:0/1041652216 learned_addr learned my addr 172.21.15.46:0/1041652216 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.695+0000 7f01dffff700 1 --2- 172.21.15.46:0/1041652216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b287a1e0 0x5598b287a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.696+0000 7f01deffd700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f01cc0b2a70 0x7f01cc0b4f80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f01e0103a70 tx=0x7f01c8009db0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.696+0000 7f01df7fe700 1 -- 172.21.15.46:0/1041652216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b287a1e0 msgr2=0x5598b287a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.696+0000 7f01df7fe700 1 --2- 172.21.15.46:0/1041652216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b287a1e0 0x5598b287a5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.696+0000 7f01df7fe700 1 -- 172.21.15.46:0/1041652216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b26f44a0 msgr2=0x5598b2876030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.696+0000 7f01df7fe700 1 --2- 172.21.15.46:0/1041652216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b26f44a0 0x5598b2876030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.696+0000 7f01df7fe700 1 -- 172.21.15.46:0/1041652216 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b272a500 con 0x5598b276ed00 2024-05-09T04:55:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.696+0000 7f01deffd700 1 --2- 172.21.15.46:0/1041652216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b26f44a0 0x5598b2876030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:00.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.696+0000 7f01df7fe700 1 --2- 172.21.15.46:0/1041652216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b276ed00 0x5598b2875ad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01d000bd90 tx=0x7f01d000f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=815820c77c41e063 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.696+0000 7f01d5ffb700 1 -- 172.21.15.46:0/1041652216 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d000fda0 con 0x5598b276ed00 2024-05-09T04:55:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.696+0000 7f01d5ffb700 1 -- 172.21.15.46:0/1041652216 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01d00118f0 con 0x5598b276ed00 2024-05-09T04:55:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.696+0000 7f01d5ffb700 1 -- 172.21.15.46:0/1041652216 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d0010730 con 0x5598b276ed00 2024-05-09T04:55:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.697+0000 7f01f4624b80 1 -- 172.21.15.46:0/1041652216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b276ed00 msgr2=0x5598b2875ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.697+0000 7f01f4624b80 1 --2- 172.21.15.46:0/1041652216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b276ed00 0x5598b2875ad0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01d000bd90 tx=0x7f01d000f640 comp rx=0 tx=0).stop 2024-05-09T04:55:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.697+0000 7f01f4624b80 1 -- 172.21.15.46:0/1041652216 shutdown_connections 2024-05-09T04:55:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.697+0000 7f01f4624b80 1 --2- 172.21.15.46:0/1041652216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b287a1e0 0x5598b287a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.697+0000 7f01f4624b80 1 --2- 172.21.15.46:0/1041652216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b276ed00 0x5598b2875ad0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.697+0000 7f01f4624b80 1 --2- 172.21.15.46:0/1041652216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b26f44a0 0x5598b2876030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.697+0000 7f01f4624b80 1 -- 172.21.15.46:0/1041652216 >> 172.21.15.46:0/1041652216 conn(0x5598b26f7f70 msgr2=0x5598b286c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:00.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.697+0000 7f01f4624b80 1 -- 172.21.15.46:0/1041652216 shutdown_connections 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.697+0000 7f01f4624b80 1 -- 172.21.15.46:0/1041652216 wait complete. 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.698+0000 7f01f4624b80 1 -- start start 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.698+0000 7f01f4624b80 1 Processor -- start 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.698+0000 7f01f4624b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 0x5598b287f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.698+0000 7f01f4624b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b276ed00 0x5598b287f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.698+0000 7f01f4624b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b287a1e0 0x5598b287ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.698+0000 7f01f4624b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5598b287af00 con 0x5598b287a1e0 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.698+0000 7f01f4624b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5598b272eb20 con 0x5598b26f44a0 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.698+0000 7f01f4624b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5598b2882470 con 0x5598b276ed00 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.698+0000 7f01f4624b80 10 client.0 _ll_register_callbacks cb 0x5598b26056c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.698+0000 7f01dffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b287a1e0 0x5598b287ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.698+0000 7f01dffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b287a1e0 0x5598b287ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51452/0 (socket says 172.21.15.46:51452) 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.698+0000 7f01dffff700 1 -- 172.21.15.46:0/3666561698 learned_addr learned my addr 172.21.15.46:0/3666561698 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.698+0000 7f01df7fe700 1 --2- 172.21.15.46:0/3666561698 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 0x5598b287f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01deffd700 1 --2- 172.21.15.46:0/3666561698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b276ed00 0x5598b287f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01dffff700 1 -- 172.21.15.46:0/3666561698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b276ed00 msgr2=0x5598b287f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01dffff700 1 --2- 172.21.15.46:0/3666561698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b276ed00 0x5598b287f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01dffff700 1 -- 172.21.15.46:0/3666561698 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 msgr2=0x5598b287f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01dffff700 1 --2- 172.21.15.46:0/3666561698 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 0x5598b287f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01dffff700 1 -- 172.21.15.46:0/3666561698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b260f490 con 0x5598b287a1e0 2024-05-09T04:55:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01deffd700 1 --2- 172.21.15.46:0/3666561698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b276ed00 0x5598b287f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01df7fe700 1 --2- 172.21.15.46:0/3666561698 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 0x5598b287f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01dffff700 1 --2- 172.21.15.46:0/3666561698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b287a1e0 0x5598b287ff30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f01d80aa9d0 tx=0x7f01d80aa690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b01f4185679302f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01beffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01beffd700 1 -- 172.21.15.46:0/3666561698 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01d80aa7e0 con 0x5598b287a1e0 2024-05-09T04:55:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01beffd700 1 -- 172.21.15.46:0/3666561698 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f01d80ab140 con 0x5598b287a1e0 2024-05-09T04:55:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01f4624b80 1 -- 172.21.15.46:0/3666561698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01d00118f0 con 0x5598b287a1e0 2024-05-09T04:55:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01beffd700 1 -- 172.21.15.46:0/3666561698 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f01d80ab3d0 con 0x5598b287a1e0 2024-05-09T04:55:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01f4624b80 10 client.16389 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01f4624b80 1 -- 172.21.15.46:0/3666561698 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5598b288ce40 con 0x5598b287a1e0 2024-05-09T04:55:00.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.699+0000 7f01beffd700 1 -- 172.21.15.46:0/3666561698 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01d00118f0 con 0x5598b287a1e0 2024-05-09T04:55:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.700+0000 7f01beffd700 1 -- 172.21.15.46:0/3666561698 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f01d8011070 con 0x5598b287a1e0 2024-05-09T04:55:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.700+0000 7f01f4624b80 10 client.16389 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.700+0000 7f01f4624b80 10 client.16389 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:55:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.700+0000 7f01f4624b80 20 client.16389 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.700+0000 7f01f4624b80 1 --2- 172.21.15.46:0/3666561698 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5598b288f3b0 0x5598b2891800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.700+0000 7f01f4624b80 4 client.16389 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:55:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.700+0000 7f01f4624b80 1 -- 172.21.15.46:0/3666561698 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5598b27a1a80 con 0x5598b288f3b0 2024-05-09T04:55:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.700+0000 7f01df7fe700 1 --2- 172.21.15.46:0/3666561698 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5598b288f3b0 0x5598b2891800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.700+0000 7f01df7fe700 1 --2- 172.21.15.46:0/3666561698 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5598b288f3b0 0x5598b2891800 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=addab0f529c74557 server_cookie=61ade169b69dd16f in_seq=0 out_seq=0 2024-05-09T04:55:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.700+0000 7f01beffd700 10 client.16389 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:55:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.701+0000 7f01beffd700 1 -- 172.21.15.46:0/3666561698 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5598b27a1a80 con 0x5598b288f3b0 2024-05-09T04:55:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.701+0000 7f01beffd700 10 client.16389 handle_command_reply: tid=0 2024-05-09T04:55:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.701+0000 7f01f4624b80 1 client.16389 shutdown 2024-05-09T04:55:00.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.701+0000 7f01f4624b80 1 -- 172.21.15.46:0/3666561698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b287a1e0 msgr2=0x5598b287ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.701+0000 7f01f4624b80 1 --2- 172.21.15.46:0/3666561698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b287a1e0 0x5598b287ff30 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f01d80aa9d0 tx=0x7f01d80aa690 comp rx=0 tx=0).stop 2024-05-09T04:55:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.702+0000 7f01f4624b80 1 -- 172.21.15.46:0/3666561698 shutdown_connections 2024-05-09T04:55:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.702+0000 7f01f4624b80 1 --2- 172.21.15.46:0/3666561698 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5598b288f3b0 0x5598b2891800 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.702+0000 7f01f4624b80 1 --2- 172.21.15.46:0/3666561698 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b287a1e0 0x5598b287ff30 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.702+0000 7f01f4624b80 1 --2- 172.21.15.46:0/3666561698 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 0x5598b287f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.702+0000 7f01beffd700 0 client.16389 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:55:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.702+0000 7f01f4624b80 1 --2- 172.21.15.46:0/3666561698 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b276ed00 0x5598b287f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.702+0000 7f01f4624b80 1 -- 172.21.15.46:0/3666561698 >> 172.21.15.46:0/3666561698 conn(0x5598b26f7f70 msgr2=0x5598b28703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.702+0000 7f01f4624b80 1 -- 172.21.15.46:0/3666561698 shutdown_connections 2024-05-09T04:55:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.702+0000 7f01f4624b80 1 -- 172.21.15.46:0/3666561698 wait complete. 2024-05-09T04:55:00.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.702+0000 7f01f4624b80 20 client.16389 trim_cache size 0 max 16384 2024-05-09T04:55:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.717+0000 7f01f4624b80 1 Processor -- start 2024-05-09T04:55:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.717+0000 7f01f4624b80 1 -- start start 2024-05-09T04:55:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.717+0000 7f01f4624b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 0x5598b287fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.717+0000 7f01f4624b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b276ed00 0x5598b2866700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.717+0000 7f01f4624b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b287a1e0 0x5598b288b220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.717+0000 7f01f4624b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01e01af5f0 con 0x5598b276ed00 2024-05-09T04:55:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.717+0000 7f01f4624b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f01e01ace80 con 0x5598b26f44a0 2024-05-09T04:55:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.717+0000 7f01f4624b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f01e01aacd0 con 0x5598b287a1e0 2024-05-09T04:55:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.717+0000 7f01deffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b276ed00 0x5598b2866700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.717+0000 7f01deffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b276ed00 0x5598b2866700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51468/0 (socket says 172.21.15.46:51468) 2024-05-09T04:55:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.717+0000 7f01deffd700 1 -- 172.21.15.46:0/537987508 learned_addr learned my addr 172.21.15.46:0/537987508 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.717+0000 7f01df7fe700 1 --2- 172.21.15.46:0/537987508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 0x5598b287fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.718+0000 7f01dffff700 1 --2- 172.21.15.46:0/537987508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b287a1e0 0x5598b288b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.718+0000 7f01deffd700 1 -- 172.21.15.46:0/537987508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b287a1e0 msgr2=0x5598b288b220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.718+0000 7f01deffd700 1 --2- 172.21.15.46:0/537987508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b287a1e0 0x5598b288b220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.718+0000 7f01deffd700 1 -- 172.21.15.46:0/537987508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 msgr2=0x5598b287fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:00.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.718+0000 7f01deffd700 1 --2- 172.21.15.46:0/537987508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 0x5598b287fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.718+0000 7f01deffd700 1 -- 172.21.15.46:0/537987508 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5598b28689d0 con 0x5598b276ed00 2024-05-09T04:55:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.718+0000 7f01df7fe700 1 --2- 172.21.15.46:0/537987508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 0x5598b287fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.718+0000 7f01deffd700 1 --2- 172.21.15.46:0/537987508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b276ed00 0x5598b2866700 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f01e006c680 tx=0x7f01c8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79d8b4111ccac6ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.718+0000 7f01d57fa700 1 -- 172.21.15.46:0/537987508 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8016d70 con 0x5598b276ed00 2024-05-09T04:55:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.718+0000 7f01d57fa700 1 -- 172.21.15.46:0/537987508 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c8026070 con 0x5598b276ed00 2024-05-09T04:55:00.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.718+0000 7f01d57fa700 1 -- 172.21.15.46:0/537987508 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c80176a0 con 0x5598b276ed00 2024-05-09T04:55:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.719+0000 7f01f4624b80 1 -- 172.21.15.46:0/537987508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b276ed00 msgr2=0x5598b2866700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.719+0000 7f01f4624b80 1 --2- 172.21.15.46:0/537987508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b276ed00 0x5598b2866700 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f01e006c680 tx=0x7f01c8018000 comp rx=0 tx=0).stop 2024-05-09T04:55:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.719+0000 7f01f4624b80 1 -- 172.21.15.46:0/537987508 shutdown_connections 2024-05-09T04:55:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.719+0000 7f01f4624b80 1 --2- 172.21.15.46:0/537987508 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b287a1e0 0x5598b288b220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.719+0000 7f01f4624b80 1 --2- 172.21.15.46:0/537987508 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 0x5598b287fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.719+0000 7f01f4624b80 1 --2- 172.21.15.46:0/537987508 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b276ed00 0x5598b2866700 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.719+0000 7f01f4624b80 1 -- 172.21.15.46:0/537987508 >> 172.21.15.46:0/537987508 conn(0x5598b26f7f70 msgr2=0x5598b2865ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.719+0000 7f01f4624b80 1 -- 172.21.15.46:0/537987508 shutdown_connections 2024-05-09T04:55:00.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.719+0000 7f01f4624b80 1 -- 172.21.15.46:0/537987508 wait complete. 2024-05-09T04:55:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.719+0000 7f01f4624b80 1 -- start start 2024-05-09T04:55:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01f4624b80 1 Processor -- start 2024-05-09T04:55:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01f4624b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 0x5598b28828f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01f4624b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b276ed00 0x5598b2882e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01f4624b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b287a1e0 0x5598b28f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01f4624b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01e01ab340 con 0x5598b276ed00 2024-05-09T04:55:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01f4624b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f01e01ac7b0 con 0x5598b26f44a0 2024-05-09T04:55:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01f4624b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f01e01aba10 con 0x5598b287a1e0 2024-05-09T04:55:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01f4624b80 10 client.0 _ll_register_callbacks cb 0x5598b26056c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01deffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b276ed00 0x5598b2882e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01deffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b276ed00 0x5598b2882e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51472/0 (socket says 172.21.15.46:51472) 2024-05-09T04:55:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01deffd700 1 -- 172.21.15.46:0/1861116825 learned_addr learned my addr 172.21.15.46:0/1861116825 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01dffff700 1 --2- 172.21.15.46:0/1861116825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b287a1e0 0x5598b28f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01df7fe700 1 --2- 172.21.15.46:0/1861116825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 0x5598b28828f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01deffd700 1 -- 172.21.15.46:0/1861116825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b287a1e0 msgr2=0x5598b28f7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01deffd700 1 --2- 172.21.15.46:0/1861116825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b287a1e0 0x5598b28f7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01deffd700 1 -- 172.21.15.46:0/1861116825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 msgr2=0x5598b28828f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.720+0000 7f01deffd700 1 --2- 172.21.15.46:0/1861116825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 0x5598b28828f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.721+0000 7f01deffd700 1 -- 172.21.15.46:0/1861116825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5598b28f80a0 con 0x5598b276ed00 2024-05-09T04:55:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.721+0000 7f01deffd700 1 --2- 172.21.15.46:0/1861116825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b276ed00 0x5598b2882e30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f01c8023040 tx=0x7f01c8021430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dcba56c928661ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.721+0000 7f01beffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.721+0000 7f01beffd700 1 -- 172.21.15.46:0/1861116825 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8021a60 con 0x5598b276ed00 2024-05-09T04:55:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.721+0000 7f01beffd700 1 -- 172.21.15.46:0/1861116825 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f01c8005a50 con 0x5598b276ed00 2024-05-09T04:55:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.721+0000 7f01f4624b80 1 -- 172.21.15.46:0/1861116825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01c8026070 con 0x5598b276ed00 2024-05-09T04:55:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.721+0000 7f01beffd700 1 -- 172.21.15.46:0/1861116825 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f01c803b050 con 0x5598b276ed00 2024-05-09T04:55:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.721+0000 7f01f4624b80 10 client.16395 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.721+0000 7f01f4624b80 1 -- 172.21.15.46:0/1861116825 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5598b28f9250 con 0x5598b276ed00 2024-05-09T04:55:00.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.721+0000 7f01beffd700 1 -- 172.21.15.46:0/1861116825 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c8026070 con 0x5598b276ed00 2024-05-09T04:55:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.722+0000 7f01beffd700 1 -- 172.21.15.46:0/1861116825 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f01c8020070 con 0x5598b276ed00 2024-05-09T04:55:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.722+0000 7f01f4624b80 10 client.16395 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.722+0000 7f01f4624b80 10 client.16395 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:55:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.722+0000 7f01f4624b80 20 client.16395 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.722+0000 7f01f4624b80 1 --2- 172.21.15.46:0/1861116825 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5598b28f9990 0x5598b28fbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.722+0000 7f01f4624b80 4 client.16395 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:55:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.722+0000 7f01f4624b80 1 -- 172.21.15.46:0/1861116825 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5598b279bb20 con 0x5598b28f9990 2024-05-09T04:55:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.722+0000 7f01df7fe700 1 --2- 172.21.15.46:0/1861116825 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5598b28f9990 0x5598b28fbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.722+0000 7f01df7fe700 1 --2- 172.21.15.46:0/1861116825 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5598b28f9990 0x5598b28fbdb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=59f75dc6967f11c2 server_cookie=4ddb381aaf151d9e in_seq=0 out_seq=0 2024-05-09T04:55:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.722+0000 7f01beffd700 10 client.16395 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:55:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.723+0000 7f01beffd700 1 -- 172.21.15.46:0/1861116825 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5598b279bb20 con 0x5598b28f9990 2024-05-09T04:55:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.723+0000 7f01beffd700 10 client.16395 handle_command_reply: tid=0 2024-05-09T04:55:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.723+0000 7f01f4624b80 1 client.16395 shutdown 2024-05-09T04:55:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.723+0000 7f01f4624b80 1 -- 172.21.15.46:0/1861116825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b276ed00 msgr2=0x5598b2882e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:00.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.723+0000 7f01f4624b80 1 --2- 172.21.15.46:0/1861116825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b276ed00 0x5598b2882e30 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f01c8023040 tx=0x7f01c8021430 comp rx=0 tx=0).stop 2024-05-09T04:55:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.723+0000 7f01f4624b80 1 -- 172.21.15.46:0/1861116825 shutdown_connections 2024-05-09T04:55:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.723+0000 7f01f4624b80 1 --2- 172.21.15.46:0/1861116825 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5598b28f9990 0x5598b28fbdb0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.723+0000 7f01f4624b80 1 --2- 172.21.15.46:0/1861116825 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5598b287a1e0 0x5598b28f7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.723+0000 7f01f4624b80 1 --2- 172.21.15.46:0/1861116825 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5598b26f44a0 0x5598b28828f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.723+0000 7f01f4624b80 1 --2- 172.21.15.46:0/1861116825 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5598b276ed00 0x5598b2882e30 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.723+0000 7f01f4624b80 1 -- 172.21.15.46:0/1861116825 >> 172.21.15.46:0/1861116825 conn(0x5598b26f7f70 msgr2=0x5598b28703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.723+0000 7f01beffd700 0 client.16395 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:55:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.723+0000 7f01f4624b80 1 -- 172.21.15.46:0/1861116825 shutdown_connections 2024-05-09T04:55:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.723+0000 7f01f4624b80 1 -- 172.21.15.46:0/1861116825 wait complete. 2024-05-09T04:55:00.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.723+0000 7f01f4624b80 20 client.16395 trim_cache size 0 max 16384 2024-05-09T04:55:00.729 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:55:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.724+0000 7f01e5c0e700 1 -- 172.21.15.46:0/3316482786 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f01cc0b2a70 msgr2=0x7f01cc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.724+0000 7f01e5c0e700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f01cc0b2a70 0x7f01cc0b4f80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f01e0103a70 tx=0x7f01c8009db0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.724+0000 7f01e5c0e700 1 -- 172.21.15.46:0/3316482786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e0109870 msgr2=0x7f01e019f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:00.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.724+0000 7f01e5c0e700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e0109870 0x7f01e019f690 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f01d80095c0 tx=0x7f01d8007ad0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.724+0000 7f01e5c0e700 1 -- 172.21.15.46:0/3316482786 shutdown_connections 2024-05-09T04:55:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.724+0000 7f01e5c0e700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f01cc0b2a70 0x7f01cc0b4f80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.724+0000 7f01e5c0e700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01e0109870 0x7f01e019f690 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.724+0000 7f01e5c0e700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01e01045f0 0x7f01e019ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.724+0000 7f01e5c0e700 1 --2- 172.21.15.46:0/3316482786 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01e0108f20 0x7f01e019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.724+0000 7f01e5c0e700 1 -- 172.21.15.46:0/3316482786 >> 172.21.15.46:0/3316482786 conn(0x7f01e0100040 msgr2=0x7f01e0108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.724+0000 7f01e5c0e700 1 -- 172.21.15.46:0/3316482786 shutdown_connections 2024-05-09T04:55:00.731 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:00.724+0000 7f01e5c0e700 1 -- 172.21.15.46:0/3316482786 wait complete. 2024-05-09T04:55:01.009 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:00 smithi046 ceph-mon[30136]: pgmap v120: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:55:01.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:00 smithi110 ceph-mon[34904]: pgmap v120: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:55:01.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:00 smithi072 ceph-mon[34088]: pgmap v120: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:55:01.557 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:55:01.821 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:03.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:02 smithi110 ceph-mon[34904]: pgmap v121: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T04:55:03.062 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:02 smithi046 ceph-mon[30136]: pgmap v121: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T04:55:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:02 smithi072 ceph-mon[34088]: pgmap v121: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 12 KiB/s wr, 10 op/s 2024-05-09T04:55:03.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.629+0000 7f01a7d01700 1 -- 172.21.15.46:0/3400465149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a01045d0 msgr2=0x7f01a01049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.629+0000 7f01a7d01700 1 --2- 172.21.15.46:0/3400465149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a01045d0 0x7f01a01049e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f019c0099c0 tx=0x7f019c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.629+0000 7f01a7d01700 1 -- 172.21.15.46:0/3400465149 shutdown_connections 2024-05-09T04:55:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.629+0000 7f01a7d01700 1 --2- 172.21.15.46:0/3400465149 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01a0109850 0x7f01a01111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.629+0000 7f01a7d01700 1 --2- 172.21.15.46:0/3400465149 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a01045d0 0x7f01a01049e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.629+0000 7f01a7d01700 1 --2- 172.21.15.46:0/3400465149 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01a0108f00 0x7f01a0109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.629+0000 7f01a7d01700 1 -- 172.21.15.46:0/3400465149 >> 172.21.15.46:0/3400465149 conn(0x7f01a00ffea0 msgr2=0x7f01a01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.629+0000 7f01a7d01700 1 -- 172.21.15.46:0/3400465149 shutdown_connections 2024-05-09T04:55:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.629+0000 7f01a7d01700 1 -- 172.21.15.46:0/3400465149 wait complete. 2024-05-09T04:55:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.630+0000 7f01a7d01700 1 Processor -- start 2024-05-09T04:55:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.630+0000 7f01a7d01700 1 -- start start 2024-05-09T04:55:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.630+0000 7f01a7d01700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01a01045d0 0x7f01a0076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.630+0000 7f01a7d01700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a0108f00 0x7f01a0077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.630+0000 7f01a7d01700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01a0109850 0x7f01a0077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.630+0000 7f01a7d01700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01a0114480 con 0x7f01a0108f00 2024-05-09T04:55:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.630+0000 7f01a7d01700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f01a0114300 con 0x7f01a01045d0 2024-05-09T04:55:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.630+0000 7f01a7d01700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f01a0114600 con 0x7f01a0109850 2024-05-09T04:55:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.630+0000 7f01a529c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a0108f00 0x7f01a0077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.630+0000 7f01a529c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a0108f00 0x7f01a0077110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51504/0 (socket says 172.21.15.46:51504) 2024-05-09T04:55:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.630+0000 7f01a529c700 1 -- 172.21.15.46:0/790165759 learned_addr learned my addr 172.21.15.46:0/790165759 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.631+0000 7f01a5a9d700 1 --2- 172.21.15.46:0/790165759 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01a01045d0 0x7f01a0076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.631+0000 7f01a629e700 1 --2- 172.21.15.46:0/790165759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01a0109850 0x7f01a0077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.631+0000 7f01a529c700 1 -- 172.21.15.46:0/790165759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01a0109850 msgr2=0x7f01a0077650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.631+0000 7f01a529c700 1 --2- 172.21.15.46:0/790165759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01a0109850 0x7f01a0077650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.631+0000 7f01a529c700 1 -- 172.21.15.46:0/790165759 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01a01045d0 msgr2=0x7f01a0076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.631+0000 7f01a529c700 1 --2- 172.21.15.46:0/790165759 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01a01045d0 0x7f01a0076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.631+0000 7f01a529c700 1 -- 172.21.15.46:0/790165759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a01a90e0 con 0x7f01a0108f00 2024-05-09T04:55:03.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.631+0000 7f01a529c700 1 --2- 172.21.15.46:0/790165759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a0108f00 0x7f01a0077110 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f019000c8f0 tx=0x7f019000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:03.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.631+0000 7f0196ffd700 1 -- 172.21.15.46:0/790165759 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190007940 con 0x7f01a0108f00 2024-05-09T04:55:03.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.631+0000 7f0196ffd700 1 -- 172.21.15.46:0/790165759 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f019000f450 con 0x7f01a0108f00 2024-05-09T04:55:03.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.631+0000 7f0196ffd700 1 -- 172.21.15.46:0/790165759 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190018660 con 0x7f01a0108f00 2024-05-09T04:55:03.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.631+0000 7f01a7d01700 1 -- 172.21.15.46:0/790165759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a01a93d0 con 0x7f01a0108f00 2024-05-09T04:55:03.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.631+0000 7f01a7d01700 1 -- 172.21.15.46:0/790165759 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a01a98c0 con 0x7f01a0108f00 2024-05-09T04:55:03.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.634+0000 7f0196ffd700 1 -- 172.21.15.46:0/790165759 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f019000f890 con 0x7f01a0108f00 2024-05-09T04:55:03.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.635+0000 7f0196ffd700 1 --2- 172.21.15.46:0/790165759 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f018c0b29f0 0x7f018c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.635+0000 7f01a5a9d700 1 --2- 172.21.15.46:0/790165759 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f018c0b29f0 0x7f018c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.635+0000 7f0196ffd700 1 -- 172.21.15.46:0/790165759 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f01900e2500 con 0x7f01a0108f00 2024-05-09T04:55:03.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.635+0000 7f01b6717b80 1 Processor -- start 2024-05-09T04:55:03.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01b6717b80 1 -- start start 2024-05-09T04:55:03.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01a5a9d700 1 --2- 172.21.15.46:0/790165759 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f018c0b29f0 0x7f018c0b4f00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f019c009990 tx=0x7f019c000bc0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:03.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01b6717b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d50006b160 0x55d5001044f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01b6717b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4fff814a0 0x55d500104a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01b6717b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5000fe6a0 0x55d5000fead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01b6717b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d500020ff0 con 0x55d4fff814a0 2024-05-09T04:55:03.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01b6717b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d4fffbf7f0 con 0x55d50006b160 2024-05-09T04:55:03.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01b6717b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d50001cfb0 con 0x55d5000fe6a0 2024-05-09T04:55:03.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01a529c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4fff814a0 0x55d500104a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01a529c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4fff814a0 0x55d500104a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51506/0 (socket says 172.21.15.46:51506) 2024-05-09T04:55:03.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01a529c700 1 -- 172.21.15.46:0/2266602570 learned_addr learned my addr 172.21.15.46:0/2266602570 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:03.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01a5a9d700 1 --2- 172.21.15.46:0/2266602570 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d50006b160 0x55d5001044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01a629e700 1 --2- 172.21.15.46:0/2266602570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5000fe6a0 0x55d5000fead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01a529c700 1 -- 172.21.15.46:0/2266602570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5000fe6a0 msgr2=0x55d5000fead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:03.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01a529c700 1 --2- 172.21.15.46:0/2266602570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5000fe6a0 0x55d5000fead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01a529c700 1 -- 172.21.15.46:0/2266602570 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d50006b160 msgr2=0x55d5001044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:03.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01a529c700 1 --2- 172.21.15.46:0/2266602570 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d50006b160 0x55d5001044f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01a529c700 1 -- 172.21.15.46:0/2266602570 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4fffb7370 con 0x55d4fff814a0 2024-05-09T04:55:03.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.636+0000 7f01a629e700 1 --2- 172.21.15.46:0/2266602570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5000fe6a0 0x55d5000fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:03.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.637+0000 7f01a529c700 1 --2- 172.21.15.46:0/2266602570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4fff814a0 0x55d500104a30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f019000fbe0 tx=0x7f01900a9240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b82c92478bbfa946 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:03.640 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.637+0000 7f018bfff700 1 -- 172.21.15.46:0/2266602570 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01900b2620 con 0x55d4fff814a0 2024-05-09T04:55:03.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.637+0000 7f018bfff700 1 -- 172.21.15.46:0/2266602570 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f019000f600 con 0x55d4fff814a0 2024-05-09T04:55:03.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.637+0000 7f018bfff700 1 -- 172.21.15.46:0/2266602570 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01900a9c60 con 0x55d4fff814a0 2024-05-09T04:55:03.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.638+0000 7f01b6717b80 1 -- 172.21.15.46:0/2266602570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4fff814a0 msgr2=0x55d500104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:03.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.638+0000 7f01b6717b80 1 --2- 172.21.15.46:0/2266602570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4fff814a0 0x55d500104a30 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f019000fbe0 tx=0x7f01900a9240 comp rx=0 tx=0).stop 2024-05-09T04:55:03.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.638+0000 7f01b6717b80 1 -- 172.21.15.46:0/2266602570 shutdown_connections 2024-05-09T04:55:03.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.638+0000 7f01b6717b80 1 --2- 172.21.15.46:0/2266602570 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5000fe6a0 0x55d5000fead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.638+0000 7f01b6717b80 1 --2- 172.21.15.46:0/2266602570 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d50006b160 0x55d5001044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.638+0000 7f01b6717b80 1 --2- 172.21.15.46:0/2266602570 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4fff814a0 0x55d500104a30 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.638+0000 7f01b6717b80 1 -- 172.21.15.46:0/2266602570 >> 172.21.15.46:0/2266602570 conn(0x55d4fffe16a0 msgr2=0x55d5000f9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:03.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.638+0000 7f01b6717b80 1 -- 172.21.15.46:0/2266602570 shutdown_connections 2024-05-09T04:55:03.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.638+0000 7f01b6717b80 1 -- 172.21.15.46:0/2266602570 wait complete. 2024-05-09T04:55:03.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.638+0000 7f01b6717b80 1 -- start start 2024-05-09T04:55:03.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.638+0000 7f01b6717b80 1 Processor -- start 2024-05-09T04:55:03.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01b6717b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4fff814a0 0x55d500108260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01b6717b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d50006b160 0x55d5001087a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01b6717b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5000fe6a0 0x55d500108ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01b6717b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d4ffe9cba0 con 0x55d5000fe6a0 2024-05-09T04:55:03.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01b6717b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d4fffdb150 con 0x55d4fff814a0 2024-05-09T04:55:03.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01b6717b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d50010b1d0 con 0x55d50006b160 2024-05-09T04:55:03.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01b6717b80 10 client.0 _ll_register_callbacks cb 0x55d4ffe926c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:03.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01a629e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5000fe6a0 0x55d500108ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01a629e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5000fe6a0 0x55d500108ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51514/0 (socket says 172.21.15.46:51514) 2024-05-09T04:55:03.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01a629e700 1 -- 172.21.15.46:0/3611487061 learned_addr learned my addr 172.21.15.46:0/3611487061 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:03.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01a529c700 1 --2- 172.21.15.46:0/3611487061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d50006b160 0x55d5001087a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01a5a9d700 1 --2- 172.21.15.46:0/3611487061 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4fff814a0 0x55d500108260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01a629e700 1 -- 172.21.15.46:0/3611487061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d50006b160 msgr2=0x55d5001087a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:03.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01a629e700 1 --2- 172.21.15.46:0/3611487061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d50006b160 0x55d5001087a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01a629e700 1 -- 172.21.15.46:0/3611487061 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4fff814a0 msgr2=0x55d500108260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:03.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.639+0000 7f01a629e700 1 --2- 172.21.15.46:0/3611487061 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4fff814a0 0x55d500108260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.640+0000 7f01a629e700 1 -- 172.21.15.46:0/3611487061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4ffe9c490 con 0x55d5000fe6a0 2024-05-09T04:55:03.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.640+0000 7f01a629e700 1 --2- 172.21.15.46:0/3611487061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5000fe6a0 0x55d500108ce0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0198010b20 tx=0x7f0198010e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6411ea4a183a00 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:03.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.640+0000 7f01897fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:03.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.640+0000 7f01897fa700 1 -- 172.21.15.46:0/3611487061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019801a020 con 0x55d5000fe6a0 2024-05-09T04:55:03.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.640+0000 7f01897fa700 1 -- 172.21.15.46:0/3611487061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f019800ea00 con 0x55d5000fe6a0 2024-05-09T04:55:03.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.640+0000 7f01b6717b80 1 -- 172.21.15.46:0/3611487061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f019000f600 con 0x55d5000fe6a0 2024-05-09T04:55:03.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.640+0000 7f01897fa700 1 -- 172.21.15.46:0/3611487061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0198025050 con 0x55d5000fe6a0 2024-05-09T04:55:03.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.640+0000 7f01b6717b80 10 client.16407 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:03.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.640+0000 7f01b6717b80 1 -- 172.21.15.46:0/3611487061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d500115ca0 con 0x55d5000fe6a0 2024-05-09T04:55:03.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.640+0000 7f01897fa700 1 -- 172.21.15.46:0/3611487061 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f019000f600 con 0x55d5000fe6a0 2024-05-09T04:55:03.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.640+0000 7f01897fa700 1 -- 172.21.15.46:0/3611487061 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f0198015b30 con 0x55d5000fe6a0 2024-05-09T04:55:03.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.641+0000 7f01b6717b80 10 client.16407 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:03.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.641+0000 7f01b6717b80 10 client.16407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:55:03.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.641+0000 7f01b6717b80 20 client.16407 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:03.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.641+0000 7f01b6717b80 1 --2- 172.21.15.46:0/3611487061 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500118330 0x55d50011a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.641+0000 7f01b6717b80 4 client.16407 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:55:03.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.641+0000 7f01b6717b80 1 -- 172.21.15.46:0/3611487061 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d50002e9d0 con 0x55d500118330 2024-05-09T04:55:03.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.641+0000 7f01a5a9d700 1 --2- 172.21.15.46:0/3611487061 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500118330 0x55d50011a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.641+0000 7f01a5a9d700 1 --2- 172.21.15.46:0/3611487061 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500118330 0x55d50011a750 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f1d2078460b6aad server_cookie=55cdd40e2103d4a8 in_seq=0 out_seq=0 2024-05-09T04:55:03.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.641+0000 7f01897fa700 10 client.16407 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:03.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.642+0000 7f01897fa700 1 -- 172.21.15.46:0/3611487061 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d50002e9d0 con 0x55d500118330 2024-05-09T04:55:03.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.642+0000 7f01897fa700 10 client.16407 handle_command_reply: tid=0 2024-05-09T04:55:03.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.642+0000 7f01b6717b80 1 client.16407 shutdown 2024-05-09T04:55:03.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.642+0000 7f01b6717b80 1 -- 172.21.15.46:0/3611487061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5000fe6a0 msgr2=0x55d500108ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:03.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.642+0000 7f01b6717b80 1 --2- 172.21.15.46:0/3611487061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5000fe6a0 0x55d500108ce0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f0198010b20 tx=0x7f0198010e30 comp rx=0 tx=0).stop 2024-05-09T04:55:03.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.642+0000 7f01b6717b80 1 -- 172.21.15.46:0/3611487061 shutdown_connections 2024-05-09T04:55:03.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.643+0000 7f01b6717b80 1 --2- 172.21.15.46:0/3611487061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5000fe6a0 0x55d500108ce0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.643+0000 7f01b6717b80 1 --2- 172.21.15.46:0/3611487061 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500118330 0x55d50011a750 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.643+0000 7f01b6717b80 1 --2- 172.21.15.46:0/3611487061 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d4fff814a0 0x55d500108260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.643+0000 7f01b6717b80 1 --2- 172.21.15.46:0/3611487061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d50006b160 0x55d5001087a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.643+0000 7f01897fa700 0 client.16407 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:03.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.643+0000 7f01b6717b80 1 -- 172.21.15.46:0/3611487061 >> 172.21.15.46:0/3611487061 conn(0x55d4fffe16a0 msgr2=0x55d5000f91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:03.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.643+0000 7f01b6717b80 1 -- 172.21.15.46:0/3611487061 shutdown_connections 2024-05-09T04:55:03.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.643+0000 7f01b6717b80 1 -- 172.21.15.46:0/3611487061 wait complete. 2024-05-09T04:55:03.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.643+0000 7f01b6717b80 20 client.16407 trim_cache size 0 max 16384 2024-05-09T04:55:03.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.658+0000 7f01b6717b80 1 Processor -- start 2024-05-09T04:55:03.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.658+0000 7f01b6717b80 1 -- start start 2024-05-09T04:55:03.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.658+0000 7f01b6717b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4fff814a0 0x55d500113fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.658+0000 7f01b6717b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d50006b160 0x55d500114510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.658+0000 7f01b6717b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5000fe6a0 0x55d500103350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.658+0000 7f01b6717b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01b39b0 con 0x55d4fff814a0 2024-05-09T04:55:03.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.658+0000 7f01b6717b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01b1240 con 0x55d50006b160 2024-05-09T04:55:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.658+0000 7f01b6717b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01af090 con 0x55d5000fe6a0 2024-05-09T04:55:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.658+0000 7f01a5a9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4fff814a0 0x55d500113fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.658+0000 7f01a5a9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4fff814a0 0x55d500113fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51518/0 (socket says 172.21.15.46:51518) 2024-05-09T04:55:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.658+0000 7f01a5a9d700 1 -- 172.21.15.46:0/39599760 learned_addr learned my addr 172.21.15.46:0/39599760 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.659+0000 7f01a629e700 1 --2- 172.21.15.46:0/39599760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5000fe6a0 0x55d500103350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.659+0000 7f01a529c700 1 --2- 172.21.15.46:0/39599760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d50006b160 0x55d500114510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.659+0000 7f01a5a9d700 1 -- 172.21.15.46:0/39599760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5000fe6a0 msgr2=0x55d500103350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.659+0000 7f01a5a9d700 1 --2- 172.21.15.46:0/39599760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5000fe6a0 0x55d500103350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.659+0000 7f01a5a9d700 1 -- 172.21.15.46:0/39599760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d50006b160 msgr2=0x55d500114510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.659+0000 7f01a5a9d700 1 --2- 172.21.15.46:0/39599760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d50006b160 0x55d500114510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.659+0000 7f01a5a9d700 1 -- 172.21.15.46:0/39599760 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d50010ab00 con 0x55d4fff814a0 2024-05-09T04:55:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.659+0000 7f01a5a9d700 1 --2- 172.21.15.46:0/39599760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4fff814a0 0x55d500113fd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f019c00b580 tx=0x7f019c01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cea901efb8db8233 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.659+0000 7f018b7fe700 1 -- 172.21.15.46:0/39599760 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c038070 con 0x55d4fff814a0 2024-05-09T04:55:03.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.659+0000 7f018b7fe700 1 -- 172.21.15.46:0/39599760 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f019c00f460 con 0x55d4fff814a0 2024-05-09T04:55:03.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.659+0000 7f018b7fe700 1 -- 172.21.15.46:0/39599760 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019c0326a0 con 0x55d4fff814a0 2024-05-09T04:55:03.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.660+0000 7f01b6717b80 1 -- 172.21.15.46:0/39599760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4fff814a0 msgr2=0x55d500113fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.660+0000 7f01b6717b80 1 --2- 172.21.15.46:0/39599760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4fff814a0 0x55d500113fd0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f019c00b580 tx=0x7f019c01e000 comp rx=0 tx=0).stop 2024-05-09T04:55:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.660+0000 7f01b6717b80 1 -- 172.21.15.46:0/39599760 shutdown_connections 2024-05-09T04:55:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.660+0000 7f01b6717b80 1 --2- 172.21.15.46:0/39599760 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5000fe6a0 0x55d500103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.660+0000 7f01b6717b80 1 --2- 172.21.15.46:0/39599760 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d4fff814a0 0x55d500113fd0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.660+0000 7f01b6717b80 1 --2- 172.21.15.46:0/39599760 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d50006b160 0x55d500114510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.660+0000 7f01b6717b80 1 -- 172.21.15.46:0/39599760 >> 172.21.15.46:0/39599760 conn(0x55d4fffe16a0 msgr2=0x55d500106b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.660+0000 7f01b6717b80 1 -- 172.21.15.46:0/39599760 shutdown_connections 2024-05-09T04:55:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.660+0000 7f01b6717b80 1 -- 172.21.15.46:0/39599760 wait complete. 2024-05-09T04:55:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.661+0000 7f01b6717b80 1 -- start start 2024-05-09T04:55:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.661+0000 7f01b6717b80 1 Processor -- start 2024-05-09T04:55:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01b6717b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4fff814a0 0x55d5001107e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01b6717b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d50006b160 0x55d50011f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01b6717b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5000fe6a0 0x55d50011fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01b6717b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01af700 con 0x55d5000fe6a0 2024-05-09T04:55:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01b6717b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01b0b70 con 0x55d50006b160 2024-05-09T04:55:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01b6717b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f01a01afdd0 con 0x55d4fff814a0 2024-05-09T04:55:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01b6717b80 10 client.0 _ll_register_callbacks cb 0x55d4ffe926c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01a629e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5000fe6a0 0x55d50011fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01a629e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5000fe6a0 0x55d50011fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51528/0 (socket says 172.21.15.46:51528) 2024-05-09T04:55:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01a629e700 1 -- 172.21.15.46:0/4085448799 learned_addr learned my addr 172.21.15.46:0/4085448799 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01a5a9d700 1 --2- 172.21.15.46:0/4085448799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4fff814a0 0x55d5001107e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01a529c700 1 --2- 172.21.15.46:0/4085448799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d50006b160 0x55d50011f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01a629e700 1 -- 172.21.15.46:0/4085448799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4fff814a0 msgr2=0x55d5001107e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01a629e700 1 --2- 172.21.15.46:0/4085448799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4fff814a0 0x55d5001107e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01a629e700 1 -- 172.21.15.46:0/4085448799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d50006b160 msgr2=0x55d50011f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:03.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01a629e700 1 --2- 172.21.15.46:0/4085448799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d50006b160 0x55d50011f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01a629e700 1 -- 172.21.15.46:0/4085448799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d500120080 con 0x55d5000fe6a0 2024-05-09T04:55:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.662+0000 7f01a5a9d700 1 --2- 172.21.15.46:0/4085448799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4fff814a0 0x55d5001107e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.663+0000 7f01a629e700 1 --2- 172.21.15.46:0/4085448799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5000fe6a0 0x55d50011fa10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0198016850 tx=0x7f0198016880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6618ed3144965422 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.663+0000 7f01897fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.663+0000 7f01897fa700 1 -- 172.21.15.46:0/4085448799 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019802d070 con 0x55d5000fe6a0 2024-05-09T04:55:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.663+0000 7f01897fa700 1 -- 172.21.15.46:0/4085448799 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f0198004a50 con 0x55d5000fe6a0 2024-05-09T04:55:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.663+0000 7f01b6717b80 1 -- 172.21.15.46:0/4085448799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f019c00f460 con 0x55d5000fe6a0 2024-05-09T04:55:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.663+0000 7f01897fa700 1 -- 172.21.15.46:0/4085448799 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f019801a050 con 0x55d5000fe6a0 2024-05-09T04:55:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.663+0000 7f01b6717b80 10 client.16413 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.663+0000 7f01b6717b80 1 -- 172.21.15.46:0/4085448799 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d500182890 con 0x55d5000fe6a0 2024-05-09T04:55:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.663+0000 7f01897fa700 1 -- 172.21.15.46:0/4085448799 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0198007e50 con 0x55d5000fe6a0 2024-05-09T04:55:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.664+0000 7f01897fa700 1 -- 172.21.15.46:0/4085448799 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f0198031400 con 0x55d5000fe6a0 2024-05-09T04:55:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.664+0000 7f01b6717b80 10 client.16413 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:03.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.664+0000 7f01b6717b80 10 client.16413 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:55:03.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.664+0000 7f01b6717b80 20 client.16413 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:03.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.664+0000 7f01b6717b80 1 --2- 172.21.15.46:0/4085448799 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500182f70 0x55d500185390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:03.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.664+0000 7f01b6717b80 4 client.16413 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:55:03.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.664+0000 7f01b6717b80 1 -- 172.21.15.46:0/4085448799 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f019c00f460 con 0x55d500182f70 2024-05-09T04:55:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.664+0000 7f01a5a9d700 1 --2- 172.21.15.46:0/4085448799 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500182f70 0x55d500185390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.664+0000 7f01897fa700 10 client.16413 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.664+0000 7f01a5a9d700 1 --2- 172.21.15.46:0/4085448799 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500182f70 0x55d500185390 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16aec597fda23edb server_cookie=f9e3cc8fcd8aa416 in_seq=0 out_seq=0 2024-05-09T04:55:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.665+0000 7f01897fa700 1 -- 172.21.15.46:0/4085448799 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f019c00f460 con 0x55d500182f70 2024-05-09T04:55:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.665+0000 7f01897fa700 10 client.16413 handle_command_reply: tid=0 2024-05-09T04:55:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.665+0000 7f01b6717b80 1 client.16413 shutdown 2024-05-09T04:55:03.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.665+0000 7f01b6717b80 1 -- 172.21.15.46:0/4085448799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5000fe6a0 msgr2=0x55d50011fa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.665+0000 7f01b6717b80 1 --2- 172.21.15.46:0/4085448799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5000fe6a0 0x55d50011fa10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f0198016850 tx=0x7f0198016880 comp rx=0 tx=0).stop 2024-05-09T04:55:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.665+0000 7f01b6717b80 1 -- 172.21.15.46:0/4085448799 shutdown_connections 2024-05-09T04:55:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.665+0000 7f01b6717b80 1 --2- 172.21.15.46:0/4085448799 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5000fe6a0 0x55d50011fa10 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.665+0000 7f01b6717b80 1 --2- 172.21.15.46:0/4085448799 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d4fff814a0 0x55d5001107e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.665+0000 7f01b6717b80 1 --2- 172.21.15.46:0/4085448799 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500182f70 0x55d500185390 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.665+0000 7f01b6717b80 1 --2- 172.21.15.46:0/4085448799 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d50006b160 0x55d50011f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.665+0000 7f01897fa700 0 client.16413 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.665+0000 7f01b6717b80 1 -- 172.21.15.46:0/4085448799 >> 172.21.15.46:0/4085448799 conn(0x55d4fffe16a0 msgr2=0x55d500103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.665+0000 7f01b6717b80 1 -- 172.21.15.46:0/4085448799 shutdown_connections 2024-05-09T04:55:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.665+0000 7f01b6717b80 1 -- 172.21.15.46:0/4085448799 wait complete. 2024-05-09T04:55:03.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.665+0000 7f01b6717b80 20 client.16413 trim_cache size 0 max 16384 2024-05-09T04:55:03.671 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:55:03.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.666+0000 7f01a7d01700 1 -- 172.21.15.46:0/790165759 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f018c0b29f0 msgr2=0x7f018c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:03.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.666+0000 7f01a7d01700 1 --2- 172.21.15.46:0/790165759 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f018c0b29f0 0x7f018c0b4f00 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f019c009990 tx=0x7f019c000bc0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.666+0000 7f01a7d01700 1 -- 172.21.15.46:0/790165759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a0108f00 msgr2=0x7f01a0077110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:03.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.666+0000 7f01a7d01700 1 --2- 172.21.15.46:0/790165759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a0108f00 0x7f01a0077110 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f019000c8f0 tx=0x7f019000cc00 comp rx=0 tx=0).stop 2024-05-09T04:55:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.666+0000 7f01a7d01700 1 -- 172.21.15.46:0/790165759 shutdown_connections 2024-05-09T04:55:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.666+0000 7f01a7d01700 1 --2- 172.21.15.46:0/790165759 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f018c0b29f0 0x7f018c0b4f00 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.666+0000 7f01a7d01700 1 --2- 172.21.15.46:0/790165759 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f01a0109850 0x7f01a0077650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.666+0000 7f01a7d01700 1 --2- 172.21.15.46:0/790165759 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f01a01045d0 0x7f01a0076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.666+0000 7f01a7d01700 1 --2- 172.21.15.46:0/790165759 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f01a0108f00 0x7f01a0077110 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.666+0000 7f01a7d01700 1 -- 172.21.15.46:0/790165759 >> 172.21.15.46:0/790165759 conn(0x7f01a00ffea0 msgr2=0x7f01a010a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.666+0000 7f01a7d01700 1 -- 172.21.15.46:0/790165759 shutdown_connections 2024-05-09T04:55:03.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:03.666+0000 7f01a7d01700 1 -- 172.21.15.46:0/790165759 wait complete. 2024-05-09T04:55:05.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:04 smithi110 ceph-mon[34904]: pgmap v122: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-09T04:55:05.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:04 smithi046 ceph-mon[30136]: pgmap v122: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-09T04:55:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:04 smithi072 ceph-mon[34088]: pgmap v122: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-09T04:55:05.442 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:55:05.442 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:55:05.712 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:06.118 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:05 smithi046 ceph-mon[30136]: pgmap v123: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-05-09T04:55:06.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:05 smithi072 ceph-mon[34088]: pgmap v123: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-05-09T04:55:06.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:05 smithi110 ceph-mon[34904]: pgmap v123: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-05-09T04:55:07.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.542+0000 7ff396046700 1 -- 172.21.15.46:0/3125677686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff390109890 msgr2=0x7ff3901111f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:07.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.542+0000 7ff396046700 1 --2- 172.21.15.46:0/3125677686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff390109890 0x7ff3901111f0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7ff38800b0d0 tx=0x7ff38800b3e0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.542+0000 7ff396046700 1 -- 172.21.15.46:0/3125677686 shutdown_connections 2024-05-09T04:55:07.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.542+0000 7ff396046700 1 --2- 172.21.15.46:0/3125677686 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff390109890 0x7ff3901111f0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.542+0000 7ff396046700 1 --2- 172.21.15.46:0/3125677686 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff390102b20 0x7ff390102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.542+0000 7ff396046700 1 --2- 172.21.15.46:0/3125677686 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff390108f40 0x7ff390109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.542+0000 7ff396046700 1 -- 172.21.15.46:0/3125677686 >> 172.21.15.46:0/3125677686 conn(0x7ff3900fe540 msgr2=0x7ff390100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:07.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.542+0000 7ff396046700 1 -- 172.21.15.46:0/3125677686 shutdown_connections 2024-05-09T04:55:07.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.542+0000 7ff396046700 1 -- 172.21.15.46:0/3125677686 wait complete. 2024-05-09T04:55:07.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.542+0000 7ff396046700 1 Processor -- start 2024-05-09T04:55:07.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.543+0000 7ff396046700 1 -- start start 2024-05-09T04:55:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.543+0000 7ff396046700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff390102b20 0x7ff390103a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.543+0000 7ff396046700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff390108f40 0x7ff390103fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.543+0000 7ff396046700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff390109890 0x7ff3901044f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.543+0000 7ff396046700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff3901144d0 con 0x7ff390108f40 2024-05-09T04:55:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.543+0000 7ff396046700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff390114350 con 0x7ff390102b20 2024-05-09T04:55:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.543+0000 7ff396046700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff390114650 con 0x7ff390109890 2024-05-09T04:55:07.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.543+0000 7ff38effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff390108f40 0x7ff390103fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.543+0000 7ff38effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff390108f40 0x7ff390103fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57374/0 (socket says 172.21.15.46:57374) 2024-05-09T04:55:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.543+0000 7ff38effd700 1 -- 172.21.15.46:0/4167389670 learned_addr learned my addr 172.21.15.46:0/4167389670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.543+0000 7ff38ffff700 1 --2- 172.21.15.46:0/4167389670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff390109890 0x7ff3901044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.543+0000 7ff38f7fe700 1 --2- 172.21.15.46:0/4167389670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff390102b20 0x7ff390103a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.544+0000 7ff38effd700 1 -- 172.21.15.46:0/4167389670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff390109890 msgr2=0x7ff3901044f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.544+0000 7ff38effd700 1 --2- 172.21.15.46:0/4167389670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff390109890 0x7ff3901044f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.544+0000 7ff38effd700 1 -- 172.21.15.46:0/4167389670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff390102b20 msgr2=0x7ff390103a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.544+0000 7ff38effd700 1 --2- 172.21.15.46:0/4167389670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff390102b20 0x7ff390103a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.544+0000 7ff38effd700 1 -- 172.21.15.46:0/4167389670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3901a9070 con 0x7ff390108f40 2024-05-09T04:55:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.544+0000 7ff38effd700 1 --2- 172.21.15.46:0/4167389670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff390108f40 0x7ff390103fb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff38400c9e0 tx=0x7ff38400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.544+0000 7ff38cff9700 1 -- 172.21.15.46:0/4167389670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff384007aa0 con 0x7ff390108f40 2024-05-09T04:55:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.544+0000 7ff38cff9700 1 -- 172.21.15.46:0/4167389670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff384007c40 con 0x7ff390108f40 2024-05-09T04:55:07.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.544+0000 7ff396046700 1 -- 172.21.15.46:0/4167389670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3901a9360 con 0x7ff390108f40 2024-05-09T04:55:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.544+0000 7ff38cff9700 1 -- 172.21.15.46:0/4167389670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38400fe60 con 0x7ff390108f40 2024-05-09T04:55:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.544+0000 7ff396046700 1 -- 172.21.15.46:0/4167389670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3901a98a0 con 0x7ff390108f40 2024-05-09T04:55:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.547+0000 7ff38cff9700 1 -- 172.21.15.46:0/4167389670 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff384004870 con 0x7ff390108f40 2024-05-09T04:55:07.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.547+0000 7ff38cff9700 1 --2- 172.21.15.46:0/4167389670 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff3780b29f0 0x7ff3780b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff38f7fe700 1 --2- 172.21.15.46:0/4167389670 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff3780b29f0 0x7ff3780b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff38cff9700 1 -- 172.21.15.46:0/4167389670 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7ff3840e2530 con 0x7ff390108f40 2024-05-09T04:55:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff3a4a5cb80 1 Processor -- start 2024-05-09T04:55:07.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff3a4a5cb80 1 -- start start 2024-05-09T04:55:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff3a4a5cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adaf7d1e0 0x556adb016570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff3a4a5cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adae93520 0x556adb016ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff38f7fe700 1 --2- 172.21.15.46:0/4167389670 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff3780b29f0 0x7ff3780b4f00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff39006bc90 tx=0x7ff380009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff3a4a5cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adb010720 0x556adb010b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff3a4a5cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556adaf33070 con 0x556adb010720 2024-05-09T04:55:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff3a4a5cb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556adaed1870 con 0x556adae93520 2024-05-09T04:55:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff3a4a5cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556adaf2f030 con 0x556adaf7d1e0 2024-05-09T04:55:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff38ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adb010720 0x556adb010b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff38ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adb010720 0x556adb010b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57386/0 (socket says 172.21.15.46:57386) 2024-05-09T04:55:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff38ffff700 1 -- 172.21.15.46:0/272258456 learned_addr learned my addr 172.21.15.46:0/272258456 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff38effd700 1 --2- 172.21.15.46:0/272258456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adae93520 0x556adb016ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.548+0000 7ff38f7fe700 1 --2- 172.21.15.46:0/272258456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adaf7d1e0 0x556adb016570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.549+0000 7ff38ffff700 1 -- 172.21.15.46:0/272258456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adaf7d1e0 msgr2=0x556adb016570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.549+0000 7ff38ffff700 1 --2- 172.21.15.46:0/272258456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adaf7d1e0 0x556adb016570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.549+0000 7ff38ffff700 1 -- 172.21.15.46:0/272258456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adae93520 msgr2=0x556adb016ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.549+0000 7ff38ffff700 1 --2- 172.21.15.46:0/272258456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adae93520 0x556adb016ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.549+0000 7ff38ffff700 1 -- 172.21.15.46:0/272258456 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556adaec93f0 con 0x556adb010720 2024-05-09T04:55:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.549+0000 7ff38ffff700 1 --2- 172.21.15.46:0/272258456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adb010720 0x556adb010b50 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff388009cb0 tx=0x7ff388009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4470fda39aacee5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.549+0000 7ff375ffb700 1 -- 172.21.15.46:0/272258456 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38800baa0 con 0x556adb010720 2024-05-09T04:55:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.549+0000 7ff375ffb700 1 -- 172.21.15.46:0/272258456 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff388020070 con 0x556adb010720 2024-05-09T04:55:07.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.549+0000 7ff375ffb700 1 -- 172.21.15.46:0/272258456 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3880054d0 con 0x556adb010720 2024-05-09T04:55:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.550+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/272258456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adb010720 msgr2=0x556adb010b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.550+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/272258456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adb010720 0x556adb010b50 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff388009cb0 tx=0x7ff388009ce0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.550+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/272258456 shutdown_connections 2024-05-09T04:55:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.550+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/272258456 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adb010720 0x556adb010b50 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.550+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/272258456 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adaf7d1e0 0x556adb016570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.550+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/272258456 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adae93520 0x556adb016ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.550+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/272258456 >> 172.21.15.46:0/272258456 conn(0x556adaef3720 msgr2=0x556adb00b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.550+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/272258456 shutdown_connections 2024-05-09T04:55:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.550+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/272258456 wait complete. 2024-05-09T04:55:07.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.551+0000 7ff3a4a5cb80 1 -- start start 2024-05-09T04:55:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.551+0000 7ff3a4a5cb80 1 Processor -- start 2024-05-09T04:55:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.551+0000 7ff3a4a5cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adae93520 0x556adb01a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.551+0000 7ff3a4a5cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adaf7d1e0 0x556adb01a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.551+0000 7ff3a4a5cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adb010720 0x556adb01ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.551+0000 7ff3a4a5cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556adadaec20 con 0x556adae93520 2024-05-09T04:55:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.551+0000 7ff3a4a5cb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556adaeed1d0 con 0x556adaf7d1e0 2024-05-09T04:55:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.551+0000 7ff3a4a5cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556adb01d250 con 0x556adb010720 2024-05-09T04:55:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.551+0000 7ff3a4a5cb80 10 client.0 _ll_register_callbacks cb 0x556adada4740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.551+0000 7ff38f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adae93520 0x556adb01a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.551+0000 7ff38f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adae93520 0x556adb01a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57390/0 (socket says 172.21.15.46:57390) 2024-05-09T04:55:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.551+0000 7ff38f7fe700 1 -- 172.21.15.46:0/1263097635 learned_addr learned my addr 172.21.15.46:0/1263097635 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.551+0000 7ff38effd700 1 --2- 172.21.15.46:0/1263097635 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adaf7d1e0 0x556adb01a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.551+0000 7ff38ffff700 1 --2- 172.21.15.46:0/1263097635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adb010720 0x556adb01ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.552+0000 7ff38f7fe700 1 -- 172.21.15.46:0/1263097635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adb010720 msgr2=0x556adb01ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:07.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.552+0000 7ff38f7fe700 1 --2- 172.21.15.46:0/1263097635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adb010720 0x556adb01ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.552+0000 7ff38f7fe700 1 -- 172.21.15.46:0/1263097635 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adaf7d1e0 msgr2=0x556adb01a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.552+0000 7ff38f7fe700 1 --2- 172.21.15.46:0/1263097635 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adaf7d1e0 0x556adb01a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.552+0000 7ff38f7fe700 1 -- 172.21.15.46:0/1263097635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556adadae510 con 0x556adae93520 2024-05-09T04:55:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.552+0000 7ff38f7fe700 1 --2- 172.21.15.46:0/1263097635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adae93520 0x556adb01a2b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff38001d8f0 tx=0x7ff3800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5da3f6ca78b05caf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.552+0000 7ff36effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.552+0000 7ff36effd700 1 -- 172.21.15.46:0/1263097635 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380004020 con 0x556adae93520 2024-05-09T04:55:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.552+0000 7ff36effd700 1 -- 172.21.15.46:0/1263097635 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7ff380009040 con 0x556adae93520 2024-05-09T04:55:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.552+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1263097635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff388020070 con 0x556adae93520 2024-05-09T04:55:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.552+0000 7ff36effd700 1 -- 172.21.15.46:0/1263097635 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff3800041c0 con 0x556adae93520 2024-05-09T04:55:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.552+0000 7ff3a4a5cb80 10 client.16425 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.552+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1263097635 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556adb027c00 con 0x556adae93520 2024-05-09T04:55:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.552+0000 7ff36effd700 1 -- 172.21.15.46:0/1263097635 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff388020070 con 0x556adae93520 2024-05-09T04:55:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.553+0000 7ff36effd700 1 -- 172.21.15.46:0/1263097635 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff38000e6d0 con 0x556adae93520 2024-05-09T04:55:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.553+0000 7ff3a4a5cb80 10 client.16425 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:07.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.553+0000 7ff3a4a5cb80 10 client.16425 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:55:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.553+0000 7ff3a4a5cb80 20 client.16425 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.553+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/1263097635 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556adb02a290 0x556adb02c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.553+0000 7ff3a4a5cb80 4 client.16425 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:55:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.553+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1263097635 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556adaf40a50 con 0x556adb02a290 2024-05-09T04:55:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.553+0000 7ff38effd700 1 --2- 172.21.15.46:0/1263097635 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556adb02a290 0x556adb02c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.554+0000 7ff36effd700 10 client.16425 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.554+0000 7ff38effd700 1 --2- 172.21.15.46:0/1263097635 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556adb02a290 0x556adb02c6f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11122380b76957ce server_cookie=ab3ccb461b9a5112 in_seq=0 out_seq=0 2024-05-09T04:55:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.554+0000 7ff36effd700 1 -- 172.21.15.46:0/1263097635 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556adaf40a50 con 0x556adb02a290 2024-05-09T04:55:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.554+0000 7ff36effd700 10 client.16425 handle_command_reply: tid=0 2024-05-09T04:55:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.554+0000 7ff3a4a5cb80 1 client.16425 shutdown 2024-05-09T04:55:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.555+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1263097635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adae93520 msgr2=0x556adb01a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.555+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/1263097635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adae93520 0x556adb01a2b0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff38001d8f0 tx=0x7ff3800047c0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.555+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1263097635 shutdown_connections 2024-05-09T04:55:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.555+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/1263097635 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adb010720 0x556adb01ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.555+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/1263097635 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adae93520 0x556adb01a2b0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.555+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/1263097635 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556adb02a290 0x556adb02c6f0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.555+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/1263097635 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adaf7d1e0 0x556adb01a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.555+0000 7ff36effd700 0 client.16425 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.555+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1263097635 >> 172.21.15.46:0/1263097635 conn(0x556adaef3720 msgr2=0x556adb00b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.555+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1263097635 shutdown_connections 2024-05-09T04:55:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.555+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1263097635 wait complete. 2024-05-09T04:55:07.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.555+0000 7ff3a4a5cb80 20 client.16425 trim_cache size 0 max 16384 2024-05-09T04:55:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff3a4a5cb80 1 Processor -- start 2024-05-09T04:55:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff3a4a5cb80 1 -- start start 2024-05-09T04:55:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff3a4a5cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adae93520 0x556adb014840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff3a4a5cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adaf7d1e0 0x556adb010140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff3a4a5cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adb010720 0x556adb019460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff3a4a5cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff3901b3800 con 0x556adae93520 2024-05-09T04:55:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff3a4a5cb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff3901b1090 con 0x556adaf7d1e0 2024-05-09T04:55:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff3a4a5cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff3901aeee0 con 0x556adb010720 2024-05-09T04:55:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff38f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adae93520 0x556adb014840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff38f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adae93520 0x556adb014840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57394/0 (socket says 172.21.15.46:57394) 2024-05-09T04:55:07.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff38f7fe700 1 -- 172.21.15.46:0/4084312587 learned_addr learned my addr 172.21.15.46:0/4084312587 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:07.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff38effd700 1 --2- 172.21.15.46:0/4084312587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adaf7d1e0 0x556adb010140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff38f7fe700 1 -- 172.21.15.46:0/4084312587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adb010720 msgr2=0x556adb019460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:07.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff38f7fe700 1 --2- 172.21.15.46:0/4084312587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adb010720 0x556adb019460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff38f7fe700 1 -- 172.21.15.46:0/4084312587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adaf7d1e0 msgr2=0x556adb010140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:07.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff38f7fe700 1 --2- 172.21.15.46:0/4084312587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adaf7d1e0 0x556adb010140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.576+0000 7ff38f7fe700 1 -- 172.21.15.46:0/4084312587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556adb004fe0 con 0x556adae93520 2024-05-09T04:55:07.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.577+0000 7ff38f7fe700 1 --2- 172.21.15.46:0/4084312587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adae93520 0x556adb014840 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff380009b20 tx=0x7ff380005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ed67d9f0c7e14 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:07.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.577+0000 7ff3757fa700 1 -- 172.21.15.46:0/4084312587 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380034030 con 0x556adae93520 2024-05-09T04:55:07.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.577+0000 7ff3757fa700 1 -- 172.21.15.46:0/4084312587 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff38000ed50 con 0x556adae93520 2024-05-09T04:55:07.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.577+0000 7ff3757fa700 1 -- 172.21.15.46:0/4084312587 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff380003b30 con 0x556adae93520 2024-05-09T04:55:07.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.578+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/4084312587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adae93520 msgr2=0x556adb014840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:07.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.578+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/4084312587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adae93520 0x556adb014840 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7ff380009b20 tx=0x7ff380005880 comp rx=0 tx=0).stop 2024-05-09T04:55:07.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.578+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/4084312587 shutdown_connections 2024-05-09T04:55:07.582 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.578+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/4084312587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adb010720 0x556adb019460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.578+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/4084312587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adae93520 0x556adb014840 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.578+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/4084312587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adaf7d1e0 0x556adb010140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.578+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/4084312587 >> 172.21.15.46:0/4084312587 conn(0x556adaef3720 msgr2=0x556adb00a960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.578+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/4084312587 shutdown_connections 2024-05-09T04:55:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.578+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/4084312587 wait complete. 2024-05-09T04:55:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.578+0000 7ff3a4a5cb80 1 -- start start 2024-05-09T04:55:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.578+0000 7ff3a4a5cb80 1 Processor -- start 2024-05-09T04:55:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff3a4a5cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adae93520 0x556adb0222a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff3a4a5cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adaf7d1e0 0x556adb0227e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff3a4a5cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adb010720 0x556adb01d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff3a4a5cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff3901af550 con 0x556adaf7d1e0 2024-05-09T04:55:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff3a4a5cb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff3901b09c0 con 0x556adae93520 2024-05-09T04:55:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff3a4a5cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff3901afc20 con 0x556adb010720 2024-05-09T04:55:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff3a4a5cb80 10 client.0 _ll_register_callbacks cb 0x556adada4740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:07.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff38effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adaf7d1e0 0x556adb0227e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff38effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adaf7d1e0 0x556adb0227e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57406/0 (socket says 172.21.15.46:57406) 2024-05-09T04:55:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff38effd700 1 -- 172.21.15.46:0/1229801844 learned_addr learned my addr 172.21.15.46:0/1229801844 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff38f7fe700 1 --2- 172.21.15.46:0/1229801844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adae93520 0x556adb0222a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff38ffff700 1 --2- 172.21.15.46:0/1229801844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adb010720 0x556adb01d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff38effd700 1 -- 172.21.15.46:0/1229801844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adb010720 msgr2=0x556adb01d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff38effd700 1 --2- 172.21.15.46:0/1229801844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adb010720 0x556adb01d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff38effd700 1 -- 172.21.15.46:0/1229801844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adae93520 msgr2=0x556adb0222a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff38effd700 1 --2- 172.21.15.46:0/1229801844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adae93520 0x556adb0222a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff38effd700 1 -- 172.21.15.46:0/1229801844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556adb022e50 con 0x556adaf7d1e0 2024-05-09T04:55:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff38ffff700 1 --2- 172.21.15.46:0/1229801844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adb010720 0x556adb01d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:55:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff38effd700 1 --2- 172.21.15.46:0/1229801844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adaf7d1e0 0x556adb0227e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff3840a6ad0 tx=0x7ff3840cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73a7e99e5b5363f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff36effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff36effd700 1 -- 172.21.15.46:0/1229801844 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff38408fe60 con 0x556adaf7d1e0 2024-05-09T04:55:07.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff36effd700 1 -- 172.21.15.46:0/1229801844 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7ff3840b21c0 con 0x556adaf7d1e0 2024-05-09T04:55:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.579+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1229801844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff38000ed50 con 0x556adaf7d1e0 2024-05-09T04:55:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.580+0000 7ff36effd700 1 -- 172.21.15.46:0/1229801844 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff3840a9ef0 con 0x556adaf7d1e0 2024-05-09T04:55:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.580+0000 7ff3a4a5cb80 10 client.16431 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.580+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1229801844 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556adb095050 con 0x556adaf7d1e0 2024-05-09T04:55:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.580+0000 7ff36effd700 1 -- 172.21.15.46:0/1229801844 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff38000ed50 con 0x556adaf7d1e0 2024-05-09T04:55:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.580+0000 7ff36effd700 1 -- 172.21.15.46:0/1229801844 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff3840c8030 con 0x556adaf7d1e0 2024-05-09T04:55:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.580+0000 7ff3a4a5cb80 10 client.16431 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.580+0000 7ff3a4a5cb80 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:55:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.580+0000 7ff3a4a5cb80 20 client.16431 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.580+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/1229801844 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556adb095710 0x556adb097b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.580+0000 7ff3a4a5cb80 4 client.16431 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:55:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.580+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1229801844 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556adaf3aaf0 con 0x556adb095710 2024-05-09T04:55:07.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.580+0000 7ff38f7fe700 1 --2- 172.21.15.46:0/1229801844 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556adb095710 0x556adb097b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:07.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.581+0000 7ff38f7fe700 1 --2- 172.21.15.46:0/1229801844 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556adb095710 0x556adb097b30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=92f2b4acac9ee1a3 server_cookie=92675a54b51bbba4 in_seq=0 out_seq=0 2024-05-09T04:55:07.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.581+0000 7ff36effd700 10 client.16431 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.582+0000 7ff36effd700 1 -- 172.21.15.46:0/1229801844 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556adaf3aaf0 con 0x556adb095710 2024-05-09T04:55:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.582+0000 7ff36effd700 10 client.16431 handle_command_reply: tid=0 2024-05-09T04:55:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.582+0000 7ff3a4a5cb80 1 client.16431 shutdown 2024-05-09T04:55:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.582+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1229801844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adaf7d1e0 msgr2=0x556adb0227e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.582+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/1229801844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adaf7d1e0 0x556adb0227e0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7ff3840a6ad0 tx=0x7ff3840cc040 comp rx=0 tx=0).stop 2024-05-09T04:55:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.582+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1229801844 shutdown_connections 2024-05-09T04:55:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.582+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/1229801844 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556adb010720 0x556adb01d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.582+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/1229801844 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556adb095710 0x556adb097b30 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/1229801844 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556adae93520 0x556adb0222a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff3a4a5cb80 1 --2- 172.21.15.46:0/1229801844 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556adaf7d1e0 0x556adb0227e0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff36effd700 0 client.16431 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1229801844 >> 172.21.15.46:0/1229801844 conn(0x556adaef3720 msgr2=0x556adb00b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:07.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1229801844 shutdown_connections 2024-05-09T04:55:07.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff3a4a5cb80 1 -- 172.21.15.46:0/1229801844 wait complete. 2024-05-09T04:55:07.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff3a4a5cb80 20 client.16431 trim_cache size 0 max 16384 2024-05-09T04:55:07.588 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:55:07.588 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:55:07.588 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "82a5fb59-3325-40b5-bd29-397fe40b478c", 2024-05-09T04:55:07.588 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:55:07.588 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:55:07.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff396046700 1 -- 172.21.15.46:0/4167389670 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff3780b29f0 msgr2=0x7ff3780b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:07.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff396046700 1 --2- 172.21.15.46:0/4167389670 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff3780b29f0 0x7ff3780b4f00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff39006bc90 tx=0x7ff380009260 comp rx=0 tx=0).stop 2024-05-09T04:55:07.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff396046700 1 -- 172.21.15.46:0/4167389670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff390108f40 msgr2=0x7ff390103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:07.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff396046700 1 --2- 172.21.15.46:0/4167389670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff390108f40 0x7ff390103fb0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff38400c9e0 tx=0x7ff38400ccf0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff396046700 1 -- 172.21.15.46:0/4167389670 shutdown_connections 2024-05-09T04:55:07.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff396046700 1 --2- 172.21.15.46:0/4167389670 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff3780b29f0 0x7ff3780b4f00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff396046700 1 --2- 172.21.15.46:0/4167389670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff390109890 0x7ff3901044f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff396046700 1 --2- 172.21.15.46:0/4167389670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff390102b20 0x7ff390103a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff396046700 1 --2- 172.21.15.46:0/4167389670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff390108f40 0x7ff390103fb0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:07.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.583+0000 7ff396046700 1 -- 172.21.15.46:0/4167389670 >> 172.21.15.46:0/4167389670 conn(0x7ff3900fe540 msgr2=0x7ff3901086b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:07.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.584+0000 7ff396046700 1 -- 172.21.15.46:0/4167389670 shutdown_connections 2024-05-09T04:55:07.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:07.584+0000 7ff396046700 1 -- 172.21.15.46:0/4167389670 wait complete. 2024-05-09T04:55:08.380 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:55:08.637 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:09.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:08 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:55:09.028 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:08 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:55:09.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:08 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:55:09.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:08 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:55:09.029 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:08 smithi046 ceph-mon[30136]: pgmap v124: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-09T04:55:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:08 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:55:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:08 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:55:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:08 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:55:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:08 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:55:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:08 smithi110 ceph-mon[34904]: pgmap v124: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-09T04:55:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:08 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:55:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:08 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:55:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:08 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:55:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:08 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:55:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:08 smithi072 ceph-mon[34088]: pgmap v124: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-09T04:55:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.340+0000 7fbdd2b2b700 1 -- 172.21.15.46:0/404570138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbdcc108760 msgr2=0x7fbdcc1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.340+0000 7fbdd2b2b700 1 --2- 172.21.15.46:0/404570138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbdcc108760 0x7fbdcc1100c0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fbdc400b0d0 tx=0x7fbdc400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.340+0000 7fbdd2b2b700 1 -- 172.21.15.46:0/404570138 shutdown_connections 2024-05-09T04:55:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.340+0000 7fbdd2b2b700 1 --2- 172.21.15.46:0/404570138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbdcc108760 0x7fbdcc1100c0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.340+0000 7fbdd2b2b700 1 --2- 172.21.15.46:0/404570138 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdcc1034e0 0x7fbdcc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.340+0000 7fbdd2b2b700 1 --2- 172.21.15.46:0/404570138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdcc107e10 0x7fbdcc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.340+0000 7fbdd2b2b700 1 -- 172.21.15.46:0/404570138 >> 172.21.15.46:0/404570138 conn(0x7fbdcc0fef30 msgr2=0x7fbdcc101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:10.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.340+0000 7fbdd2b2b700 1 -- 172.21.15.46:0/404570138 shutdown_connections 2024-05-09T04:55:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.341+0000 7fbdd2b2b700 1 -- 172.21.15.46:0/404570138 wait complete. 2024-05-09T04:55:10.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd2b2b700 1 Processor -- start 2024-05-09T04:55:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd2b2b700 1 -- start start 2024-05-09T04:55:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd2b2b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdcc1034e0 0x7fbdcc19ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd2b2b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdcc107e10 0x7fbdcc19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd2b2b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbdcc108760 0x7fbdcc19f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd2b2b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc1133b0 con 0x7fbdcc108760 2024-05-09T04:55:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd2b2b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc113230 con 0x7fbdcc107e10 2024-05-09T04:55:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd2b2b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc113530 con 0x7fbdcc1034e0 2024-05-09T04:55:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd10c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbdcc108760 0x7fbdcc19f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd10c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbdcc108760 0x7fbdcc19f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57436/0 (socket says 172.21.15.46:57436) 2024-05-09T04:55:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd10c8700 1 -- 172.21.15.46:0/3545731443 learned_addr learned my addr 172.21.15.46:0/3545731443 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd08c7700 1 --2- 172.21.15.46:0/3545731443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdcc1034e0 0x7fbdcc19ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdcbfff700 1 --2- 172.21.15.46:0/3545731443 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdcc107e10 0x7fbdcc19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd10c8700 1 -- 172.21.15.46:0/3545731443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdcc1034e0 msgr2=0x7fbdcc19ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd10c8700 1 --2- 172.21.15.46:0/3545731443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdcc1034e0 0x7fbdcc19ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd10c8700 1 -- 172.21.15.46:0/3545731443 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdcc107e10 msgr2=0x7fbdcc19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd10c8700 1 --2- 172.21.15.46:0/3545731443 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdcc107e10 0x7fbdcc19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.342+0000 7fbdd10c8700 1 -- 172.21.15.46:0/3545731443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbdcc19fce0 con 0x7fbdcc108760 2024-05-09T04:55:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.343+0000 7fbdcbfff700 1 --2- 172.21.15.46:0/3545731443 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdcc107e10 0x7fbdcc19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.343+0000 7fbdd10c8700 1 --2- 172.21.15.46:0/3545731443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbdcc108760 0x7fbdcc19f640 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40095c0 tx=0x7fbdc4007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.343+0000 7fbdc9ffb700 1 -- 172.21.15.46:0/3545731443 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc401f070 con 0x7fbdcc108760 2024-05-09T04:55:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.343+0000 7fbdd2b2b700 1 -- 172.21.15.46:0/3545731443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbdcc19ff70 con 0x7fbdcc108760 2024-05-09T04:55:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.343+0000 7fbdc9ffb700 1 -- 172.21.15.46:0/3545731443 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc4003c00 con 0x7fbdcc108760 2024-05-09T04:55:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.343+0000 7fbdc9ffb700 1 -- 172.21.15.46:0/3545731443 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc401b6a0 con 0x7fbdcc108760 2024-05-09T04:55:10.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.343+0000 7fbdd2b2b700 1 -- 172.21.15.46:0/3545731443 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbdcc1a78f0 con 0x7fbdcc108760 2024-05-09T04:55:10.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.346+0000 7fbdc9ffb700 1 -- 172.21.15.46:0/3545731443 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbdc4022070 con 0x7fbdcc108760 2024-05-09T04:55:10.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.347+0000 7fbdc9ffb700 1 --2- 172.21.15.46:0/3545731443 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbdbc0b29f0 0x7fbdbc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.347+0000 7fbdd08c7700 1 --2- 172.21.15.46:0/3545731443 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbdbc0b29f0 0x7fbdbc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.347+0000 7fbdc9ffb700 1 -- 172.21.15.46:0/3545731443 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fbdc400f020 con 0x7fbdcc108760 2024-05-09T04:55:10.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.347+0000 7fbde1541b80 1 Processor -- start 2024-05-09T04:55:10.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.347+0000 7fbde1541b80 1 -- start start 2024-05-09T04:55:10.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.347+0000 7fbdd08c7700 1 --2- 172.21.15.46:0/3545731443 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbdbc0b29f0 0x7fbdbc0b4f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0005950 tx=0x7fbdc000b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:10.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbde1541b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbde1541b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe7f224a0 0x55cfe80a5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbde1541b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe809f6a0 0x55cfe809fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbde1541b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cfe7fc1ff0 con 0x55cfe800c160 2024-05-09T04:55:10.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbde1541b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55cfe7f607f0 con 0x55cfe809f6a0 2024-05-09T04:55:10.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbde1541b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cfe7fbdfb0 con 0x55cfe7f224a0 2024-05-09T04:55:10.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbdd08c7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbdd08c7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57442/0 (socket says 172.21.15.46:57442) 2024-05-09T04:55:10.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbdd08c7700 1 -- 172.21.15.46:0/1745904557 learned_addr learned my addr 172.21.15.46:0/1745904557 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:10.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbdcbfff700 1 --2- 172.21.15.46:0/1745904557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe7f224a0 0x55cfe80a5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbdd10c8700 1 --2- 172.21.15.46:0/1745904557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe809f6a0 0x55cfe809fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbdd08c7700 1 -- 172.21.15.46:0/1745904557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe7f224a0 msgr2=0x55cfe80a5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:10.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbdd08c7700 1 --2- 172.21.15.46:0/1745904557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe7f224a0 0x55cfe80a5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbdd08c7700 1 -- 172.21.15.46:0/1745904557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe809f6a0 msgr2=0x55cfe809fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:10.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbdd08c7700 1 --2- 172.21.15.46:0/1745904557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe809f6a0 0x55cfe809fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbdd08c7700 1 -- 172.21.15.46:0/1745904557 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfe7f58370 con 0x55cfe800c160 2024-05-09T04:55:10.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbdcbfff700 1 --2- 172.21.15.46:0/1745904557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe7f224a0 0x55cfe80a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:10.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.348+0000 7fbdd08c7700 1 --2- 172.21.15.46:0/1745904557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a54f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0010fa0 tx=0x7fbdc0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7375f986becbf4b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:10.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.349+0000 7fbdb6ffd700 1 -- 172.21.15.46:0/1745904557 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc000be60 con 0x55cfe800c160 2024-05-09T04:55:10.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.349+0000 7fbdb6ffd700 1 -- 172.21.15.46:0/1745904557 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc0011db0 con 0x55cfe800c160 2024-05-09T04:55:10.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.349+0000 7fbdb6ffd700 1 -- 172.21.15.46:0/1745904557 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc001c700 con 0x55cfe800c160 2024-05-09T04:55:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.350+0000 7fbde1541b80 1 -- 172.21.15.46:0/1745904557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 msgr2=0x55cfe80a54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.350+0000 7fbde1541b80 1 --2- 172.21.15.46:0/1745904557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a54f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0010fa0 tx=0x7fbdc0013000 comp rx=0 tx=0).stop 2024-05-09T04:55:10.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.350+0000 7fbde1541b80 1 -- 172.21.15.46:0/1745904557 shutdown_connections 2024-05-09T04:55:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.350+0000 7fbde1541b80 1 --2- 172.21.15.46:0/1745904557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe809f6a0 0x55cfe809fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.350+0000 7fbde1541b80 1 --2- 172.21.15.46:0/1745904557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a54f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.350+0000 7fbde1541b80 1 --2- 172.21.15.46:0/1745904557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe7f224a0 0x55cfe80a5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.350+0000 7fbde1541b80 1 -- 172.21.15.46:0/1745904557 >> 172.21.15.46:0/1745904557 conn(0x55cfe7f826a0 msgr2=0x55cfe809a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.350+0000 7fbde1541b80 1 -- 172.21.15.46:0/1745904557 shutdown_connections 2024-05-09T04:55:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.350+0000 7fbde1541b80 1 -- 172.21.15.46:0/1745904557 wait complete. 2024-05-09T04:55:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.350+0000 7fbde1541b80 1 -- start start 2024-05-09T04:55:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.350+0000 7fbde1541b80 1 Processor -- start 2024-05-09T04:55:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbde1541b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe7f224a0 0x55cfe80a9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbde1541b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbde1541b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe809f6a0 0x55cfe80a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbde1541b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55cfe7e3dba0 con 0x55cfe800c160 2024-05-09T04:55:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbde1541b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55cfe7f7c150 con 0x55cfe7f224a0 2024-05-09T04:55:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbde1541b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55cfe80ac1d0 con 0x55cfe809f6a0 2024-05-09T04:55:10.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbde1541b80 10 client.0 _ll_register_callbacks cb 0x55cfe7e336c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbdcbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbdcbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a9770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57456/0 (socket says 172.21.15.46:57456) 2024-05-09T04:55:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbdd08c7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe7f224a0 0x55cfe80a9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbdcbfff700 1 -- 172.21.15.46:0/2200149614 learned_addr learned my addr 172.21.15.46:0/2200149614 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:10.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbdd10c8700 1 --2- 172.21.15.46:0/2200149614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe809f6a0 0x55cfe80a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbdcbfff700 1 -- 172.21.15.46:0/2200149614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe809f6a0 msgr2=0x55cfe80a9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbdcbfff700 1 --2- 172.21.15.46:0/2200149614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe809f6a0 0x55cfe80a9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbdcbfff700 1 -- 172.21.15.46:0/2200149614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe7f224a0 msgr2=0x55cfe80a9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbdcbfff700 1 --2- 172.21.15.46:0/2200149614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe7f224a0 0x55cfe80a9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbdcbfff700 1 -- 172.21.15.46:0/2200149614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfe7e3d490 con 0x55cfe800c160 2024-05-09T04:55:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbdd08c7700 1 --2- 172.21.15.46:0/2200149614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe7f224a0 0x55cfe80a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbdcbfff700 1 --2- 172.21.15.46:0/2200149614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a9770 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8012730 tx=0x7fbdb8012a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2742a842755f564b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.351+0000 7fbdabfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.352+0000 7fbdabfff700 1 -- 172.21.15.46:0/2200149614 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8004a30 con 0x55cfe800c160 2024-05-09T04:55:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.352+0000 7fbdabfff700 1 -- 172.21.15.46:0/2200149614 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fbdb8004bd0 con 0x55cfe800c160 2024-05-09T04:55:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.352+0000 7fbde1541b80 1 -- 172.21.15.46:0/2200149614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc0011db0 con 0x55cfe800c160 2024-05-09T04:55:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.352+0000 7fbdabfff700 1 -- 172.21.15.46:0/2200149614 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbdb8004dd0 con 0x55cfe800c160 2024-05-09T04:55:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.352+0000 7fbde1541b80 10 client.16443 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:10.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.352+0000 7fbde1541b80 1 -- 172.21.15.46:0/2200149614 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cfe80b6c70 con 0x55cfe800c160 2024-05-09T04:55:10.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.352+0000 7fbdabfff700 1 -- 172.21.15.46:0/2200149614 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc0011db0 con 0x55cfe800c160 2024-05-09T04:55:10.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.352+0000 7fbdabfff700 1 -- 172.21.15.46:0/2200149614 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fbdb8019030 con 0x55cfe800c160 2024-05-09T04:55:10.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.352+0000 7fbde1541b80 10 client.16443 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:10.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.352+0000 7fbde1541b80 10 client.16443 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:55:10.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.352+0000 7fbde1541b80 20 client.16443 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:10.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.352+0000 7fbde1541b80 1 --2- 172.21.15.46:0/2200149614 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55cfe80b9300 0x55cfe80bb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.352+0000 7fbde1541b80 4 client.16443 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:55:10.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.352+0000 7fbde1541b80 1 -- 172.21.15.46:0/2200149614 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfe7fcf9d0 con 0x55cfe80b9300 2024-05-09T04:55:10.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.353+0000 7fbdd08c7700 1 --2- 172.21.15.46:0/2200149614 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55cfe80b9300 0x55cfe80bb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.353+0000 7fbdd08c7700 1 --2- 172.21.15.46:0/2200149614 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55cfe80b9300 0x55cfe80bb7b0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3afa0822feaa75 server_cookie=4dd64dcea9c47fa1 in_seq=0 out_seq=0 2024-05-09T04:55:10.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.353+0000 7fbdabfff700 10 client.16443 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.354+0000 7fbdabfff700 1 -- 172.21.15.46:0/2200149614 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cfe7fcf9d0 con 0x55cfe80b9300 2024-05-09T04:55:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.354+0000 7fbdabfff700 10 client.16443 handle_command_reply: tid=0 2024-05-09T04:55:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.354+0000 7fbde1541b80 1 client.16443 shutdown 2024-05-09T04:55:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.354+0000 7fbde1541b80 1 -- 172.21.15.46:0/2200149614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 msgr2=0x55cfe80a9770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.354+0000 7fbde1541b80 1 --2- 172.21.15.46:0/2200149614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a9770 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8012730 tx=0x7fbdb8012a40 comp rx=0 tx=0).stop 2024-05-09T04:55:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.354+0000 7fbde1541b80 1 -- 172.21.15.46:0/2200149614 shutdown_connections 2024-05-09T04:55:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.354+0000 7fbde1541b80 1 --2- 172.21.15.46:0/2200149614 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe809f6a0 0x55cfe80a9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.354+0000 7fbde1541b80 1 --2- 172.21.15.46:0/2200149614 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55cfe80b9300 0x55cfe80bb7b0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.354+0000 7fbde1541b80 1 --2- 172.21.15.46:0/2200149614 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe7f224a0 0x55cfe80a9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.354+0000 7fbde1541b80 1 --2- 172.21.15.46:0/2200149614 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a9770 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.354+0000 7fbde1541b80 1 -- 172.21.15.46:0/2200149614 >> 172.21.15.46:0/2200149614 conn(0x55cfe7f826a0 msgr2=0x55cfe809a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.354+0000 7fbdabfff700 0 client.16443 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.354+0000 7fbde1541b80 1 -- 172.21.15.46:0/2200149614 shutdown_connections 2024-05-09T04:55:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.354+0000 7fbde1541b80 1 -- 172.21.15.46:0/2200149614 wait complete. 2024-05-09T04:55:10.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.354+0000 7fbde1541b80 20 client.16443 trim_cache size 0 max 16384 2024-05-09T04:55:10.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.375+0000 7fbde1541b80 1 Processor -- start 2024-05-09T04:55:10.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.375+0000 7fbde1541b80 1 -- start start 2024-05-09T04:55:10.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.375+0000 7fbde1541b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe7f224a0 0x55cfe80a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.375+0000 7fbde1541b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe800c160 0x55cfe80a3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.375+0000 7fbde1541b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe809f6a0 0x55cfe80c0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.375+0000 7fbde1541b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc1b17b0 con 0x55cfe809f6a0 2024-05-09T04:55:10.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.375+0000 7fbde1541b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc1af040 con 0x55cfe7f224a0 2024-05-09T04:55:10.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.375+0000 7fbde1541b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc1ace90 con 0x55cfe800c160 2024-05-09T04:55:10.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.375+0000 7fbdd10c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe809f6a0 0x55cfe80c0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.375+0000 7fbdd10c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe809f6a0 0x55cfe80c0f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57458/0 (socket says 172.21.15.46:57458) 2024-05-09T04:55:10.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.375+0000 7fbdd10c8700 1 -- 172.21.15.46:0/1448500773 learned_addr learned my addr 172.21.15.46:0/1448500773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:10.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.376+0000 7fbdcbfff700 1 --2- 172.21.15.46:0/1448500773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe800c160 0x55cfe80a3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.376+0000 7fbdd08c7700 1 --2- 172.21.15.46:0/1448500773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe7f224a0 0x55cfe80a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.376+0000 7fbdd10c8700 1 -- 172.21.15.46:0/1448500773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe800c160 msgr2=0x55cfe80a3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:10.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.376+0000 7fbdd10c8700 1 --2- 172.21.15.46:0/1448500773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe800c160 0x55cfe80a3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.376+0000 7fbdd10c8700 1 -- 172.21.15.46:0/1448500773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe7f224a0 msgr2=0x55cfe80a9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:10.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.376+0000 7fbdd10c8700 1 --2- 172.21.15.46:0/1448500773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe7f224a0 0x55cfe80a9c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.376+0000 7fbdd10c8700 1 -- 172.21.15.46:0/1448500773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfe80a4d80 con 0x55cfe809f6a0 2024-05-09T04:55:10.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.376+0000 7fbdd10c8700 1 --2- 172.21.15.46:0/1448500773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe809f6a0 0x55cfe80c0f70 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009fd0 tx=0x7fbdc40099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d4ba65b4e00a14e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:10.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.376+0000 7fbdb67fc700 1 -- 172.21.15.46:0/1448500773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc40b1040 con 0x55cfe809f6a0 2024-05-09T04:55:10.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.376+0000 7fbdb67fc700 1 -- 172.21.15.46:0/1448500773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc40cbe50 con 0x55cfe809f6a0 2024-05-09T04:55:10.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.376+0000 7fbdb67fc700 1 -- 172.21.15.46:0/1448500773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdc40aefe0 con 0x55cfe809f6a0 2024-05-09T04:55:10.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.377+0000 7fbde1541b80 1 -- 172.21.15.46:0/1448500773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe809f6a0 msgr2=0x55cfe80c0f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:10.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.377+0000 7fbde1541b80 1 --2- 172.21.15.46:0/1448500773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe809f6a0 0x55cfe80c0f70 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fbdc4009fd0 tx=0x7fbdc40099e0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.377+0000 7fbde1541b80 1 -- 172.21.15.46:0/1448500773 shutdown_connections 2024-05-09T04:55:10.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.377+0000 7fbde1541b80 1 --2- 172.21.15.46:0/1448500773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe809f6a0 0x55cfe80c0f70 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.377+0000 7fbde1541b80 1 --2- 172.21.15.46:0/1448500773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe7f224a0 0x55cfe80a9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.377+0000 7fbde1541b80 1 --2- 172.21.15.46:0/1448500773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe800c160 0x55cfe80a3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.377+0000 7fbde1541b80 1 -- 172.21.15.46:0/1448500773 >> 172.21.15.46:0/1448500773 conn(0x55cfe7f826a0 msgr2=0x55cfe80a2e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:10.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.377+0000 7fbde1541b80 1 -- 172.21.15.46:0/1448500773 shutdown_connections 2024-05-09T04:55:10.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.377+0000 7fbde1541b80 1 -- 172.21.15.46:0/1448500773 wait complete. 2024-05-09T04:55:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.377+0000 7fbde1541b80 1 -- start start 2024-05-09T04:55:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.377+0000 7fbde1541b80 1 Processor -- start 2024-05-09T04:55:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbde1541b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe7f224a0 0x55cfe80b1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbde1541b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbde1541b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe809f6a0 0x55cfe80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbde1541b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc1ad500 con 0x55cfe800c160 2024-05-09T04:55:10.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbde1541b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc1ae970 con 0x55cfe809f6a0 2024-05-09T04:55:10.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbde1541b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fbdcc1adbd0 con 0x55cfe7f224a0 2024-05-09T04:55:10.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbde1541b80 10 client.0 _ll_register_callbacks cb 0x55cfe7e336c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:10.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbdcbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbdcbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57468/0 (socket says 172.21.15.46:57468) 2024-05-09T04:55:10.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbdd08c7700 1 --2- 172.21.15.46:0/51535779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe7f224a0 0x55cfe80b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbdcbfff700 1 -- 172.21.15.46:0/51535779 learned_addr learned my addr 172.21.15.46:0/51535779 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:10.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbdd10c8700 1 --2- 172.21.15.46:0/51535779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe809f6a0 0x55cfe80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbdcbfff700 1 -- 172.21.15.46:0/51535779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe7f224a0 msgr2=0x55cfe80b1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:10.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbdcbfff700 1 --2- 172.21.15.46:0/51535779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe7f224a0 0x55cfe80b1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbdcbfff700 1 -- 172.21.15.46:0/51535779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe809f6a0 msgr2=0x55cfe80a6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:10.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbdcbfff700 1 --2- 172.21.15.46:0/51535779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe809f6a0 0x55cfe80a6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.378+0000 7fbdcbfff700 1 -- 172.21.15.46:0/51535779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfe80a6940 con 0x55cfe800c160 2024-05-09T04:55:10.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.379+0000 7fbdcbfff700 1 --2- 172.21.15.46:0/51535779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a5d20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8015040 tx=0x7fbdb8004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c6e035c03877898 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:10.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.379+0000 7fbdabfff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:10.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.379+0000 7fbdabfff700 1 -- 172.21.15.46:0/51535779 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8031060 con 0x55cfe800c160 2024-05-09T04:55:10.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.379+0000 7fbdabfff700 1 -- 172.21.15.46:0/51535779 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fbdb800fb70 con 0x55cfe800c160 2024-05-09T04:55:10.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.379+0000 7fbde1541b80 1 -- 172.21.15.46:0/51535779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdc40cbe50 con 0x55cfe800c160 2024-05-09T04:55:10.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.379+0000 7fbdabfff700 1 -- 172.21.15.46:0/51535779 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fbdb8004be0 con 0x55cfe800c160 2024-05-09T04:55:10.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.379+0000 7fbde1541b80 10 client.16449 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:10.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.379+0000 7fbde1541b80 1 -- 172.21.15.46:0/51535779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cfe8123980 con 0x55cfe800c160 2024-05-09T04:55:10.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.379+0000 7fbdabfff700 1 -- 172.21.15.46:0/51535779 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdc40cbe50 con 0x55cfe800c160 2024-05-09T04:55:10.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.379+0000 7fbdabfff700 1 -- 172.21.15.46:0/51535779 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fbdb8019040 con 0x55cfe800c160 2024-05-09T04:55:10.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.379+0000 7fbde1541b80 10 client.16449 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:10.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.379+0000 7fbde1541b80 10 client.16449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:55:10.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.380+0000 7fbde1541b80 20 client.16449 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:10.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.380+0000 7fbde1541b80 1 --2- 172.21.15.46:0/51535779 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55cfe8124030 0x55cfe8126450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:10.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.380+0000 7fbde1541b80 4 client.16449 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:55:10.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.380+0000 7fbde1541b80 1 -- 172.21.15.46:0/51535779 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cfe7fc9a70 con 0x55cfe8124030 2024-05-09T04:55:10.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.380+0000 7fbdd08c7700 1 --2- 172.21.15.46:0/51535779 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55cfe8124030 0x55cfe8126450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:10.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.380+0000 7fbdd08c7700 1 --2- 172.21.15.46:0/51535779 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55cfe8124030 0x55cfe8126450 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4dacf365f4be205 server_cookie=c1cbf79264a89d7a in_seq=0 out_seq=0 2024-05-09T04:55:10.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.380+0000 7fbdabfff700 10 client.16449 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:10.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.380+0000 7fbdabfff700 1 -- 172.21.15.46:0/51535779 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cfe7fc9a70 con 0x55cfe8124030 2024-05-09T04:55:10.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.380+0000 7fbdabfff700 10 client.16449 handle_command_reply: tid=0 2024-05-09T04:55:10.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.380+0000 7fbde1541b80 1 client.16449 shutdown 2024-05-09T04:55:10.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.381+0000 7fbde1541b80 1 -- 172.21.15.46:0/51535779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 msgr2=0x55cfe80a5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:10.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.381+0000 7fbde1541b80 1 --2- 172.21.15.46:0/51535779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a5d20 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8015040 tx=0x7fbdb8004940 comp rx=0 tx=0).stop 2024-05-09T04:55:10.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.381+0000 7fbde1541b80 1 -- 172.21.15.46:0/51535779 shutdown_connections 2024-05-09T04:55:10.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.381+0000 7fbde1541b80 1 --2- 172.21.15.46:0/51535779 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55cfe8124030 0x55cfe8126450 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.381+0000 7fbde1541b80 1 --2- 172.21.15.46:0/51535779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55cfe809f6a0 0x55cfe80a6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.381+0000 7fbde1541b80 1 --2- 172.21.15.46:0/51535779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55cfe7f224a0 0x55cfe80b1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.381+0000 7fbdabfff700 0 client.16449 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:10.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.381+0000 7fbde1541b80 1 --2- 172.21.15.46:0/51535779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55cfe800c160 0x55cfe80a5d20 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.381+0000 7fbde1541b80 1 -- 172.21.15.46:0/51535779 >> 172.21.15.46:0/51535779 conn(0x55cfe7f826a0 msgr2=0x55cfe80a9980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:10.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.381+0000 7fbde1541b80 1 -- 172.21.15.46:0/51535779 shutdown_connections 2024-05-09T04:55:10.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.381+0000 7fbde1541b80 1 -- 172.21.15.46:0/51535779 wait complete. 2024-05-09T04:55:10.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.381+0000 7fbde1541b80 20 client.16449 trim_cache size 0 max 16384 2024-05-09T04:55:10.387 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:55:10.387 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T04:55:10.387 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-09T04:55:10.387 INFO:teuthology.orchestra.run.smithi046.stdout: "82a5fb59-3325-40b5-bd29-397fe40b478c": { 2024-05-09T04:55:10.388 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-09T04:55:10.388 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "82a5fb59-3325-40b5-bd29-397fe40b478c", 2024-05-09T04:55:10.388 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-09T04:55:10.388 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T04:55:10.388 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T04:55:10.388 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:55:10.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.382+0000 7fbdd2b2b700 1 -- 172.21.15.46:0/3545731443 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbdbc0b29f0 msgr2=0x7fbdbc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:10.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.382+0000 7fbdd2b2b700 1 --2- 172.21.15.46:0/3545731443 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbdbc0b29f0 0x7fbdbc0b4f00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fbdc0005950 tx=0x7fbdc000b410 comp rx=0 tx=0).stop 2024-05-09T04:55:10.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.382+0000 7fbdd2b2b700 1 -- 172.21.15.46:0/3545731443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbdcc108760 msgr2=0x7fbdcc19f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:10.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.382+0000 7fbdd2b2b700 1 --2- 172.21.15.46:0/3545731443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbdcc108760 0x7fbdcc19f640 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fbdc40095c0 tx=0x7fbdc4007a20 comp rx=0 tx=0).stop 2024-05-09T04:55:10.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.382+0000 7fbdd2b2b700 1 -- 172.21.15.46:0/3545731443 shutdown_connections 2024-05-09T04:55:10.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.382+0000 7fbdd2b2b700 1 --2- 172.21.15.46:0/3545731443 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fbdbc0b29f0 0x7fbdbc0b4f00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.382+0000 7fbdd2b2b700 1 --2- 172.21.15.46:0/3545731443 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fbdcc108760 0x7fbdcc19f640 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.382+0000 7fbdd2b2b700 1 --2- 172.21.15.46:0/3545731443 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fbdcc1034e0 0x7fbdcc19ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.382+0000 7fbdd2b2b700 1 --2- 172.21.15.46:0/3545731443 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fbdcc107e10 0x7fbdcc19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:10.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.382+0000 7fbdd2b2b700 1 -- 172.21.15.46:0/3545731443 >> 172.21.15.46:0/3545731443 conn(0x7fbdcc0fef30 msgr2=0x7fbdcc107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:10.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.382+0000 7fbdd2b2b700 1 -- 172.21.15.46:0/3545731443 shutdown_connections 2024-05-09T04:55:10.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:10.382+0000 7fbdd2b2b700 1 -- 172.21.15.46:0/3545731443 wait complete. 2024-05-09T04:55:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:10 smithi110 ceph-mon[34904]: pgmap v125: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-09T04:55:11.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:10 smithi046 ceph-mon[30136]: pgmap v125: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-09T04:55:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:10 smithi072 ceph-mon[34088]: pgmap v125: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-09T04:55:11.282 INFO:tasks.cephfs.filesystem:scrub status for tag:82a5fb59-3325-40b5-bd29-397fe40b478c - {'path': '/', 'tag': '82a5fb59-3325-40b5-bd29-397fe40b478c', 'options': 'recursive,force'} 2024-05-09T04:55:12.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:11 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:55:12.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:11 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:55:12.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:11 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:55:13.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:12 smithi110 ceph-mon[34904]: pgmap v126: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T04:55:13.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:12 smithi046 ceph-mon[30136]: pgmap v126: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T04:55:13.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:12 smithi072 ceph-mon[34088]: pgmap v126: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T04:55:15.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:14 smithi110 ceph-mon[34904]: pgmap v127: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-05-09T04:55:15.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:14 smithi046 ceph-mon[30136]: pgmap v127: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-05-09T04:55:15.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:14 smithi072 ceph-mon[34088]: pgmap v127: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-05-09T04:55:17.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:16 smithi110 ceph-mon[34904]: pgmap v128: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-09T04:55:17.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:16 smithi046 ceph-mon[30136]: pgmap v128: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-09T04:55:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:16 smithi072 ceph-mon[34088]: pgmap v128: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-09T04:55:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:18 smithi110 ceph-mon[34904]: pgmap v129: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-09T04:55:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:18 smithi046 ceph-mon[30136]: pgmap v129: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-09T04:55:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:18 smithi072 ceph-mon[34088]: pgmap v129: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-09T04:55:21.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:20 smithi110 ceph-mon[34904]: pgmap v130: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T04:55:21.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:20 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:55:21.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:20 smithi046 ceph-mon[30136]: pgmap v130: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T04:55:21.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:20 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:55:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:20 smithi072 ceph-mon[34088]: pgmap v130: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-05-09T04:55:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:20 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:55:21.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:21 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:55:21.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:21 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:55:21.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:21 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:55:22.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:21 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:55:22.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:21 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:55:22.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:21 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:55:22.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:21 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:55:22.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:21 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:55:22.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:21 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:55:23.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:22 smithi110 ceph-mon[34904]: pgmap v131: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T04:55:23.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:22 smithi046 ceph-mon[30136]: pgmap v131: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T04:55:23.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:22 smithi072 ceph-mon[34088]: pgmap v131: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-09T04:55:25.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:24 smithi110 ceph-mon[34904]: pgmap v132: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:55:25.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:24 smithi046 ceph-mon[30136]: pgmap v132: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:55:25.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:24 smithi072 ceph-mon[34088]: pgmap v132: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:55:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:26 smithi110 ceph-mon[34904]: pgmap v133: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:27.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:26 smithi046 ceph-mon[30136]: pgmap v133: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:26 smithi072 ceph-mon[34088]: pgmap v133: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:28 smithi110 ceph-mon[34904]: pgmap v134: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:55:29.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:28 smithi046 ceph-mon[30136]: pgmap v134: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:55:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:28 smithi072 ceph-mon[34088]: pgmap v134: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:55:31.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:30 smithi110 ceph-mon[34904]: pgmap v135: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:31.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:30 smithi046 ceph-mon[30136]: pgmap v135: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:31.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:30 smithi072 ceph-mon[34088]: pgmap v135: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:32.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:32 smithi046 ceph-mon[30136]: pgmap v136: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:55:33.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:32 smithi110 ceph-mon[34904]: pgmap v136: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:55:33.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:32 smithi072 ceph-mon[34088]: pgmap v136: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:55:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:34 smithi110 ceph-mon[34904]: pgmap v137: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:55:35.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:34 smithi046 ceph-mon[30136]: pgmap v137: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:55:35.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:34 smithi072 ceph-mon[34088]: pgmap v137: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:55:37.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:36 smithi110 ceph-mon[34904]: pgmap v138: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:37.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:36 smithi046 ceph-mon[30136]: pgmap v138: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:37.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:36 smithi072 ceph-mon[34088]: pgmap v138: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:39.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:38 smithi110 ceph-mon[34904]: pgmap v139: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:55:39.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:38 smithi046 ceph-mon[30136]: pgmap v139: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:55:39.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:38 smithi072 ceph-mon[34088]: pgmap v139: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:55:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:40 smithi110 ceph-mon[34904]: pgmap v140: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:41.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:40 smithi046 ceph-mon[30136]: pgmap v140: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:40 smithi072 ceph-mon[34088]: pgmap v140: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:41.284 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:55:41.551 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:43.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:42 smithi110 ceph-mon[34904]: pgmap v141: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:55:43.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:42 smithi046 ceph-mon[30136]: pgmap v141: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:55:43.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:42 smithi072 ceph-mon[34088]: pgmap v141: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:55:43.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.344+0000 7f20f7fb8700 1 -- 172.21.15.46:0/590079945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f010e8d0 msgr2=0x7f20f0110de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.344+0000 7f20f7fb8700 1 --2- 172.21.15.46:0/590079945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f010e8d0 0x7f20f0110de0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f20e800b0a0 tx=0x7f20e800b3b0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.344+0000 7f20f7fb8700 1 -- 172.21.15.46:0/590079945 shutdown_connections 2024-05-09T04:55:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.344+0000 7f20f7fb8700 1 --2- 172.21.15.46:0/590079945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f010e8d0 0x7f20f0110de0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.344+0000 7f20f7fb8700 1 --2- 172.21.15.46:0/590079945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20f0105460 0x7f20f0105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.344+0000 7f20f7fb8700 1 --2- 172.21.15.46:0/590079945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20f0105db0 0x7f20f010e300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.344+0000 7f20f7fb8700 1 -- 172.21.15.46:0/590079945 >> 172.21.15.46:0/590079945 conn(0x7f20f00fedf0 msgr2=0x7f20f0101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.344+0000 7f20f7fb8700 1 -- 172.21.15.46:0/590079945 shutdown_connections 2024-05-09T04:55:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.344+0000 7f20f7fb8700 1 -- 172.21.15.46:0/590079945 wait complete. 2024-05-09T04:55:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.345+0000 7f20f7fb8700 1 Processor -- start 2024-05-09T04:55:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.345+0000 7f20f7fb8700 1 -- start start 2024-05-09T04:55:43.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.345+0000 7f20f7fb8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20f0105460 0x7f20f019adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.345+0000 7f20f7fb8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20f0105db0 0x7f20f019b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.345+0000 7f20f7fb8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f010e8d0 0x7f20f019f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.345+0000 7f20f7fb8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0114480 con 0x7f20f010e8d0 2024-05-09T04:55:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.345+0000 7f20f7fb8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0114300 con 0x7f20f0105460 2024-05-09T04:55:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.345+0000 7f20f7fb8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f20f0114600 con 0x7f20f0105db0 2024-05-09T04:55:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20f6555700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f010e8d0 0x7f20f019f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20f6555700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f010e8d0 0x7f20f019f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36598/0 (socket says 172.21.15.46:36598) 2024-05-09T04:55:43.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20f6555700 1 -- 172.21.15.46:0/3641566453 learned_addr learned my addr 172.21.15.46:0/3641566453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20f5553700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20f0105db0 0x7f20f019b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20f5d54700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20f0105460 0x7f20f019adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20f6555700 1 -- 172.21.15.46:0/3641566453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20f0105db0 msgr2=0x7f20f019b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20f6555700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20f0105db0 0x7f20f019b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20f6555700 1 -- 172.21.15.46:0/3641566453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20f0105460 msgr2=0x7f20f019adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:43.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20f6555700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20f0105460 0x7f20f019adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20f6555700 1 -- 172.21.15.46:0/3641566453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f019fc10 con 0x7f20f010e8d0 2024-05-09T04:55:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20f5d54700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20f0105460 0x7f20f019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20f5553700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20f0105db0 0x7f20f019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20f6555700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f010e8d0 0x7f20f019f620 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f20e800b830 tx=0x7f20e8004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20e6ffd700 1 -- 172.21.15.46:0/3641566453 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8021070 con 0x7f20f010e8d0 2024-05-09T04:55:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20e6ffd700 1 -- 172.21.15.46:0/3641566453 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e8003c00 con 0x7f20f010e8d0 2024-05-09T04:55:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.346+0000 7f20e6ffd700 1 -- 172.21.15.46:0/3641566453 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e8004730 con 0x7f20f010e8d0 2024-05-09T04:55:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.347+0000 7f20f7fb8700 1 -- 172.21.15.46:0/3641566453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f019fea0 con 0x7f20f010e8d0 2024-05-09T04:55:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.347+0000 7f20f7fb8700 1 -- 172.21.15.46:0/3641566453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f01a0390 con 0x7f20f010e8d0 2024-05-09T04:55:43.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.349+0000 7f20e6ffd700 1 -- 172.21.15.46:0/3641566453 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f20e8007a00 con 0x7f20f010e8d0 2024-05-09T04:55:43.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.350+0000 7f20e6ffd700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f20dc0b2a70 0x7f20dc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.350+0000 7f21069ceb80 1 Processor -- start 2024-05-09T04:55:43.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.350+0000 7f20f5d54700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f20dc0b2a70 0x7f20dc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.350+0000 7f20e6ffd700 1 -- 172.21.15.46:0/3641566453 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f20e800f020 con 0x7f20f010e8d0 2024-05-09T04:55:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f21069ceb80 1 -- start start 2024-05-09T04:55:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f20f5d54700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f20dc0b2a70 0x7f20dc0b4f80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f20f006d150 tx=0x7f20ec00a680 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f21069ceb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f723fd00 0x5637f7348240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f21069ceb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f71c54a0 0x5637f7348780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f21069ceb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f7342810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f21069ceb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5637f72650a0 con 0x5637f71c54a0 2024-05-09T04:55:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f21069ceb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5637f720a9a0 con 0x5637f723fd00 2024-05-09T04:55:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f21069ceb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5637f7261060 con 0x5637f7342400 2024-05-09T04:55:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f20f5553700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f71c54a0 0x5637f7348780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f20f5553700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f71c54a0 0x5637f7348780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36612/0 (socket says 172.21.15.46:36612) 2024-05-09T04:55:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f20f5553700 1 -- 172.21.15.46:0/1046304632 learned_addr learned my addr 172.21.15.46:0/1046304632 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f20f6555700 1 --2- 172.21.15.46:0/1046304632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f7342810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f20f5d54700 1 --2- 172.21.15.46:0/1046304632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f723fd00 0x5637f7348240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f20f5553700 1 -- 172.21.15.46:0/1046304632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 msgr2=0x5637f7342810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f20f5553700 1 --2- 172.21.15.46:0/1046304632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f7342810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f20f5553700 1 -- 172.21.15.46:0/1046304632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f723fd00 msgr2=0x5637f7348240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f20f5553700 1 --2- 172.21.15.46:0/1046304632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f723fd00 0x5637f7348240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.351+0000 7f20f5553700 1 -- 172.21.15.46:0/1046304632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637f71fb500 con 0x5637f71c54a0 2024-05-09T04:55:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.352+0000 7f20f5553700 1 --2- 172.21.15.46:0/1046304632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f71c54a0 0x5637f7348780 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f20e000dc40 tx=0x7f20e000df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3665574214f2ba2c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.352+0000 7f20dbfff700 1 -- 172.21.15.46:0/1046304632 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0015070 con 0x5637f71c54a0 2024-05-09T04:55:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.352+0000 7f20dbfff700 1 -- 172.21.15.46:0/1046304632 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e0011440 con 0x5637f71c54a0 2024-05-09T04:55:43.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.352+0000 7f20dbfff700 1 -- 172.21.15.46:0/1046304632 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0010630 con 0x5637f71c54a0 2024-05-09T04:55:43.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.353+0000 7f21069ceb80 1 -- 172.21.15.46:0/1046304632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f71c54a0 msgr2=0x5637f7348780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.353+0000 7f21069ceb80 1 --2- 172.21.15.46:0/1046304632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f71c54a0 0x5637f7348780 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f20e000dc40 tx=0x7f20e000df50 comp rx=0 tx=0).stop 2024-05-09T04:55:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.353+0000 7f21069ceb80 1 -- 172.21.15.46:0/1046304632 shutdown_connections 2024-05-09T04:55:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.353+0000 7f21069ceb80 1 --2- 172.21.15.46:0/1046304632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f7342810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.353+0000 7f21069ceb80 1 --2- 172.21.15.46:0/1046304632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f723fd00 0x5637f7348240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.353+0000 7f21069ceb80 1 --2- 172.21.15.46:0/1046304632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f71c54a0 0x5637f7348780 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.353+0000 7f21069ceb80 1 -- 172.21.15.46:0/1046304632 >> 172.21.15.46:0/1046304632 conn(0x5637f71c8f70 msgr2=0x5637f733d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.353+0000 7f21069ceb80 1 -- 172.21.15.46:0/1046304632 shutdown_connections 2024-05-09T04:55:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.353+0000 7f21069ceb80 1 -- 172.21.15.46:0/1046304632 wait complete. 2024-05-09T04:55:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.353+0000 7f21069ceb80 1 -- start start 2024-05-09T04:55:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.354+0000 7f21069ceb80 1 Processor -- start 2024-05-09T04:55:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.354+0000 7f21069ceb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f71c54a0 0x5637f734c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.354+0000 7f21069ceb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f723fd00 0x5637f734c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.354+0000 7f21069ceb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f734cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.354+0000 7f21069ceb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5637f7343160 con 0x5637f71c54a0 2024-05-09T04:55:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.354+0000 7f21069ceb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5637f71ffb20 con 0x5637f723fd00 2024-05-09T04:55:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.354+0000 7f21069ceb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5637f734f020 con 0x5637f7342400 2024-05-09T04:55:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.354+0000 7f21069ceb80 10 client.0 _ll_register_callbacks cb 0x5637f70d66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.354+0000 7f20f5d54700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f71c54a0 0x5637f734c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.354+0000 7f20f5d54700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f71c54a0 0x5637f734c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36614/0 (socket says 172.21.15.46:36614) 2024-05-09T04:55:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.354+0000 7f20f5d54700 1 -- 172.21.15.46:0/2912465133 learned_addr learned my addr 172.21.15.46:0/2912465133 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.354+0000 7f20f6555700 1 --2- 172.21.15.46:0/2912465133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f734cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.354+0000 7f20f5553700 1 --2- 172.21.15.46:0/2912465133 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f723fd00 0x5637f734c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.355+0000 7f20f5d54700 1 -- 172.21.15.46:0/2912465133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 msgr2=0x5637f734cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.355+0000 7f20f5d54700 1 --2- 172.21.15.46:0/2912465133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f734cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.355+0000 7f20f5d54700 1 -- 172.21.15.46:0/2912465133 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f723fd00 msgr2=0x5637f734c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:43.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.355+0000 7f20f5d54700 1 --2- 172.21.15.46:0/2912465133 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f723fd00 0x5637f734c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.355+0000 7f20f5d54700 1 -- 172.21.15.46:0/2912465133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637f70e0490 con 0x5637f71c54a0 2024-05-09T04:55:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.355+0000 7f20f5d54700 1 --2- 172.21.15.46:0/2912465133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f71c54a0 0x5637f734c110 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0109c0 tx=0x7f20ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a486820a6375927 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.355+0000 7f20d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.355+0000 7f20d97fa700 1 -- 172.21.15.46:0/2912465133 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20ec004230 con 0x5637f71c54a0 2024-05-09T04:55:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.355+0000 7f21069ceb80 1 -- 172.21.15.46:0/2912465133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e0011440 con 0x5637f71c54a0 2024-05-09T04:55:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.355+0000 7f20d97fa700 1 -- 172.21.15.46:0/2912465133 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f20ec0043d0 con 0x5637f71c54a0 2024-05-09T04:55:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.355+0000 7f20d97fa700 1 -- 172.21.15.46:0/2912465133 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f20ec01d850 con 0x5637f71c54a0 2024-05-09T04:55:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.355+0000 7f21069ceb80 10 client.16461 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.355+0000 7f21069ceb80 1 -- 172.21.15.46:0/2912465133 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5637f7359bd0 con 0x5637f71c54a0 2024-05-09T04:55:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.355+0000 7f20d97fa700 1 -- 172.21.15.46:0/2912465133 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e0011440 con 0x5637f71c54a0 2024-05-09T04:55:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.356+0000 7f20d97fa700 1 -- 172.21.15.46:0/2912465133 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f20ec003c50 con 0x5637f71c54a0 2024-05-09T04:55:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.356+0000 7f21069ceb80 10 client.16461 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.356+0000 7f21069ceb80 10 client.16461 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:55:43.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.356+0000 7f21069ceb80 20 client.16461 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.356+0000 7f21069ceb80 1 --2- 172.21.15.46:0/2912465133 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5637f735c1b0 0x5637f735e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.356+0000 7f21069ceb80 4 client.16461 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:55:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.356+0000 7f21069ceb80 1 -- 172.21.15.46:0/2912465133 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5637f7272a80 con 0x5637f735c1b0 2024-05-09T04:55:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.356+0000 7f20f5553700 1 --2- 172.21.15.46:0/2912465133 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5637f735c1b0 0x5637f735e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.356+0000 7f20f5553700 1 --2- 172.21.15.46:0/2912465133 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5637f735c1b0 0x5637f735e5f0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c870b91d9f673b6 server_cookie=ff9e63f870f4e7ab in_seq=0 out_seq=0 2024-05-09T04:55:43.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.356+0000 7f20d97fa700 10 client.16461 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.357+0000 7f20d97fa700 1 -- 172.21.15.46:0/2912465133 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5637f7272a80 con 0x5637f735c1b0 2024-05-09T04:55:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.357+0000 7f20d97fa700 10 client.16461 handle_command_reply: tid=0 2024-05-09T04:55:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.357+0000 7f21069ceb80 1 client.16461 shutdown 2024-05-09T04:55:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.357+0000 7f21069ceb80 1 -- 172.21.15.46:0/2912465133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f71c54a0 msgr2=0x5637f734c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.357+0000 7f21069ceb80 1 --2- 172.21.15.46:0/2912465133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f71c54a0 0x5637f734c110 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f20ec0109c0 tx=0x7f20ec004750 comp rx=0 tx=0).stop 2024-05-09T04:55:43.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.357+0000 7f21069ceb80 1 -- 172.21.15.46:0/2912465133 shutdown_connections 2024-05-09T04:55:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.357+0000 7f21069ceb80 1 --2- 172.21.15.46:0/2912465133 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f734cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.357+0000 7f21069ceb80 1 --2- 172.21.15.46:0/2912465133 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f71c54a0 0x5637f734c110 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.357+0000 7f21069ceb80 1 --2- 172.21.15.46:0/2912465133 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5637f735c1b0 0x5637f735e5f0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.357+0000 7f21069ceb80 1 --2- 172.21.15.46:0/2912465133 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f723fd00 0x5637f734c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.357+0000 7f20d97fa700 0 client.16461 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.357+0000 7f21069ceb80 1 -- 172.21.15.46:0/2912465133 >> 172.21.15.46:0/2912465133 conn(0x5637f71c8f70 msgr2=0x5637f7336ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.357+0000 7f21069ceb80 1 -- 172.21.15.46:0/2912465133 shutdown_connections 2024-05-09T04:55:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.358+0000 7f21069ceb80 1 -- 172.21.15.46:0/2912465133 wait complete. 2024-05-09T04:55:43.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.358+0000 7f21069ceb80 20 client.16461 trim_cache size 0 max 16384 2024-05-09T04:55:43.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.378+0000 7f21069ceb80 1 Processor -- start 2024-05-09T04:55:43.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.378+0000 7f21069ceb80 1 -- start start 2024-05-09T04:55:43.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.378+0000 7f21069ceb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f71c54a0 0x5637f7341e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.378+0000 7f21069ceb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f723fd00 0x5637f734c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.378+0000 7f21069ceb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f7357e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.378+0000 7f21069ceb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01b16b0 con 0x5637f723fd00 2024-05-09T04:55:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.378+0000 7f21069ceb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01aef40 con 0x5637f71c54a0 2024-05-09T04:55:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.378+0000 7f21069ceb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01acd90 con 0x5637f7342400 2024-05-09T04:55:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.378+0000 7f20f5553700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f723fd00 0x5637f734c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.379+0000 7f20f5553700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f723fd00 0x5637f734c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36622/0 (socket says 172.21.15.46:36622) 2024-05-09T04:55:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.379+0000 7f20f5553700 1 -- 172.21.15.46:0/2008862160 learned_addr learned my addr 172.21.15.46:0/2008862160 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.379+0000 7f20f6555700 1 --2- 172.21.15.46:0/2008862160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f7357e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.379+0000 7f20f5d54700 1 --2- 172.21.15.46:0/2008862160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f71c54a0 0x5637f7341e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.379+0000 7f20f5553700 1 -- 172.21.15.46:0/2008862160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 msgr2=0x5637f7357e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.379+0000 7f20f5553700 1 --2- 172.21.15.46:0/2008862160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f7357e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.379+0000 7f20f5553700 1 -- 172.21.15.46:0/2008862160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f71c54a0 msgr2=0x5637f7341e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.379+0000 7f20f5553700 1 --2- 172.21.15.46:0/2008862160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f71c54a0 0x5637f7341e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.379+0000 7f20f5553700 1 -- 172.21.15.46:0/2008862160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637f73399d0 con 0x5637f723fd00 2024-05-09T04:55:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.379+0000 7f20f5553700 1 --2- 172.21.15.46:0/2008862160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f723fd00 0x5637f734c9d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f20e0011d40 tx=0x7f20e000d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df99fd6cef357e5b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.379+0000 7f20db7fe700 1 -- 172.21.15.46:0/2008862160 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e002e070 con 0x5637f723fd00 2024-05-09T04:55:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.379+0000 7f20db7fe700 1 -- 172.21.15.46:0/2008862160 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e00044e0 con 0x5637f723fd00 2024-05-09T04:55:43.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.379+0000 7f20db7fe700 1 -- 172.21.15.46:0/2008862160 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0015070 con 0x5637f723fd00 2024-05-09T04:55:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.380+0000 7f21069ceb80 1 -- 172.21.15.46:0/2008862160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f723fd00 msgr2=0x5637f734c9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.380+0000 7f21069ceb80 1 --2- 172.21.15.46:0/2008862160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f723fd00 0x5637f734c9d0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f20e0011d40 tx=0x7f20e000d560 comp rx=0 tx=0).stop 2024-05-09T04:55:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.380+0000 7f21069ceb80 1 -- 172.21.15.46:0/2008862160 shutdown_connections 2024-05-09T04:55:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.380+0000 7f21069ceb80 1 --2- 172.21.15.46:0/2008862160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f7357e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.380+0000 7f21069ceb80 1 --2- 172.21.15.46:0/2008862160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f71c54a0 0x5637f7341e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.380+0000 7f21069ceb80 1 --2- 172.21.15.46:0/2008862160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f723fd00 0x5637f734c9d0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.380+0000 7f21069ceb80 1 -- 172.21.15.46:0/2008862160 >> 172.21.15.46:0/2008862160 conn(0x5637f71c8f70 msgr2=0x5637f734ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.380+0000 7f21069ceb80 1 -- 172.21.15.46:0/2008862160 shutdown_connections 2024-05-09T04:55:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.380+0000 7f21069ceb80 1 -- 172.21.15.46:0/2008862160 wait complete. 2024-05-09T04:55:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.381+0000 7f21069ceb80 1 -- start start 2024-05-09T04:55:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.381+0000 7f21069ceb80 1 Processor -- start 2024-05-09T04:55:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.381+0000 7f21069ceb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f71c54a0 0x5637f7351d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.381+0000 7f21069ceb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f723fd00 0x5637f7352260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.381+0000 7f21069ceb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f73527a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.381+0000 7f21069ceb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01ad400 con 0x5637f723fd00 2024-05-09T04:55:43.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.381+0000 7f21069ceb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01ae870 con 0x5637f71c54a0 2024-05-09T04:55:43.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.381+0000 7f21069ceb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f20f01adad0 con 0x5637f7342400 2024-05-09T04:55:43.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.381+0000 7f21069ceb80 10 client.0 _ll_register_callbacks cb 0x5637f70d66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:43.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.381+0000 7f20f5553700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f723fd00 0x5637f7352260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.381+0000 7f20f5553700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f723fd00 0x5637f7352260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36626/0 (socket says 172.21.15.46:36626) 2024-05-09T04:55:43.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.381+0000 7f20f5553700 1 -- 172.21.15.46:0/4007852490 learned_addr learned my addr 172.21.15.46:0/4007852490 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:43.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.382+0000 7f20f5d54700 1 --2- 172.21.15.46:0/4007852490 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f71c54a0 0x5637f7351d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.382+0000 7f20f6555700 1 --2- 172.21.15.46:0/4007852490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f73527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.382+0000 7f20f5553700 1 -- 172.21.15.46:0/4007852490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 msgr2=0x5637f73527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:43.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.382+0000 7f20f5553700 1 --2- 172.21.15.46:0/4007852490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f73527a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.382+0000 7f20f5553700 1 -- 172.21.15.46:0/4007852490 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f71c54a0 msgr2=0x5637f7351d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:43.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.382+0000 7f20f5553700 1 --2- 172.21.15.46:0/4007852490 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f71c54a0 0x5637f7351d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.382+0000 7f20f5553700 1 -- 172.21.15.46:0/4007852490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637f73c5970 con 0x5637f723fd00 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.382+0000 7f20f6555700 1 --2- 172.21.15.46:0/4007852490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f73527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.382+0000 7f20f5553700 1 --2- 172.21.15.46:0/4007852490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f723fd00 0x5637f7352260 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f20e0005f80 tx=0x7f20e000f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154eb1baedb26890 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.382+0000 7f20d97fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.382+0000 7f20d97fa700 1 -- 172.21.15.46:0/4007852490 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e0038060 con 0x5637f723fd00 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.382+0000 7f20d97fa700 1 -- 172.21.15.46:0/4007852490 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f20e000fb70 con 0x5637f723fd00 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.382+0000 7f21069ceb80 1 -- 172.21.15.46:0/4007852490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20e00044e0 con 0x5637f723fd00 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.382+0000 7f20d97fa700 1 -- 172.21.15.46:0/4007852490 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f20e002e070 con 0x5637f723fd00 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.383+0000 7f21069ceb80 10 client.16467 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.383+0000 7f21069ceb80 1 -- 172.21.15.46:0/4007852490 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5637f73c6b10 con 0x5637f723fd00 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.383+0000 7f20d97fa700 1 -- 172.21.15.46:0/4007852490 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e00044e0 con 0x5637f723fd00 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.383+0000 7f20d97fa700 1 -- 172.21.15.46:0/4007852490 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f20e00048d0 con 0x5637f723fd00 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.383+0000 7f21069ceb80 10 client.16467 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.383+0000 7f21069ceb80 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.383+0000 7f21069ceb80 20 client.16467 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:43.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.383+0000 7f21069ceb80 1 --2- 172.21.15.46:0/4007852490 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5637f73c6ef0 0x5637f73c9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.383+0000 7f21069ceb80 4 client.16467 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.383+0000 7f21069ceb80 1 -- 172.21.15.46:0/4007852490 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5637f726cb20 con 0x5637f73c6ef0 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.383+0000 7f20f5d54700 1 --2- 172.21.15.46:0/4007852490 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5637f73c6ef0 0x5637f73c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.384+0000 7f20f5d54700 1 --2- 172.21.15.46:0/4007852490 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5637f73c6ef0 0x5637f73c9310 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d84fac444c3247ac server_cookie=942902ea505b3ace in_seq=0 out_seq=0 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.384+0000 7f20d97fa700 10 client.16467 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.384+0000 7f20d97fa700 1 -- 172.21.15.46:0/4007852490 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5637f726cb20 con 0x5637f73c6ef0 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.384+0000 7f20d97fa700 10 client.16467 handle_command_reply: tid=0 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.384+0000 7f21069ceb80 1 client.16467 shutdown 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.384+0000 7f21069ceb80 1 -- 172.21.15.46:0/4007852490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f723fd00 msgr2=0x5637f7352260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.384+0000 7f21069ceb80 1 --2- 172.21.15.46:0/4007852490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f723fd00 0x5637f7352260 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f20e0005f80 tx=0x7f20e000f6f0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.384+0000 7f21069ceb80 1 -- 172.21.15.46:0/4007852490 shutdown_connections 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.384+0000 7f21069ceb80 1 --2- 172.21.15.46:0/4007852490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5637f7342400 0x5637f73527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.384+0000 7f21069ceb80 1 --2- 172.21.15.46:0/4007852490 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5637f73c6ef0 0x5637f73c9310 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.384+0000 7f21069ceb80 1 --2- 172.21.15.46:0/4007852490 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5637f71c54a0 0x5637f7351d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.384+0000 7f21069ceb80 1 --2- 172.21.15.46:0/4007852490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5637f723fd00 0x5637f7352260 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.384+0000 7f20d97fa700 0 client.16467 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.385+0000 7f21069ceb80 1 -- 172.21.15.46:0/4007852490 >> 172.21.15.46:0/4007852490 conn(0x5637f71c8f70 msgr2=0x5637f7345c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.385+0000 7f21069ceb80 1 -- 172.21.15.46:0/4007852490 shutdown_connections 2024-05-09T04:55:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.385+0000 7f21069ceb80 1 -- 172.21.15.46:0/4007852490 wait complete. 2024-05-09T04:55:43.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.385+0000 7f21069ceb80 20 client.16467 trim_cache size 0 max 16384 2024-05-09T04:55:43.389 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:55:43.389 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:55:43.389 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:55:43.389 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:55:43.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.385+0000 7f20f7fb8700 1 -- 172.21.15.46:0/3641566453 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f20dc0b2a70 msgr2=0x7f20dc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:43.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.385+0000 7f20f7fb8700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f20dc0b2a70 0x7f20dc0b4f80 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f20f006d150 tx=0x7f20ec00a680 comp rx=0 tx=0).stop 2024-05-09T04:55:43.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.385+0000 7f20f7fb8700 1 -- 172.21.15.46:0/3641566453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f010e8d0 msgr2=0x7f20f019f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:43.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.385+0000 7f20f7fb8700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f010e8d0 0x7f20f019f620 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f20e800b830 tx=0x7f20e8004d90 comp rx=0 tx=0).stop 2024-05-09T04:55:43.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.386+0000 7f20f7fb8700 1 -- 172.21.15.46:0/3641566453 shutdown_connections 2024-05-09T04:55:43.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.386+0000 7f20f7fb8700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f20dc0b2a70 0x7f20dc0b4f80 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.386+0000 7f20f7fb8700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f20f010e8d0 0x7f20f019f620 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.386+0000 7f20f7fb8700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f20f0105460 0x7f20f019adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.386+0000 7f20f7fb8700 1 --2- 172.21.15.46:0/3641566453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f20f0105db0 0x7f20f019b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:43.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.386+0000 7f20f7fb8700 1 -- 172.21.15.46:0/3641566453 >> 172.21.15.46:0/3641566453 conn(0x7f20f00fedf0 msgr2=0x7f20f00ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:43.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.386+0000 7f20f7fb8700 1 -- 172.21.15.46:0/3641566453 shutdown_connections 2024-05-09T04:55:43.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:43.386+0000 7f20f7fb8700 1 -- 172.21.15.46:0/3641566453 wait complete. 2024-05-09T04:55:44.143 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:55:44.143 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:55:44.407 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:44.817 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:44 smithi046 ceph-mon[30136]: pgmap v142: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:55:45.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:44 smithi110 ceph-mon[34904]: pgmap v142: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:55:45.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:44 smithi072 ceph-mon[34088]: pgmap v142: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:55:46.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.203+0000 7f25c869d700 1 -- 172.21.15.46:0/3198726196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c0105e40 msgr2=0x7f25c010e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.203+0000 7f25c869d700 1 --2- 172.21.15.46:0/3198726196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c0105e40 0x7f25c010e390 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009990 tx=0x7f25b0009ca0 comp rx=0 tx=0).stop 2024-05-09T04:55:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.203+0000 7f25c869d700 1 -- 172.21.15.46:0/3198726196 shutdown_connections 2024-05-09T04:55:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.203+0000 7f25c869d700 1 --2- 172.21.15.46:0/3198726196 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25c010e920 0x7f25c0110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.203+0000 7f25c869d700 1 --2- 172.21.15.46:0/3198726196 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c01054f0 0x7f25c0105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.203+0000 7f25c869d700 1 --2- 172.21.15.46:0/3198726196 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c0105e40 0x7f25c010e390 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.203+0000 7f25c869d700 1 -- 172.21.15.46:0/3198726196 >> 172.21.15.46:0/3198726196 conn(0x7f25c0078e00 msgr2=0x7f25c0079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.203+0000 7f25c869d700 1 -- 172.21.15.46:0/3198726196 shutdown_connections 2024-05-09T04:55:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.203+0000 7f25c869d700 1 -- 172.21.15.46:0/3198726196 wait complete. 2024-05-09T04:55:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.204+0000 7f25c869d700 1 Processor -- start 2024-05-09T04:55:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.204+0000 7f25c869d700 1 -- start start 2024-05-09T04:55:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.204+0000 7f25c869d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25c01054f0 0x7f25c01031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:46.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.204+0000 7f25c869d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c0105e40 0x7f25c0103720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.204+0000 7f25c869d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c010e920 0x7f25c0103c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.204+0000 7f25c869d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25c0114480 con 0x7f25c010e920 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.204+0000 7f25c869d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f25c0114300 con 0x7f25c01054f0 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.204+0000 7f25c869d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25c0114600 con 0x7f25c0105e40 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.204+0000 7f25c6c3a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c010e920 0x7f25c0103c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.204+0000 7f25c6c3a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c010e920 0x7f25c0103c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41870/0 (socket says 172.21.15.46:41870) 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.204+0000 7f25c6c3a700 1 -- 172.21.15.46:0/3453568553 learned_addr learned my addr 172.21.15.46:0/3453568553 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.205+0000 7f25c5c38700 1 --2- 172.21.15.46:0/3453568553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c0105e40 0x7f25c0103720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.205+0000 7f25c6439700 1 --2- 172.21.15.46:0/3453568553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25c01054f0 0x7f25c01031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.205+0000 7f25c6c3a700 1 -- 172.21.15.46:0/3453568553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c0105e40 msgr2=0x7f25c0103720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.205+0000 7f25c6c3a700 1 --2- 172.21.15.46:0/3453568553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c0105e40 0x7f25c0103720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.205+0000 7f25c6c3a700 1 -- 172.21.15.46:0/3453568553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25c01054f0 msgr2=0x7f25c01031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.205+0000 7f25c6c3a700 1 --2- 172.21.15.46:0/3453568553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25c01054f0 0x7f25c01031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.205+0000 7f25c6c3a700 1 -- 172.21.15.46:0/3453568553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25c01a91b0 con 0x7f25c010e920 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.205+0000 7f25c6c3a700 1 --2- 172.21.15.46:0/3453568553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c010e920 0x7f25c0103c60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f25b8007ee0 tx=0x7f25b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:46.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.205+0000 7f25b77fe700 1 -- 172.21.15.46:0/3453568553 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b800dd20 con 0x7f25c010e920 2024-05-09T04:55:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.205+0000 7f25b77fe700 1 -- 172.21.15.46:0/3453568553 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25b8004020 con 0x7f25c010e920 2024-05-09T04:55:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.205+0000 7f25c869d700 1 -- 172.21.15.46:0/3453568553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25c01a94a0 con 0x7f25c010e920 2024-05-09T04:55:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.205+0000 7f25b77fe700 1 -- 172.21.15.46:0/3453568553 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25b8015660 con 0x7f25c010e920 2024-05-09T04:55:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.205+0000 7f25c869d700 1 -- 172.21.15.46:0/3453568553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25c01a99b0 con 0x7f25c010e920 2024-05-09T04:55:46.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.206+0000 7f25c869d700 1 -- 172.21.15.46:0/3453568553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25c0076280 con 0x7f25c010e920 2024-05-09T04:55:46.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.209+0000 7f25b77fe700 1 -- 172.21.15.46:0/3453568553 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f25b80041d0 con 0x7f25c010e920 2024-05-09T04:55:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.210+0000 7f25b77fe700 1 --2- 172.21.15.46:0/3453568553 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f25ac0b2a70 0x7f25ac0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.210+0000 7f25c6439700 1 --2- 172.21.15.46:0/3453568553 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f25ac0b2a70 0x7f25ac0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.210+0000 7f25b77fe700 1 -- 172.21.15.46:0/3453568553 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f25b80e2760 con 0x7f25c010e920 2024-05-09T04:55:46.210 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.210+0000 7f25c6439700 1 --2- 172.21.15.46:0/3453568553 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f25ac0b2a70 0x7f25ac0b4f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f25bc005950 tx=0x7f25bc00b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:46.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.214+0000 7f25b77fe700 1 -- 172.21.15.46:0/3453568553 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f25b8010070 con 0x7f25c010e920 2024-05-09T04:55:46.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.475+0000 7f25c869d700 1 -- 172.21.15.46:0/3453568553 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f25c002e5f0 con 0x7f25c010e920 2024-05-09T04:55:46.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.476+0000 7f25b77fe700 1 -- 172.21.15.46:0/3453568553 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7f25b80aa0c0 con 0x7f25c010e920 2024-05-09T04:55:46.476 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:55:46.476 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:55:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.479+0000 7f25c869d700 1 -- 172.21.15.46:0/3453568553 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f25ac0b2a70 msgr2=0x7f25ac0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.479+0000 7f25c869d700 1 --2- 172.21.15.46:0/3453568553 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f25ac0b2a70 0x7f25ac0b4f80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f25bc005950 tx=0x7f25bc00b410 comp rx=0 tx=0).stop 2024-05-09T04:55:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.479+0000 7f25c869d700 1 -- 172.21.15.46:0/3453568553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c010e920 msgr2=0x7f25c0103c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.479+0000 7f25c869d700 1 --2- 172.21.15.46:0/3453568553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c010e920 0x7f25c0103c60 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f25b8007ee0 tx=0x7f25b800d3b0 comp rx=0 tx=0).stop 2024-05-09T04:55:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.479+0000 7f25c869d700 1 -- 172.21.15.46:0/3453568553 shutdown_connections 2024-05-09T04:55:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.479+0000 7f25c869d700 1 --2- 172.21.15.46:0/3453568553 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f25ac0b2a70 0x7f25ac0b4f80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:46.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.479+0000 7f25c869d700 1 --2- 172.21.15.46:0/3453568553 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f25c010e920 0x7f25c0103c60 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:46.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.479+0000 7f25c869d700 1 --2- 172.21.15.46:0/3453568553 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25c01054f0 0x7f25c01031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:46.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.479+0000 7f25c869d700 1 --2- 172.21.15.46:0/3453568553 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25c0105e40 0x7f25c0103720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:46.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.479+0000 7f25c869d700 1 -- 172.21.15.46:0/3453568553 >> 172.21.15.46:0/3453568553 conn(0x7f25c0078e00 msgr2=0x7f25c0100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:46.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.479+0000 7f25c869d700 1 -- 172.21.15.46:0/3453568553 shutdown_connections 2024-05-09T04:55:46.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:46.479+0000 7f25c869d700 1 -- 172.21.15.46:0/3453568553 wait complete. 2024-05-09T04:55:46.480 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:55:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:46 smithi110 ceph-mon[34904]: pgmap v143: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:46 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3453568553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:55:47.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:46 smithi046 ceph-mon[30136]: pgmap v143: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:47.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:46 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3453568553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:55:47.127 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:55:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:46 smithi072 ceph-mon[34088]: pgmap v143: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:46 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3453568553' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:55:47.380 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:48.904 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:48 smithi046 ceph-mon[30136]: pgmap v144: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:55:49.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:48 smithi110 ceph-mon[34904]: pgmap v144: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:55:49.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:48 smithi072 ceph-mon[34088]: pgmap v144: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:55:49.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.165+0000 7f006e854700 1 -- 172.21.15.46:0/2402776286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0068110bd0 msgr2=0x7f0068113010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:49.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.165+0000 7f006e854700 1 --2- 172.21.15.46:0/2402776286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0068110bd0 0x7f0068113010 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f006000b0a0 tx=0x7f006000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.165+0000 7f006e854700 1 -- 172.21.15.46:0/2402776286 shutdown_connections 2024-05-09T04:55:49.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.165+0000 7f006e854700 1 --2- 172.21.15.46:0/2402776286 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0068110bd0 0x7f0068113010 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.165+0000 7f006e854700 1 --2- 172.21.15.46:0/2402776286 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0068076c30 0x7f0068077040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.165+0000 7f006e854700 1 --2- 172.21.15.46:0/2402776286 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0068077580 0x7f0068110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.165+0000 7f006e854700 1 -- 172.21.15.46:0/2402776286 >> 172.21.15.46:0/2402776286 conn(0x7f00680ffef0 msgr2=0x7f0068102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:49.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.165+0000 7f006e854700 1 -- 172.21.15.46:0/2402776286 shutdown_connections 2024-05-09T04:55:49.166 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.165+0000 7f006e854700 1 -- 172.21.15.46:0/2402776286 wait complete. 2024-05-09T04:55:49.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.166+0000 7f006e854700 1 Processor -- start 2024-05-09T04:55:49.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.166+0000 7f006e854700 1 -- start start 2024-05-09T04:55:49.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.166+0000 7f006e854700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0068076c30 0x7f006819d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.166+0000 7f006e854700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0068077580 0x7f006819d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.166+0000 7f006e854700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0068110bd0 0x7f00681a1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.166+0000 7f006e854700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f00681166e0 con 0x7f0068110bd0 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.167+0000 7f006e854700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0068116560 con 0x7f0068076c30 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.167+0000 7f006e854700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0068116860 con 0x7f0068077580 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.167+0000 7f006cdf1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0068110bd0 0x7f00681a1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.167+0000 7f006cdf1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0068110bd0 0x7f00681a1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41882/0 (socket says 172.21.15.46:41882) 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.167+0000 7f006cdf1700 1 -- 172.21.15.46:0/455230832 learned_addr learned my addr 172.21.15.46:0/455230832 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.167+0000 7f0067fff700 1 --2- 172.21.15.46:0/455230832 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0068076c30 0x7f006819d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.167+0000 7f005ffff700 1 --2- 172.21.15.46:0/455230832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0068077580 0x7f006819d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.167+0000 7f006cdf1700 1 -- 172.21.15.46:0/455230832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0068077580 msgr2=0x7f006819d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.167+0000 7f006cdf1700 1 --2- 172.21.15.46:0/455230832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0068077580 0x7f006819d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.167+0000 7f006cdf1700 1 -- 172.21.15.46:0/455230832 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0068076c30 msgr2=0x7f006819d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.167+0000 7f006cdf1700 1 --2- 172.21.15.46:0/455230832 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0068076c30 0x7f006819d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.167+0000 7f006cdf1700 1 -- 172.21.15.46:0/455230832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00681a1e30 con 0x7f0068110bd0 2024-05-09T04:55:49.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.167+0000 7f006cdf1700 1 --2- 172.21.15.46:0/455230832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0068110bd0 0x7f00681a1840 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f006000b070 tx=0x7f0060004dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:49.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.167+0000 7f0065ffb700 1 -- 172.21.15.46:0/455230832 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060021070 con 0x7f0068110bd0 2024-05-09T04:55:49.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.167+0000 7f0065ffb700 1 -- 172.21.15.46:0/455230832 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0060007e30 con 0x7f0068110bd0 2024-05-09T04:55:49.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.168+0000 7f0065ffb700 1 -- 172.21.15.46:0/455230832 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00600046f0 con 0x7f0068110bd0 2024-05-09T04:55:49.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.168+0000 7f006e854700 1 -- 172.21.15.46:0/455230832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00681a20c0 con 0x7f0068110bd0 2024-05-09T04:55:49.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.168+0000 7f006e854700 1 -- 172.21.15.46:0/455230832 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00681a2600 con 0x7f0068110bd0 2024-05-09T04:55:49.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.170+0000 7f0065ffb700 1 -- 172.21.15.46:0/455230832 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0060007980 con 0x7f0068110bd0 2024-05-09T04:55:49.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.171+0000 7f0065ffb700 1 --2- 172.21.15.46:0/455230832 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f00540b2a70 0x7f00540b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.171+0000 7f0067fff700 1 --2- 172.21.15.46:0/455230832 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f00540b2a70 0x7f00540b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.171+0000 7f0065ffb700 1 -- 172.21.15.46:0/455230832 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f006000f020 con 0x7f0068110bd0 2024-05-09T04:55:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.172+0000 7f007d26ab80 1 Processor -- start 2024-05-09T04:55:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.172+0000 7f0067fff700 1 --2- 172.21.15.46:0/455230832 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f00540b2a70 0x7f00540b4f80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0068078ce0 tx=0x7f005800a3b0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:49.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.172+0000 7f007d26ab80 1 -- start start 2024-05-09T04:55:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.172+0000 7f007d26ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40ca3d00 0x55fb40dac240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.172+0000 7f007d26ab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40c294a0 0x55fb40dac780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.172+0000 7f007d26ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40da6400 0x55fb40da6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.172+0000 7f007d26ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fb40cc90a0 con 0x55fb40da6400 2024-05-09T04:55:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.172+0000 7f007d26ab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55fb40c6e9a0 con 0x55fb40c294a0 2024-05-09T04:55:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.172+0000 7f007d26ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fb40cc5060 con 0x55fb40ca3d00 2024-05-09T04:55:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.172+0000 7f006cdf1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40da6400 0x55fb40da6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.172+0000 7f006cdf1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40da6400 0x55fb40da6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41898/0 (socket says 172.21.15.46:41898) 2024-05-09T04:55:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.172+0000 7f006cdf1700 1 -- 172.21.15.46:0/1897238700 learned_addr learned my addr 172.21.15.46:0/1897238700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.173+0000 7f005ffff700 1 --2- 172.21.15.46:0/1897238700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40c294a0 0x55fb40dac780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.173+0000 7f0067fff700 1 --2- 172.21.15.46:0/1897238700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40ca3d00 0x55fb40dac240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.173+0000 7f006cdf1700 1 -- 172.21.15.46:0/1897238700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40ca3d00 msgr2=0x55fb40dac240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.173+0000 7f006cdf1700 1 --2- 172.21.15.46:0/1897238700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40ca3d00 0x55fb40dac240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.173+0000 7f006cdf1700 1 -- 172.21.15.46:0/1897238700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40c294a0 msgr2=0x55fb40dac780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:49.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.173+0000 7f006cdf1700 1 --2- 172.21.15.46:0/1897238700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40c294a0 0x55fb40dac780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.173+0000 7f006cdf1700 1 -- 172.21.15.46:0/1897238700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb40c5f500 con 0x55fb40da6400 2024-05-09T04:55:49.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.173+0000 7f006cdf1700 1 --2- 172.21.15.46:0/1897238700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40da6400 0x55fb40da6810 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f00600a6f00 tx=0x7f006000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a271d3a934cfa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:49.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.173+0000 7f005e7fc700 1 -- 172.21.15.46:0/1897238700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0060007c70 con 0x55fb40da6400 2024-05-09T04:55:49.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.173+0000 7f005e7fc700 1 -- 172.21.15.46:0/1897238700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0060004890 con 0x55fb40da6400 2024-05-09T04:55:49.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.173+0000 7f005e7fc700 1 -- 172.21.15.46:0/1897238700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00600ab2e0 con 0x55fb40da6400 2024-05-09T04:55:49.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.174+0000 7f007d26ab80 1 -- 172.21.15.46:0/1897238700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40da6400 msgr2=0x55fb40da6810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:49.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.174+0000 7f007d26ab80 1 --2- 172.21.15.46:0/1897238700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40da6400 0x55fb40da6810 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f00600a6f00 tx=0x7f006000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.174+0000 7f007d26ab80 1 -- 172.21.15.46:0/1897238700 shutdown_connections 2024-05-09T04:55:49.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.174+0000 7f007d26ab80 1 --2- 172.21.15.46:0/1897238700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40da6400 0x55fb40da6810 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.174+0000 7f007d26ab80 1 --2- 172.21.15.46:0/1897238700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40ca3d00 0x55fb40dac240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.174+0000 7f007d26ab80 1 --2- 172.21.15.46:0/1897238700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40c294a0 0x55fb40dac780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.174+0000 7f007d26ab80 1 -- 172.21.15.46:0/1897238700 >> 172.21.15.46:0/1897238700 conn(0x55fb40c2cf70 msgr2=0x55fb40da1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:49.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.174+0000 7f007d26ab80 1 -- 172.21.15.46:0/1897238700 shutdown_connections 2024-05-09T04:55:49.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.174+0000 7f007d26ab80 1 -- 172.21.15.46:0/1897238700 wait complete. 2024-05-09T04:55:49.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.175+0000 7f007d26ab80 1 -- start start 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.175+0000 7f007d26ab80 1 Processor -- start 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.175+0000 7f007d26ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40c294a0 0x55fb40db5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.175+0000 7f007d26ab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40ca3d00 0x55fb40db60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.175+0000 7f007d26ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40da6400 0x55fb40daff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.175+0000 7f007d26ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fb40da6fb0 con 0x55fb40da6400 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.175+0000 7f007d26ab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55fb40c63b20 con 0x55fb40ca3d00 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.175+0000 7f007d26ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fb40da7130 con 0x55fb40c294a0 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.175+0000 7f007d26ab80 10 client.0 _ll_register_callbacks cb 0x55fb40b3a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.175+0000 7f006cdf1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40da6400 0x55fb40daff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.175+0000 7f006cdf1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40da6400 0x55fb40daff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41904/0 (socket says 172.21.15.46:41904) 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.175+0000 7f006cdf1700 1 -- 172.21.15.46:0/107407870 learned_addr learned my addr 172.21.15.46:0/107407870 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.175+0000 7f0067fff700 1 --2- 172.21.15.46:0/107407870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40c294a0 0x55fb40db5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f005ffff700 1 --2- 172.21.15.46:0/107407870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40ca3d00 0x55fb40db60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f006cdf1700 1 -- 172.21.15.46:0/107407870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40c294a0 msgr2=0x55fb40db5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f006cdf1700 1 --2- 172.21.15.46:0/107407870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40c294a0 0x55fb40db5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f006cdf1700 1 -- 172.21.15.46:0/107407870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40ca3d00 msgr2=0x55fb40db60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:49.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f006cdf1700 1 --2- 172.21.15.46:0/107407870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40ca3d00 0x55fb40db60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f006cdf1700 1 -- 172.21.15.46:0/107407870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb40b44490 con 0x55fb40da6400 2024-05-09T04:55:49.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f0067fff700 1 --2- 172.21.15.46:0/107407870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40c294a0 0x55fb40db5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:49.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f006cdf1700 1 --2- 172.21.15.46:0/107407870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40da6400 0x55fb40daff60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f00600041d0 tx=0x7f00600ab440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7878daaaab0e200 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:49.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f00477fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:49.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f00477fe700 1 -- 172.21.15.46:0/107407870 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00600c9070 con 0x55fb40da6400 2024-05-09T04:55:49.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f00477fe700 1 -- 172.21.15.46:0/107407870 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f00600b39d0 con 0x55fb40da6400 2024-05-09T04:55:49.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f007d26ab80 1 -- 172.21.15.46:0/107407870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55fb40cd6a80 con 0x55fb40da6400 2024-05-09T04:55:49.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f00477fe700 1 -- 172.21.15.46:0/107407870 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0060007c70 con 0x55fb40da6400 2024-05-09T04:55:49.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f007d26ab80 10 client.16485 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:49.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f007d26ab80 1 -- 172.21.15.46:0/107407870 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb40da6fb0 con 0x55fb40da6400 2024-05-09T04:55:49.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.176+0000 7f00477fe700 1 -- 172.21.15.46:0/107407870 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55fb40cd6a80 con 0x55fb40da6400 2024-05-09T04:55:49.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.177+0000 7f00477fe700 1 -- 172.21.15.46:0/107407870 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f0060011070 con 0x55fb40da6400 2024-05-09T04:55:49.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.177+0000 7f007d26ab80 10 client.16485 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:49.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.177+0000 7f007d26ab80 10 client.16485 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:55:49.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.177+0000 7f007d26ab80 20 client.16485 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:49.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.177+0000 7f007d26ab80 1 --2- 172.21.15.46:0/107407870 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55fb40dbfb70 0x55fb40dc1f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.177+0000 7f007d26ab80 4 client.16485 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:55:49.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.177+0000 7f007d26ab80 1 -- 172.21.15.46:0/107407870 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb40cd0b20 con 0x55fb40dbfb70 2024-05-09T04:55:49.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.177+0000 7f0067fff700 1 --2- 172.21.15.46:0/107407870 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55fb40dbfb70 0x55fb40dc1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.177+0000 7f0067fff700 1 --2- 172.21.15.46:0/107407870 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55fb40dbfb70 0x55fb40dc1f90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44ecc7fc16c91024 server_cookie=2a54e3312ccc6de2 in_seq=0 out_seq=0 2024-05-09T04:55:49.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.177+0000 7f00477fe700 10 client.16485 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:55:49.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.178+0000 7f00477fe700 1 -- 172.21.15.46:0/107407870 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fb40cd0b20 con 0x55fb40dbfb70 2024-05-09T04:55:49.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.178+0000 7f00477fe700 10 client.16485 handle_command_reply: tid=0 2024-05-09T04:55:49.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.178+0000 7f007d26ab80 1 client.16485 shutdown 2024-05-09T04:55:49.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.178+0000 7f007d26ab80 1 -- 172.21.15.46:0/107407870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40da6400 msgr2=0x55fb40daff60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:49.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.178+0000 7f007d26ab80 1 --2- 172.21.15.46:0/107407870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40da6400 0x55fb40daff60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f00600041d0 tx=0x7f00600ab440 comp rx=0 tx=0).stop 2024-05-09T04:55:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.178+0000 7f007d26ab80 1 -- 172.21.15.46:0/107407870 shutdown_connections 2024-05-09T04:55:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.178+0000 7f007d26ab80 1 --2- 172.21.15.46:0/107407870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40da6400 0x55fb40daff60 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.178+0000 7f007d26ab80 1 --2- 172.21.15.46:0/107407870 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55fb40dbfb70 0x55fb40dc1f90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.178+0000 7f007d26ab80 1 --2- 172.21.15.46:0/107407870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40c294a0 0x55fb40db5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.178+0000 7f007d26ab80 1 --2- 172.21.15.46:0/107407870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40ca3d00 0x55fb40db60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.178+0000 7f007d26ab80 1 -- 172.21.15.46:0/107407870 >> 172.21.15.46:0/107407870 conn(0x55fb40c2cf70 msgr2=0x55fb40d9aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.179+0000 7f00477fe700 0 client.16485 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:55:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.179+0000 7f007d26ab80 1 -- 172.21.15.46:0/107407870 shutdown_connections 2024-05-09T04:55:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.179+0000 7f007d26ab80 1 -- 172.21.15.46:0/107407870 wait complete. 2024-05-09T04:55:49.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.179+0000 7f007d26ab80 20 client.16485 trim_cache size 0 max 16384 2024-05-09T04:55:49.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.194+0000 7f007d26ab80 1 Processor -- start 2024-05-09T04:55:49.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.194+0000 7f007d26ab80 1 -- start start 2024-05-09T04:55:49.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.194+0000 7f007d26ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40c294a0 0x55fb40daff20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.194+0000 7f007d26ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40ca3d00 0x55fb40dc7770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.194+0000 7f007d26ab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40da6400 0x55fb40dc7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.194+0000 7f007d26ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f00681b38c0 con 0x55fb40ca3d00 2024-05-09T04:55:49.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.194+0000 7f007d26ab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f00681b1150 con 0x55fb40da6400 2024-05-09T04:55:49.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.194+0000 7f007d26ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f00681aefa0 con 0x55fb40c294a0 2024-05-09T04:55:49.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.194+0000 7f005ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40ca3d00 0x55fb40dc7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.194+0000 7f005ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40ca3d00 0x55fb40dc7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41916/0 (socket says 172.21.15.46:41916) 2024-05-09T04:55:49.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.194+0000 7f005ffff700 1 -- 172.21.15.46:0/944045916 learned_addr learned my addr 172.21.15.46:0/944045916 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:49.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.195+0000 7f006cdf1700 1 --2- 172.21.15.46:0/944045916 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40da6400 0x55fb40dc7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.195+0000 7f0067fff700 1 --2- 172.21.15.46:0/944045916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40c294a0 0x55fb40daff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.195+0000 7f005ffff700 1 -- 172.21.15.46:0/944045916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40c294a0 msgr2=0x55fb40daff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:49.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.195+0000 7f005ffff700 1 --2- 172.21.15.46:0/944045916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40c294a0 0x55fb40daff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.195+0000 7f005ffff700 1 -- 172.21.15.46:0/944045916 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40da6400 msgr2=0x55fb40dc7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:49.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.195+0000 7f005ffff700 1 --2- 172.21.15.46:0/944045916 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40da6400 0x55fb40dc7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.195+0000 7f005ffff700 1 -- 172.21.15.46:0/944045916 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb40d9d9d0 con 0x55fb40ca3d00 2024-05-09T04:55:49.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.195+0000 7f0067fff700 1 --2- 172.21.15.46:0/944045916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40c294a0 0x55fb40daff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:55:49.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.195+0000 7f005ffff700 1 --2- 172.21.15.46:0/944045916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40ca3d00 0x55fb40dc7770 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0050007d50 tx=0x7f0050011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd607fa59f3aae06 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:49.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.195+0000 7f005dffb700 1 -- 172.21.15.46:0/944045916 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0050020070 con 0x55fb40ca3d00 2024-05-09T04:55:49.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.195+0000 7f005dffb700 1 -- 172.21.15.46:0/944045916 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0050004500 con 0x55fb40ca3d00 2024-05-09T04:55:49.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.195+0000 7f005dffb700 1 -- 172.21.15.46:0/944045916 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0050003e60 con 0x55fb40ca3d00 2024-05-09T04:55:49.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.196+0000 7f007d26ab80 1 -- 172.21.15.46:0/944045916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40ca3d00 msgr2=0x55fb40dc7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:49.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.196+0000 7f007d26ab80 1 --2- 172.21.15.46:0/944045916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40ca3d00 0x55fb40dc7770 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f0050007d50 tx=0x7f0050011720 comp rx=0 tx=0).stop 2024-05-09T04:55:49.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.196+0000 7f007d26ab80 1 -- 172.21.15.46:0/944045916 shutdown_connections 2024-05-09T04:55:49.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.196+0000 7f007d26ab80 1 --2- 172.21.15.46:0/944045916 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40da6400 0x55fb40dc7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.196+0000 7f007d26ab80 1 --2- 172.21.15.46:0/944045916 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40c294a0 0x55fb40daff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.196+0000 7f007d26ab80 1 --2- 172.21.15.46:0/944045916 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40ca3d00 0x55fb40dc7770 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.196+0000 7f007d26ab80 1 -- 172.21.15.46:0/944045916 >> 172.21.15.46:0/944045916 conn(0x55fb40c2cf70 msgr2=0x55fb40daf810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:49.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.196+0000 7f007d26ab80 1 -- 172.21.15.46:0/944045916 shutdown_connections 2024-05-09T04:55:49.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.196+0000 7f007d26ab80 1 -- 172.21.15.46:0/944045916 wait complete. 2024-05-09T04:55:49.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.196+0000 7f007d26ab80 1 -- start start 2024-05-09T04:55:49.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f007d26ab80 1 Processor -- start 2024-05-09T04:55:49.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f007d26ab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40c294a0 0x55fb40db8db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f007d26ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40ca3d00 0x55fb40db92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f007d26ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40da6400 0x55fb40db9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f007d26ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f00681af610 con 0x55fb40ca3d00 2024-05-09T04:55:49.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f007d26ab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f00681b0a80 con 0x55fb40c294a0 2024-05-09T04:55:49.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f007d26ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f00681afce0 con 0x55fb40da6400 2024-05-09T04:55:49.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f007d26ab80 10 client.0 _ll_register_callbacks cb 0x55fb40b3a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:49.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f005ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40ca3d00 0x55fb40db92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f0067fff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40c294a0 0x55fb40db8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f005ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40ca3d00 0x55fb40db92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41920/0 (socket says 172.21.15.46:41920) 2024-05-09T04:55:49.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f005ffff700 1 -- 172.21.15.46:0/1558294404 learned_addr learned my addr 172.21.15.46:0/1558294404 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f006cdf1700 1 --2- 172.21.15.46:0/1558294404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40da6400 0x55fb40db9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f005ffff700 1 -- 172.21.15.46:0/1558294404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40da6400 msgr2=0x55fb40db9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f005ffff700 1 --2- 172.21.15.46:0/1558294404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40da6400 0x55fb40db9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f005ffff700 1 -- 172.21.15.46:0/1558294404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40c294a0 msgr2=0x55fb40db8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f005ffff700 1 --2- 172.21.15.46:0/1558294404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40c294a0 0x55fb40db8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f005ffff700 1 -- 172.21.15.46:0/1558294404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb40db6590 con 0x55fb40ca3d00 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.197+0000 7f0067fff700 1 --2- 172.21.15.46:0/1558294404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40c294a0 0x55fb40db8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f005ffff700 1 --2- 172.21.15.46:0/1558294404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40ca3d00 0x55fb40db92f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0050005fa0 tx=0x7f00500039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87a76bd8e2978e6b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f00477fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f00477fe700 1 -- 172.21.15.46:0/1558294404 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0050020030 con 0x55fb40ca3d00 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f00477fe700 1 -- 172.21.15.46:0/1558294404 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f0050004250 con 0x55fb40ca3d00 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f007d26ab80 1 -- 172.21.15.46:0/1558294404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0050004500 con 0x55fb40ca3d00 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f00477fe700 1 -- 172.21.15.46:0/1558294404 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0050033050 con 0x55fb40ca3d00 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f007d26ab80 10 client.16491 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f007d26ab80 1 -- 172.21.15.46:0/1558294404 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fb40e29530 con 0x55fb40ca3d00 2024-05-09T04:55:49.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f00477fe700 1 -- 172.21.15.46:0/1558294404 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0050004500 con 0x55fb40ca3d00 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f00477fe700 1 -- 172.21.15.46:0/1558294404 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f005002e030 con 0x55fb40ca3d00 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f007d26ab80 10 client.16491 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f007d26ab80 10 client.16491 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f007d26ab80 20 client.16491 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f007d26ab80 1 --2- 172.21.15.46:0/1558294404 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55fb40e29b30 0x55fb40e2bf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f007d26ab80 4 client.16491 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.198+0000 7f007d26ab80 1 -- 172.21.15.46:0/1558294404 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb40b44970 con 0x55fb40e29b30 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.199+0000 7f0067fff700 1 --2- 172.21.15.46:0/1558294404 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55fb40e29b30 0x55fb40e2bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.199+0000 7f00477fe700 10 client.16491 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.199+0000 7f0067fff700 1 --2- 172.21.15.46:0/1558294404 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55fb40e29b30 0x55fb40e2bf50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c2560711b27f5763 server_cookie=f35496471f5d9141 in_seq=0 out_seq=0 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.200+0000 7f00477fe700 1 -- 172.21.15.46:0/1558294404 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb40b44970 con 0x55fb40e29b30 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.200+0000 7f00477fe700 10 client.16491 handle_command_reply: tid=0 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.200+0000 7f007d26ab80 1 client.16491 shutdown 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.200+0000 7f007d26ab80 1 -- 172.21.15.46:0/1558294404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40ca3d00 msgr2=0x55fb40db92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.200+0000 7f007d26ab80 1 --2- 172.21.15.46:0/1558294404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40ca3d00 0x55fb40db92f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f0050005fa0 tx=0x7f00500039e0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.200+0000 7f007d26ab80 1 -- 172.21.15.46:0/1558294404 shutdown_connections 2024-05-09T04:55:49.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.200+0000 7f007d26ab80 1 --2- 172.21.15.46:0/1558294404 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55fb40e29b30 0x55fb40e2bf50 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.200+0000 7f007d26ab80 1 --2- 172.21.15.46:0/1558294404 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fb40da6400 0x55fb40db9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.200+0000 7f007d26ab80 1 --2- 172.21.15.46:0/1558294404 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fb40c294a0 0x55fb40db8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.200+0000 7f007d26ab80 1 --2- 172.21.15.46:0/1558294404 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fb40ca3d00 0x55fb40db92f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.200+0000 7f007d26ab80 1 -- 172.21.15.46:0/1558294404 >> 172.21.15.46:0/1558294404 conn(0x55fb40c2cf70 msgr2=0x55fb40da9cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:49.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.200+0000 7f00477fe700 0 client.16491 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:55:49.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.200+0000 7f007d26ab80 1 -- 172.21.15.46:0/1558294404 shutdown_connections 2024-05-09T04:55:49.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.200+0000 7f007d26ab80 1 -- 172.21.15.46:0/1558294404 wait complete. 2024-05-09T04:55:49.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.200+0000 7f007d26ab80 20 client.16491 trim_cache size 0 max 16384 2024-05-09T04:55:49.205 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:55:49.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.201+0000 7f006e854700 1 -- 172.21.15.46:0/455230832 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f00540b2a70 msgr2=0x7f00540b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:49.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.201+0000 7f006e854700 1 --2- 172.21.15.46:0/455230832 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f00540b2a70 0x7f00540b4f80 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0068078ce0 tx=0x7f005800a3b0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.201+0000 7f006e854700 1 -- 172.21.15.46:0/455230832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0068110bd0 msgr2=0x7f00681a1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:49.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.201+0000 7f006e854700 1 --2- 172.21.15.46:0/455230832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0068110bd0 0x7f00681a1840 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f006000b070 tx=0x7f0060004dc0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.201+0000 7f006e854700 1 -- 172.21.15.46:0/455230832 shutdown_connections 2024-05-09T04:55:49.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.201+0000 7f006e854700 1 --2- 172.21.15.46:0/455230832 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f00540b2a70 0x7f00540b4f80 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.201+0000 7f006e854700 1 --2- 172.21.15.46:0/455230832 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0068110bd0 0x7f00681a1840 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.201+0000 7f006e854700 1 --2- 172.21.15.46:0/455230832 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0068076c30 0x7f006819d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.201+0000 7f006e854700 1 --2- 172.21.15.46:0/455230832 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0068077580 0x7f006819d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:49.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.201+0000 7f006e854700 1 -- 172.21.15.46:0/455230832 >> 172.21.15.46:0/455230832 conn(0x7f00680ffef0 msgr2=0x7f0068100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:49.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.201+0000 7f006e854700 1 -- 172.21.15.46:0/455230832 shutdown_connections 2024-05-09T04:55:49.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:49.201+0000 7f006e854700 1 -- 172.21.15.46:0/455230832 wait complete. 2024-05-09T04:55:50.010 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:55:50.265 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:51.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:50 smithi110 ceph-mon[34904]: pgmap v145: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:51.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:50 smithi046 ceph-mon[30136]: pgmap v145: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:51.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:50 smithi072 ceph-mon[34088]: pgmap v145: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.145+0000 7f58969e8700 1 -- 172.21.15.46:0/1245571764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5890107df0 msgr2=0x7f5890108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.145+0000 7f58969e8700 1 --2- 172.21.15.46:0/1245571764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5890107df0 0x7f5890108200 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f58780099c0 tx=0x7f5878009cd0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.145+0000 7f58969e8700 1 -- 172.21.15.46:0/1245571764 shutdown_connections 2024-05-09T04:55:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.145+0000 7f58969e8700 1 --2- 172.21.15.46:0/1245571764 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5890108740 0x7f58901100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.145+0000 7f58969e8700 1 --2- 172.21.15.46:0/1245571764 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58901034c0 0x7f58901038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.145+0000 7f58969e8700 1 --2- 172.21.15.46:0/1245571764 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5890107df0 0x7f5890108200 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.145+0000 7f58969e8700 1 -- 172.21.15.46:0/1245571764 >> 172.21.15.46:0/1245571764 conn(0x7f58900fedc0 msgr2=0x7f58901011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:52.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.145+0000 7f58969e8700 1 -- 172.21.15.46:0/1245571764 shutdown_connections 2024-05-09T04:55:52.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.146+0000 7f58969e8700 1 -- 172.21.15.46:0/1245571764 wait complete. 2024-05-09T04:55:52.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.146+0000 7f58969e8700 1 Processor -- start 2024-05-09T04:55:52.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.146+0000 7f58969e8700 1 -- start start 2024-05-09T04:55:52.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f58969e8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58901034c0 0x7f589019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f58969e8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5890107df0 0x7f589019b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f58969e8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5890108740 0x7f589019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f58969e8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5890113350 con 0x7f5890108740 2024-05-09T04:55:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f58969e8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f58901131d0 con 0x7f5890107df0 2024-05-09T04:55:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f58969e8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f58901134d0 con 0x7f58901034c0 2024-05-09T04:55:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f5894f85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5890108740 0x7f589019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f5894f85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5890108740 0x7f589019f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41948/0 (socket says 172.21.15.46:41948) 2024-05-09T04:55:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f5894f85700 1 -- 172.21.15.46:0/3313664433 learned_addr learned my addr 172.21.15.46:0/3313664433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f5887fff700 1 --2- 172.21.15.46:0/3313664433 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5890107df0 0x7f589019b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f588ffff700 1 --2- 172.21.15.46:0/3313664433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58901034c0 0x7f589019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f5894f85700 1 -- 172.21.15.46:0/3313664433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58901034c0 msgr2=0x7f589019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f5894f85700 1 --2- 172.21.15.46:0/3313664433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58901034c0 0x7f589019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f5894f85700 1 -- 172.21.15.46:0/3313664433 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5890107df0 msgr2=0x7f589019b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f5894f85700 1 --2- 172.21.15.46:0/3313664433 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5890107df0 0x7f589019b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f5894f85700 1 -- 172.21.15.46:0/3313664433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f589019fbf0 con 0x7f5890108740 2024-05-09T04:55:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.147+0000 7f5894f85700 1 --2- 172.21.15.46:0/3313664433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5890108740 0x7f589019f600 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5888007ee0 tx=0x7f588800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.148+0000 7f588dffb700 1 -- 172.21.15.46:0/3313664433 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5888004bb0 con 0x7f5890108740 2024-05-09T04:55:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.148+0000 7f58969e8700 1 -- 172.21.15.46:0/3313664433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f589019fee0 con 0x7f5890108740 2024-05-09T04:55:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.148+0000 7f588dffb700 1 -- 172.21.15.46:0/3313664433 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5888004d50 con 0x7f5890108740 2024-05-09T04:55:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.148+0000 7f588dffb700 1 -- 172.21.15.46:0/3313664433 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5888005920 con 0x7f5890108740 2024-05-09T04:55:52.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.148+0000 7f58969e8700 1 -- 172.21.15.46:0/3313664433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58901a03a0 con 0x7f5890108740 2024-05-09T04:55:52.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.151+0000 7f588dffb700 1 -- 172.21.15.46:0/3313664433 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f588801a020 con 0x7f5890108740 2024-05-09T04:55:52.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.152+0000 7f58a53feb80 1 Processor -- start 2024-05-09T04:55:52.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.152+0000 7f58a53feb80 1 -- start start 2024-05-09T04:55:52.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.152+0000 7f588dffb700 1 --2- 172.21.15.46:0/3313664433 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f587c0b29f0 0x7f587c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.152+0000 7f58a53feb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b686160 0x558f1b71f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.152+0000 7f5887fff700 1 --2- 172.21.15.46:0/3313664433 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f587c0b29f0 0x7f587c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.152+0000 7f58a53feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b59c4a0 0x558f1b71fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.152+0000 7f58a53feb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b7196a0 0x558f1b719ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.152+0000 7f58a53feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558f1b63bff0 con 0x558f1b59c4a0 2024-05-09T04:55:52.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.152+0000 7f58a53feb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558f1b5da7f0 con 0x558f1b686160 2024-05-09T04:55:52.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.152+0000 7f58a53feb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558f1b637fb0 con 0x558f1b7196a0 2024-05-09T04:55:52.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.152+0000 7f5887fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b59c4a0 0x558f1b71fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.152+0000 7f588dffb700 1 -- 172.21.15.46:0/3313664433 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f58880e1cb0 con 0x7f5890108740 2024-05-09T04:55:52.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.152+0000 7f5887fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b59c4a0 0x558f1b71fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41962/0 (socket says 172.21.15.46:41962) 2024-05-09T04:55:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.152+0000 7f5887fff700 1 -- 172.21.15.46:0/782028599 learned_addr learned my addr 172.21.15.46:0/782028599 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.152+0000 7f588ffff700 1 --2- 172.21.15.46:0/782028599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b686160 0x558f1b71f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.153+0000 7f5894f85700 1 --2- 172.21.15.46:0/782028599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b7196a0 0x558f1b719ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.153+0000 7f5887fff700 1 -- 172.21.15.46:0/782028599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b7196a0 msgr2=0x558f1b719ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.153+0000 7f5887fff700 1 --2- 172.21.15.46:0/782028599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b7196a0 0x558f1b719ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.153+0000 7f5887fff700 1 -- 172.21.15.46:0/782028599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b686160 msgr2=0x558f1b71f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.153+0000 7f5887fff700 1 --2- 172.21.15.46:0/782028599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b686160 0x558f1b71f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.153+0000 7f5887fff700 1 -- 172.21.15.46:0/782028599 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f1b5d2370 con 0x558f1b59c4a0 2024-05-09T04:55:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.153+0000 7f588ffff700 1 --2- 172.21.15.46:0/782028599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b686160 0x558f1b71f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.153+0000 7f5887fff700 1 --2- 172.21.15.46:0/782028599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b59c4a0 0x558f1b71fa30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f58780099c0 tx=0x7f5878027000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89bd90e851f4f3e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.153+0000 7f5887fff700 1 --2- 172.21.15.46:0/3313664433 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f587c0b29f0 0x7f587c0b4f00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5878009990 tx=0x7f5878009690 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.153+0000 7f58867fc700 1 -- 172.21.15.46:0/782028599 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587800b8b0 con 0x558f1b59c4a0 2024-05-09T04:55:52.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.153+0000 7f58867fc700 1 -- 172.21.15.46:0/782028599 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5878004d20 con 0x558f1b59c4a0 2024-05-09T04:55:52.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.153+0000 7f58867fc700 1 -- 172.21.15.46:0/782028599 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878005590 con 0x558f1b59c4a0 2024-05-09T04:55:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.154+0000 7f58a53feb80 1 -- 172.21.15.46:0/782028599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b59c4a0 msgr2=0x558f1b71fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.154+0000 7f58a53feb80 1 --2- 172.21.15.46:0/782028599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b59c4a0 0x558f1b71fa30 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f58780099c0 tx=0x7f5878027000 comp rx=0 tx=0).stop 2024-05-09T04:55:52.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.154+0000 7f58a53feb80 1 -- 172.21.15.46:0/782028599 shutdown_connections 2024-05-09T04:55:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.154+0000 7f58a53feb80 1 --2- 172.21.15.46:0/782028599 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b7196a0 0x558f1b719ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.154+0000 7f58a53feb80 1 --2- 172.21.15.46:0/782028599 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b686160 0x558f1b71f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.154+0000 7f58a53feb80 1 --2- 172.21.15.46:0/782028599 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b59c4a0 0x558f1b71fa30 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.154+0000 7f58a53feb80 1 -- 172.21.15.46:0/782028599 >> 172.21.15.46:0/782028599 conn(0x558f1b5fc6a0 msgr2=0x558f1b714400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.154+0000 7f58a53feb80 1 -- 172.21.15.46:0/782028599 shutdown_connections 2024-05-09T04:55:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.154+0000 7f58a53feb80 1 -- 172.21.15.46:0/782028599 wait complete. 2024-05-09T04:55:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.155+0000 7f58a53feb80 1 -- start start 2024-05-09T04:55:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.155+0000 7f58a53feb80 1 Processor -- start 2024-05-09T04:55:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.155+0000 7f58a53feb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b59c4a0 0x558f1b723230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.155+0000 7f58a53feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b723770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.155+0000 7f58a53feb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b7196a0 0x558f1b723cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.155+0000 7f58a53feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x558f1b4b7ba0 con 0x558f1b686160 2024-05-09T04:55:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.155+0000 7f58a53feb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x558f1b5f6150 con 0x558f1b7196a0 2024-05-09T04:55:52.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.155+0000 7f58a53feb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x558f1b7261d0 con 0x558f1b59c4a0 2024-05-09T04:55:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.155+0000 7f58a53feb80 10 client.0 _ll_register_callbacks cb 0x558f1b4ad6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f5887fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b723770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f5887fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b723770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41968/0 (socket says 172.21.15.46:41968) 2024-05-09T04:55:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f5887fff700 1 -- 172.21.15.46:0/3677947016 learned_addr learned my addr 172.21.15.46:0/3677947016 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f588ffff700 1 --2- 172.21.15.46:0/3677947016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b59c4a0 0x558f1b723230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f5894f85700 1 --2- 172.21.15.46:0/3677947016 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b7196a0 0x558f1b723cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f588ffff700 1 -- 172.21.15.46:0/3677947016 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b7196a0 msgr2=0x558f1b723cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f588ffff700 1 --2- 172.21.15.46:0/3677947016 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b7196a0 0x558f1b723cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f588ffff700 1 -- 172.21.15.46:0/3677947016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 msgr2=0x558f1b723770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f588ffff700 1 --2- 172.21.15.46:0/3677947016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b723770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f588ffff700 1 -- 172.21.15.46:0/3677947016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f1b4b7490 con 0x558f1b59c4a0 2024-05-09T04:55:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f5887fff700 1 --2- 172.21.15.46:0/3677947016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b723770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f588ffff700 1 --2- 172.21.15.46:0/3677947016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b59c4a0 0x558f1b723230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5880010a50 tx=0x7f5880010d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=837e8c88c480cb61 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f586f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:55:52.161 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f586f7fe700 1 -- 172.21.15.46:0/3677947016 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f588000fb60 con 0x558f1b59c4a0 2024-05-09T04:55:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f58a53feb80 1 -- 172.21.15.46:0/3677947016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5878004d20 con 0x558f1b59c4a0 2024-05-09T04:55:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.156+0000 7f586f7fe700 1 -- 172.21.15.46:0/3677947016 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f588000fd00 con 0x558f1b59c4a0 2024-05-09T04:55:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.157+0000 7f586f7fe700 1 -- 172.21.15.46:0/3677947016 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5880004030 con 0x558f1b59c4a0 2024-05-09T04:55:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.157+0000 7f58a53feb80 10 client.25996 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.157+0000 7f58a53feb80 1 -- 172.21.15.46:0/3677947016 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558f1b730c20 con 0x558f1b59c4a0 2024-05-09T04:55:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.157+0000 7f586f7fe700 1 -- 172.21.15.46:0/3677947016 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5878004d20 con 0x558f1b59c4a0 2024-05-09T04:55:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.157+0000 7f586f7fe700 1 -- 172.21.15.46:0/3677947016 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5880005020 con 0x558f1b59c4a0 2024-05-09T04:55:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.157+0000 7f58a53feb80 10 client.25996 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.157+0000 7f58a53feb80 10 client.25996 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:55:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.157+0000 7f58a53feb80 20 client.25996 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.157+0000 7f58a53feb80 1 --2- 172.21.15.46:0/3677947016 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x558f1b733250 0x558f1b7356c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.157+0000 7f58a53feb80 4 client.25996 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:55:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.157+0000 7f58a53feb80 1 -- 172.21.15.46:0/3677947016 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558f1b6499d0 con 0x558f1b733250 2024-05-09T04:55:52.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.158+0000 7f5887fff700 1 --2- 172.21.15.46:0/3677947016 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x558f1b733250 0x558f1b7356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.158+0000 7f5887fff700 1 --2- 172.21.15.46:0/3677947016 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x558f1b733250 0x558f1b7356c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2f9b0040141f3a server_cookie=cfb958c464039bdd in_seq=0 out_seq=0 2024-05-09T04:55:52.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.158+0000 7f586f7fe700 10 client.25996 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:55:52.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.159+0000 7f586f7fe700 1 -- 172.21.15.46:0/3677947016 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558f1b6499d0 con 0x558f1b733250 2024-05-09T04:55:52.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.159+0000 7f586f7fe700 10 client.25996 handle_command_reply: tid=0 2024-05-09T04:55:52.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.159+0000 7f58a53feb80 1 client.25996 shutdown 2024-05-09T04:55:52.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.159+0000 7f58a53feb80 1 -- 172.21.15.46:0/3677947016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b59c4a0 msgr2=0x558f1b723230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:52.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.159+0000 7f58a53feb80 1 --2- 172.21.15.46:0/3677947016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b59c4a0 0x558f1b723230 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5880010a50 tx=0x7f5880010d60 comp rx=0 tx=0).stop 2024-05-09T04:55:52.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.159+0000 7f58a53feb80 1 -- 172.21.15.46:0/3677947016 shutdown_connections 2024-05-09T04:55:52.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.159+0000 7f58a53feb80 1 --2- 172.21.15.46:0/3677947016 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x558f1b733250 0x558f1b7356c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.159+0000 7f58a53feb80 1 --2- 172.21.15.46:0/3677947016 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b7196a0 0x558f1b723cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.159+0000 7f58a53feb80 1 --2- 172.21.15.46:0/3677947016 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b59c4a0 0x558f1b723230 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.159+0000 7f586f7fe700 0 client.25996 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:55:52.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.159+0000 7f58a53feb80 1 --2- 172.21.15.46:0/3677947016 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b723770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.163 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.159+0000 7f58a53feb80 1 -- 172.21.15.46:0/3677947016 >> 172.21.15.46:0/3677947016 conn(0x558f1b5fc6a0 msgr2=0x558f1b7141c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:52.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.159+0000 7f58a53feb80 1 -- 172.21.15.46:0/3677947016 shutdown_connections 2024-05-09T04:55:52.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.159+0000 7f58a53feb80 1 -- 172.21.15.46:0/3677947016 wait complete. 2024-05-09T04:55:52.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.159+0000 7f58a53feb80 20 client.25996 trim_cache size 0 max 16384 2024-05-09T04:55:52.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.174+0000 7f58a53feb80 1 Processor -- start 2024-05-09T04:55:52.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.174+0000 7f58a53feb80 1 -- start start 2024-05-09T04:55:52.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f58a53feb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b59c4a0 0x558f1b723af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f58a53feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b722230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f58a53feb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b7196a0 0x558f1b72f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f58a53feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58901b1680 con 0x558f1b686160 2024-05-09T04:55:52.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f58a53feb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f58901aef10 con 0x558f1b7196a0 2024-05-09T04:55:52.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f58a53feb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f58901acd60 con 0x558f1b59c4a0 2024-05-09T04:55:52.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f5887fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b722230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f5887fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b722230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41984/0 (socket says 172.21.15.46:41984) 2024-05-09T04:55:52.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f5887fff700 1 -- 172.21.15.46:0/3761755222 learned_addr learned my addr 172.21.15.46:0/3761755222 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:52.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f5894f85700 1 --2- 172.21.15.46:0/3761755222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b7196a0 0x558f1b72f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f588ffff700 1 --2- 172.21.15.46:0/3761755222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b59c4a0 0x558f1b723af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f5887fff700 1 -- 172.21.15.46:0/3761755222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b59c4a0 msgr2=0x558f1b723af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:52.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f5887fff700 1 --2- 172.21.15.46:0/3761755222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b59c4a0 0x558f1b723af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f5887fff700 1 -- 172.21.15.46:0/3761755222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b7196a0 msgr2=0x558f1b72f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:52.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f5887fff700 1 --2- 172.21.15.46:0/3761755222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b7196a0 0x558f1b72f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f5887fff700 1 -- 172.21.15.46:0/3761755222 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558f1b71f050 con 0x558f1b686160 2024-05-09T04:55:52.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f5894f85700 1 --2- 172.21.15.46:0/3761755222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b7196a0 0x558f1b72f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:52.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f588ffff700 1 --2- 172.21.15.46:0/3761755222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b59c4a0 0x558f1b723af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:55:52.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.175+0000 7f5887fff700 1 --2- 172.21.15.46:0/3761755222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b722230 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5878005140 tx=0x7f5878005170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a85bf791ba5e913 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:52.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.176+0000 7f5885ffb700 1 -- 172.21.15.46:0/3761755222 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587800b8b0 con 0x558f1b686160 2024-05-09T04:55:52.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.176+0000 7f5885ffb700 1 -- 172.21.15.46:0/3761755222 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5878005690 con 0x558f1b686160 2024-05-09T04:55:52.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.176+0000 7f5885ffb700 1 -- 172.21.15.46:0/3761755222 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f587803e780 con 0x558f1b686160 2024-05-09T04:55:52.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.176+0000 7f58a53feb80 1 -- 172.21.15.46:0/3761755222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 msgr2=0x558f1b722230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:52.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.176+0000 7f58a53feb80 1 --2- 172.21.15.46:0/3761755222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b722230 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f5878005140 tx=0x7f5878005170 comp rx=0 tx=0).stop 2024-05-09T04:55:52.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.176+0000 7f58a53feb80 1 -- 172.21.15.46:0/3761755222 shutdown_connections 2024-05-09T04:55:52.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.176+0000 7f58a53feb80 1 --2- 172.21.15.46:0/3761755222 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b7196a0 0x558f1b72f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.176+0000 7f58a53feb80 1 --2- 172.21.15.46:0/3761755222 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b59c4a0 0x558f1b723af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.176+0000 7f58a53feb80 1 --2- 172.21.15.46:0/3761755222 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b722230 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.176+0000 7f58a53feb80 1 -- 172.21.15.46:0/3761755222 >> 172.21.15.46:0/3761755222 conn(0x558f1b5fc6a0 msgr2=0x558f1b721b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:52.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.176+0000 7f58a53feb80 1 -- 172.21.15.46:0/3761755222 shutdown_connections 2024-05-09T04:55:52.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.176+0000 7f58a53feb80 1 -- 172.21.15.46:0/3761755222 wait complete. 2024-05-09T04:55:52.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.177+0000 7f58a53feb80 1 -- start start 2024-05-09T04:55:52.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.177+0000 7f58a53feb80 1 Processor -- start 2024-05-09T04:55:52.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.177+0000 7f58a53feb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b59c4a0 0x558f1b72b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.177+0000 7f58a53feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b72b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.177+0000 7f58a53feb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b7196a0 0x558f1b739e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.177+0000 7f58a53feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f58901ad3d0 con 0x558f1b686160 2024-05-09T04:55:52.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f58a53feb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f58901ae840 con 0x558f1b59c4a0 2024-05-09T04:55:52.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f58a53feb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f58901adaa0 con 0x558f1b7196a0 2024-05-09T04:55:52.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f5887fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b72b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f58a53feb80 10 client.0 _ll_register_callbacks cb 0x558f1b4ad6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:52.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f5887fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b72b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:41996/0 (socket says 172.21.15.46:41996) 2024-05-09T04:55:52.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f5887fff700 1 -- 172.21.15.46:0/1978832167 learned_addr learned my addr 172.21.15.46:0/1978832167 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:52.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f588ffff700 1 --2- 172.21.15.46:0/1978832167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b59c4a0 0x558f1b72b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f5894f85700 1 --2- 172.21.15.46:0/1978832167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b7196a0 0x558f1b739e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f5887fff700 1 -- 172.21.15.46:0/1978832167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b7196a0 msgr2=0x558f1b739e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:52.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f5887fff700 1 --2- 172.21.15.46:0/1978832167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b7196a0 0x558f1b739e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f5887fff700 1 -- 172.21.15.46:0/1978832167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b59c4a0 msgr2=0x558f1b72b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:52.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f5887fff700 1 --2- 172.21.15.46:0/1978832167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b59c4a0 0x558f1b72b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f5887fff700 1 -- 172.21.15.46:0/1978832167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558f1b73a3b0 con 0x558f1b686160 2024-05-09T04:55:52.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f588ffff700 1 --2- 172.21.15.46:0/1978832167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b59c4a0 0x558f1b72b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:52.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f5887fff700 1 --2- 172.21.15.46:0/1978832167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b72b8a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f58780099c0 tx=0x7f587800b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad2058f498e192f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:52.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f586f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:52.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f586f7fe700 1 -- 172.21.15.46:0/1978832167 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5878004b80 con 0x558f1b686160 2024-05-09T04:55:52.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f586f7fe700 1 -- 172.21.15.46:0/1978832167 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f58780051e0 con 0x558f1b686160 2024-05-09T04:55:52.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.178+0000 7f58a53feb80 1 -- 172.21.15.46:0/1978832167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5878005690 con 0x558f1b686160 2024-05-09T04:55:52.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.179+0000 7f586f7fe700 1 -- 172.21.15.46:0/1978832167 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5878004d20 con 0x558f1b686160 2024-05-09T04:55:52.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.179+0000 7f58a53feb80 10 client.16509 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:52.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.179+0000 7f58a53feb80 1 -- 172.21.15.46:0/1978832167 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x558f1b79cf50 con 0x558f1b686160 2024-05-09T04:55:52.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.179+0000 7f586f7fe700 1 -- 172.21.15.46:0/1978832167 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5878005690 con 0x558f1b686160 2024-05-09T04:55:52.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.179+0000 7f586f7fe700 1 -- 172.21.15.46:0/1978832167 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5878027650 con 0x558f1b686160 2024-05-09T04:55:52.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.179+0000 7f58a53feb80 10 client.16509 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:52.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.179+0000 7f58a53feb80 10 client.16509 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:55:52.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.179+0000 7f58a53feb80 20 client.16509 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:52.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.179+0000 7f58a53feb80 1 --2- 172.21.15.46:0/1978832167 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x558f1b79d1d0 0x558f1b79f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:52.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.179+0000 7f58a53feb80 4 client.16509 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:55:52.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.179+0000 7f58a53feb80 1 -- 172.21.15.46:0/1978832167 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558f1b643a70 con 0x558f1b79d1d0 2024-05-09T04:55:52.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.179+0000 7f588ffff700 1 --2- 172.21.15.46:0/1978832167 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x558f1b79d1d0 0x558f1b79f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:52.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f588ffff700 1 --2- 172.21.15.46:0/1978832167 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x558f1b79d1d0 0x558f1b79f5f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1822cc92c8beb5e7 server_cookie=6dd3c6bd9dec242d in_seq=0 out_seq=0 2024-05-09T04:55:52.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f586f7fe700 10 client.16509 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:55:52.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f586f7fe700 1 -- 172.21.15.46:0/1978832167 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558f1b643a70 con 0x558f1b79d1d0 2024-05-09T04:55:52.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f586f7fe700 10 client.16509 handle_command_reply: tid=0 2024-05-09T04:55:52.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f58a53feb80 1 client.16509 shutdown 2024-05-09T04:55:52.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f58a53feb80 1 -- 172.21.15.46:0/1978832167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 msgr2=0x558f1b72b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:52.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f58a53feb80 1 --2- 172.21.15.46:0/1978832167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b72b8a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f58780099c0 tx=0x7f587800b6e0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f58a53feb80 1 -- 172.21.15.46:0/1978832167 shutdown_connections 2024-05-09T04:55:52.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f58a53feb80 1 --2- 172.21.15.46:0/1978832167 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x558f1b79d1d0 0x558f1b79f5f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f58a53feb80 1 --2- 172.21.15.46:0/1978832167 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x558f1b7196a0 0x558f1b739e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f58a53feb80 1 --2- 172.21.15.46:0/1978832167 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x558f1b59c4a0 0x558f1b72b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f58a53feb80 1 --2- 172.21.15.46:0/1978832167 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x558f1b686160 0x558f1b72b8a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f58a53feb80 1 -- 172.21.15.46:0/1978832167 >> 172.21.15.46:0/1978832167 conn(0x558f1b5fc6a0 msgr2=0x558f1b718690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:52.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f586f7fe700 0 client.16509 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:55:52.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f58a53feb80 1 -- 172.21.15.46:0/1978832167 shutdown_connections 2024-05-09T04:55:52.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f58a53feb80 1 -- 172.21.15.46:0/1978832167 wait complete. 2024-05-09T04:55:52.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.180+0000 7f58a53feb80 20 client.16509 trim_cache size 0 max 16384 2024-05-09T04:55:52.186 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:55:52.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.181+0000 7f58969e8700 1 -- 172.21.15.46:0/3313664433 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f587c0b29f0 msgr2=0x7f587c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:52.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.181+0000 7f58969e8700 1 --2- 172.21.15.46:0/3313664433 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f587c0b29f0 0x7f587c0b4f00 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f5878009990 tx=0x7f5878009690 comp rx=0 tx=0).stop 2024-05-09T04:55:52.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.181+0000 7f58969e8700 1 -- 172.21.15.46:0/3313664433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5890108740 msgr2=0x7f589019f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:52.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.181+0000 7f58969e8700 1 --2- 172.21.15.46:0/3313664433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5890108740 0x7f589019f600 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f5888007ee0 tx=0x7f588800d3b0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.181+0000 7f58969e8700 1 -- 172.21.15.46:0/3313664433 shutdown_connections 2024-05-09T04:55:52.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.181+0000 7f58969e8700 1 --2- 172.21.15.46:0/3313664433 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f587c0b29f0 0x7f587c0b4f00 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.181+0000 7f58969e8700 1 --2- 172.21.15.46:0/3313664433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5890108740 0x7f589019f600 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.181+0000 7f58969e8700 1 --2- 172.21.15.46:0/3313664433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f58901034c0 0x7f589019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.181+0000 7f58969e8700 1 --2- 172.21.15.46:0/3313664433 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5890107df0 0x7f589019b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:52.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.181+0000 7f58969e8700 1 -- 172.21.15.46:0/3313664433 >> 172.21.15.46:0/3313664433 conn(0x7f58900fedc0 msgr2=0x7f589006b890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:52.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.181+0000 7f58969e8700 1 -- 172.21.15.46:0/3313664433 shutdown_connections 2024-05-09T04:55:52.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:52.181+0000 7f58969e8700 1 -- 172.21.15.46:0/3313664433 wait complete. 2024-05-09T04:55:52.818 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:55:53.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:52 smithi110 ceph-mon[34904]: pgmap v146: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:55:53.071 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:53.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:52 smithi046 ceph-mon[30136]: pgmap v146: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:55:53.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:52 smithi072 ceph-mon[34088]: pgmap v146: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:55:54.893 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:54 smithi046 ceph-mon[30136]: pgmap v147: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:55:54.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.910+0000 7f4e43834700 1 -- 172.21.15.46:0/3729008549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c105db0 msgr2=0x7f4e3c10e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:54.910 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.910+0000 7f4e43834700 1 --2- 172.21.15.46:0/3729008549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c105db0 0x7f4e3c10e300 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009990 tx=0x7f4e2c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.910+0000 7f4e43834700 1 -- 172.21.15.46:0/3729008549 shutdown_connections 2024-05-09T04:55:54.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.910+0000 7f4e43834700 1 --2- 172.21.15.46:0/3729008549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e3c10e8d0 0x7f4e3c110de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.910+0000 7f4e43834700 1 --2- 172.21.15.46:0/3729008549 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e3c105460 0x7f4e3c105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.910+0000 7f4e43834700 1 --2- 172.21.15.46:0/3729008549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c105db0 0x7f4e3c10e300 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.910+0000 7f4e43834700 1 -- 172.21.15.46:0/3729008549 >> 172.21.15.46:0/3729008549 conn(0x7f4e3c0fedf0 msgr2=0x7f4e3c101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:54.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.910+0000 7f4e43834700 1 -- 172.21.15.46:0/3729008549 shutdown_connections 2024-05-09T04:55:54.911 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.910+0000 7f4e43834700 1 -- 172.21.15.46:0/3729008549 wait complete. 2024-05-09T04:55:54.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.911+0000 7f4e43834700 1 Processor -- start 2024-05-09T04:55:54.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.911+0000 7f4e43834700 1 -- start start 2024-05-09T04:55:54.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.911+0000 7f4e43834700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c105460 0x7f4e3c103150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.911+0000 7f4e43834700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e3c105db0 0x7f4e3c103690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.911+0000 7f4e43834700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e3c10e8d0 0x7f4e3c103bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.911+0000 7f4e43834700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c114460 con 0x7f4e3c105460 2024-05-09T04:55:54.913 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.911+0000 7f4e43834700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1142e0 con 0x7f4e3c10e8d0 2024-05-09T04:55:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.911+0000 7f4e43834700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1145e0 con 0x7f4e3c105db0 2024-05-09T04:55:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.911+0000 7f4e415d0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c105460 0x7f4e3c103150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.911+0000 7f4e415d0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c105460 0x7f4e3c103150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56908/0 (socket says 172.21.15.46:56908) 2024-05-09T04:55:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.911+0000 7f4e415d0700 1 -- 172.21.15.46:0/242143945 learned_addr learned my addr 172.21.15.46:0/242143945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.911+0000 7f4e40dcf700 1 --2- 172.21.15.46:0/242143945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e3c105db0 0x7f4e3c103690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.912+0000 7f4e41dd1700 1 --2- 172.21.15.46:0/242143945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e3c10e8d0 0x7f4e3c103bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.912+0000 7f4e415d0700 1 -- 172.21.15.46:0/242143945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e3c105db0 msgr2=0x7f4e3c103690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.912+0000 7f4e415d0700 1 --2- 172.21.15.46:0/242143945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e3c105db0 0x7f4e3c103690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.912+0000 7f4e415d0700 1 -- 172.21.15.46:0/242143945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e3c10e8d0 msgr2=0x7f4e3c103bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.912+0000 7f4e415d0700 1 --2- 172.21.15.46:0/242143945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e3c10e8d0 0x7f4e3c103bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.912+0000 7f4e415d0700 1 -- 172.21.15.46:0/242143945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3c1a9190 con 0x7f4e3c105460 2024-05-09T04:55:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.912+0000 7f4e415d0700 1 --2- 172.21.15.46:0/242143945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c105460 0x7f4e3c103150 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800d980 tx=0x7f4e3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.912+0000 7f4e327fc700 1 -- 172.21.15.46:0/242143945 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e38017030 con 0x7f4e3c105460 2024-05-09T04:55:54.914 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.912+0000 7f4e327fc700 1 -- 172.21.15.46:0/242143945 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e380044d0 con 0x7f4e3c105460 2024-05-09T04:55:54.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.912+0000 7f4e43834700 1 -- 172.21.15.46:0/242143945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3c1a9480 con 0x7f4e3c105460 2024-05-09T04:55:54.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.912+0000 7f4e327fc700 1 -- 172.21.15.46:0/242143945 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e380039e0 con 0x7f4e3c105460 2024-05-09T04:55:54.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.912+0000 7f4e43834700 1 -- 172.21.15.46:0/242143945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e3c1a9990 con 0x7f4e3c105460 2024-05-09T04:55:54.915 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.915+0000 7f4e327fc700 1 -- 172.21.15.46:0/242143945 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4e3801a070 con 0x7f4e3c105460 2024-05-09T04:55:54.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.916+0000 7f4e327fc700 1 --2- 172.21.15.46:0/242143945 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4e280b29f0 0x7f4e280b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e40dcf700 1 --2- 172.21.15.46:0/242143945 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4e280b29f0 0x7f4e280b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e5224ab80 1 Processor -- start 2024-05-09T04:55:54.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e327fc700 1 -- 172.21.15.46:0/242143945 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f4e380e1f80 con 0x7f4e3c105460 2024-05-09T04:55:54.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e5224ab80 1 -- start start 2024-05-09T04:55:54.918 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e5224ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ef6d00 0x55e326fff240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e5224ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326e7c4a0 0x55e326fff780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e5224ab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326ff9400 0x55e326ff9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e5224ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e326f1c0a0 con 0x55e326ef6d00 2024-05-09T04:55:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e5224ab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e326ec19a0 con 0x55e326ff9400 2024-05-09T04:55:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e5224ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e326f18060 con 0x55e326e7c4a0 2024-05-09T04:55:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e40dcf700 1 --2- 172.21.15.46:0/242143945 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4e280b29f0 0x7f4e280b4f00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c006010 tx=0x7f4e2c005960 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e415d0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ef6d00 0x55e326fff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e415d0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ef6d00 0x55e326fff240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56910/0 (socket says 172.21.15.46:56910) 2024-05-09T04:55:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e415d0700 1 -- 172.21.15.46:0/4279575471 learned_addr learned my addr 172.21.15.46:0/4279575471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e40dcf700 1 --2- 172.21.15.46:0/4279575471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326e7c4a0 0x55e326fff780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.917+0000 7f4e41dd1700 1 --2- 172.21.15.46:0/4279575471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326ff9400 0x55e326ff9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.918+0000 7f4e415d0700 1 -- 172.21.15.46:0/4279575471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326e7c4a0 msgr2=0x55e326fff780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.918+0000 7f4e415d0700 1 --2- 172.21.15.46:0/4279575471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326e7c4a0 0x55e326fff780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.918+0000 7f4e415d0700 1 -- 172.21.15.46:0/4279575471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326ff9400 msgr2=0x55e326ff9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:54.919 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.918+0000 7f4e415d0700 1 --2- 172.21.15.46:0/4279575471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326ff9400 0x55e326ff9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.918+0000 7f4e415d0700 1 -- 172.21.15.46:0/4279575471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e326eb2500 con 0x55e326ef6d00 2024-05-09T04:55:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.918+0000 7f4e415d0700 1 --2- 172.21.15.46:0/4279575471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ef6d00 0x55e326fff240 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4e380a91d0 tx=0x7f4e380a9940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c58057a6c3f6e4b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.918+0000 7f4e277fe700 1 -- 172.21.15.46:0/4279575471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e380ad5f0 con 0x55e326ef6d00 2024-05-09T04:55:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.918+0000 7f4e277fe700 1 -- 172.21.15.46:0/4279575471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e380b20a0 con 0x55e326ef6d00 2024-05-09T04:55:54.920 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.918+0000 7f4e277fe700 1 -- 172.21.15.46:0/4279575471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e380b1e50 con 0x55e326ef6d00 2024-05-09T04:55:54.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.919+0000 7f4e5224ab80 1 -- 172.21.15.46:0/4279575471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ef6d00 msgr2=0x55e326fff240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:54.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.919+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/4279575471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ef6d00 0x55e326fff240 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f4e380a91d0 tx=0x7f4e380a9940 comp rx=0 tx=0).stop 2024-05-09T04:55:54.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.919+0000 7f4e5224ab80 1 -- 172.21.15.46:0/4279575471 shutdown_connections 2024-05-09T04:55:54.923 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.919+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/4279575471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326ff9400 0x55e326ff9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.919+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/4279575471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ef6d00 0x55e326fff240 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.919+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/4279575471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326e7c4a0 0x55e326fff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.919+0000 7f4e5224ab80 1 -- 172.21.15.46:0/4279575471 >> 172.21.15.46:0/4279575471 conn(0x55e326e7ff70 msgr2=0x55e326ff4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.919+0000 7f4e5224ab80 1 -- 172.21.15.46:0/4279575471 shutdown_connections 2024-05-09T04:55:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.919+0000 7f4e5224ab80 1 -- 172.21.15.46:0/4279575471 wait complete. 2024-05-09T04:55:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.920+0000 7f4e5224ab80 1 -- start start 2024-05-09T04:55:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.920+0000 7f4e5224ab80 1 Processor -- start 2024-05-09T04:55:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.920+0000 7f4e5224ab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326e7c4a0 0x55e327003040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.920+0000 7f4e5224ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326ef6d00 0x55e327003580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.920+0000 7f4e5224ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ff9400 0x55e327003ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.920+0000 7f4e5224ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e326ffa160 con 0x55e326ff9400 2024-05-09T04:55:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.920+0000 7f4e5224ab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e326eb6b20 con 0x55e326e7c4a0 2024-05-09T04:55:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.920+0000 7f4e5224ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e327006020 con 0x55e326ef6d00 2024-05-09T04:55:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.920+0000 7f4e5224ab80 10 client.0 _ll_register_callbacks cb 0x55e326d8d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:54.924 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.920+0000 7f4e41dd1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ff9400 0x55e327003ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.920+0000 7f4e41dd1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ff9400 0x55e327003ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56918/0 (socket says 172.21.15.46:56918) 2024-05-09T04:55:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.920+0000 7f4e41dd1700 1 -- 172.21.15.46:0/3340737690 learned_addr learned my addr 172.21.15.46:0/3340737690 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.920+0000 7f4e415d0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326e7c4a0 0x55e327003040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.920+0000 7f4e40dcf700 1 --2- 172.21.15.46:0/3340737690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326ef6d00 0x55e327003580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.921+0000 7f4e41dd1700 1 -- 172.21.15.46:0/3340737690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326ef6d00 msgr2=0x55e327003580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.921+0000 7f4e41dd1700 1 --2- 172.21.15.46:0/3340737690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326ef6d00 0x55e327003580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.921+0000 7f4e41dd1700 1 -- 172.21.15.46:0/3340737690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326e7c4a0 msgr2=0x55e327003040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.921+0000 7f4e41dd1700 1 --2- 172.21.15.46:0/3340737690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326e7c4a0 0x55e327003040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.921+0000 7f4e41dd1700 1 -- 172.21.15.46:0/3340737690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e326d97490 con 0x55e326ff9400 2024-05-09T04:55:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.921+0000 7f4e40dcf700 1 --2- 172.21.15.46:0/3340737690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326ef6d00 0x55e327003580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.921+0000 7f4e415d0700 1 --2- 172.21.15.46:0/3340737690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326e7c4a0 0x55e327003040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.921+0000 7f4e41dd1700 1 --2- 172.21.15.46:0/3340737690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ff9400 0x55e327003ac0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4e340090e0 tx=0x7f4e34007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10b7822cbca8b929 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.921+0000 7f4e24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:54.925 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.921+0000 7f4e24ff9700 1 -- 172.21.15.46:0/3340737690 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34003a00 con 0x55e326ff9400 2024-05-09T04:55:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.921+0000 7f4e24ff9700 1 -- 172.21.15.46:0/3340737690 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f4e34003ba0 con 0x55e326ff9400 2024-05-09T04:55:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.921+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3340737690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e380b20a0 con 0x55e326ff9400 2024-05-09T04:55:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.921+0000 7f4e24ff9700 1 -- 172.21.15.46:0/3340737690 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e34024050 con 0x55e326ff9400 2024-05-09T04:55:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.921+0000 7f4e5224ab80 10 client.16521 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.921+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3340737690 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e327010a70 con 0x55e326ff9400 2024-05-09T04:55:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.922+0000 7f4e24ff9700 1 -- 172.21.15.46:0/3340737690 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e380b20a0 con 0x55e326ff9400 2024-05-09T04:55:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.922+0000 7f4e24ff9700 1 -- 172.21.15.46:0/3340737690 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f4e34004370 con 0x55e326ff9400 2024-05-09T04:55:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.922+0000 7f4e5224ab80 10 client.16521 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.922+0000 7f4e5224ab80 10 client.16521 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:55:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.922+0000 7f4e5224ab80 20 client.16521 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.922+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/3340737690 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e3270130b0 0x55e327015540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.922+0000 7f4e5224ab80 4 client.16521 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:55:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.922+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3340737690 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e326f29a80 con 0x55e3270130b0 2024-05-09T04:55:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.922+0000 7f4e415d0700 1 --2- 172.21.15.46:0/3340737690 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e3270130b0 0x55e327015540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.926 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.923+0000 7f4e24ff9700 10 client.16521 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.923+0000 7f4e415d0700 1 --2- 172.21.15.46:0/3340737690 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e3270130b0 0x55e327015540 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12302fc06a5da374 server_cookie=ecd0c09ec6ec8e5d in_seq=0 out_seq=0 2024-05-09T04:55:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.923+0000 7f4e24ff9700 1 -- 172.21.15.46:0/3340737690 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e326f29a80 con 0x55e3270130b0 2024-05-09T04:55:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.923+0000 7f4e24ff9700 10 client.16521 handle_command_reply: tid=0 2024-05-09T04:55:54.927 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.923+0000 7f4e5224ab80 1 client.16521 shutdown 2024-05-09T04:55:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.924+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3340737690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ff9400 msgr2=0x55e327003ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.924+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/3340737690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ff9400 0x55e327003ac0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f4e340090e0 tx=0x7f4e34007dd0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.924+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3340737690 shutdown_connections 2024-05-09T04:55:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.924+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/3340737690 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ff9400 0x55e327003ac0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.924+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/3340737690 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e3270130b0 0x55e327015540 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.924+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/3340737690 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326e7c4a0 0x55e327003040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.924+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/3340737690 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326ef6d00 0x55e327003580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.924+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3340737690 >> 172.21.15.46:0/3340737690 conn(0x55e326e7ff70 msgr2=0x55e326fedff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.924+0000 7f4e24ff9700 0 client.16521 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.924+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3340737690 shutdown_connections 2024-05-09T04:55:54.928 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.924+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3340737690 wait complete. 2024-05-09T04:55:54.929 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.924+0000 7f4e5224ab80 20 client.16521 trim_cache size 0 max 16384 2024-05-09T04:55:54.939 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.939+0000 7f4e5224ab80 1 Processor -- start 2024-05-09T04:55:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.939+0000 7f4e5224ab80 1 -- start start 2024-05-09T04:55:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.939+0000 7f4e5224ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326e7c4a0 0x55e327003710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.939+0000 7f4e5224ab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326ef6d00 0x55e32700ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.939+0000 7f4e5224ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ff9400 0x55e32701ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.939+0000 7f4e5224ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1b3ad0 con 0x55e326ff9400 2024-05-09T04:55:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.939+0000 7f4e5224ab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1b1360 con 0x55e326ef6d00 2024-05-09T04:55:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.939+0000 7f4e5224ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1af1b0 con 0x55e326e7c4a0 2024-05-09T04:55:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.940+0000 7f4e41dd1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ff9400 0x55e32701ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.940+0000 7f4e41dd1700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ff9400 0x55e32701ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56928/0 (socket says 172.21.15.46:56928) 2024-05-09T04:55:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.940+0000 7f4e41dd1700 1 -- 172.21.15.46:0/2861100586 learned_addr learned my addr 172.21.15.46:0/2861100586 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.940+0000 7f4e40dcf700 1 --2- 172.21.15.46:0/2861100586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326ef6d00 0x55e32700ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.940 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.940+0000 7f4e415d0700 1 --2- 172.21.15.46:0/2861100586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326e7c4a0 0x55e327003710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.940+0000 7f4e41dd1700 1 -- 172.21.15.46:0/2861100586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326e7c4a0 msgr2=0x55e327003710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.940+0000 7f4e41dd1700 1 --2- 172.21.15.46:0/2861100586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326e7c4a0 0x55e327003710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.940+0000 7f4e41dd1700 1 -- 172.21.15.46:0/2861100586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326ef6d00 msgr2=0x55e32700ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.940+0000 7f4e41dd1700 1 --2- 172.21.15.46:0/2861100586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326ef6d00 0x55e32700ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.940+0000 7f4e41dd1700 1 -- 172.21.15.46:0/2861100586 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e326ff5e80 con 0x55e326ff9400 2024-05-09T04:55:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.940+0000 7f4e40dcf700 1 --2- 172.21.15.46:0/2861100586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326ef6d00 0x55e32700ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.940+0000 7f4e41dd1700 1 --2- 172.21.15.46:0/2861100586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ff9400 0x55e32701ad50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4e340047e0 tx=0x7f4e340114b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aee23d5852e316 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.940+0000 7f4e26ffd700 1 -- 172.21.15.46:0/2861100586 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34003c00 con 0x55e326ff9400 2024-05-09T04:55:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.940+0000 7f4e26ffd700 1 -- 172.21.15.46:0/2861100586 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e34003da0 con 0x55e326ff9400 2024-05-09T04:55:54.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.940+0000 7f4e26ffd700 1 -- 172.21.15.46:0/2861100586 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e34011c50 con 0x55e326ff9400 2024-05-09T04:55:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.941+0000 7f4e5224ab80 1 -- 172.21.15.46:0/2861100586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ff9400 msgr2=0x55e32701ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.941+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/2861100586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ff9400 0x55e32701ad50 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f4e340047e0 tx=0x7f4e340114b0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.941+0000 7f4e5224ab80 1 -- 172.21.15.46:0/2861100586 shutdown_connections 2024-05-09T04:55:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.941+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/2861100586 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ff9400 0x55e32701ad50 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.941+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/2861100586 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326e7c4a0 0x55e327003710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.941+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/2861100586 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326ef6d00 0x55e32700ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.941+0000 7f4e5224ab80 1 -- 172.21.15.46:0/2861100586 >> 172.21.15.46:0/2861100586 conn(0x55e326e7ff70 msgr2=0x55e327003000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.941+0000 7f4e5224ab80 1 -- 172.21.15.46:0/2861100586 shutdown_connections 2024-05-09T04:55:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.941+0000 7f4e5224ab80 1 -- 172.21.15.46:0/2861100586 wait complete. 2024-05-09T04:55:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.942+0000 7f4e5224ab80 1 -- start start 2024-05-09T04:55:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.942+0000 7f4e5224ab80 1 Processor -- start 2024-05-09T04:55:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.942+0000 7f4e5224ab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326e7c4a0 0x55e3270083f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.942+0000 7f4e5224ab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ef6d00 0x55e327008930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.942+0000 7f4e5224ab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326ff9400 0x55e32707b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.942+0000 7f4e5224ab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1af820 con 0x55e326ef6d00 2024-05-09T04:55:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.942+0000 7f4e5224ab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1b0c90 con 0x55e326e7c4a0 2024-05-09T04:55:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.942+0000 7f4e5224ab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3c1afef0 con 0x55e326ff9400 2024-05-09T04:55:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.942+0000 7f4e5224ab80 10 client.0 _ll_register_callbacks cb 0x55e326d8d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.942+0000 7f4e40dcf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ef6d00 0x55e327008930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.942+0000 7f4e40dcf700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ef6d00 0x55e327008930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56930/0 (socket says 172.21.15.46:56930) 2024-05-09T04:55:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.942+0000 7f4e40dcf700 1 -- 172.21.15.46:0/3225349893 learned_addr learned my addr 172.21.15.46:0/3225349893 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.942+0000 7f4e41dd1700 1 --2- 172.21.15.46:0/3225349893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326ff9400 0x55e32707b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.943+0000 7f4e415d0700 1 --2- 172.21.15.46:0/3225349893 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326e7c4a0 0x55e3270083f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.943+0000 7f4e40dcf700 1 -- 172.21.15.46:0/3225349893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326ff9400 msgr2=0x55e32707b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.943+0000 7f4e40dcf700 1 --2- 172.21.15.46:0/3225349893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326ff9400 0x55e32707b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.943+0000 7f4e40dcf700 1 -- 172.21.15.46:0/3225349893 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326e7c4a0 msgr2=0x55e3270083f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.943+0000 7f4e40dcf700 1 --2- 172.21.15.46:0/3225349893 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326e7c4a0 0x55e3270083f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.943+0000 7f4e40dcf700 1 -- 172.21.15.46:0/3225349893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e32707be40 con 0x55e326ef6d00 2024-05-09T04:55:54.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.943+0000 7f4e40dcf700 1 --2- 172.21.15.46:0/3225349893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ef6d00 0x55e327008930 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c003800 tx=0x7f4e2c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6cf4bcacbb846ec server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.943+0000 7f4e24ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.943+0000 7f4e24ff9700 1 -- 172.21.15.46:0/3225349893 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c00f870 con 0x55e326ef6d00 2024-05-09T04:55:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.943+0000 7f4e24ff9700 1 -- 172.21.15.46:0/3225349893 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f4e2c00fa10 con 0x55e326ef6d00 2024-05-09T04:55:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.943+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3225349893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4e34003da0 con 0x55e326ef6d00 2024-05-09T04:55:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.943+0000 7f4e24ff9700 1 -- 172.21.15.46:0/3225349893 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f4e2c00fc10 con 0x55e326ef6d00 2024-05-09T04:55:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.943+0000 7f4e5224ab80 10 client.16527 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.943+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3225349893 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e32707cf30 con 0x55e326ef6d00 2024-05-09T04:55:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.944+0000 7f4e24ff9700 1 -- 172.21.15.46:0/3225349893 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e34003da0 con 0x55e326ef6d00 2024-05-09T04:55:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.944+0000 7f4e24ff9700 1 -- 172.21.15.46:0/3225349893 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f4e2c022070 con 0x55e326ef6d00 2024-05-09T04:55:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.944+0000 7f4e5224ab80 10 client.16527 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.944+0000 7f4e5224ab80 10 client.16527 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:55:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.944+0000 7f4e5224ab80 20 client.16527 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.944+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/3225349893 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e32707d6c0 0x55e32707fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.944+0000 7f4e5224ab80 4 client.16527 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:55:54.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.944+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3225349893 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e326f23b20 con 0x55e32707d6c0 2024-05-09T04:55:54.948 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.944+0000 7f4e415d0700 1 --2- 172.21.15.46:0/3225349893 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e32707d6c0 0x55e32707fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.944+0000 7f4e24ff9700 10 client.16527 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.944+0000 7f4e415d0700 1 --2- 172.21.15.46:0/3225349893 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e32707d6c0 0x55e32707fae0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ba363dc995552e3 server_cookie=a68dddb8bd7a38dd in_seq=0 out_seq=0 2024-05-09T04:55:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.945+0000 7f4e24ff9700 1 -- 172.21.15.46:0/3225349893 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e326f23b20 con 0x55e32707d6c0 2024-05-09T04:55:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.945+0000 7f4e24ff9700 10 client.16527 handle_command_reply: tid=0 2024-05-09T04:55:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.945+0000 7f4e5224ab80 1 client.16527 shutdown 2024-05-09T04:55:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.945+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3225349893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ef6d00 msgr2=0x55e327008930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.945+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/3225349893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ef6d00 0x55e327008930 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c003800 tx=0x7f4e2c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.945+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3225349893 shutdown_connections 2024-05-09T04:55:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.945+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/3225349893 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e326ff9400 0x55e32707b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.945+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/3225349893 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e32707d6c0 0x55e32707fae0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.949 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.945+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/3225349893 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e326e7c4a0 0x55e3270083f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.945+0000 7f4e5224ab80 1 --2- 172.21.15.46:0/3225349893 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e326ef6d00 0x55e327008930 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.945+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3225349893 >> 172.21.15.46:0/3225349893 conn(0x55e326e7ff70 msgr2=0x55e326ffccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.945+0000 7f4e24ff9700 0 client.16527 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.945+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3225349893 shutdown_connections 2024-05-09T04:55:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.945+0000 7f4e5224ab80 1 -- 172.21.15.46:0/3225349893 wait complete. 2024-05-09T04:55:54.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.945+0000 7f4e5224ab80 20 client.16527 trim_cache size 0 max 16384 2024-05-09T04:55:54.950 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:55:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.946+0000 7f4e43834700 1 -- 172.21.15.46:0/242143945 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4e280b29f0 msgr2=0x7f4e280b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.946+0000 7f4e43834700 1 --2- 172.21.15.46:0/242143945 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4e280b29f0 0x7f4e280b4f00 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c006010 tx=0x7f4e2c005960 comp rx=0 tx=0).stop 2024-05-09T04:55:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.946+0000 7f4e43834700 1 -- 172.21.15.46:0/242143945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c105460 msgr2=0x7f4e3c103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.946+0000 7f4e43834700 1 --2- 172.21.15.46:0/242143945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c105460 0x7f4e3c103150 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f4e3800d980 tx=0x7f4e3800dc90 comp rx=0 tx=0).stop 2024-05-09T04:55:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.946+0000 7f4e43834700 1 -- 172.21.15.46:0/242143945 shutdown_connections 2024-05-09T04:55:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.946+0000 7f4e43834700 1 --2- 172.21.15.46:0/242143945 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4e280b29f0 0x7f4e280b4f00 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.946+0000 7f4e43834700 1 --2- 172.21.15.46:0/242143945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e3c10e8d0 0x7f4e3c103bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.946+0000 7f4e43834700 1 --2- 172.21.15.46:0/242143945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e3c105460 0x7f4e3c103150 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.951 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.946+0000 7f4e43834700 1 --2- 172.21.15.46:0/242143945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e3c105db0 0x7f4e3c103690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.946+0000 7f4e43834700 1 -- 172.21.15.46:0/242143945 >> 172.21.15.46:0/242143945 conn(0x7f4e3c0fedf0 msgr2=0x7f4e3c0ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.946+0000 7f4e43834700 1 -- 172.21.15.46:0/242143945 shutdown_connections 2024-05-09T04:55:54.952 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:54.946+0000 7f4e43834700 1 -- 172.21.15.46:0/242143945 wait complete. 2024-05-09T04:55:55.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:54 smithi110 ceph-mon[34904]: pgmap v147: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:55:55.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:54 smithi072 ceph-mon[34088]: pgmap v147: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:55:56.657 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:55:56.657 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:55:56.927 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:55:57.050 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:56 smithi046 ceph-mon[30136]: pgmap v148: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:57.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:56 smithi110 ceph-mon[34904]: pgmap v148: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:56 smithi072 ceph-mon[34088]: pgmap v148: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:55:58.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.803+0000 7f2860a97700 1 -- 172.21.15.46:0/3819149870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285c1034e0 msgr2=0x7f285c1038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.803+0000 7f2860a97700 1 --2- 172.21.15.46:0/3819149870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285c1034e0 0x7f285c1038f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f284c0099c0 tx=0x7f284c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.803+0000 7f2860a97700 1 -- 172.21.15.46:0/3819149870 shutdown_connections 2024-05-09T04:55:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.803+0000 7f2860a97700 1 --2- 172.21.15.46:0/3819149870 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f285c108760 0x7f285c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.803+0000 7f2860a97700 1 --2- 172.21.15.46:0/3819149870 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285c1034e0 0x7f285c1038f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.803+0000 7f2860a97700 1 --2- 172.21.15.46:0/3819149870 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f285c107e10 0x7f285c108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.803+0000 7f2860a97700 1 -- 172.21.15.46:0/3819149870 >> 172.21.15.46:0/3819149870 conn(0x7f285c0fef30 msgr2=0x7f285c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.803+0000 7f2860a97700 1 -- 172.21.15.46:0/3819149870 shutdown_connections 2024-05-09T04:55:58.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.803+0000 7f2860a97700 1 -- 172.21.15.46:0/3819149870 wait complete. 2024-05-09T04:55:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.804+0000 7f2860a97700 1 Processor -- start 2024-05-09T04:55:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.804+0000 7f2860a97700 1 -- start start 2024-05-09T04:55:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.804+0000 7f2860a97700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f285c1034e0 0x7f285c19bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.804+0000 7f2860a97700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f285c107e10 0x7f285c19c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.804+0000 7f2860a97700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285c108760 0x7f285c19c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.804+0000 7f2860a97700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f285c1133e0 con 0x7f285c108760 2024-05-09T04:55:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.804+0000 7f2860a97700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f285c113260 con 0x7f285c107e10 2024-05-09T04:55:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.804+0000 7f2860a97700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f285c113560 con 0x7f285c1034e0 2024-05-09T04:55:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.804+0000 7f285ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285c108760 0x7f285c19c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.804+0000 7f285ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285c108760 0x7f285c19c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56938/0 (socket says 172.21.15.46:56938) 2024-05-09T04:55:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.804+0000 7f285ad9d700 1 -- 172.21.15.46:0/3266878024 learned_addr learned my addr 172.21.15.46:0/3266878024 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:58.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.805+0000 7f2859d9b700 1 --2- 172.21.15.46:0/3266878024 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f285c107e10 0x7f285c19c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.805+0000 7f285a59c700 1 --2- 172.21.15.46:0/3266878024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f285c1034e0 0x7f285c19bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.805+0000 7f285ad9d700 1 -- 172.21.15.46:0/3266878024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f285c1034e0 msgr2=0x7f285c19bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.805+0000 7f285ad9d700 1 --2- 172.21.15.46:0/3266878024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f285c1034e0 0x7f285c19bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.805+0000 7f285ad9d700 1 -- 172.21.15.46:0/3266878024 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f285c107e10 msgr2=0x7f285c19c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.805+0000 7f285ad9d700 1 --2- 172.21.15.46:0/3266878024 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f285c107e10 0x7f285c19c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.805+0000 7f285ad9d700 1 -- 172.21.15.46:0/3266878024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285c1a6fc0 con 0x7f285c108760 2024-05-09T04:55:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.805+0000 7f285a59c700 1 --2- 172.21.15.46:0/3266878024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f285c1034e0 0x7f285c19bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.805+0000 7f285ad9d700 1 --2- 172.21.15.46:0/3266878024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285c108760 0x7f285c19c7a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2854007ee0 tx=0x7f285400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.805+0000 7f284b7fe700 1 -- 172.21.15.46:0/3266878024 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285400dbd0 con 0x7f285c108760 2024-05-09T04:55:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.805+0000 7f2860a97700 1 -- 172.21.15.46:0/3266878024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f285c1a72b0 con 0x7f285c108760 2024-05-09T04:55:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.805+0000 7f284b7fe700 1 -- 172.21.15.46:0/3266878024 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f285400dd70 con 0x7f285c108760 2024-05-09T04:55:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.805+0000 7f284b7fe700 1 -- 172.21.15.46:0/3266878024 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2854005990 con 0x7f285c108760 2024-05-09T04:55:58.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.805+0000 7f2860a97700 1 -- 172.21.15.46:0/3266878024 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f285c1a77f0 con 0x7f285c108760 2024-05-09T04:55:58.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.808+0000 7f284b7fe700 1 -- 172.21.15.46:0/3266878024 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f285401a020 con 0x7f285c108760 2024-05-09T04:55:58.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.809+0000 7f284b7fe700 1 --2- 172.21.15.46:0/3266878024 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f28440b29f0 0x7f28440b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.809+0000 7f285a59c700 1 --2- 172.21.15.46:0/3266878024 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f28440b29f0 0x7f28440b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.809+0000 7f284b7fe700 1 -- 172.21.15.46:0/3266878024 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f28540e1de0 con 0x7f285c108760 2024-05-09T04:55:58.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.809+0000 7f286f4adb80 1 Processor -- start 2024-05-09T04:55:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.809+0000 7f286f4adb80 1 -- start start 2024-05-09T04:55:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.809+0000 7f285a59c700 1 --2- 172.21.15.46:0/3266878024 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f28440b29f0 0x7f28440b4f00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f284c000c00 tx=0x7f284c01a040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f286f4adb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c835dd80 0x55f9c84662c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f286f4adb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c82e3520 0x55f9c8466800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f286f4adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c8460480 0x55f9c8460890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f286f4adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f9c8383120 con 0x55f9c8460480 2024-05-09T04:55:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f286f4adb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f9c8328a20 con 0x55f9c82e3520 2024-05-09T04:55:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f286f4adb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f9c837f0e0 con 0x55f9c835dd80 2024-05-09T04:55:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f285ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c8460480 0x55f9c8460890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f285ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c8460480 0x55f9c8460890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56948/0 (socket says 172.21.15.46:56948) 2024-05-09T04:55:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f285ad9d700 1 -- 172.21.15.46:0/1418552865 learned_addr learned my addr 172.21.15.46:0/1418552865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:58.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f2859d9b700 1 --2- 172.21.15.46:0/1418552865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c82e3520 0x55f9c8466800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f285a59c700 1 --2- 172.21.15.46:0/1418552865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c835dd80 0x55f9c84662c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f285ad9d700 1 -- 172.21.15.46:0/1418552865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c835dd80 msgr2=0x55f9c84662c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f285ad9d700 1 --2- 172.21.15.46:0/1418552865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c835dd80 0x55f9c84662c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f285ad9d700 1 -- 172.21.15.46:0/1418552865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c82e3520 msgr2=0x55f9c8466800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f285ad9d700 1 --2- 172.21.15.46:0/1418552865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c82e3520 0x55f9c8466800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f285ad9d700 1 -- 172.21.15.46:0/1418552865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c8319580 con 0x55f9c8460480 2024-05-09T04:55:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f285a59c700 1 --2- 172.21.15.46:0/1418552865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c835dd80 0x55f9c84662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:55:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f285ad9d700 1 --2- 172.21.15.46:0/1418552865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c8460480 0x55f9c8460890 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f28540a8fd0 tx=0x7f2854009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de773d1fd2c27fef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.810+0000 7f2848ff9700 1 -- 172.21.15.46:0/1418552865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28540a9740 con 0x55f9c8460480 2024-05-09T04:55:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.811+0000 7f2848ff9700 1 -- 172.21.15.46:0/1418552865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28540a9040 con 0x55f9c8460480 2024-05-09T04:55:58.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.811+0000 7f2848ff9700 1 -- 172.21.15.46:0/1418552865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28540a92a0 con 0x55f9c8460480 2024-05-09T04:55:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.811+0000 7f286f4adb80 1 -- 172.21.15.46:0/1418552865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c8460480 msgr2=0x55f9c8460890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.811+0000 7f286f4adb80 1 --2- 172.21.15.46:0/1418552865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c8460480 0x55f9c8460890 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f28540a8fd0 tx=0x7f2854009570 comp rx=0 tx=0).stop 2024-05-09T04:55:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.811+0000 7f286f4adb80 1 -- 172.21.15.46:0/1418552865 shutdown_connections 2024-05-09T04:55:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.811+0000 7f286f4adb80 1 --2- 172.21.15.46:0/1418552865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c8460480 0x55f9c8460890 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.811+0000 7f286f4adb80 1 --2- 172.21.15.46:0/1418552865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c835dd80 0x55f9c84662c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.811+0000 7f286f4adb80 1 --2- 172.21.15.46:0/1418552865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c82e3520 0x55f9c8466800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.811+0000 7f286f4adb80 1 -- 172.21.15.46:0/1418552865 >> 172.21.15.46:0/1418552865 conn(0x55f9c82e6ff0 msgr2=0x55f9c845b2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.812+0000 7f286f4adb80 1 -- 172.21.15.46:0/1418552865 shutdown_connections 2024-05-09T04:55:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.812+0000 7f286f4adb80 1 -- 172.21.15.46:0/1418552865 wait complete. 2024-05-09T04:55:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.812+0000 7f286f4adb80 1 -- start start 2024-05-09T04:55:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.812+0000 7f286f4adb80 1 Processor -- start 2024-05-09T04:55:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.812+0000 7f286f4adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c82e3520 0x55f9c846a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.812+0000 7f286f4adb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c835dd80 0x55f9c846a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.812+0000 7f286f4adb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c8460480 0x55f9c846abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f286f4adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f9c84611e0 con 0x55f9c82e3520 2024-05-09T04:55:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f286f4adb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f9c831dba0 con 0x55f9c8460480 2024-05-09T04:55:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f286f4adb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f9c846d0a0 con 0x55f9c835dd80 2024-05-09T04:55:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f285a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c82e3520 0x55f9c846a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f286f4adb80 10 client.0 _ll_register_callbacks cb 0x55f9c81f4740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f285a59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c82e3520 0x55f9c846a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56962/0 (socket says 172.21.15.46:56962) 2024-05-09T04:55:58.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f285a59c700 1 -- 172.21.15.46:0/2834195708 learned_addr learned my addr 172.21.15.46:0/2834195708 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f285ad9d700 1 --2- 172.21.15.46:0/2834195708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c8460480 0x55f9c846abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f2859d9b700 1 --2- 172.21.15.46:0/2834195708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c835dd80 0x55f9c846a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f285a59c700 1 -- 172.21.15.46:0/2834195708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c835dd80 msgr2=0x55f9c846a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f285a59c700 1 --2- 172.21.15.46:0/2834195708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c835dd80 0x55f9c846a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f285a59c700 1 -- 172.21.15.46:0/2834195708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c8460480 msgr2=0x55f9c846abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f285a59c700 1 --2- 172.21.15.46:0/2834195708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c8460480 0x55f9c846abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f285a59c700 1 -- 172.21.15.46:0/2834195708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c81fe510 con 0x55f9c82e3520 2024-05-09T04:55:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f285a59c700 1 --2- 172.21.15.46:0/2834195708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c82e3520 0x55f9c846a130 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f284c0098e0 tx=0x7f284c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d9430f901e74b9f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f2841ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:55:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f2841ffb700 1 -- 172.21.15.46:0/2834195708 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c033060 con 0x55f9c82e3520 2024-05-09T04:55:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f2841ffb700 1 -- 172.21.15.46:0/2834195708 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f284c00f660 con 0x55f9c82e3520 2024-05-09T04:55:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.813+0000 7f286f4adb80 1 -- 172.21.15.46:0/2834195708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28540a9040 con 0x55f9c82e3520 2024-05-09T04:55:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.814+0000 7f2841ffb700 1 -- 172.21.15.46:0/2834195708 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f284c0056c0 con 0x55f9c82e3520 2024-05-09T04:55:58.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.814+0000 7f286f4adb80 10 client.16539 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.814+0000 7f286f4adb80 1 -- 172.21.15.46:0/2834195708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9c847bf20 con 0x55f9c82e3520 2024-05-09T04:55:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.814+0000 7f2841ffb700 1 -- 172.21.15.46:0/2834195708 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28540a9040 con 0x55f9c82e3520 2024-05-09T04:55:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.814+0000 7f2841ffb700 1 -- 172.21.15.46:0/2834195708 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f284c02d070 con 0x55f9c82e3520 2024-05-09T04:55:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.814+0000 7f286f4adb80 10 client.16539 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.814+0000 7f286f4adb80 10 client.16539 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:55:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.814+0000 7f286f4adb80 20 client.16539 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.814+0000 7f286f4adb80 1 --2- 172.21.15.46:0/2834195708 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f9c847e600 0x55f9c8480a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.814+0000 7f286f4adb80 4 client.16539 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:55:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.814+0000 7f286f4adb80 1 -- 172.21.15.46:0/2834195708 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f9c8390b00 con 0x55f9c847e600 2024-05-09T04:55:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.814+0000 7f2859d9b700 1 --2- 172.21.15.46:0/2834195708 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f9c847e600 0x55f9c8480a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.815+0000 7f2859d9b700 1 --2- 172.21.15.46:0/2834195708 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f9c847e600 0x55f9c8480a80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=28b360c48e5f3359 server_cookie=eed480cb47592471 in_seq=0 out_seq=0 2024-05-09T04:55:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.815+0000 7f2841ffb700 10 client.16539 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.816+0000 7f2841ffb700 1 -- 172.21.15.46:0/2834195708 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f9c8390b00 con 0x55f9c847e600 2024-05-09T04:55:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.816+0000 7f2841ffb700 10 client.16539 handle_command_reply: tid=0 2024-05-09T04:55:58.819 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.816+0000 7f286f4adb80 1 client.16539 shutdown 2024-05-09T04:55:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.816+0000 7f286f4adb80 1 -- 172.21.15.46:0/2834195708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c82e3520 msgr2=0x55f9c846a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.816+0000 7f286f4adb80 1 --2- 172.21.15.46:0/2834195708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c82e3520 0x55f9c846a130 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f284c0098e0 tx=0x7f284c009310 comp rx=0 tx=0).stop 2024-05-09T04:55:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.816+0000 7f286f4adb80 1 -- 172.21.15.46:0/2834195708 shutdown_connections 2024-05-09T04:55:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.816+0000 7f286f4adb80 1 --2- 172.21.15.46:0/2834195708 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f9c847e600 0x55f9c8480a80 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.816+0000 7f286f4adb80 1 --2- 172.21.15.46:0/2834195708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c8460480 0x55f9c846abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.816+0000 7f286f4adb80 1 --2- 172.21.15.46:0/2834195708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c82e3520 0x55f9c846a130 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.816+0000 7f286f4adb80 1 --2- 172.21.15.46:0/2834195708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c835dd80 0x55f9c846a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.816+0000 7f286f4adb80 1 -- 172.21.15.46:0/2834195708 >> 172.21.15.46:0/2834195708 conn(0x55f9c82e6ff0 msgr2=0x55f9c8455070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.816+0000 7f2841ffb700 0 client.16539 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.816+0000 7f286f4adb80 1 -- 172.21.15.46:0/2834195708 shutdown_connections 2024-05-09T04:55:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.816+0000 7f286f4adb80 1 -- 172.21.15.46:0/2834195708 wait complete. 2024-05-09T04:55:58.820 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.816+0000 7f286f4adb80 20 client.16539 trim_cache size 0 max 16384 2024-05-09T04:55:58.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.837+0000 7f286f4adb80 1 Processor -- start 2024-05-09T04:55:58.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.837+0000 7f286f4adb80 1 -- start start 2024-05-09T04:55:58.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.837+0000 7f286f4adb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c82e3520 0x55f9c8455aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.837+0000 7f286f4adb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c835dd80 0x55f9c846a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.837+0000 7f286f4adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c8460480 0x55f9c8463d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.837+0000 7f286f4adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f285c1af690 con 0x55f9c8460480 2024-05-09T04:55:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.837+0000 7f286f4adb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f285c1acf20 con 0x55f9c835dd80 2024-05-09T04:55:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.837+0000 7f286f4adb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f285c1aad70 con 0x55f9c82e3520 2024-05-09T04:55:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.837+0000 7f285ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c8460480 0x55f9c8463d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.837+0000 7f285a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c82e3520 0x55f9c8455aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.837+0000 7f285ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c8460480 0x55f9c8463d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56964/0 (socket says 172.21.15.46:56964) 2024-05-09T04:55:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.837+0000 7f285ad9d700 1 -- 172.21.15.46:0/3330891226 learned_addr learned my addr 172.21.15.46:0/3330891226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.837+0000 7f2859d9b700 1 --2- 172.21.15.46:0/3330891226 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c835dd80 0x55f9c846a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.838+0000 7f285ad9d700 1 -- 172.21.15.46:0/3330891226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c82e3520 msgr2=0x55f9c8455aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.838+0000 7f285ad9d700 1 --2- 172.21.15.46:0/3330891226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c82e3520 0x55f9c8455aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.838+0000 7f285ad9d700 1 -- 172.21.15.46:0/3330891226 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c835dd80 msgr2=0x55f9c846a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.838+0000 7f285ad9d700 1 --2- 172.21.15.46:0/3330891226 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c835dd80 0x55f9c846a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.838+0000 7f285ad9d700 1 -- 172.21.15.46:0/3330891226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9c8457a50 con 0x55f9c8460480 2024-05-09T04:55:58.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.838+0000 7f285a59c700 1 --2- 172.21.15.46:0/3330891226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c82e3520 0x55f9c8455aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.838+0000 7f285ad9d700 1 --2- 172.21.15.46:0/3330891226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c8460480 0x55f9c8463d20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2854004db0 tx=0x7f28540a8e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cccf8bb7e880edc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.838+0000 7f2843fff700 1 -- 172.21.15.46:0/3330891226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28540a9740 con 0x55f9c8460480 2024-05-09T04:55:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.838+0000 7f2843fff700 1 -- 172.21.15.46:0/3330891226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28540b1ea0 con 0x55f9c8460480 2024-05-09T04:55:58.840 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.838+0000 7f2843fff700 1 -- 172.21.15.46:0/3330891226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28540a92a0 con 0x55f9c8460480 2024-05-09T04:55:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.839+0000 7f286f4adb80 1 -- 172.21.15.46:0/3330891226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c8460480 msgr2=0x55f9c8463d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.839+0000 7f286f4adb80 1 --2- 172.21.15.46:0/3330891226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c8460480 0x55f9c8463d20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f2854004db0 tx=0x7f28540a8e10 comp rx=0 tx=0).stop 2024-05-09T04:55:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.839+0000 7f286f4adb80 1 -- 172.21.15.46:0/3330891226 shutdown_connections 2024-05-09T04:55:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.839+0000 7f286f4adb80 1 --2- 172.21.15.46:0/3330891226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c8460480 0x55f9c8463d20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.839+0000 7f286f4adb80 1 --2- 172.21.15.46:0/3330891226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c82e3520 0x55f9c8455aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.839+0000 7f286f4adb80 1 --2- 172.21.15.46:0/3330891226 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c835dd80 0x55f9c846a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.839+0000 7f286f4adb80 1 -- 172.21.15.46:0/3330891226 >> 172.21.15.46:0/3330891226 conn(0x55f9c82e6ff0 msgr2=0x55f9c8469090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.839+0000 7f286f4adb80 1 -- 172.21.15.46:0/3330891226 shutdown_connections 2024-05-09T04:55:58.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.839+0000 7f286f4adb80 1 -- 172.21.15.46:0/3330891226 wait complete. 2024-05-09T04:55:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.839+0000 7f286f4adb80 1 -- start start 2024-05-09T04:55:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.839+0000 7f286f4adb80 1 Processor -- start 2024-05-09T04:55:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f286f4adb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c82e3520 0x55f9c8472220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f286f4adb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c835dd80 0x55f9c8472760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f286f4adb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c8460480 0x55f9c846f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f286f4adb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f285c1ab3e0 con 0x55f9c835dd80 2024-05-09T04:55:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f286f4adb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f285c1ac850 con 0x55f9c8460480 2024-05-09T04:55:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f286f4adb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f285c1abab0 con 0x55f9c82e3520 2024-05-09T04:55:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f286f4adb80 10 client.0 _ll_register_callbacks cb 0x55f9c81f4740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:55:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f2859d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c835dd80 0x55f9c8472760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f285ad9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c8460480 0x55f9c846f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f2859d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c835dd80 0x55f9c8472760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56980/0 (socket says 172.21.15.46:56980) 2024-05-09T04:55:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f2859d9b700 1 -- 172.21.15.46:0/2302483878 learned_addr learned my addr 172.21.15.46:0/2302483878 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:55:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f285a59c700 1 --2- 172.21.15.46:0/2302483878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c82e3520 0x55f9c8472220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f285ad9d700 1 -- 172.21.15.46:0/2302483878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c82e3520 msgr2=0x55f9c8472220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f285ad9d700 1 --2- 172.21.15.46:0/2302483878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c82e3520 0x55f9c8472220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f285ad9d700 1 -- 172.21.15.46:0/2302483878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c835dd80 msgr2=0x55f9c8472760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:55:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f285ad9d700 1 --2- 172.21.15.46:0/2302483878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c835dd80 0x55f9c8472760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f285ad9d700 1 -- 172.21.15.46:0/2302483878 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9c846f910 con 0x55f9c8460480 2024-05-09T04:55:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.840+0000 7f2859d9b700 1 --2- 172.21.15.46:0/2302483878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c835dd80 0x55f9c8472760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:55:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.841+0000 7f285ad9d700 1 --2- 172.21.15.46:0/2302483878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c8460480 0x55f9c846f1f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f28540a8e80 tx=0x7f2854005260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8af137b2993ccb29 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:55:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.841+0000 7f2841ffb700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T04:55:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.841+0000 7f2841ffb700 1 -- 172.21.15.46:0/2302483878 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28540a9040 con 0x55f9c8460480 2024-05-09T04:55:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.841+0000 7f2841ffb700 1 -- 172.21.15.46:0/2302483878 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f28540a94b0 con 0x55f9c8460480 2024-05-09T04:55:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.841+0000 7f286f4adb80 1 -- 172.21.15.46:0/2302483878 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28540b1ea0 con 0x55f9c8460480 2024-05-09T04:55:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.841+0000 7f2841ffb700 1 -- 172.21.15.46:0/2302483878 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f28540b1a10 con 0x55f9c8460480 2024-05-09T04:55:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.841+0000 7f286f4adb80 10 client.26132 fetch_fsmap learned FSMap version 25 2024-05-09T04:55:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.841+0000 7f286f4adb80 1 -- 172.21.15.46:0/2302483878 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f9c84e97e0 con 0x55f9c8460480 2024-05-09T04:55:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.841+0000 7f2841ffb700 1 -- 172.21.15.46:0/2302483878 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28540b1ea0 con 0x55f9c8460480 2024-05-09T04:55:58.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.841+0000 7f2841ffb700 1 -- 172.21.15.46:0/2302483878 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f28540cb040 con 0x55f9c8460480 2024-05-09T04:55:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.842+0000 7f286f4adb80 10 client.26132 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:55:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.842+0000 7f286f4adb80 10 client.26132 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:55:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.842+0000 7f286f4adb80 20 client.26132 populate_metadata read hostname 'smithi046' 2024-05-09T04:55:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.842+0000 7f286f4adb80 1 --2- 172.21.15.46:0/2302483878 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f9c84e9a60 0x55f9c84ebe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:55:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.842+0000 7f286f4adb80 4 client.26132 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:55:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.842+0000 7f286f4adb80 1 -- 172.21.15.46:0/2302483878 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f9c838aba0 con 0x55f9c84e9a60 2024-05-09T04:55:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.842+0000 7f285a59c700 1 --2- 172.21.15.46:0/2302483878 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f9c84e9a60 0x55f9c84ebe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:55:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.842+0000 7f285a59c700 1 --2- 172.21.15.46:0/2302483878 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f9c84e9a60 0x55f9c84ebe80 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1e85ea0359af41c server_cookie=628ab41676413f54 in_seq=0 out_seq=0 2024-05-09T04:55:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.842+0000 7f2841ffb700 10 client.26132 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.843+0000 7f2841ffb700 1 -- 172.21.15.46:0/2302483878 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f9c838aba0 con 0x55f9c84e9a60 2024-05-09T04:55:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.843+0000 7f2841ffb700 10 client.26132 handle_command_reply: tid=0 2024-05-09T04:55:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.843+0000 7f286f4adb80 1 client.26132 shutdown 2024-05-09T04:55:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.843+0000 7f286f4adb80 1 -- 172.21.15.46:0/2302483878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c8460480 msgr2=0x55f9c846f1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.843+0000 7f286f4adb80 1 --2- 172.21.15.46:0/2302483878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c8460480 0x55f9c846f1f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f28540a8e80 tx=0x7f2854005260 comp rx=0 tx=0).stop 2024-05-09T04:55:58.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.843+0000 7f286f4adb80 1 -- 172.21.15.46:0/2302483878 shutdown_connections 2024-05-09T04:55:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.843+0000 7f286f4adb80 1 --2- 172.21.15.46:0/2302483878 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55f9c84e9a60 0x55f9c84ebe80 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f286f4adb80 1 --2- 172.21.15.46:0/2302483878 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f9c8460480 0x55f9c846f1f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f286f4adb80 1 --2- 172.21.15.46:0/2302483878 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f9c82e3520 0x55f9c8472220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f286f4adb80 1 --2- 172.21.15.46:0/2302483878 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f9c835dd80 0x55f9c8472760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f286f4adb80 1 -- 172.21.15.46:0/2302483878 >> 172.21.15.46:0/2302483878 conn(0x55f9c82e6ff0 msgr2=0x55f9c8463d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f2841ffb700 0 client.26132 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:55:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f286f4adb80 1 -- 172.21.15.46:0/2302483878 shutdown_connections 2024-05-09T04:55:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f286f4adb80 1 -- 172.21.15.46:0/2302483878 wait complete. 2024-05-09T04:55:58.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f286f4adb80 20 client.26132 trim_cache size 0 max 16384 2024-05-09T04:55:58.848 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:55:58.848 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:55:58.848 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "71e428c2-659e-474a-a01f-b8e8f748e929", 2024-05-09T04:55:58.849 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:55:58.849 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:55:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f2860a97700 1 -- 172.21.15.46:0/3266878024 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f28440b29f0 msgr2=0x7f28440b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f2860a97700 1 --2- 172.21.15.46:0/3266878024 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f28440b29f0 0x7f28440b4f00 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f284c000c00 tx=0x7f284c01a040 comp rx=0 tx=0).stop 2024-05-09T04:55:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f2860a97700 1 -- 172.21.15.46:0/3266878024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285c108760 msgr2=0x7f285c19c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:55:58.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f2860a97700 1 --2- 172.21.15.46:0/3266878024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285c108760 0x7f285c19c7a0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f2854007ee0 tx=0x7f285400d3b0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f2860a97700 1 -- 172.21.15.46:0/3266878024 shutdown_connections 2024-05-09T04:55:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f2860a97700 1 --2- 172.21.15.46:0/3266878024 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f28440b29f0 0x7f28440b4f00 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f2860a97700 1 --2- 172.21.15.46:0/3266878024 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f285c108760 0x7f285c19c7a0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f2860a97700 1 --2- 172.21.15.46:0/3266878024 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f285c1034e0 0x7f285c19bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f2860a97700 1 --2- 172.21.15.46:0/3266878024 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f285c107e10 0x7f285c19c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:55:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.844+0000 7f2860a97700 1 -- 172.21.15.46:0/3266878024 >> 172.21.15.46:0/3266878024 conn(0x7f285c0fef30 msgr2=0x7f285c1096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:55:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.845+0000 7f2860a97700 1 -- 172.21.15.46:0/3266878024 shutdown_connections 2024-05-09T04:55:58.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:55:58.845+0000 7f2860a97700 1 -- 172.21.15.46:0/3266878024 wait complete. 2024-05-09T04:55:59.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:58 smithi110 ceph-mon[34904]: pgmap v149: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:55:59.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:58 smithi046 ceph-mon[30136]: pgmap v149: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:55:59.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:58 smithi072 ceph-mon[34088]: pgmap v149: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:55:59.681 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:55:59.937 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:59 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:56:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:59 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:56:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:59 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:56:00.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:55:59 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:56:00.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:59 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:56:00.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:59 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:56:00.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:59 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:56:00.059 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:55:59 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:56:00.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:59 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:56:00.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:59 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:56:00.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:59 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:56:00.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:55:59 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:56:01.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:00 smithi110 ceph-mon[34904]: pgmap v150: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-09T04:56:01.077 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:00 smithi046 ceph-mon[30136]: pgmap v150: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-09T04:56:01.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:00 smithi072 ceph-mon[34088]: pgmap v150: 97 pgs: 97 active+clean; 11 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-09T04:56:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.679+0000 7f687a37b700 1 -- 172.21.15.46:0/795747807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6874108740 msgr2=0x7f68741100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.679+0000 7f687a37b700 1 --2- 172.21.15.46:0/795747807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6874108740 0x7f68741100a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f686c00b0d0 tx=0x7f686c00b3e0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.680+0000 7f687a37b700 1 -- 172.21.15.46:0/795747807 shutdown_connections 2024-05-09T04:56:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.680+0000 7f687a37b700 1 --2- 172.21.15.46:0/795747807 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6874108740 0x7f68741100a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.680+0000 7f687a37b700 1 --2- 172.21.15.46:0/795747807 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68741034c0 0x7f68741038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.680+0000 7f687a37b700 1 --2- 172.21.15.46:0/795747807 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6874107df0 0x7f6874108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.680+0000 7f687a37b700 1 -- 172.21.15.46:0/795747807 >> 172.21.15.46:0/795747807 conn(0x7f68740fedc0 msgr2=0x7f68741011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.680+0000 7f687a37b700 1 -- 172.21.15.46:0/795747807 shutdown_connections 2024-05-09T04:56:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.680+0000 7f687a37b700 1 -- 172.21.15.46:0/795747807 wait complete. 2024-05-09T04:56:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.681+0000 7f687a37b700 1 Processor -- start 2024-05-09T04:56:01.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.681+0000 7f687a37b700 1 -- start start 2024-05-09T04:56:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.681+0000 7f687a37b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68741034c0 0x7f687419bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.681+0000 7f687a37b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6874107df0 0x7f687419c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.681+0000 7f687a37b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6874108740 0x7f687419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.681+0000 7f687a37b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6874113390 con 0x7f6874108740 2024-05-09T04:56:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.681+0000 7f687a37b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6874113210 con 0x7f68741034c0 2024-05-09T04:56:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.681+0000 7f687a37b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6874113510 con 0x7f6874107df0 2024-05-09T04:56:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.682+0000 7f6878918700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6874108740 0x7f687419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.682+0000 7f6878918700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6874108740 0x7f687419c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57002/0 (socket says 172.21.15.46:57002) 2024-05-09T04:56:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.682+0000 7f6878918700 1 -- 172.21.15.46:0/1203866853 learned_addr learned my addr 172.21.15.46:0/1203866853 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.682+0000 7f68737fe700 1 --2- 172.21.15.46:0/1203866853 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6874107df0 0x7f687419c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.682+0000 7f6873fff700 1 --2- 172.21.15.46:0/1203866853 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68741034c0 0x7f687419bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.682+0000 7f6878918700 1 -- 172.21.15.46:0/1203866853 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6874107df0 msgr2=0x7f687419c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.682+0000 7f6878918700 1 --2- 172.21.15.46:0/1203866853 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6874107df0 0x7f687419c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.682+0000 7f6878918700 1 -- 172.21.15.46:0/1203866853 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68741034c0 msgr2=0x7f687419bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.682+0000 7f6878918700 1 --2- 172.21.15.46:0/1203866853 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68741034c0 0x7f687419bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.682+0000 7f6878918700 1 -- 172.21.15.46:0/1203866853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68741a6fa0 con 0x7f6874108740 2024-05-09T04:56:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.682+0000 7f6878918700 1 --2- 172.21.15.46:0/1203866853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6874108740 0x7f687419c6f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f686c009850 tx=0x7f686c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:01.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.683+0000 7f68717fa700 1 -- 172.21.15.46:0/1203866853 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c01f070 con 0x7f6874108740 2024-05-09T04:56:01.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.683+0000 7f687a37b700 1 -- 172.21.15.46:0/1203866853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68741a7230 con 0x7f6874108740 2024-05-09T04:56:01.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.683+0000 7f68717fa700 1 -- 172.21.15.46:0/1203866853 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f686c004c50 con 0x7f6874108740 2024-05-09T04:56:01.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.683+0000 7f68717fa700 1 -- 172.21.15.46:0/1203866853 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686c01b690 con 0x7f6874108740 2024-05-09T04:56:01.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.683+0000 7f687a37b700 1 -- 172.21.15.46:0/1203866853 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68741a7680 con 0x7f6874108740 2024-05-09T04:56:01.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.686+0000 7f68717fa700 1 -- 172.21.15.46:0/1203866853 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f686c022070 con 0x7f6874108740 2024-05-09T04:56:01.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.687+0000 7f68717fa700 1 --2- 172.21.15.46:0/1203866853 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f685c0b2a70 0x7f685c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.687+0000 7f68717fa700 1 -- 172.21.15.46:0/1203866853 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f686c0e3220 con 0x7f6874108740 2024-05-09T04:56:01.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.687+0000 7f6873fff700 1 --2- 172.21.15.46:0/1203866853 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f685c0b2a70 0x7f685c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.687+0000 7f6888d91b80 1 Processor -- start 2024-05-09T04:56:01.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.687+0000 7f6888d91b80 1 -- start start 2024-05-09T04:56:01.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6873fff700 1 --2- 172.21.15.46:0/1203866853 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f685c0b2a70 0x7f685c0b4f80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6864005950 tx=0x7f686400b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:01.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6888d91b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c449fd00 0x55e1c45a8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6888d91b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c44254a0 0x55e1c45a8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6888d91b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 0x55e1c45a2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6888d91b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e1c44c50a0 con 0x55e1c449fd00 2024-05-09T04:56:01.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6888d91b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e1c446a9a0 con 0x55e1c45a2400 2024-05-09T04:56:01.689 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6873fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c449fd00 0x55e1c45a8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6888d91b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e1c44c1060 con 0x55e1c44254a0 2024-05-09T04:56:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6873fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c449fd00 0x55e1c45a8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57004/0 (socket says 172.21.15.46:57004) 2024-05-09T04:56:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6873fff700 1 -- 172.21.15.46:0/946272037 learned_addr learned my addr 172.21.15.46:0/946272037 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f68737fe700 1 --2- 172.21.15.46:0/946272037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c44254a0 0x55e1c45a8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6878918700 1 --2- 172.21.15.46:0/946272037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 0x55e1c45a2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6873fff700 1 -- 172.21.15.46:0/946272037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c44254a0 msgr2=0x55e1c45a8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6873fff700 1 --2- 172.21.15.46:0/946272037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c44254a0 0x55e1c45a8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6873fff700 1 -- 172.21.15.46:0/946272037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 msgr2=0x55e1c45a2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6873fff700 1 --2- 172.21.15.46:0/946272037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 0x55e1c45a2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6873fff700 1 -- 172.21.15.46:0/946272037 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c445b500 con 0x55e1c449fd00 2024-05-09T04:56:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.688+0000 7f6873fff700 1 --2- 172.21.15.46:0/946272037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c449fd00 0x55e1c45a8240 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6864010fa0 tx=0x7f6864013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fab6b41bfab974 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.689+0000 7f685a7fc700 1 -- 172.21.15.46:0/946272037 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686400be60 con 0x55e1c449fd00 2024-05-09T04:56:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.689+0000 7f685a7fc700 1 -- 172.21.15.46:0/946272037 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6864011db0 con 0x55e1c449fd00 2024-05-09T04:56:01.690 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.689+0000 7f685a7fc700 1 -- 172.21.15.46:0/946272037 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686401c770 con 0x55e1c449fd00 2024-05-09T04:56:01.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.690+0000 7f6888d91b80 1 -- 172.21.15.46:0/946272037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c449fd00 msgr2=0x55e1c45a8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:01.693 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.690+0000 7f6888d91b80 1 --2- 172.21.15.46:0/946272037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c449fd00 0x55e1c45a8240 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f6864010fa0 tx=0x7f6864013000 comp rx=0 tx=0).stop 2024-05-09T04:56:01.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.690+0000 7f6888d91b80 1 -- 172.21.15.46:0/946272037 shutdown_connections 2024-05-09T04:56:01.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.690+0000 7f6888d91b80 1 --2- 172.21.15.46:0/946272037 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 0x55e1c45a2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.690+0000 7f6888d91b80 1 --2- 172.21.15.46:0/946272037 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c449fd00 0x55e1c45a8240 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.690+0000 7f6888d91b80 1 --2- 172.21.15.46:0/946272037 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c44254a0 0x55e1c45a8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.690+0000 7f6888d91b80 1 -- 172.21.15.46:0/946272037 >> 172.21.15.46:0/946272037 conn(0x55e1c4428f70 msgr2=0x55e1c459d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:01.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.690+0000 7f6888d91b80 1 -- 172.21.15.46:0/946272037 shutdown_connections 2024-05-09T04:56:01.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.690+0000 7f6888d91b80 1 -- 172.21.15.46:0/946272037 wait complete. 2024-05-09T04:56:01.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.691+0000 7f6888d91b80 1 -- start start 2024-05-09T04:56:01.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.691+0000 7f6888d91b80 1 Processor -- start 2024-05-09T04:56:01.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.691+0000 7f6888d91b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c44254a0 0x55e1c45ac080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f6888d91b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c449fd00 0x55e1c45ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.694 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f6888d91b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 0x55e1c45acb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f6888d91b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e1c45a3160 con 0x55e1c44254a0 2024-05-09T04:56:01.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f6888d91b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e1c445fb20 con 0x55e1c45a2400 2024-05-09T04:56:01.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f6888d91b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e1c45af020 con 0x55e1c449fd00 2024-05-09T04:56:01.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f6888d91b80 10 client.0 _ll_register_callbacks cb 0x55e1c43366c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:01.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f6873fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c44254a0 0x55e1c45ac080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f6873fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c44254a0 0x55e1c45ac080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57012/0 (socket says 172.21.15.46:57012) 2024-05-09T04:56:01.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f6873fff700 1 -- 172.21.15.46:0/988076000 learned_addr learned my addr 172.21.15.46:0/988076000 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:01.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f68737fe700 1 --2- 172.21.15.46:0/988076000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c449fd00 0x55e1c45ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f6878918700 1 --2- 172.21.15.46:0/988076000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 0x55e1c45acb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f68737fe700 1 -- 172.21.15.46:0/988076000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 msgr2=0x55e1c45acb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:01.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f68737fe700 1 --2- 172.21.15.46:0/988076000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 0x55e1c45acb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f68737fe700 1 -- 172.21.15.46:0/988076000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c44254a0 msgr2=0x55e1c45ac080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:01.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f68737fe700 1 --2- 172.21.15.46:0/988076000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c44254a0 0x55e1c45ac080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.695 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f68737fe700 1 -- 172.21.15.46:0/988076000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c4340490 con 0x55e1c449fd00 2024-05-09T04:56:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f6873fff700 1 --2- 172.21.15.46:0/988076000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c44254a0 0x55e1c45ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:56:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f68737fe700 1 --2- 172.21.15.46:0/988076000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c449fd00 0x55e1c45ac5c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55e1c459ec90 tx=0x7f6868012960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f320aa5a7ea1592f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.692+0000 7f68537fe700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:56:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.693+0000 7f68537fe700 1 -- 172.21.15.46:0/988076000 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868004020 con 0x55e1c449fd00 2024-05-09T04:56:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.693+0000 7f68537fe700 1 -- 172.21.15.46:0/988076000 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f68680041c0 con 0x55e1c449fd00 2024-05-09T04:56:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.693+0000 7f6888d91b80 1 -- 172.21.15.46:0/988076000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6864011db0 con 0x55e1c449fd00 2024-05-09T04:56:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.693+0000 7f68537fe700 1 -- 172.21.15.46:0/988076000 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f68680043c0 con 0x55e1c449fd00 2024-05-09T04:56:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.693+0000 7f6888d91b80 10 client.26047 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.693+0000 7f6888d91b80 1 -- 172.21.15.46:0/988076000 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1c45b9a10 con 0x55e1c449fd00 2024-05-09T04:56:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.693+0000 7f68537fe700 1 -- 172.21.15.46:0/988076000 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6864011db0 con 0x55e1c449fd00 2024-05-09T04:56:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.693+0000 7f68537fe700 1 -- 172.21.15.46:0/988076000 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f686801c6d0 con 0x55e1c449fd00 2024-05-09T04:56:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.693+0000 7f6888d91b80 10 client.26047 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.693+0000 7f6888d91b80 10 client.26047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:56:01.696 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.693+0000 7f6888d91b80 20 client.26047 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:01.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.693+0000 7f6888d91b80 1 --2- 172.21.15.46:0/988076000 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e1c45bbff0 0x55e1c45be410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.693+0000 7f6888d91b80 4 client.26047 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:56:01.697 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.693+0000 7f6888d91b80 1 -- 172.21.15.46:0/988076000 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e1c44d2a80 con 0x55e1c45bbff0 2024-05-09T04:56:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.694+0000 7f6873fff700 1 --2- 172.21.15.46:0/988076000 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e1c45bbff0 0x55e1c45be410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.694+0000 7f68537fe700 10 client.26047 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.694+0000 7f6873fff700 1 --2- 172.21.15.46:0/988076000 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e1c45bbff0 0x55e1c45be410 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab75ae5a483f3114 server_cookie=a0e7e39e2c8457e9 in_seq=0 out_seq=0 2024-05-09T04:56:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.695+0000 7f68537fe700 1 -- 172.21.15.46:0/988076000 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e1c44d2a80 con 0x55e1c45bbff0 2024-05-09T04:56:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.695+0000 7f68537fe700 10 client.26047 handle_command_reply: tid=0 2024-05-09T04:56:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.695+0000 7f6888d91b80 1 client.26047 shutdown 2024-05-09T04:56:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.696+0000 7f6888d91b80 1 -- 172.21.15.46:0/988076000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c449fd00 msgr2=0x55e1c45ac5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.696+0000 7f6888d91b80 1 --2- 172.21.15.46:0/988076000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c449fd00 0x55e1c45ac5c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x55e1c459ec90 tx=0x7f6868012960 comp rx=0 tx=0).stop 2024-05-09T04:56:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.696+0000 7f6888d91b80 1 -- 172.21.15.46:0/988076000 shutdown_connections 2024-05-09T04:56:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.696+0000 7f6888d91b80 1 --2- 172.21.15.46:0/988076000 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e1c45bbff0 0x55e1c45be410 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.696+0000 7f6888d91b80 1 --2- 172.21.15.46:0/988076000 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 0x55e1c45acb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.696+0000 7f6888d91b80 1 --2- 172.21.15.46:0/988076000 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c44254a0 0x55e1c45ac080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.696+0000 7f6888d91b80 1 --2- 172.21.15.46:0/988076000 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c449fd00 0x55e1c45ac5c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.696+0000 7f6888d91b80 1 -- 172.21.15.46:0/988076000 >> 172.21.15.46:0/988076000 conn(0x55e1c4428f70 msgr2=0x55e1c4596ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.696+0000 7f68537fe700 0 client.26047 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.696+0000 7f6888d91b80 1 -- 172.21.15.46:0/988076000 shutdown_connections 2024-05-09T04:56:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.696+0000 7f6888d91b80 1 -- 172.21.15.46:0/988076000 wait complete. 2024-05-09T04:56:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.696+0000 7f6888d91b80 20 client.26047 trim_cache size 0 max 16384 2024-05-09T04:56:01.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.717+0000 7f6888d91b80 1 Processor -- start 2024-05-09T04:56:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.717+0000 7f6888d91b80 1 -- start start 2024-05-09T04:56:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.717+0000 7f6888d91b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c44254a0 0x55e1c45ab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.717+0000 7f6888d91b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c449fd00 0x55e1c4597a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.717+0000 7f6888d91b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 0x55e1c45ac750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.717+0000 7f6888d91b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68741b1740 con 0x55e1c449fd00 2024-05-09T04:56:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.717+0000 7f6888d91b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f68741aefd0 con 0x55e1c45a2400 2024-05-09T04:56:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.717+0000 7f6888d91b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f68741ace20 con 0x55e1c44254a0 2024-05-09T04:56:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.717+0000 7f68737fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c449fd00 0x55e1c4597a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.717+0000 7f68737fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c449fd00 0x55e1c4597a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57020/0 (socket says 172.21.15.46:57020) 2024-05-09T04:56:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.717+0000 7f68737fe700 1 -- 172.21.15.46:0/1453122935 learned_addr learned my addr 172.21.15.46:0/1453122935 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.717+0000 7f6878918700 1 --2- 172.21.15.46:0/1453122935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 0x55e1c45ac750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.717+0000 7f6873fff700 1 --2- 172.21.15.46:0/1453122935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c44254a0 0x55e1c45ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.718+0000 7f68737fe700 1 -- 172.21.15.46:0/1453122935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c44254a0 msgr2=0x55e1c45ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.718+0000 7f68737fe700 1 --2- 172.21.15.46:0/1453122935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c44254a0 0x55e1c45ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.718+0000 7f68737fe700 1 -- 172.21.15.46:0/1453122935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 msgr2=0x55e1c45ac750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.718+0000 7f68737fe700 1 --2- 172.21.15.46:0/1453122935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 0x55e1c45ac750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.718+0000 7f68737fe700 1 -- 172.21.15.46:0/1453122935 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1c45abe50 con 0x55e1c449fd00 2024-05-09T04:56:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.718+0000 7f6873fff700 1 --2- 172.21.15.46:0/1453122935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c44254a0 0x55e1c45ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:56:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.718+0000 7f6878918700 1 --2- 172.21.15.46:0/1453122935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 0x55e1c45ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:56:01.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.718+0000 7f68737fe700 1 --2- 172.21.15.46:0/1453122935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c449fd00 0x55e1c4597a20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6868005950 tx=0x7f6868005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61705556a6619c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.718+0000 7f6859ffb700 1 -- 172.21.15.46:0/1453122935 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6868028030 con 0x55e1c449fd00 2024-05-09T04:56:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.718+0000 7f6859ffb700 1 -- 172.21.15.46:0/1453122935 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f686801c470 con 0x55e1c449fd00 2024-05-09T04:56:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.718+0000 7f6859ffb700 1 -- 172.21.15.46:0/1453122935 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f686801a700 con 0x55e1c449fd00 2024-05-09T04:56:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.719+0000 7f6888d91b80 1 -- 172.21.15.46:0/1453122935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c449fd00 msgr2=0x55e1c4597a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.719+0000 7f6888d91b80 1 --2- 172.21.15.46:0/1453122935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c449fd00 0x55e1c4597a20 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f6868005950 tx=0x7f6868005780 comp rx=0 tx=0).stop 2024-05-09T04:56:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.720+0000 7f6888d91b80 1 -- 172.21.15.46:0/1453122935 shutdown_connections 2024-05-09T04:56:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.720+0000 7f6888d91b80 1 --2- 172.21.15.46:0/1453122935 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c45a2400 0x55e1c45ac750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.720+0000 7f6888d91b80 1 --2- 172.21.15.46:0/1453122935 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c44254a0 0x55e1c45ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.720+0000 7f6888d91b80 1 --2- 172.21.15.46:0/1453122935 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c449fd00 0x55e1c4597a20 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.720+0000 7f6888d91b80 1 -- 172.21.15.46:0/1453122935 >> 172.21.15.46:0/1453122935 conn(0x55e1c4428f70 msgr2=0x55e1c45ac040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.720+0000 7f6888d91b80 1 -- 172.21.15.46:0/1453122935 shutdown_connections 2024-05-09T04:56:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.720+0000 7f6888d91b80 1 -- 172.21.15.46:0/1453122935 wait complete. 2024-05-09T04:56:01.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.720+0000 7f6888d91b80 1 -- start start 2024-05-09T04:56:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.720+0000 7f6888d91b80 1 Processor -- start 2024-05-09T04:56:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.721+0000 7f6888d91b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c44254a0 0x55e1c45af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.721+0000 7f6888d91b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c449fd00 0x55e1c45afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.721+0000 7f6888d91b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c45a2400 0x55e1c4624770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.721+0000 7f6888d91b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f68741ad490 con 0x55e1c44254a0 2024-05-09T04:56:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.721+0000 7f6888d91b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f68741ae900 con 0x55e1c449fd00 2024-05-09T04:56:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.721+0000 7f6888d91b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f68741adb60 con 0x55e1c45a2400 2024-05-09T04:56:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.721+0000 7f6888d91b80 10 client.0 _ll_register_callbacks cb 0x55e1c43366c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.721+0000 7f6873fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c44254a0 0x55e1c45af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.721+0000 7f6873fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c44254a0 0x55e1c45af550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57026/0 (socket says 172.21.15.46:57026) 2024-05-09T04:56:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.721+0000 7f6873fff700 1 -- 172.21.15.46:0/1956771671 learned_addr learned my addr 172.21.15.46:0/1956771671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.721+0000 7f68737fe700 1 --2- 172.21.15.46:0/1956771671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c449fd00 0x55e1c45afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.721+0000 7f6878918700 1 --2- 172.21.15.46:0/1956771671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c45a2400 0x55e1c4624770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.721+0000 7f6873fff700 1 -- 172.21.15.46:0/1956771671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c45a2400 msgr2=0x55e1c4624770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.721+0000 7f6873fff700 1 --2- 172.21.15.46:0/1956771671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c45a2400 0x55e1c4624770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.721+0000 7f6873fff700 1 -- 172.21.15.46:0/1956771671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c449fd00 msgr2=0x55e1c45afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.722+0000 7f6873fff700 1 --2- 172.21.15.46:0/1956771671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c449fd00 0x55e1c45afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.722+0000 7f6873fff700 1 -- 172.21.15.46:0/1956771671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1c4624cb0 con 0x55e1c44254a0 2024-05-09T04:56:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.722+0000 7f6873fff700 1 --2- 172.21.15.46:0/1956771671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c44254a0 0x55e1c45af550 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f686401eef0 tx=0x7f686401ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=682b679c52933f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.722+0000 7f68537fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:56:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.722+0000 7f68537fe700 1 -- 172.21.15.46:0/1956771671 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6864011db0 con 0x55e1c44254a0 2024-05-09T04:56:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.722+0000 7f68537fe700 1 -- 172.21.15.46:0/1956771671 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f686401e420 con 0x55e1c44254a0 2024-05-09T04:56:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.722+0000 7f6888d91b80 1 -- 172.21.15.46:0/1956771671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f686801c470 con 0x55e1c44254a0 2024-05-09T04:56:01.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.722+0000 7f68537fe700 1 -- 172.21.15.46:0/1956771671 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f686401e660 con 0x55e1c44254a0 2024-05-09T04:56:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.722+0000 7f6888d91b80 10 client.16563 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.722+0000 7f6888d91b80 1 -- 172.21.15.46:0/1956771671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e1c4625f00 con 0x55e1c44254a0 2024-05-09T04:56:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.722+0000 7f68537fe700 1 -- 172.21.15.46:0/1956771671 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f686801c470 con 0x55e1c44254a0 2024-05-09T04:56:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.723+0000 7f68537fe700 1 -- 172.21.15.46:0/1956771671 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f6864031410 con 0x55e1c44254a0 2024-05-09T04:56:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.723+0000 7f6888d91b80 10 client.16563 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.723+0000 7f6888d91b80 10 client.16563 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:56:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.723+0000 7f6888d91b80 20 client.16563 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.723+0000 7f6888d91b80 1 --2- 172.21.15.46:0/1956771671 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e1c46262d0 0x55e1c46286f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.723+0000 7f6888d91b80 4 client.16563 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:56:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.723+0000 7f6888d91b80 1 -- 172.21.15.46:0/1956771671 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55e1c44ccb20 con 0x55e1c46262d0 2024-05-09T04:56:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.723+0000 7f68737fe700 1 --2- 172.21.15.46:0/1956771671 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e1c46262d0 0x55e1c46286f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.723+0000 7f68537fe700 10 client.16563 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.724+0000 7f68737fe700 1 --2- 172.21.15.46:0/1956771671 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e1c46262d0 0x55e1c46286f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a68fe3f3473ff548 server_cookie=23e39bdb588cc66a in_seq=0 out_seq=0 2024-05-09T04:56:01.727 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.724+0000 7f68537fe700 1 -- 172.21.15.46:0/1956771671 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55e1c44ccb20 con 0x55e1c46262d0 2024-05-09T04:56:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.724+0000 7f68537fe700 10 client.16563 handle_command_reply: tid=0 2024-05-09T04:56:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.724+0000 7f6888d91b80 1 client.16563 shutdown 2024-05-09T04:56:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.724+0000 7f6888d91b80 1 -- 172.21.15.46:0/1956771671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c44254a0 msgr2=0x55e1c45af550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.724+0000 7f6888d91b80 1 --2- 172.21.15.46:0/1956771671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c44254a0 0x55e1c45af550 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f686401eef0 tx=0x7f686401ef20 comp rx=0 tx=0).stop 2024-05-09T04:56:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.724+0000 7f6888d91b80 1 -- 172.21.15.46:0/1956771671 shutdown_connections 2024-05-09T04:56:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.724+0000 7f6888d91b80 1 --2- 172.21.15.46:0/1956771671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e1c45a2400 0x55e1c4624770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.724+0000 7f6888d91b80 1 --2- 172.21.15.46:0/1956771671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e1c44254a0 0x55e1c45af550 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.724+0000 7f6888d91b80 1 --2- 172.21.15.46:0/1956771671 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55e1c46262d0 0x55e1c46286f0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.724+0000 7f6888d91b80 1 --2- 172.21.15.46:0/1956771671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e1c449fd00 0x55e1c45afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.724+0000 7f68537fe700 0 client.16563 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.724+0000 7f6888d91b80 1 -- 172.21.15.46:0/1956771671 >> 172.21.15.46:0/1956771671 conn(0x55e1c4428f70 msgr2=0x55e1c4597780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.724+0000 7f6888d91b80 1 -- 172.21.15.46:0/1956771671 shutdown_connections 2024-05-09T04:56:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.724+0000 7f6888d91b80 1 -- 172.21.15.46:0/1956771671 wait complete. 2024-05-09T04:56:01.728 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.725+0000 7f6888d91b80 20 client.16563 trim_cache size 0 max 16384 2024-05-09T04:56:01.729 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:56:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:56:01.729 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:56:01.729 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:56:01.729 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.725+0000 7f687a37b700 1 -- 172.21.15.46:0/1203866853 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f685c0b2a70 msgr2=0x7f685c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.725+0000 7f687a37b700 1 --2- 172.21.15.46:0/1203866853 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f685c0b2a70 0x7f685c0b4f80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f6864005950 tx=0x7f686400b410 comp rx=0 tx=0).stop 2024-05-09T04:56:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.725+0000 7f687a37b700 1 -- 172.21.15.46:0/1203866853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6874108740 msgr2=0x7f687419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.725+0000 7f687a37b700 1 --2- 172.21.15.46:0/1203866853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6874108740 0x7f687419c6f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f686c009850 tx=0x7f686c007a20 comp rx=0 tx=0).stop 2024-05-09T04:56:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.725+0000 7f687a37b700 1 -- 172.21.15.46:0/1203866853 shutdown_connections 2024-05-09T04:56:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.725+0000 7f687a37b700 1 --2- 172.21.15.46:0/1203866853 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f685c0b2a70 0x7f685c0b4f80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.725+0000 7f687a37b700 1 --2- 172.21.15.46:0/1203866853 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6874108740 0x7f687419c6f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.725+0000 7f687a37b700 1 --2- 172.21.15.46:0/1203866853 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f68741034c0 0x7f687419bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.725+0000 7f687a37b700 1 --2- 172.21.15.46:0/1203866853 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6874107df0 0x7f687419c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.725+0000 7f687a37b700 1 -- 172.21.15.46:0/1203866853 >> 172.21.15.46:0/1203866853 conn(0x7f68740fedc0 msgr2=0x7f6874107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.726+0000 7f687a37b700 1 -- 172.21.15.46:0/1203866853 shutdown_connections 2024-05-09T04:56:01.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:01.726+0000 7f687a37b700 1 -- 172.21.15.46:0/1203866853 wait complete. 2024-05-09T04:56:02.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:01 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:56:02.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:01 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:56:02.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:01 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:56:02.510 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:56:02.510 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:56:02.766 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:03.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:02 smithi046 ceph-mon[30136]: pgmap v151: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T04:56:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:02 smithi072 ceph-mon[34088]: pgmap v151: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T04:56:03.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:02 smithi110 ceph-mon[34904]: pgmap v151: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T04:56:04.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.399+0000 7f6c4cc75700 1 -- 172.21.15.46:0/3343366490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c48108770 msgr2=0x7f6c481100d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:04.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.399+0000 7f6c4cc75700 1 --2- 172.21.15.46:0/3343366490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c48108770 0x7f6c481100d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6c4000b0d0 tx=0x7f6c4000b3e0 comp rx=0 tx=0).stop 2024-05-09T04:56:04.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.399+0000 7f6c4cc75700 1 -- 172.21.15.46:0/3343366490 shutdown_connections 2024-05-09T04:56:04.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.399+0000 7f6c4cc75700 1 --2- 172.21.15.46:0/3343366490 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c48108770 0x7f6c481100d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:04.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.399+0000 7f6c4cc75700 1 --2- 172.21.15.46:0/3343366490 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c48103540 0x7f6c48103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:04.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.399+0000 7f6c4cc75700 1 --2- 172.21.15.46:0/3343366490 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c48107e20 0x7f6c48108230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:04.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.399+0000 7f6c4cc75700 1 -- 172.21.15.46:0/3343366490 >> 172.21.15.46:0/3343366490 conn(0x7f6c48078e50 msgr2=0x7f6c48079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:04.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.399+0000 7f6c4cc75700 1 -- 172.21.15.46:0/3343366490 shutdown_connections 2024-05-09T04:56:04.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.399+0000 7f6c4cc75700 1 -- 172.21.15.46:0/3343366490 wait complete. 2024-05-09T04:56:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.400+0000 7f6c4cc75700 1 Processor -- start 2024-05-09T04:56:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.400+0000 7f6c4cc75700 1 -- start start 2024-05-09T04:56:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.400+0000 7f6c4cc75700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c48103540 0x7f6c4819aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.400+0000 7f6c4cc75700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c48107e20 0x7f6c4819b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.400+0000 7f6c4cc75700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c48108770 0x7f6c4819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.400+0000 7f6c4cc75700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f6c481133a0 con 0x7f6c48107e20 2024-05-09T04:56:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.400+0000 7f6c4cc75700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f6c48113220 con 0x7f6c48108770 2024-05-09T04:56:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.400+0000 7f6c4cc75700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f6c48113520 con 0x7f6c48103540 2024-05-09T04:56:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.400+0000 7f6c45d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c48107e20 0x7f6c4819b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.400+0000 7f6c45d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c48107e20 0x7f6c4819b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36414/0 (socket says 172.21.15.46:36414) 2024-05-09T04:56:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.400+0000 7f6c45d9b700 1 -- 172.21.15.46:0/3346740543 learned_addr learned my addr 172.21.15.46:0/3346740543 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.401+0000 7f6c4659c700 1 --2- 172.21.15.46:0/3346740543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c48103540 0x7f6c4819aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.401+0000 7f6c46d9d700 1 --2- 172.21.15.46:0/3346740543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c48108770 0x7f6c4819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:04.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.401+0000 7f6c45d9b700 1 -- 172.21.15.46:0/3346740543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c48103540 msgr2=0x7f6c4819aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.401+0000 7f6c45d9b700 1 --2- 172.21.15.46:0/3346740543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c48103540 0x7f6c4819aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.401+0000 7f6c45d9b700 1 -- 172.21.15.46:0/3346740543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c48108770 msgr2=0x7f6c4819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.401+0000 7f6c45d9b700 1 --2- 172.21.15.46:0/3346740543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c48108770 0x7f6c4819f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.401+0000 7f6c45d9b700 1 -- 172.21.15.46:0/3346740543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4819fcc0 con 0x7f6c48107e20 2024-05-09T04:56:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.401+0000 7f6c45d9b700 1 --2- 172.21.15.46:0/3346740543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c48107e20 0x7f6c4819b440 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00d950 tx=0x7f6c3c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.401+0000 7f6c377fe700 1 -- 172.21.15.46:0/3346740543 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c016070 con 0x7f6c48107e20 2024-05-09T04:56:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.401+0000 7f6c377fe700 1 -- 172.21.15.46:0/3346740543 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c3c00de30 con 0x7f6c48107e20 2024-05-09T04:56:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.401+0000 7f6c4cc75700 1 -- 172.21.15.46:0/3346740543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4819ffb0 con 0x7f6c48107e20 2024-05-09T04:56:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.401+0000 7f6c377fe700 1 -- 172.21.15.46:0/3346740543 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c005350 con 0x7f6c48107e20 2024-05-09T04:56:04.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.401+0000 7f6c4cc75700 1 -- 172.21.15.46:0/3346740543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c481a04f0 con 0x7f6c48107e20 2024-05-09T04:56:04.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.402+0000 7f6c4cc75700 1 -- 172.21.15.46:0/3346740543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c480512c0 con 0x7f6c48107e20 2024-05-09T04:56:04.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.404+0000 7f6c377fe700 1 -- 172.21.15.46:0/3346740543 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6c3c0054f0 con 0x7f6c48107e20 2024-05-09T04:56:04.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.405+0000 7f6c377fe700 1 --2- 172.21.15.46:0/3346740543 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f6c300b29f0 0x7f6c300b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:04.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.405+0000 7f6c4659c700 1 --2- 172.21.15.46:0/3346740543 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f6c300b29f0 0x7f6c300b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:04.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.405+0000 7f6c377fe700 1 -- 172.21.15.46:0/3346740543 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f6c3c0e2270 con 0x7f6c48107e20 2024-05-09T04:56:04.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.405+0000 7f6c4659c700 1 --2- 172.21.15.46:0/3346740543 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f6c300b29f0 0x7f6c300b4f00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6c38005ea0 tx=0x7f6c38005e30 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:04.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.408+0000 7f6c377fe700 1 -- 172.21.15.46:0/3346740543 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6c3c01f070 con 0x7f6c48107e20 2024-05-09T04:56:04.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.675+0000 7f6c4cc75700 1 -- 172.21.15.46:0/3346740543 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6c4802e5f0 con 0x7f6c48107e20 2024-05-09T04:56:04.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.676+0000 7f6c377fe700 1 -- 172.21.15.46:0/3346740543 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7f6c3c0a9bd0 con 0x7f6c48107e20 2024-05-09T04:56:04.676 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:56:04.677 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:56:04.678 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.678+0000 7f6c4cc75700 1 -- 172.21.15.46:0/3346740543 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f6c300b29f0 msgr2=0x7f6c300b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.678+0000 7f6c4cc75700 1 --2- 172.21.15.46:0/3346740543 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f6c300b29f0 0x7f6c300b4f00 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f6c38005ea0 tx=0x7f6c38005e30 comp rx=0 tx=0).stop 2024-05-09T04:56:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.678+0000 7f6c4cc75700 1 -- 172.21.15.46:0/3346740543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c48107e20 msgr2=0x7f6c4819b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.678+0000 7f6c4cc75700 1 --2- 172.21.15.46:0/3346740543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c48107e20 0x7f6c4819b440 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00d950 tx=0x7f6c3c00dd10 comp rx=0 tx=0).stop 2024-05-09T04:56:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.678+0000 7f6c4cc75700 1 -- 172.21.15.46:0/3346740543 shutdown_connections 2024-05-09T04:56:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.678+0000 7f6c4cc75700 1 --2- 172.21.15.46:0/3346740543 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f6c300b29f0 0x7f6c300b4f00 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.678+0000 7f6c4cc75700 1 --2- 172.21.15.46:0/3346740543 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f6c48108770 0x7f6c4819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.678+0000 7f6c4cc75700 1 --2- 172.21.15.46:0/3346740543 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f6c48103540 0x7f6c4819aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.678+0000 7f6c4cc75700 1 --2- 172.21.15.46:0/3346740543 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f6c48107e20 0x7f6c4819b440 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.678+0000 7f6c4cc75700 1 -- 172.21.15.46:0/3346740543 >> 172.21.15.46:0/3346740543 conn(0x7f6c48078e50 msgr2=0x7f6c48075d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.679+0000 7f6c4cc75700 1 -- 172.21.15.46:0/3346740543 shutdown_connections 2024-05-09T04:56:04.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:04.679+0000 7f6c4cc75700 1 -- 172.21.15.46:0/3346740543 wait complete. 2024-05-09T04:56:04.680 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:56:05.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:04 smithi046 ceph-mon[30136]: pgmap v152: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-05-09T04:56:05.045 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:04 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3346740543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:56:05.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:04 smithi072 ceph-mon[34088]: pgmap v152: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-05-09T04:56:05.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:04 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3346740543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:56:05.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:04 smithi110 ceph-mon[34904]: pgmap v152: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-05-09T04:56:05.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:04 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3346740543' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:56:05.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:56:05.755 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:07.073 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:06 smithi046 ceph-mon[30136]: pgmap v153: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-09T04:56:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:06 smithi072 ceph-mon[34088]: pgmap v153: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-09T04:56:07.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:06 smithi110 ceph-mon[34904]: pgmap v153: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-09T04:56:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.473+0000 7f03cffa2700 1 -- 172.21.15.46:0/3189212978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c8109860 msgr2=0x7f03c81111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.473+0000 7f03cffa2700 1 --2- 172.21.15.46:0/3189212978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c8109860 0x7f03c81111c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f03c000b0d0 tx=0x7f03c000b3e0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.473+0000 7f03cffa2700 1 -- 172.21.15.46:0/3189212978 shutdown_connections 2024-05-09T04:56:07.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.473+0000 7f03cffa2700 1 --2- 172.21.15.46:0/3189212978 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c8109860 0x7f03c81111c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.473+0000 7f03cffa2700 1 --2- 172.21.15.46:0/3189212978 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03c81045e0 0x7f03c81049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.473+0000 7f03cffa2700 1 --2- 172.21.15.46:0/3189212978 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03c8108f10 0x7f03c8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.473+0000 7f03cffa2700 1 -- 172.21.15.46:0/3189212978 >> 172.21.15.46:0/3189212978 conn(0x7f03c80ffed0 msgr2=0x7f03c8102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:07.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.473+0000 7f03cffa2700 1 -- 172.21.15.46:0/3189212978 shutdown_connections 2024-05-09T04:56:07.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.473+0000 7f03cffa2700 1 -- 172.21.15.46:0/3189212978 wait complete. 2024-05-09T04:56:07.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.473+0000 7f03cffa2700 1 Processor -- start 2024-05-09T04:56:07.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03cffa2700 1 -- start start 2024-05-09T04:56:07.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03cffa2700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03c81045e0 0x7f03c819ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03cffa2700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03c8108f10 0x7f03c819b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03cffa2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c8109860 0x7f03c819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03cffa2700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03c8114490 con 0x7f03c8109860 2024-05-09T04:56:07.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03cffa2700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f03c8114310 con 0x7f03c8108f10 2024-05-09T04:56:07.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03cffa2700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03c8114610 con 0x7f03c81045e0 2024-05-09T04:56:07.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03ce53f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c8109860 0x7f03c819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03ce53f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c8109860 0x7f03c819f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36420/0 (socket says 172.21.15.46:36420) 2024-05-09T04:56:07.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03ce53f700 1 -- 172.21.15.46:0/599278447 learned_addr learned my addr 172.21.15.46:0/599278447 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:07.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03cd53d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03c8108f10 0x7f03c819b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03cdd3e700 1 --2- 172.21.15.46:0/599278447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03c81045e0 0x7f03c819ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03ce53f700 1 -- 172.21.15.46:0/599278447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03c81045e0 msgr2=0x7f03c819ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:07.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03ce53f700 1 --2- 172.21.15.46:0/599278447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03c81045e0 0x7f03c819ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03ce53f700 1 -- 172.21.15.46:0/599278447 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03c8108f10 msgr2=0x7f03c819b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:07.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03ce53f700 1 --2- 172.21.15.46:0/599278447 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03c8108f10 0x7f03c819b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.474+0000 7f03ce53f700 1 -- 172.21.15.46:0/599278447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c819fc50 con 0x7f03c8109860 2024-05-09T04:56:07.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.475+0000 7f03ce53f700 1 --2- 172.21.15.46:0/599278447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c8109860 0x7f03c819f5b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009850 tx=0x7f03c0007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:07.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.475+0000 7f03baffd700 1 -- 172.21.15.46:0/599278447 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c001f070 con 0x7f03c8109860 2024-05-09T04:56:07.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.475+0000 7f03baffd700 1 -- 172.21.15.46:0/599278447 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c0004c50 con 0x7f03c8109860 2024-05-09T04:56:07.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.475+0000 7f03cffa2700 1 -- 172.21.15.46:0/599278447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c819fee0 con 0x7f03c8109860 2024-05-09T04:56:07.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.475+0000 7f03baffd700 1 -- 172.21.15.46:0/599278447 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c001b7b0 con 0x7f03c8109860 2024-05-09T04:56:07.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.475+0000 7f03cffa2700 1 -- 172.21.15.46:0/599278447 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c81a7860 con 0x7f03c8109860 2024-05-09T04:56:07.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.478+0000 7f03baffd700 1 -- 172.21.15.46:0/599278447 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f03c0022070 con 0x7f03c8109860 2024-05-09T04:56:07.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.479+0000 7f03baffd700 1 --2- 172.21.15.46:0/599278447 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f03b40b2a70 0x7f03b40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.479+0000 7f03cdd3e700 1 --2- 172.21.15.46:0/599278447 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f03b40b2a70 0x7f03b40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.479+0000 7f03de9b8b80 1 Processor -- start 2024-05-09T04:56:07.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.479+0000 7f03baffd700 1 -- 172.21.15.46:0/599278447 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f03c000f020 con 0x7f03c8109860 2024-05-09T04:56:07.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03de9b8b80 1 -- start start 2024-05-09T04:56:07.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03de9b8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569af1160 0x55e569b88d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03de9b8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b892f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03de9b8b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 0x55e569b8d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03cdd3e700 1 --2- 172.21.15.46:0/599278447 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f03b40b2a70 0x7f03b40b4f80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009bf0 tx=0x7f03c4009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:07.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03de9b8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e569aa6ff0 con 0x55e569a074a0 2024-05-09T04:56:07.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03de9b8b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e569a457f0 con 0x55e569b8d400 2024-05-09T04:56:07.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03de9b8b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e569aa2fb0 con 0x55e569af1160 2024-05-09T04:56:07.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03cd53d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b892f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03cd53d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b892f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36424/0 (socket says 172.21.15.46:36424) 2024-05-09T04:56:07.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03cd53d700 1 -- 172.21.15.46:0/1439371769 learned_addr learned my addr 172.21.15.46:0/1439371769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:07.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03cdd3e700 1 --2- 172.21.15.46:0/1439371769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569af1160 0x55e569b88d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03ce53f700 1 --2- 172.21.15.46:0/1439371769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 0x55e569b8d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03cd53d700 1 -- 172.21.15.46:0/1439371769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569af1160 msgr2=0x55e569b88d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:07.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03cd53d700 1 --2- 172.21.15.46:0/1439371769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569af1160 0x55e569b88d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03cd53d700 1 -- 172.21.15.46:0/1439371769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 msgr2=0x55e569b8d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:07.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03cd53d700 1 --2- 172.21.15.46:0/1439371769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 0x55e569b8d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.480+0000 7f03cd53d700 1 -- 172.21.15.46:0/1439371769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e569a3d370 con 0x55e569a074a0 2024-05-09T04:56:07.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.481+0000 7f03ce53f700 1 --2- 172.21.15.46:0/1439371769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 0x55e569b8d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:56:07.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.481+0000 7f03cd53d700 1 --2- 172.21.15.46:0/1439371769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b892f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00bc70 tx=0x7f03bc00bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d747ea3707a74636 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:07.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.481+0000 7f03b3fff700 1 -- 172.21.15.46:0/1439371769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc010be0 con 0x55e569a074a0 2024-05-09T04:56:07.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.481+0000 7f03b3fff700 1 -- 172.21.15.46:0/1439371769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03bc0044e0 con 0x55e569a074a0 2024-05-09T04:56:07.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.481+0000 7f03b3fff700 1 -- 172.21.15.46:0/1439371769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc005220 con 0x55e569a074a0 2024-05-09T04:56:07.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.482+0000 7f03de9b8b80 1 -- 172.21.15.46:0/1439371769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 msgr2=0x55e569b892f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.482+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/1439371769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b892f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00bc70 tx=0x7f03bc00bf80 comp rx=0 tx=0).stop 2024-05-09T04:56:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.482+0000 7f03de9b8b80 1 -- 172.21.15.46:0/1439371769 shutdown_connections 2024-05-09T04:56:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.482+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/1439371769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 0x55e569b8d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.482+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/1439371769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569af1160 0x55e569b88d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.482+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/1439371769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b892f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.482+0000 7f03de9b8b80 1 -- 172.21.15.46:0/1439371769 >> 172.21.15.46:0/1439371769 conn(0x55e569a676a0 msgr2=0x55e569b7f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.482+0000 7f03de9b8b80 1 -- 172.21.15.46:0/1439371769 shutdown_connections 2024-05-09T04:56:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.482+0000 7f03de9b8b80 1 -- 172.21.15.46:0/1439371769 wait complete. 2024-05-09T04:56:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.482+0000 7f03de9b8b80 1 -- start start 2024-05-09T04:56:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.483+0000 7f03de9b8b80 1 Processor -- start 2024-05-09T04:56:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.483+0000 7f03de9b8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b925b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.483+0000 7f03de9b8b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569af1160 0x55e569b92af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.483+0000 7f03de9b8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569b8d400 0x55e569b93030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.483+0000 7f03de9b8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e569922ba0 con 0x55e569a074a0 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.483+0000 7f03de9b8b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e569a61150 con 0x55e569af1160 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.483+0000 7f03de9b8b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e569b8e1b0 con 0x55e569b8d400 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.483+0000 7f03de9b8b80 10 client.0 _ll_register_callbacks cb 0x55e5699186c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.483+0000 7f03cdd3e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b925b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.483+0000 7f03cdd3e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b925b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36428/0 (socket says 172.21.15.46:36428) 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.483+0000 7f03cdd3e700 1 -- 172.21.15.46:0/1637390345 learned_addr learned my addr 172.21.15.46:0/1637390345 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.483+0000 7f03ce53f700 1 --2- 172.21.15.46:0/1637390345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569b8d400 0x55e569b93030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.483+0000 7f03cd53d700 1 --2- 172.21.15.46:0/1637390345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569af1160 0x55e569b92af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.484+0000 7f03cdd3e700 1 -- 172.21.15.46:0/1637390345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569b8d400 msgr2=0x55e569b93030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.484+0000 7f03cdd3e700 1 --2- 172.21.15.46:0/1637390345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569b8d400 0x55e569b93030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.484+0000 7f03cdd3e700 1 -- 172.21.15.46:0/1637390345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569af1160 msgr2=0x55e569b92af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.484+0000 7f03cdd3e700 1 --2- 172.21.15.46:0/1637390345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569af1160 0x55e569b92af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.484+0000 7f03cdd3e700 1 -- 172.21.15.46:0/1637390345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e569922490 con 0x55e569a074a0 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.484+0000 7f03cdd3e700 1 --2- 172.21.15.46:0/1637390345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b925b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f03c4012910 tx=0x7f03c40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36d69233a7001f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.484+0000 7f03b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.484+0000 7f03b17fa700 1 -- 172.21.15.46:0/1637390345 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4029670 con 0x55e569a074a0 2024-05-09T04:56:07.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.484+0000 7f03b17fa700 1 -- 172.21.15.46:0/1637390345 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f03c4029810 con 0x55e569a074a0 2024-05-09T04:56:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.484+0000 7f03de9b8b80 1 -- 172.21.15.46:0/1637390345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03bc0044e0 con 0x55e569a074a0 2024-05-09T04:56:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.484+0000 7f03b17fa700 1 -- 172.21.15.46:0/1637390345 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f03c4029a10 con 0x55e569a074a0 2024-05-09T04:56:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.484+0000 7f03de9b8b80 10 client.16581 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.484+0000 7f03de9b8b80 1 -- 172.21.15.46:0/1637390345 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e569b9ff00 con 0x55e569a074a0 2024-05-09T04:56:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.485+0000 7f03b17fa700 1 -- 172.21.15.46:0/1637390345 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03bc0044e0 con 0x55e569a074a0 2024-05-09T04:56:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.485+0000 7f03b17fa700 1 -- 172.21.15.46:0/1637390345 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f03c401f5f0 con 0x55e569a074a0 2024-05-09T04:56:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.485+0000 7f03de9b8b80 10 client.16581 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.485+0000 7f03de9b8b80 10 client.16581 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:56:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.485+0000 7f03de9b8b80 20 client.16581 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.485+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/1637390345 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55e569ba2590 0x55e569ba49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.485+0000 7f03de9b8b80 4 client.16581 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:56:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.485+0000 7f03de9b8b80 1 -- 172.21.15.46:0/1637390345 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e569ab49d0 con 0x55e569ba2590 2024-05-09T04:56:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.485+0000 7f03cd53d700 1 --2- 172.21.15.46:0/1637390345 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55e569ba2590 0x55e569ba49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.486+0000 7f03cd53d700 1 --2- 172.21.15.46:0/1637390345 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55e569ba2590 0x55e569ba49d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ea31596bf309df94 server_cookie=303ae917164eda6b in_seq=0 out_seq=0 2024-05-09T04:56:07.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.486+0000 7f03b17fa700 10 client.16581 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:56:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.486+0000 7f03b17fa700 1 -- 172.21.15.46:0/1637390345 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e569ab49d0 con 0x55e569ba2590 2024-05-09T04:56:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.486+0000 7f03b17fa700 10 client.16581 handle_command_reply: tid=0 2024-05-09T04:56:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.486+0000 7f03de9b8b80 1 client.16581 shutdown 2024-05-09T04:56:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.486+0000 7f03de9b8b80 1 -- 172.21.15.46:0/1637390345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 msgr2=0x55e569b925b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.486+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/1637390345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b925b0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f03c4012910 tx=0x7f03c40047c0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.487+0000 7f03de9b8b80 1 -- 172.21.15.46:0/1637390345 shutdown_connections 2024-05-09T04:56:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.487+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/1637390345 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55e569ba2590 0x55e569ba49d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.487+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/1637390345 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569b8d400 0x55e569b93030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.487+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/1637390345 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b925b0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.487+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/1637390345 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569af1160 0x55e569b92af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.487+0000 7f03b17fa700 0 client.16581 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:56:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.487+0000 7f03de9b8b80 1 -- 172.21.15.46:0/1637390345 >> 172.21.15.46:0/1637390345 conn(0x55e569a676a0 msgr2=0x55e569b7f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:07.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.487+0000 7f03de9b8b80 1 -- 172.21.15.46:0/1637390345 shutdown_connections 2024-05-09T04:56:07.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.487+0000 7f03de9b8b80 1 -- 172.21.15.46:0/1637390345 wait complete. 2024-05-09T04:56:07.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.487+0000 7f03de9b8b80 20 client.16581 trim_cache size 0 max 16384 2024-05-09T04:56:07.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.502+0000 7f03de9b8b80 1 Processor -- start 2024-05-09T04:56:07.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.502+0000 7f03de9b8b80 1 -- start start 2024-05-09T04:56:07.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.502+0000 7f03de9b8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b8cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.502+0000 7f03de9b8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569af1160 0x55e569b91510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.502+0000 7f03de9b8b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 0x55e569b92e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.502+0000 7f03de9b8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03c81af510 con 0x55e569a074a0 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.502+0000 7f03de9b8b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f03c81acda0 con 0x55e569b8d400 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.502+0000 7f03de9b8b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03c81aabf0 con 0x55e569af1160 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.502+0000 7f03cdd3e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b8cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.503+0000 7f03cdd3e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b8cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36434/0 (socket says 172.21.15.46:36434) 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.503+0000 7f03cdd3e700 1 -- 172.21.15.46:0/4135936796 learned_addr learned my addr 172.21.15.46:0/4135936796 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.503+0000 7f03cd53d700 1 --2- 172.21.15.46:0/4135936796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569af1160 0x55e569b91510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.503+0000 7f03ce53f700 1 --2- 172.21.15.46:0/4135936796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 0x55e569b92e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.503+0000 7f03cdd3e700 1 -- 172.21.15.46:0/4135936796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569af1160 msgr2=0x55e569b91510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.503+0000 7f03cdd3e700 1 --2- 172.21.15.46:0/4135936796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569af1160 0x55e569b91510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.503+0000 7f03cdd3e700 1 -- 172.21.15.46:0/4135936796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 msgr2=0x55e569b92e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.503+0000 7f03cdd3e700 1 --2- 172.21.15.46:0/4135936796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 0x55e569b92e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.503+0000 7f03cdd3e700 1 -- 172.21.15.46:0/4135936796 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e569b90b20 con 0x55e569a074a0 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.503+0000 7f03cdd3e700 1 --2- 172.21.15.46:0/4135936796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b8cc60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f03c401fa10 tx=0x7f03c4009a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b80bedc94ef940 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:07.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.503+0000 7f03b37fe700 1 -- 172.21.15.46:0/4135936796 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4036040 con 0x55e569a074a0 2024-05-09T04:56:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.503+0000 7f03b37fe700 1 -- 172.21.15.46:0/4135936796 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c4024070 con 0x55e569a074a0 2024-05-09T04:56:07.505 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.503+0000 7f03b37fe700 1 -- 172.21.15.46:0/4135936796 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4004020 con 0x55e569a074a0 2024-05-09T04:56:07.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.504+0000 7f03de9b8b80 1 -- 172.21.15.46:0/4135936796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 msgr2=0x55e569b8cc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:07.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.504+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/4135936796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b8cc60 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f03c401fa10 tx=0x7f03c4009a30 comp rx=0 tx=0).stop 2024-05-09T04:56:07.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.504+0000 7f03de9b8b80 1 -- 172.21.15.46:0/4135936796 shutdown_connections 2024-05-09T04:56:07.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.504+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/4135936796 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 0x55e569b92e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.504+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/4135936796 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b8cc60 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.504+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/4135936796 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569af1160 0x55e569b91510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.504+0000 7f03de9b8b80 1 -- 172.21.15.46:0/4135936796 >> 172.21.15.46:0/4135936796 conn(0x55e569a676a0 msgr2=0x55e569b93c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.504+0000 7f03de9b8b80 1 -- 172.21.15.46:0/4135936796 shutdown_connections 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.504+0000 7f03de9b8b80 1 -- 172.21.15.46:0/4135936796 wait complete. 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.505+0000 7f03de9b8b80 1 -- start start 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.505+0000 7f03de9b8b80 1 Processor -- start 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.505+0000 7f03de9b8b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b9ad20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.505+0000 7f03de9b8b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569af1160 0x55e569b8f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.505+0000 7f03de9b8b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 0x55e569b8f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.505+0000 7f03de9b8b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f03c81ab260 con 0x55e569a074a0 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.505+0000 7f03de9b8b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f03c81ac6d0 con 0x55e569b8d400 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.505+0000 7f03de9b8b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f03c81ab930 con 0x55e569af1160 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.505+0000 7f03de9b8b80 10 client.0 _ll_register_callbacks cb 0x55e5699186c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.505+0000 7f03cdd3e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b9ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.505+0000 7f03cdd3e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b9ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36448/0 (socket says 172.21.15.46:36448) 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.505+0000 7f03cdd3e700 1 -- 172.21.15.46:0/4264459242 learned_addr learned my addr 172.21.15.46:0/4264459242 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.506+0000 7f03ce53f700 1 --2- 172.21.15.46:0/4264459242 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 0x55e569b8f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.506+0000 7f03cd53d700 1 --2- 172.21.15.46:0/4264459242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569af1160 0x55e569b8f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.506+0000 7f03cdd3e700 1 -- 172.21.15.46:0/4264459242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569af1160 msgr2=0x55e569b8f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.506+0000 7f03cdd3e700 1 --2- 172.21.15.46:0/4264459242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569af1160 0x55e569b8f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.506+0000 7f03cdd3e700 1 -- 172.21.15.46:0/4264459242 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 msgr2=0x55e569b8f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.506+0000 7f03cdd3e700 1 --2- 172.21.15.46:0/4264459242 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 0x55e569b8f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.506+0000 7f03cdd3e700 1 -- 172.21.15.46:0/4264459242 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e569b8fed0 con 0x55e569a074a0 2024-05-09T04:56:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.506+0000 7f03cdd3e700 1 --2- 172.21.15.46:0/4264459242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b9ad20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f03c40057d0 tx=0x7f03c4011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dda599becf8ea20e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.506+0000 7f03b17fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:56:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.506+0000 7f03b17fa700 1 -- 172.21.15.46:0/4264459242 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03c4036040 con 0x55e569a074a0 2024-05-09T04:56:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.506+0000 7f03b17fa700 1 -- 172.21.15.46:0/4264459242 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f03c4029db0 con 0x55e569a074a0 2024-05-09T04:56:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.506+0000 7f03de9b8b80 1 -- 172.21.15.46:0/4264459242 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03c4024070 con 0x55e569a074a0 2024-05-09T04:56:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.506+0000 7f03b17fa700 1 -- 172.21.15.46:0/4264459242 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f03c404c050 con 0x55e569a074a0 2024-05-09T04:56:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.506+0000 7f03de9b8b80 10 client.16587 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.506+0000 7f03de9b8b80 1 -- 172.21.15.46:0/4264459242 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e569c0ccd0 con 0x55e569a074a0 2024-05-09T04:56:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.507+0000 7f03b17fa700 1 -- 172.21.15.46:0/4264459242 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03c4024070 con 0x55e569a074a0 2024-05-09T04:56:07.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.507+0000 7f03b17fa700 1 -- 172.21.15.46:0/4264459242 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f03c403ba00 con 0x55e569a074a0 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.507+0000 7f03de9b8b80 10 client.16587 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.507+0000 7f03de9b8b80 10 client.16587 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.507+0000 7f03de9b8b80 20 client.16587 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.507+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/4264459242 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55e569c0d210 0x55e569c0f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.507+0000 7f03de9b8b80 4 client.16587 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.507+0000 7f03de9b8b80 1 -- 172.21.15.46:0/4264459242 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e569aaea70 con 0x55e569c0d210 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.507+0000 7f03cd53d700 1 --2- 172.21.15.46:0/4264459242 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55e569c0d210 0x55e569c0f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03cd53d700 1 --2- 172.21.15.46:0/4264459242 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55e569c0d210 0x55e569c0f630 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9d79bb4950092cd2 server_cookie=bb25197e94c828de in_seq=0 out_seq=0 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03b17fa700 10 client.16587 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03b17fa700 1 -- 172.21.15.46:0/4264459242 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e569aaea70 con 0x55e569c0d210 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03b17fa700 10 client.16587 handle_command_reply: tid=0 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03de9b8b80 1 client.16587 shutdown 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03de9b8b80 1 -- 172.21.15.46:0/4264459242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 msgr2=0x55e569b9ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/4264459242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b9ad20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f03c40057d0 tx=0x7f03c4011c70 comp rx=0 tx=0).stop 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03de9b8b80 1 -- 172.21.15.46:0/4264459242 shutdown_connections 2024-05-09T04:56:07.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/4264459242 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e569b8d400 0x55e569b8f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/4264459242 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e569a074a0 0x55e569b9ad20 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/4264459242 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55e569c0d210 0x55e569c0f630 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03de9b8b80 1 --2- 172.21.15.46:0/4264459242 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e569af1160 0x55e569b8f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03b17fa700 0 client.16587 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:56:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03de9b8b80 1 -- 172.21.15.46:0/4264459242 >> 172.21.15.46:0/4264459242 conn(0x55e569a676a0 msgr2=0x55e569b8c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03de9b8b80 1 -- 172.21.15.46:0/4264459242 shutdown_connections 2024-05-09T04:56:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03de9b8b80 1 -- 172.21.15.46:0/4264459242 wait complete. 2024-05-09T04:56:07.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.508+0000 7f03de9b8b80 20 client.16587 trim_cache size 0 max 16384 2024-05-09T04:56:07.512 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:56:07.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.509+0000 7f03cffa2700 1 -- 172.21.15.46:0/599278447 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f03b40b2a70 msgr2=0x7f03b40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:07.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.509+0000 7f03cffa2700 1 --2- 172.21.15.46:0/599278447 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f03b40b2a70 0x7f03b40b4f80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f03c4009bf0 tx=0x7f03c4009260 comp rx=0 tx=0).stop 2024-05-09T04:56:07.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.509+0000 7f03cffa2700 1 -- 172.21.15.46:0/599278447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c8109860 msgr2=0x7f03c819f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:07.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.509+0000 7f03cffa2700 1 --2- 172.21.15.46:0/599278447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c8109860 0x7f03c819f5b0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f03c0009850 tx=0x7f03c0007a20 comp rx=0 tx=0).stop 2024-05-09T04:56:07.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.509+0000 7f03cffa2700 1 -- 172.21.15.46:0/599278447 shutdown_connections 2024-05-09T04:56:07.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.509+0000 7f03cffa2700 1 --2- 172.21.15.46:0/599278447 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f03b40b2a70 0x7f03b40b4f80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.509+0000 7f03cffa2700 1 --2- 172.21.15.46:0/599278447 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f03c8109860 0x7f03c819f5b0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.509+0000 7f03cffa2700 1 --2- 172.21.15.46:0/599278447 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f03c81045e0 0x7f03c819ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.509+0000 7f03cffa2700 1 --2- 172.21.15.46:0/599278447 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f03c8108f10 0x7f03c819b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:07.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.509+0000 7f03cffa2700 1 -- 172.21.15.46:0/599278447 >> 172.21.15.46:0/599278447 conn(0x7f03c80ffed0 msgr2=0x7f03c8108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:07.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.509+0000 7f03cffa2700 1 -- 172.21.15.46:0/599278447 shutdown_connections 2024-05-09T04:56:07.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:07.509+0000 7f03cffa2700 1 -- 172.21.15.46:0/599278447 wait complete. 2024-05-09T04:56:08.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:07 smithi046 ceph-mon[30136]: pgmap v154: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T04:56:08.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:07 smithi072 ceph-mon[34088]: pgmap v154: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T04:56:08.303 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:56:08.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:07 smithi110 ceph-mon[34904]: pgmap v154: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T04:56:08.556 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:10.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.294+0000 7f8f4e598700 1 -- 172.21.15.46:0/2617442180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f48108780 msgr2=0x7f8f481100e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.294+0000 7f8f4e598700 1 --2- 172.21.15.46:0/2617442180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f48108780 0x7f8f481100e0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f8f4000b0d0 tx=0x7f8f4000b3e0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.294+0000 7f8f4e598700 1 -- 172.21.15.46:0/2617442180 shutdown_connections 2024-05-09T04:56:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.294+0000 7f8f4e598700 1 --2- 172.21.15.46:0/2617442180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f48108780 0x7f8f481100e0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.294+0000 7f8f4e598700 1 --2- 172.21.15.46:0/2617442180 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f48103550 0x7f8f48103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.294+0000 7f8f4e598700 1 --2- 172.21.15.46:0/2617442180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f48107e30 0x7f8f48108240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.294+0000 7f8f4e598700 1 -- 172.21.15.46:0/2617442180 >> 172.21.15.46:0/2617442180 conn(0x7f8f48078e00 msgr2=0x7f8f48079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.294+0000 7f8f4e598700 1 -- 172.21.15.46:0/2617442180 shutdown_connections 2024-05-09T04:56:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.294+0000 7f8f4e598700 1 -- 172.21.15.46:0/2617442180 wait complete. 2024-05-09T04:56:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.295+0000 7f8f4e598700 1 Processor -- start 2024-05-09T04:56:10.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.295+0000 7f8f4e598700 1 -- start start 2024-05-09T04:56:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f4e598700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f48103550 0x7f8f4819bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f4e598700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f48107e30 0x7f8f4819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f4e598700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f48108780 0x7f8f4819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f4e598700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481133b0 con 0x7f8f48107e30 2024-05-09T04:56:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f4e598700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48113230 con 0x7f8f48103550 2024-05-09T04:56:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f4e598700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8f48113530 con 0x7f8f48108780 2024-05-09T04:56:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f48107e30 0x7f8f4819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f477fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f48107e30 0x7f8f4819c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36464/0 (socket says 172.21.15.46:36464) 2024-05-09T04:56:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f477fe700 1 -- 172.21.15.46:0/3079731248 learned_addr learned my addr 172.21.15.46:0/3079731248 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f47fff700 1 --2- 172.21.15.46:0/3079731248 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f48103550 0x7f8f4819bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f4cb35700 1 --2- 172.21.15.46:0/3079731248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f48108780 0x7f8f4819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f477fe700 1 -- 172.21.15.46:0/3079731248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f48108780 msgr2=0x7f8f4819c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f477fe700 1 --2- 172.21.15.46:0/3079731248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f48108780 0x7f8f4819c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f477fe700 1 -- 172.21.15.46:0/3079731248 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f48103550 msgr2=0x7f8f4819bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f477fe700 1 --2- 172.21.15.46:0/3079731248 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f48103550 0x7f8f4819bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.296+0000 7f8f477fe700 1 -- 172.21.15.46:0/3079731248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f481a6f60 con 0x7f8f48107e30 2024-05-09T04:56:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.297+0000 7f8f477fe700 1 --2- 172.21.15.46:0/3079731248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f48107e30 0x7f8f4819c150 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00d980 tx=0x7f8f3c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:10.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.297+0000 7f8f457fa700 1 -- 172.21.15.46:0/3079731248 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c017070 con 0x7f8f48107e30 2024-05-09T04:56:10.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.297+0000 7f8f457fa700 1 -- 172.21.15.46:0/3079731248 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f3c00fd40 con 0x7f8f48107e30 2024-05-09T04:56:10.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.297+0000 7f8f4e598700 1 -- 172.21.15.46:0/3079731248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f481a7250 con 0x7f8f48107e30 2024-05-09T04:56:10.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.297+0000 7f8f457fa700 1 -- 172.21.15.46:0/3079731248 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3c00e3b0 con 0x7f8f48107e30 2024-05-09T04:56:10.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.297+0000 7f8f4e598700 1 -- 172.21.15.46:0/3079731248 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f481a7790 con 0x7f8f48107e30 2024-05-09T04:56:10.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.300+0000 7f8f457fa700 1 -- 172.21.15.46:0/3079731248 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8f3c010610 con 0x7f8f48107e30 2024-05-09T04:56:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f457fa700 1 --2- 172.21.15.46:0/3079731248 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8f300b29f0 0x7f8f300b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f47fff700 1 --2- 172.21.15.46:0/3079731248 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8f300b29f0 0x7f8f300b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f457fa700 1 -- 172.21.15.46:0/3079731248 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f8f3c0e20a0 con 0x7f8f48107e30 2024-05-09T04:56:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f5cfaeb80 1 Processor -- start 2024-05-09T04:56:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f5cfaeb80 1 -- start start 2024-05-09T04:56:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f5cfaeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c296160 0x55f18c333900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f5cfaeb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c1ac4a0 0x55f18c333e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f5cfaeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c32dab0 0x55f18c32dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f5cfaeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f18c24bff0 con 0x55f18c32dab0 2024-05-09T04:56:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f5cfaeb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f18c1ea7f0 con 0x55f18c1ac4a0 2024-05-09T04:56:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f5cfaeb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f18c247fb0 con 0x55f18c296160 2024-05-09T04:56:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f47fff700 1 --2- 172.21.15.46:0/3079731248 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8f300b29f0 0x7f8f300b4f00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009c60 tx=0x7f8f38009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f4cb35700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c32dab0 0x55f18c32dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f4cb35700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c32dab0 0x55f18c32dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36476/0 (socket says 172.21.15.46:36476) 2024-05-09T04:56:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f4cb35700 1 -- 172.21.15.46:0/2215410484 learned_addr learned my addr 172.21.15.46:0/2215410484 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.301+0000 7f8f47fff700 1 --2- 172.21.15.46:0/2215410484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c296160 0x55f18c333900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.302+0000 7f8f477fe700 1 --2- 172.21.15.46:0/2215410484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c1ac4a0 0x55f18c333e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.302+0000 7f8f4cb35700 1 -- 172.21.15.46:0/2215410484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c296160 msgr2=0x55f18c333900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:10.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.302+0000 7f8f4cb35700 1 --2- 172.21.15.46:0/2215410484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c296160 0x55f18c333900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.302+0000 7f8f4cb35700 1 -- 172.21.15.46:0/2215410484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c1ac4a0 msgr2=0x55f18c333e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.302+0000 7f8f4cb35700 1 --2- 172.21.15.46:0/2215410484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c1ac4a0 0x55f18c333e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.302+0000 7f8f4cb35700 1 -- 172.21.15.46:0/2215410484 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18c1e2370 con 0x55f18c32dab0 2024-05-09T04:56:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.302+0000 7f8f477fe700 1 --2- 172.21.15.46:0/2215410484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c1ac4a0 0x55f18c333e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:56:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.302+0000 7f8f4cb35700 1 --2- 172.21.15.46:0/2215410484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c32dab0 0x55f18c32dee0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8f40015000 tx=0x7f8f40009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf07e66f8371563 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.302+0000 7f8f2e7fc700 1 -- 172.21.15.46:0/2215410484 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40006020 con 0x55f18c32dab0 2024-05-09T04:56:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.302+0000 7f8f2e7fc700 1 -- 172.21.15.46:0/2215410484 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f40020070 con 0x55f18c32dab0 2024-05-09T04:56:10.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.302+0000 7f8f2e7fc700 1 -- 172.21.15.46:0/2215410484 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f400053f0 con 0x55f18c32dab0 2024-05-09T04:56:10.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.303+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/2215410484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c32dab0 msgr2=0x55f18c32dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:10.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.303+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/2215410484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c32dab0 0x55f18c32dee0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f8f40015000 tx=0x7f8f40009c00 comp rx=0 tx=0).stop 2024-05-09T04:56:10.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.303+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/2215410484 shutdown_connections 2024-05-09T04:56:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.303+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/2215410484 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c32dab0 0x55f18c32dee0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.303+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/2215410484 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c296160 0x55f18c333900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.303+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/2215410484 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c1ac4a0 0x55f18c333e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.303+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/2215410484 >> 172.21.15.46:0/2215410484 conn(0x55f18c20c6a0 msgr2=0x55f18c324400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.303+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/2215410484 shutdown_connections 2024-05-09T04:56:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.303+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/2215410484 wait complete. 2024-05-09T04:56:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.304+0000 7f8f5cfaeb80 1 -- start start 2024-05-09T04:56:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.304+0000 7f8f5cfaeb80 1 Processor -- start 2024-05-09T04:56:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.304+0000 7f8f5cfaeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c1ac4a0 0x55f18c337600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.304+0000 7f8f5cfaeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c296160 0x55f18c337b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.304+0000 7f8f5cfaeb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c32dab0 0x55f18c338080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.304+0000 7f8f5cfaeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55f18c0c7ba0 con 0x55f18c1ac4a0 2024-05-09T04:56:10.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.304+0000 7f8f5cfaeb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55f18c206150 con 0x55f18c32dab0 2024-05-09T04:56:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.304+0000 7f8f5cfaeb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55f18c33a5e0 con 0x55f18c296160 2024-05-09T04:56:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.304+0000 7f8f5cfaeb80 10 client.0 _ll_register_callbacks cb 0x55f18c0bd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.304+0000 7f8f47fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c1ac4a0 0x55f18c337600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.304+0000 7f8f47fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c1ac4a0 0x55f18c337600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36492/0 (socket says 172.21.15.46:36492) 2024-05-09T04:56:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.304+0000 7f8f47fff700 1 -- 172.21.15.46:0/527026770 learned_addr learned my addr 172.21.15.46:0/527026770 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.304+0000 7f8f477fe700 1 --2- 172.21.15.46:0/527026770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c296160 0x55f18c337b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.304+0000 7f8f4cb35700 1 --2- 172.21.15.46:0/527026770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c32dab0 0x55f18c338080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.305+0000 7f8f47fff700 1 -- 172.21.15.46:0/527026770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c296160 msgr2=0x55f18c337b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.305+0000 7f8f47fff700 1 --2- 172.21.15.46:0/527026770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c296160 0x55f18c337b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.305+0000 7f8f47fff700 1 -- 172.21.15.46:0/527026770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c32dab0 msgr2=0x55f18c338080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.305+0000 7f8f47fff700 1 --2- 172.21.15.46:0/527026770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c32dab0 0x55f18c338080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.305+0000 7f8f47fff700 1 -- 172.21.15.46:0/527026770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18c0c7490 con 0x55f18c1ac4a0 2024-05-09T04:56:10.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.305+0000 7f8f277fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:56:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.305+0000 7f8f47fff700 1 --2- 172.21.15.46:0/527026770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c1ac4a0 0x55f18c337600 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8f38003900 tx=0x7f8f380047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194e90000465fb9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.305+0000 7f8f277fe700 1 -- 172.21.15.46:0/527026770 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3801c430 con 0x55f18c1ac4a0 2024-05-09T04:56:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.305+0000 7f8f277fe700 1 -- 172.21.15.46:0/527026770 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f8f3801c5d0 con 0x55f18c1ac4a0 2024-05-09T04:56:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.305+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/527026770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f40020070 con 0x55f18c1ac4a0 2024-05-09T04:56:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.305+0000 7f8f277fe700 1 -- 172.21.15.46:0/527026770 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8f3801c7d0 con 0x55f18c1ac4a0 2024-05-09T04:56:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.305+0000 7f8f5cfaeb80 10 client.16599 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.305+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/527026770 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f18c344ff0 con 0x55f18c1ac4a0 2024-05-09T04:56:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.306+0000 7f8f277fe700 1 -- 172.21.15.46:0/527026770 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f40020070 con 0x55f18c1ac4a0 2024-05-09T04:56:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.306+0000 7f8f277fe700 1 -- 172.21.15.46:0/527026770 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f8f38027a90 con 0x55f18c1ac4a0 2024-05-09T04:56:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.306+0000 7f8f5cfaeb80 10 client.16599 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.306+0000 7f8f5cfaeb80 10 client.16599 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:56:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.306+0000 7f8f5cfaeb80 20 client.16599 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.306+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/527026770 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55f18c347630 0x55f18c349a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.306+0000 7f8f5cfaeb80 4 client.16599 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:56:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.306+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/527026770 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f18c2599d0 con 0x55f18c347630 2024-05-09T04:56:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.306+0000 7f8f477fe700 1 --2- 172.21.15.46:0/527026770 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55f18c347630 0x55f18c349a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.307+0000 7f8f277fe700 10 client.16599 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:56:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.307+0000 7f8f477fe700 1 --2- 172.21.15.46:0/527026770 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55f18c347630 0x55f18c349a50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=37e9f0fd5ff21e6e server_cookie=4848e6e3333a375c in_seq=0 out_seq=0 2024-05-09T04:56:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.307+0000 7f8f277fe700 1 -- 172.21.15.46:0/527026770 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f18c2599d0 con 0x55f18c347630 2024-05-09T04:56:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.307+0000 7f8f277fe700 10 client.16599 handle_command_reply: tid=0 2024-05-09T04:56:10.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.307+0000 7f8f5cfaeb80 1 client.16599 shutdown 2024-05-09T04:56:10.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.308+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/527026770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c1ac4a0 msgr2=0x55f18c337600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:10.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.308+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/527026770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c1ac4a0 0x55f18c337600 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f8f38003900 tx=0x7f8f380047c0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.308+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/527026770 shutdown_connections 2024-05-09T04:56:10.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.308+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/527026770 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55f18c347630 0x55f18c349a50 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.308+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/527026770 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c32dab0 0x55f18c338080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.308+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/527026770 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c1ac4a0 0x55f18c337600 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.308+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/527026770 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c296160 0x55f18c337b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.308+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/527026770 >> 172.21.15.46:0/527026770 conn(0x55f18c20c6a0 msgr2=0x55f18c3241c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.308+0000 7f8f277fe700 0 client.16599 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:56:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.308+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/527026770 shutdown_connections 2024-05-09T04:56:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.308+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/527026770 wait complete. 2024-05-09T04:56:10.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.308+0000 7f8f5cfaeb80 20 client.16599 trim_cache size 0 max 16384 2024-05-09T04:56:10.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.323+0000 7f8f5cfaeb80 1 Processor -- start 2024-05-09T04:56:10.324 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.323+0000 7f8f5cfaeb80 1 -- start start 2024-05-09T04:56:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.323+0000 7f8f5cfaeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c20c6a0 0x55f18c332d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.323+0000 7f8f5cfaeb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c1ac4a0 0x55f18c331bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.323+0000 7f8f5cfaeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c296160 0x55f18c337ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.323+0000 7f8f5cfaeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481af540 con 0x55f18c296160 2024-05-09T04:56:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f5cfaeb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481acdd0 con 0x55f18c1ac4a0 2024-05-09T04:56:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f5cfaeb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481aac20 con 0x55f18c20c6a0 2024-05-09T04:56:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f4cb35700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c296160 0x55f18c337ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f4cb35700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c296160 0x55f18c337ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36502/0 (socket says 172.21.15.46:36502) 2024-05-09T04:56:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f4cb35700 1 -- 172.21.15.46:0/1182854988 learned_addr learned my addr 172.21.15.46:0/1182854988 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f477fe700 1 --2- 172.21.15.46:0/1182854988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c1ac4a0 0x55f18c331bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f47fff700 1 --2- 172.21.15.46:0/1182854988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c20c6a0 0x55f18c332d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f4cb35700 1 -- 172.21.15.46:0/1182854988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c20c6a0 msgr2=0x55f18c332d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:10.325 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f4cb35700 1 --2- 172.21.15.46:0/1182854988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c20c6a0 0x55f18c332d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f4cb35700 1 -- 172.21.15.46:0/1182854988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c1ac4a0 msgr2=0x55f18c331bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:10.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f4cb35700 1 --2- 172.21.15.46:0/1182854988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c1ac4a0 0x55f18c331bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f4cb35700 1 -- 172.21.15.46:0/1182854988 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f18c338400 con 0x55f18c296160 2024-05-09T04:56:10.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f47fff700 1 --2- 172.21.15.46:0/1182854988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c20c6a0 0x55f18c332d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:56:10.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f477fe700 1 --2- 172.21.15.46:0/1182854988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c1ac4a0 0x55f18c331bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:56:10.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f4cb35700 1 --2- 172.21.15.46:0/1182854988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c296160 0x55f18c337ec0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009a20 tx=0x7f8f40009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda48b2584c7390e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:10.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.324+0000 7f8f2dffb700 1 -- 172.21.15.46:0/1182854988 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f40003cc0 con 0x55f18c296160 2024-05-09T04:56:10.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.325+0000 7f8f2dffb700 1 -- 172.21.15.46:0/1182854988 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f40020040 con 0x55f18c296160 2024-05-09T04:56:10.326 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.325+0000 7f8f2dffb700 1 -- 172.21.15.46:0/1182854988 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f400075c0 con 0x55f18c296160 2024-05-09T04:56:10.329 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.325+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/1182854988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c296160 msgr2=0x55f18c337ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.325+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/1182854988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c296160 0x55f18c337ec0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8f40009a20 tx=0x7f8f40009e20 comp rx=0 tx=0).stop 2024-05-09T04:56:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.325+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/1182854988 shutdown_connections 2024-05-09T04:56:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.325+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/1182854988 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c296160 0x55f18c337ec0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.325+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/1182854988 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c20c6a0 0x55f18c332d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.325+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/1182854988 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c1ac4a0 0x55f18c331bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.325+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/1182854988 >> 172.21.15.46:0/1182854988 conn(0x55f18c32e220 msgr2=0x55f18c332600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.325+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/1182854988 shutdown_connections 2024-05-09T04:56:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.325+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/1182854988 wait complete. 2024-05-09T04:56:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.326+0000 7f8f5cfaeb80 1 -- start start 2024-05-09T04:56:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.326+0000 7f8f5cfaeb80 1 Processor -- start 2024-05-09T04:56:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.326+0000 7f8f5cfaeb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c1ac4a0 0x55f18c34e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.326+0000 7f8f5cfaeb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c20c6a0 0x55f18c34e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.330 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.326+0000 7f8f5cfaeb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c296160 0x55f18c34ec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.326+0000 7f8f5cfaeb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481ab290 con 0x55f18c1ac4a0 2024-05-09T04:56:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.326+0000 7f8f5cfaeb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481ac700 con 0x55f18c20c6a0 2024-05-09T04:56:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.326+0000 7f8f5cfaeb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8f481ab960 con 0x55f18c296160 2024-05-09T04:56:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.326+0000 7f8f5cfaeb80 10 client.0 _ll_register_callbacks cb 0x55f18c0bd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f47fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c1ac4a0 0x55f18c34e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f47fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c1ac4a0 0x55f18c34e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36506/0 (socket says 172.21.15.46:36506) 2024-05-09T04:56:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f47fff700 1 -- 172.21.15.46:0/1635714552 learned_addr learned my addr 172.21.15.46:0/1635714552 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f4cb35700 1 --2- 172.21.15.46:0/1635714552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c296160 0x55f18c34ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f477fe700 1 --2- 172.21.15.46:0/1635714552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c20c6a0 0x55f18c34e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f47fff700 1 -- 172.21.15.46:0/1635714552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c296160 msgr2=0x55f18c34ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f47fff700 1 --2- 172.21.15.46:0/1635714552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c296160 0x55f18c34ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f47fff700 1 -- 172.21.15.46:0/1635714552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c20c6a0 msgr2=0x55f18c34e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f47fff700 1 --2- 172.21.15.46:0/1635714552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c20c6a0 0x55f18c34e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f47fff700 1 -- 172.21.15.46:0/1635714552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f18c33e790 con 0x55f18c1ac4a0 2024-05-09T04:56:10.331 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f477fe700 1 --2- 172.21.15.46:0/1635714552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c20c6a0 0x55f18c34e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:56:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f47fff700 1 --2- 172.21.15.46:0/1635714552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c1ac4a0 0x55f18c34e200 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8f38027eb0 tx=0x7f8f3801c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8e77d306803fa9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f277fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:56:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f277fe700 1 -- 172.21.15.46:0/1635714552 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f3803d020 con 0x55f18c1ac4a0 2024-05-09T04:56:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f277fe700 1 -- 172.21.15.46:0/1635714552 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f8f3801c970 con 0x55f18c1ac4a0 2024-05-09T04:56:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.327+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/1635714552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8f40020040 con 0x55f18c1ac4a0 2024-05-09T04:56:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.328+0000 7f8f277fe700 1 -- 172.21.15.46:0/1635714552 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8f38033040 con 0x55f18c1ac4a0 2024-05-09T04:56:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.328+0000 7f8f5cfaeb80 10 client.16605 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.328+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/1635714552 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f18c33fa50 con 0x55f18c1ac4a0 2024-05-09T04:56:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.328+0000 7f8f277fe700 1 -- 172.21.15.46:0/1635714552 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f40020040 con 0x55f18c1ac4a0 2024-05-09T04:56:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.328+0000 7f8f277fe700 1 -- 172.21.15.46:0/1635714552 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f8f38027420 con 0x55f18c1ac4a0 2024-05-09T04:56:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.328+0000 7f8f5cfaeb80 10 client.16605 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.328+0000 7f8f5cfaeb80 10 client.16605 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:56:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.328+0000 7f8f5cfaeb80 20 client.16605 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:10.332 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.328+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/1635714552 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55f18c32dab0 0x55f18c3b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.328+0000 7f8f5cfaeb80 4 client.16605 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:56:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.328+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/1635714552 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f18c253a70 con 0x55f18c32dab0 2024-05-09T04:56:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f477fe700 1 --2- 172.21.15.46:0/1635714552 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55f18c32dab0 0x55f18c3b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f477fe700 1 --2- 172.21.15.46:0/1635714552 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55f18c32dab0 0x55f18c3b3dd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3365766fc6f04465 server_cookie=e53efb187eb5dbde in_seq=0 out_seq=0 2024-05-09T04:56:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f277fe700 10 client.16605 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:56:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f277fe700 1 -- 172.21.15.46:0/1635714552 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f18c253a70 con 0x55f18c32dab0 2024-05-09T04:56:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f277fe700 10 client.16605 handle_command_reply: tid=0 2024-05-09T04:56:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f5cfaeb80 1 client.16605 shutdown 2024-05-09T04:56:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/1635714552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c1ac4a0 msgr2=0x55f18c34e200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/1635714552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c1ac4a0 0x55f18c34e200 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8f38027eb0 tx=0x7f8f3801c6e0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/1635714552 shutdown_connections 2024-05-09T04:56:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/1635714552 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55f18c32dab0 0x55f18c3b3dd0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.333 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/1635714552 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55f18c296160 0x55f18c34ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/1635714552 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55f18c1ac4a0 0x55f18c34e200 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f5cfaeb80 1 --2- 172.21.15.46:0/1635714552 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55f18c20c6a0 0x55f18c34e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f277fe700 0 client.16605 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:56:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/1635714552 >> 172.21.15.46:0/1635714552 conn(0x55f18c32e220 msgr2=0x55f18c331210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.329+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/1635714552 shutdown_connections 2024-05-09T04:56:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.330+0000 7f8f5cfaeb80 1 -- 172.21.15.46:0/1635714552 wait complete. 2024-05-09T04:56:10.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.330+0000 7f8f5cfaeb80 20 client.16605 trim_cache size 0 max 16384 2024-05-09T04:56:10.334 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:56:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.330+0000 7f8f4e598700 1 -- 172.21.15.46:0/3079731248 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8f300b29f0 msgr2=0x7f8f300b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.330+0000 7f8f4e598700 1 --2- 172.21.15.46:0/3079731248 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8f300b29f0 0x7f8f300b4f00 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8f38009c60 tx=0x7f8f38009260 comp rx=0 tx=0).stop 2024-05-09T04:56:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.330+0000 7f8f4e598700 1 -- 172.21.15.46:0/3079731248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f48107e30 msgr2=0x7f8f4819c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.330+0000 7f8f4e598700 1 --2- 172.21.15.46:0/3079731248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f48107e30 0x7f8f4819c150 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8f3c00d980 tx=0x7f8f3c00dc90 comp rx=0 tx=0).stop 2024-05-09T04:56:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.330+0000 7f8f4e598700 1 -- 172.21.15.46:0/3079731248 shutdown_connections 2024-05-09T04:56:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.330+0000 7f8f4e598700 1 --2- 172.21.15.46:0/3079731248 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8f300b29f0 0x7f8f300b4f00 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.330+0000 7f8f4e598700 1 --2- 172.21.15.46:0/3079731248 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8f48108780 0x7f8f4819c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.330+0000 7f8f4e598700 1 --2- 172.21.15.46:0/3079731248 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8f48103550 0x7f8f4819bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.330+0000 7f8f4e598700 1 --2- 172.21.15.46:0/3079731248 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8f48107e30 0x7f8f4819c150 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.330+0000 7f8f4e598700 1 -- 172.21.15.46:0/3079731248 >> 172.21.15.46:0/3079731248 conn(0x7f8f48078e00 msgr2=0x7f8f48101dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.330+0000 7f8f4e598700 1 -- 172.21.15.46:0/3079731248 shutdown_connections 2024-05-09T04:56:10.336 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:10.330+0000 7f8f4e598700 1 -- 172.21.15.46:0/3079731248 wait complete. 2024-05-09T04:56:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:10 smithi110 ceph-mon[34904]: pgmap v155: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-05-09T04:56:11.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:10 smithi046 ceph-mon[30136]: pgmap v155: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-05-09T04:56:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:10 smithi072 ceph-mon[34088]: pgmap v155: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 21 op/s 2024-05-09T04:56:11.137 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:56:11.387 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:13.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:12 smithi110 ceph-mon[34904]: pgmap v156: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T04:56:13.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:12 smithi046 ceph-mon[30136]: pgmap v156: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T04:56:13.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:12 smithi072 ceph-mon[34088]: pgmap v156: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T04:56:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.345+0000 7f55dd86b700 1 -- 172.21.15.46:0/2075364237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55d8108740 msgr2=0x7f55d81100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.345+0000 7f55dd86b700 1 --2- 172.21.15.46:0/2075364237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55d8108740 0x7f55d81100a0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f55d00099c0 tx=0x7f55d0009cd0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.345+0000 7f55dd86b700 1 -- 172.21.15.46:0/2075364237 shutdown_connections 2024-05-09T04:56:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.345+0000 7f55dd86b700 1 --2- 172.21.15.46:0/2075364237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55d8108740 0x7f55d81100a0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.345+0000 7f55dd86b700 1 --2- 172.21.15.46:0/2075364237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55d81034c0 0x7f55d81038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.345+0000 7f55dd86b700 1 --2- 172.21.15.46:0/2075364237 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55d8107df0 0x7f55d8108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.346+0000 7f55dd86b700 1 -- 172.21.15.46:0/2075364237 >> 172.21.15.46:0/2075364237 conn(0x7f55d80fedf0 msgr2=0x7f55d8101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:13.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.346+0000 7f55dd86b700 1 -- 172.21.15.46:0/2075364237 shutdown_connections 2024-05-09T04:56:13.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.346+0000 7f55dd86b700 1 -- 172.21.15.46:0/2075364237 wait complete. 2024-05-09T04:56:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.346+0000 7f55dd86b700 1 Processor -- start 2024-05-09T04:56:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55dd86b700 1 -- start start 2024-05-09T04:56:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55dd86b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55d81034c0 0x7f55d80ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55dd86b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55d8107df0 0x7f55d80ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55dd86b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55d8108740 0x7f55d80ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55dd86b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8113390 con 0x7f55d8107df0 2024-05-09T04:56:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55dd86b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8113210 con 0x7f55d8108740 2024-05-09T04:56:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55dd86b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55d8113510 con 0x7f55d81034c0 2024-05-09T04:56:13.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55d67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55d8107df0 0x7f55d80ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55d67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55d8107df0 0x7f55d80ff810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36520/0 (socket says 172.21.15.46:36520) 2024-05-09T04:56:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55d67fc700 1 -- 172.21.15.46:0/3658492516 learned_addr learned my addr 172.21.15.46:0/3658492516 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55d6ffd700 1 --2- 172.21.15.46:0/3658492516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55d81034c0 0x7f55d80ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55d77fe700 1 --2- 172.21.15.46:0/3658492516 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55d8108740 0x7f55d80ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55d67fc700 1 -- 172.21.15.46:0/3658492516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55d81034c0 msgr2=0x7f55d80ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55d67fc700 1 --2- 172.21.15.46:0/3658492516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55d81034c0 0x7f55d80ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55d67fc700 1 -- 172.21.15.46:0/3658492516 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55d8108740 msgr2=0x7f55d80ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55d67fc700 1 --2- 172.21.15.46:0/3658492516 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55d8108740 0x7f55d80ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.347+0000 7f55d67fc700 1 -- 172.21.15.46:0/3658492516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55d81a91f0 con 0x7f55d8107df0 2024-05-09T04:56:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.348+0000 7f55d67fc700 1 --2- 172.21.15.46:0/3658492516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55d8107df0 0x7f55d80ff810 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f55c000e970 tx=0x7f55c000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.348+0000 7f55dc869700 1 -- 172.21.15.46:0/3658492516 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c000cbd0 con 0x7f55d8107df0 2024-05-09T04:56:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.348+0000 7f55dd86b700 1 -- 172.21.15.46:0/3658492516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55d81a94e0 con 0x7f55d8107df0 2024-05-09T04:56:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.348+0000 7f55dc869700 1 -- 172.21.15.46:0/3658492516 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55c000cd70 con 0x7f55d8107df0 2024-05-09T04:56:13.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.348+0000 7f55dc869700 1 -- 172.21.15.46:0/3658492516 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c0005680 con 0x7f55d8107df0 2024-05-09T04:56:13.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.348+0000 7f55dd86b700 1 -- 172.21.15.46:0/3658492516 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55d81a9a70 con 0x7f55d8107df0 2024-05-09T04:56:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.351+0000 7f55dc869700 1 -- 172.21.15.46:0/3658492516 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f55c0004750 con 0x7f55d8107df0 2024-05-09T04:56:13.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.351+0000 7f55dc869700 1 --2- 172.21.15.46:0/3658492516 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f55c40b29f0 0x7f55c40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.352+0000 7f55dc869700 1 -- 172.21.15.46:0/3658492516 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f55c00e2810 con 0x7f55d8107df0 2024-05-09T04:56:13.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.352+0000 7f55d6ffd700 1 --2- 172.21.15.46:0/3658492516 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f55c40b29f0 0x7f55c40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.352+0000 7f55ec281b80 1 Processor -- start 2024-05-09T04:56:13.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.352+0000 7f55ec281b80 1 -- start start 2024-05-09T04:56:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.352+0000 7f55d6ffd700 1 --2- 172.21.15.46:0/3658492516 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f55c40b29f0 0x7f55c40b4f00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f55c800f540 tx=0x7f55c800b040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.352+0000 7f55ec281b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa9814d00 0x55faa991d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.352+0000 7f55ec281b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa979a4a0 0x55faa991d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.352+0000 7f55ec281b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9917400 0x55faa9917810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.352+0000 7f55ec281b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55faa983a0a0 con 0x55faa979a4a0 2024-05-09T04:56:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.352+0000 7f55ec281b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55faa97df9a0 con 0x55faa9917400 2024-05-09T04:56:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.352+0000 7f55ec281b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55faa9836060 con 0x55faa9814d00 2024-05-09T04:56:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.352+0000 7f55d67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa979a4a0 0x55faa991d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.352+0000 7f55d67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa979a4a0 0x55faa991d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36522/0 (socket says 172.21.15.46:36522) 2024-05-09T04:56:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.352+0000 7f55d67fc700 1 -- 172.21.15.46:0/324950019 learned_addr learned my addr 172.21.15.46:0/324950019 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.352+0000 7f55d6ffd700 1 --2- 172.21.15.46:0/324950019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa9814d00 0x55faa991d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.353+0000 7f55d77fe700 1 --2- 172.21.15.46:0/324950019 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9917400 0x55faa9917810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.353+0000 7f55d67fc700 1 -- 172.21.15.46:0/324950019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa9814d00 msgr2=0x55faa991d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:13.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.353+0000 7f55d67fc700 1 --2- 172.21.15.46:0/324950019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa9814d00 0x55faa991d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.353+0000 7f55d67fc700 1 -- 172.21.15.46:0/324950019 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9917400 msgr2=0x55faa9917810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:13.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.353+0000 7f55d67fc700 1 --2- 172.21.15.46:0/324950019 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9917400 0x55faa9917810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.353+0000 7f55d67fc700 1 -- 172.21.15.46:0/324950019 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa97d0500 con 0x55faa979a4a0 2024-05-09T04:56:13.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.353+0000 7f55d67fc700 1 --2- 172.21.15.46:0/324950019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa979a4a0 0x55faa991d780 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009230 tx=0x7f55c00a9240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b51e16c4405be946 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:13.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.353+0000 7f55cdffb700 1 -- 172.21.15.46:0/324950019 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c00c3070 con 0x55faa979a4a0 2024-05-09T04:56:13.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.353+0000 7f55cdffb700 1 -- 172.21.15.46:0/324950019 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55c00a95e0 con 0x55faa979a4a0 2024-05-09T04:56:13.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.353+0000 7f55cdffb700 1 -- 172.21.15.46:0/324950019 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c00a9d30 con 0x55faa979a4a0 2024-05-09T04:56:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.354+0000 7f55ec281b80 1 -- 172.21.15.46:0/324950019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa979a4a0 msgr2=0x55faa991d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.354+0000 7f55ec281b80 1 --2- 172.21.15.46:0/324950019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa979a4a0 0x55faa991d780 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009230 tx=0x7f55c00a9240 comp rx=0 tx=0).stop 2024-05-09T04:56:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.354+0000 7f55ec281b80 1 -- 172.21.15.46:0/324950019 shutdown_connections 2024-05-09T04:56:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.354+0000 7f55ec281b80 1 --2- 172.21.15.46:0/324950019 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9917400 0x55faa9917810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.354+0000 7f55ec281b80 1 --2- 172.21.15.46:0/324950019 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa9814d00 0x55faa991d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.354+0000 7f55ec281b80 1 --2- 172.21.15.46:0/324950019 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa979a4a0 0x55faa991d780 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.354+0000 7f55ec281b80 1 -- 172.21.15.46:0/324950019 >> 172.21.15.46:0/324950019 conn(0x55faa979df70 msgr2=0x55faa9912220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.354+0000 7f55ec281b80 1 -- 172.21.15.46:0/324950019 shutdown_connections 2024-05-09T04:56:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.354+0000 7f55ec281b80 1 -- 172.21.15.46:0/324950019 wait complete. 2024-05-09T04:56:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.355+0000 7f55ec281b80 1 -- start start 2024-05-09T04:56:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.355+0000 7f55ec281b80 1 Processor -- start 2024-05-09T04:56:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.355+0000 7f55ec281b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa979a4a0 0x55faa9921040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.355+0000 7f55ec281b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa9814d00 0x55faa9921580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.355+0000 7f55ec281b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9917400 0x55faa9921ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.355+0000 7f55ec281b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55faa9918100 con 0x55faa979a4a0 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.355+0000 7f55ec281b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55faa97d4b20 con 0x55faa9917400 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.355+0000 7f55ec281b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55faa992d1d0 con 0x55faa9814d00 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.355+0000 7f55ec281b80 10 client.0 _ll_register_callbacks cb 0x55faa96ab6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.355+0000 7f55d6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa979a4a0 0x55faa9921040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.355+0000 7f55d6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa979a4a0 0x55faa9921040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36532/0 (socket says 172.21.15.46:36532) 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.355+0000 7f55d6ffd700 1 -- 172.21.15.46:0/55298911 learned_addr learned my addr 172.21.15.46:0/55298911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.356+0000 7f55d77fe700 1 --2- 172.21.15.46:0/55298911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9917400 0x55faa9921ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.356+0000 7f55d67fc700 1 --2- 172.21.15.46:0/55298911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa9814d00 0x55faa9921580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.356+0000 7f55d6ffd700 1 -- 172.21.15.46:0/55298911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa9814d00 msgr2=0x55faa9921580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.356+0000 7f55d6ffd700 1 --2- 172.21.15.46:0/55298911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa9814d00 0x55faa9921580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.356+0000 7f55d6ffd700 1 -- 172.21.15.46:0/55298911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9917400 msgr2=0x55faa9921ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.356+0000 7f55d6ffd700 1 --2- 172.21.15.46:0/55298911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9917400 0x55faa9921ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.356+0000 7f55d6ffd700 1 -- 172.21.15.46:0/55298911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa96b5490 con 0x55faa979a4a0 2024-05-09T04:56:13.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.356+0000 7f55d6ffd700 1 --2- 172.21.15.46:0/55298911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa979a4a0 0x55faa9921040 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f55c8008bd0 tx=0x7f55c8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7498e76bf2d2ea9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.356+0000 7f55b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.356+0000 7f55b6ffd700 1 -- 172.21.15.46:0/55298911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c8027070 con 0x55faa979a4a0 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.356+0000 7f55ec281b80 1 -- 172.21.15.46:0/55298911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55c00a95e0 con 0x55faa979a4a0 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.356+0000 7f55b6ffd700 1 -- 172.21.15.46:0/55298911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f55c8017c60 con 0x55faa979a4a0 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.356+0000 7f55b6ffd700 1 -- 172.21.15.46:0/55298911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f55c8033050 con 0x55faa979a4a0 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.356+0000 7f55ec281b80 10 client.16617 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.356+0000 7f55ec281b80 1 -- 172.21.15.46:0/55298911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55faa992eb80 con 0x55faa979a4a0 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.357+0000 7f55b6ffd700 1 -- 172.21.15.46:0/55298911 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55c00a95e0 con 0x55faa979a4a0 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.357+0000 7f55b6ffd700 1 -- 172.21.15.46:0/55298911 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f55c8015410 con 0x55faa979a4a0 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.357+0000 7f55ec281b80 10 client.16617 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.357+0000 7f55ec281b80 10 client.16617 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.357+0000 7f55ec281b80 20 client.16617 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.357+0000 7f55ec281b80 1 --2- 172.21.15.46:0/55298911 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55faa9931160 0x55faa9933580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.357+0000 7f55ec281b80 4 client.16617 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.357+0000 7f55ec281b80 1 -- 172.21.15.46:0/55298911 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faa9847a80 con 0x55faa9931160 2024-05-09T04:56:13.360 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.357+0000 7f55d67fc700 1 --2- 172.21.15.46:0/55298911 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55faa9931160 0x55faa9933580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.358+0000 7f55b6ffd700 10 client.16617 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.358+0000 7f55d67fc700 1 --2- 172.21.15.46:0/55298911 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55faa9931160 0x55faa9933580 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26e69bbac05a1e9c server_cookie=de092239db2b0702 in_seq=0 out_seq=0 2024-05-09T04:56:13.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.358+0000 7f55b6ffd700 1 -- 172.21.15.46:0/55298911 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55faa9847a80 con 0x55faa9931160 2024-05-09T04:56:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.358+0000 7f55b6ffd700 10 client.16617 handle_command_reply: tid=0 2024-05-09T04:56:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.358+0000 7f55ec281b80 1 client.16617 shutdown 2024-05-09T04:56:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.359+0000 7f55ec281b80 1 -- 172.21.15.46:0/55298911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa979a4a0 msgr2=0x55faa9921040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.359+0000 7f55ec281b80 1 --2- 172.21.15.46:0/55298911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa979a4a0 0x55faa9921040 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f55c8008bd0 tx=0x7f55c8004ad0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.359+0000 7f55ec281b80 1 -- 172.21.15.46:0/55298911 shutdown_connections 2024-05-09T04:56:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.359+0000 7f55ec281b80 1 --2- 172.21.15.46:0/55298911 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55faa9931160 0x55faa9933580 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.359+0000 7f55ec281b80 1 --2- 172.21.15.46:0/55298911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9917400 0x55faa9921ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.359+0000 7f55ec281b80 1 --2- 172.21.15.46:0/55298911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa979a4a0 0x55faa9921040 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.359+0000 7f55b6ffd700 0 client.16617 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.359+0000 7f55ec281b80 1 --2- 172.21.15.46:0/55298911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa9814d00 0x55faa9921580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.359+0000 7f55ec281b80 1 -- 172.21.15.46:0/55298911 >> 172.21.15.46:0/55298911 conn(0x55faa979df70 msgr2=0x55faa990bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.359+0000 7f55ec281b80 1 -- 172.21.15.46:0/55298911 shutdown_connections 2024-05-09T04:56:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.359+0000 7f55ec281b80 1 -- 172.21.15.46:0/55298911 wait complete. 2024-05-09T04:56:13.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.359+0000 7f55ec281b80 20 client.16617 trim_cache size 0 max 16384 2024-05-09T04:56:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.374+0000 7f55ec281b80 1 Processor -- start 2024-05-09T04:56:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.374+0000 7f55ec281b80 1 -- start start 2024-05-09T04:56:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.374+0000 7f55ec281b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa979a4a0 0x55faa990ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.374+0000 7f55ec281b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9814d00 0x55faa991c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.374+0000 7f55ec281b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa9917400 0x55faa9921900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.374+0000 7f55ec281b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81b3b00 con 0x55faa9917400 2024-05-09T04:56:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.374+0000 7f55ec281b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81b1390 con 0x55faa9814d00 2024-05-09T04:56:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.374+0000 7f55ec281b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81af1e0 con 0x55faa979a4a0 2024-05-09T04:56:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.374+0000 7f55d77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa9917400 0x55faa9921900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.374+0000 7f55d77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa9917400 0x55faa9921900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36542/0 (socket says 172.21.15.46:36542) 2024-05-09T04:56:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.374+0000 7f55d77fe700 1 -- 172.21.15.46:0/1423226692 learned_addr learned my addr 172.21.15.46:0/1423226692 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.374+0000 7f55d6ffd700 1 --2- 172.21.15.46:0/1423226692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa979a4a0 0x55faa990ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.374+0000 7f55d67fc700 1 --2- 172.21.15.46:0/1423226692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9814d00 0x55faa991c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.375+0000 7f55d77fe700 1 -- 172.21.15.46:0/1423226692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa979a4a0 msgr2=0x55faa990ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.375+0000 7f55d77fe700 1 --2- 172.21.15.46:0/1423226692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa979a4a0 0x55faa990ca20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.375+0000 7f55d77fe700 1 -- 172.21.15.46:0/1423226692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9814d00 msgr2=0x55faa991c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.375+0000 7f55d77fe700 1 --2- 172.21.15.46:0/1423226692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9814d00 0x55faa991c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.375+0000 7f55d77fe700 1 -- 172.21.15.46:0/1423226692 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faa99238e0 con 0x55faa9917400 2024-05-09T04:56:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.375+0000 7f55d67fc700 1 --2- 172.21.15.46:0/1423226692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9814d00 0x55faa991c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:56:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.375+0000 7f55d77fe700 1 --2- 172.21.15.46:0/1423226692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa9917400 0x55faa9921900 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f55d0015000 tx=0x7f55d0005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b85595e714832e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.375+0000 7f55cd7fa700 1 -- 172.21.15.46:0/1423226692 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d00052b0 con 0x55faa9917400 2024-05-09T04:56:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.375+0000 7f55cd7fa700 1 -- 172.21.15.46:0/1423226692 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d0003c70 con 0x55faa9917400 2024-05-09T04:56:13.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.375+0000 7f55cd7fa700 1 -- 172.21.15.46:0/1423226692 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d000bbf0 con 0x55faa9917400 2024-05-09T04:56:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.376+0000 7f55ec281b80 1 -- 172.21.15.46:0/1423226692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa9917400 msgr2=0x55faa9921900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.376+0000 7f55ec281b80 1 --2- 172.21.15.46:0/1423226692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa9917400 0x55faa9921900 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f55d0015000 tx=0x7f55d0005040 comp rx=0 tx=0).stop 2024-05-09T04:56:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.376+0000 7f55ec281b80 1 -- 172.21.15.46:0/1423226692 shutdown_connections 2024-05-09T04:56:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.376+0000 7f55ec281b80 1 --2- 172.21.15.46:0/1423226692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa9917400 0x55faa9921900 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.376+0000 7f55ec281b80 1 --2- 172.21.15.46:0/1423226692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa979a4a0 0x55faa990ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.376+0000 7f55ec281b80 1 --2- 172.21.15.46:0/1423226692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa9814d00 0x55faa991c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.376+0000 7f55ec281b80 1 -- 172.21.15.46:0/1423226692 >> 172.21.15.46:0/1423226692 conn(0x55faa979df70 msgr2=0x55faa991acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.376+0000 7f55ec281b80 1 -- 172.21.15.46:0/1423226692 shutdown_connections 2024-05-09T04:56:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.376+0000 7f55ec281b80 1 -- 172.21.15.46:0/1423226692 wait complete. 2024-05-09T04:56:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.376+0000 7f55ec281b80 1 -- start start 2024-05-09T04:56:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.376+0000 7f55ec281b80 1 Processor -- start 2024-05-09T04:56:13.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55ec281b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa979a4a0 0x55faa9926ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55ec281b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa9814d00 0x55faa99271e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55ec281b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa9917400 0x55faa9927720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55ec281b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81af850 con 0x55faa9917400 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55ec281b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81b0cc0 con 0x55faa979a4a0 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55ec281b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f55d81aff20 con 0x55faa9814d00 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55ec281b80 10 client.0 _ll_register_callbacks cb 0x55faa96ab6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55d77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa9917400 0x55faa9927720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55d6ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa979a4a0 0x55faa9926ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55d77fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa9917400 0x55faa9927720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36550/0 (socket says 172.21.15.46:36550) 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55d67fc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa9814d00 0x55faa99271e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55d77fe700 1 -- 172.21.15.46:0/3407534300 learned_addr learned my addr 172.21.15.46:0/3407534300 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55d77fe700 1 -- 172.21.15.46:0/3407534300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa9814d00 msgr2=0x55faa99271e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55d77fe700 1 --2- 172.21.15.46:0/3407534300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa9814d00 0x55faa99271e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55d77fe700 1 -- 172.21.15.46:0/3407534300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa979a4a0 msgr2=0x55faa9926ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55d77fe700 1 --2- 172.21.15.46:0/3407534300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa979a4a0 0x55faa9926ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.377+0000 7f55d77fe700 1 -- 172.21.15.46:0/3407534300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faa999a940 con 0x55faa9917400 2024-05-09T04:56:13.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.378+0000 7f55d77fe700 1 --2- 172.21.15.46:0/3407534300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa9917400 0x55faa9927720 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f55d00094e0 tx=0x7f55d0005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b391ee3f953f442 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.378+0000 7f55b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.378+0000 7f55b6ffd700 1 -- 172.21.15.46:0/3407534300 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55d0032030 con 0x55faa9917400 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.378+0000 7f55b6ffd700 1 -- 172.21.15.46:0/3407534300 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f55d000ba40 con 0x55faa9917400 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.378+0000 7f55ec281b80 1 -- 172.21.15.46:0/3407534300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f55d0003c70 con 0x55faa9917400 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.378+0000 7f55b6ffd700 1 -- 172.21.15.46:0/3407534300 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f55d00055e0 con 0x55faa9917400 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.378+0000 7f55ec281b80 10 client.16623 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.378+0000 7f55ec281b80 1 -- 172.21.15.46:0/3407534300 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55faa999ba70 con 0x55faa9917400 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.378+0000 7f55b6ffd700 1 -- 172.21.15.46:0/3407534300 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55d0003c70 con 0x55faa9917400 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.378+0000 7f55b6ffd700 1 -- 172.21.15.46:0/3407534300 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f55d000fbe0 con 0x55faa9917400 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.378+0000 7f55ec281b80 10 client.16623 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.378+0000 7f55ec281b80 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.378+0000 7f55ec281b80 20 client.16623 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.378+0000 7f55ec281b80 1 --2- 172.21.15.46:0/3407534300 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55faa999c060 0x55faa999e480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.379+0000 7f55ec281b80 4 client.16623 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.379+0000 7f55ec281b80 1 -- 172.21.15.46:0/3407534300 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55faa9841b20 con 0x55faa999c060 2024-05-09T04:56:13.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.379+0000 7f55d6ffd700 1 --2- 172.21.15.46:0/3407534300 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55faa999c060 0x55faa999e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.379+0000 7f55d6ffd700 1 --2- 172.21.15.46:0/3407534300 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55faa999c060 0x55faa999e480 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c13d888858aa6604 server_cookie=1b8ed1fa8cbc3fd4 in_seq=0 out_seq=0 2024-05-09T04:56:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.379+0000 7f55b6ffd700 10 client.16623 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.379+0000 7f55b6ffd700 1 -- 172.21.15.46:0/3407534300 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55faa9841b20 con 0x55faa999c060 2024-05-09T04:56:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.379+0000 7f55b6ffd700 10 client.16623 handle_command_reply: tid=0 2024-05-09T04:56:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.379+0000 7f55ec281b80 1 client.16623 shutdown 2024-05-09T04:56:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55ec281b80 1 -- 172.21.15.46:0/3407534300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa9917400 msgr2=0x55faa9927720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55ec281b80 1 --2- 172.21.15.46:0/3407534300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa9917400 0x55faa9927720 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f55d00094e0 tx=0x7f55d0005360 comp rx=0 tx=0).stop 2024-05-09T04:56:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55ec281b80 1 -- 172.21.15.46:0/3407534300 shutdown_connections 2024-05-09T04:56:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55ec281b80 1 --2- 172.21.15.46:0/3407534300 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55faa9917400 0x55faa9927720 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55ec281b80 1 --2- 172.21.15.46:0/3407534300 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55faa999c060 0x55faa999e480 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55ec281b80 1 --2- 172.21.15.46:0/3407534300 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55faa979a4a0 0x55faa9926ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55ec281b80 1 --2- 172.21.15.46:0/3407534300 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55faa9814d00 0x55faa99271e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55ec281b80 1 -- 172.21.15.46:0/3407534300 >> 172.21.15.46:0/3407534300 conn(0x55faa979df70 msgr2=0x55faa990c780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55b6ffd700 0 client.16623 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55ec281b80 1 -- 172.21.15.46:0/3407534300 shutdown_connections 2024-05-09T04:56:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55ec281b80 1 -- 172.21.15.46:0/3407534300 wait complete. 2024-05-09T04:56:13.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55ec281b80 20 client.16623 trim_cache size 0 max 16384 2024-05-09T04:56:13.384 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:56:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55dd86b700 1 -- 172.21.15.46:0/3658492516 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f55c40b29f0 msgr2=0x7f55c40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55dd86b700 1 --2- 172.21.15.46:0/3658492516 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f55c40b29f0 0x7f55c40b4f00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f55c800f540 tx=0x7f55c800b040 comp rx=0 tx=0).stop 2024-05-09T04:56:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55dd86b700 1 -- 172.21.15.46:0/3658492516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55d8107df0 msgr2=0x7f55d80ff810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.380+0000 7f55dd86b700 1 --2- 172.21.15.46:0/3658492516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55d8107df0 0x7f55d80ff810 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f55c000e970 tx=0x7f55c000ec80 comp rx=0 tx=0).stop 2024-05-09T04:56:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.381+0000 7f55dd86b700 1 -- 172.21.15.46:0/3658492516 shutdown_connections 2024-05-09T04:56:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.381+0000 7f55dd86b700 1 --2- 172.21.15.46:0/3658492516 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f55c40b29f0 0x7f55c40b4f00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.381+0000 7f55dd86b700 1 --2- 172.21.15.46:0/3658492516 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f55d8108740 0x7f55d80ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.381+0000 7f55dd86b700 1 --2- 172.21.15.46:0/3658492516 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f55d81034c0 0x7f55d80ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.381+0000 7f55dd86b700 1 --2- 172.21.15.46:0/3658492516 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f55d8107df0 0x7f55d80ff810 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.381+0000 7f55dd86b700 1 -- 172.21.15.46:0/3658492516 >> 172.21.15.46:0/3658492516 conn(0x7f55d80fedf0 msgr2=0x7f55d8075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.381+0000 7f55dd86b700 1 -- 172.21.15.46:0/3658492516 shutdown_connections 2024-05-09T04:56:13.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:13.381+0000 7f55dd86b700 1 -- 172.21.15.46:0/3658492516 wait complete. 2024-05-09T04:56:15.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:14 smithi110 ceph-mon[34904]: pgmap v157: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-05-09T04:56:15.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:14 smithi046 ceph-mon[30136]: pgmap v157: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-05-09T04:56:15.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:14 smithi072 ceph-mon[34088]: pgmap v157: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 11 KiB/s wr, 18 op/s 2024-05-09T04:56:15.234 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:56:15.234 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:56:15.501 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:16.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:16 smithi046 ceph-mon[30136]: pgmap v158: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T04:56:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:16 smithi110 ceph-mon[34904]: pgmap v158: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T04:56:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:16 smithi072 ceph-mon[34088]: pgmap v158: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T04:56:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.164+0000 7f9afd6e5700 1 -- 172.21.15.46:0/294511107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af8109850 msgr2=0x7f9af81111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.164+0000 7f9afd6e5700 1 --2- 172.21.15.46:0/294511107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af8109850 0x7f9af81111b0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f9af00099c0 tx=0x7f9af0009cd0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.164+0000 7f9afd6e5700 1 -- 172.21.15.46:0/294511107 shutdown_connections 2024-05-09T04:56:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.164+0000 7f9afd6e5700 1 --2- 172.21.15.46:0/294511107 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af8109850 0x7f9af81111b0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.164+0000 7f9afd6e5700 1 --2- 172.21.15.46:0/294511107 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9af8102ae0 0x7f9af8102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.164+0000 7f9afd6e5700 1 --2- 172.21.15.46:0/294511107 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9af8108f00 0x7f9af8109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.164+0000 7f9afd6e5700 1 -- 172.21.15.46:0/294511107 >> 172.21.15.46:0/294511107 conn(0x7f9af80fe4e0 msgr2=0x7f9af8100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.164+0000 7f9afd6e5700 1 -- 172.21.15.46:0/294511107 shutdown_connections 2024-05-09T04:56:17.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.164+0000 7f9afd6e5700 1 -- 172.21.15.46:0/294511107 wait complete. 2024-05-09T04:56:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.165+0000 7f9afd6e5700 1 Processor -- start 2024-05-09T04:56:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.165+0000 7f9afd6e5700 1 -- start start 2024-05-09T04:56:17.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9afd6e5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9af8102ae0 0x7f9af819ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9afd6e5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af8108f00 0x7f9af819b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9afd6e5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9af8109850 0x7f9af819f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9afd6e5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9af8114500 con 0x7f9af8108f00 2024-05-09T04:56:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9afd6e5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9af8114380 con 0x7f9af8109850 2024-05-09T04:56:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9afd6e5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9af8114680 con 0x7f9af8102ae0 2024-05-09T04:56:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9af67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af8108f00 0x7f9af819b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9af67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af8108f00 0x7f9af819b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57786/0 (socket says 172.21.15.46:57786) 2024-05-09T04:56:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9af67fc700 1 -- 172.21.15.46:0/1432342285 learned_addr learned my addr 172.21.15.46:0/1432342285 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9af77fe700 1 --2- 172.21.15.46:0/1432342285 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9af8109850 0x7f9af819f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9af6ffd700 1 --2- 172.21.15.46:0/1432342285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9af8102ae0 0x7f9af819ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9af67fc700 1 -- 172.21.15.46:0/1432342285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9af8102ae0 msgr2=0x7f9af819ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9af67fc700 1 --2- 172.21.15.46:0/1432342285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9af8102ae0 0x7f9af819ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9af67fc700 1 -- 172.21.15.46:0/1432342285 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9af8109850 msgr2=0x7f9af819f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9af67fc700 1 --2- 172.21.15.46:0/1432342285 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9af8109850 0x7f9af819f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9af67fc700 1 -- 172.21.15.46:0/1432342285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9af819fce0 con 0x7f9af8108f00 2024-05-09T04:56:17.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.166+0000 7f9af67fc700 1 --2- 172.21.15.46:0/1432342285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af8108f00 0x7f9af819b3b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000d980 tx=0x7f9ae000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:17.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.167+0000 7f9aeffff700 1 -- 172.21.15.46:0/1432342285 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0017070 con 0x7f9af8108f00 2024-05-09T04:56:17.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.167+0000 7f9afd6e5700 1 -- 172.21.15.46:0/1432342285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9af819ffd0 con 0x7f9af8108f00 2024-05-09T04:56:17.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.167+0000 7f9aeffff700 1 -- 172.21.15.46:0/1432342285 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae000fd40 con 0x7f9af8108f00 2024-05-09T04:56:17.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.167+0000 7f9aeffff700 1 -- 172.21.15.46:0/1432342285 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae000e3b0 con 0x7f9af8108f00 2024-05-09T04:56:17.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.167+0000 7f9afd6e5700 1 -- 172.21.15.46:0/1432342285 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9af81a78f0 con 0x7f9af8108f00 2024-05-09T04:56:17.170 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.170+0000 7f9aeffff700 1 -- 172.21.15.46:0/1432342285 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9ae000e550 con 0x7f9af8108f00 2024-05-09T04:56:17.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.171+0000 7f9b0c0fbb80 1 Processor -- start 2024-05-09T04:56:17.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.171+0000 7f9b0c0fbb80 1 -- start start 2024-05-09T04:56:17.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.171+0000 7f9aeffff700 1 --2- 172.21.15.46:0/1432342285 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9ae40b29f0 0x7f9ae40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9af77fe700 1 --2- 172.21.15.46:0/1432342285 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9ae40b29f0 0x7f9ae40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9aeffff700 1 -- 172.21.15.46:0/1432342285 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f9ae00e2140 con 0x7f9af8108f00 2024-05-09T04:56:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9b0c0fbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4411e0 0x5555ed4d8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9b0c0fbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed357520 0x5555ed4d9370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9b0c0fbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4dd480 0x5555ed4dd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9b0c0fbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5555ed3f7070 con 0x5555ed4dd480 2024-05-09T04:56:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9b0c0fbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5555ed395870 con 0x5555ed4411e0 2024-05-09T04:56:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9b0c0fbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5555ed3f3030 con 0x5555ed357520 2024-05-09T04:56:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9af6ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4411e0 0x5555ed4d8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9af6ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4411e0 0x5555ed4d8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:50058/0 (socket says 172.21.15.46:50058) 2024-05-09T04:56:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9af6ffd700 1 -- 172.21.15.46:0/989848645 learned_addr learned my addr 172.21.15.46:0/989848645 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9af67fc700 1 --2- 172.21.15.46:0/989848645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed357520 0x5555ed4d9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9af77fe700 1 --2- 172.21.15.46:0/989848645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4dd480 0x5555ed4dd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9af6ffd700 1 -- 172.21.15.46:0/989848645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed357520 msgr2=0x5555ed4d9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9af6ffd700 1 --2- 172.21.15.46:0/989848645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed357520 0x5555ed4d9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9af6ffd700 1 -- 172.21.15.46:0/989848645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4dd480 msgr2=0x5555ed4dd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.172+0000 7f9af6ffd700 1 --2- 172.21.15.46:0/989848645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4dd480 0x5555ed4dd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.173+0000 7f9af6ffd700 1 -- 172.21.15.46:0/989848645 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ed38d3f0 con 0x5555ed4411e0 2024-05-09T04:56:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.173+0000 7f9af77fe700 1 --2- 172.21.15.46:0/1432342285 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9ae40b29f0 0x7f9ae40b4f00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9af0003800 tx=0x7f9af00096a0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.173+0000 7f9af6ffd700 1 --2- 172.21.15.46:0/989848645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4411e0 0x5555ed4d8dd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800a4b0 tx=0x7f9ae80096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ec1d054b1a2292f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.173+0000 7f9aed7fa700 1 -- 172.21.15.46:0/989848645 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8014620 con 0x5555ed4411e0 2024-05-09T04:56:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.173+0000 7f9aed7fa700 1 -- 172.21.15.46:0/989848645 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae8019070 con 0x5555ed4411e0 2024-05-09T04:56:17.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.173+0000 7f9aed7fa700 1 -- 172.21.15.46:0/989848645 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae8013a70 con 0x5555ed4411e0 2024-05-09T04:56:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.174+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/989848645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4411e0 msgr2=0x5555ed4d8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.174+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/989848645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4411e0 0x5555ed4d8dd0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800a4b0 tx=0x7f9ae80096d0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.174+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/989848645 shutdown_connections 2024-05-09T04:56:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.174+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/989848645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4dd480 0x5555ed4dd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.174+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/989848645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4411e0 0x5555ed4d8dd0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.174+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/989848645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed357520 0x5555ed4d9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.174+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/989848645 >> 172.21.15.46:0/989848645 conn(0x5555ed3b7720 msgr2=0x5555ed4cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.174+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/989848645 shutdown_connections 2024-05-09T04:56:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.174+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/989848645 wait complete. 2024-05-09T04:56:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.174+0000 7f9b0c0fbb80 1 -- start start 2024-05-09T04:56:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.175+0000 7f9b0c0fbb80 1 Processor -- start 2024-05-09T04:56:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.175+0000 7f9b0c0fbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed357520 0x5555ed4e25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.175+0000 7f9b0c0fbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4411e0 0x5555ed4e2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.175+0000 7f9b0c0fbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4dd480 0x5555ed4e3070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.180 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.175+0000 7f9b0c0fbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5555ed272c20 con 0x5555ed4411e0 2024-05-09T04:56:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.175+0000 7f9b0c0fbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5555ed3b11d0 con 0x5555ed4dd480 2024-05-09T04:56:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.175+0000 7f9b0c0fbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5555ed4de230 con 0x5555ed357520 2024-05-09T04:56:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.175+0000 7f9b0c0fbb80 10 client.0 _ll_register_callbacks cb 0x5555ed268740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.175+0000 7f9af67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4411e0 0x5555ed4e2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.175+0000 7f9af67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4411e0 0x5555ed4e2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57808/0 (socket says 172.21.15.46:57808) 2024-05-09T04:56:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.175+0000 7f9af67fc700 1 -- 172.21.15.46:0/38404583 learned_addr learned my addr 172.21.15.46:0/38404583 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.175+0000 7f9af77fe700 1 --2- 172.21.15.46:0/38404583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4dd480 0x5555ed4e3070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.175+0000 7f9af6ffd700 1 --2- 172.21.15.46:0/38404583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed357520 0x5555ed4e25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.175+0000 7f9af67fc700 1 -- 172.21.15.46:0/38404583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed357520 msgr2=0x5555ed4e25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.175+0000 7f9af67fc700 1 --2- 172.21.15.46:0/38404583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed357520 0x5555ed4e25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.176+0000 7f9af67fc700 1 -- 172.21.15.46:0/38404583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4dd480 msgr2=0x5555ed4e3070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.176+0000 7f9af67fc700 1 --2- 172.21.15.46:0/38404583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4dd480 0x5555ed4e3070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.176+0000 7f9af67fc700 1 -- 172.21.15.46:0/38404583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ed272510 con 0x5555ed4411e0 2024-05-09T04:56:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.176+0000 7f9af6ffd700 1 --2- 172.21.15.46:0/38404583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed357520 0x5555ed4e25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:56:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.176+0000 7f9af67fc700 1 --2- 172.21.15.46:0/38404583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4411e0 0x5555ed4e2b30 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000bfd0 tx=0x7f9ae00a98e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf2aa67215844ae server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.176+0000 7f9ade7fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:56:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.176+0000 7f9ade7fc700 1 -- 172.21.15.46:0/38404583 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0020040 con 0x5555ed4411e0 2024-05-09T04:56:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.176+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/38404583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ae8019070 con 0x5555ed4411e0 2024-05-09T04:56:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.176+0000 7f9ade7fc700 1 -- 172.21.15.46:0/38404583 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f9ae000faa0 con 0x5555ed4411e0 2024-05-09T04:56:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.176+0000 7f9ade7fc700 1 -- 172.21.15.46:0/38404583 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9ae0010610 con 0x5555ed4411e0 2024-05-09T04:56:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.176+0000 7f9b0c0fbb80 10 client.16635 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:17.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.176+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/38404583 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555ed4effc0 con 0x5555ed4411e0 2024-05-09T04:56:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.176+0000 7f9ade7fc700 1 -- 172.21.15.46:0/38404583 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae8019070 con 0x5555ed4411e0 2024-05-09T04:56:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.177+0000 7f9ade7fc700 1 -- 172.21.15.46:0/38404583 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9ae00a4d10 con 0x5555ed4411e0 2024-05-09T04:56:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.177+0000 7f9b0c0fbb80 10 client.16635 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.177+0000 7f9b0c0fbb80 10 client.16635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:56:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.177+0000 7f9b0c0fbb80 20 client.16635 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.177+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/38404583 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5555ed4f2600 0x5555ed4f4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.177+0000 7f9b0c0fbb80 4 client.16635 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:56:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.177+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/38404583 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555ed404a50 con 0x5555ed4f2600 2024-05-09T04:56:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.177+0000 7f9af6ffd700 1 --2- 172.21.15.46:0/38404583 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5555ed4f2600 0x5555ed4f4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.177+0000 7f9af6ffd700 1 --2- 172.21.15.46:0/38404583 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5555ed4f2600 0x5555ed4f4a80 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=399f89d14ba4c977 server_cookie=20644aa52f30a738 in_seq=0 out_seq=0 2024-05-09T04:56:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.177+0000 7f9ade7fc700 10 client.16635 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.178+0000 7f9ade7fc700 1 -- 172.21.15.46:0/38404583 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5555ed404a50 con 0x5555ed4f2600 2024-05-09T04:56:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.178+0000 7f9ade7fc700 10 client.16635 handle_command_reply: tid=0 2024-05-09T04:56:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.178+0000 7f9b0c0fbb80 1 client.16635 shutdown 2024-05-09T04:56:17.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.178+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/38404583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4411e0 msgr2=0x5555ed4e2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:17.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.178+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/38404583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4411e0 0x5555ed4e2b30 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000bfd0 tx=0x7f9ae00a98e0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.178+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/38404583 shutdown_connections 2024-05-09T04:56:17.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.178+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/38404583 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5555ed4f2600 0x5555ed4f4a80 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.178+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/38404583 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4dd480 0x5555ed4e3070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.178+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/38404583 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed357520 0x5555ed4e25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.178+0000 7f9ade7fc700 0 client.16635 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:17.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.178+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/38404583 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4411e0 0x5555ed4e2b30 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.178+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/38404583 >> 172.21.15.46:0/38404583 conn(0x5555ed3b7720 msgr2=0x5555ed4cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:17.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.178+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/38404583 shutdown_connections 2024-05-09T04:56:17.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.178+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/38404583 wait complete. 2024-05-09T04:56:17.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.178+0000 7f9b0c0fbb80 20 client.16635 trim_cache size 0 max 16384 2024-05-09T04:56:17.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.200+0000 7f9b0c0fbb80 1 Processor -- start 2024-05-09T04:56:17.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.200+0000 7f9b0c0fbb80 1 -- start start 2024-05-09T04:56:17.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.200+0000 7f9b0c0fbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed357520 0x5555ed4e2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.200+0000 7f9b0c0fbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4411e0 0x5555ed4d3e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.200+0000 7f9b0c0fbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed4dd480 0x5555ed4ee480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.200+0000 7f9b0c0fbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81af5d0 con 0x5555ed4411e0 2024-05-09T04:56:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.200+0000 7f9b0c0fbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81ace60 con 0x5555ed357520 2024-05-09T04:56:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.200+0000 7f9b0c0fbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81aacb0 con 0x5555ed4dd480 2024-05-09T04:56:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.200+0000 7f9af67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4411e0 0x5555ed4d3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.200+0000 7f9af67fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4411e0 0x5555ed4d3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57820/0 (socket says 172.21.15.46:57820) 2024-05-09T04:56:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.200+0000 7f9af67fc700 1 -- 172.21.15.46:0/1049285657 learned_addr learned my addr 172.21.15.46:0/1049285657 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.200+0000 7f9af77fe700 1 --2- 172.21.15.46:0/1049285657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed4dd480 0x5555ed4ee480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.200+0000 7f9af6ffd700 1 --2- 172.21.15.46:0/1049285657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed357520 0x5555ed4e2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.201+0000 7f9af67fc700 1 -- 172.21.15.46:0/1049285657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed4dd480 msgr2=0x5555ed4ee480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.201+0000 7f9af67fc700 1 --2- 172.21.15.46:0/1049285657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed4dd480 0x5555ed4ee480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.201+0000 7f9af67fc700 1 -- 172.21.15.46:0/1049285657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed357520 msgr2=0x5555ed4e2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.201+0000 7f9af67fc700 1 --2- 172.21.15.46:0/1049285657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed357520 0x5555ed4e2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.202 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.201+0000 7f9af67fc700 1 -- 172.21.15.46:0/1049285657 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555ed4e0b80 con 0x5555ed4411e0 2024-05-09T04:56:17.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.201+0000 7f9af77fe700 1 --2- 172.21.15.46:0/1049285657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed4dd480 0x5555ed4ee480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:56:17.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.201+0000 7f9af67fc700 1 --2- 172.21.15.46:0/1049285657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4411e0 0x5555ed4d3e20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00107b0 tx=0x7f9ae00109b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdbbfce82ac53695 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:17.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.201+0000 7f9aecff9700 1 -- 172.21.15.46:0/1049285657 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae0020070 con 0x5555ed4411e0 2024-05-09T04:56:17.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.201+0000 7f9aecff9700 1 -- 172.21.15.46:0/1049285657 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae00a8eb0 con 0x5555ed4411e0 2024-05-09T04:56:17.203 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.201+0000 7f9aecff9700 1 -- 172.21.15.46:0/1049285657 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae00b0ad0 con 0x5555ed4411e0 2024-05-09T04:56:17.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.202+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/1049285657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4411e0 msgr2=0x5555ed4d3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:17.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.202+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/1049285657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4411e0 0x5555ed4d3e20 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f9ae00107b0 tx=0x7f9ae00109b0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.202+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/1049285657 shutdown_connections 2024-05-09T04:56:17.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.202+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/1049285657 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed4dd480 0x5555ed4ee480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.202+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/1049285657 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed357520 0x5555ed4e2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.202+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/1049285657 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed4411e0 0x5555ed4d3e20 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.202+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/1049285657 >> 172.21.15.46:0/1049285657 conn(0x5555ed3b7720 msgr2=0x5555ed4d3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:17.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.202+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/1049285657 shutdown_connections 2024-05-09T04:56:17.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.202+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/1049285657 wait complete. 2024-05-09T04:56:17.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.202+0000 7f9b0c0fbb80 1 -- start start 2024-05-09T04:56:17.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.202+0000 7f9b0c0fbb80 1 Processor -- start 2024-05-09T04:56:17.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9b0c0fbb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed357520 0x5555ed4eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9b0c0fbb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4411e0 0x5555ed4da2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9b0c0fbb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed4dd480 0x5555ed4da7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9b0c0fbb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81ab320 con 0x5555ed357520 2024-05-09T04:56:17.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9b0c0fbb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81ac790 con 0x5555ed4411e0 2024-05-09T04:56:17.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9b0c0fbb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9af81ab9f0 con 0x5555ed4dd480 2024-05-09T04:56:17.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9b0c0fbb80 10 client.0 _ll_register_callbacks cb 0x5555ed268740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:17.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9af6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed357520 0x5555ed4eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9af6ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed357520 0x5555ed4eaa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57822/0 (socket says 172.21.15.46:57822) 2024-05-09T04:56:17.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9af6ffd700 1 -- 172.21.15.46:0/3003213350 learned_addr learned my addr 172.21.15.46:0/3003213350 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:17.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9af67fc700 1 --2- 172.21.15.46:0/3003213350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4411e0 0x5555ed4da2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9af77fe700 1 --2- 172.21.15.46:0/3003213350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed4dd480 0x5555ed4da7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9af6ffd700 1 -- 172.21.15.46:0/3003213350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed4dd480 msgr2=0x5555ed4da7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:17.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9af6ffd700 1 --2- 172.21.15.46:0/3003213350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed4dd480 0x5555ed4da7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9af6ffd700 1 -- 172.21.15.46:0/3003213350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4411e0 msgr2=0x5555ed4da2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:17.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9af6ffd700 1 --2- 172.21.15.46:0/3003213350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4411e0 0x5555ed4da2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9af6ffd700 1 -- 172.21.15.46:0/3003213350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555ed4eb260 con 0x5555ed357520 2024-05-09T04:56:17.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9af67fc700 1 --2- 172.21.15.46:0/3003213350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4411e0 0x5555ed4da2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:56:17.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9af6ffd700 1 --2- 172.21.15.46:0/3003213350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed357520 0x5555ed4eaa00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800a4b0 tx=0x7f9ae8013610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e54ab07cffeab877 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:17.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.203+0000 7f9ade7fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:56:17.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.204+0000 7f9ade7fc700 1 -- 172.21.15.46:0/3003213350 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ae802c060 con 0x5555ed357520 2024-05-09T04:56:17.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.204+0000 7f9ade7fc700 1 -- 172.21.15.46:0/3003213350 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f9ae8013c90 con 0x5555ed357520 2024-05-09T04:56:17.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.204+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/3003213350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ae00a8eb0 con 0x5555ed357520 2024-05-09T04:56:17.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.204+0000 7f9ade7fc700 1 -- 172.21.15.46:0/3003213350 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9ae8013e90 con 0x5555ed357520 2024-05-09T04:56:17.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.204+0000 7f9b0c0fbb80 10 client.16641 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:17.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.204+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/3003213350 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5555ed4dbdc0 con 0x5555ed357520 2024-05-09T04:56:17.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.204+0000 7f9ade7fc700 1 -- 172.21.15.46:0/3003213350 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ae00a8eb0 con 0x5555ed357520 2024-05-09T04:56:17.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.204+0000 7f9ade7fc700 1 -- 172.21.15.46:0/3003213350 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9ae80147c0 con 0x5555ed357520 2024-05-09T04:56:17.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.204+0000 7f9b0c0fbb80 10 client.16641 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:17.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.204+0000 7f9b0c0fbb80 10 client.16641 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:56:17.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.204+0000 7f9b0c0fbb80 20 client.16641 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:17.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.204+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/3003213350 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5555ed55de50 0x5555ed560270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:17.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.204+0000 7f9b0c0fbb80 4 client.16641 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:56:17.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.204+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/3003213350 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5555ed3feaf0 con 0x5555ed55de50 2024-05-09T04:56:17.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.205+0000 7f9af67fc700 1 --2- 172.21.15.46:0/3003213350 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5555ed55de50 0x5555ed560270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:17.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.205+0000 7f9ade7fc700 10 client.16641 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:17.209 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.205+0000 7f9af67fc700 1 --2- 172.21.15.46:0/3003213350 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5555ed55de50 0x5555ed560270 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96d98eecef08fdc9 server_cookie=c4e81d2bb208c9f in_seq=0 out_seq=0 2024-05-09T04:56:17.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.206+0000 7f9ade7fc700 1 -- 172.21.15.46:0/3003213350 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5555ed3feaf0 con 0x5555ed55de50 2024-05-09T04:56:17.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.206+0000 7f9ade7fc700 10 client.16641 handle_command_reply: tid=0 2024-05-09T04:56:17.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.206+0000 7f9b0c0fbb80 1 client.16641 shutdown 2024-05-09T04:56:17.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.206+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/3003213350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed357520 msgr2=0x5555ed4eaa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:17.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.206+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/3003213350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed357520 0x5555ed4eaa00 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f9ae800a4b0 tx=0x7f9ae8013610 comp rx=0 tx=0).stop 2024-05-09T04:56:17.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.206+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/3003213350 shutdown_connections 2024-05-09T04:56:17.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.206+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/3003213350 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5555ed4dd480 0x5555ed4da7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.206+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/3003213350 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5555ed357520 0x5555ed4eaa00 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.206+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/3003213350 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5555ed55de50 0x5555ed560270 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.206+0000 7f9b0c0fbb80 1 --2- 172.21.15.46:0/3003213350 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5555ed4411e0 0x5555ed4da2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.206+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/3003213350 >> 172.21.15.46:0/3003213350 conn(0x5555ed3b7720 msgr2=0x5555ed4d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:17.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.207+0000 7f9ade7fc700 0 client.16641 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:17.211 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.207+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/3003213350 shutdown_connections 2024-05-09T04:56:17.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.207+0000 7f9b0c0fbb80 1 -- 172.21.15.46:0/3003213350 wait complete. 2024-05-09T04:56:17.212 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.207+0000 7f9b0c0fbb80 20 client.16641 trim_cache size 0 max 16384 2024-05-09T04:56:17.212 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:56:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:56:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "7740c373-9930-47d8-93da-881a7564632f", 2024-05-09T04:56:17.212 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:56:17.212 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:56:17.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.207+0000 7f9afd6e5700 1 -- 172.21.15.46:0/1432342285 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9ae40b29f0 msgr2=0x7f9ae40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:17.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.207+0000 7f9afd6e5700 1 --2- 172.21.15.46:0/1432342285 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9ae40b29f0 0x7f9ae40b4f00 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9af0003800 tx=0x7f9af00096a0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.207+0000 7f9afd6e5700 1 -- 172.21.15.46:0/1432342285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af8108f00 msgr2=0x7f9af819b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:17.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.207+0000 7f9afd6e5700 1 --2- 172.21.15.46:0/1432342285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af8108f00 0x7f9af819b3b0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f9ae000d980 tx=0x7f9ae000dc90 comp rx=0 tx=0).stop 2024-05-09T04:56:17.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.207+0000 7f9afd6e5700 1 -- 172.21.15.46:0/1432342285 shutdown_connections 2024-05-09T04:56:17.213 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.207+0000 7f9afd6e5700 1 --2- 172.21.15.46:0/1432342285 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9ae40b29f0 0x7f9ae40b4f00 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.207+0000 7f9afd6e5700 1 --2- 172.21.15.46:0/1432342285 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9af8109850 0x7f9af819f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.208+0000 7f9afd6e5700 1 --2- 172.21.15.46:0/1432342285 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9af8102ae0 0x7f9af819ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.208+0000 7f9afd6e5700 1 --2- 172.21.15.46:0/1432342285 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9af8108f00 0x7f9af819b3b0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:17.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.208+0000 7f9afd6e5700 1 -- 172.21.15.46:0/1432342285 >> 172.21.15.46:0/1432342285 conn(0x7f9af80fe4e0 msgr2=0x7f9af8108620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:17.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.208+0000 7f9afd6e5700 1 -- 172.21.15.46:0/1432342285 shutdown_connections 2024-05-09T04:56:17.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:17.208+0000 7f9afd6e5700 1 -- 172.21.15.46:0/1432342285 wait complete. 2024-05-09T04:56:18.038 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:56:18.293 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:18 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:56:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:18 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:56:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:18 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:56:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:18 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:56:19.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:18 smithi110 ceph-mon[34904]: pgmap v159: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-09T04:56:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:18 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:56:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:18 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:56:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:18 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:56:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:18 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:56:19.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:18 smithi046 ceph-mon[30136]: pgmap v159: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-09T04:56:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:18 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:56:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:18 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:56:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:18 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:56:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:18 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:56:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:18 smithi072 ceph-mon[34088]: pgmap v159: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 7 op/s 2024-05-09T04:56:20.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.180+0000 7ff2453a4700 1 -- 172.21.15.46:0/1428138692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2401034e0 msgr2=0x7ff2401038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.180+0000 7ff2453a4700 1 --2- 172.21.15.46:0/1428138692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2401034e0 0x7ff2401038f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff2300099c0 tx=0x7ff230009cd0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.180+0000 7ff2453a4700 1 -- 172.21.15.46:0/1428138692 shutdown_connections 2024-05-09T04:56:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.180+0000 7ff2453a4700 1 --2- 172.21.15.46:0/1428138692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff240108760 0x7ff2401100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.180+0000 7ff2453a4700 1 --2- 172.21.15.46:0/1428138692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff2401034e0 0x7ff2401038f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.180+0000 7ff2453a4700 1 --2- 172.21.15.46:0/1428138692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff240107e10 0x7ff240108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.180+0000 7ff2453a4700 1 -- 172.21.15.46:0/1428138692 >> 172.21.15.46:0/1428138692 conn(0x7ff2400fef30 msgr2=0x7ff240101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.180+0000 7ff2453a4700 1 -- 172.21.15.46:0/1428138692 shutdown_connections 2024-05-09T04:56:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.180+0000 7ff2453a4700 1 -- 172.21.15.46:0/1428138692 wait complete. 2024-05-09T04:56:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.181+0000 7ff2453a4700 1 Processor -- start 2024-05-09T04:56:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.181+0000 7ff2453a4700 1 -- start start 2024-05-09T04:56:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.181+0000 7ff2453a4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2401034e0 0x7ff24019aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.181+0000 7ff2453a4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff240107e10 0x7ff24019b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.181+0000 7ff2453a4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff240108760 0x7ff24019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.181+0000 7ff2453a4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff2401133b0 con 0x7ff240108760 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.181+0000 7ff2453a4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff240113230 con 0x7ff240107e10 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.181+0000 7ff23f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff240108760 0x7ff24019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.181+0000 7ff2453a4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff240113530 con 0x7ff2401034e0 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.181+0000 7ff23f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff240108760 0x7ff24019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57834/0 (socket says 172.21.15.46:57834) 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.181+0000 7ff23f7fe700 1 -- 172.21.15.46:0/2104463331 learned_addr learned my addr 172.21.15.46:0/2104463331 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.182+0000 7ff23e7fc700 1 --2- 172.21.15.46:0/2104463331 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff240107e10 0x7ff24019b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.182+0000 7ff23effd700 1 --2- 172.21.15.46:0/2104463331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2401034e0 0x7ff24019aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.182+0000 7ff23f7fe700 1 -- 172.21.15.46:0/2104463331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2401034e0 msgr2=0x7ff24019aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.182+0000 7ff23f7fe700 1 --2- 172.21.15.46:0/2104463331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2401034e0 0x7ff24019aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.182+0000 7ff23f7fe700 1 -- 172.21.15.46:0/2104463331 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff240107e10 msgr2=0x7ff24019b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.182+0000 7ff23f7fe700 1 --2- 172.21.15.46:0/2104463331 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff240107e10 0x7ff24019b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.182+0000 7ff23f7fe700 1 -- 172.21.15.46:0/2104463331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff24019fdc0 con 0x7ff240108760 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.182+0000 7ff23f7fe700 1 --2- 172.21.15.46:0/2104463331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff240108760 0x7ff24019f720 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff238007ee0 tx=0x7ff23800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.182+0000 7ff237fff700 1 -- 172.21.15.46:0/2104463331 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238004bb0 con 0x7ff240108760 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.182+0000 7ff237fff700 1 -- 172.21.15.46:0/2104463331 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff238004d50 con 0x7ff240108760 2024-05-09T04:56:20.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.182+0000 7ff2453a4700 1 -- 172.21.15.46:0/2104463331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2401a00b0 con 0x7ff240108760 2024-05-09T04:56:20.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.182+0000 7ff237fff700 1 -- 172.21.15.46:0/2104463331 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff238005920 con 0x7ff240108760 2024-05-09T04:56:20.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.182+0000 7ff2453a4700 1 -- 172.21.15.46:0/2104463331 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2401a79d0 con 0x7ff240108760 2024-05-09T04:56:20.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.186+0000 7ff237fff700 1 -- 172.21.15.46:0/2104463331 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff23801a020 con 0x7ff240108760 2024-05-09T04:56:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.187+0000 7ff237fff700 1 --2- 172.21.15.46:0/2104463331 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff22c0b2a70 0x7ff22c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.187+0000 7ff23effd700 1 --2- 172.21.15.46:0/2104463331 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff22c0b2a70 0x7ff22c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.187+0000 7ff237fff700 1 -- 172.21.15.46:0/2104463331 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7ff2380e1c20 con 0x7ff240108760 2024-05-09T04:56:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.187+0000 7ff253dbab80 1 Processor -- start 2024-05-09T04:56:20.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.187+0000 7ff253dbab80 1 -- start start 2024-05-09T04:56:20.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.187+0000 7ff23effd700 1 --2- 172.21.15.46:0/2104463331 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff22c0b2a70 0x7ff22c0b4f80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff230009990 tx=0x7ff2300036e0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:20.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff253dbab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde868d00 0x560cde971240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff253dbab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde7ee4a0 0x560cde971780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff253dbab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde96b400 0x560cde96b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff253dbab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560cde88e0a0 con 0x560cde868d00 2024-05-09T04:56:20.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff253dbab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x560cde8339a0 con 0x560cde7ee4a0 2024-05-09T04:56:20.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff253dbab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560cde88a060 con 0x560cde96b400 2024-05-09T04:56:20.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff23effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde868d00 0x560cde971240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff23effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde868d00 0x560cde971240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57842/0 (socket says 172.21.15.46:57842) 2024-05-09T04:56:20.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff23effd700 1 -- 172.21.15.46:0/725384030 learned_addr learned my addr 172.21.15.46:0/725384030 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:20.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff23e7fc700 1 --2- 172.21.15.46:0/725384030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde7ee4a0 0x560cde971780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff23f7fe700 1 --2- 172.21.15.46:0/725384030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde96b400 0x560cde96b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff23effd700 1 -- 172.21.15.46:0/725384030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde96b400 msgr2=0x560cde96b810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:20.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff23effd700 1 --2- 172.21.15.46:0/725384030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde96b400 0x560cde96b810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff23effd700 1 -- 172.21.15.46:0/725384030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde7ee4a0 msgr2=0x560cde971780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:20.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff23effd700 1 --2- 172.21.15.46:0/725384030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde7ee4a0 0x560cde971780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff23effd700 1 -- 172.21.15.46:0/725384030 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cde824500 con 0x560cde868d00 2024-05-09T04:56:20.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.188+0000 7ff23effd700 1 --2- 172.21.15.46:0/725384030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde868d00 0x560cde971240 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff2300099c0 tx=0x7ff23001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1bd9f7fe9ac180 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:20.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.189+0000 7ff2357fa700 1 -- 172.21.15.46:0/725384030 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff230005430 con 0x560cde868d00 2024-05-09T04:56:20.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.189+0000 7ff2357fa700 1 -- 172.21.15.46:0/725384030 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2300055d0 con 0x560cde868d00 2024-05-09T04:56:20.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.189+0000 7ff2357fa700 1 -- 172.21.15.46:0/725384030 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23000f940 con 0x560cde868d00 2024-05-09T04:56:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.190+0000 7ff253dbab80 1 -- 172.21.15.46:0/725384030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde868d00 msgr2=0x560cde971240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.190+0000 7ff253dbab80 1 --2- 172.21.15.46:0/725384030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde868d00 0x560cde971240 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ff2300099c0 tx=0x7ff23001e000 comp rx=0 tx=0).stop 2024-05-09T04:56:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.190+0000 7ff253dbab80 1 -- 172.21.15.46:0/725384030 shutdown_connections 2024-05-09T04:56:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.190+0000 7ff253dbab80 1 --2- 172.21.15.46:0/725384030 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde96b400 0x560cde96b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.190+0000 7ff253dbab80 1 --2- 172.21.15.46:0/725384030 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde868d00 0x560cde971240 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.190+0000 7ff253dbab80 1 --2- 172.21.15.46:0/725384030 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde7ee4a0 0x560cde971780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.190+0000 7ff253dbab80 1 -- 172.21.15.46:0/725384030 >> 172.21.15.46:0/725384030 conn(0x560cde7f1f70 msgr2=0x560cde966220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.190+0000 7ff253dbab80 1 -- 172.21.15.46:0/725384030 shutdown_connections 2024-05-09T04:56:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.190+0000 7ff253dbab80 1 -- 172.21.15.46:0/725384030 wait complete. 2024-05-09T04:56:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.190+0000 7ff253dbab80 1 -- start start 2024-05-09T04:56:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.190+0000 7ff253dbab80 1 Processor -- start 2024-05-09T04:56:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff253dbab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde7ee4a0 0x560cde974ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.195 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff253dbab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde868d00 0x560cde975530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff253dbab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde96b400 0x560cde975a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff253dbab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x560cde96c0a0 con 0x560cde7ee4a0 2024-05-09T04:56:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff253dbab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x560cde828b20 con 0x560cde96b400 2024-05-09T04:56:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff253dbab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x560cde96c220 con 0x560cde868d00 2024-05-09T04:56:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff253dbab80 10 client.0 _ll_register_callbacks cb 0x560cde6ff6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff23effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde7ee4a0 0x560cde974ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff23effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde7ee4a0 0x560cde974ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57858/0 (socket says 172.21.15.46:57858) 2024-05-09T04:56:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff23effd700 1 -- 172.21.15.46:0/2908764094 learned_addr learned my addr 172.21.15.46:0/2908764094 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff23e7fc700 1 --2- 172.21.15.46:0/2908764094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde868d00 0x560cde975530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff23f7fe700 1 --2- 172.21.15.46:0/2908764094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde96b400 0x560cde975a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff23effd700 1 -- 172.21.15.46:0/2908764094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde868d00 msgr2=0x560cde975530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff23effd700 1 --2- 172.21.15.46:0/2908764094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde868d00 0x560cde975530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff23effd700 1 -- 172.21.15.46:0/2908764094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde96b400 msgr2=0x560cde975a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff23effd700 1 --2- 172.21.15.46:0/2908764094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde96b400 0x560cde975a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff23effd700 1 -- 172.21.15.46:0/2908764094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cde709490 con 0x560cde7ee4a0 2024-05-09T04:56:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff23f7fe700 1 --2- 172.21.15.46:0/2908764094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde96b400 0x560cde975a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:56:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff23effd700 1 --2- 172.21.15.46:0/2908764094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde7ee4a0 0x560cde974ff0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff230006010 tx=0x7ff23000ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1330ee1395f92179 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.191+0000 7ff2267fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:56:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.192+0000 7ff2267fc700 1 -- 172.21.15.46:0/2908764094 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23003c060 con 0x560cde7ee4a0 2024-05-09T04:56:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.192+0000 7ff2267fc700 1 -- 172.21.15.46:0/2908764094 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7ff23000b9a0 con 0x560cde7ee4a0 2024-05-09T04:56:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.192+0000 7ff253dbab80 1 -- 172.21.15.46:0/2908764094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2300055d0 con 0x560cde7ee4a0 2024-05-09T04:56:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.192+0000 7ff2267fc700 1 -- 172.21.15.46:0/2908764094 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff230005430 con 0x560cde7ee4a0 2024-05-09T04:56:20.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.192+0000 7ff253dbab80 10 client.16653 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.192+0000 7ff253dbab80 1 -- 172.21.15.46:0/2908764094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560cde982910 con 0x560cde7ee4a0 2024-05-09T04:56:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.192+0000 7ff2267fc700 1 -- 172.21.15.46:0/2908764094 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2300055d0 con 0x560cde7ee4a0 2024-05-09T04:56:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.192+0000 7ff2267fc700 1 -- 172.21.15.46:0/2908764094 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff230022070 con 0x560cde7ee4a0 2024-05-09T04:56:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.192+0000 7ff253dbab80 10 client.16653 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.192+0000 7ff253dbab80 10 client.16653 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:56:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.192+0000 7ff253dbab80 20 client.16653 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.192+0000 7ff253dbab80 1 --2- 172.21.15.46:0/2908764094 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x560cde984de0 0x560cde987200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.192+0000 7ff253dbab80 4 client.16653 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:56:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.192+0000 7ff253dbab80 1 -- 172.21.15.46:0/2908764094 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560cde89ba80 con 0x560cde984de0 2024-05-09T04:56:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.193+0000 7ff23e7fc700 1 --2- 172.21.15.46:0/2908764094 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x560cde984de0 0x560cde987200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.193+0000 7ff23e7fc700 1 --2- 172.21.15.46:0/2908764094 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x560cde984de0 0x560cde987200 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db621c051eb65b9d server_cookie=64e75a120df8eef3 in_seq=0 out_seq=0 2024-05-09T04:56:20.199 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.193+0000 7ff2267fc700 10 client.16653 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:20.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.194+0000 7ff2267fc700 1 -- 172.21.15.46:0/2908764094 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x560cde89ba80 con 0x560cde984de0 2024-05-09T04:56:20.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.194+0000 7ff2267fc700 10 client.16653 handle_command_reply: tid=0 2024-05-09T04:56:20.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.194+0000 7ff253dbab80 1 client.16653 shutdown 2024-05-09T04:56:20.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.194+0000 7ff253dbab80 1 -- 172.21.15.46:0/2908764094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde7ee4a0 msgr2=0x560cde974ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:20.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.194+0000 7ff253dbab80 1 --2- 172.21.15.46:0/2908764094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde7ee4a0 0x560cde974ff0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ff230006010 tx=0x7ff23000ff90 comp rx=0 tx=0).stop 2024-05-09T04:56:20.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.194+0000 7ff253dbab80 1 -- 172.21.15.46:0/2908764094 shutdown_connections 2024-05-09T04:56:20.200 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.194+0000 7ff253dbab80 1 --2- 172.21.15.46:0/2908764094 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x560cde984de0 0x560cde987200 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.194+0000 7ff2267fc700 0 client.16653 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:20.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.194+0000 7ff253dbab80 1 --2- 172.21.15.46:0/2908764094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde96b400 0x560cde975a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.194+0000 7ff253dbab80 1 --2- 172.21.15.46:0/2908764094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde7ee4a0 0x560cde974ff0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.194+0000 7ff253dbab80 1 --2- 172.21.15.46:0/2908764094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde868d00 0x560cde975530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.194+0000 7ff253dbab80 1 -- 172.21.15.46:0/2908764094 >> 172.21.15.46:0/2908764094 conn(0x560cde7f1f70 msgr2=0x560cde95fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:20.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.194+0000 7ff253dbab80 1 -- 172.21.15.46:0/2908764094 shutdown_connections 2024-05-09T04:56:20.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.194+0000 7ff253dbab80 1 -- 172.21.15.46:0/2908764094 wait complete. 2024-05-09T04:56:20.201 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.194+0000 7ff253dbab80 20 client.16653 trim_cache size 0 max 16384 2024-05-09T04:56:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.215+0000 7ff253dbab80 1 Processor -- start 2024-05-09T04:56:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.215+0000 7ff253dbab80 1 -- start start 2024-05-09T04:56:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.215+0000 7ff253dbab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde7ee4a0 0x560cde96ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.215+0000 7ff253dbab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde868d00 0x560cde9707a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.215+0000 7ff253dbab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde96b400 0x560cde981100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.215+0000 7ff253dbab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff2401b1750 con 0x560cde96b400 2024-05-09T04:56:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.215+0000 7ff253dbab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff2401aefe0 con 0x560cde868d00 2024-05-09T04:56:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.215+0000 7ff253dbab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff2401ace30 con 0x560cde7ee4a0 2024-05-09T04:56:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.216+0000 7ff23f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde96b400 0x560cde981100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.216+0000 7ff23f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde96b400 0x560cde981100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57868/0 (socket says 172.21.15.46:57868) 2024-05-09T04:56:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.216+0000 7ff23f7fe700 1 -- 172.21.15.46:0/219331276 learned_addr learned my addr 172.21.15.46:0/219331276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.216+0000 7ff23effd700 1 --2- 172.21.15.46:0/219331276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde7ee4a0 0x560cde96ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.216+0000 7ff23e7fc700 1 --2- 172.21.15.46:0/219331276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde868d00 0x560cde9707a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.216+0000 7ff23f7fe700 1 -- 172.21.15.46:0/219331276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde7ee4a0 msgr2=0x560cde96ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.216+0000 7ff23f7fe700 1 --2- 172.21.15.46:0/219331276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde7ee4a0 0x560cde96ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.216+0000 7ff23f7fe700 1 -- 172.21.15.46:0/219331276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde868d00 msgr2=0x560cde9707a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.216+0000 7ff23f7fe700 1 --2- 172.21.15.46:0/219331276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde868d00 0x560cde9707a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.216+0000 7ff23f7fe700 1 -- 172.21.15.46:0/219331276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560cde9629d0 con 0x560cde96b400 2024-05-09T04:56:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.216+0000 7ff23f7fe700 1 --2- 172.21.15.46:0/219331276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde96b400 0x560cde981100 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff2380df350 tx=0x7ff2380a8d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff67d68f76d34c19 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.216+0000 7ff234ff9700 1 -- 172.21.15.46:0/219331276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2380a8990 con 0x560cde96b400 2024-05-09T04:56:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.216+0000 7ff234ff9700 1 -- 172.21.15.46:0/219331276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2380a9580 con 0x560cde96b400 2024-05-09T04:56:20.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.216+0000 7ff234ff9700 1 -- 172.21.15.46:0/219331276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2380b1850 con 0x560cde96b400 2024-05-09T04:56:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.217+0000 7ff253dbab80 1 -- 172.21.15.46:0/219331276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde96b400 msgr2=0x560cde981100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.217+0000 7ff253dbab80 1 --2- 172.21.15.46:0/219331276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde96b400 0x560cde981100 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ff2380df350 tx=0x7ff2380a8d60 comp rx=0 tx=0).stop 2024-05-09T04:56:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.217+0000 7ff253dbab80 1 -- 172.21.15.46:0/219331276 shutdown_connections 2024-05-09T04:56:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.217+0000 7ff253dbab80 1 --2- 172.21.15.46:0/219331276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde96b400 0x560cde981100 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.217+0000 7ff253dbab80 1 --2- 172.21.15.46:0/219331276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde7ee4a0 0x560cde96ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.217+0000 7ff253dbab80 1 --2- 172.21.15.46:0/219331276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde868d00 0x560cde9707a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.217+0000 7ff253dbab80 1 -- 172.21.15.46:0/219331276 >> 172.21.15.46:0/219331276 conn(0x560cde7f1f70 msgr2=0x560cde960660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.217+0000 7ff253dbab80 1 -- 172.21.15.46:0/219331276 shutdown_connections 2024-05-09T04:56:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.217+0000 7ff253dbab80 1 -- 172.21.15.46:0/219331276 wait complete. 2024-05-09T04:56:20.223 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.218+0000 7ff253dbab80 1 -- start start 2024-05-09T04:56:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.218+0000 7ff253dbab80 1 Processor -- start 2024-05-09T04:56:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.218+0000 7ff253dbab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde7ee4a0 0x560cde988480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.218+0000 7ff253dbab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde868d00 0x560cde9889c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.218+0000 7ff253dbab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde96b400 0x560cde9ee330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.218+0000 7ff253dbab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff2401ad4a0 con 0x560cde7ee4a0 2024-05-09T04:56:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.218+0000 7ff253dbab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff2401ae910 con 0x560cde96b400 2024-05-09T04:56:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.218+0000 7ff253dbab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff2401adb70 con 0x560cde868d00 2024-05-09T04:56:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.218+0000 7ff253dbab80 10 client.0 _ll_register_callbacks cb 0x560cde6ff6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.218+0000 7ff23effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde7ee4a0 0x560cde988480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.218+0000 7ff23effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde7ee4a0 0x560cde988480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:57870/0 (socket says 172.21.15.46:57870) 2024-05-09T04:56:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.218+0000 7ff23effd700 1 -- 172.21.15.46:0/4138354630 learned_addr learned my addr 172.21.15.46:0/4138354630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.218+0000 7ff23e7fc700 1 --2- 172.21.15.46:0/4138354630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde868d00 0x560cde9889c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.218+0000 7ff23f7fe700 1 --2- 172.21.15.46:0/4138354630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde96b400 0x560cde9ee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.219+0000 7ff23effd700 1 -- 172.21.15.46:0/4138354630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde868d00 msgr2=0x560cde9889c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.219+0000 7ff23effd700 1 --2- 172.21.15.46:0/4138354630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde868d00 0x560cde9889c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.219+0000 7ff23effd700 1 -- 172.21.15.46:0/4138354630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde96b400 msgr2=0x560cde9ee330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.219+0000 7ff23effd700 1 --2- 172.21.15.46:0/4138354630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde96b400 0x560cde9ee330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.219+0000 7ff23effd700 1 -- 172.21.15.46:0/4138354630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560cde9ee870 con 0x560cde7ee4a0 2024-05-09T04:56:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.219+0000 7ff23effd700 1 --2- 172.21.15.46:0/4138354630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde7ee4a0 0x560cde988480 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff23000b600 tx=0x7ff23000f4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37d5a472771c689 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.219+0000 7ff2267fc700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:56:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.219+0000 7ff2267fc700 1 -- 172.21.15.46:0/4138354630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff23003c070 con 0x560cde7ee4a0 2024-05-09T04:56:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.219+0000 7ff253dbab80 1 -- 172.21.15.46:0/4138354630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff2380a9580 con 0x560cde7ee4a0 2024-05-09T04:56:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.219+0000 7ff2267fc700 1 -- 172.21.15.46:0/4138354630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7ff23000bbb0 con 0x560cde7ee4a0 2024-05-09T04:56:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.219+0000 7ff2267fc700 1 -- 172.21.15.46:0/4138354630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff23000bdb0 con 0x560cde7ee4a0 2024-05-09T04:56:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.219+0000 7ff253dbab80 10 client.16659 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.219+0000 7ff253dbab80 1 -- 172.21.15.46:0/4138354630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x560cde9ef9b0 con 0x560cde7ee4a0 2024-05-09T04:56:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.219+0000 7ff2267fc700 1 -- 172.21.15.46:0/4138354630 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2380a9580 con 0x560cde7ee4a0 2024-05-09T04:56:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.220+0000 7ff2267fc700 1 -- 172.21.15.46:0/4138354630 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff230022040 con 0x560cde7ee4a0 2024-05-09T04:56:20.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.220+0000 7ff253dbab80 10 client.16659 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:20.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.220+0000 7ff253dbab80 10 client.16659 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:56:20.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.220+0000 7ff253dbab80 20 client.16659 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:20.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.220+0000 7ff253dbab80 1 --2- 172.21.15.46:0/4138354630 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x560cde984de0 0x560cde9f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:20.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.220+0000 7ff253dbab80 4 client.16659 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:56:20.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.220+0000 7ff253dbab80 1 -- 172.21.15.46:0/4138354630 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560cde895b20 con 0x560cde984de0 2024-05-09T04:56:20.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.220+0000 7ff23e7fc700 1 --2- 172.21.15.46:0/4138354630 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x560cde984de0 0x560cde9f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:20.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.220+0000 7ff23e7fc700 1 --2- 172.21.15.46:0/4138354630 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x560cde984de0 0x560cde9f1dd0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2702b581a9de75f server_cookie=1986eff46dcbd603 in_seq=0 out_seq=0 2024-05-09T04:56:20.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff2267fc700 10 client.16659 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:20.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff2267fc700 1 -- 172.21.15.46:0/4138354630 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560cde895b20 con 0x560cde984de0 2024-05-09T04:56:20.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff2267fc700 10 client.16659 handle_command_reply: tid=0 2024-05-09T04:56:20.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff253dbab80 1 client.16659 shutdown 2024-05-09T04:56:20.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff253dbab80 1 -- 172.21.15.46:0/4138354630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde7ee4a0 msgr2=0x560cde988480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:20.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff253dbab80 1 --2- 172.21.15.46:0/4138354630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde7ee4a0 0x560cde988480 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ff23000b600 tx=0x7ff23000f4f0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff253dbab80 1 -- 172.21.15.46:0/4138354630 shutdown_connections 2024-05-09T04:56:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff253dbab80 1 --2- 172.21.15.46:0/4138354630 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x560cde984de0 0x560cde9f1dd0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff253dbab80 1 --2- 172.21.15.46:0/4138354630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x560cde96b400 0x560cde9ee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff253dbab80 1 --2- 172.21.15.46:0/4138354630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x560cde7ee4a0 0x560cde988480 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff253dbab80 1 --2- 172.21.15.46:0/4138354630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x560cde868d00 0x560cde9889c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff2267fc700 0 client.16659 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff253dbab80 1 -- 172.21.15.46:0/4138354630 >> 172.21.15.46:0/4138354630 conn(0x560cde7f1f70 msgr2=0x560cde973560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff253dbab80 1 -- 172.21.15.46:0/4138354630 shutdown_connections 2024-05-09T04:56:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff253dbab80 1 -- 172.21.15.46:0/4138354630 wait complete. 2024-05-09T04:56:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.221+0000 7ff253dbab80 20 client.16659 trim_cache size 0 max 16384 2024-05-09T04:56:20.228 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:56:20.228 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T04:56:20.228 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-09T04:56:20.228 INFO:teuthology.orchestra.run.smithi046.stdout: "7740c373-9930-47d8-93da-881a7564632f": { 2024-05-09T04:56:20.229 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-09T04:56:20.229 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "7740c373-9930-47d8-93da-881a7564632f", 2024-05-09T04:56:20.229 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-09T04:56:20.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T04:56:20.229 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T04:56:20.229 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:56:20.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.222+0000 7ff2453a4700 1 -- 172.21.15.46:0/2104463331 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff22c0b2a70 msgr2=0x7ff22c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:20.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.222+0000 7ff2453a4700 1 --2- 172.21.15.46:0/2104463331 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff22c0b2a70 0x7ff22c0b4f80 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ff230009990 tx=0x7ff2300036e0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.222+0000 7ff2453a4700 1 -- 172.21.15.46:0/2104463331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff240108760 msgr2=0x7ff24019f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:20.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.222+0000 7ff2453a4700 1 --2- 172.21.15.46:0/2104463331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff240108760 0x7ff24019f720 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff238007ee0 tx=0x7ff23800d3b0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.222+0000 7ff2453a4700 1 -- 172.21.15.46:0/2104463331 shutdown_connections 2024-05-09T04:56:20.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.222+0000 7ff2453a4700 1 --2- 172.21.15.46:0/2104463331 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff22c0b2a70 0x7ff22c0b4f80 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.222+0000 7ff2453a4700 1 --2- 172.21.15.46:0/2104463331 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff240108760 0x7ff24019f720 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.222+0000 7ff2453a4700 1 --2- 172.21.15.46:0/2104463331 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff2401034e0 0x7ff24019aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.222+0000 7ff2453a4700 1 --2- 172.21.15.46:0/2104463331 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff240107e10 0x7ff24019b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:20.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.222+0000 7ff2453a4700 1 -- 172.21.15.46:0/2104463331 >> 172.21.15.46:0/2104463331 conn(0x7ff2400fef30 msgr2=0x7ff2401096c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:20.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.222+0000 7ff2453a4700 1 -- 172.21.15.46:0/2104463331 shutdown_connections 2024-05-09T04:56:20.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:20.222+0000 7ff2453a4700 1 -- 172.21.15.46:0/2104463331 wait complete. 2024-05-09T04:56:21.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:20 smithi110 ceph-mon[34904]: pgmap v160: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T04:56:21.061 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:20 smithi046 ceph-mon[30136]: pgmap v160: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T04:56:21.092 INFO:tasks.cephfs.filesystem:scrub status for tag:7740c373-9930-47d8-93da-881a7564632f - {'path': '/', 'tag': '7740c373-9930-47d8-93da-881a7564632f', 'options': 'recursive,force'} 2024-05-09T04:56:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:20 smithi072 ceph-mon[34088]: pgmap v160: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2024-05-09T04:56:22.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:21 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:56:22.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:21 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:56:22.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:21 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:56:22.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:21 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:56:22.129 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:21 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:56:22.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:21 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:56:23.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:22 smithi110 ceph-mon[34904]: pgmap v161: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T04:56:23.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:22 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:56:23.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:22 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:56:23.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:22 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:56:23.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:22 smithi046 ceph-mon[30136]: pgmap v161: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T04:56:23.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:22 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:56:23.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:22 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:56:23.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:22 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:56:23.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:22 smithi072 ceph-mon[34088]: pgmap v161: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T04:56:23.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:22 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:56:23.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:22 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:56:23.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:22 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:56:25.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:24 smithi110 ceph-mon[34904]: pgmap v162: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T04:56:25.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:24 smithi046 ceph-mon[30136]: pgmap v162: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T04:56:25.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:24 smithi072 ceph-mon[34088]: pgmap v162: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T04:56:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:26 smithi110 ceph-mon[34904]: pgmap v163: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:56:27.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:26 smithi046 ceph-mon[30136]: pgmap v163: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:56:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:26 smithi072 ceph-mon[34088]: pgmap v163: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:56:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:28 smithi110 ceph-mon[34904]: pgmap v164: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T04:56:29.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:28 smithi046 ceph-mon[30136]: pgmap v164: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T04:56:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:28 smithi072 ceph-mon[34088]: pgmap v164: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T04:56:31.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:30 smithi110 ceph-mon[34904]: pgmap v165: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:56:31.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:30 smithi046 ceph-mon[30136]: pgmap v165: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:56:31.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:30 smithi072 ceph-mon[34088]: pgmap v165: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:56:33.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:32 smithi110 ceph-mon[34904]: pgmap v166: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T04:56:33.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:32 smithi046 ceph-mon[30136]: pgmap v166: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T04:56:33.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:32 smithi072 ceph-mon[34088]: pgmap v166: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T04:56:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:34 smithi110 ceph-mon[34904]: pgmap v167: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:56:35.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:34 smithi046 ceph-mon[30136]: pgmap v167: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:56:35.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:34 smithi072 ceph-mon[34088]: pgmap v167: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:56:37.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:36 smithi110 ceph-mon[34904]: pgmap v168: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:56:37.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:36 smithi046 ceph-mon[30136]: pgmap v168: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:56:37.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:36 smithi072 ceph-mon[34088]: pgmap v168: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:56:39.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:38 smithi110 ceph-mon[34904]: pgmap v169: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:56:39.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:38 smithi046 ceph-mon[30136]: pgmap v169: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:56:39.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:38 smithi072 ceph-mon[34088]: pgmap v169: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:56:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:40 smithi110 ceph-mon[34904]: pgmap v170: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:56:41.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:40 smithi046 ceph-mon[30136]: pgmap v170: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:56:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:40 smithi072 ceph-mon[34088]: pgmap v170: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:56:43.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:42 smithi110 ceph-mon[34904]: pgmap v171: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:56:43.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:42 smithi046 ceph-mon[30136]: pgmap v171: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:56:43.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:42 smithi072 ceph-mon[34088]: pgmap v171: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:56:45.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:44 smithi110 ceph-mon[34904]: pgmap v172: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:56:45.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:44 smithi046 ceph-mon[30136]: pgmap v172: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:56:45.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:44 smithi072 ceph-mon[34088]: pgmap v172: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:56:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:46 smithi110 ceph-mon[34904]: pgmap v173: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:56:47.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:46 smithi046 ceph-mon[30136]: pgmap v173: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:56:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:46 smithi072 ceph-mon[34088]: pgmap v173: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:56:49.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:48 smithi110 ceph-mon[34904]: pgmap v174: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:56:49.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:48 smithi046 ceph-mon[30136]: pgmap v174: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:56:49.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:48 smithi072 ceph-mon[34088]: pgmap v174: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:56:51.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:50 smithi110 ceph-mon[34904]: pgmap v175: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:56:51.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:50 smithi046 ceph-mon[30136]: pgmap v175: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:56:51.094 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:56:51.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:50 smithi072 ceph-mon[34088]: pgmap v175: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:56:51.359 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:53.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:52 smithi110 ceph-mon[34904]: pgmap v176: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:56:53.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.079+0000 7f397d925700 1 -- 172.21.15.46:0/2879920538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39781089f0 msgr2=0x7f3978108e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:53.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.079+0000 7f397d925700 1 --2- 172.21.15.46:0/2879920538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39781089f0 0x7f3978108e00 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f396800c210 tx=0x7f396800c520 comp rx=0 tx=0).stop 2024-05-09T04:56:53.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.079+0000 7f397d925700 1 -- 172.21.15.46:0/2879920538 shutdown_connections 2024-05-09T04:56:53.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.079+0000 7f397d925700 1 --2- 172.21.15.46:0/2879920538 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3978103310 0x7f397810f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.079+0000 7f397d925700 1 --2- 172.21.15.46:0/2879920538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39781089f0 0x7f3978108e00 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.079+0000 7f397d925700 1 --2- 172.21.15.46:0/2879920538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39781029c0 0x7f3978102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.079 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.079+0000 7f397d925700 1 -- 172.21.15.46:0/2879920538 >> 172.21.15.46:0/2879920538 conn(0x7f39780fe4e0 msgr2=0x7f3978100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:53.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.079+0000 7f397d925700 1 -- 172.21.15.46:0/2879920538 shutdown_connections 2024-05-09T04:56:53.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:52 smithi046 ceph-mon[30136]: pgmap v176: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:56:53.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.079+0000 7f397d925700 1 -- 172.21.15.46:0/2879920538 wait complete. 2024-05-09T04:56:53.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.080+0000 7f397d925700 1 Processor -- start 2024-05-09T04:56:53.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.080+0000 7f397d925700 1 -- start start 2024-05-09T04:56:53.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.080+0000 7f397d925700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39781029c0 0x7f3978104590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.081 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.080+0000 7f397d925700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3978103310 0x7f3978104ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.080+0000 7f397d925700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39781089f0 0x7f3978105010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.080+0000 7f397d925700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39781122e0 con 0x7f39781029c0 2024-05-09T04:56:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.080+0000 7f397d925700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3978112160 con 0x7f3978103310 2024-05-09T04:56:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.080+0000 7f397d925700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3978112460 con 0x7f39781089f0 2024-05-09T04:56:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.081+0000 7f3976ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39781029c0 0x7f3978104590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.081+0000 7f3976ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39781029c0 0x7f3978104590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47122/0 (socket says 172.21.15.46:47122) 2024-05-09T04:56:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.081+0000 7f3976ffd700 1 -- 172.21.15.46:0/1074542390 learned_addr learned my addr 172.21.15.46:0/1074542390 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.081+0000 7f39767fc700 1 --2- 172.21.15.46:0/1074542390 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3978103310 0x7f3978104ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.081+0000 7f39777fe700 1 --2- 172.21.15.46:0/1074542390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39781089f0 0x7f3978105010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.081+0000 7f3976ffd700 1 -- 172.21.15.46:0/1074542390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39781089f0 msgr2=0x7f3978105010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.081+0000 7f3976ffd700 1 --2- 172.21.15.46:0/1074542390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39781089f0 0x7f3978105010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.081+0000 7f3976ffd700 1 -- 172.21.15.46:0/1074542390 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3978103310 msgr2=0x7f3978104ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:53.082 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.081+0000 7f3976ffd700 1 --2- 172.21.15.46:0/1074542390 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3978103310 0x7f3978104ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.081+0000 7f3976ffd700 1 -- 172.21.15.46:0/1074542390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39781a9220 con 0x7f39781029c0 2024-05-09T04:56:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.081+0000 7f3976ffd700 1 --2- 172.21.15.46:0/1074542390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39781029c0 0x7f3978104590 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f396800a6a0 tx=0x7f396800a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.082+0000 7f397c923700 1 -- 172.21.15.46:0/1074542390 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968022020 con 0x7f39781029c0 2024-05-09T04:56:53.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.082+0000 7f397c923700 1 -- 172.21.15.46:0/1074542390 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3968008c90 con 0x7f39781029c0 2024-05-09T04:56:53.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.082+0000 7f397d925700 1 -- 172.21.15.46:0/1074542390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39781a94b0 con 0x7f39781029c0 2024-05-09T04:56:53.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.083+0000 7f397c923700 1 -- 172.21.15.46:0/1074542390 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39680052a0 con 0x7f39781029c0 2024-05-09T04:56:53.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.083+0000 7f397d925700 1 -- 172.21.15.46:0/1074542390 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39781a98a0 con 0x7f39781029c0 2024-05-09T04:56:53.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.085+0000 7f397c923700 1 -- 172.21.15.46:0/1074542390 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3968025020 con 0x7f39781029c0 2024-05-09T04:56:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.086+0000 7f397c923700 1 --2- 172.21.15.46:0/1074542390 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f39600b2b40 0x7f39600b5050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.086+0000 7f39767fc700 1 --2- 172.21.15.46:0/1074542390 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f39600b2b40 0x7f39600b5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.086+0000 7f397c923700 1 -- 172.21.15.46:0/1074542390 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f39680e3810 con 0x7f39781029c0 2024-05-09T04:56:53.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.086+0000 7f39767fc700 1 --2- 172.21.15.46:0/1074542390 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f39600b2b40 0x7f39600b5050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3978100610 tx=0x7f396c00b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:53.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.087+0000 7f398c33bb80 1 Processor -- start 2024-05-09T04:56:53.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.087+0000 7f398c33bb80 1 -- start start 2024-05-09T04:56:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.087+0000 7f398c33bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7651c3160 0x55c76525c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.087+0000 7f398c33bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7650d94a0 0x55c76525ca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.089 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.087+0000 7f398c33bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7652566a0 0x55c765256ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.087+0000 7f398c33bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c765178ff0 con 0x55c7651c3160 2024-05-09T04:56:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.087+0000 7f398c33bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c7651177f0 con 0x55c7650d94a0 2024-05-09T04:56:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.087+0000 7f398c33bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c765174fb0 con 0x55c7652566a0 2024-05-09T04:56:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.087+0000 7f3976ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7651c3160 0x55c76525c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.087+0000 7f3976ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7651c3160 0x55c76525c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47128/0 (socket says 172.21.15.46:47128) 2024-05-09T04:56:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.087+0000 7f3976ffd700 1 -- 172.21.15.46:0/480226251 learned_addr learned my addr 172.21.15.46:0/480226251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.087+0000 7f39767fc700 1 --2- 172.21.15.46:0/480226251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7650d94a0 0x55c76525ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.088+0000 7f39777fe700 1 --2- 172.21.15.46:0/480226251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7652566a0 0x55c765256ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.088+0000 7f3976ffd700 1 -- 172.21.15.46:0/480226251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7652566a0 msgr2=0x55c765256ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:53.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.088+0000 7f3976ffd700 1 --2- 172.21.15.46:0/480226251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7652566a0 0x55c765256ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.088+0000 7f3976ffd700 1 -- 172.21.15.46:0/480226251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7650d94a0 msgr2=0x55c76525ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.088+0000 7f3976ffd700 1 --2- 172.21.15.46:0/480226251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7650d94a0 0x55c76525ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.088+0000 7f3976ffd700 1 -- 172.21.15.46:0/480226251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c76510f370 con 0x55c7651c3160 2024-05-09T04:56:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.088+0000 7f39767fc700 1 --2- 172.21.15.46:0/480226251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7650d94a0 0x55c76525ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:56:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.088+0000 7f39777fe700 1 --2- 172.21.15.46:0/480226251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7652566a0 0x55c765256ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:56:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.088+0000 7f3976ffd700 1 --2- 172.21.15.46:0/480226251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7651c3160 0x55c76525c4f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f39680e0e80 tx=0x7f396800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a69856621c524e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.088+0000 7f395dffb700 1 -- 172.21.15.46:0/480226251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39680b3220 con 0x55c7651c3160 2024-05-09T04:56:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.088+0000 7f395dffb700 1 -- 172.21.15.46:0/480226251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39680aa610 con 0x55c7651c3160 2024-05-09T04:56:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.088+0000 7f395dffb700 1 -- 172.21.15.46:0/480226251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396800ae30 con 0x55c7651c3160 2024-05-09T04:56:53.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.089+0000 7f398c33bb80 1 -- 172.21.15.46:0/480226251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7651c3160 msgr2=0x55c76525c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:53.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.089+0000 7f398c33bb80 1 --2- 172.21.15.46:0/480226251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7651c3160 0x55c76525c4f0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f39680e0e80 tx=0x7f396800c520 comp rx=0 tx=0).stop 2024-05-09T04:56:53.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.089+0000 7f398c33bb80 1 -- 172.21.15.46:0/480226251 shutdown_connections 2024-05-09T04:56:53.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.089+0000 7f398c33bb80 1 --2- 172.21.15.46:0/480226251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7652566a0 0x55c765256ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.089+0000 7f398c33bb80 1 --2- 172.21.15.46:0/480226251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7651c3160 0x55c76525c4f0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.089+0000 7f398c33bb80 1 --2- 172.21.15.46:0/480226251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7650d94a0 0x55c76525ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.089+0000 7f398c33bb80 1 -- 172.21.15.46:0/480226251 >> 172.21.15.46:0/480226251 conn(0x55c7651396a0 msgr2=0x55c765251400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:53.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.089+0000 7f398c33bb80 1 -- 172.21.15.46:0/480226251 shutdown_connections 2024-05-09T04:56:53.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.089+0000 7f398c33bb80 1 -- 172.21.15.46:0/480226251 wait complete. 2024-05-09T04:56:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.090+0000 7f398c33bb80 1 -- start start 2024-05-09T04:56:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.090+0000 7f398c33bb80 1 Processor -- start 2024-05-09T04:56:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.090+0000 7f398c33bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7650d94a0 0x55c765265d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.090+0000 7f398c33bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7651c3160 0x55c765266290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.090+0000 7f398c33bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c765260150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.090+0000 7f398c33bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c764ff4ba0 con 0x55c7652566a0 2024-05-09T04:56:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.090+0000 7f398c33bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c765133150 con 0x55c7650d94a0 2024-05-09T04:56:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.090+0000 7f398c33bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c765257360 con 0x55c7651c3160 2024-05-09T04:56:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.090+0000 7f398c33bb80 10 client.0 _ll_register_callbacks cb 0x55c764fea6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.090+0000 7f39777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c765260150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.090+0000 7f39777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c765260150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47130/0 (socket says 172.21.15.46:47130) 2024-05-09T04:56:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.090+0000 7f39777fe700 1 -- 172.21.15.46:0/2867623578 learned_addr learned my addr 172.21.15.46:0/2867623578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.091+0000 7f39767fc700 1 --2- 172.21.15.46:0/2867623578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7651c3160 0x55c765266290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.091+0000 7f3976ffd700 1 --2- 172.21.15.46:0/2867623578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7650d94a0 0x55c765265d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.091+0000 7f39777fe700 1 -- 172.21.15.46:0/2867623578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7651c3160 msgr2=0x55c765266290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.091+0000 7f39777fe700 1 --2- 172.21.15.46:0/2867623578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7651c3160 0x55c765266290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.091+0000 7f39777fe700 1 -- 172.21.15.46:0/2867623578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7650d94a0 msgr2=0x55c765265d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.091+0000 7f39777fe700 1 --2- 172.21.15.46:0/2867623578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7650d94a0 0x55c765265d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.091+0000 7f39777fe700 1 -- 172.21.15.46:0/2867623578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c764ff4490 con 0x55c7652566a0 2024-05-09T04:56:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.091+0000 7f39777fe700 1 --2- 172.21.15.46:0/2867623578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c765260150 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f39700127d0 tx=0x7f3970012ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcc1facfd36ec587 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.091+0000 7f3956ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:56:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.091+0000 7f3956ffd700 1 -- 172.21.15.46:0/2867623578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397001f060 con 0x55c7652566a0 2024-05-09T04:56:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.091+0000 7f3956ffd700 1 -- 172.21.15.46:0/2867623578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f3970004460 con 0x55c7652566a0 2024-05-09T04:56:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.091+0000 7f398c33bb80 1 -- 172.21.15.46:0/2867623578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39680aa610 con 0x55c7652566a0 2024-05-09T04:56:53.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.091+0000 7f3956ffd700 1 -- 172.21.15.46:0/2867623578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3970003680 con 0x55c7652566a0 2024-05-09T04:56:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.091+0000 7f398c33bb80 10 client.16671 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.092+0000 7f398c33bb80 1 -- 172.21.15.46:0/2867623578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c76526d9d0 con 0x55c7652566a0 2024-05-09T04:56:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.092+0000 7f3956ffd700 1 -- 172.21.15.46:0/2867623578 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39680aa610 con 0x55c7652566a0 2024-05-09T04:56:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.092+0000 7f3956ffd700 1 -- 172.21.15.46:0/2867623578 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3970010b20 con 0x55c7652566a0 2024-05-09T04:56:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.092+0000 7f398c33bb80 10 client.16671 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.092+0000 7f398c33bb80 10 client.16671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:56:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.092+0000 7f398c33bb80 20 client.16671 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.092+0000 7f398c33bb80 1 --2- 172.21.15.46:0/2867623578 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c7652703e0 0x55c765272830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.092+0000 7f398c33bb80 4 client.16671 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:56:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.092+0000 7f398c33bb80 1 -- 172.21.15.46:0/2867623578 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7651869d0 con 0x55c7652703e0 2024-05-09T04:56:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.093+0000 7f3976ffd700 1 --2- 172.21.15.46:0/2867623578 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c7652703e0 0x55c765272830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.093+0000 7f3976ffd700 1 --2- 172.21.15.46:0/2867623578 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c7652703e0 0x55c765272830 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91246b5a17787670 server_cookie=806d8085286d432 in_seq=0 out_seq=0 2024-05-09T04:56:53.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.093+0000 7f3956ffd700 10 client.16671 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.094+0000 7f3956ffd700 1 -- 172.21.15.46:0/2867623578 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c7651869d0 con 0x55c7652703e0 2024-05-09T04:56:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.094+0000 7f3956ffd700 10 client.16671 handle_command_reply: tid=0 2024-05-09T04:56:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.094+0000 7f398c33bb80 1 client.16671 shutdown 2024-05-09T04:56:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.094+0000 7f398c33bb80 1 -- 172.21.15.46:0/2867623578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 msgr2=0x55c765260150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.094+0000 7f398c33bb80 1 --2- 172.21.15.46:0/2867623578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c765260150 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f39700127d0 tx=0x7f3970012ae0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.094+0000 7f398c33bb80 1 -- 172.21.15.46:0/2867623578 shutdown_connections 2024-05-09T04:56:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.094+0000 7f398c33bb80 1 --2- 172.21.15.46:0/2867623578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c765260150 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.094+0000 7f398c33bb80 1 --2- 172.21.15.46:0/2867623578 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c7652703e0 0x55c765272830 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.094+0000 7f398c33bb80 1 --2- 172.21.15.46:0/2867623578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7650d94a0 0x55c765265d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.094+0000 7f3956ffd700 0 client.16671 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.094+0000 7f398c33bb80 1 --2- 172.21.15.46:0/2867623578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7651c3160 0x55c765266290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.094+0000 7f398c33bb80 1 -- 172.21.15.46:0/2867623578 >> 172.21.15.46:0/2867623578 conn(0x55c7651396a0 msgr2=0x55c7652511c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.094+0000 7f398c33bb80 1 -- 172.21.15.46:0/2867623578 shutdown_connections 2024-05-09T04:56:53.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.094+0000 7f398c33bb80 1 -- 172.21.15.46:0/2867623578 wait complete. 2024-05-09T04:56:53.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.095+0000 7f398c33bb80 20 client.16671 trim_cache size 0 max 16384 2024-05-09T04:56:53.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:52 smithi072 ceph-mon[34088]: pgmap v176: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:56:53.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.137+0000 7f398c33bb80 1 Processor -- start 2024-05-09T04:56:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.137+0000 7f398c33bb80 1 -- start start 2024-05-09T04:56:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.137+0000 7f398c33bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7650d94a0 0x55c76526c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.137+0000 7f398c33bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7651c3160 0x55c76525b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.139 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.137+0000 7f398c33bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c765278010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.137+0000 7f398c33bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39781b38d0 con 0x55c7652566a0 2024-05-09T04:56:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.137+0000 7f398c33bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f39781b1160 con 0x55c7651c3160 2024-05-09T04:56:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.137+0000 7f398c33bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f39781aefb0 con 0x55c7650d94a0 2024-05-09T04:56:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.137+0000 7f39777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c765278010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.137+0000 7f39777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c765278010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47146/0 (socket says 172.21.15.46:47146) 2024-05-09T04:56:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.138+0000 7f39777fe700 1 -- 172.21.15.46:0/3271746503 learned_addr learned my addr 172.21.15.46:0/3271746503 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:53.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.138+0000 7f3976ffd700 1 --2- 172.21.15.46:0/3271746503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7650d94a0 0x55c76526c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.138+0000 7f39767fc700 1 --2- 172.21.15.46:0/3271746503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7651c3160 0x55c76525b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.138+0000 7f39777fe700 1 -- 172.21.15.46:0/3271746503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7650d94a0 msgr2=0x55c76526c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.138+0000 7f39777fe700 1 --2- 172.21.15.46:0/3271746503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7650d94a0 0x55c76526c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.138+0000 7f39777fe700 1 -- 172.21.15.46:0/3271746503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7651c3160 msgr2=0x55c76525b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.138+0000 7f39777fe700 1 --2- 172.21.15.46:0/3271746503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7651c3160 0x55c76525b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.138+0000 7f39777fe700 1 -- 172.21.15.46:0/3271746503 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7652523f0 con 0x55c7652566a0 2024-05-09T04:56:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.138+0000 7f39777fe700 1 --2- 172.21.15.46:0/3271746503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c765278010 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3970000c00 tx=0x7f39700227b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197ebdc739c04df9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.138+0000 7f395d7fa700 1 -- 172.21.15.46:0/3271746503 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970030070 con 0x55c7652566a0 2024-05-09T04:56:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.138+0000 7f395d7fa700 1 -- 172.21.15.46:0/3271746503 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39700228a0 con 0x55c7652566a0 2024-05-09T04:56:53.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.138+0000 7f395d7fa700 1 -- 172.21.15.46:0/3271746503 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3970003ce0 con 0x55c7652566a0 2024-05-09T04:56:53.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.139+0000 7f398c33bb80 1 -- 172.21.15.46:0/3271746503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 msgr2=0x55c765278010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:53.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.139+0000 7f398c33bb80 1 --2- 172.21.15.46:0/3271746503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c765278010 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f3970000c00 tx=0x7f39700227b0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.139+0000 7f398c33bb80 1 -- 172.21.15.46:0/3271746503 shutdown_connections 2024-05-09T04:56:53.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.139+0000 7f398c33bb80 1 --2- 172.21.15.46:0/3271746503 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c765278010 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.139+0000 7f398c33bb80 1 --2- 172.21.15.46:0/3271746503 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7650d94a0 0x55c76526c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.139+0000 7f398c33bb80 1 --2- 172.21.15.46:0/3271746503 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7651c3160 0x55c76525b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.139+0000 7f398c33bb80 1 -- 172.21.15.46:0/3271746503 >> 172.21.15.46:0/3271746503 conn(0x55c7651396a0 msgr2=0x55c76525b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:53.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.139+0000 7f398c33bb80 1 -- 172.21.15.46:0/3271746503 shutdown_connections 2024-05-09T04:56:53.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.139+0000 7f398c33bb80 1 -- 172.21.15.46:0/3271746503 wait complete. 2024-05-09T04:56:53.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.140+0000 7f398c33bb80 1 -- start start 2024-05-09T04:56:53.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.140+0000 7f398c33bb80 1 Processor -- start 2024-05-09T04:56:53.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.140+0000 7f398c33bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7650d94a0 0x55c765269dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.140+0000 7f398c33bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7651c3160 0x55c76526a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.140+0000 7f398c33bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c76526a850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.140+0000 7f398c33bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f39781af620 con 0x55c7652566a0 2024-05-09T04:56:53.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.140+0000 7f398c33bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f39781b0a90 con 0x55c7651c3160 2024-05-09T04:56:53.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.140+0000 7f398c33bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f39781afcf0 con 0x55c7650d94a0 2024-05-09T04:56:53.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f398c33bb80 10 client.0 _ll_register_callbacks cb 0x55c764fea6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:53.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f39777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c76526a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f39777fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c76526a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47152/0 (socket says 172.21.15.46:47152) 2024-05-09T04:56:53.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f39777fe700 1 -- 172.21.15.46:0/13220581 learned_addr learned my addr 172.21.15.46:0/13220581 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:53.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f3976ffd700 1 --2- 172.21.15.46:0/13220581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7650d94a0 0x55c765269dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f39767fc700 1 --2- 172.21.15.46:0/13220581 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7651c3160 0x55c76526a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f39777fe700 1 -- 172.21.15.46:0/13220581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7650d94a0 msgr2=0x55c765269dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:53.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f39777fe700 1 --2- 172.21.15.46:0/13220581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7650d94a0 0x55c765269dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f39777fe700 1 -- 172.21.15.46:0/13220581 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7651c3160 msgr2=0x55c76526a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:53.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f39777fe700 1 --2- 172.21.15.46:0/13220581 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7651c3160 0x55c76526a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f39777fe700 1 -- 172.21.15.46:0/13220581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7652d9c50 con 0x55c7652566a0 2024-05-09T04:56:53.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f39777fe700 1 --2- 172.21.15.46:0/13220581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c76526a850 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f39700223f0 tx=0x7f3970018f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e5ab4529098cfe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:53.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f3956ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:56:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f3956ffd700 1 -- 172.21.15.46:0/13220581 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f397001f050 con 0x55c7652566a0 2024-05-09T04:56:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f3956ffd700 1 -- 172.21.15.46:0/13220581 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f3970022e10 con 0x55c7652566a0 2024-05-09T04:56:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.141+0000 7f398c33bb80 1 -- 172.21.15.46:0/13220581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39700228a0 con 0x55c7652566a0 2024-05-09T04:56:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.142+0000 7f3956ffd700 1 -- 172.21.15.46:0/13220581 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3970041050 con 0x55c7652566a0 2024-05-09T04:56:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.142+0000 7f398c33bb80 10 client.16677 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.142+0000 7f398c33bb80 1 -- 172.21.15.46:0/13220581 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c7652daec0 con 0x55c7652566a0 2024-05-09T04:56:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.142+0000 7f3956ffd700 1 -- 172.21.15.46:0/13220581 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39700228a0 con 0x55c7652566a0 2024-05-09T04:56:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.142+0000 7f3956ffd700 1 -- 172.21.15.46:0/13220581 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f397003c030 con 0x55c7652566a0 2024-05-09T04:56:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.142+0000 7f398c33bb80 10 client.16677 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.142+0000 7f398c33bb80 10 client.16677 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:56:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.142+0000 7f398c33bb80 20 client.16677 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.142+0000 7f398c33bb80 1 --2- 172.21.15.46:0/13220581 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c7652703e0 0x55c7652dd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.142+0000 7f398c33bb80 4 client.16677 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:56:53.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.142+0000 7f398c33bb80 1 -- 172.21.15.46:0/13220581 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c765180a70 con 0x55c7652703e0 2024-05-09T04:56:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.143+0000 7f3976ffd700 1 --2- 172.21.15.46:0/13220581 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c7652703e0 0x55c7652dd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.143+0000 7f3956ffd700 10 client.16677 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.143+0000 7f3976ffd700 1 --2- 172.21.15.46:0/13220581 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c7652703e0 0x55c7652dd4b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a196fcd279073da4 server_cookie=57cf08b3534dbf0e in_seq=0 out_seq=0 2024-05-09T04:56:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.143+0000 7f3956ffd700 1 -- 172.21.15.46:0/13220581 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c765180a70 con 0x55c7652703e0 2024-05-09T04:56:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.143+0000 7f3956ffd700 10 client.16677 handle_command_reply: tid=0 2024-05-09T04:56:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.143+0000 7f398c33bb80 1 client.16677 shutdown 2024-05-09T04:56:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.143+0000 7f398c33bb80 1 -- 172.21.15.46:0/13220581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 msgr2=0x55c76526a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.143+0000 7f398c33bb80 1 --2- 172.21.15.46:0/13220581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c76526a850 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f39700223f0 tx=0x7f3970018f20 comp rx=0 tx=0).stop 2024-05-09T04:56:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.144+0000 7f398c33bb80 1 -- 172.21.15.46:0/13220581 shutdown_connections 2024-05-09T04:56:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.144+0000 7f398c33bb80 1 --2- 172.21.15.46:0/13220581 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c7652566a0 0x55c76526a850 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.144+0000 7f398c33bb80 1 --2- 172.21.15.46:0/13220581 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c7650d94a0 0x55c765269dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.144+0000 7f398c33bb80 1 --2- 172.21.15.46:0/13220581 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c7652703e0 0x55c7652dd4b0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.144+0000 7f398c33bb80 1 --2- 172.21.15.46:0/13220581 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c7651c3160 0x55c76526a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.144+0000 7f3956ffd700 0 client.16677 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:56:53.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.144+0000 7f398c33bb80 1 -- 172.21.15.46:0/13220581 >> 172.21.15.46:0/13220581 conn(0x55c7651396a0 msgr2=0x55c76525e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:53.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.144+0000 7f398c33bb80 1 -- 172.21.15.46:0/13220581 shutdown_connections 2024-05-09T04:56:53.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.144+0000 7f398c33bb80 1 -- 172.21.15.46:0/13220581 wait complete. 2024-05-09T04:56:53.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.144+0000 7f398c33bb80 20 client.16677 trim_cache size 0 max 16384 2024-05-09T04:56:53.150 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:56:53.150 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:56:53.150 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:56:53.150 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:56:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.144+0000 7f397d925700 1 -- 172.21.15.46:0/1074542390 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f39600b2b40 msgr2=0x7f39600b5050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.144+0000 7f397d925700 1 --2- 172.21.15.46:0/1074542390 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f39600b2b40 0x7f39600b5050 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3978100610 tx=0x7f396c00b410 comp rx=0 tx=0).stop 2024-05-09T04:56:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.144+0000 7f397d925700 1 -- 172.21.15.46:0/1074542390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39781029c0 msgr2=0x7f3978104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.144+0000 7f397d925700 1 --2- 172.21.15.46:0/1074542390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39781029c0 0x7f3978104590 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f396800a6a0 tx=0x7f396800a6d0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.145+0000 7f397d925700 1 -- 172.21.15.46:0/1074542390 shutdown_connections 2024-05-09T04:56:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.145+0000 7f397d925700 1 --2- 172.21.15.46:0/1074542390 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f39600b2b40 0x7f39600b5050 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.145+0000 7f397d925700 1 --2- 172.21.15.46:0/1074542390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f39781089f0 0x7f3978105010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.145+0000 7f397d925700 1 --2- 172.21.15.46:0/1074542390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f39781029c0 0x7f3978104590 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.145+0000 7f397d925700 1 --2- 172.21.15.46:0/1074542390 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3978103310 0x7f3978104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.145+0000 7f397d925700 1 -- 172.21.15.46:0/1074542390 >> 172.21.15.46:0/1074542390 conn(0x7f39780fe4e0 msgr2=0x7f3978106eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.145+0000 7f397d925700 1 -- 172.21.15.46:0/1074542390 shutdown_connections 2024-05-09T04:56:53.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:53.145+0000 7f397d925700 1 -- 172.21.15.46:0/1074542390 wait complete. 2024-05-09T04:56:53.857 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:56:53.858 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:56:54.124 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:55.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:54 smithi110 ceph-mon[34904]: pgmap v177: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:56:55.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:54 smithi046 ceph-mon[30136]: pgmap v177: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:56:55.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:54 smithi072 ceph-mon[34088]: pgmap v177: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:56:55.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.939+0000 7fa6344da700 1 -- 172.21.15.46:0/4135908790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c109850 msgr2=0x7fa62c1111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:55.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.939+0000 7fa6344da700 1 --2- 172.21.15.46:0/4135908790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c109850 0x7fa62c1111b0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa62400b0d0 tx=0x7fa62400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:56:55.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.939+0000 7fa6344da700 1 -- 172.21.15.46:0/4135908790 shutdown_connections 2024-05-09T04:56:55.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.940+0000 7fa6344da700 1 --2- 172.21.15.46:0/4135908790 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c109850 0x7fa62c1111b0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:55.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.940+0000 7fa6344da700 1 --2- 172.21.15.46:0/4135908790 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa62c1045d0 0x7fa62c1049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:55.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.940+0000 7fa6344da700 1 --2- 172.21.15.46:0/4135908790 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa62c108f00 0x7fa62c109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:55.942 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.940+0000 7fa6344da700 1 -- 172.21.15.46:0/4135908790 >> 172.21.15.46:0/4135908790 conn(0x7fa62c0ffea0 msgr2=0x7fa62c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.940+0000 7fa6344da700 1 -- 172.21.15.46:0/4135908790 shutdown_connections 2024-05-09T04:56:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.940+0000 7fa6344da700 1 -- 172.21.15.46:0/4135908790 wait complete. 2024-05-09T04:56:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.940+0000 7fa6344da700 1 Processor -- start 2024-05-09T04:56:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.940+0000 7fa6344da700 1 -- start start 2024-05-09T04:56:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa6344da700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c1045d0 0x7fa62c19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa6344da700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa62c108f00 0x7fa62c19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa6344da700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa62c109850 0x7fa62c19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa6344da700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c1144a0 con 0x7fa62c1045d0 2024-05-09T04:56:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa6344da700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c114320 con 0x7fa62c108f00 2024-05-09T04:56:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa6344da700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa62c114620 con 0x7fa62c109850 2024-05-09T04:56:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa632276700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c1045d0 0x7fa62c19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa632276700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c1045d0 0x7fa62c19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35056/0 (socket says 172.21.15.46:35056) 2024-05-09T04:56:55.943 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa632276700 1 -- 172.21.15.46:0/3752802550 learned_addr learned my addr 172.21.15.46:0/3752802550 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa631a75700 1 --2- 172.21.15.46:0/3752802550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa62c108f00 0x7fa62c19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa632a77700 1 --2- 172.21.15.46:0/3752802550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa62c109850 0x7fa62c19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa632276700 1 -- 172.21.15.46:0/3752802550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa62c109850 msgr2=0x7fa62c19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa632276700 1 --2- 172.21.15.46:0/3752802550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa62c109850 0x7fa62c19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa632276700 1 -- 172.21.15.46:0/3752802550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa62c108f00 msgr2=0x7fa62c19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa632276700 1 --2- 172.21.15.46:0/3752802550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa62c108f00 0x7fa62c19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.941+0000 7fa632276700 1 -- 172.21.15.46:0/3752802550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa62c19fc80 con 0x7fa62c1045d0 2024-05-09T04:56:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.942+0000 7fa632276700 1 --2- 172.21.15.46:0/3752802550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c1045d0 0x7fa62c19adb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00d980 tx=0x7fa61c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.942+0000 7fa62b7fe700 1 -- 172.21.15.46:0/3752802550 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c017070 con 0x7fa62c1045d0 2024-05-09T04:56:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.942+0000 7fa62b7fe700 1 -- 172.21.15.46:0/3752802550 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa61c005020 con 0x7fa62c1045d0 2024-05-09T04:56:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.942+0000 7fa62b7fe700 1 -- 172.21.15.46:0/3752802550 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa61c00fa10 con 0x7fa62c1045d0 2024-05-09T04:56:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.942+0000 7fa6344da700 1 -- 172.21.15.46:0/3752802550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa62c19ff70 con 0x7fa62c1045d0 2024-05-09T04:56:55.944 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.942+0000 7fa6344da700 1 -- 172.21.15.46:0/3752802550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa62c1a7890 con 0x7fa62c1045d0 2024-05-09T04:56:55.945 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.943+0000 7fa6344da700 1 -- 172.21.15.46:0/3752802550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa62c0512c0 con 0x7fa62c1045d0 2024-05-09T04:56:55.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.946+0000 7fa62b7fe700 1 -- 172.21.15.46:0/3752802550 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa61c004a30 con 0x7fa62c1045d0 2024-05-09T04:56:55.946 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.946+0000 7fa62b7fe700 1 --2- 172.21.15.46:0/3752802550 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa6140b2a70 0x7fa6140b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.947+0000 7fa631a75700 1 --2- 172.21.15.46:0/3752802550 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa6140b2a70 0x7fa6140b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.947+0000 7fa62b7fe700 1 -- 172.21.15.46:0/3752802550 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa61c0e2860 con 0x7fa62c1045d0 2024-05-09T04:56:55.947 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.947+0000 7fa631a75700 1 --2- 172.21.15.46:0/3752802550 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa6140b2a70 0x7fa6140b4f80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa62c103440 tx=0x7fa62000b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:55.950 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:55.950+0000 7fa62b7fe700 1 -- 172.21.15.46:0/3752802550 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa61c020020 con 0x7fa62c1045d0 2024-05-09T04:56:56.214 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:56.214+0000 7fa6344da700 1 -- 172.21.15.46:0/3752802550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa62c02e5f0 con 0x7fa62c1045d0 2024-05-09T04:56:56.215 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:56.215+0000 7fa62b7fe700 1 -- 172.21.15.46:0/3752802550 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7fa61c0aa2a0 con 0x7fa62c1045d0 2024-05-09T04:56:56.216 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:56:56.216 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:56:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:56.217+0000 7fa6344da700 1 -- 172.21.15.46:0/3752802550 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa6140b2a70 msgr2=0x7fa6140b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:56.217+0000 7fa6344da700 1 --2- 172.21.15.46:0/3752802550 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa6140b2a70 0x7fa6140b4f80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fa62c103440 tx=0x7fa62000b410 comp rx=0 tx=0).stop 2024-05-09T04:56:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:56.217+0000 7fa6344da700 1 -- 172.21.15.46:0/3752802550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c1045d0 msgr2=0x7fa62c19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:56.217+0000 7fa6344da700 1 --2- 172.21.15.46:0/3752802550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c1045d0 0x7fa62c19adb0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa61c00d980 tx=0x7fa61c00dc90 comp rx=0 tx=0).stop 2024-05-09T04:56:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:56.217+0000 7fa6344da700 1 -- 172.21.15.46:0/3752802550 shutdown_connections 2024-05-09T04:56:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:56.217+0000 7fa6344da700 1 --2- 172.21.15.46:0/3752802550 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa6140b2a70 0x7fa6140b4f80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:56.217+0000 7fa6344da700 1 --2- 172.21.15.46:0/3752802550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa62c109850 0x7fa62c19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:56.217+0000 7fa6344da700 1 --2- 172.21.15.46:0/3752802550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa62c1045d0 0x7fa62c19adb0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:56.217+0000 7fa6344da700 1 --2- 172.21.15.46:0/3752802550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa62c108f00 0x7fa62c19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:56.217+0000 7fa6344da700 1 -- 172.21.15.46:0/3752802550 >> 172.21.15.46:0/3752802550 conn(0x7fa62c0ffea0 msgr2=0x7fa62c108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:56.218 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:56.218+0000 7fa6344da700 1 -- 172.21.15.46:0/3752802550 shutdown_connections 2024-05-09T04:56:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:56.218+0000 7fa6344da700 1 -- 172.21.15.46:0/3752802550 wait complete. 2024-05-09T04:56:56.219 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:56:56.827 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:56 smithi046 ceph-mon[30136]: pgmap v178: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:56:56.827 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:56 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3752802550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:56:56.829 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:56:57.086 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:56:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:56 smithi072 ceph-mon[34088]: pgmap v178: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:56:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:56 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3752802550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:56:57.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:56 smithi110 ceph-mon[34904]: pgmap v178: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:56:57.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:56 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3752802550' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:56:58.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:56:58 smithi046 ceph-mon[30136]: pgmap v179: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:56:58.853 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.853+0000 7f71ab51a700 1 -- 172.21.15.46:0/558443991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71a4108760 msgr2=0x7f71a41100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.853+0000 7f71ab51a700 1 --2- 172.21.15.46:0/558443991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71a4108760 0x7f71a41100c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f719c00b0d0 tx=0x7f719c00b3e0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.853+0000 7f71ab51a700 1 -- 172.21.15.46:0/558443991 shutdown_connections 2024-05-09T04:56:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.853+0000 7f71ab51a700 1 --2- 172.21.15.46:0/558443991 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71a4108760 0x7f71a41100c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.853+0000 7f71ab51a700 1 --2- 172.21.15.46:0/558443991 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71a41034e0 0x7f71a41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.853+0000 7f71ab51a700 1 --2- 172.21.15.46:0/558443991 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a4107e10 0x7f71a4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.853+0000 7f71ab51a700 1 -- 172.21.15.46:0/558443991 >> 172.21.15.46:0/558443991 conn(0x7f71a40fef30 msgr2=0x7f71a4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.853+0000 7f71ab51a700 1 -- 172.21.15.46:0/558443991 shutdown_connections 2024-05-09T04:56:58.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.853+0000 7f71ab51a700 1 -- 172.21.15.46:0/558443991 wait complete. 2024-05-09T04:56:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.854+0000 7f71ab51a700 1 Processor -- start 2024-05-09T04:56:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.854+0000 7f71ab51a700 1 -- start start 2024-05-09T04:56:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.854+0000 7f71ab51a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71a41034e0 0x7f71a419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.854+0000 7f71ab51a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a4107e10 0x7f71a419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.854+0000 7f71ab51a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71a4108760 0x7f71a419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.854+0000 7f71ab51a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f71a41133b0 con 0x7f71a41034e0 2024-05-09T04:56:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.854+0000 7f71ab51a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f71a4113230 con 0x7f71a4107e10 2024-05-09T04:56:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.854+0000 7f71ab51a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71a4113530 con 0x7f71a4108760 2024-05-09T04:56:58.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.855+0000 7f71a92b6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71a41034e0 0x7f71a419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.855+0000 7f71a92b6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71a41034e0 0x7f71a419aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35080/0 (socket says 172.21.15.46:35080) 2024-05-09T04:56:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.855+0000 7f71a92b6700 1 -- 172.21.15.46:0/887673982 learned_addr learned my addr 172.21.15.46:0/887673982 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.855+0000 7f71a9ab7700 1 --2- 172.21.15.46:0/887673982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71a4108760 0x7f71a419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.855+0000 7f71a8ab5700 1 --2- 172.21.15.46:0/887673982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a4107e10 0x7f71a419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.855+0000 7f71a9ab7700 1 -- 172.21.15.46:0/887673982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a4107e10 msgr2=0x7f71a419b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.855+0000 7f71a9ab7700 1 --2- 172.21.15.46:0/887673982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a4107e10 0x7f71a419b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.855+0000 7f71a9ab7700 1 -- 172.21.15.46:0/887673982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71a41034e0 msgr2=0x7f71a419aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.855+0000 7f71a9ab7700 1 --2- 172.21.15.46:0/887673982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71a41034e0 0x7f71a419aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.855+0000 7f71a9ab7700 1 -- 172.21.15.46:0/887673982 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71a419fda0 con 0x7f71a4108760 2024-05-09T04:56:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.855+0000 7f71a9ab7700 1 --2- 172.21.15.46:0/887673982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71a4108760 0x7f71a419f700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f719c00b0d0 tx=0x7f719c007a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.855+0000 7f71967fc700 1 -- 172.21.15.46:0/887673982 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c01f070 con 0x7f71a4108760 2024-05-09T04:56:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.855+0000 7f71967fc700 1 -- 172.21.15.46:0/887673982 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f719c004c50 con 0x7f71a4108760 2024-05-09T04:56:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.855+0000 7f71ab51a700 1 -- 172.21.15.46:0/887673982 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71a41a0030 con 0x7f71a4108760 2024-05-09T04:56:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.855+0000 7f71967fc700 1 -- 172.21.15.46:0/887673982 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f719c01b690 con 0x7f71a4108760 2024-05-09T04:56:58.857 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.856+0000 7f71ab51a700 1 -- 172.21.15.46:0/887673982 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71a41a0510 con 0x7f71a4108760 2024-05-09T04:56:58.858 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.858+0000 7f71967fc700 1 -- 172.21.15.46:0/887673982 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f719c022070 con 0x7f71a4108760 2024-05-09T04:56:58.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.860+0000 7f71967fc700 1 --2- 172.21.15.46:0/887673982 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f71900b29f0 0x7f71900b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.860+0000 7f71a92b6700 1 --2- 172.21.15.46:0/887673982 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f71900b29f0 0x7f71900b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.860+0000 7f71b9f30b80 1 Processor -- start 2024-05-09T04:56:58.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.860+0000 7f71967fc700 1 -- 172.21.15.46:0/887673982 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f719c00f020 con 0x7f71a4108760 2024-05-09T04:56:58.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.860+0000 7f71b9f30b80 1 -- start start 2024-05-09T04:56:58.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.860+0000 7f71b9f30b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0e95d00 0x5564d0f9cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.860+0000 7f71b9f30b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e1b4a0 0x5564d0f9d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.860+0000 7f71b9f30b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0fa11e0 0x5564d0fa15f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.860+0000 7f71b9f30b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5564d0ebb0a0 con 0x5564d0e1b4a0 2024-05-09T04:56:58.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.860+0000 7f71b9f30b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5564d0e609a0 con 0x5564d0fa11e0 2024-05-09T04:56:58.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.860+0000 7f71b9f30b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5564d0eb7060 con 0x5564d0e95d00 2024-05-09T04:56:58.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.860+0000 7f71a92b6700 1 --2- 172.21.15.46:0/887673982 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f71900b29f0 0x7f71900b4f00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009cb0 tx=0x7f71a0009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:58.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.861+0000 7f71a8ab5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e1b4a0 0x5564d0f9d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.861+0000 7f71a8ab5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e1b4a0 0x5564d0f9d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35084/0 (socket says 172.21.15.46:35084) 2024-05-09T04:56:58.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.861+0000 7f71a8ab5700 1 -- 172.21.15.46:0/3718370351 learned_addr learned my addr 172.21.15.46:0/3718370351 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:58.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.861+0000 7f71a92b6700 1 --2- 172.21.15.46:0/3718370351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0e95d00 0x5564d0f9cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.861+0000 7f71a9ab7700 1 --2- 172.21.15.46:0/3718370351 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0fa11e0 0x5564d0fa15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.861+0000 7f71a8ab5700 1 -- 172.21.15.46:0/3718370351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0e95d00 msgr2=0x5564d0f9cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.861+0000 7f71a8ab5700 1 --2- 172.21.15.46:0/3718370351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0e95d00 0x5564d0f9cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.861+0000 7f71a8ab5700 1 -- 172.21.15.46:0/3718370351 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0fa11e0 msgr2=0x5564d0fa15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.861+0000 7f71a8ab5700 1 --2- 172.21.15.46:0/3718370351 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0fa11e0 0x5564d0fa15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.861+0000 7f71a8ab5700 1 -- 172.21.15.46:0/3718370351 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d0e51500 con 0x5564d0e1b4a0 2024-05-09T04:56:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.861+0000 7f71a8ab5700 1 --2- 172.21.15.46:0/3718370351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e1b4a0 0x5564d0f9d030 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7198007be0 tx=0x7f7198007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1870d826cf3acaff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.861+0000 7f718f7fe700 1 -- 172.21.15.46:0/3718370351 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198004310 con 0x5564d0e1b4a0 2024-05-09T04:56:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.861+0000 7f718f7fe700 1 -- 172.21.15.46:0/3718370351 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7198015070 con 0x5564d0e1b4a0 2024-05-09T04:56:58.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.861+0000 7f718f7fe700 1 -- 172.21.15.46:0/3718370351 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198003cf0 con 0x5564d0e1b4a0 2024-05-09T04:56:58.867 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.862+0000 7f71b9f30b80 1 -- 172.21.15.46:0/3718370351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e1b4a0 msgr2=0x5564d0f9d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.862+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/3718370351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e1b4a0 0x5564d0f9d030 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f7198007be0 tx=0x7f7198007ef0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.862+0000 7f71b9f30b80 1 -- 172.21.15.46:0/3718370351 shutdown_connections 2024-05-09T04:56:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.862+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/3718370351 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0fa11e0 0x5564d0fa15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.862+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/3718370351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0e95d00 0x5564d0f9cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.862+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/3718370351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e1b4a0 0x5564d0f9d030 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.862+0000 7f71b9f30b80 1 -- 172.21.15.46:0/3718370351 >> 172.21.15.46:0/3718370351 conn(0x5564d0e1ef70 msgr2=0x5564d0f93220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.862+0000 7f71b9f30b80 1 -- 172.21.15.46:0/3718370351 shutdown_connections 2024-05-09T04:56:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.862+0000 7f71b9f30b80 1 -- 172.21.15.46:0/3718370351 wait complete. 2024-05-09T04:56:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.863+0000 7f71b9f30b80 1 -- start start 2024-05-09T04:56:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.863+0000 7f71b9f30b80 1 Processor -- start 2024-05-09T04:56:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.863+0000 7f71b9f30b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0e1b4a0 0x5564d0fa63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.863+0000 7f71b9f30b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e95d00 0x5564d0fa6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.863+0000 7f71b9f30b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0fa11e0 0x5564d0fa6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.863+0000 7f71b9f30b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5564d0fa1f00 con 0x5564d0e95d00 2024-05-09T04:56:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.863+0000 7f71b9f30b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5564d0e55b20 con 0x5564d0e1b4a0 2024-05-09T04:56:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.863+0000 7f71b9f30b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5564d0fa9470 con 0x5564d0fa11e0 2024-05-09T04:56:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.863+0000 7f71b9f30b80 10 client.0 _ll_register_callbacks cb 0x5564d0d2c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.863+0000 7f71a8ab5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e95d00 0x5564d0fa6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.864+0000 7f71a8ab5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e95d00 0x5564d0fa6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35090/0 (socket says 172.21.15.46:35090) 2024-05-09T04:56:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.864+0000 7f71a8ab5700 1 -- 172.21.15.46:0/1099626578 learned_addr learned my addr 172.21.15.46:0/1099626578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.864+0000 7f71a92b6700 1 --2- 172.21.15.46:0/1099626578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0e1b4a0 0x5564d0fa63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.864+0000 7f71a9ab7700 1 --2- 172.21.15.46:0/1099626578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0fa11e0 0x5564d0fa6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.864+0000 7f71a8ab5700 1 -- 172.21.15.46:0/1099626578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0fa11e0 msgr2=0x5564d0fa6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.864+0000 7f71a8ab5700 1 --2- 172.21.15.46:0/1099626578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0fa11e0 0x5564d0fa6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.864+0000 7f71a8ab5700 1 -- 172.21.15.46:0/1099626578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0e1b4a0 msgr2=0x5564d0fa63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.864+0000 7f71a8ab5700 1 --2- 172.21.15.46:0/1099626578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0e1b4a0 0x5564d0fa63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.864+0000 7f71a8ab5700 1 -- 172.21.15.46:0/1099626578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d0d36490 con 0x5564d0e95d00 2024-05-09T04:56:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.864+0000 7f71a92b6700 1 --2- 172.21.15.46:0/1099626578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0e1b4a0 0x5564d0fa63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:56:58.869 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.864+0000 7f71a8ab5700 1 --2- 172.21.15.46:0/1099626578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e95d00 0x5564d0fa6930 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7198000c00 tx=0x7f71980043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a87ce7b19ead87d3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.864+0000 7f718cff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:56:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.864+0000 7f718cff9700 1 -- 172.21.15.46:0/1099626578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198028060 con 0x5564d0e95d00 2024-05-09T04:56:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.864+0000 7f718cff9700 1 -- 172.21.15.46:0/1099626578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f7198010b90 con 0x5564d0e95d00 2024-05-09T04:56:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.864+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1099626578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7198015070 con 0x5564d0e95d00 2024-05-09T04:56:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.865+0000 7f718cff9700 1 -- 172.21.15.46:0/1099626578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7198010d90 con 0x5564d0e95d00 2024-05-09T04:56:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.865+0000 7f71b9f30b80 10 client.16695 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.865+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1099626578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5564d0fb3d80 con 0x5564d0e95d00 2024-05-09T04:56:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.865+0000 7f718cff9700 1 -- 172.21.15.46:0/1099626578 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7198015070 con 0x5564d0e95d00 2024-05-09T04:56:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.865+0000 7f718cff9700 1 -- 172.21.15.46:0/1099626578 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f719801a9d0 con 0x5564d0e95d00 2024-05-09T04:56:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.865+0000 7f71b9f30b80 10 client.16695 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.865+0000 7f71b9f30b80 10 client.16695 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:56:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.865+0000 7f71b9f30b80 20 client.16695 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:58.870 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.865+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/1099626578 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5564d0fb6360 0x5564d0fb8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.865+0000 7f71b9f30b80 4 client.16695 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:56:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.865+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1099626578 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564d0ec8a80 con 0x5564d0fb6360 2024-05-09T04:56:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.865+0000 7f71a92b6700 1 --2- 172.21.15.46:0/1099626578 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5564d0fb6360 0x5564d0fb8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.866+0000 7f71a92b6700 1 --2- 172.21.15.46:0/1099626578 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5564d0fb6360 0x5564d0fb8780 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71e23d16b0a9e910 server_cookie=f9a402e896e9a2e9 in_seq=0 out_seq=0 2024-05-09T04:56:58.871 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.866+0000 7f718cff9700 10 client.16695 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:56:58.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.866+0000 7f718cff9700 1 -- 172.21.15.46:0/1099626578 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5564d0ec8a80 con 0x5564d0fb6360 2024-05-09T04:56:58.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.867+0000 7f718cff9700 10 client.16695 handle_command_reply: tid=0 2024-05-09T04:56:58.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.867+0000 7f71b9f30b80 1 client.16695 shutdown 2024-05-09T04:56:58.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.867+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1099626578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e95d00 msgr2=0x5564d0fa6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:58.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.867+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/1099626578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e95d00 0x5564d0fa6930 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f7198000c00 tx=0x7f71980043f0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.867+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1099626578 shutdown_connections 2024-05-09T04:56:58.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.867+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/1099626578 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5564d0fb6360 0x5564d0fb8780 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.867+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/1099626578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0fa11e0 0x5564d0fa6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.867+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/1099626578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0e1b4a0 0x5564d0fa63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.874 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.867+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/1099626578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e95d00 0x5564d0fa6930 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.867+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1099626578 >> 172.21.15.46:0/1099626578 conn(0x5564d0e1ef70 msgr2=0x5564d0f8cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:58.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.867+0000 7f718cff9700 0 client.16695 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:56:58.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.867+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1099626578 shutdown_connections 2024-05-09T04:56:58.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.867+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1099626578 wait complete. 2024-05-09T04:56:58.875 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.867+0000 7f71b9f30b80 20 client.16695 trim_cache size 0 max 16384 2024-05-09T04:56:58.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.882+0000 7f71b9f30b80 1 Processor -- start 2024-05-09T04:56:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71b9f30b80 1 -- start start 2024-05-09T04:56:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71b9f30b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e1b4a0 0x5564d0fa6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71b9f30b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0e95d00 0x5564d0f8d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71b9f30b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0fa11e0 0x5564d0fa3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71b9f30b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f71a41af500 con 0x5564d0e1b4a0 2024-05-09T04:56:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71b9f30b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f71a41acd90 con 0x5564d0e95d00 2024-05-09T04:56:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71b9f30b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71a41aabe0 con 0x5564d0fa11e0 2024-05-09T04:56:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71a92b6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e1b4a0 0x5564d0fa6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71a92b6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e1b4a0 0x5564d0fa6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35098/0 (socket says 172.21.15.46:35098) 2024-05-09T04:56:58.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71a92b6700 1 -- 172.21.15.46:0/398223630 learned_addr learned my addr 172.21.15.46:0/398223630 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:58.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71a8ab5700 1 --2- 172.21.15.46:0/398223630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0e95d00 0x5564d0f8d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71a9ab7700 1 --2- 172.21.15.46:0/398223630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0fa11e0 0x5564d0fa3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71a92b6700 1 -- 172.21.15.46:0/398223630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0fa11e0 msgr2=0x5564d0fa3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:58.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71a92b6700 1 --2- 172.21.15.46:0/398223630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0fa11e0 0x5564d0fa3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71a92b6700 1 -- 172.21.15.46:0/398223630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0e95d00 msgr2=0x5564d0f8d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:58.892 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71a92b6700 1 --2- 172.21.15.46:0/398223630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0e95d00 0x5564d0f8d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71a92b6700 1 -- 172.21.15.46:0/398223630 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564d0f8f9d0 con 0x5564d0e1b4a0 2024-05-09T04:56:58.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71a9ab7700 1 --2- 172.21.15.46:0/398223630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0fa11e0 0x5564d0fa3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:56:58.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.883+0000 7f71a92b6700 1 --2- 172.21.15.46:0/398223630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e1b4a0 0x5564d0fa6cb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009b70 tx=0x7f71a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44c9c22b20d7f31 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:58.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.884+0000 7f718effd700 1 -- 172.21.15.46:0/398223630 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a002b5b0 con 0x5564d0e1b4a0 2024-05-09T04:56:58.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.884+0000 7f718effd700 1 -- 172.21.15.46:0/398223630 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71a001f420 con 0x5564d0e1b4a0 2024-05-09T04:56:58.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.884+0000 7f718effd700 1 -- 172.21.15.46:0/398223630 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71a002a680 con 0x5564d0e1b4a0 2024-05-09T04:56:58.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.884+0000 7f71b9f30b80 1 -- 172.21.15.46:0/398223630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e1b4a0 msgr2=0x5564d0fa6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:58.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.884+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/398223630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e1b4a0 0x5564d0fa6cb0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009b70 tx=0x7f71a0014000 comp rx=0 tx=0).stop 2024-05-09T04:56:58.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.884+0000 7f71b9f30b80 1 -- 172.21.15.46:0/398223630 shutdown_connections 2024-05-09T04:56:58.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.885+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/398223630 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0fa11e0 0x5564d0fa3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.885+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/398223630 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e1b4a0 0x5564d0fa6cb0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.885+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/398223630 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0e95d00 0x5564d0f8d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.885+0000 7f71b9f30b80 1 -- 172.21.15.46:0/398223630 >> 172.21.15.46:0/398223630 conn(0x5564d0e1ef70 msgr2=0x5564d0f8cff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:58.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.885+0000 7f71b9f30b80 1 -- 172.21.15.46:0/398223630 shutdown_connections 2024-05-09T04:56:58.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.885+0000 7f71b9f30b80 1 -- 172.21.15.46:0/398223630 wait complete. 2024-05-09T04:56:58.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.885+0000 7f71b9f30b80 1 -- start start 2024-05-09T04:56:58.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.885+0000 7f71b9f30b80 1 Processor -- start 2024-05-09T04:56:58.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71b9f30b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0e1b4a0 0x5564d0fb8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71b9f30b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e95d00 0x5564d0fb88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71b9f30b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0fa11e0 0x5564d0fb8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71b9f30b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f71a41ab250 con 0x5564d0e95d00 2024-05-09T04:56:58.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71b9f30b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f71a41ac6c0 con 0x5564d0fa11e0 2024-05-09T04:56:58.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71b9f30b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71a41ab920 con 0x5564d0e1b4a0 2024-05-09T04:56:58.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71b9f30b80 10 client.0 _ll_register_callbacks cb 0x5564d0d2c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:56:58.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71a8ab5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e95d00 0x5564d0fb88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71a8ab5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e95d00 0x5564d0fb88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35114/0 (socket says 172.21.15.46:35114) 2024-05-09T04:56:58.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71a8ab5700 1 -- 172.21.15.46:0/1542309370 learned_addr learned my addr 172.21.15.46:0/1542309370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:56:58.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71a9ab7700 1 --2- 172.21.15.46:0/1542309370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0fa11e0 0x5564d0fb8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71a92b6700 1 --2- 172.21.15.46:0/1542309370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0e1b4a0 0x5564d0fb8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71a8ab5700 1 -- 172.21.15.46:0/1542309370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0e1b4a0 msgr2=0x5564d0fb8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:58.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71a8ab5700 1 --2- 172.21.15.46:0/1542309370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0e1b4a0 0x5564d0fb8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71a8ab5700 1 -- 172.21.15.46:0/1542309370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0fa11e0 msgr2=0x5564d0fb8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:56:58.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71a8ab5700 1 --2- 172.21.15.46:0/1542309370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0fa11e0 0x5564d0fb8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71a8ab5700 1 -- 172.21.15.46:0/1542309370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564d0fbbdd0 con 0x5564d0e95d00 2024-05-09T04:56:58.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f71a8ab5700 1 --2- 172.21.15.46:0/1542309370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e95d00 0x5564d0fb88b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7198007790 tx=0x7f7198011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23f77e040b072159 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:56:58.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.886+0000 7f718cff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:56:58.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.887+0000 7f718cff9700 1 -- 172.21.15.46:0/1542309370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7198028040 con 0x5564d0e95d00 2024-05-09T04:56:58.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.887+0000 7f718cff9700 1 -- 172.21.15.46:0/1542309370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f7198010b90 con 0x5564d0e95d00 2024-05-09T04:56:58.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.887+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1542309370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71a001f420 con 0x5564d0e95d00 2024-05-09T04:56:58.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.887+0000 7f718cff9700 1 -- 172.21.15.46:0/1542309370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f71980073d0 con 0x5564d0e95d00 2024-05-09T04:56:58.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.887+0000 7f71b9f30b80 10 client.16701 fetch_fsmap learned FSMap version 25 2024-05-09T04:56:58.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.887+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1542309370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5564d101f860 con 0x5564d0e95d00 2024-05-09T04:56:58.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.887+0000 7f718cff9700 1 -- 172.21.15.46:0/1542309370 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71a001f420 con 0x5564d0e95d00 2024-05-09T04:56:58.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.887+0000 7f718cff9700 1 -- 172.21.15.46:0/1542309370 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7198003d10 con 0x5564d0e95d00 2024-05-09T04:56:58.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.887+0000 7f71b9f30b80 10 client.16701 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:56:58.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.887+0000 7f71b9f30b80 10 client.16701 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:56:58.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.887+0000 7f71b9f30b80 20 client.16701 populate_metadata read hostname 'smithi046' 2024-05-09T04:56:58.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.887+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/1542309370 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5564d101ff60 0x5564d1022380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:56:58.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.887+0000 7f71b9f30b80 4 client.16701 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:56:58.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.887+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1542309370 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5564d0ec2b20 con 0x5564d101ff60 2024-05-09T04:56:58.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.887+0000 7f71a92b6700 1 --2- 172.21.15.46:0/1542309370 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5564d101ff60 0x5564d1022380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:56:58.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.888+0000 7f71a92b6700 1 --2- 172.21.15.46:0/1542309370 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5564d101ff60 0x5564d1022380 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4984f322c8da417b server_cookie=a2c19f84a09b9df0 in_seq=0 out_seq=0 2024-05-09T04:56:58.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.888+0000 7f718cff9700 10 client.16701 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:56:58.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.888+0000 7f718cff9700 1 -- 172.21.15.46:0/1542309370 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5564d0ec2b20 con 0x5564d101ff60 2024-05-09T04:56:58.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.888+0000 7f718cff9700 10 client.16701 handle_command_reply: tid=0 2024-05-09T04:56:58.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.888+0000 7f71b9f30b80 1 client.16701 shutdown 2024-05-09T04:56:58.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.888+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1542309370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e95d00 msgr2=0x5564d0fb88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:58.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.888+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/1542309370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e95d00 0x5564d0fb88b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f7198007790 tx=0x7f7198011870 comp rx=0 tx=0).stop 2024-05-09T04:56:58.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.888+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1542309370 shutdown_connections 2024-05-09T04:56:58.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.888+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/1542309370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5564d0fa11e0 0x5564d0fb8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.888+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/1542309370 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x5564d101ff60 0x5564d1022380 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.888+0000 7f718cff9700 0 client.16701 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:56:58.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.888+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/1542309370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5564d0e1b4a0 0x5564d0fb8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.888+0000 7f71b9f30b80 1 --2- 172.21.15.46:0/1542309370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5564d0e95d00 0x5564d0fb88b0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.900 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.888+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1542309370 >> 172.21.15.46:0/1542309370 conn(0x5564d0e1ef70 msgr2=0x5564d0f973f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:58.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.888+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1542309370 shutdown_connections 2024-05-09T04:56:58.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.889+0000 7f71b9f30b80 1 -- 172.21.15.46:0/1542309370 wait complete. 2024-05-09T04:56:58.901 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.889+0000 7f71b9f30b80 20 client.16701 trim_cache size 0 max 16384 2024-05-09T04:56:58.901 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:56:58.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.889+0000 7f71ab51a700 1 -- 172.21.15.46:0/887673982 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f71900b29f0 msgr2=0x7f71900b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:58.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.889+0000 7f71ab51a700 1 --2- 172.21.15.46:0/887673982 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f71900b29f0 0x7f71900b4f00 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f71a0009cb0 tx=0x7f71a0009260 comp rx=0 tx=0).stop 2024-05-09T04:56:58.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.889+0000 7f71ab51a700 1 -- 172.21.15.46:0/887673982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71a4108760 msgr2=0x7f71a419f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:56:58.902 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.889+0000 7f71ab51a700 1 --2- 172.21.15.46:0/887673982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71a4108760 0x7f71a419f700 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f719c00b0d0 tx=0x7f719c007a00 comp rx=0 tx=0).stop 2024-05-09T04:56:58.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.889+0000 7f71ab51a700 1 -- 172.21.15.46:0/887673982 shutdown_connections 2024-05-09T04:56:58.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.889+0000 7f71ab51a700 1 --2- 172.21.15.46:0/887673982 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f71900b29f0 0x7f71900b4f00 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.889+0000 7f71ab51a700 1 --2- 172.21.15.46:0/887673982 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f71a4108760 0x7f71a419f700 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.889+0000 7f71ab51a700 1 --2- 172.21.15.46:0/887673982 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71a41034e0 0x7f71a419aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.889+0000 7f71ab51a700 1 --2- 172.21.15.46:0/887673982 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f71a4107e10 0x7f71a419b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:56:58.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.889+0000 7f71ab51a700 1 -- 172.21.15.46:0/887673982 >> 172.21.15.46:0/887673982 conn(0x7f71a40fef30 msgr2=0x7f71a406c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:56:58.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.889+0000 7f71ab51a700 1 -- 172.21.15.46:0/887673982 shutdown_connections 2024-05-09T04:56:58.903 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:56:58.889+0000 7f71ab51a700 1 -- 172.21.15.46:0/887673982 wait complete. 2024-05-09T04:56:59.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:56:58 smithi072 ceph-mon[34088]: pgmap v179: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:56:59.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:56:58 smithi110 ceph-mon[34904]: pgmap v179: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:56:59.733 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:56:59.989 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:57:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:00 smithi046 ceph-mon[30136]: pgmap v180: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:01.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:00 smithi072 ceph-mon[34088]: pgmap v180: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:01.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:00 smithi110 ceph-mon[34904]: pgmap v180: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:01.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.668+0000 7f8ff8b00700 1 -- 172.21.15.46:0/4224440641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff41045e0 msgr2=0x7f8ff41049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:01.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.668+0000 7f8ff8b00700 1 --2- 172.21.15.46:0/4224440641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff41045e0 0x7f8ff41049f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f8fe400b0d0 tx=0x7f8fe400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.668+0000 7f8ff8b00700 1 -- 172.21.15.46:0/4224440641 shutdown_connections 2024-05-09T04:57:01.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.668+0000 7f8ff8b00700 1 --2- 172.21.15.46:0/4224440641 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ff4109860 0x7f8ff41111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.668+0000 7f8ff8b00700 1 --2- 172.21.15.46:0/4224440641 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff41045e0 0x7f8ff41049f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.668+0000 7f8ff8b00700 1 --2- 172.21.15.46:0/4224440641 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ff4108f10 0x7f8ff4109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.668+0000 7f8ff8b00700 1 -- 172.21.15.46:0/4224440641 >> 172.21.15.46:0/4224440641 conn(0x7f8ff40ffed0 msgr2=0x7f8ff4102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:01.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.668+0000 7f8ff8b00700 1 -- 172.21.15.46:0/4224440641 shutdown_connections 2024-05-09T04:57:01.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.668+0000 7f8ff8b00700 1 -- 172.21.15.46:0/4224440641 wait complete. 2024-05-09T04:57:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff8b00700 1 Processor -- start 2024-05-09T04:57:01.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff8b00700 1 -- start start 2024-05-09T04:57:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff8b00700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff41045e0 0x7f8ff419ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff8b00700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ff4108f10 0x7f8ff419b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff8b00700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ff4109860 0x7f8ff419f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff8b00700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4114490 con 0x7f8ff41045e0 2024-05-09T04:57:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff8b00700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4114310 con 0x7f8ff4108f10 2024-05-09T04:57:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff8b00700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff4114610 con 0x7f8ff4109860 2024-05-09T04:57:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff41045e0 0x7f8ff419ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff41045e0 0x7f8ff419ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35134/0 (socket says 172.21.15.46:35134) 2024-05-09T04:57:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff259c700 1 -- 172.21.15.46:0/794100214 learned_addr learned my addr 172.21.15.46:0/794100214 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff1d9b700 1 --2- 172.21.15.46:0/794100214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ff4108f10 0x7f8ff419b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff2d9d700 1 --2- 172.21.15.46:0/794100214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ff4109860 0x7f8ff419f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff259c700 1 -- 172.21.15.46:0/794100214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ff4109860 msgr2=0x7f8ff419f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:01.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff259c700 1 --2- 172.21.15.46:0/794100214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ff4109860 0x7f8ff419f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff259c700 1 -- 172.21.15.46:0/794100214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ff4108f10 msgr2=0x7f8ff419b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:01.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff259c700 1 --2- 172.21.15.46:0/794100214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ff4108f10 0x7f8ff419b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.669+0000 7f8ff259c700 1 -- 172.21.15.46:0/794100214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ff419fcb0 con 0x7f8ff41045e0 2024-05-09T04:57:01.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.670+0000 7f8ff259c700 1 --2- 172.21.15.46:0/794100214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff41045e0 0x7f8ff419ade0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4007a20 tx=0x7f8fe4007a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:01.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.670+0000 7f8feb7fe700 1 -- 172.21.15.46:0/794100214 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe401f070 con 0x7f8ff41045e0 2024-05-09T04:57:01.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.670+0000 7f8ff8b00700 1 -- 172.21.15.46:0/794100214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ff419ff40 con 0x7f8ff41045e0 2024-05-09T04:57:01.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.670+0000 7f8feb7fe700 1 -- 172.21.15.46:0/794100214 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe4004c50 con 0x7f8ff41045e0 2024-05-09T04:57:01.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.670+0000 7f8feb7fe700 1 -- 172.21.15.46:0/794100214 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe401b670 con 0x7f8ff41045e0 2024-05-09T04:57:01.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.670+0000 7f8ff8b00700 1 -- 172.21.15.46:0/794100214 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ff41a78c0 con 0x7f8ff41045e0 2024-05-09T04:57:01.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.673+0000 7f8feb7fe700 1 -- 172.21.15.46:0/794100214 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8fe4022070 con 0x7f8ff41045e0 2024-05-09T04:57:01.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.673+0000 7f8feb7fe700 1 --2- 172.21.15.46:0/794100214 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8fe00b29f0 0x7f8fe00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.673+0000 7f8ff1d9b700 1 --2- 172.21.15.46:0/794100214 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8fe00b29f0 0x7f8fe00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.673+0000 7f8feb7fe700 1 -- 172.21.15.46:0/794100214 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f8fe400f020 con 0x7f8ff41045e0 2024-05-09T04:57:01.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.674+0000 7f9007516b80 1 Processor -- start 2024-05-09T04:57:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.674+0000 7f8ff1d9b700 1 --2- 172.21.15.46:0/794100214 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8fe00b29f0 0x7f8fe00b4f00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4103470 tx=0x7f8fdc00b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.674+0000 7f9007516b80 1 -- start start 2024-05-09T04:57:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.674+0000 7f9007516b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321b60d00 0x55d321c69240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.674+0000 7f9007516b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321ae64a0 0x55d321c69780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.674+0000 7f9007516b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321c63400 0x55d321c63810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.674+0000 7f9007516b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d321b860a0 con 0x55d321b60d00 2024-05-09T04:57:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.674+0000 7f9007516b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d321b2b9a0 con 0x55d321c63400 2024-05-09T04:57:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.674+0000 7f9007516b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d321b82060 con 0x55d321ae64a0 2024-05-09T04:57:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.674+0000 7f8ff259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321b60d00 0x55d321c69240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.674+0000 7f8ff259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321b60d00 0x55d321c69240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35150/0 (socket says 172.21.15.46:35150) 2024-05-09T04:57:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.674+0000 7f8ff259c700 1 -- 172.21.15.46:0/2804044871 learned_addr learned my addr 172.21.15.46:0/2804044871 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.674+0000 7f8ff1d9b700 1 --2- 172.21.15.46:0/2804044871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321ae64a0 0x55d321c69780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.675+0000 7f8ff2d9d700 1 --2- 172.21.15.46:0/2804044871 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321c63400 0x55d321c63810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.675+0000 7f8ff259c700 1 -- 172.21.15.46:0/2804044871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321ae64a0 msgr2=0x55d321c69780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.675+0000 7f8ff259c700 1 --2- 172.21.15.46:0/2804044871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321ae64a0 0x55d321c69780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.675+0000 7f8ff259c700 1 -- 172.21.15.46:0/2804044871 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321c63400 msgr2=0x55d321c63810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.675+0000 7f8ff259c700 1 --2- 172.21.15.46:0/2804044871 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321c63400 0x55d321c63810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.675+0000 7f8ff259c700 1 -- 172.21.15.46:0/2804044871 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d321b1c500 con 0x55d321b60d00 2024-05-09T04:57:01.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.675+0000 7f8ff259c700 1 --2- 172.21.15.46:0/2804044871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321b60d00 0x55d321c69240 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40e0f70 tx=0x7f8fe40aa390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b33b326919f3ca7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.675+0000 7f8fe8ff9700 1 -- 172.21.15.46:0/2804044871 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40abe60 con 0x55d321b60d00 2024-05-09T04:57:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.675+0000 7f8fe8ff9700 1 -- 172.21.15.46:0/2804044871 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe40aa640 con 0x55d321b60d00 2024-05-09T04:57:01.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.675+0000 7f8fe8ff9700 1 -- 172.21.15.46:0/2804044871 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fe40a6dc0 con 0x55d321b60d00 2024-05-09T04:57:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.676+0000 7f9007516b80 1 -- 172.21.15.46:0/2804044871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321b60d00 msgr2=0x55d321c69240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.676+0000 7f9007516b80 1 --2- 172.21.15.46:0/2804044871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321b60d00 0x55d321c69240 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f8fe40e0f70 tx=0x7f8fe40aa390 comp rx=0 tx=0).stop 2024-05-09T04:57:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.676+0000 7f9007516b80 1 -- 172.21.15.46:0/2804044871 shutdown_connections 2024-05-09T04:57:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.676+0000 7f9007516b80 1 --2- 172.21.15.46:0/2804044871 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321c63400 0x55d321c63810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.676+0000 7f9007516b80 1 --2- 172.21.15.46:0/2804044871 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321b60d00 0x55d321c69240 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.676+0000 7f9007516b80 1 --2- 172.21.15.46:0/2804044871 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321ae64a0 0x55d321c69780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.676+0000 7f9007516b80 1 -- 172.21.15.46:0/2804044871 >> 172.21.15.46:0/2804044871 conn(0x55d321ae9f70 msgr2=0x55d321c5e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.676+0000 7f9007516b80 1 -- 172.21.15.46:0/2804044871 shutdown_connections 2024-05-09T04:57:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.676+0000 7f9007516b80 1 -- 172.21.15.46:0/2804044871 wait complete. 2024-05-09T04:57:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.677+0000 7f9007516b80 1 -- start start 2024-05-09T04:57:01.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.677+0000 7f9007516b80 1 Processor -- start 2024-05-09T04:57:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.677+0000 7f9007516b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321ae64a0 0x55d321c72b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.677+0000 7f9007516b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321b60d00 0x55d321c730d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.677+0000 7f9007516b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321c63400 0x55d321c6cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.677+0000 7f9007516b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d321b860a0 con 0x55d321c63400 2024-05-09T04:57:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.677+0000 7f9007516b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d321b20b20 con 0x55d321ae64a0 2024-05-09T04:57:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.677+0000 7f9007516b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d321c63fb0 con 0x55d321b60d00 2024-05-09T04:57:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.677+0000 7f9007516b80 10 client.0 _ll_register_callbacks cb 0x55d3219f76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.677+0000 7f8ff2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321c63400 0x55d321c6cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.677+0000 7f8ff2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321c63400 0x55d321c6cf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35160/0 (socket says 172.21.15.46:35160) 2024-05-09T04:57:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.677+0000 7f8ff2d9d700 1 -- 172.21.15.46:0/4106092088 learned_addr learned my addr 172.21.15.46:0/4106092088 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.677+0000 7f8ff1d9b700 1 --2- 172.21.15.46:0/4106092088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321b60d00 0x55d321c730d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.678+0000 7f8ff259c700 1 --2- 172.21.15.46:0/4106092088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321ae64a0 0x55d321c72b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.678+0000 7f8ff1d9b700 1 -- 172.21.15.46:0/4106092088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321ae64a0 msgr2=0x55d321c72b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.678+0000 7f8ff1d9b700 1 --2- 172.21.15.46:0/4106092088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321ae64a0 0x55d321c72b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.678+0000 7f8ff1d9b700 1 -- 172.21.15.46:0/4106092088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321c63400 msgr2=0x55d321c6cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.678+0000 7f8ff1d9b700 1 --2- 172.21.15.46:0/4106092088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321c63400 0x55d321c6cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.678+0000 7f8ff1d9b700 1 -- 172.21.15.46:0/4106092088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d321a01490 con 0x55d321b60d00 2024-05-09T04:57:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.678+0000 7f8ff2d9d700 1 --2- 172.21.15.46:0/4106092088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321c63400 0x55d321c6cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.678+0000 7f8ff1d9b700 1 --2- 172.21.15.46:0/4106092088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321b60d00 0x55d321c730d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55d321c5fc90 tx=0x7f8fdc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb3931d1630a6ee3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.678+0000 7f8fd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:57:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.678+0000 7f8fd9ffb700 1 -- 172.21.15.46:0/4106092088 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc004830 con 0x55d321b60d00 2024-05-09T04:57:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.678+0000 7f9007516b80 1 -- 172.21.15.46:0/4106092088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fe40aa640 con 0x55d321b60d00 2024-05-09T04:57:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.678+0000 7f8fd9ffb700 1 -- 172.21.15.46:0/4106092088 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f8fdc0049d0 con 0x55d321b60d00 2024-05-09T04:57:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.679+0000 7f8fd9ffb700 1 -- 172.21.15.46:0/4106092088 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8fdc004bd0 con 0x55d321b60d00 2024-05-09T04:57:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.679+0000 7f9007516b80 10 client.26194 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.679+0000 7f9007516b80 1 -- 172.21.15.46:0/4106092088 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d321c7a860 con 0x55d321b60d00 2024-05-09T04:57:01.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.679+0000 7f8fd9ffb700 1 -- 172.21.15.46:0/4106092088 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fe40aa640 con 0x55d321b60d00 2024-05-09T04:57:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.679+0000 7f8fd9ffb700 1 -- 172.21.15.46:0/4106092088 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f8fdc01f530 con 0x55d321b60d00 2024-05-09T04:57:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.679+0000 7f9007516b80 10 client.26194 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.679+0000 7f9007516b80 10 client.26194 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:57:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.679+0000 7f9007516b80 20 client.26194 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.679+0000 7f9007516b80 1 --2- 172.21.15.46:0/4106092088 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55d321c7cdf0 0x55d321c7f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.679+0000 7f9007516b80 4 client.26194 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:57:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.679+0000 7f9007516b80 1 -- 172.21.15.46:0/4106092088 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d321b93a80 con 0x55d321c7cdf0 2024-05-09T04:57:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.680+0000 7f8ff259c700 1 --2- 172.21.15.46:0/4106092088 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55d321c7cdf0 0x55d321c7f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.680+0000 7f8fd9ffb700 10 client.26194 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:57:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.680+0000 7f8ff259c700 1 --2- 172.21.15.46:0/4106092088 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55d321c7cdf0 0x55d321c7f210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7e1e5029f7ca73c0 server_cookie=725e77666669b4b0 in_seq=0 out_seq=0 2024-05-09T04:57:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.681+0000 7f8fd9ffb700 1 -- 172.21.15.46:0/4106092088 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d321b93a80 con 0x55d321c7cdf0 2024-05-09T04:57:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.681+0000 7f8fd9ffb700 10 client.26194 handle_command_reply: tid=0 2024-05-09T04:57:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.681+0000 7f9007516b80 1 client.26194 shutdown 2024-05-09T04:57:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.681+0000 7f9007516b80 1 -- 172.21.15.46:0/4106092088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321b60d00 msgr2=0x55d321c730d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.681+0000 7f9007516b80 1 --2- 172.21.15.46:0/4106092088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321b60d00 0x55d321c730d0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55d321c5fc90 tx=0x7f8fdc0047c0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.681+0000 7f9007516b80 1 -- 172.21.15.46:0/4106092088 shutdown_connections 2024-05-09T04:57:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.681+0000 7f9007516b80 1 --2- 172.21.15.46:0/4106092088 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55d321c7cdf0 0x55d321c7f210 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.681+0000 7f9007516b80 1 --2- 172.21.15.46:0/4106092088 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321c63400 0x55d321c6cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.681+0000 7f9007516b80 1 --2- 172.21.15.46:0/4106092088 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321ae64a0 0x55d321c72b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.681+0000 7f9007516b80 1 --2- 172.21.15.46:0/4106092088 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321b60d00 0x55d321c730d0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.681+0000 7f9007516b80 1 -- 172.21.15.46:0/4106092088 >> 172.21.15.46:0/4106092088 conn(0x55d321ae9f70 msgr2=0x55d321c57ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:01.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.681+0000 7f8fd9ffb700 0 client.26194 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:57:01.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.681+0000 7f9007516b80 1 -- 172.21.15.46:0/4106092088 shutdown_connections 2024-05-09T04:57:01.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.681+0000 7f9007516b80 1 -- 172.21.15.46:0/4106092088 wait complete. 2024-05-09T04:57:01.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.681+0000 7f9007516b80 20 client.26194 trim_cache size 0 max 16384 2024-05-09T04:57:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.696+0000 7f9007516b80 1 Processor -- start 2024-05-09T04:57:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.696+0000 7f9007516b80 1 -- start start 2024-05-09T04:57:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f9007516b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321ae64a0 0x55d321c6cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f9007516b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321b60d00 0x55d321c84960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f9007516b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321c63400 0x55d321c84ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f9007516b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41b17e0 con 0x55d321b60d00 2024-05-09T04:57:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f9007516b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41af070 con 0x55d321ae64a0 2024-05-09T04:57:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f9007516b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41acec0 con 0x55d321c63400 2024-05-09T04:57:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f8ff1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321b60d00 0x55d321c84960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f8ff1d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321b60d00 0x55d321c84960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35164/0 (socket says 172.21.15.46:35164) 2024-05-09T04:57:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f8ff1d9b700 1 -- 172.21.15.46:0/3970484160 learned_addr learned my addr 172.21.15.46:0/3970484160 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:01.698 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f8ff259c700 1 --2- 172.21.15.46:0/3970484160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321ae64a0 0x55d321c6cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f8ff2d9d700 1 --2- 172.21.15.46:0/3970484160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321c63400 0x55d321c84ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f8ff1d9b700 1 -- 172.21.15.46:0/3970484160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321c63400 msgr2=0x55d321c84ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f8ff1d9b700 1 --2- 172.21.15.46:0/3970484160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321c63400 0x55d321c84ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f8ff1d9b700 1 -- 172.21.15.46:0/3970484160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321ae64a0 msgr2=0x55d321c6cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f8ff1d9b700 1 --2- 172.21.15.46:0/3970484160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321ae64a0 0x55d321c6cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f8ff1d9b700 1 -- 172.21.15.46:0/3970484160 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d321c5f1b0 con 0x55d321b60d00 2024-05-09T04:57:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.697+0000 7f8ff1d9b700 1 --2- 172.21.15.46:0/3970484160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321b60d00 0x55d321c84960 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc01ffb0 tx=0x7f8fdc012ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f2bfcee8527d8e7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.698+0000 7f8fdbfff700 1 -- 172.21.15.46:0/3970484160 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc034040 con 0x55d321b60d00 2024-05-09T04:57:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.698+0000 7f8fdbfff700 1 -- 172.21.15.46:0/3970484160 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fdc03d070 con 0x55d321b60d00 2024-05-09T04:57:01.699 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.698+0000 7f8fdbfff700 1 -- 172.21.15.46:0/3970484160 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fdc005130 con 0x55d321b60d00 2024-05-09T04:57:01.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.698+0000 7f9007516b80 1 -- 172.21.15.46:0/3970484160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321b60d00 msgr2=0x55d321c84960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:01.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.698+0000 7f9007516b80 1 --2- 172.21.15.46:0/3970484160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321b60d00 0x55d321c84960 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f8fdc01ffb0 tx=0x7f8fdc012ed0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.699+0000 7f9007516b80 1 -- 172.21.15.46:0/3970484160 shutdown_connections 2024-05-09T04:57:01.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.699+0000 7f9007516b80 1 --2- 172.21.15.46:0/3970484160 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321c63400 0x55d321c84ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.699+0000 7f9007516b80 1 --2- 172.21.15.46:0/3970484160 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321ae64a0 0x55d321c6cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.703 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.699+0000 7f9007516b80 1 --2- 172.21.15.46:0/3970484160 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321b60d00 0x55d321c84960 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.699+0000 7f9007516b80 1 -- 172.21.15.46:0/3970484160 >> 172.21.15.46:0/3970484160 conn(0x55d321ae9f70 msgr2=0x55d321c6c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.699+0000 7f9007516b80 1 -- 172.21.15.46:0/3970484160 shutdown_connections 2024-05-09T04:57:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.699+0000 7f9007516b80 1 -- 172.21.15.46:0/3970484160 wait complete. 2024-05-09T04:57:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.699+0000 7f9007516b80 1 -- start start 2024-05-09T04:57:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.699+0000 7f9007516b80 1 Processor -- start 2024-05-09T04:57:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.699+0000 7f9007516b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321ae64a0 0x55d321c79520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.699+0000 7f9007516b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321b60d00 0x55d321c758a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.699+0000 7f9007516b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321c63400 0x55d321c75ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.699+0000 7f9007516b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41ad530 con 0x55d321c63400 2024-05-09T04:57:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.699+0000 7f9007516b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41ae9a0 con 0x55d321b60d00 2024-05-09T04:57:01.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.699+0000 7f9007516b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8ff41adc00 con 0x55d321ae64a0 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f9007516b80 10 client.0 _ll_register_callbacks cb 0x55d3219f76c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f8ff2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321c63400 0x55d321c75ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f8ff2d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321c63400 0x55d321c75ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35168/0 (socket says 172.21.15.46:35168) 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f8ff1d9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321b60d00 0x55d321c758a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f8ff2d9d700 1 -- 172.21.15.46:0/4122764874 learned_addr learned my addr 172.21.15.46:0/4122764874 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f8ff259c700 1 --2- 172.21.15.46:0/4122764874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321ae64a0 0x55d321c79520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f8ff2d9d700 1 -- 172.21.15.46:0/4122764874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321ae64a0 msgr2=0x55d321c79520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f8ff2d9d700 1 --2- 172.21.15.46:0/4122764874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321ae64a0 0x55d321c79520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f8ff2d9d700 1 -- 172.21.15.46:0/4122764874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321b60d00 msgr2=0x55d321c758a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f8ff2d9d700 1 --2- 172.21.15.46:0/4122764874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321b60d00 0x55d321c758a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f8ff2d9d700 1 -- 172.21.15.46:0/4122764874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d321c76690 con 0x55d321c63400 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f8ff2d9d700 1 --2- 172.21.15.46:0/4122764874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321c63400 0x55d321c75ef0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0036f0 tx=0x7f8fec003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6641c5a03725b162 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f8fd9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f8fd9ffb700 1 -- 172.21.15.46:0/4122764874 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fec011850 con 0x55d321c63400 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f8fd9ffb700 1 -- 172.21.15.46:0/4122764874 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f8fec0119f0 con 0x55d321c63400 2024-05-09T04:57:01.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.700+0000 7f9007516b80 1 -- 172.21.15.46:0/4122764874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8fdc03d070 con 0x55d321c63400 2024-05-09T04:57:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.701+0000 7f8fd9ffb700 1 -- 172.21.15.46:0/4122764874 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8fec029050 con 0x55d321c63400 2024-05-09T04:57:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.701+0000 7f9007516b80 10 client.16719 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.701+0000 7f9007516b80 1 -- 172.21.15.46:0/4122764874 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d321ce6550 con 0x55d321c63400 2024-05-09T04:57:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.701+0000 7f8fd9ffb700 1 -- 172.21.15.46:0/4122764874 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fdc03d070 con 0x55d321c63400 2024-05-09T04:57:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.701+0000 7f8fd9ffb700 1 -- 172.21.15.46:0/4122764874 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f8fec010430 con 0x55d321c63400 2024-05-09T04:57:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.701+0000 7f9007516b80 10 client.16719 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.701+0000 7f9007516b80 10 client.16719 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:57:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.701+0000 7f9007516b80 20 client.16719 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.701+0000 7f9007516b80 1 --2- 172.21.15.46:0/4122764874 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55d321ce6cb0 0x55d321ce90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.701+0000 7f9007516b80 4 client.16719 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:57:01.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.701+0000 7f9007516b80 1 -- 172.21.15.46:0/4122764874 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d321b8db20 con 0x55d321ce6cb0 2024-05-09T04:57:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.701+0000 7f8ff259c700 1 --2- 172.21.15.46:0/4122764874 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55d321ce6cb0 0x55d321ce90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f8ff259c700 1 --2- 172.21.15.46:0/4122764874 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55d321ce6cb0 0x55d321ce90d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa4ae4a52caa9b37 server_cookie=3cc7bb0e46753f56 in_seq=0 out_seq=0 2024-05-09T04:57:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f8fd9ffb700 10 client.16719 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:57:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f8fd9ffb700 1 -- 172.21.15.46:0/4122764874 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d321b8db20 con 0x55d321ce6cb0 2024-05-09T04:57:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f8fd9ffb700 10 client.16719 handle_command_reply: tid=0 2024-05-09T04:57:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f9007516b80 1 client.16719 shutdown 2024-05-09T04:57:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f9007516b80 1 -- 172.21.15.46:0/4122764874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321c63400 msgr2=0x55d321c75ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f9007516b80 1 --2- 172.21.15.46:0/4122764874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321c63400 0x55d321c75ef0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f8fec0036f0 tx=0x7f8fec003a00 comp rx=0 tx=0).stop 2024-05-09T04:57:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f9007516b80 1 -- 172.21.15.46:0/4122764874 shutdown_connections 2024-05-09T04:57:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f9007516b80 1 --2- 172.21.15.46:0/4122764874 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55d321ce6cb0 0x55d321ce90d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f8fd9ffb700 0 client.16719 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:57:01.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f9007516b80 1 --2- 172.21.15.46:0/4122764874 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d321c63400 0x55d321c75ef0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f9007516b80 1 --2- 172.21.15.46:0/4122764874 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d321ae64a0 0x55d321c79520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f9007516b80 1 --2- 172.21.15.46:0/4122764874 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d321b60d00 0x55d321c758a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f9007516b80 1 -- 172.21.15.46:0/4122764874 >> 172.21.15.46:0/4122764874 conn(0x55d321ae9f70 msgr2=0x55d321c66cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f9007516b80 1 -- 172.21.15.46:0/4122764874 shutdown_connections 2024-05-09T04:57:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.702+0000 7f9007516b80 1 -- 172.21.15.46:0/4122764874 wait complete. 2024-05-09T04:57:01.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.703+0000 7f9007516b80 20 client.16719 trim_cache size 0 max 16384 2024-05-09T04:57:01.708 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:57:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.703+0000 7f8ff8b00700 1 -- 172.21.15.46:0/794100214 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8fe00b29f0 msgr2=0x7f8fe00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.703+0000 7f8ff8b00700 1 --2- 172.21.15.46:0/794100214 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8fe00b29f0 0x7f8fe00b4f00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4103470 tx=0x7f8fdc00b410 comp rx=0 tx=0).stop 2024-05-09T04:57:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.703+0000 7f8ff8b00700 1 -- 172.21.15.46:0/794100214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff41045e0 msgr2=0x7f8ff419ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.703+0000 7f8ff8b00700 1 --2- 172.21.15.46:0/794100214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff41045e0 0x7f8ff419ade0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4007a20 tx=0x7f8fe4007a50 comp rx=0 tx=0).stop 2024-05-09T04:57:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.703+0000 7f8ff8b00700 1 -- 172.21.15.46:0/794100214 shutdown_connections 2024-05-09T04:57:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.703+0000 7f8ff8b00700 1 --2- 172.21.15.46:0/794100214 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f8fe00b29f0 0x7f8fe00b4f00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.703+0000 7f8ff8b00700 1 --2- 172.21.15.46:0/794100214 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8ff4109860 0x7f8ff419f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.703+0000 7f8ff8b00700 1 --2- 172.21.15.46:0/794100214 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8ff41045e0 0x7f8ff419ade0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.703+0000 7f8ff8b00700 1 --2- 172.21.15.46:0/794100214 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8ff4108f10 0x7f8ff419b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:01.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.703+0000 7f8ff8b00700 1 -- 172.21.15.46:0/794100214 >> 172.21.15.46:0/794100214 conn(0x7f8ff40ffed0 msgr2=0x7f8ff41086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.703+0000 7f8ff8b00700 1 -- 172.21.15.46:0/794100214 shutdown_connections 2024-05-09T04:57:01.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:01.703+0000 7f8ff8b00700 1 -- 172.21.15.46:0/794100214 wait complete. 2024-05-09T04:57:02.497 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:57:02.757 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:57:03.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:02 smithi046 ceph-mon[30136]: pgmap v181: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T04:57:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:02 smithi072 ceph-mon[34088]: pgmap v181: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T04:57:03.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:02 smithi110 ceph-mon[34904]: pgmap v181: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 12 KiB/s wr, 9 op/s 2024-05-09T04:57:04.455 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.454+0000 7f34c226d700 1 -- 172.21.15.46:0/2858358114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34bc1045e0 msgr2=0x7f34bc1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.454+0000 7f34c226d700 1 --2- 172.21.15.46:0/2858358114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34bc1045e0 0x7f34bc1049f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f34ac0099c0 tx=0x7f34ac009cd0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.455+0000 7f34c226d700 1 -- 172.21.15.46:0/2858358114 shutdown_connections 2024-05-09T04:57:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.455+0000 7f34c226d700 1 --2- 172.21.15.46:0/2858358114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34bc109860 0x7f34bc1111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.455+0000 7f34c226d700 1 --2- 172.21.15.46:0/2858358114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34bc1045e0 0x7f34bc1049f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.455+0000 7f34c226d700 1 --2- 172.21.15.46:0/2858358114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34bc108f10 0x7f34bc109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.455+0000 7f34c226d700 1 -- 172.21.15.46:0/2858358114 >> 172.21.15.46:0/2858358114 conn(0x7f34bc0ffe70 msgr2=0x7f34bc1022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.455+0000 7f34c226d700 1 -- 172.21.15.46:0/2858358114 shutdown_connections 2024-05-09T04:57:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.455+0000 7f34c226d700 1 -- 172.21.15.46:0/2858358114 wait complete. 2024-05-09T04:57:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.455+0000 7f34c226d700 1 Processor -- start 2024-05-09T04:57:04.456 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.455+0000 7f34c226d700 1 -- start start 2024-05-09T04:57:04.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.456+0000 7f34c226d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34bc1045e0 0x7f34bc075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.456+0000 7f34c226d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34bc108f10 0x7f34bc076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.456+0000 7f34c226d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34bc109860 0x7f34bc076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.456+0000 7f34c226d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc114490 con 0x7f34bc109860 2024-05-09T04:57:04.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.456+0000 7f34c226d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc114310 con 0x7f34bc1045e0 2024-05-09T04:57:04.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.456+0000 7f34c226d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f34bc114610 con 0x7f34bc108f10 2024-05-09T04:57:04.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.456+0000 7f34c080a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34bc109860 0x7f34bc076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.456+0000 7f34bbfff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34bc1045e0 0x7f34bc075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.456+0000 7f34c080a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34bc109860 0x7f34bc076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39004/0 (socket says 172.21.15.46:39004) 2024-05-09T04:57:04.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.456+0000 7f34c080a700 1 -- 172.21.15.46:0/834340907 learned_addr learned my addr 172.21.15.46:0/834340907 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:04.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.456+0000 7f34bb7fe700 1 --2- 172.21.15.46:0/834340907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34bc108f10 0x7f34bc076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.457+0000 7f34c080a700 1 -- 172.21.15.46:0/834340907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34bc108f10 msgr2=0x7f34bc076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:04.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.457+0000 7f34c080a700 1 --2- 172.21.15.46:0/834340907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34bc108f10 0x7f34bc076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.457+0000 7f34c080a700 1 -- 172.21.15.46:0/834340907 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34bc1045e0 msgr2=0x7f34bc075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:04.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.457+0000 7f34c080a700 1 --2- 172.21.15.46:0/834340907 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34bc1045e0 0x7f34bc075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.457+0000 7f34c080a700 1 -- 172.21.15.46:0/834340907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34bc1a91c0 con 0x7f34bc109860 2024-05-09T04:57:04.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.457+0000 7f34bbfff700 1 --2- 172.21.15.46:0/834340907 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34bc1045e0 0x7f34bc075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:04.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.457+0000 7f34c080a700 1 --2- 172.21.15.46:0/834340907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34bc109860 0x7f34bc076540 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f34b4007ee0 tx=0x7f34b400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:04.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.457+0000 7f34b97fa700 1 -- 172.21.15.46:0/834340907 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4004bb0 con 0x7f34bc109860 2024-05-09T04:57:04.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.457+0000 7f34b97fa700 1 -- 172.21.15.46:0/834340907 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34b4004d50 con 0x7f34bc109860 2024-05-09T04:57:04.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.457+0000 7f34c226d700 1 -- 172.21.15.46:0/834340907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34bc1a94b0 con 0x7f34bc109860 2024-05-09T04:57:04.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.457+0000 7f34c226d700 1 -- 172.21.15.46:0/834340907 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34bc1a99f0 con 0x7f34bc109860 2024-05-09T04:57:04.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.457+0000 7f34b97fa700 1 -- 172.21.15.46:0/834340907 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b4005920 con 0x7f34bc109860 2024-05-09T04:57:04.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.461+0000 7f34b97fa700 1 -- 172.21.15.46:0/834340907 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f34b401a020 con 0x7f34bc109860 2024-05-09T04:57:04.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.461+0000 7f34b97fa700 1 --2- 172.21.15.46:0/834340907 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f34a40b2a70 0x7f34a40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34d0c83b80 1 Processor -- start 2024-05-09T04:57:04.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34d0c83b80 1 -- start start 2024-05-09T04:57:04.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34bbfff700 1 --2- 172.21.15.46:0/834340907 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f34a40b2a70 0x7f34a40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34b97fa700 1 -- 172.21.15.46:0/834340907 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f34b40e1b20 con 0x7f34bc109860 2024-05-09T04:57:04.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34d0c83b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b86802d00 0x556b8690b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34d0c83b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b867884a0 0x556b8690b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34d0c83b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 0x556b86905810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34d0c83b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556b868280a0 con 0x556b867884a0 2024-05-09T04:57:04.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34d0c83b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556b867cd9a0 con 0x556b86905400 2024-05-09T04:57:04.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34d0c83b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556b86824060 con 0x556b86802d00 2024-05-09T04:57:04.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34bb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b867884a0 0x556b8690b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34bb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b867884a0 0x556b8690b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39016/0 (socket says 172.21.15.46:39016) 2024-05-09T04:57:04.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34bb7fe700 1 -- 172.21.15.46:0/1104613761 learned_addr learned my addr 172.21.15.46:0/1104613761 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:04.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34bbfff700 1 --2- 172.21.15.46:0/1104613761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b86802d00 0x556b8690b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34bbfff700 1 --2- 172.21.15.46:0/834340907 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f34a40b2a70 0x7f34a40b4f80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f34ac000c00 tx=0x7f34ac01a040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:04.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34c080a700 1 --2- 172.21.15.46:0/1104613761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 0x556b86905810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34bb7fe700 1 -- 172.21.15.46:0/1104613761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b86802d00 msgr2=0x556b8690b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:04.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34bb7fe700 1 --2- 172.21.15.46:0/1104613761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b86802d00 0x556b8690b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.462+0000 7f34bb7fe700 1 -- 172.21.15.46:0/1104613761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 msgr2=0x556b86905810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:04.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.463+0000 7f34bb7fe700 1 --2- 172.21.15.46:0/1104613761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 0x556b86905810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.463+0000 7f34bb7fe700 1 -- 172.21.15.46:0/1104613761 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b867be500 con 0x556b867884a0 2024-05-09T04:57:04.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.463+0000 7f34bb7fe700 1 --2- 172.21.15.46:0/1104613761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b867884a0 0x556b8690b780 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009cf0 tx=0x7f34b0011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45f076974432dafc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:04.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.463+0000 7f34a27fc700 1 -- 172.21.15.46:0/1104613761 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0011d80 con 0x556b867884a0 2024-05-09T04:57:04.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.463+0000 7f34a27fc700 1 -- 172.21.15.46:0/1104613761 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34b0004500 con 0x556b867884a0 2024-05-09T04:57:04.465 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.463+0000 7f34a27fc700 1 -- 172.21.15.46:0/1104613761 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0005310 con 0x556b867884a0 2024-05-09T04:57:04.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.464+0000 7f34d0c83b80 1 -- 172.21.15.46:0/1104613761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b867884a0 msgr2=0x556b8690b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:04.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.464+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/1104613761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b867884a0 0x556b8690b780 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009cf0 tx=0x7f34b0011440 comp rx=0 tx=0).stop 2024-05-09T04:57:04.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.464+0000 7f34d0c83b80 1 -- 172.21.15.46:0/1104613761 shutdown_connections 2024-05-09T04:57:04.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.464+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/1104613761 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 0x556b86905810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.464+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/1104613761 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b86802d00 0x556b8690b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.464+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/1104613761 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b867884a0 0x556b8690b780 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.464+0000 7f34d0c83b80 1 -- 172.21.15.46:0/1104613761 >> 172.21.15.46:0/1104613761 conn(0x556b8678bf70 msgr2=0x556b86900220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:04.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.464+0000 7f34d0c83b80 1 -- 172.21.15.46:0/1104613761 shutdown_connections 2024-05-09T04:57:04.468 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.464+0000 7f34d0c83b80 1 -- 172.21.15.46:0/1104613761 wait complete. 2024-05-09T04:57:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.464+0000 7f34d0c83b80 1 -- start start 2024-05-09T04:57:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34d0c83b80 1 Processor -- start 2024-05-09T04:57:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34d0c83b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b867884a0 0x556b8690eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34d0c83b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b86802d00 0x556b8690f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34d0c83b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86905400 0x556b8690f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34d0c83b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556b86906160 con 0x556b86905400 2024-05-09T04:57:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34d0c83b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556b867c2b20 con 0x556b867884a0 2024-05-09T04:57:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34d0c83b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556b86912020 con 0x556b86802d00 2024-05-09T04:57:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34d0c83b80 10 client.0 _ll_register_callbacks cb 0x556b866996c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34c080a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86905400 0x556b8690f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34c080a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86905400 0x556b8690f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39026/0 (socket says 172.21.15.46:39026) 2024-05-09T04:57:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34c080a700 1 -- 172.21.15.46:0/4206068432 learned_addr learned my addr 172.21.15.46:0/4206068432 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34bbfff700 1 --2- 172.21.15.46:0/4206068432 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b867884a0 0x556b8690eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34bb7fe700 1 --2- 172.21.15.46:0/4206068432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b86802d00 0x556b8690f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.469 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34c080a700 1 -- 172.21.15.46:0/4206068432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b86802d00 msgr2=0x556b8690f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34c080a700 1 --2- 172.21.15.46:0/4206068432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b86802d00 0x556b8690f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34c080a700 1 -- 172.21.15.46:0/4206068432 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b867884a0 msgr2=0x556b8690eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34c080a700 1 --2- 172.21.15.46:0/4206068432 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b867884a0 0x556b8690eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.465+0000 7f34c080a700 1 -- 172.21.15.46:0/4206068432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b866a3490 con 0x556b86905400 2024-05-09T04:57:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f34c080a700 1 --2- 172.21.15.46:0/4206068432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86905400 0x556b8690f950 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f34b40a8d70 tx=0x7f34b40898a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0948ac3edf4d8ea server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f349b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f349b7fe700 1 -- 172.21.15.46:0/4206068432 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b40ac070 con 0x556b86905400 2024-05-09T04:57:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f349b7fe700 1 -- 172.21.15.46:0/4206068432 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f34b40b19c0 con 0x556b86905400 2024-05-09T04:57:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f34d0c83b80 1 -- 172.21.15.46:0/4206068432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34b0004500 con 0x556b86905400 2024-05-09T04:57:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f349b7fe700 1 -- 172.21.15.46:0/4206068432 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f34b4005ac0 con 0x556b86905400 2024-05-09T04:57:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f34d0c83b80 10 client.16731 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f34d0c83b80 1 -- 172.21.15.46:0/4206068432 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b8691c990 con 0x556b86905400 2024-05-09T04:57:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f349b7fe700 1 -- 172.21.15.46:0/4206068432 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34b0004500 con 0x556b86905400 2024-05-09T04:57:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f349b7fe700 1 -- 172.21.15.46:0/4206068432 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f34b4005000 con 0x556b86905400 2024-05-09T04:57:04.470 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f34d0c83b80 10 client.16731 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f34d0c83b80 10 client.16731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:57:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f34d0c83b80 20 client.16731 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/4206068432 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8691ef20 0x556b86921390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f34d0c83b80 4 client.16731 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:57:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.466+0000 7f34d0c83b80 1 -- 172.21.15.46:0/4206068432 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b86835a80 con 0x556b8691ef20 2024-05-09T04:57:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.467+0000 7f34bbfff700 1 --2- 172.21.15.46:0/4206068432 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8691ef20 0x556b86921390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.467+0000 7f34bbfff700 1 --2- 172.21.15.46:0/4206068432 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8691ef20 0x556b86921390 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b82b5da96d862a6 server_cookie=25e1c26ef9c1ab3e in_seq=0 out_seq=0 2024-05-09T04:57:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.467+0000 7f349b7fe700 10 client.16731 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.468+0000 7f349b7fe700 1 -- 172.21.15.46:0/4206068432 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556b86835a80 con 0x556b8691ef20 2024-05-09T04:57:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.468+0000 7f349b7fe700 10 client.16731 handle_command_reply: tid=0 2024-05-09T04:57:04.471 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.468+0000 7f34d0c83b80 1 client.16731 shutdown 2024-05-09T04:57:04.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.468+0000 7f34d0c83b80 1 -- 172.21.15.46:0/4206068432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86905400 msgr2=0x556b8690f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:04.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.468+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/4206068432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86905400 0x556b8690f950 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f34b40a8d70 tx=0x7f34b40898a0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.468+0000 7f34d0c83b80 1 -- 172.21.15.46:0/4206068432 shutdown_connections 2024-05-09T04:57:04.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.468+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/4206068432 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86905400 0x556b8690f950 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.468+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/4206068432 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8691ef20 0x556b86921390 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.468+0000 7f349b7fe700 0 client.16731 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:04.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.468+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/4206068432 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b867884a0 0x556b8690eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.468+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/4206068432 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b86802d00 0x556b8690f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.468+0000 7f34d0c83b80 1 -- 172.21.15.46:0/4206068432 >> 172.21.15.46:0/4206068432 conn(0x556b8678bf70 msgr2=0x556b868f9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.468+0000 7f34d0c83b80 1 -- 172.21.15.46:0/4206068432 shutdown_connections 2024-05-09T04:57:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.468+0000 7f34d0c83b80 1 -- 172.21.15.46:0/4206068432 wait complete. 2024-05-09T04:57:04.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.469+0000 7f34d0c83b80 20 client.16731 trim_cache size 0 max 16384 2024-05-09T04:57:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.484+0000 7f34d0c83b80 1 Processor -- start 2024-05-09T04:57:04.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.484+0000 7f34d0c83b80 1 -- start start 2024-05-09T04:57:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.484+0000 7f34d0c83b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b867884a0 0x556b8690f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.484+0000 7f34d0c83b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86802d00 0x556b869093b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.484+0000 7f34d0c83b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 0x556b8691aef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.484+0000 7f34d0c83b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34a800ff40 con 0x556b86802d00 2024-05-09T04:57:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.484+0000 7f34d0c83b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f34a800d7d0 con 0x556b86905400 2024-05-09T04:57:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.484+0000 7f34d0c83b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f34a800b620 con 0x556b867884a0 2024-05-09T04:57:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.484+0000 7f34bb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86802d00 0x556b869093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.484+0000 7f34bb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86802d00 0x556b869093b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39038/0 (socket says 172.21.15.46:39038) 2024-05-09T04:57:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.484+0000 7f34bb7fe700 1 -- 172.21.15.46:0/2759242940 learned_addr learned my addr 172.21.15.46:0/2759242940 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.484+0000 7f34bbfff700 1 --2- 172.21.15.46:0/2759242940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b867884a0 0x556b8690f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.485+0000 7f34c080a700 1 --2- 172.21.15.46:0/2759242940 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 0x556b8691aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.485+0000 7f34bb7fe700 1 -- 172.21.15.46:0/2759242940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b867884a0 msgr2=0x556b8690f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.485+0000 7f34bb7fe700 1 --2- 172.21.15.46:0/2759242940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b867884a0 0x556b8690f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.485+0000 7f34bb7fe700 1 -- 172.21.15.46:0/2759242940 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 msgr2=0x556b8691aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:04.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.485+0000 7f34bb7fe700 1 --2- 172.21.15.46:0/2759242940 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 0x556b8691aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.485+0000 7f34bb7fe700 1 -- 172.21.15.46:0/2759242940 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b869065d0 con 0x556b86802d00 2024-05-09T04:57:04.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.485+0000 7f34c080a700 1 --2- 172.21.15.46:0/2759242940 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 0x556b8691aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:57:04.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.485+0000 7f34bb7fe700 1 --2- 172.21.15.46:0/2759242940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86802d00 0x556b869093b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x556b86901c90 tx=0x7f34b0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc14a442ee35315 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:04.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.485+0000 7f34a1ffb700 1 -- 172.21.15.46:0/2759242940 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0011c80 con 0x556b86802d00 2024-05-09T04:57:04.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.485+0000 7f34a1ffb700 1 -- 172.21.15.46:0/2759242940 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34b0005560 con 0x556b86802d00 2024-05-09T04:57:04.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.485+0000 7f34a1ffb700 1 -- 172.21.15.46:0/2759242940 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0023470 con 0x556b86802d00 2024-05-09T04:57:04.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.486+0000 7f34d0c83b80 1 -- 172.21.15.46:0/2759242940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86802d00 msgr2=0x556b869093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:04.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.486+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/2759242940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86802d00 0x556b869093b0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x556b86901c90 tx=0x7f34b0009750 comp rx=0 tx=0).stop 2024-05-09T04:57:04.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.486+0000 7f34d0c83b80 1 -- 172.21.15.46:0/2759242940 shutdown_connections 2024-05-09T04:57:04.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.486+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/2759242940 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 0x556b8691aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.486+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/2759242940 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b867884a0 0x556b8690f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.486+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/2759242940 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86802d00 0x556b869093b0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.486+0000 7f34d0c83b80 1 -- 172.21.15.46:0/2759242940 >> 172.21.15.46:0/2759242940 conn(0x556b8678bf70 msgr2=0x556b86908ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:04.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.486+0000 7f34d0c83b80 1 -- 172.21.15.46:0/2759242940 shutdown_connections 2024-05-09T04:57:04.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.486+0000 7f34d0c83b80 1 -- 172.21.15.46:0/2759242940 wait complete. 2024-05-09T04:57:04.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.486+0000 7f34d0c83b80 1 -- start start 2024-05-09T04:57:04.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.487+0000 7f34d0c83b80 1 Processor -- start 2024-05-09T04:57:04.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.487+0000 7f34d0c83b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b867884a0 0x556b86917270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.487+0000 7f34d0c83b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86802d00 0x556b869177b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.487+0000 7f34d0c83b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 0x556b86925b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.487+0000 7f34d0c83b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f34a800bc90 con 0x556b86802d00 2024-05-09T04:57:04.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.487+0000 7f34d0c83b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f34a800d100 con 0x556b86905400 2024-05-09T04:57:04.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.487+0000 7f34d0c83b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f34a800c360 con 0x556b867884a0 2024-05-09T04:57:04.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.487+0000 7f34d0c83b80 10 client.0 _ll_register_callbacks cb 0x556b866996c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:04.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.487+0000 7f34bb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86802d00 0x556b869177b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.487+0000 7f34bb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86802d00 0x556b869177b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39052/0 (socket says 172.21.15.46:39052) 2024-05-09T04:57:04.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.487+0000 7f34bb7fe700 1 -- 172.21.15.46:0/390541572 learned_addr learned my addr 172.21.15.46:0/390541572 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:04.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.487+0000 7f34bbfff700 1 --2- 172.21.15.46:0/390541572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b867884a0 0x556b86917270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.487+0000 7f34c080a700 1 --2- 172.21.15.46:0/390541572 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 0x556b86925b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f34bb7fe700 1 -- 172.21.15.46:0/390541572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b867884a0 msgr2=0x556b86917270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:04.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f34bb7fe700 1 --2- 172.21.15.46:0/390541572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b867884a0 0x556b86917270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f34bb7fe700 1 -- 172.21.15.46:0/390541572 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 msgr2=0x556b86925b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:04.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f34bb7fe700 1 --2- 172.21.15.46:0/390541572 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 0x556b86925b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f34bb7fe700 1 -- 172.21.15.46:0/390541572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b86926290 con 0x556b86802d00 2024-05-09T04:57:04.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f34c080a700 1 --2- 172.21.15.46:0/390541572 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 0x556b86925b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:57:04.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f34bbfff700 1 --2- 172.21.15.46:0/390541572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b867884a0 0x556b86917270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:04.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f34bb7fe700 1 --2- 172.21.15.46:0/390541572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86802d00 0x556b869177b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009cc0 tx=0x7f34b00039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d131bd92ec8c609 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:04.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f349b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:04.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f349b7fe700 1 -- 172.21.15.46:0/390541572 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34b0033060 con 0x556b86802d00 2024-05-09T04:57:04.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f349b7fe700 1 -- 172.21.15.46:0/390541572 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f34b00050b0 con 0x556b86802d00 2024-05-09T04:57:04.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f34d0c83b80 1 -- 172.21.15.46:0/390541572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556b8682fb20 con 0x556b86802d00 2024-05-09T04:57:04.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f349b7fe700 1 -- 172.21.15.46:0/390541572 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f34b00052d0 con 0x556b86802d00 2024-05-09T04:57:04.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f34d0c83b80 10 client.16737 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:04.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f34d0c83b80 1 -- 172.21.15.46:0/390541572 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556b86989020 con 0x556b86802d00 2024-05-09T04:57:04.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.488+0000 7f349b7fe700 1 -- 172.21.15.46:0/390541572 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556b8682fb20 con 0x556b86802d00 2024-05-09T04:57:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.489+0000 7f349b7fe700 1 -- 172.21.15.46:0/390541572 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f34b0018070 con 0x556b86802d00 2024-05-09T04:57:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.489+0000 7f34d0c83b80 10 client.16737 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.489+0000 7f34d0c83b80 10 client.16737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:57:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.489+0000 7f34d0c83b80 20 client.16737 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.489+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/390541572 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8691ef20 0x556b8698b490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.489+0000 7f34d0c83b80 4 client.16737 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:57:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.489+0000 7f34d0c83b80 1 -- 172.21.15.46:0/390541572 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556b866a3970 con 0x556b8691ef20 2024-05-09T04:57:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.489+0000 7f34bbfff700 1 --2- 172.21.15.46:0/390541572 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8691ef20 0x556b8698b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.489+0000 7f349b7fe700 10 client.16737 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.489+0000 7f34bbfff700 1 --2- 172.21.15.46:0/390541572 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8691ef20 0x556b8698b490 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82d9dede500857ef server_cookie=a5b0beaad6ab13ee in_seq=0 out_seq=0 2024-05-09T04:57:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.489+0000 7f349b7fe700 1 -- 172.21.15.46:0/390541572 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556b866a3970 con 0x556b8691ef20 2024-05-09T04:57:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.489+0000 7f349b7fe700 10 client.16737 handle_command_reply: tid=0 2024-05-09T04:57:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.489+0000 7f34d0c83b80 1 client.16737 shutdown 2024-05-09T04:57:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f34d0c83b80 1 -- 172.21.15.46:0/390541572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86802d00 msgr2=0x556b869177b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:04.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/390541572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86802d00 0x556b869177b0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f34b0009cc0 tx=0x7f34b00039f0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f34d0c83b80 1 -- 172.21.15.46:0/390541572 shutdown_connections 2024-05-09T04:57:04.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/390541572 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x556b8691ef20 0x556b8698b490 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f349b7fe700 0 client.16737 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:04.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/390541572 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556b86905400 0x556b86925b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/390541572 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556b867884a0 0x556b86917270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f34d0c83b80 1 --2- 172.21.15.46:0/390541572 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556b86802d00 0x556b869177b0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f34d0c83b80 1 -- 172.21.15.46:0/390541572 >> 172.21.15.46:0/390541572 conn(0x556b8678bf70 msgr2=0x556b8690a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:04.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f34d0c83b80 1 -- 172.21.15.46:0/390541572 shutdown_connections 2024-05-09T04:57:04.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f34d0c83b80 1 -- 172.21.15.46:0/390541572 wait complete. 2024-05-09T04:57:04.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f34d0c83b80 20 client.16737 trim_cache size 0 max 16384 2024-05-09T04:57:04.495 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:57:04.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f34c226d700 1 -- 172.21.15.46:0/834340907 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f34a40b2a70 msgr2=0x7f34a40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:04.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f34c226d700 1 --2- 172.21.15.46:0/834340907 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f34a40b2a70 0x7f34a40b4f80 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f34ac000c00 tx=0x7f34ac01a040 comp rx=0 tx=0).stop 2024-05-09T04:57:04.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f34c226d700 1 -- 172.21.15.46:0/834340907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34bc109860 msgr2=0x7f34bc076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:04.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.490+0000 7f34c226d700 1 --2- 172.21.15.46:0/834340907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34bc109860 0x7f34bc076540 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f34b4007ee0 tx=0x7f34b400d3b0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.491+0000 7f34c226d700 1 -- 172.21.15.46:0/834340907 shutdown_connections 2024-05-09T04:57:04.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.491+0000 7f34c226d700 1 --2- 172.21.15.46:0/834340907 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f34a40b2a70 0x7f34a40b4f80 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.491+0000 7f34c226d700 1 --2- 172.21.15.46:0/834340907 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f34bc109860 0x7f34bc076540 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.491+0000 7f34c226d700 1 --2- 172.21.15.46:0/834340907 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f34bc1045e0 0x7f34bc075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.491+0000 7f34c226d700 1 --2- 172.21.15.46:0/834340907 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f34bc108f10 0x7f34bc076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:04.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.491+0000 7f34c226d700 1 -- 172.21.15.46:0/834340907 >> 172.21.15.46:0/834340907 conn(0x7f34bc0ffe70 msgr2=0x7f34bc1071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:04.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.491+0000 7f34c226d700 1 -- 172.21.15.46:0/834340907 shutdown_connections 2024-05-09T04:57:04.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:04.491+0000 7f34c226d700 1 -- 172.21.15.46:0/834340907 wait complete. 2024-05-09T04:57:05.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:04 smithi046 ceph-mon[30136]: pgmap v182: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-09T04:57:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:04 smithi072 ceph-mon[34088]: pgmap v182: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-09T04:57:05.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:04 smithi110 ceph-mon[34904]: pgmap v182: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-09T04:57:06.321 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:57:06.322 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:57:06.590 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:57:07.042 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:06 smithi046 ceph-mon[30136]: pgmap v183: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-05-09T04:57:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:06 smithi072 ceph-mon[34088]: pgmap v183: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-05-09T04:57:07.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:06 smithi110 ceph-mon[34904]: pgmap v183: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-05-09T04:57:08.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:07 smithi046 ceph-mon[30136]: pgmap v184: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-09T04:57:08.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:07 smithi072 ceph-mon[34088]: pgmap v184: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-09T04:57:08.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:07 smithi110 ceph-mon[34904]: pgmap v184: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 22 op/s 2024-05-09T04:57:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.338+0000 7f966b89f700 1 -- 172.21.15.46:0/3536780741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9664108760 msgr2=0x7f96641100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.338+0000 7f966b89f700 1 --2- 172.21.15.46:0/3536780741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9664108760 0x7f96641100c0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f965c00b0d0 tx=0x7f965c00b3e0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.338+0000 7f966b89f700 1 -- 172.21.15.46:0/3536780741 shutdown_connections 2024-05-09T04:57:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.338+0000 7f966b89f700 1 --2- 172.21.15.46:0/3536780741 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9664108760 0x7f96641100c0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.338+0000 7f966b89f700 1 --2- 172.21.15.46:0/3536780741 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96641034e0 0x7f96641038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.338+0000 7f966b89f700 1 --2- 172.21.15.46:0/3536780741 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9664107e10 0x7f9664108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.338+0000 7f966b89f700 1 -- 172.21.15.46:0/3536780741 >> 172.21.15.46:0/3536780741 conn(0x7f96640fef30 msgr2=0x7f9664101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.338+0000 7f966b89f700 1 -- 172.21.15.46:0/3536780741 shutdown_connections 2024-05-09T04:57:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.338+0000 7f966b89f700 1 -- 172.21.15.46:0/3536780741 wait complete. 2024-05-09T04:57:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.339+0000 7f966b89f700 1 Processor -- start 2024-05-09T04:57:08.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.339+0000 7f966b89f700 1 -- start start 2024-05-09T04:57:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.339+0000 7f966b89f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96641034e0 0x7f9664199da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.339+0000 7f966b89f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9664107e10 0x7f966419a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.339+0000 7f966b89f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9664108760 0x7f966419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.339+0000 7f966b89f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9664113390 con 0x7f9664107e10 2024-05-09T04:57:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.339+0000 7f966b89f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9664113210 con 0x7f9664108760 2024-05-09T04:57:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.339+0000 7f966b89f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9664113510 con 0x7f96641034e0 2024-05-09T04:57:08.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.339+0000 7f9668e3a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9664107e10 0x7f966419a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.339+0000 7f9668e3a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9664107e10 0x7f966419a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39068/0 (socket says 172.21.15.46:39068) 2024-05-09T04:57:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.339+0000 7f9668e3a700 1 -- 172.21.15.46:0/2773798944 learned_addr learned my addr 172.21.15.46:0/2773798944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.339+0000 7f966963b700 1 --2- 172.21.15.46:0/2773798944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96641034e0 0x7f9664199da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.340+0000 7f9669e3c700 1 --2- 172.21.15.46:0/2773798944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9664108760 0x7f966419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.340+0000 7f9668e3a700 1 -- 172.21.15.46:0/2773798944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96641034e0 msgr2=0x7f9664199da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.340+0000 7f9668e3a700 1 --2- 172.21.15.46:0/2773798944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96641034e0 0x7f9664199da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.340+0000 7f9668e3a700 1 -- 172.21.15.46:0/2773798944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9664108760 msgr2=0x7f966419e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.340+0000 7f9668e3a700 1 --2- 172.21.15.46:0/2773798944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9664108760 0x7f966419e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.340+0000 7f9668e3a700 1 -- 172.21.15.46:0/2773798944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f966419ebc0 con 0x7f9664107e10 2024-05-09T04:57:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.340+0000 7f9669e3c700 1 --2- 172.21.15.46:0/2773798944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9664108760 0x7f966419e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:57:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.340+0000 7f9668e3a700 1 --2- 172.21.15.46:0/2773798944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9664107e10 0x7f966419a340 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f965400c9e0 tx=0x7f965400ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.340+0000 7f965a7fc700 1 -- 172.21.15.46:0/2773798944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654007aa0 con 0x7f9664107e10 2024-05-09T04:57:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.340+0000 7f965a7fc700 1 -- 172.21.15.46:0/2773798944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9654007c40 con 0x7f9664107e10 2024-05-09T04:57:08.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.340+0000 7f966b89f700 1 -- 172.21.15.46:0/2773798944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f966419eeb0 con 0x7f9664107e10 2024-05-09T04:57:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.340+0000 7f965a7fc700 1 -- 172.21.15.46:0/2773798944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9654018660 con 0x7f9664107e10 2024-05-09T04:57:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.340+0000 7f966b89f700 1 -- 172.21.15.46:0/2773798944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96641a6880 con 0x7f9664107e10 2024-05-09T04:57:08.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.343+0000 7f965a7fc700 1 -- 172.21.15.46:0/2773798944 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9654004830 con 0x7f9664107e10 2024-05-09T04:57:08.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.344+0000 7f965a7fc700 1 --2- 172.21.15.46:0/2773798944 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f96500b2a70 0x7f96500b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.344+0000 7f967a2b5b80 1 Processor -- start 2024-05-09T04:57:08.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.344+0000 7f966963b700 1 --2- 172.21.15.46:0/2773798944 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f96500b2a70 0x7f96500b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.344+0000 7f967a2b5b80 1 -- start start 2024-05-09T04:57:08.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.344+0000 7f965a7fc700 1 -- 172.21.15.46:0/2773798944 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f96540e2550 con 0x7f9664107e10 2024-05-09T04:57:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.344+0000 7f967a2b5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b853340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.344+0000 7f967a2b5b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 0x559a9b84d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.344+0000 7f967a2b5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b84ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.344+0000 7f967a2b5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559a9b770120 con 0x559a9b74ad80 2024-05-09T04:57:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.344+0000 7f967a2b5b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559a9b715a20 con 0x559a9b6d0520 2024-05-09T04:57:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.344+0000 7f967a2b5b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559a9b76c0e0 con 0x559a9b84d9c0 2024-05-09T04:57:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f966963b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b853340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f966963b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b853340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39082/0 (socket says 172.21.15.46:39082) 2024-05-09T04:57:08.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f966963b700 1 -- 172.21.15.46:0/1525538676 learned_addr learned my addr 172.21.15.46:0/1525538676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f966963b700 1 --2- 172.21.15.46:0/2773798944 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f96500b2a70 0x7f96500b4f80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9660005950 tx=0x7f966000a300 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f9668e3a700 1 --2- 172.21.15.46:0/1525538676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 0x559a9b84d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f9669e3c700 1 --2- 172.21.15.46:0/1525538676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b84ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f966963b700 1 -- 172.21.15.46:0/1525538676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 msgr2=0x559a9b84ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f966963b700 1 --2- 172.21.15.46:0/1525538676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b84ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f966963b700 1 -- 172.21.15.46:0/1525538676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 msgr2=0x559a9b84d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f966963b700 1 --2- 172.21.15.46:0/1525538676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 0x559a9b84d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f966963b700 1 -- 172.21.15.46:0/1525538676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a9b706580 con 0x559a9b74ad80 2024-05-09T04:57:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f9668e3a700 1 --2- 172.21.15.46:0/1525538676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 0x559a9b84d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:57:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f966963b700 1 --2- 172.21.15.46:0/1525538676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b853340 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f966000fe20 tx=0x7f9660012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=202fa04d2ccc6192 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f964f7fe700 1 -- 172.21.15.46:0/1525538676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660016070 con 0x559a9b74ad80 2024-05-09T04:57:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f964f7fe700 1 -- 172.21.15.46:0/1525538676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f966001c420 con 0x559a9b74ad80 2024-05-09T04:57:08.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.345+0000 7f964f7fe700 1 -- 172.21.15.46:0/1525538676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f966001b680 con 0x559a9b74ad80 2024-05-09T04:57:08.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.346+0000 7f967a2b5b80 1 -- 172.21.15.46:0/1525538676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 msgr2=0x559a9b853340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.346+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/1525538676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b853340 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f966000fe20 tx=0x7f9660012000 comp rx=0 tx=0).stop 2024-05-09T04:57:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.346+0000 7f967a2b5b80 1 -- 172.21.15.46:0/1525538676 shutdown_connections 2024-05-09T04:57:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.346+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/1525538676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b84ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.346+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/1525538676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b853340 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.346+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/1525538676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 0x559a9b84d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.346+0000 7f967a2b5b80 1 -- 172.21.15.46:0/1525538676 >> 172.21.15.46:0/1525538676 conn(0x559a9b6d3ff0 msgr2=0x559a9b8482a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.346+0000 7f967a2b5b80 1 -- 172.21.15.46:0/1525538676 shutdown_connections 2024-05-09T04:57:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.346+0000 7f967a2b5b80 1 -- 172.21.15.46:0/1525538676 wait complete. 2024-05-09T04:57:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.347+0000 7f967a2b5b80 1 -- start start 2024-05-09T04:57:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.347+0000 7f967a2b5b80 1 Processor -- start 2024-05-09T04:57:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.347+0000 7f967a2b5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b6d0520 0x559a9b85f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.347+0000 7f967a2b5b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b74ad80 0x559a9b85f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.347+0000 7f967a2b5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b85fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.347+0000 7f967a2b5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x559a9b862620 con 0x559a9b6d0520 2024-05-09T04:57:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.347+0000 7f967a2b5b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x559a9b70aba0 con 0x559a9b74ad80 2024-05-09T04:57:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.347+0000 7f967a2b5b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x559a9b8627a0 con 0x559a9b84d9c0 2024-05-09T04:57:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f967a2b5b80 10 client.0 _ll_register_callbacks cb 0x559a9b5e1740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f966963b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b6d0520 0x559a9b85f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f966963b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b6d0520 0x559a9b85f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39090/0 (socket says 172.21.15.46:39090) 2024-05-09T04:57:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f966963b700 1 -- 172.21.15.46:0/3196112501 learned_addr learned my addr 172.21.15.46:0/3196112501 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f9669e3c700 1 --2- 172.21.15.46:0/3196112501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b85fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f9668e3a700 1 --2- 172.21.15.46:0/3196112501 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b74ad80 0x559a9b85f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f966963b700 1 -- 172.21.15.46:0/3196112501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 msgr2=0x559a9b85fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f966963b700 1 --2- 172.21.15.46:0/3196112501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b85fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f966963b700 1 -- 172.21.15.46:0/3196112501 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b74ad80 msgr2=0x559a9b85f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f966963b700 1 --2- 172.21.15.46:0/3196112501 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b74ad80 0x559a9b85f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f966963b700 1 -- 172.21.15.46:0/3196112501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a9b5eb510 con 0x559a9b6d0520 2024-05-09T04:57:08.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f966963b700 1 --2- 172.21.15.46:0/3196112501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b6d0520 0x559a9b85f340 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f966001cde0 tx=0x7f966001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8362d00f8b4d96 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f964cff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f964cff9700 1 -- 172.21.15.46:0/3196112501 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9660016030 con 0x559a9b6d0520 2024-05-09T04:57:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f964cff9700 1 -- 172.21.15.46:0/3196112501 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f9660010ad0 con 0x559a9b6d0520 2024-05-09T04:57:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.348+0000 7f967a2b5b80 1 -- 172.21.15.46:0/3196112501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f966001c420 con 0x559a9b6d0520 2024-05-09T04:57:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.349+0000 7f964cff9700 1 -- 172.21.15.46:0/3196112501 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9660010cd0 con 0x559a9b6d0520 2024-05-09T04:57:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.349+0000 7f967a2b5b80 10 client.16749 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.349+0000 7f967a2b5b80 1 -- 172.21.15.46:0/3196112501 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a9b86d260 con 0x559a9b6d0520 2024-05-09T04:57:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.349+0000 7f964cff9700 1 -- 172.21.15.46:0/3196112501 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f966001c420 con 0x559a9b6d0520 2024-05-09T04:57:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.349+0000 7f964cff9700 1 -- 172.21.15.46:0/3196112501 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9660004af0 con 0x559a9b6d0520 2024-05-09T04:57:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.349+0000 7f967a2b5b80 10 client.16749 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.349+0000 7f967a2b5b80 10 client.16749 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:57:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.349+0000 7f967a2b5b80 20 client.16749 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.349+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/3196112501 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559a9b86f7e0 0x559a9b871c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.349+0000 7f967a2b5b80 4 client.16749 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:57:08.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.349+0000 7f967a2b5b80 1 -- 172.21.15.46:0/3196112501 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a9b77db00 con 0x559a9b86f7e0 2024-05-09T04:57:08.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.349+0000 7f9668e3a700 1 --2- 172.21.15.46:0/3196112501 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559a9b86f7e0 0x559a9b871c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.350+0000 7f9668e3a700 1 --2- 172.21.15.46:0/3196112501 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559a9b86f7e0 0x559a9b871c00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fdc0b52ef677362 server_cookie=eedeb15601220c04 in_seq=0 out_seq=0 2024-05-09T04:57:08.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.350+0000 7f964cff9700 10 client.16749 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:08.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.351+0000 7f964cff9700 1 -- 172.21.15.46:0/3196112501 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559a9b77db00 con 0x559a9b86f7e0 2024-05-09T04:57:08.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.351+0000 7f964cff9700 10 client.16749 handle_command_reply: tid=0 2024-05-09T04:57:08.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.351+0000 7f967a2b5b80 1 client.16749 shutdown 2024-05-09T04:57:08.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.351+0000 7f967a2b5b80 1 -- 172.21.15.46:0/3196112501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b6d0520 msgr2=0x559a9b85f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:08.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.351+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/3196112501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b6d0520 0x559a9b85f340 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f966001cde0 tx=0x7f966001c880 comp rx=0 tx=0).stop 2024-05-09T04:57:08.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.351+0000 7f967a2b5b80 1 -- 172.21.15.46:0/3196112501 shutdown_connections 2024-05-09T04:57:08.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.351+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/3196112501 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b85fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.351+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/3196112501 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b6d0520 0x559a9b85f340 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.351+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/3196112501 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559a9b86f7e0 0x559a9b871c00 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.351+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/3196112501 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b74ad80 0x559a9b85f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.351+0000 7f967a2b5b80 1 -- 172.21.15.46:0/3196112501 >> 172.21.15.46:0/3196112501 conn(0x559a9b6d3ff0 msgr2=0x559a9b84c470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.351+0000 7f964cff9700 0 client.16749 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.351+0000 7f967a2b5b80 1 -- 172.21.15.46:0/3196112501 shutdown_connections 2024-05-09T04:57:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.351+0000 7f967a2b5b80 1 -- 172.21.15.46:0/3196112501 wait complete. 2024-05-09T04:57:08.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.351+0000 7f967a2b5b80 20 client.16749 trim_cache size 0 max 16384 2024-05-09T04:57:08.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.372+0000 7f967a2b5b80 1 Processor -- start 2024-05-09T04:57:08.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.372+0000 7f967a2b5b80 1 -- start start 2024-05-09T04:57:08.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.372+0000 7f967a2b5b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 0x559a9b84d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.372+0000 7f967a2b5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b851780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.372+0000 7f967a2b5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b85fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.372+0000 7f967a2b5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f96641af680 con 0x559a9b74ad80 2024-05-09T04:57:08.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.372+0000 7f967a2b5b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f96641acf10 con 0x559a9b6d0520 2024-05-09T04:57:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.372+0000 7f967a2b5b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f96641aad60 con 0x559a9b84d9c0 2024-05-09T04:57:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.372+0000 7f9668e3a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b851780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.372+0000 7f9668e3a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b851780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39094/0 (socket says 172.21.15.46:39094) 2024-05-09T04:57:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.372+0000 7f9668e3a700 1 -- 172.21.15.46:0/1718934394 learned_addr learned my addr 172.21.15.46:0/1718934394 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.372+0000 7f966963b700 1 --2- 172.21.15.46:0/1718934394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 0x559a9b84d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.372+0000 7f9668e3a700 1 -- 172.21.15.46:0/1718934394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 msgr2=0x559a9b85fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.372+0000 7f9669e3c700 1 --2- 172.21.15.46:0/1718934394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b85fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.373+0000 7f9668e3a700 1 --2- 172.21.15.46:0/1718934394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b85fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.373+0000 7f9668e3a700 1 -- 172.21.15.46:0/1718934394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 msgr2=0x559a9b84d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.373+0000 7f9668e3a700 1 --2- 172.21.15.46:0/1718934394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 0x559a9b84d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.373+0000 7f9668e3a700 1 -- 172.21.15.46:0/1718934394 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a9b861be0 con 0x559a9b74ad80 2024-05-09T04:57:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.373+0000 7f9668e3a700 1 --2- 172.21.15.46:0/1718934394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b851780 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f96540a9740 tx=0x7f9654091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c401143235a8c5fe server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:08.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.373+0000 7f964effd700 1 -- 172.21.15.46:0/1718934394 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96540a97b0 con 0x559a9b74ad80 2024-05-09T04:57:08.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.373+0000 7f964effd700 1 -- 172.21.15.46:0/1718934394 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96540adb60 con 0x559a9b74ad80 2024-05-09T04:57:08.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.373+0000 7f964effd700 1 -- 172.21.15.46:0/1718934394 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96540b2180 con 0x559a9b74ad80 2024-05-09T04:57:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.374+0000 7f967a2b5b80 1 -- 172.21.15.46:0/1718934394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 msgr2=0x559a9b851780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.374+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/1718934394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b851780 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f96540a9740 tx=0x7f9654091040 comp rx=0 tx=0).stop 2024-05-09T04:57:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.374+0000 7f967a2b5b80 1 -- 172.21.15.46:0/1718934394 shutdown_connections 2024-05-09T04:57:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.374+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/1718934394 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b85fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.374+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/1718934394 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 0x559a9b84d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.374+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/1718934394 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b851780 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.374+0000 7f967a2b5b80 1 -- 172.21.15.46:0/1718934394 >> 172.21.15.46:0/1718934394 conn(0x559a9b6d3ff0 msgr2=0x559a9b85dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.374+0000 7f967a2b5b80 1 -- 172.21.15.46:0/1718934394 shutdown_connections 2024-05-09T04:57:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.374+0000 7f967a2b5b80 1 -- 172.21.15.46:0/1718934394 wait complete. 2024-05-09T04:57:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.374+0000 7f967a2b5b80 1 -- start start 2024-05-09T04:57:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.374+0000 7f967a2b5b80 1 Processor -- start 2024-05-09T04:57:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f967a2b5b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 0x559a9b874780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f967a2b5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b874cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f967a2b5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b872ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f967a2b5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f96641ab3d0 con 0x559a9b74ad80 2024-05-09T04:57:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f967a2b5b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f96641ac840 con 0x559a9b6d0520 2024-05-09T04:57:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f967a2b5b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f96641abaa0 con 0x559a9b84d9c0 2024-05-09T04:57:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f967a2b5b80 10 client.0 _ll_register_callbacks cb 0x559a9b5e1740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f9668e3a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b874cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f9668e3a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b874cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39100/0 (socket says 172.21.15.46:39100) 2024-05-09T04:57:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f9668e3a700 1 -- 172.21.15.46:0/213209801 learned_addr learned my addr 172.21.15.46:0/213209801 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f966963b700 1 --2- 172.21.15.46:0/213209801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 0x559a9b874780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f9669e3c700 1 --2- 172.21.15.46:0/213209801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b872ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f9668e3a700 1 -- 172.21.15.46:0/213209801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 msgr2=0x559a9b872ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f9668e3a700 1 --2- 172.21.15.46:0/213209801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b872ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f9668e3a700 1 -- 172.21.15.46:0/213209801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 msgr2=0x559a9b874780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f9668e3a700 1 --2- 172.21.15.46:0/213209801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 0x559a9b874780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f9668e3a700 1 -- 172.21.15.46:0/213209801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a9b8732b0 con 0x559a9b74ad80 2024-05-09T04:57:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f966963b700 1 --2- 172.21.15.46:0/213209801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 0x559a9b874780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:57:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.375+0000 7f9669e3c700 1 --2- 172.21.15.46:0/213209801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b872ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:57:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f9668e3a700 1 --2- 172.21.15.46:0/213209801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b874cc0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9654017000 tx=0x7f96540a98d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d70a705c80873d4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f964cff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f964cff9700 1 -- 172.21.15.46:0/213209801 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96540a5a40 con 0x559a9b74ad80 2024-05-09T04:57:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f964cff9700 1 -- 172.21.15.46:0/213209801 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f96540a9a10 con 0x559a9b74ad80 2024-05-09T04:57:08.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f967a2b5b80 1 -- 172.21.15.46:0/213209801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96540adb60 con 0x559a9b74ad80 2024-05-09T04:57:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f964cff9700 1 -- 172.21.15.46:0/213209801 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f96540a9c10 con 0x559a9b74ad80 2024-05-09T04:57:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f967a2b5b80 10 client.16755 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f967a2b5b80 1 -- 172.21.15.46:0/213209801 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x559a9b8da700 con 0x559a9b74ad80 2024-05-09T04:57:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f964cff9700 1 -- 172.21.15.46:0/213209801 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96540adb60 con 0x559a9b74ad80 2024-05-09T04:57:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f964cff9700 1 -- 172.21.15.46:0/213209801 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f96540aaae0 con 0x559a9b74ad80 2024-05-09T04:57:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f967a2b5b80 10 client.16755 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f967a2b5b80 10 client.16755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:57:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f967a2b5b80 20 client.16755 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/213209801 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559a9b8dad00 0x559a9b8dd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f967a2b5b80 4 client.16755 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:57:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.376+0000 7f967a2b5b80 1 -- 172.21.15.46:0/213209801 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559a9b777ba0 con 0x559a9b8dad00 2024-05-09T04:57:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.377+0000 7f966963b700 1 --2- 172.21.15.46:0/213209801 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559a9b8dad00 0x559a9b8dd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.377+0000 7f966963b700 1 --2- 172.21.15.46:0/213209801 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559a9b8dad00 0x559a9b8dd120 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c228bfb179e29575 server_cookie=e6d41a8042e2e7e8 in_seq=0 out_seq=0 2024-05-09T04:57:08.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.377+0000 7f964cff9700 10 client.16755 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.378+0000 7f964cff9700 1 -- 172.21.15.46:0/213209801 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559a9b777ba0 con 0x559a9b8dad00 2024-05-09T04:57:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.378+0000 7f964cff9700 10 client.16755 handle_command_reply: tid=0 2024-05-09T04:57:08.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.378+0000 7f967a2b5b80 1 client.16755 shutdown 2024-05-09T04:57:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.378+0000 7f967a2b5b80 1 -- 172.21.15.46:0/213209801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 msgr2=0x559a9b874cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.378+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/213209801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b874cc0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9654017000 tx=0x7f96540a98d0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.378+0000 7f967a2b5b80 1 -- 172.21.15.46:0/213209801 shutdown_connections 2024-05-09T04:57:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.378+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/213209801 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x559a9b84d9c0 0x559a9b872ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.378+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/213209801 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x559a9b8dad00 0x559a9b8dd120 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.378+0000 7f964cff9700 0 client.16755 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.378+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/213209801 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x559a9b6d0520 0x559a9b874780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.378+0000 7f967a2b5b80 1 --2- 172.21.15.46:0/213209801 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x559a9b74ad80 0x559a9b874cc0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.379+0000 7f967a2b5b80 1 -- 172.21.15.46:0/213209801 >> 172.21.15.46:0/213209801 conn(0x559a9b6d3ff0 msgr2=0x559a9b8514e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.379+0000 7f967a2b5b80 1 -- 172.21.15.46:0/213209801 shutdown_connections 2024-05-09T04:57:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.379+0000 7f967a2b5b80 1 -- 172.21.15.46:0/213209801 wait complete. 2024-05-09T04:57:08.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.379+0000 7f967a2b5b80 20 client.16755 trim_cache size 0 max 16384 2024-05-09T04:57:08.384 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:57:08.384 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:57:08.384 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "b667465f-b9d8-4a54-87d1-74969a440796", 2024-05-09T04:57:08.384 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:57:08.384 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:57:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.379+0000 7f966b89f700 1 -- 172.21.15.46:0/2773798944 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f96500b2a70 msgr2=0x7f96500b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:08.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.379+0000 7f966b89f700 1 --2- 172.21.15.46:0/2773798944 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f96500b2a70 0x7f96500b4f80 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f9660005950 tx=0x7f966000a300 comp rx=0 tx=0).stop 2024-05-09T04:57:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.379+0000 7f966b89f700 1 -- 172.21.15.46:0/2773798944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9664107e10 msgr2=0x7f966419a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.379+0000 7f966b89f700 1 --2- 172.21.15.46:0/2773798944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9664107e10 0x7f966419a340 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f965400c9e0 tx=0x7f965400ccf0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.379+0000 7f966b89f700 1 -- 172.21.15.46:0/2773798944 shutdown_connections 2024-05-09T04:57:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.379+0000 7f966b89f700 1 --2- 172.21.15.46:0/2773798944 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f96500b2a70 0x7f96500b4f80 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.379+0000 7f966b89f700 1 --2- 172.21.15.46:0/2773798944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9664108760 0x7f966419e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.379+0000 7f966b89f700 1 --2- 172.21.15.46:0/2773798944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f96641034e0 0x7f9664199da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.379+0000 7f966b89f700 1 --2- 172.21.15.46:0/2773798944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9664107e10 0x7f966419a340 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.379+0000 7f966b89f700 1 -- 172.21.15.46:0/2773798944 >> 172.21.15.46:0/2773798944 conn(0x7f96640fef30 msgr2=0x7f9664107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.380+0000 7f966b89f700 1 -- 172.21.15.46:0/2773798944 shutdown_connections 2024-05-09T04:57:08.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:08.380+0000 7f966b89f700 1 -- 172.21.15.46:0/2773798944 wait complete. 2024-05-09T04:57:09.143 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:57:09.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:09 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:57:09.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:09 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:57:09.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:09 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:57:09.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:09 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:57:09.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:09 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:57:09.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:09 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:57:09.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:09 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:57:09.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:09 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:57:09.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:09 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:57:09.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:09 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:57:09.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:09 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:57:09.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:09 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:57:09.392 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:57:10.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:10 smithi046 ceph-mon[30136]: pgmap v185: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-09T04:57:10.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:10 smithi072 ceph-mon[34088]: pgmap v185: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-09T04:57:10.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:10 smithi110 ceph-mon[34904]: pgmap v185: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 22 KiB/s wr, 20 op/s 2024-05-09T04:57:11.145 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:11 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:57:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.265+0000 7fa3f0f15700 1 -- 172.21.15.46:0/816144628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ec1054a0 msgr2=0x7fa3ec1058b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.265+0000 7fa3f0f15700 1 --2- 172.21.15.46:0/816144628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ec1054a0 0x7fa3ec1058b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc009990 tx=0x7fa3dc009ca0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.265+0000 7fa3f0f15700 1 -- 172.21.15.46:0/816144628 shutdown_connections 2024-05-09T04:57:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.265+0000 7fa3f0f15700 1 --2- 172.21.15.46:0/816144628 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3ec10e910 0x7fa3ec110e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.265+0000 7fa3f0f15700 1 --2- 172.21.15.46:0/816144628 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ec1054a0 0x7fa3ec1058b0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.265+0000 7fa3f0f15700 1 --2- 172.21.15.46:0/816144628 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3ec105df0 0x7fa3ec10e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.265+0000 7fa3f0f15700 1 -- 172.21.15.46:0/816144628 >> 172.21.15.46:0/816144628 conn(0x7fa3ec078e50 msgr2=0x7fa3ec079290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.265+0000 7fa3f0f15700 1 -- 172.21.15.46:0/816144628 shutdown_connections 2024-05-09T04:57:11.267 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.265+0000 7fa3f0f15700 1 -- 172.21.15.46:0/816144628 wait complete. 2024-05-09T04:57:11.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.266+0000 7fa3f0f15700 1 Processor -- start 2024-05-09T04:57:11.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.266+0000 7fa3f0f15700 1 -- start start 2024-05-09T04:57:11.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3f0f15700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3ec1054a0 0x7fa3ec19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3f0f15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ec105df0 0x7fa3ec19b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3f0f15700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3ec10e910 0x7fa3ec19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3f0f15700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec1144c0 con 0x7fa3ec105df0 2024-05-09T04:57:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3f0f15700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec114340 con 0x7fa3ec10e910 2024-05-09T04:57:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3f0f15700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec114640 con 0x7fa3ec1054a0 2024-05-09T04:57:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3e9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ec105df0 0x7fa3ec19b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3e9d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ec105df0 0x7fa3ec19b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39124/0 (socket says 172.21.15.46:39124) 2024-05-09T04:57:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3e9d9b700 1 -- 172.21.15.46:0/2236888368 learned_addr learned my addr 172.21.15.46:0/2236888368 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3ea59c700 1 --2- 172.21.15.46:0/2236888368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3ec1054a0 0x7fa3ec19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3ead9d700 1 --2- 172.21.15.46:0/2236888368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3ec10e910 0x7fa3ec19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3e9d9b700 1 -- 172.21.15.46:0/2236888368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3ec1054a0 msgr2=0x7fa3ec19aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3e9d9b700 1 --2- 172.21.15.46:0/2236888368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3ec1054a0 0x7fa3ec19aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3e9d9b700 1 -- 172.21.15.46:0/2236888368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3ec10e910 msgr2=0x7fa3ec19f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3e9d9b700 1 --2- 172.21.15.46:0/2236888368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3ec10e910 0x7fa3ec19f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3e9d9b700 1 -- 172.21.15.46:0/2236888368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3ec19fd80 con 0x7fa3ec105df0 2024-05-09T04:57:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.267+0000 7fa3e9d9b700 1 --2- 172.21.15.46:0/2236888368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ec105df0 0x7fa3ec19b450 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000d980 tx=0x7fa3e000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.268+0000 7fa3db7fe700 1 -- 172.21.15.46:0/2236888368 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e0017070 con 0x7fa3ec105df0 2024-05-09T04:57:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.268+0000 7fa3f0f15700 1 -- 172.21.15.46:0/2236888368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3ec1a0070 con 0x7fa3ec105df0 2024-05-09T04:57:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.268+0000 7fa3db7fe700 1 -- 172.21.15.46:0/2236888368 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e000fd40 con 0x7fa3ec105df0 2024-05-09T04:57:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.268+0000 7fa3db7fe700 1 -- 172.21.15.46:0/2236888368 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e000e3b0 con 0x7fa3ec105df0 2024-05-09T04:57:11.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.268+0000 7fa3f0f15700 1 -- 172.21.15.46:0/2236888368 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3ec1a7990 con 0x7fa3ec105df0 2024-05-09T04:57:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.270+0000 7fa3db7fe700 1 -- 172.21.15.46:0/2236888368 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa3e000e550 con 0x7fa3ec105df0 2024-05-09T04:57:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.271+0000 7fa3db7fe700 1 --2- 172.21.15.46:0/2236888368 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3d40b29f0 0x7fa3d40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.271+0000 7fa3ff92bb80 1 Processor -- start 2024-05-09T04:57:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.271+0000 7fa3ea59c700 1 --2- 172.21.15.46:0/2236888368 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3d40b29f0 0x7fa3d40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.271+0000 7fa3db7fe700 1 -- 172.21.15.46:0/2236888368 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa3e00e2470 con 0x7fa3ec105df0 2024-05-09T04:57:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.272+0000 7fa3ff92bb80 1 -- start start 2024-05-09T04:57:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.272+0000 7fa3ff92bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.272+0000 7fa3ff92bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.272+0000 7fa3ff92bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2bfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.272+0000 7fa3ff92bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ac9e1e1ff0 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.272+0000 7fa3ff92bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ac9e1807f0 con 0x55ac9e1424a0 2024-05-09T04:57:11.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.272+0000 7fa3ff92bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ac9e1ddfb0 con 0x55ac9e22c160 2024-05-09T04:57:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.272+0000 7fa3ea59c700 1 --2- 172.21.15.46:0/2236888368 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3d40b29f0 0x7fa3d40b4f00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc006010 tx=0x7fa3dc005960 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.272+0000 7fa3ead9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2bfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.272+0000 7fa3ead9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2bfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39138/0 (socket says 172.21.15.46:39138) 2024-05-09T04:57:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.272+0000 7fa3ead9d700 1 -- 172.21.15.46:0/1311317396 learned_addr learned my addr 172.21.15.46:0/1311317396 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.272+0000 7fa3ea59c700 1 --2- 172.21.15.46:0/1311317396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.272+0000 7fa3e9d9b700 1 --2- 172.21.15.46:0/1311317396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.273+0000 7fa3ead9d700 1 -- 172.21.15.46:0/1311317396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 msgr2=0x55ac9e2c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.273+0000 7fa3ead9d700 1 --2- 172.21.15.46:0/1311317396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.273+0000 7fa3ead9d700 1 -- 172.21.15.46:0/1311317396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 msgr2=0x55ac9e2c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.273+0000 7fa3ead9d700 1 --2- 172.21.15.46:0/1311317396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.273+0000 7fa3ead9d700 1 -- 172.21.15.46:0/1311317396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9e178370 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.273+0000 7fa3ea59c700 1 --2- 172.21.15.46:0/1311317396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.273+0000 7fa3ead9d700 1 --2- 172.21.15.46:0/1311317396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2bfad0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4003fa0 tx=0x7fa3e4009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3c1faa99976e0f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:11.275 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.273+0000 7fa3d8ff9700 1 -- 172.21.15.46:0/1311317396 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4004bb0 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.273+0000 7fa3d8ff9700 1 -- 172.21.15.46:0/1311317396 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e4004d50 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.276 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.273+0000 7fa3d8ff9700 1 -- 172.21.15.46:0/1311317396 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4007d20 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.274+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/1311317396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 msgr2=0x55ac9e2bfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:11.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.274+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/1311317396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2bfad0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4003fa0 tx=0x7fa3e4009ba0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.274+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/1311317396 shutdown_connections 2024-05-09T04:57:11.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.274+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/1311317396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2bfad0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.274+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/1311317396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.274+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/1311317396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.274+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/1311317396 >> 172.21.15.46:0/1311317396 conn(0x55ac9e1a26a0 msgr2=0x55ac9e2ba400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.274+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/1311317396 shutdown_connections 2024-05-09T04:57:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.274+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/1311317396 wait complete. 2024-05-09T04:57:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.275+0000 7fa3ff92bb80 1 -- start start 2024-05-09T04:57:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.275+0000 7fa3ff92bb80 1 Processor -- start 2024-05-09T04:57:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.275+0000 7fa3ff92bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.275+0000 7fa3ff92bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c96e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.275+0000 7fa3ff92bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2c9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.275+0000 7fa3ff92bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ac9e05dba0 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.275+0000 7fa3ff92bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ac9e19c150 con 0x55ac9e1424a0 2024-05-09T04:57:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.275+0000 7fa3ff92bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ac9e2cc1d0 con 0x55ac9e22c160 2024-05-09T04:57:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.275+0000 7fa3ff92bb80 10 client.0 _ll_register_callbacks cb 0x55ac9e0536c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.275+0000 7fa3ead9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2c9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.275+0000 7fa3ead9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2c9c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39148/0 (socket says 172.21.15.46:39148) 2024-05-09T04:57:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.275+0000 7fa3ead9d700 1 -- 172.21.15.46:0/2561676190 learned_addr learned my addr 172.21.15.46:0/2561676190 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:11.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.275+0000 7fa3e9d9b700 1 --2- 172.21.15.46:0/2561676190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.275+0000 7fa3ea59c700 1 --2- 172.21.15.46:0/2561676190 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.276+0000 7fa3ead9d700 1 -- 172.21.15.46:0/2561676190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 msgr2=0x55ac9e2c96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:11.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.276+0000 7fa3ead9d700 1 --2- 172.21.15.46:0/2561676190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.276+0000 7fa3ead9d700 1 -- 172.21.15.46:0/2561676190 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 msgr2=0x55ac9e2c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:11.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.276+0000 7fa3ead9d700 1 --2- 172.21.15.46:0/2561676190 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c91a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.276+0000 7fa3ead9d700 1 -- 172.21.15.46:0/2561676190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac9e05d490 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.276+0000 7fa3ead9d700 1 --2- 172.21.15.46:0/2561676190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2c9c20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4004700 tx=0x7fa3e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f326030ebff2d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:11.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.276+0000 7fa3d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:11.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.276+0000 7fa3d1ffb700 1 -- 172.21.15.46:0/2561676190 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4022070 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.276+0000 7fa3d1ffb700 1 -- 172.21.15.46:0/2561676190 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa3e4007b10 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.276+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/2561676190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3e4004d50 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.276+0000 7fa3d1ffb700 1 -- 172.21.15.46:0/2561676190 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa3e402e050 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.276+0000 7fa3ff92bb80 10 client.16767 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:11.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.276+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/2561676190 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac9e2d6b70 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.276+0000 7fa3d1ffb700 1 -- 172.21.15.46:0/2561676190 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e4004d50 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.276+0000 7fa3d1ffb700 1 -- 172.21.15.46:0/2561676190 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa3e401e410 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.277+0000 7fa3ff92bb80 10 client.16767 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:11.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.277+0000 7fa3ff92bb80 10 client.16767 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:57:11.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.277+0000 7fa3ff92bb80 20 client.16767 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:11.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.277+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/2561676190 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ac9e2d90a0 0x55ac9e2db4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.277+0000 7fa3ff92bb80 4 client.16767 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:57:11.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.277+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/2561676190 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac9e1ef9d0 con 0x55ac9e2d90a0 2024-05-09T04:57:11.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.277+0000 7fa3ea59c700 1 --2- 172.21.15.46:0/2561676190 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ac9e2d90a0 0x55ac9e2db4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.277+0000 7fa3d1ffb700 10 client.16767 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:11.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.277+0000 7fa3ea59c700 1 --2- 172.21.15.46:0/2561676190 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ac9e2d90a0 0x55ac9e2db4c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73c4c17225defff2 server_cookie=9a23e4bf6c708782 in_seq=0 out_seq=0 2024-05-09T04:57:11.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.278+0000 7fa3d1ffb700 1 -- 172.21.15.46:0/2561676190 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ac9e1ef9d0 con 0x55ac9e2d90a0 2024-05-09T04:57:11.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.278+0000 7fa3d1ffb700 10 client.16767 handle_command_reply: tid=0 2024-05-09T04:57:11.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.278+0000 7fa3ff92bb80 1 client.16767 shutdown 2024-05-09T04:57:11.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.278+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/2561676190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 msgr2=0x55ac9e2c9c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:11.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.278+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/2561676190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2c9c20 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa3e4004700 tx=0x7fa3e4004730 comp rx=0 tx=0).stop 2024-05-09T04:57:11.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.279+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/2561676190 shutdown_connections 2024-05-09T04:57:11.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.279+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/2561676190 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2c9c20 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.279+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/2561676190 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ac9e2d90a0 0x55ac9e2db4c0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.279+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/2561676190 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.279+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/2561676190 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.279+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/2561676190 >> 172.21.15.46:0/2561676190 conn(0x55ac9e1a26a0 msgr2=0x55ac9e2ba1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:11.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.279+0000 7fa3d1ffb700 0 client.16767 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:11.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.279+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/2561676190 shutdown_connections 2024-05-09T04:57:11.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.279+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/2561676190 wait complete. 2024-05-09T04:57:11.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.279+0000 7fa3ff92bb80 20 client.16767 trim_cache size 0 max 16384 2024-05-09T04:57:11.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.299+0000 7fa3ff92bb80 1 Processor -- start 2024-05-09T04:57:11.300 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.299+0000 7fa3ff92bb80 1 -- start start 2024-05-09T04:57:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.299+0000 7fa3ff92bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.299+0000 7fa3ff92bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.299+0000 7fa3ff92bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2d5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.299+0000 7fa3ff92bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec1b18e0 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.299+0000 7fa3ff92bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec1af170 con 0x55ac9e1424a0 2024-05-09T04:57:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.299+0000 7fa3ff92bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec1acfc0 con 0x55ac9e22c160 2024-05-09T04:57:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.300+0000 7fa3ead9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2d5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.300+0000 7fa3ead9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2d5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39162/0 (socket says 172.21.15.46:39162) 2024-05-09T04:57:11.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.300+0000 7fa3ead9d700 1 -- 172.21.15.46:0/3516459782 learned_addr learned my addr 172.21.15.46:0/3516459782 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:11.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.300+0000 7fa3ea59c700 1 --2- 172.21.15.46:0/3516459782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.300+0000 7fa3e9d9b700 1 --2- 172.21.15.46:0/3516459782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.300+0000 7fa3ead9d700 1 -- 172.21.15.46:0/3516459782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 msgr2=0x55ac9e2c8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:11.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.300+0000 7fa3ead9d700 1 --2- 172.21.15.46:0/3516459782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.300+0000 7fa3ead9d700 1 -- 172.21.15.46:0/3516459782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 msgr2=0x55ac9e2c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.300+0000 7fa3ead9d700 1 --2- 172.21.15.46:0/3516459782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.300+0000 7fa3ead9d700 1 -- 172.21.15.46:0/3516459782 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac9e2c7710 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.300+0000 7fa3ea59c700 1 --2- 172.21.15.46:0/3516459782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:57:11.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.300+0000 7fa3ead9d700 1 --2- 172.21.15.46:0/3516459782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2d5010 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40046d0 tx=0x7fa3e4007890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f99395aaaa59e60 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:11.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.300+0000 7fa3d3fff700 1 -- 172.21.15.46:0/3516459782 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4022070 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.300+0000 7fa3d3fff700 1 -- 172.21.15.46:0/3516459782 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e4004850 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.300+0000 7fa3d3fff700 1 -- 172.21.15.46:0/3516459782 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4026c40 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.301+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/3516459782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 msgr2=0x55ac9e2d5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.301+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/3516459782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2d5010 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa3e40046d0 tx=0x7fa3e4007890 comp rx=0 tx=0).stop 2024-05-09T04:57:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.301+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/3516459782 shutdown_connections 2024-05-09T04:57:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.301+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/3516459782 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2d5010 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.301+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/3516459782 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.301+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/3516459782 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.301+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/3516459782 >> 172.21.15.46:0/3516459782 conn(0x55ac9e1a26a0 msgr2=0x55ac9e2bed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.301+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/3516459782 shutdown_connections 2024-05-09T04:57:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.301+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/3516459782 wait complete. 2024-05-09T04:57:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.302+0000 7fa3ff92bb80 1 -- start start 2024-05-09T04:57:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.302+0000 7fa3ff92bb80 1 Processor -- start 2024-05-09T04:57:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.302+0000 7fa3ff92bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.302+0000 7fa3ff92bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.302+0000 7fa3ff92bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.302+0000 7fa3ff92bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec1ad630 con 0x55ac9e1424a0 2024-05-09T04:57:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.302+0000 7fa3ff92bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec1aeaa0 con 0x55ac9e2bf6a0 2024-05-09T04:57:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.302+0000 7fa3ff92bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa3ec1add00 con 0x55ac9e22c160 2024-05-09T04:57:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.302+0000 7fa3ff92bb80 10 client.0 _ll_register_callbacks cb 0x55ac9e0536c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.302+0000 7fa3ea59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.302+0000 7fa3ea59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39176/0 (socket says 172.21.15.46:39176) 2024-05-09T04:57:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.302+0000 7fa3ea59c700 1 -- 172.21.15.46:0/1163002020 learned_addr learned my addr 172.21.15.46:0/1163002020 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.302+0000 7fa3e9d9b700 1 --2- 172.21.15.46:0/1163002020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.302+0000 7fa3ead9d700 1 --2- 172.21.15.46:0/1163002020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.303+0000 7fa3ea59c700 1 -- 172.21.15.46:0/1163002020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 msgr2=0x55ac9e2c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.303+0000 7fa3ea59c700 1 --2- 172.21.15.46:0/1163002020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.303+0000 7fa3ea59c700 1 -- 172.21.15.46:0/1163002020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e2bf6a0 msgr2=0x55ac9e2c6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.303+0000 7fa3ea59c700 1 --2- 172.21.15.46:0/1163002020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2c6a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.303+0000 7fa3ea59c700 1 -- 172.21.15.46:0/1163002020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac9e2cf670 con 0x55ac9e1424a0 2024-05-09T04:57:11.309 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.303+0000 7fa3ead9d700 1 --2- 172.21.15.46:0/1163002020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:57:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.303+0000 7fa3ea59c700 1 --2- 172.21.15.46:0/1163002020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c5ff0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec06d130 tx=0x7fa3dc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de889567b7f2dcd1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.303+0000 7fa3d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.303+0000 7fa3d1ffb700 1 -- 172.21.15.46:0/1163002020 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc00bbe0 con 0x55ac9e1424a0 2024-05-09T04:57:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.303+0000 7fa3d1ffb700 1 -- 172.21.15.46:0/1163002020 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fa3dc00bd80 con 0x55ac9e1424a0 2024-05-09T04:57:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.303+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/1163002020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3e4004850 con 0x55ac9e1424a0 2024-05-09T04:57:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.303+0000 7fa3d1ffb700 1 -- 172.21.15.46:0/1163002020 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa3dc03f050 con 0x55ac9e1424a0 2024-05-09T04:57:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.303+0000 7fa3ff92bb80 10 client.16773 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.303+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/1163002020 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ac9e343780 con 0x55ac9e1424a0 2024-05-09T04:57:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.303+0000 7fa3d1ffb700 1 -- 172.21.15.46:0/1163002020 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3e4004850 con 0x55ac9e1424a0 2024-05-09T04:57:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.304+0000 7fa3d1ffb700 1 -- 172.21.15.46:0/1163002020 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fa3dc022070 con 0x55ac9e1424a0 2024-05-09T04:57:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.304+0000 7fa3ff92bb80 10 client.16773 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.304+0000 7fa3ff92bb80 10 client.16773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:57:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.304+0000 7fa3ff92bb80 20 client.16773 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:11.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.304+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/1163002020 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ac9e343ec0 0x55ac9e3462e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.304+0000 7fa3ff92bb80 4 client.16773 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:57:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.304+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/1163002020 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ac9e1e9a70 con 0x55ac9e343ec0 2024-05-09T04:57:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.304+0000 7fa3e9d9b700 1 --2- 172.21.15.46:0/1163002020 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ac9e343ec0 0x55ac9e3462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.304+0000 7fa3e9d9b700 1 --2- 172.21.15.46:0/1163002020 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ac9e343ec0 0x55ac9e3462e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bf3ea654574c4b3 server_cookie=d6338040c587c1ff in_seq=0 out_seq=0 2024-05-09T04:57:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.304+0000 7fa3d1ffb700 10 client.16773 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.305+0000 7fa3d1ffb700 1 -- 172.21.15.46:0/1163002020 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ac9e1e9a70 con 0x55ac9e343ec0 2024-05-09T04:57:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.305+0000 7fa3d1ffb700 10 client.16773 handle_command_reply: tid=0 2024-05-09T04:57:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.305+0000 7fa3ff92bb80 1 client.16773 shutdown 2024-05-09T04:57:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.305+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/1163002020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e1424a0 msgr2=0x55ac9e2c5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.305+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/1163002020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c5ff0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec06d130 tx=0x7fa3dc009ca0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.305+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/1163002020 shutdown_connections 2024-05-09T04:57:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.305+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/1163002020 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ac9e343ec0 0x55ac9e3462e0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.305+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/1163002020 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ac9e2bf6a0 0x55ac9e2c6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.305+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/1163002020 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ac9e1424a0 0x55ac9e2c5ff0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.305+0000 7fa3ff92bb80 1 --2- 172.21.15.46:0/1163002020 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ac9e22c160 0x55ac9e2c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.305+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/1163002020 >> 172.21.15.46:0/1163002020 conn(0x55ac9e1a26a0 msgr2=0x55ac9e2c7960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:11.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.305+0000 7fa3d1ffb700 0 client.16773 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:11.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.305+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/1163002020 shutdown_connections 2024-05-09T04:57:11.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.305+0000 7fa3ff92bb80 1 -- 172.21.15.46:0/1163002020 wait complete. 2024-05-09T04:57:11.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.305+0000 7fa3ff92bb80 20 client.16773 trim_cache size 0 max 16384 2024-05-09T04:57:11.312 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:57:11.312 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:57:11.312 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:57:11.312 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:57:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.306+0000 7fa3f0f15700 1 -- 172.21.15.46:0/2236888368 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3d40b29f0 msgr2=0x7fa3d40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.306+0000 7fa3f0f15700 1 --2- 172.21.15.46:0/2236888368 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3d40b29f0 0x7fa3d40b4f00 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc006010 tx=0x7fa3dc005960 comp rx=0 tx=0).stop 2024-05-09T04:57:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.306+0000 7fa3f0f15700 1 -- 172.21.15.46:0/2236888368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ec105df0 msgr2=0x7fa3ec19b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.306+0000 7fa3f0f15700 1 --2- 172.21.15.46:0/2236888368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ec105df0 0x7fa3ec19b450 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa3e000d980 tx=0x7fa3e000dc90 comp rx=0 tx=0).stop 2024-05-09T04:57:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.306+0000 7fa3f0f15700 1 -- 172.21.15.46:0/2236888368 shutdown_connections 2024-05-09T04:57:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.306+0000 7fa3f0f15700 1 --2- 172.21.15.46:0/2236888368 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3d40b29f0 0x7fa3d40b4f00 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.306+0000 7fa3f0f15700 1 --2- 172.21.15.46:0/2236888368 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa3ec10e910 0x7fa3ec19f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.306+0000 7fa3f0f15700 1 --2- 172.21.15.46:0/2236888368 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa3ec1054a0 0x7fa3ec19aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.306+0000 7fa3f0f15700 1 --2- 172.21.15.46:0/2236888368 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa3ec105df0 0x7fa3ec19b450 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:11.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.306+0000 7fa3f0f15700 1 -- 172.21.15.46:0/2236888368 >> 172.21.15.46:0/2236888368 conn(0x7fa3ec078e50 msgr2=0x7fa3ec101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:11.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.306+0000 7fa3f0f15700 1 -- 172.21.15.46:0/2236888368 shutdown_connections 2024-05-09T04:57:11.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:11.306+0000 7fa3f0f15700 1 -- 172.21.15.46:0/2236888368 wait complete. 2024-05-09T04:57:11.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:11 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:57:11.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:11 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:57:12.113 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:57:12.113 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:57:12.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:12 smithi046 ceph-mon[30136]: pgmap v186: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T04:57:12.368 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:57:12.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:12 smithi072 ceph-mon[34088]: pgmap v186: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T04:57:12.458 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:12 smithi110 ceph-mon[34904]: pgmap v186: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 22 KiB/s wr, 23 op/s 2024-05-09T04:57:14.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.244+0000 7fc551d56700 1 -- 172.21.15.46:0/2823459768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c076e40 msgr2=0x7fc54c077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:14.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.244+0000 7fc551d56700 1 --2- 172.21.15.46:0/2823459768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c076e40 0x7fc54c077250 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc53c009990 tx=0x7fc53c009ca0 comp rx=0 tx=0).stop 2024-05-09T04:57:14.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.244+0000 7fc551d56700 1 -- 172.21.15.46:0/2823459768 shutdown_connections 2024-05-09T04:57:14.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.244+0000 7fc551d56700 1 --2- 172.21.15.46:0/2823459768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc54c110b80 0x7fc54c112fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:14.245 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.244+0000 7fc551d56700 1 --2- 172.21.15.46:0/2823459768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c076e40 0x7fc54c077250 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.244+0000 7fc551d56700 1 --2- 172.21.15.46:0/2823459768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc54c077790 0x7fc54c110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.244+0000 7fc551d56700 1 -- 172.21.15.46:0/2823459768 >> 172.21.15.46:0/2823459768 conn(0x7fc54c0ffea0 msgr2=0x7fc54c1022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.244+0000 7fc551d56700 1 -- 172.21.15.46:0/2823459768 shutdown_connections 2024-05-09T04:57:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.244+0000 7fc551d56700 1 -- 172.21.15.46:0/2823459768 wait complete. 2024-05-09T04:57:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.245+0000 7fc551d56700 1 Processor -- start 2024-05-09T04:57:14.246 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.245+0000 7fc551d56700 1 -- start start 2024-05-09T04:57:14.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.245+0000 7fc551d56700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc54c076e40 0x7fc54c0753a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:14.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.245+0000 7fc551d56700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc54c077790 0x7fc54c06e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:14.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.245+0000 7fc551d56700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c110b80 0x7fc54c06eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:14.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.245+0000 7fc551d56700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c116670 con 0x7fc54c110b80 2024-05-09T04:57:14.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.245+0000 7fc551d56700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c1164f0 con 0x7fc54c077790 2024-05-09T04:57:14.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.245+0000 7fc551d56700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc54c1167f0 con 0x7fc54c076e40 2024-05-09T04:57:14.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.245+0000 7fc54bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c110b80 0x7fc54c06eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:14.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc54bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c110b80 0x7fc54c06eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40136/0 (socket says 172.21.15.46:40136) 2024-05-09T04:57:14.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc54bfff700 1 -- 172.21.15.46:0/2942640866 learned_addr learned my addr 172.21.15.46:0/2942640866 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:14.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc54affd700 1 --2- 172.21.15.46:0/2942640866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc54c077790 0x7fc54c06e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:14.248 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc54b7fe700 1 --2- 172.21.15.46:0/2942640866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc54c076e40 0x7fc54c0753a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc54bfff700 1 -- 172.21.15.46:0/2942640866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc54c076e40 msgr2=0x7fc54c0753a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc54bfff700 1 --2- 172.21.15.46:0/2942640866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc54c076e40 0x7fc54c0753a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc54bfff700 1 -- 172.21.15.46:0/2942640866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc54c077790 msgr2=0x7fc54c06e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc54bfff700 1 --2- 172.21.15.46:0/2942640866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc54c077790 0x7fc54c06e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc54bfff700 1 -- 172.21.15.46:0/2942640866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc54c072ed0 con 0x7fc54c110b80 2024-05-09T04:57:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc54b7fe700 1 --2- 172.21.15.46:0/2942640866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc54c076e40 0x7fc54c0753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:57:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc54bfff700 1 --2- 172.21.15.46:0/2942640866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c110b80 0x7fc54c06eb20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc544007ec0 tx=0x7fc54400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc548ff9700 1 -- 172.21.15.46:0/2942640866 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc54400dbd0 con 0x7fc54c110b80 2024-05-09T04:57:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc551d56700 1 -- 172.21.15.46:0/2942640866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc54c0731c0 con 0x7fc54c110b80 2024-05-09T04:57:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc548ff9700 1 -- 172.21.15.46:0/2942640866 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc54400dd70 con 0x7fc54c110b80 2024-05-09T04:57:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc548ff9700 1 -- 172.21.15.46:0/2942640866 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc544004560 con 0x7fc54c110b80 2024-05-09T04:57:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.246+0000 7fc551d56700 1 -- 172.21.15.46:0/2942640866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc54c073700 con 0x7fc54c110b80 2024-05-09T04:57:14.249 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.248+0000 7fc551d56700 1 -- 172.21.15.46:0/2942640866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc54c0783b0 con 0x7fc54c110b80 2024-05-09T04:57:14.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.249+0000 7fc548ff9700 1 -- 172.21.15.46:0/2942640866 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc54401a030 con 0x7fc54c110b80 2024-05-09T04:57:14.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.250+0000 7fc548ff9700 1 --2- 172.21.15.46:0/2942640866 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5340b29f0 0x7fc5340b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:14.250 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.250+0000 7fc54b7fe700 1 --2- 172.21.15.46:0/2942640866 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5340b29f0 0x7fc5340b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:14.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.250+0000 7fc548ff9700 1 -- 172.21.15.46:0/2942640866 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fc5440e1bb0 con 0x7fc54c110b80 2024-05-09T04:57:14.251 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.251+0000 7fc54b7fe700 1 --2- 172.21.15.46:0/2942640866 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5340b29f0 0x7fc5340b4f00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc53c003800 tx=0x7fc53c01a040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:14.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.255+0000 7fc548ff9700 1 -- 172.21.15.46:0/2942640866 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc544021020 con 0x7fc54c110b80 2024-05-09T04:57:14.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.519+0000 7fc551d56700 1 -- 172.21.15.46:0/2942640866 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc54c06fa00 con 0x7fc54c110b80 2024-05-09T04:57:14.519 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.520+0000 7fc548ff9700 1 -- 172.21.15.46:0/2942640866 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7fc544010070 con 0x7fc54c110b80 2024-05-09T04:57:14.520 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:57:14.520 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:57:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.522+0000 7fc551d56700 1 -- 172.21.15.46:0/2942640866 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5340b29f0 msgr2=0x7fc5340b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.522+0000 7fc551d56700 1 --2- 172.21.15.46:0/2942640866 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5340b29f0 0x7fc5340b4f00 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc53c003800 tx=0x7fc53c01a040 comp rx=0 tx=0).stop 2024-05-09T04:57:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.522+0000 7fc551d56700 1 -- 172.21.15.46:0/2942640866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c110b80 msgr2=0x7fc54c06eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.522+0000 7fc551d56700 1 --2- 172.21.15.46:0/2942640866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c110b80 0x7fc54c06eb20 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc544007ec0 tx=0x7fc54400d3b0 comp rx=0 tx=0).stop 2024-05-09T04:57:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.522+0000 7fc551d56700 1 -- 172.21.15.46:0/2942640866 shutdown_connections 2024-05-09T04:57:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.522+0000 7fc551d56700 1 --2- 172.21.15.46:0/2942640866 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc5340b29f0 0x7fc5340b4f00 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:14.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.522+0000 7fc551d56700 1 --2- 172.21.15.46:0/2942640866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc54c110b80 0x7fc54c06eb20 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.522+0000 7fc551d56700 1 --2- 172.21.15.46:0/2942640866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc54c076e40 0x7fc54c0753a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.522+0000 7fc551d56700 1 --2- 172.21.15.46:0/2942640866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc54c077790 0x7fc54c06e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.522+0000 7fc551d56700 1 -- 172.21.15.46:0/2942640866 >> 172.21.15.46:0/2942640866 conn(0x7fc54c0ffea0 msgr2=0x7fc54c1004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.522+0000 7fc551d56700 1 -- 172.21.15.46:0/2942640866 shutdown_connections 2024-05-09T04:57:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:14.522+0000 7fc551d56700 1 -- 172.21.15.46:0/2942640866 wait complete. 2024-05-09T04:57:14.523 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:57:14.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:14 smithi046 ceph-mon[30136]: pgmap v187: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-05-09T04:57:14.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:14 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2942640866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:57:15.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:14 smithi110 ceph-mon[34904]: pgmap v187: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-05-09T04:57:15.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:14 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2942640866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:57:15.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:14 smithi072 ceph-mon[34088]: pgmap v187: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 19 op/s 2024-05-09T04:57:15.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:14 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2942640866' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:57:15.343 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:57:15.600 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:57:17.016 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:16 smithi046 ceph-mon[30136]: pgmap v188: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-09T04:57:17.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:16 smithi110 ceph-mon[34904]: pgmap v188: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-09T04:57:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:16 smithi072 ceph-mon[34088]: pgmap v188: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-05-09T04:57:17.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.409+0000 7fe06f7cb700 1 -- 172.21.15.46:0/3895232271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe068108740 msgr2=0x7fe0681100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:17.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.409+0000 7fe06f7cb700 1 --2- 172.21.15.46:0/3895232271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe068108740 0x7fe0681100a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fe06000b0d0 tx=0x7fe06000b3e0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.409+0000 7fe06f7cb700 1 -- 172.21.15.46:0/3895232271 shutdown_connections 2024-05-09T04:57:17.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.409+0000 7fe06f7cb700 1 --2- 172.21.15.46:0/3895232271 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe068108740 0x7fe0681100a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.409+0000 7fe06f7cb700 1 --2- 172.21.15.46:0/3895232271 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe0681034c0 0x7fe0681038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.409+0000 7fe06f7cb700 1 --2- 172.21.15.46:0/3895232271 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe068107df0 0x7fe068108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.409+0000 7fe06f7cb700 1 -- 172.21.15.46:0/3895232271 >> 172.21.15.46:0/3895232271 conn(0x7fe0680fedf0 msgr2=0x7fe068101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:17.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.409+0000 7fe06f7cb700 1 -- 172.21.15.46:0/3895232271 shutdown_connections 2024-05-09T04:57:17.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.409+0000 7fe06f7cb700 1 -- 172.21.15.46:0/3895232271 wait complete. 2024-05-09T04:57:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.410+0000 7fe06f7cb700 1 Processor -- start 2024-05-09T04:57:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.410+0000 7fe06f7cb700 1 -- start start 2024-05-09T04:57:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.410+0000 7fe06f7cb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0681034c0 0x7fe06819ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.410+0000 7fe06f7cb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe068107df0 0x7fe06819b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.410+0000 7fe06f7cb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe068108740 0x7fe06819f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.410+0000 7fe06f7cb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe068113370 con 0x7fe068107df0 2024-05-09T04:57:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.410+0000 7fe06f7cb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681131f0 con 0x7fe0681034c0 2024-05-09T04:57:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.410+0000 7fe06f7cb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681134f0 con 0x7fe068108740 2024-05-09T04:57:17.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.410+0000 7fe06cd66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe068107df0 0x7fe06819b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.410+0000 7fe06cd66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe068107df0 0x7fe06819b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40152/0 (socket says 172.21.15.46:40152) 2024-05-09T04:57:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.410+0000 7fe06cd66700 1 -- 172.21.15.46:0/4003357767 learned_addr learned my addr 172.21.15.46:0/4003357767 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.411+0000 7fe06d567700 1 --2- 172.21.15.46:0/4003357767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0681034c0 0x7fe06819ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.411+0000 7fe06dd68700 1 --2- 172.21.15.46:0/4003357767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe068108740 0x7fe06819f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.411+0000 7fe06cd66700 1 -- 172.21.15.46:0/4003357767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe068108740 msgr2=0x7fe06819f5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.411+0000 7fe06cd66700 1 --2- 172.21.15.46:0/4003357767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe068108740 0x7fe06819f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.411+0000 7fe06cd66700 1 -- 172.21.15.46:0/4003357767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0681034c0 msgr2=0x7fe06819ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.411+0000 7fe06cd66700 1 --2- 172.21.15.46:0/4003357767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0681034c0 0x7fe06819ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.411+0000 7fe06cd66700 1 -- 172.21.15.46:0/4003357767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe06819fbb0 con 0x7fe068107df0 2024-05-09T04:57:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.411+0000 7fe06d567700 1 --2- 172.21.15.46:0/4003357767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0681034c0 0x7fe06819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:57:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.411+0000 7fe06cd66700 1 --2- 172.21.15.46:0/4003357767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe068107df0 0x7fe06819b330 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe05800e970 tx=0x7fe05800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.411+0000 7fe05e7fc700 1 -- 172.21.15.46:0/4003357767 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05800cbd0 con 0x7fe068107df0 2024-05-09T04:57:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.411+0000 7fe06f7cb700 1 -- 172.21.15.46:0/4003357767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe06819fea0 con 0x7fe068107df0 2024-05-09T04:57:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.411+0000 7fe05e7fc700 1 -- 172.21.15.46:0/4003357767 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe05800cd70 con 0x7fe068107df0 2024-05-09T04:57:17.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.411+0000 7fe05e7fc700 1 -- 172.21.15.46:0/4003357767 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe058010430 con 0x7fe068107df0 2024-05-09T04:57:17.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.411+0000 7fe06f7cb700 1 -- 172.21.15.46:0/4003357767 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0681a03e0 con 0x7fe068107df0 2024-05-09T04:57:17.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.414+0000 7fe05e7fc700 1 -- 172.21.15.46:0/4003357767 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe058004d70 con 0x7fe068107df0 2024-05-09T04:57:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.415+0000 7fe05e7fc700 1 --2- 172.21.15.46:0/4003357767 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe0540b29f0 0x7fe0540b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.415+0000 7fe07e1e1b80 1 Processor -- start 2024-05-09T04:57:17.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.415+0000 7fe06d567700 1 --2- 172.21.15.46:0/4003357767 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe0540b29f0 0x7fe0540b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.415+0000 7fe05e7fc700 1 -- 172.21.15.46:0/4003357767 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fe0580e2940 con 0x7fe068107df0 2024-05-09T04:57:17.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.415+0000 7fe07e1e1b80 1 -- start start 2024-05-09T04:57:17.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe07e1e1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddbca160 0x55a1ddc634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe07e1e1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddae04a0 0x55a1ddc63a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe07e1e1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddc5d6a0 0x55a1ddc5dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe07e1e1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a1ddb7fff0 con 0x55a1ddae04a0 2024-05-09T04:57:17.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe07e1e1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a1ddb1e7f0 con 0x55a1ddbca160 2024-05-09T04:57:17.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe07e1e1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a1ddb7bfb0 con 0x55a1ddc5d6a0 2024-05-09T04:57:17.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe06cd66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddae04a0 0x55a1ddc63a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe06cd66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddae04a0 0x55a1ddc63a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40154/0 (socket says 172.21.15.46:40154) 2024-05-09T04:57:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe06cd66700 1 -- 172.21.15.46:0/1681117919 learned_addr learned my addr 172.21.15.46:0/1681117919 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe06d567700 1 --2- 172.21.15.46:0/4003357767 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe0540b29f0 0x7fe0540b4f00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe064005fd0 tx=0x7fe064005e30 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe06d567700 1 --2- 172.21.15.46:0/1681117919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddbca160 0x55a1ddc634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe06dd68700 1 --2- 172.21.15.46:0/1681117919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddc5d6a0 0x55a1ddc5dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe06cd66700 1 -- 172.21.15.46:0/1681117919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddc5d6a0 msgr2=0x55a1ddc5dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe06cd66700 1 --2- 172.21.15.46:0/1681117919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddc5d6a0 0x55a1ddc5dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe06cd66700 1 -- 172.21.15.46:0/1681117919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddbca160 msgr2=0x55a1ddc634f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe06cd66700 1 --2- 172.21.15.46:0/1681117919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddbca160 0x55a1ddc634f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe06cd66700 1 -- 172.21.15.46:0/1681117919 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1ddb16370 con 0x55a1ddae04a0 2024-05-09T04:57:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.416+0000 7fe06cd66700 1 --2- 172.21.15.46:0/1681117919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddae04a0 0x55a1ddc63a30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe0580a9b30 tx=0x7fe0580a5330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f29c1614006da3e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.417+0000 7fe0537fe700 1 -- 172.21.15.46:0/1681117919 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0580b2350 con 0x55a1ddae04a0 2024-05-09T04:57:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.417+0000 7fe0537fe700 1 -- 172.21.15.46:0/1681117919 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0580cae50 con 0x55a1ddae04a0 2024-05-09T04:57:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.417+0000 7fe0537fe700 1 -- 172.21.15.46:0/1681117919 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0580aa060 con 0x55a1ddae04a0 2024-05-09T04:57:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.418+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/1681117919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddae04a0 msgr2=0x55a1ddc63a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:17.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.418+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/1681117919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddae04a0 0x55a1ddc63a30 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fe0580a9b30 tx=0x7fe0580a5330 comp rx=0 tx=0).stop 2024-05-09T04:57:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.418+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/1681117919 shutdown_connections 2024-05-09T04:57:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.418+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/1681117919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddc5d6a0 0x55a1ddc5dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.418+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/1681117919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddbca160 0x55a1ddc634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.418+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/1681117919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddae04a0 0x55a1ddc63a30 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.418+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/1681117919 >> 172.21.15.46:0/1681117919 conn(0x55a1ddb406a0 msgr2=0x55a1ddc58400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.418+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/1681117919 shutdown_connections 2024-05-09T04:57:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.418+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/1681117919 wait complete. 2024-05-09T04:57:17.420 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.418+0000 7fe07e1e1b80 1 -- start start 2024-05-09T04:57:17.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.418+0000 7fe07e1e1b80 1 Processor -- start 2024-05-09T04:57:17.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe07e1e1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddae04a0 0x55a1ddc67280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe07e1e1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddbca160 0x55a1ddc677c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.423 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe07e1e1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddc5d6a0 0x55a1ddc67d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe07e1e1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55a1dd9fbba0 con 0x55a1ddc5d6a0 2024-05-09T04:57:17.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe07e1e1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55a1ddb3a150 con 0x55a1ddbca160 2024-05-09T04:57:17.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe07e1e1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55a1ddc6a1d0 con 0x55a1ddae04a0 2024-05-09T04:57:17.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe07e1e1b80 10 client.0 _ll_register_callbacks cb 0x55a1dd9f16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:17.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe06dd68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddc5d6a0 0x55a1ddc67d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe06dd68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddc5d6a0 0x55a1ddc67d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40166/0 (socket says 172.21.15.46:40166) 2024-05-09T04:57:17.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe06dd68700 1 -- 172.21.15.46:0/2171040660 learned_addr learned my addr 172.21.15.46:0/2171040660 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:17.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe06d567700 1 --2- 172.21.15.46:0/2171040660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddae04a0 0x55a1ddc67280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe06cd66700 1 --2- 172.21.15.46:0/2171040660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddbca160 0x55a1ddc677c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe06dd68700 1 -- 172.21.15.46:0/2171040660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddae04a0 msgr2=0x55a1ddc67280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:17.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe06dd68700 1 --2- 172.21.15.46:0/2171040660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddae04a0 0x55a1ddc67280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe06dd68700 1 -- 172.21.15.46:0/2171040660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddbca160 msgr2=0x55a1ddc677c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:17.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe06dd68700 1 --2- 172.21.15.46:0/2171040660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddbca160 0x55a1ddc677c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe06dd68700 1 -- 172.21.15.46:0/2171040660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1dd9fb490 con 0x55a1ddc5d6a0 2024-05-09T04:57:17.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.419+0000 7fe06cd66700 1 --2- 172.21.15.46:0/2171040660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddbca160 0x55a1ddc677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:57:17.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.420+0000 7fe06dd68700 1 --2- 172.21.15.46:0/2171040660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddc5d6a0 0x55a1ddc67d00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe06000bcd0 tx=0x7fe06000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36329b7f905ebf9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:17.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.420+0000 7fe050ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:17.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.420+0000 7fe050ff9700 1 -- 172.21.15.46:0/2171040660 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060004e60 con 0x55a1ddc5d6a0 2024-05-09T04:57:17.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.420+0000 7fe050ff9700 1 -- 172.21.15.46:0/2171040660 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fe0600075c0 con 0x55a1ddc5d6a0 2024-05-09T04:57:17.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.420+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/2171040660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe0580cae50 con 0x55a1ddc5d6a0 2024-05-09T04:57:17.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.420+0000 7fe050ff9700 1 -- 172.21.15.46:0/2171040660 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe0600077e0 con 0x55a1ddc5d6a0 2024-05-09T04:57:17.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.420+0000 7fe07e1e1b80 10 client.16791 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:17.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.420+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/2171040660 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a1ddc74c60 con 0x55a1ddc5d6a0 2024-05-09T04:57:17.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.420+0000 7fe050ff9700 1 -- 172.21.15.46:0/2171040660 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe0580cae50 con 0x55a1ddc5d6a0 2024-05-09T04:57:17.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.420+0000 7fe050ff9700 1 -- 172.21.15.46:0/2171040660 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fe060011070 con 0x55a1ddc5d6a0 2024-05-09T04:57:17.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.420+0000 7fe07e1e1b80 10 client.16791 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:17.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.420+0000 7fe07e1e1b80 10 client.16791 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:57:17.425 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.421+0000 7fe07e1e1b80 20 client.16791 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.421+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/2171040660 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a1ddc772a0 0x55a1ddc796c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.421+0000 7fe07e1e1b80 4 client.16791 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:57:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.421+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/2171040660 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1ddb8d9d0 con 0x55a1ddc772a0 2024-05-09T04:57:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.421+0000 7fe06d567700 1 --2- 172.21.15.46:0/2171040660 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a1ddc772a0 0x55a1ddc796c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.421+0000 7fe050ff9700 10 client.16791 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:57:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.421+0000 7fe06d567700 1 --2- 172.21.15.46:0/2171040660 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a1ddc772a0 0x55a1ddc796c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d40c9e5740aff5cb server_cookie=42de34f389553413 in_seq=0 out_seq=0 2024-05-09T04:57:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.422+0000 7fe050ff9700 1 -- 172.21.15.46:0/2171040660 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a1ddb8d9d0 con 0x55a1ddc772a0 2024-05-09T04:57:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.422+0000 7fe050ff9700 10 client.16791 handle_command_reply: tid=0 2024-05-09T04:57:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.422+0000 7fe07e1e1b80 1 client.16791 shutdown 2024-05-09T04:57:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.422+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/2171040660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddc5d6a0 msgr2=0x55a1ddc67d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.422+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/2171040660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddc5d6a0 0x55a1ddc67d00 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fe06000bcd0 tx=0x7fe06000bd00 comp rx=0 tx=0).stop 2024-05-09T04:57:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.422+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/2171040660 shutdown_connections 2024-05-09T04:57:17.426 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.422+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/2171040660 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddc5d6a0 0x55a1ddc67d00 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.422+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/2171040660 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a1ddc772a0 0x55a1ddc796c0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.422+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/2171040660 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddae04a0 0x55a1ddc67280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.422+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/2171040660 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddbca160 0x55a1ddc677c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.422+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/2171040660 >> 172.21.15.46:0/2171040660 conn(0x55a1ddb406a0 msgr2=0x55a1ddc581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.422+0000 7fe050ff9700 0 client.16791 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:57:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.423+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/2171040660 shutdown_connections 2024-05-09T04:57:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.423+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/2171040660 wait complete. 2024-05-09T04:57:17.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.423+0000 7fe07e1e1b80 20 client.16791 trim_cache size 0 max 16384 2024-05-09T04:57:17.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.438+0000 7fe07e1e1b80 1 Processor -- start 2024-05-09T04:57:17.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.438+0000 7fe07e1e1b80 1 -- start start 2024-05-09T04:57:17.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.438+0000 7fe07e1e1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddb406a0 0x55a1ddc628f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.438+0000 7fe07e1e1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddae04a0 0x55a1ddc61830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.438+0000 7fe07e1e1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddbca160 0x55a1ddc69040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.438+0000 7fe07e1e1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681b16e0 con 0x55a1ddbca160 2024-05-09T04:57:17.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.438+0000 7fe07e1e1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681aef70 con 0x55a1ddb406a0 2024-05-09T04:57:17.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.438+0000 7fe07e1e1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681acdc0 con 0x55a1ddae04a0 2024-05-09T04:57:17.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.438+0000 7fe06dd68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddbca160 0x55a1ddc69040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.438+0000 7fe06dd68700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddbca160 0x55a1ddc69040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40178/0 (socket says 172.21.15.46:40178) 2024-05-09T04:57:17.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.438+0000 7fe06dd68700 1 -- 172.21.15.46:0/599043216 learned_addr learned my addr 172.21.15.46:0/599043216 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:17.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.438+0000 7fe06d567700 1 --2- 172.21.15.46:0/599043216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddb406a0 0x55a1ddc628f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.438+0000 7fe06dd68700 1 -- 172.21.15.46:0/599043216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddae04a0 msgr2=0x55a1ddc61830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:17.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.438+0000 7fe06cd66700 1 --2- 172.21.15.46:0/599043216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddae04a0 0x55a1ddc61830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.438+0000 7fe06dd68700 1 --2- 172.21.15.46:0/599043216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddae04a0 0x55a1ddc61830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.439+0000 7fe06dd68700 1 -- 172.21.15.46:0/599043216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddb406a0 msgr2=0x55a1ddc628f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:17.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.439+0000 7fe06dd68700 1 --2- 172.21.15.46:0/599043216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddb406a0 0x55a1ddc628f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.439+0000 7fe06dd68700 1 -- 172.21.15.46:0/599043216 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1ddc51f60 con 0x55a1ddbca160 2024-05-09T04:57:17.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.439+0000 7fe06cd66700 1 --2- 172.21.15.46:0/599043216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddae04a0 0x55a1ddc61830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:57:17.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.439+0000 7fe06d567700 1 --2- 172.21.15.46:0/599043216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddb406a0 0x55a1ddc628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:57:17.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.439+0000 7fe06dd68700 1 --2- 172.21.15.46:0/599043216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddbca160 0x55a1ddc69040 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe060009a20 tx=0x7fe06000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853123e72595e9f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:17.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.439+0000 7fe052ffd700 1 -- 172.21.15.46:0/599043216 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060039070 con 0x55a1ddbca160 2024-05-09T04:57:17.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.439+0000 7fe052ffd700 1 -- 172.21.15.46:0/599043216 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe060005990 con 0x55a1ddbca160 2024-05-09T04:57:17.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.439+0000 7fe052ffd700 1 -- 172.21.15.46:0/599043216 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe060030070 con 0x55a1ddbca160 2024-05-09T04:57:17.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.440+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/599043216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddbca160 msgr2=0x55a1ddc69040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:17.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.440+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/599043216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddbca160 0x55a1ddc69040 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7fe060009a20 tx=0x7fe06000ce30 comp rx=0 tx=0).stop 2024-05-09T04:57:17.444 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.440+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/599043216 shutdown_connections 2024-05-09T04:57:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.440+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/599043216 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddbca160 0x55a1ddc69040 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.440+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/599043216 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddb406a0 0x55a1ddc628f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.440+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/599043216 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddae04a0 0x55a1ddc61830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.440+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/599043216 >> 172.21.15.46:0/599043216 conn(0x55a1ddc67e50 msgr2=0x55a1ddc68930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.440+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/599043216 shutdown_connections 2024-05-09T04:57:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.440+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/599043216 wait complete. 2024-05-09T04:57:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.440+0000 7fe07e1e1b80 1 -- start start 2024-05-09T04:57:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.440+0000 7fe07e1e1b80 1 Processor -- start 2024-05-09T04:57:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe07e1e1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddae04a0 0x55a1ddc6cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe07e1e1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddb406a0 0x55a1ddc6d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe07e1e1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddbca160 0x55a1ddc6d990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe07e1e1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681ad430 con 0x55a1ddb406a0 2024-05-09T04:57:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe07e1e1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681ae8a0 con 0x55a1ddae04a0 2024-05-09T04:57:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe07e1e1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe0681adb00 con 0x55a1ddbca160 2024-05-09T04:57:17.445 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe07e1e1b80 10 client.0 _ll_register_callbacks cb 0x55a1dd9f16c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:17.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe06cd66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddb406a0 0x55a1ddc6d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe06cd66700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddb406a0 0x55a1ddc6d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40184/0 (socket says 172.21.15.46:40184) 2024-05-09T04:57:17.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe06cd66700 1 -- 172.21.15.46:0/980116423 learned_addr learned my addr 172.21.15.46:0/980116423 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:17.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe06d567700 1 --2- 172.21.15.46:0/980116423 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddae04a0 0x55a1ddc6cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe06dd68700 1 --2- 172.21.15.46:0/980116423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddbca160 0x55a1ddc6d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe06cd66700 1 -- 172.21.15.46:0/980116423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddbca160 msgr2=0x55a1ddc6d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:17.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe06cd66700 1 --2- 172.21.15.46:0/980116423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddbca160 0x55a1ddc6d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe06cd66700 1 -- 172.21.15.46:0/980116423 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddae04a0 msgr2=0x55a1ddc6cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:17.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe06cd66700 1 --2- 172.21.15.46:0/980116423 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddae04a0 0x55a1ddc6cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe06cd66700 1 -- 172.21.15.46:0/980116423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1ddce0a40 con 0x55a1ddb406a0 2024-05-09T04:57:17.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe06cd66700 1 --2- 172.21.15.46:0/980116423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddb406a0 0x55a1ddc6d450 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe058000c00 tx=0x7fe0580b2780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcfc0d4f9bcf6c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:17.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe050ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:17.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.441+0000 7fe050ff9700 1 -- 172.21.15.46:0/980116423 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe05806d060 con 0x55a1ddb406a0 2024-05-09T04:57:17.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.442+0000 7fe050ff9700 1 -- 172.21.15.46:0/980116423 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fe0580a6190 con 0x55a1ddb406a0 2024-05-09T04:57:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.442+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/980116423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe060005990 con 0x55a1ddb406a0 2024-05-09T04:57:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.442+0000 7fe050ff9700 1 -- 172.21.15.46:0/980116423 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe0580c9050 con 0x55a1ddb406a0 2024-05-09T04:57:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.442+0000 7fe07e1e1b80 10 client.16797 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.442+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/980116423 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55a1ddce1ab0 con 0x55a1ddb406a0 2024-05-09T04:57:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.442+0000 7fe050ff9700 1 -- 172.21.15.46:0/980116423 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe060005990 con 0x55a1ddb406a0 2024-05-09T04:57:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.442+0000 7fe050ff9700 1 -- 172.21.15.46:0/980116423 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fe05806f300 con 0x55a1ddb406a0 2024-05-09T04:57:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.442+0000 7fe07e1e1b80 10 client.16797 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.442+0000 7fe07e1e1b80 10 client.16797 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:57:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.442+0000 7fe07e1e1b80 20 client.16797 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.442+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/980116423 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a1ddc5d6a0 0x55a1ddce3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.442+0000 7fe07e1e1b80 4 client.16797 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:57:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.442+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/980116423 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1ddb87a70 con 0x55a1ddc5d6a0 2024-05-09T04:57:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.442+0000 7fe06d567700 1 --2- 172.21.15.46:0/980116423 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a1ddc5d6a0 0x55a1ddce3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:17.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.443+0000 7fe06d567700 1 --2- 172.21.15.46:0/980116423 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a1ddc5d6a0 0x55a1ddce3da0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2b4f9aefc14c0774 server_cookie=d7f4f1c6c48a2688 in_seq=0 out_seq=0 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.443+0000 7fe050ff9700 10 client.16797 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.443+0000 7fe050ff9700 1 -- 172.21.15.46:0/980116423 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1ddb87a70 con 0x55a1ddc5d6a0 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.443+0000 7fe050ff9700 10 client.16797 handle_command_reply: tid=0 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.443+0000 7fe07e1e1b80 1 client.16797 shutdown 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.443+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/980116423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddb406a0 msgr2=0x55a1ddc6d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.443+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/980116423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddb406a0 0x55a1ddc6d450 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fe058000c00 tx=0x7fe0580b2780 comp rx=0 tx=0).stop 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.444+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/980116423 shutdown_connections 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.444+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/980116423 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55a1ddc5d6a0 0x55a1ddce3da0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.444+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/980116423 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55a1ddbca160 0x55a1ddc6d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.444+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/980116423 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55a1ddae04a0 0x55a1ddc6cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.444+0000 7fe07e1e1b80 1 --2- 172.21.15.46:0/980116423 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55a1ddb406a0 0x55a1ddc6d450 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.444+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/980116423 >> 172.21.15.46:0/980116423 conn(0x55a1ddc67e50 msgr2=0x55a1ddc61590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.444+0000 7fe050ff9700 0 client.16797 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.444+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/980116423 shutdown_connections 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.444+0000 7fe07e1e1b80 1 -- 172.21.15.46:0/980116423 wait complete. 2024-05-09T04:57:17.448 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.444+0000 7fe07e1e1b80 20 client.16797 trim_cache size 0 max 16384 2024-05-09T04:57:17.449 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:57:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.445+0000 7fe06f7cb700 1 -- 172.21.15.46:0/4003357767 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe0540b29f0 msgr2=0x7fe0540b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.445+0000 7fe06f7cb700 1 --2- 172.21.15.46:0/4003357767 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe0540b29f0 0x7fe0540b4f00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe064005fd0 tx=0x7fe064005e30 comp rx=0 tx=0).stop 2024-05-09T04:57:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.445+0000 7fe06f7cb700 1 -- 172.21.15.46:0/4003357767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe068107df0 msgr2=0x7fe06819b330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.445+0000 7fe06f7cb700 1 --2- 172.21.15.46:0/4003357767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe068107df0 0x7fe06819b330 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fe05800e970 tx=0x7fe05800ed30 comp rx=0 tx=0).stop 2024-05-09T04:57:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.445+0000 7fe06f7cb700 1 -- 172.21.15.46:0/4003357767 shutdown_connections 2024-05-09T04:57:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.445+0000 7fe06f7cb700 1 --2- 172.21.15.46:0/4003357767 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe0540b29f0 0x7fe0540b4f00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.445+0000 7fe06f7cb700 1 --2- 172.21.15.46:0/4003357767 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe068108740 0x7fe06819f5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.445+0000 7fe06f7cb700 1 --2- 172.21.15.46:0/4003357767 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe0681034c0 0x7fe06819ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.445+0000 7fe06f7cb700 1 --2- 172.21.15.46:0/4003357767 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe068107df0 0x7fe06819b330 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.445+0000 7fe06f7cb700 1 -- 172.21.15.46:0/4003357767 >> 172.21.15.46:0/4003357767 conn(0x7fe0680fedf0 msgr2=0x7fe068075c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.445+0000 7fe06f7cb700 1 -- 172.21.15.46:0/4003357767 shutdown_connections 2024-05-09T04:57:17.450 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:17.445+0000 7fe06f7cb700 1 -- 172.21.15.46:0/4003357767 wait complete. 2024-05-09T04:57:18.171 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:57:18.428 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:57:18.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:18 smithi046 ceph-mon[30136]: pgmap v189: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-09T04:57:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:18 smithi110 ceph-mon[34904]: pgmap v189: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-09T04:57:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:18 smithi072 ceph-mon[34088]: pgmap v189: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 85 B/s wr, 8 op/s 2024-05-09T04:57:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.223+0000 7f980dd7b700 1 -- 172.21.15.46:0/458071001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808108f10 msgr2=0x7f9808109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.223+0000 7f980dd7b700 1 --2- 172.21.15.46:0/458071001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808108f10 0x7f9808109320 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f97f00099c0 tx=0x7f97f0009cd0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.223+0000 7f980dd7b700 1 -- 172.21.15.46:0/458071001 shutdown_connections 2024-05-09T04:57:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.223+0000 7f980dd7b700 1 --2- 172.21.15.46:0/458071001 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9808109860 0x7f98081111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.223+0000 7f980dd7b700 1 --2- 172.21.15.46:0/458071001 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f98081045e0 0x7f98081049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.223+0000 7f980dd7b700 1 --2- 172.21.15.46:0/458071001 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808108f10 0x7f9808109320 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.223+0000 7f980dd7b700 1 -- 172.21.15.46:0/458071001 >> 172.21.15.46:0/458071001 conn(0x7f98080ffed0 msgr2=0x7f9808102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.223+0000 7f980dd7b700 1 -- 172.21.15.46:0/458071001 shutdown_connections 2024-05-09T04:57:20.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.223+0000 7f980dd7b700 1 -- 172.21.15.46:0/458071001 wait complete. 2024-05-09T04:57:20.226 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.224+0000 7f980dd7b700 1 Processor -- start 2024-05-09T04:57:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.224+0000 7f980dd7b700 1 -- start start 2024-05-09T04:57:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.224+0000 7f980dd7b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98081045e0 0x7f980819ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.224+0000 7f980dd7b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808108f10 0x7f980819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f980dd7b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9808109860 0x7f980819f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f980dd7b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9808114470 con 0x7f9808108f10 2024-05-09T04:57:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f980dd7b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f98081142f0 con 0x7f9808109860 2024-05-09T04:57:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f980dd7b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f98081145f0 con 0x7f98081045e0 2024-05-09T04:57:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f9806ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808108f10 0x7f980819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f9806ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808108f10 0x7f980819b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40200/0 (socket says 172.21.15.46:40200) 2024-05-09T04:57:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f9806ffd700 1 -- 172.21.15.46:0/888466880 learned_addr learned my addr 172.21.15.46:0/888466880 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f98077fe700 1 --2- 172.21.15.46:0/888466880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98081045e0 0x7f980819ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f9807fff700 1 --2- 172.21.15.46:0/888466880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9808109860 0x7f980819f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f9806ffd700 1 -- 172.21.15.46:0/888466880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98081045e0 msgr2=0x7f980819ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f9806ffd700 1 --2- 172.21.15.46:0/888466880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98081045e0 0x7f980819ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f9806ffd700 1 -- 172.21.15.46:0/888466880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9808109860 msgr2=0x7f980819f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f9806ffd700 1 --2- 172.21.15.46:0/888466880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9808109860 0x7f980819f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f9806ffd700 1 -- 172.21.15.46:0/888466880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f980819fd10 con 0x7f9808108f10 2024-05-09T04:57:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f9807fff700 1 --2- 172.21.15.46:0/888466880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9808109860 0x7f980819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:57:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f9806ffd700 1 --2- 172.21.15.46:0/888466880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808108f10 0x7f980819b3e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f97f00098e0 tx=0x7f97f00057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.225+0000 7f9804ff9700 1 -- 172.21.15.46:0/888466880 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0027070 con 0x7f9808108f10 2024-05-09T04:57:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.226+0000 7f980dd7b700 1 -- 172.21.15.46:0/888466880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f980819ffa0 con 0x7f9808108f10 2024-05-09T04:57:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.226+0000 7f9804ff9700 1 -- 172.21.15.46:0/888466880 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97f0003f20 con 0x7f9808108f10 2024-05-09T04:57:20.228 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.226+0000 7f980dd7b700 1 -- 172.21.15.46:0/888466880 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98081a7920 con 0x7f9808108f10 2024-05-09T04:57:20.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.226+0000 7f9804ff9700 1 -- 172.21.15.46:0/888466880 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0022d70 con 0x7f9808108f10 2024-05-09T04:57:20.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.228+0000 7f9804ff9700 1 -- 172.21.15.46:0/888466880 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f97f002a070 con 0x7f9808108f10 2024-05-09T04:57:20.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.230+0000 7f9804ff9700 1 --2- 172.21.15.46:0/888466880 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f97f40b29f0 0x7f97f40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.230+0000 7f981c791b80 1 Processor -- start 2024-05-09T04:57:20.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.230+0000 7f981c791b80 1 -- start start 2024-05-09T04:57:20.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.230+0000 7f98077fe700 1 --2- 172.21.15.46:0/888466880 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f97f40b29f0 0x7f97f40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.230+0000 7f9804ff9700 1 -- 172.21.15.46:0/888466880 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f97f00e3260 con 0x7f9808108f10 2024-05-09T04:57:20.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.230+0000 7f981c791b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 0x55fa8fcf8240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.230+0000 7f981c791b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fcf8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.230+0000 7f981c791b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fcf2400 0x55fa8fcf2810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.230+0000 7f981c791b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fa8fc150a0 con 0x55fa8fb754a0 2024-05-09T04:57:20.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.230+0000 7f981c791b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55fa8fbba9a0 con 0x55fa8fcf2400 2024-05-09T04:57:20.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.230+0000 7f981c791b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fa8fc11060 con 0x55fa8fbefd00 2024-05-09T04:57:20.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.230+0000 7f9806ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fcf8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.230+0000 7f9806ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fcf8780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40208/0 (socket says 172.21.15.46:40208) 2024-05-09T04:57:20.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.230+0000 7f98077fe700 1 --2- 172.21.15.46:0/888466880 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f97f40b29f0 0x7f97f40b4f00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009550 tx=0x7f97f80091b0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:20.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.230+0000 7f9806ffd700 1 -- 172.21.15.46:0/1531145136 learned_addr learned my addr 172.21.15.46:0/1531145136 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:20.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.231+0000 7f98077fe700 1 --2- 172.21.15.46:0/1531145136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 0x55fa8fcf8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.231+0000 7f9807fff700 1 --2- 172.21.15.46:0/1531145136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fcf2400 0x55fa8fcf2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.231+0000 7f9806ffd700 1 -- 172.21.15.46:0/1531145136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 msgr2=0x55fa8fcf8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:20.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.231+0000 7f9806ffd700 1 --2- 172.21.15.46:0/1531145136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 0x55fa8fcf8240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.231+0000 7f9806ffd700 1 -- 172.21.15.46:0/1531145136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fcf2400 msgr2=0x55fa8fcf2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:20.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.231+0000 7f9806ffd700 1 --2- 172.21.15.46:0/1531145136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fcf2400 0x55fa8fcf2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.231+0000 7f9806ffd700 1 -- 172.21.15.46:0/1531145136 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa8fbab500 con 0x55fa8fb754a0 2024-05-09T04:57:20.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.231+0000 7f9806ffd700 1 --2- 172.21.15.46:0/1531145136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fcf8780 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f97f00aa4e0 tx=0x7f97f0009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af9e11a726bc509a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:20.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.231+0000 7f97fdffb700 1 -- 172.21.15.46:0/1531145136 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f0092110 con 0x55fa8fb754a0 2024-05-09T04:57:20.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.231+0000 7f97fdffb700 1 -- 172.21.15.46:0/1531145136 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97f00aa550 con 0x55fa8fb754a0 2024-05-09T04:57:20.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.231+0000 7f97fdffb700 1 -- 172.21.15.46:0/1531145136 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f00b30f0 con 0x55fa8fb754a0 2024-05-09T04:57:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.232+0000 7f981c791b80 1 -- 172.21.15.46:0/1531145136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 msgr2=0x55fa8fcf8780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.232+0000 7f981c791b80 1 --2- 172.21.15.46:0/1531145136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fcf8780 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f97f00aa4e0 tx=0x7f97f0009cd0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.232+0000 7f981c791b80 1 -- 172.21.15.46:0/1531145136 shutdown_connections 2024-05-09T04:57:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.232+0000 7f981c791b80 1 --2- 172.21.15.46:0/1531145136 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fcf2400 0x55fa8fcf2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.232+0000 7f981c791b80 1 --2- 172.21.15.46:0/1531145136 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 0x55fa8fcf8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.232+0000 7f981c791b80 1 --2- 172.21.15.46:0/1531145136 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fcf8780 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.232+0000 7f981c791b80 1 -- 172.21.15.46:0/1531145136 >> 172.21.15.46:0/1531145136 conn(0x55fa8fb78f70 msgr2=0x55fa8fced220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:20.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.232+0000 7f981c791b80 1 -- 172.21.15.46:0/1531145136 shutdown_connections 2024-05-09T04:57:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.232+0000 7f981c791b80 1 -- 172.21.15.46:0/1531145136 wait complete. 2024-05-09T04:57:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.233+0000 7f981c791b80 1 -- start start 2024-05-09T04:57:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.233+0000 7f981c791b80 1 Processor -- start 2024-05-09T04:57:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.233+0000 7f981c791b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fd01bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.233+0000 7f981c791b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fbefd00 0x55fa8fd02100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.233+0000 7f981c791b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fcf2400 0x55fa8fcfbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.233+0000 7f981c791b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fa8fcf2fb0 con 0x55fa8fb754a0 2024-05-09T04:57:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.233+0000 7f981c791b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55fa8fbafb20 con 0x55fa8fbefd00 2024-05-09T04:57:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.233+0000 7f981c791b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fa8fcf3130 con 0x55fa8fcf2400 2024-05-09T04:57:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.233+0000 7f981c791b80 10 client.0 _ll_register_callbacks cb 0x55fa8fa866c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.233+0000 7f98077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fd01bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.233+0000 7f98077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fd01bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40222/0 (socket says 172.21.15.46:40222) 2024-05-09T04:57:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f98077fe700 1 -- 172.21.15.46:0/3179474389 learned_addr learned my addr 172.21.15.46:0/3179474389 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f9807fff700 1 --2- 172.21.15.46:0/3179474389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fcf2400 0x55fa8fcfbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.237 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f9806ffd700 1 --2- 172.21.15.46:0/3179474389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fbefd00 0x55fa8fd02100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f98077fe700 1 -- 172.21.15.46:0/3179474389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fcf2400 msgr2=0x55fa8fcfbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f98077fe700 1 --2- 172.21.15.46:0/3179474389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fcf2400 0x55fa8fcfbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f98077fe700 1 -- 172.21.15.46:0/3179474389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fbefd00 msgr2=0x55fa8fd02100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f98077fe700 1 --2- 172.21.15.46:0/3179474389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fbefd00 0x55fa8fd02100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f98077fe700 1 -- 172.21.15.46:0/3179474389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa8fa90490 con 0x55fa8fb754a0 2024-05-09T04:57:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f9806ffd700 1 --2- 172.21.15.46:0/3179474389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fbefd00 0x55fa8fd02100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:57:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f98077fe700 1 --2- 172.21.15.46:0/3179474389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fd01bc0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f97f8007ab0 tx=0x7f97f8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a019db0d82a8b16 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f97e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f97e6ffd700 1 -- 172.21.15.46:0/3179474389 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f8026020 con 0x55fa8fb754a0 2024-05-09T04:57:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f97e6ffd700 1 -- 172.21.15.46:0/3179474389 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f97f8015ce0 con 0x55fa8fb754a0 2024-05-09T04:57:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f981c791b80 1 -- 172.21.15.46:0/3179474389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97f00aa550 con 0x55fa8fb754a0 2024-05-09T04:57:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f97e6ffd700 1 -- 172.21.15.46:0/3179474389 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f97f801c050 con 0x55fa8fb754a0 2024-05-09T04:57:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f981c791b80 10 client.16809 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:20.238 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.234+0000 7f981c791b80 1 -- 172.21.15.46:0/3179474389 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa8fd09840 con 0x55fa8fb754a0 2024-05-09T04:57:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.235+0000 7f97e6ffd700 1 -- 172.21.15.46:0/3179474389 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97f00aa550 con 0x55fa8fb754a0 2024-05-09T04:57:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.235+0000 7f97e6ffd700 1 -- 172.21.15.46:0/3179474389 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f97f80163f0 con 0x55fa8fb754a0 2024-05-09T04:57:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.235+0000 7f981c791b80 10 client.16809 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.235+0000 7f981c791b80 10 client.16809 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:57:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.235+0000 7f981c791b80 20 client.16809 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.235+0000 7f981c791b80 1 --2- 172.21.15.46:0/3179474389 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55fa8fd0be80 0x55fa8fd0e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.235+0000 7f981c791b80 4 client.16809 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:57:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.235+0000 7f981c791b80 1 -- 172.21.15.46:0/3179474389 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa8fc22a80 con 0x55fa8fd0be80 2024-05-09T04:57:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.235+0000 7f9806ffd700 1 --2- 172.21.15.46:0/3179474389 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55fa8fd0be80 0x55fa8fd0e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.236+0000 7f9806ffd700 1 --2- 172.21.15.46:0/3179474389 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55fa8fd0be80 0x55fa8fd0e310 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d6cb6c50af3e2f3b server_cookie=6cfd7e0d38c8cecc in_seq=0 out_seq=0 2024-05-09T04:57:20.239 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.236+0000 7f97e6ffd700 10 client.16809 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:57:20.240 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.236+0000 7f97e6ffd700 1 -- 172.21.15.46:0/3179474389 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fa8fc22a80 con 0x55fa8fd0be80 2024-05-09T04:57:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.236+0000 7f97e6ffd700 10 client.16809 handle_command_reply: tid=0 2024-05-09T04:57:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.237+0000 7f981c791b80 1 client.16809 shutdown 2024-05-09T04:57:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.237+0000 7f981c791b80 1 -- 172.21.15.46:0/3179474389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 msgr2=0x55fa8fd01bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.237+0000 7f981c791b80 1 --2- 172.21.15.46:0/3179474389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fd01bc0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f97f8007ab0 tx=0x7f97f8004b40 comp rx=0 tx=0).stop 2024-05-09T04:57:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.237+0000 7f981c791b80 1 -- 172.21.15.46:0/3179474389 shutdown_connections 2024-05-09T04:57:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.237+0000 7f981c791b80 1 --2- 172.21.15.46:0/3179474389 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55fa8fd0be80 0x55fa8fd0e310 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.237+0000 7f981c791b80 1 --2- 172.21.15.46:0/3179474389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fcf2400 0x55fa8fcfbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.237+0000 7f981c791b80 1 --2- 172.21.15.46:0/3179474389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fd01bc0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.237+0000 7f981c791b80 1 --2- 172.21.15.46:0/3179474389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fbefd00 0x55fa8fd02100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.237+0000 7f981c791b80 1 -- 172.21.15.46:0/3179474389 >> 172.21.15.46:0/3179474389 conn(0x55fa8fb78f70 msgr2=0x55fa8fce6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.237+0000 7f97e6ffd700 0 client.16809 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:57:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.237+0000 7f981c791b80 1 -- 172.21.15.46:0/3179474389 shutdown_connections 2024-05-09T04:57:20.241 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.237+0000 7f981c791b80 1 -- 172.21.15.46:0/3179474389 wait complete. 2024-05-09T04:57:20.242 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.237+0000 7f981c791b80 20 client.16809 trim_cache size 0 max 16384 2024-05-09T04:57:20.253 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.252+0000 7f981c791b80 1 Processor -- start 2024-05-09T04:57:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.252+0000 7f981c791b80 1 -- start start 2024-05-09T04:57:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.252+0000 7f981c791b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fb754a0 0x55fa8fd0e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f981c791b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 0x55fa8fce7a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f981c791b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fcf2400 0x55fa8fcf1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f981c791b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f98081b1760 con 0x55fa8fcf2400 2024-05-09T04:57:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f981c791b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f98081aeff0 con 0x55fa8fb754a0 2024-05-09T04:57:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f981c791b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f98081ace40 con 0x55fa8fbefd00 2024-05-09T04:57:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f9807fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fcf2400 0x55fa8fcf1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f9807fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fcf2400 0x55fa8fcf1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40228/0 (socket says 172.21.15.46:40228) 2024-05-09T04:57:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f9807fff700 1 -- 172.21.15.46:0/1813303938 learned_addr learned my addr 172.21.15.46:0/1813303938 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f9806ffd700 1 --2- 172.21.15.46:0/1813303938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 0x55fa8fce7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.254 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f98077fe700 1 --2- 172.21.15.46:0/1813303938 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fb754a0 0x55fa8fd0e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f9807fff700 1 -- 172.21.15.46:0/1813303938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 msgr2=0x55fa8fce7a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f9807fff700 1 --2- 172.21.15.46:0/1813303938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 0x55fa8fce7a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f9807fff700 1 -- 172.21.15.46:0/1813303938 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fb754a0 msgr2=0x55fa8fd0e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f9807fff700 1 --2- 172.21.15.46:0/1813303938 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fb754a0 0x55fa8fd0e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f9807fff700 1 -- 172.21.15.46:0/1813303938 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa8fce99d0 con 0x55fa8fcf2400 2024-05-09T04:57:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f9807fff700 1 --2- 172.21.15.46:0/1813303938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fcf2400 0x55fa8fcf1b00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f980000d8c0 tx=0x7f980000dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfca9eab5fc7161e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f97fd7fa700 1 -- 172.21.15.46:0/1813303938 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800004830 con 0x55fa8fcf2400 2024-05-09T04:57:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f97fd7fa700 1 -- 172.21.15.46:0/1813303938 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9800005020 con 0x55fa8fcf2400 2024-05-09T04:57:20.255 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.253+0000 7f97fd7fa700 1 -- 172.21.15.46:0/1813303938 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9800003a30 con 0x55fa8fcf2400 2024-05-09T04:57:20.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.254+0000 7f981c791b80 1 -- 172.21.15.46:0/1813303938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fcf2400 msgr2=0x55fa8fcf1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:20.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.254+0000 7f981c791b80 1 --2- 172.21.15.46:0/1813303938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fcf2400 0x55fa8fcf1b00 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f980000d8c0 tx=0x7f980000dbd0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.254+0000 7f981c791b80 1 -- 172.21.15.46:0/1813303938 shutdown_connections 2024-05-09T04:57:20.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.254+0000 7f981c791b80 1 --2- 172.21.15.46:0/1813303938 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fcf2400 0x55fa8fcf1b00 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.254+0000 7f981c791b80 1 --2- 172.21.15.46:0/1813303938 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fb754a0 0x55fa8fd0e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.259 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.254+0000 7f981c791b80 1 --2- 172.21.15.46:0/1813303938 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 0x55fa8fce7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.254+0000 7f981c791b80 1 -- 172.21.15.46:0/1813303938 >> 172.21.15.46:0/1813303938 conn(0x55fa8fb78f70 msgr2=0x55fa8fcf13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.254+0000 7f981c791b80 1 -- 172.21.15.46:0/1813303938 shutdown_connections 2024-05-09T04:57:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.254+0000 7f981c791b80 1 -- 172.21.15.46:0/1813303938 wait complete. 2024-05-09T04:57:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.255+0000 7f981c791b80 1 -- start start 2024-05-09T04:57:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.255+0000 7f981c791b80 1 Processor -- start 2024-05-09T04:57:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.255+0000 7f981c791b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fd052f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.255+0000 7f981c791b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 0x55fa8fd05830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.255+0000 7f981c791b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fcf2400 0x55fa8fd74680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.255+0000 7f981c791b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f98081ad4b0 con 0x55fa8fb754a0 2024-05-09T04:57:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.255+0000 7f981c791b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f98081ae920 con 0x55fa8fcf2400 2024-05-09T04:57:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.255+0000 7f981c791b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f98081adb80 con 0x55fa8fbefd00 2024-05-09T04:57:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.255+0000 7f981c791b80 10 client.0 _ll_register_callbacks cb 0x55fa8fa866c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.255+0000 7f98077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fd052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.260 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.255+0000 7f98077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fd052f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40236/0 (socket says 172.21.15.46:40236) 2024-05-09T04:57:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.255+0000 7f98077fe700 1 -- 172.21.15.46:0/3241765421 learned_addr learned my addr 172.21.15.46:0/3241765421 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.255+0000 7f9806ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 0x55fa8fd05830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.255+0000 7f9807fff700 1 --2- 172.21.15.46:0/3241765421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fcf2400 0x55fa8fd74680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.256+0000 7f98077fe700 1 -- 172.21.15.46:0/3241765421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 msgr2=0x55fa8fd05830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.256+0000 7f98077fe700 1 --2- 172.21.15.46:0/3241765421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 0x55fa8fd05830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.256+0000 7f98077fe700 1 -- 172.21.15.46:0/3241765421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fcf2400 msgr2=0x55fa8fd74680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.256+0000 7f98077fe700 1 --2- 172.21.15.46:0/3241765421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fcf2400 0x55fa8fd74680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.256+0000 7f98077fe700 1 -- 172.21.15.46:0/3241765421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa8fce6cc0 con 0x55fa8fb754a0 2024-05-09T04:57:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.256+0000 7f9806ffd700 1 --2- 172.21.15.46:0/3241765421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 0x55fa8fd05830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.256+0000 7f98077fe700 1 --2- 172.21.15.46:0/3241765421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fd052f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f97f8016860 tx=0x7f97f80166a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74ce41954f70d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.256+0000 7f97e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.256+0000 7f97e6ffd700 1 -- 172.21.15.46:0/3241765421 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f803a060 con 0x55fa8fb754a0 2024-05-09T04:57:20.261 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.256+0000 7f97e6ffd700 1 -- 172.21.15.46:0/3241765421 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f97f8035480 con 0x55fa8fb754a0 2024-05-09T04:57:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.256+0000 7f981c791b80 1 -- 172.21.15.46:0/3241765421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9800005020 con 0x55fa8fb754a0 2024-05-09T04:57:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.256+0000 7f97e6ffd700 1 -- 172.21.15.46:0/3241765421 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f97f8026040 con 0x55fa8fb754a0 2024-05-09T04:57:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.256+0000 7f981c791b80 10 client.16815 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.256+0000 7f981c791b80 1 -- 172.21.15.46:0/3241765421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fa8fd75be0 con 0x55fa8fb754a0 2024-05-09T04:57:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.257+0000 7f97e6ffd700 1 -- 172.21.15.46:0/3241765421 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9800005020 con 0x55fa8fb754a0 2024-05-09T04:57:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.257+0000 7f97e6ffd700 1 -- 172.21.15.46:0/3241765421 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f97f8022470 con 0x55fa8fb754a0 2024-05-09T04:57:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.257+0000 7f981c791b80 10 client.16815 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.257+0000 7f981c791b80 10 client.16815 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:57:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.257+0000 7f981c791b80 20 client.16815 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:20.262 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.257+0000 7f981c791b80 1 --2- 172.21.15.46:0/3241765421 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55fa8fd762f0 0x55fa8fd78710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.257+0000 7f981c791b80 4 client.16815 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:57:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.257+0000 7f981c791b80 1 -- 172.21.15.46:0/3241765421 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fa8fc1cb20 con 0x55fa8fd762f0 2024-05-09T04:57:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.257+0000 7f9806ffd700 1 --2- 172.21.15.46:0/3241765421 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55fa8fd762f0 0x55fa8fd78710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.257+0000 7f97e6ffd700 10 client.16815 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:57:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.257+0000 7f9806ffd700 1 --2- 172.21.15.46:0/3241765421 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55fa8fd762f0 0x55fa8fd78710 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9babe6c6f6caf5bf server_cookie=f301493fe6deb7e6 in_seq=0 out_seq=0 2024-05-09T04:57:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.258+0000 7f97e6ffd700 1 -- 172.21.15.46:0/3241765421 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fa8fc1cb20 con 0x55fa8fd762f0 2024-05-09T04:57:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.258+0000 7f97e6ffd700 10 client.16815 handle_command_reply: tid=0 2024-05-09T04:57:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.258+0000 7f981c791b80 1 client.16815 shutdown 2024-05-09T04:57:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.258+0000 7f981c791b80 1 -- 172.21.15.46:0/3241765421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 msgr2=0x55fa8fd052f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.258+0000 7f981c791b80 1 --2- 172.21.15.46:0/3241765421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fd052f0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f97f8016860 tx=0x7f97f80166a0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.258+0000 7f981c791b80 1 -- 172.21.15.46:0/3241765421 shutdown_connections 2024-05-09T04:57:20.263 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.258+0000 7f981c791b80 1 --2- 172.21.15.46:0/3241765421 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55fa8fd762f0 0x55fa8fd78710 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.258+0000 7f981c791b80 1 --2- 172.21.15.46:0/3241765421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fa8fcf2400 0x55fa8fd74680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.258+0000 7f981c791b80 1 --2- 172.21.15.46:0/3241765421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fa8fb754a0 0x55fa8fd052f0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.258+0000 7f981c791b80 1 --2- 172.21.15.46:0/3241765421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fa8fbefd00 0x55fa8fd05830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.258+0000 7f981c791b80 1 -- 172.21.15.46:0/3241765421 >> 172.21.15.46:0/3241765421 conn(0x55fa8fb78f70 msgr2=0x55fa8fce7780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.258+0000 7f97e6ffd700 0 client.16815 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:57:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.258+0000 7f981c791b80 1 -- 172.21.15.46:0/3241765421 shutdown_connections 2024-05-09T04:57:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.258+0000 7f981c791b80 1 -- 172.21.15.46:0/3241765421 wait complete. 2024-05-09T04:57:20.264 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.258+0000 7f981c791b80 20 client.16815 trim_cache size 0 max 16384 2024-05-09T04:57:20.264 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:57:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.259+0000 7f980dd7b700 1 -- 172.21.15.46:0/888466880 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f97f40b29f0 msgr2=0x7f97f40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.259+0000 7f980dd7b700 1 --2- 172.21.15.46:0/888466880 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f97f40b29f0 0x7f97f40b4f00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f97f8009550 tx=0x7f97f80091b0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.259+0000 7f980dd7b700 1 -- 172.21.15.46:0/888466880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808108f10 msgr2=0x7f980819b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.259+0000 7f980dd7b700 1 --2- 172.21.15.46:0/888466880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808108f10 0x7f980819b3e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f97f00098e0 tx=0x7f97f00057c0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.259+0000 7f980dd7b700 1 -- 172.21.15.46:0/888466880 shutdown_connections 2024-05-09T04:57:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.259+0000 7f980dd7b700 1 --2- 172.21.15.46:0/888466880 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f97f40b29f0 0x7f97f40b4f00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.259+0000 7f980dd7b700 1 --2- 172.21.15.46:0/888466880 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9808109860 0x7f980819f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.259+0000 7f980dd7b700 1 --2- 172.21.15.46:0/888466880 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f98081045e0 0x7f980819ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.265 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.259+0000 7f980dd7b700 1 --2- 172.21.15.46:0/888466880 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9808108f10 0x7f980819b3e0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:20.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.259+0000 7f980dd7b700 1 -- 172.21.15.46:0/888466880 >> 172.21.15.46:0/888466880 conn(0x7f98080ffed0 msgr2=0x7f980810a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:20.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.259+0000 7f980dd7b700 1 -- 172.21.15.46:0/888466880 shutdown_connections 2024-05-09T04:57:20.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:20.259+0000 7f980dd7b700 1 -- 172.21.15.46:0/888466880 wait complete. 2024-05-09T04:57:20.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:20 smithi046 ceph-mon[30136]: pgmap v190: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:57:21.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:20 smithi110 ceph-mon[34904]: pgmap v190: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:57:21.093 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:57:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:20 smithi072 ceph-mon[34088]: pgmap v190: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:57:21.349 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:57:23.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:22 smithi110 ceph-mon[34904]: pgmap v191: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T04:57:23.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:22 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:57:23.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:22 smithi046 ceph-mon[30136]: pgmap v191: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T04:57:23.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:22 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:57:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.118+0000 7fde9ea63700 1 -- 172.21.15.46:0/2326699428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde98108740 msgr2=0x7fde981100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.118+0000 7fde9ea63700 1 --2- 172.21.15.46:0/2326699428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde98108740 0x7fde981100a0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fde9000b0d0 tx=0x7fde9000b3e0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.118+0000 7fde9ea63700 1 -- 172.21.15.46:0/2326699428 shutdown_connections 2024-05-09T04:57:23.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.118+0000 7fde9ea63700 1 --2- 172.21.15.46:0/2326699428 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde98108740 0x7fde981100a0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.118+0000 7fde9ea63700 1 --2- 172.21.15.46:0/2326699428 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde981034c0 0x7fde981038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.118+0000 7fde9ea63700 1 --2- 172.21.15.46:0/2326699428 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde98107df0 0x7fde98108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.118+0000 7fde9ea63700 1 -- 172.21.15.46:0/2326699428 >> 172.21.15.46:0/2326699428 conn(0x7fde980fedf0 msgr2=0x7fde98101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.118+0000 7fde9ea63700 1 -- 172.21.15.46:0/2326699428 shutdown_connections 2024-05-09T04:57:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.118+0000 7fde9ea63700 1 -- 172.21.15.46:0/2326699428 wait complete. 2024-05-09T04:57:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.119+0000 7fde9ea63700 1 Processor -- start 2024-05-09T04:57:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.119+0000 7fde9ea63700 1 -- start start 2024-05-09T04:57:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.119+0000 7fde9ea63700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde981034c0 0x7fde9819aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.119+0000 7fde9ea63700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde98107df0 0x7fde9819b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.119+0000 7fde9ea63700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde98108740 0x7fde9819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.119+0000 7fde9ea63700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde98113390 con 0x7fde98108740 2024-05-09T04:57:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.119+0000 7fde9ea63700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fde98113210 con 0x7fde98107df0 2024-05-09T04:57:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.119+0000 7fde9ea63700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fde98113510 con 0x7fde981034c0 2024-05-09T04:57:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.119+0000 7fde97fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde981034c0 0x7fde9819aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.119+0000 7fde9d000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde98108740 0x7fde9819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.119+0000 7fde97fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde981034c0 0x7fde9819aef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:55234/0 (socket says 172.21.15.46:55234) 2024-05-09T04:57:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.119+0000 7fde97fff700 1 -- 172.21.15.46:0/3175054263 learned_addr learned my addr 172.21.15.46:0/3175054263 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.119+0000 7fde977fe700 1 --2- 172.21.15.46:0/3175054263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde98107df0 0x7fde9819b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.120+0000 7fde97fff700 1 -- 172.21.15.46:0/3175054263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde98107df0 msgr2=0x7fde9819b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.120+0000 7fde97fff700 1 --2- 172.21.15.46:0/3175054263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde98107df0 0x7fde9819b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.120+0000 7fde97fff700 1 -- 172.21.15.46:0/3175054263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde98108740 msgr2=0x7fde9819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.120+0000 7fde97fff700 1 --2- 172.21.15.46:0/3175054263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde98108740 0x7fde9819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.120+0000 7fde97fff700 1 -- 172.21.15.46:0/3175054263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde9819fdc0 con 0x7fde981034c0 2024-05-09T04:57:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.120+0000 7fde97fff700 1 --2- 172.21.15.46:0/3175054263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde981034c0 0x7fde9819aef0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fde8800eb30 tx=0x7fde8800ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.120+0000 7fde957fa700 1 -- 172.21.15.46:0/3175054263 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8800cbd0 con 0x7fde981034c0 2024-05-09T04:57:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.120+0000 7fde957fa700 1 -- 172.21.15.46:0/3175054263 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde880044d0 con 0x7fde981034c0 2024-05-09T04:57:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.120+0000 7fde9ea63700 1 -- 172.21.15.46:0/3175054263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde981a00b0 con 0x7fde981034c0 2024-05-09T04:57:23.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.120+0000 7fde957fa700 1 -- 172.21.15.46:0/3175054263 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde88003e30 con 0x7fde981034c0 2024-05-09T04:57:23.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.120+0000 7fde9ea63700 1 -- 172.21.15.46:0/3175054263 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde981a79d0 con 0x7fde981034c0 2024-05-09T04:57:23.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.123+0000 7fde957fa700 1 -- 172.21.15.46:0/3175054263 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fde88018430 con 0x7fde981034c0 2024-05-09T04:57:23.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.124+0000 7fde957fa700 1 --2- 172.21.15.46:0/3175054263 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fde800b29f0 0x7fde800b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.124+0000 7fde977fe700 1 --2- 172.21.15.46:0/3175054263 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fde800b29f0 0x7fde800b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.124+0000 7fde957fa700 1 -- 172.21.15.46:0/3175054263 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fde880e29f0 con 0x7fde981034c0 2024-05-09T04:57:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.124+0000 7fdead479b80 1 Processor -- start 2024-05-09T04:57:23.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.124+0000 7fdead479b80 1 -- start start 2024-05-09T04:57:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.124+0000 7fde977fe700 1 --2- 172.21.15.46:0/3175054263 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fde800b29f0 0x7fde800b4f00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fde98109900 tx=0x7fde8c009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.124+0000 7fdead479b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7c7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.124+0000 7fdead479b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 0x5617eb7c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.124+0000 7fdead479b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 0x5617eb7c1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.124+0000 7fdead479b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5617eb6e00a0 con 0x5617eb6bad00 2024-05-09T04:57:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.124+0000 7fdead479b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5617eb6859a0 con 0x5617eb7c1810 2024-05-09T04:57:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.124+0000 7fdead479b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5617eb6dc060 con 0x5617eb6404a0 2024-05-09T04:57:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.125+0000 7fde97fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.125+0000 7fde97fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7c7620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40280/0 (socket says 172.21.15.46:40280) 2024-05-09T04:57:23.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.125+0000 7fde97fff700 1 -- 172.21.15.46:0/390952123 learned_addr learned my addr 172.21.15.46:0/390952123 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.125+0000 7fde977fe700 1 --2- 172.21.15.46:0/390952123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 0x5617eb7c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.125+0000 7fde9d000700 1 --2- 172.21.15.46:0/390952123 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 0x5617eb7c1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.125+0000 7fde977fe700 1 -- 172.21.15.46:0/390952123 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 msgr2=0x5617eb7c1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.125+0000 7fde977fe700 1 --2- 172.21.15.46:0/390952123 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 0x5617eb7c1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.125+0000 7fde977fe700 1 -- 172.21.15.46:0/390952123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 msgr2=0x5617eb7c7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.125+0000 7fde977fe700 1 --2- 172.21.15.46:0/390952123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7c7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.125+0000 7fde977fe700 1 -- 172.21.15.46:0/390952123 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617eb676500 con 0x5617eb6404a0 2024-05-09T04:57:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.125+0000 7fde97fff700 1 --2- 172.21.15.46:0/390952123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.125+0000 7fde977fe700 1 --2- 172.21.15.46:0/390952123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 0x5617eb7c7bc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fde8c011c30 tx=0x7fde8c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a1d444ac668dd97 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.125+0000 7fde7e7fc700 1 -- 172.21.15.46:0/390952123 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c022070 con 0x5617eb6404a0 2024-05-09T04:57:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.125+0000 7fde7e7fc700 1 -- 172.21.15.46:0/390952123 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde8c004500 con 0x5617eb6404a0 2024-05-09T04:57:23.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.125+0000 7fde7e7fc700 1 -- 172.21.15.46:0/390952123 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c003a00 con 0x5617eb6404a0 2024-05-09T04:57:23.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.126+0000 7fdead479b80 1 -- 172.21.15.46:0/390952123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 msgr2=0x5617eb7c7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:23.131 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.126+0000 7fdead479b80 1 --2- 172.21.15.46:0/390952123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 0x5617eb7c7bc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fde8c011c30 tx=0x7fde8c014000 comp rx=0 tx=0).stop 2024-05-09T04:57:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.126+0000 7fdead479b80 1 -- 172.21.15.46:0/390952123 shutdown_connections 2024-05-09T04:57:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.126+0000 7fdead479b80 1 --2- 172.21.15.46:0/390952123 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 0x5617eb7c1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.126+0000 7fdead479b80 1 --2- 172.21.15.46:0/390952123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7c7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.126+0000 7fdead479b80 1 --2- 172.21.15.46:0/390952123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 0x5617eb7c7bc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.126+0000 7fdead479b80 1 -- 172.21.15.46:0/390952123 >> 172.21.15.46:0/390952123 conn(0x5617eb643f70 msgr2=0x5617eb7b8220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.126+0000 7fdead479b80 1 -- 172.21.15.46:0/390952123 shutdown_connections 2024-05-09T04:57:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.126+0000 7fdead479b80 1 -- 172.21.15.46:0/390952123 wait complete. 2024-05-09T04:57:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.127+0000 7fdead479b80 1 -- start start 2024-05-09T04:57:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.127+0000 7fdead479b80 1 Processor -- start 2024-05-09T04:57:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.127+0000 7fdead479b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 0x5617eb7cb440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.127+0000 7fdead479b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7cb980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.127+0000 7fdead479b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 0x5617eb7cbec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.127+0000 7fdead479b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5617eb7c2570 con 0x5617eb6bad00 2024-05-09T04:57:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.127+0000 7fdead479b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5617eb67ab20 con 0x5617eb7c1810 2024-05-09T04:57:23.132 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.127+0000 7fdead479b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5617eb7ce420 con 0x5617eb6404a0 2024-05-09T04:57:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fdead479b80 10 client.0 _ll_register_callbacks cb 0x5617eb5516c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fde977fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7cb980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fde97fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 0x5617eb7cb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fde977fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7cb980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40282/0 (socket says 172.21.15.46:40282) 2024-05-09T04:57:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fde977fe700 1 -- 172.21.15.46:0/1581036412 learned_addr learned my addr 172.21.15.46:0/1581036412 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fde9d000700 1 --2- 172.21.15.46:0/1581036412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 0x5617eb7cbec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fde97fff700 1 -- 172.21.15.46:0/1581036412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 msgr2=0x5617eb7cbec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fde97fff700 1 --2- 172.21.15.46:0/1581036412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 0x5617eb7cbec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fde97fff700 1 -- 172.21.15.46:0/1581036412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 msgr2=0x5617eb7cb980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fde97fff700 1 --2- 172.21.15.46:0/1581036412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7cb980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fde97fff700 1 -- 172.21.15.46:0/1581036412 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617eb55b490 con 0x5617eb6404a0 2024-05-09T04:57:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fde9d000700 1 --2- 172.21.15.46:0/1581036412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 0x5617eb7cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fde97fff700 1 --2- 172.21.15.46:0/1581036412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 0x5617eb7cb440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fde880a5d30 tx=0x7fde880e0650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=947f11d472dd48ca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:23.133 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fde777fe700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:57:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fde777fe700 1 -- 172.21.15.46:0/1581036412 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde880a9420 con 0x5617eb6404a0 2024-05-09T04:57:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fde777fe700 1 -- 172.21.15.46:0/1581036412 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fde880a62a0 con 0x5617eb6404a0 2024-05-09T04:57:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.128+0000 7fdead479b80 1 -- 172.21.15.46:0/1581036412 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde8c004500 con 0x5617eb6404a0 2024-05-09T04:57:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.129+0000 7fde777fe700 1 -- 172.21.15.46:0/1581036412 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde880a5440 con 0x5617eb6404a0 2024-05-09T04:57:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.129+0000 7fdead479b80 10 client.26284 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.129+0000 7fdead479b80 1 -- 172.21.15.46:0/1581036412 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5617eb7d8e70 con 0x5617eb6404a0 2024-05-09T04:57:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.129+0000 7fde777fe700 1 -- 172.21.15.46:0/1581036412 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde8c004500 con 0x5617eb6404a0 2024-05-09T04:57:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.129+0000 7fde777fe700 1 -- 172.21.15.46:0/1581036412 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fde880a8f10 con 0x5617eb6404a0 2024-05-09T04:57:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.129+0000 7fdead479b80 10 client.26284 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.129+0000 7fdead479b80 10 client.26284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:57:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.129+0000 7fdead479b80 20 client.26284 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.129+0000 7fdead479b80 1 --2- 172.21.15.46:0/1581036412 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5617eb7db4b0 0x5617eb7dd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.129+0000 7fdead479b80 4 client.26284 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:57:23.134 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.129+0000 7fdead479b80 1 -- 172.21.15.46:0/1581036412 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5617eb6eda80 con 0x5617eb7db4b0 2024-05-09T04:57:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.129+0000 7fde977fe700 1 --2- 172.21.15.46:0/1581036412 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5617eb7db4b0 0x5617eb7dd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.130+0000 7fde777fe700 10 client.26284 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.130+0000 7fde977fe700 1 --2- 172.21.15.46:0/1581036412 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5617eb7db4b0 0x5617eb7dd8d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7e0162b522174e0 server_cookie=3fbac26bfd7a3be3 in_seq=0 out_seq=0 2024-05-09T04:57:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.130+0000 7fde777fe700 1 -- 172.21.15.46:0/1581036412 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5617eb6eda80 con 0x5617eb7db4b0 2024-05-09T04:57:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.130+0000 7fde777fe700 10 client.26284 handle_command_reply: tid=0 2024-05-09T04:57:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.131+0000 7fdead479b80 1 client.26284 shutdown 2024-05-09T04:57:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.131+0000 7fdead479b80 1 -- 172.21.15.46:0/1581036412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 msgr2=0x5617eb7cb440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.131+0000 7fdead479b80 1 --2- 172.21.15.46:0/1581036412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 0x5617eb7cb440 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fde880a5d30 tx=0x7fde880e0650 comp rx=0 tx=0).stop 2024-05-09T04:57:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.131+0000 7fdead479b80 1 -- 172.21.15.46:0/1581036412 shutdown_connections 2024-05-09T04:57:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.131+0000 7fdead479b80 1 --2- 172.21.15.46:0/1581036412 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5617eb7db4b0 0x5617eb7dd8d0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.131+0000 7fdead479b80 1 --2- 172.21.15.46:0/1581036412 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 0x5617eb7cbec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.131+0000 7fde777fe700 0 client.26284 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.131+0000 7fdead479b80 1 --2- 172.21.15.46:0/1581036412 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 0x5617eb7cb440 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.131+0000 7fdead479b80 1 --2- 172.21.15.46:0/1581036412 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7cb980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.135 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.131+0000 7fdead479b80 1 -- 172.21.15.46:0/1581036412 >> 172.21.15.46:0/1581036412 conn(0x5617eb643f70 msgr2=0x5617eb7bc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:23.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.131+0000 7fdead479b80 1 -- 172.21.15.46:0/1581036412 shutdown_connections 2024-05-09T04:57:23.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.131+0000 7fdead479b80 1 -- 172.21.15.46:0/1581036412 wait complete. 2024-05-09T04:57:23.136 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.131+0000 7fdead479b80 20 client.26284 trim_cache size 0 max 16384 2024-05-09T04:57:23.136 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:22 smithi072 ceph-mon[34088]: pgmap v191: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T04:57:23.136 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:22 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:57:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.146+0000 7fdead479b80 1 Processor -- start 2024-05-09T04:57:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.146+0000 7fdead479b80 1 -- start start 2024-05-09T04:57:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.146+0000 7fdead479b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 0x5617eb7ca780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.146+0000 7fdead479b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.146+0000 7fdead479b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 0x5617eb7b2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.146+0000 7fdead479b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde981af600 con 0x5617eb6bad00 2024-05-09T04:57:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.146+0000 7fdead479b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fde981ace90 con 0x5617eb7c1810 2024-05-09T04:57:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.146+0000 7fdead479b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fde981aace0 con 0x5617eb6404a0 2024-05-09T04:57:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.147+0000 7fde977fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.147+0000 7fde977fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40292/0 (socket says 172.21.15.46:40292) 2024-05-09T04:57:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.147+0000 7fde977fe700 1 -- 172.21.15.46:0/385861486 learned_addr learned my addr 172.21.15.46:0/385861486 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.147+0000 7fde97fff700 1 --2- 172.21.15.46:0/385861486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 0x5617eb7ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.147+0000 7fde9d000700 1 --2- 172.21.15.46:0/385861486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 0x5617eb7b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.147+0000 7fde977fe700 1 -- 172.21.15.46:0/385861486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 msgr2=0x5617eb7ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.147+0000 7fde977fe700 1 --2- 172.21.15.46:0/385861486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 0x5617eb7ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.147+0000 7fde977fe700 1 -- 172.21.15.46:0/385861486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 msgr2=0x5617eb7b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.147+0000 7fde977fe700 1 --2- 172.21.15.46:0/385861486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 0x5617eb7b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.147+0000 7fde977fe700 1 -- 172.21.15.46:0/385861486 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5617eb7b49d0 con 0x5617eb6bad00 2024-05-09T04:57:23.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.147+0000 7fde97fff700 1 --2- 172.21.15.46:0/385861486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 0x5617eb7ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.147+0000 7fde977fe700 1 --2- 172.21.15.46:0/385861486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7cbd00 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00e690 tx=0x7fde8c0092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6ec2cb1bb2cf1e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.147+0000 7fde7dffb700 1 -- 172.21.15.46:0/385861486 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c022040 con 0x5617eb6bad00 2024-05-09T04:57:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.147+0000 7fde7dffb700 1 -- 172.21.15.46:0/385861486 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde8c037070 con 0x5617eb6bad00 2024-05-09T04:57:23.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.147+0000 7fde7dffb700 1 -- 172.21.15.46:0/385861486 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8c0276b0 con 0x5617eb6bad00 2024-05-09T04:57:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.148+0000 7fdead479b80 1 -- 172.21.15.46:0/385861486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 msgr2=0x5617eb7cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:23.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.148+0000 7fdead479b80 1 --2- 172.21.15.46:0/385861486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7cbd00 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fde8c00e690 tx=0x7fde8c0092d0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.148+0000 7fdead479b80 1 -- 172.21.15.46:0/385861486 shutdown_connections 2024-05-09T04:57:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.148+0000 7fdead479b80 1 --2- 172.21.15.46:0/385861486 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb7c1810 0x5617eb7b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.148+0000 7fdead479b80 1 --2- 172.21.15.46:0/385861486 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6404a0 0x5617eb7ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.148+0000 7fdead479b80 1 --2- 172.21.15.46:0/385861486 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb6bad00 0x5617eb7cbd00 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.148+0000 7fdead479b80 1 -- 172.21.15.46:0/385861486 >> 172.21.15.46:0/385861486 conn(0x5617eb643f70 msgr2=0x5617eb7b1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.148+0000 7fdead479b80 1 -- 172.21.15.46:0/385861486 shutdown_connections 2024-05-09T04:57:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.148+0000 7fdead479b80 1 -- 172.21.15.46:0/385861486 wait complete. 2024-05-09T04:57:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.149+0000 7fdead479b80 1 -- start start 2024-05-09T04:57:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.149+0000 7fdead479b80 1 Processor -- start 2024-05-09T04:57:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.149+0000 7fdead479b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb6404a0 0x5617eb7d3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.149+0000 7fdead479b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6bad00 0x5617eb7d3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.149+0000 7fdead479b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb7c1810 0x5617eb7e2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.149+0000 7fdead479b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fde981ab350 con 0x5617eb7c1810 2024-05-09T04:57:23.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.149+0000 7fdead479b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fde981ac7c0 con 0x5617eb6404a0 2024-05-09T04:57:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.149+0000 7fdead479b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fde981aba20 con 0x5617eb6bad00 2024-05-09T04:57:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.149+0000 7fdead479b80 10 client.0 _ll_register_callbacks cb 0x5617eb5516c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.149+0000 7fde9d000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb7c1810 0x5617eb7e2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde9d000700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb7c1810 0x5617eb7e2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:40302/0 (socket says 172.21.15.46:40302) 2024-05-09T04:57:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde9d000700 1 -- 172.21.15.46:0/4118986110 learned_addr learned my addr 172.21.15.46:0/4118986110 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde97fff700 1 --2- 172.21.15.46:0/4118986110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb6404a0 0x5617eb7d3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde977fe700 1 --2- 172.21.15.46:0/4118986110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6bad00 0x5617eb7d3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde9d000700 1 -- 172.21.15.46:0/4118986110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6bad00 msgr2=0x5617eb7d3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde9d000700 1 --2- 172.21.15.46:0/4118986110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6bad00 0x5617eb7d3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde9d000700 1 -- 172.21.15.46:0/4118986110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb6404a0 msgr2=0x5617eb7d3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde9d000700 1 --2- 172.21.15.46:0/4118986110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb6404a0 0x5617eb7d3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde9d000700 1 -- 172.21.15.46:0/4118986110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5617eb7e25a0 con 0x5617eb7c1810 2024-05-09T04:57:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde97fff700 1 --2- 172.21.15.46:0/4118986110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb6404a0 0x5617eb7d3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:23.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde977fe700 1 --2- 172.21.15.46:0/4118986110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6bad00 0x5617eb7d3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde9d000700 1 --2- 172.21.15.46:0/4118986110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb7c1810 0x5617eb7e2060 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fde90009610 tx=0x7fde9000bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5bd30de391419 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde777fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde777fe700 1 -- 172.21.15.46:0/4118986110 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde9002a060 con 0x5617eb7c1810 2024-05-09T04:57:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde777fe700 1 -- 172.21.15.46:0/4118986110 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fde900075c0 con 0x5617eb7c1810 2024-05-09T04:57:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fdead479b80 1 -- 172.21.15.46:0/4118986110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde8c037070 con 0x5617eb7c1810 2024-05-09T04:57:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.150+0000 7fde777fe700 1 -- 172.21.15.46:0/4118986110 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fde900077c0 con 0x5617eb7c1810 2024-05-09T04:57:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.151+0000 7fdead479b80 10 client.16833 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.151+0000 7fdead479b80 1 -- 172.21.15.46:0/4118986110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5617eb845380 con 0x5617eb7c1810 2024-05-09T04:57:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.151+0000 7fde777fe700 1 -- 172.21.15.46:0/4118986110 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde8c037070 con 0x5617eb7c1810 2024-05-09T04:57:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.151+0000 7fde777fe700 1 -- 172.21.15.46:0/4118986110 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fde90011070 con 0x5617eb7c1810 2024-05-09T04:57:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.151+0000 7fdead479b80 10 client.16833 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.151+0000 7fdead479b80 10 client.16833 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:57:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.151+0000 7fdead479b80 20 client.16833 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:23.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.151+0000 7fdead479b80 1 --2- 172.21.15.46:0/4118986110 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5617eb845ac0 0x5617eb847ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.151+0000 7fdead479b80 4 client.16833 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:57:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.151+0000 7fdead479b80 1 -- 172.21.15.46:0/4118986110 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5617eb6e7b20 con 0x5617eb845ac0 2024-05-09T04:57:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.151+0000 7fde97fff700 1 --2- 172.21.15.46:0/4118986110 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5617eb845ac0 0x5617eb847ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fde777fe700 10 client.16833 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fde97fff700 1 --2- 172.21.15.46:0/4118986110 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5617eb845ac0 0x5617eb847ee0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=378db3f2120b963c server_cookie=cc028f41c80f4f2f in_seq=0 out_seq=0 2024-05-09T04:57:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fde777fe700 1 -- 172.21.15.46:0/4118986110 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5617eb6e7b20 con 0x5617eb845ac0 2024-05-09T04:57:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fde777fe700 10 client.16833 handle_command_reply: tid=0 2024-05-09T04:57:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fdead479b80 1 client.16833 shutdown 2024-05-09T04:57:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fdead479b80 1 -- 172.21.15.46:0/4118986110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb7c1810 msgr2=0x5617eb7e2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fdead479b80 1 --2- 172.21.15.46:0/4118986110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb7c1810 0x5617eb7e2060 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fde90009610 tx=0x7fde9000bf70 comp rx=0 tx=0).stop 2024-05-09T04:57:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fdead479b80 1 -- 172.21.15.46:0/4118986110 shutdown_connections 2024-05-09T04:57:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fdead479b80 1 --2- 172.21.15.46:0/4118986110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5617eb7c1810 0x5617eb7e2060 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fdead479b80 1 --2- 172.21.15.46:0/4118986110 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5617eb845ac0 0x5617eb847ee0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fdead479b80 1 --2- 172.21.15.46:0/4118986110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5617eb6404a0 0x5617eb7d3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fde777fe700 0 client.16833 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fdead479b80 1 --2- 172.21.15.46:0/4118986110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5617eb6bad00 0x5617eb7d3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fdead479b80 1 -- 172.21.15.46:0/4118986110 >> 172.21.15.46:0/4118986110 conn(0x5617eb643f70 msgr2=0x5617eb7ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fdead479b80 1 -- 172.21.15.46:0/4118986110 shutdown_connections 2024-05-09T04:57:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fdead479b80 1 -- 172.21.15.46:0/4118986110 wait complete. 2024-05-09T04:57:23.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.152+0000 7fdead479b80 20 client.16833 trim_cache size 0 max 16384 2024-05-09T04:57:23.157 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:57:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.153+0000 7fde9ea63700 1 -- 172.21.15.46:0/3175054263 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fde800b29f0 msgr2=0x7fde800b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.153+0000 7fde9ea63700 1 --2- 172.21.15.46:0/3175054263 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fde800b29f0 0x7fde800b4f00 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fde98109900 tx=0x7fde8c009260 comp rx=0 tx=0).stop 2024-05-09T04:57:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.153+0000 7fde9ea63700 1 -- 172.21.15.46:0/3175054263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde981034c0 msgr2=0x7fde9819aef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.153+0000 7fde9ea63700 1 --2- 172.21.15.46:0/3175054263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde981034c0 0x7fde9819aef0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fde8800eb30 tx=0x7fde8800ee40 comp rx=0 tx=0).stop 2024-05-09T04:57:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.153+0000 7fde9ea63700 1 -- 172.21.15.46:0/3175054263 shutdown_connections 2024-05-09T04:57:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.153+0000 7fde9ea63700 1 --2- 172.21.15.46:0/3175054263 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fde800b29f0 0x7fde800b4f00 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.153+0000 7fde9ea63700 1 --2- 172.21.15.46:0/3175054263 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fde98108740 0x7fde9819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.153+0000 7fde9ea63700 1 --2- 172.21.15.46:0/3175054263 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fde981034c0 0x7fde9819aef0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.153+0000 7fde9ea63700 1 --2- 172.21.15.46:0/3175054263 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fde98107df0 0x7fde9819b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.153+0000 7fde9ea63700 1 -- 172.21.15.46:0/3175054263 >> 172.21.15.46:0/3175054263 conn(0x7fde980fedf0 msgr2=0x7fde98075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:23.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.153+0000 7fde9ea63700 1 -- 172.21.15.46:0/3175054263 shutdown_connections 2024-05-09T04:57:23.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:23.153+0000 7fde9ea63700 1 -- 172.21.15.46:0/3175054263 wait complete. 2024-05-09T04:57:24.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:23 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:57:24.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:23 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:57:24.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:23 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:57:24.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:23 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:57:24.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:23 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:57:24.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:23 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:57:24.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:23 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:57:24.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:23 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:57:24.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:23 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:57:24.895 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:57:24.895 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:57:25.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:24 smithi110 ceph-mon[34904]: pgmap v192: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:57:25.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:24 smithi046 ceph-mon[30136]: pgmap v192: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:57:25.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:24 smithi072 ceph-mon[34088]: pgmap v192: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:57:25.163 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:57:26.831 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:26 smithi046 ceph-mon[30136]: pgmap v193: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:26.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.979+0000 7fd36759e700 1 -- 172.21.15.46:0/1445614738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3681032f0 msgr2=0x7fd368103700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:26.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.979+0000 7fd36759e700 1 --2- 172.21.15.46:0/1445614738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3681032f0 0x7fd368103700 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fd3580099c0 tx=0x7fd358009cd0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.979+0000 7fd36759e700 1 -- 172.21.15.46:0/1445614738 shutdown_connections 2024-05-09T04:57:26.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.979+0000 7fd36759e700 1 --2- 172.21.15.46:0/1445614738 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd368108570 0x7fd36810fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.979+0000 7fd36759e700 1 --2- 172.21.15.46:0/1445614738 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd3681032f0 0x7fd368103700 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.979+0000 7fd36759e700 1 --2- 172.21.15.46:0/1445614738 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd368107c20 0x7fd368108030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.979+0000 7fd36759e700 1 -- 172.21.15.46:0/1445614738 >> 172.21.15.46:0/1445614738 conn(0x7fd368078b50 msgr2=0x7fd368078f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:26.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.979+0000 7fd36759e700 1 -- 172.21.15.46:0/1445614738 shutdown_connections 2024-05-09T04:57:26.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.979+0000 7fd36759e700 1 -- 172.21.15.46:0/1445614738 wait complete. 2024-05-09T04:57:26.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.980+0000 7fd36759e700 1 Processor -- start 2024-05-09T04:57:26.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.980+0000 7fd36759e700 1 -- start start 2024-05-09T04:57:26.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.980+0000 7fd36759e700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3681032f0 0x7fd36819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:26.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.980+0000 7fd36759e700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd368107c20 0x7fd36819d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:26.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd36759e700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd368108570 0x7fd3681a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:26.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd36759e700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681142f0 con 0x7fd368107c20 2024-05-09T04:57:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd36759e700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd368114170 con 0x7fd3681032f0 2024-05-09T04:57:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd36759e700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd368114470 con 0x7fd368108570 2024-05-09T04:57:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd365d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd368107c20 0x7fd36819d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd365d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd368107c20 0x7fd36819d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58862/0 (socket says 172.21.15.46:58862) 2024-05-09T04:57:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd365d9b700 1 -- 172.21.15.46:0/2787227696 learned_addr learned my addr 172.21.15.46:0/2787227696 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd36659c700 1 --2- 172.21.15.46:0/2787227696 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3681032f0 0x7fd36819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd366d9d700 1 --2- 172.21.15.46:0/2787227696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd368108570 0x7fd3681a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd365d9b700 1 -- 172.21.15.46:0/2787227696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd368108570 msgr2=0x7fd3681a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd365d9b700 1 --2- 172.21.15.46:0/2787227696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd368108570 0x7fd3681a1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd365d9b700 1 -- 172.21.15.46:0/2787227696 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3681032f0 msgr2=0x7fd36819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd365d9b700 1 --2- 172.21.15.46:0/2787227696 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3681032f0 0x7fd36819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd365d9b700 1 -- 172.21.15.46:0/2787227696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3681a1cf0 con 0x7fd368107c20 2024-05-09T04:57:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd366d9d700 1 --2- 172.21.15.46:0/2787227696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd368108570 0x7fd3681a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:57:26.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd365d9b700 1 --2- 172.21.15.46:0/2787227696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd368107c20 0x7fd36819d3c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00d980 tx=0x7fd35c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:26.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd3577fe700 1 -- 172.21.15.46:0/2787227696 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c017070 con 0x7fd368107c20 2024-05-09T04:57:26.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd3577fe700 1 -- 172.21.15.46:0/2787227696 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd35c009de0 con 0x7fd368107c20 2024-05-09T04:57:26.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd36759e700 1 -- 172.21.15.46:0/2787227696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3681a1fe0 con 0x7fd368107c20 2024-05-09T04:57:26.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.981+0000 7fd3577fe700 1 -- 172.21.15.46:0/2787227696 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c00f3c0 con 0x7fd368107c20 2024-05-09T04:57:26.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.982+0000 7fd36759e700 1 -- 172.21.15.46:0/2787227696 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3681a9900 con 0x7fd368107c20 2024-05-09T04:57:26.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.984+0000 7fd3577fe700 1 -- 172.21.15.46:0/2787227696 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd35c00f560 con 0x7fd368107c20 2024-05-09T04:57:26.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.985+0000 7fd3577fe700 1 --2- 172.21.15.46:0/2787227696 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd3500b29f0 0x7fd3500b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:26.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.985+0000 7fd36659c700 1 --2- 172.21.15.46:0/2787227696 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd3500b29f0 0x7fd3500b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:26.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.986+0000 7fd3577fe700 1 -- 172.21.15.46:0/2787227696 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fd35c0e2540 con 0x7fd368107c20 2024-05-09T04:57:26.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.986+0000 7fd36659c700 1 --2- 172.21.15.46:0/2787227696 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd3500b29f0 0x7fd3500b4f00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd358009990 tx=0x7fd3580036e0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:26.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.986+0000 7fd37b419b80 1 Processor -- start 2024-05-09T04:57:26.986 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.986+0000 7fd37b419b80 1 -- start start 2024-05-09T04:57:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.986+0000 7fd37b419b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fe491e0 0x55da6fee26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd37b419b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fd5f520 0x55da6fee2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd37b419b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 0x55da6fedcc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd37b419b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55da6fdff070 con 0x55da6fe491e0 2024-05-09T04:57:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd37b419b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55da6fd9d870 con 0x55da6fd5f520 2024-05-09T04:57:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd37b419b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55da6fdfb030 con 0x55da6fedc880 2024-05-09T04:57:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd36659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fe491e0 0x55da6fee26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:26.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd36659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fe491e0 0x55da6fee26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58868/0 (socket says 172.21.15.46:58868) 2024-05-09T04:57:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd36659c700 1 -- 172.21.15.46:0/2677173511 learned_addr learned my addr 172.21.15.46:0/2677173511 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd365d9b700 1 --2- 172.21.15.46:0/2677173511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fd5f520 0x55da6fee2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd366d9d700 1 --2- 172.21.15.46:0/2677173511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 0x55da6fedcc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd36659c700 1 -- 172.21.15.46:0/2677173511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 msgr2=0x55da6fedcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd36659c700 1 --2- 172.21.15.46:0/2677173511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 0x55da6fedcc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd36659c700 1 -- 172.21.15.46:0/2677173511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fd5f520 msgr2=0x55da6fee2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd36659c700 1 --2- 172.21.15.46:0/2677173511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fd5f520 0x55da6fee2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd36659c700 1 -- 172.21.15.46:0/2677173511 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da6fd953f0 con 0x55da6fe491e0 2024-05-09T04:57:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd366d9d700 1 --2- 172.21.15.46:0/2677173511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 0x55da6fedcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:57:26.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd36659c700 1 --2- 172.21.15.46:0/2677173511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fe491e0 0x55da6fee26c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd3580099c0 tx=0x7fd35801e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bb7c3dcac592361 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd354ff9700 1 -- 172.21.15.46:0/2677173511 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358005430 con 0x55da6fe491e0 2024-05-09T04:57:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd354ff9700 1 -- 172.21.15.46:0/2677173511 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3580055d0 con 0x55da6fe491e0 2024-05-09T04:57:26.990 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.987+0000 7fd354ff9700 1 -- 172.21.15.46:0/2677173511 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35800f940 con 0x55da6fe491e0 2024-05-09T04:57:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.988+0000 7fd37b419b80 1 -- 172.21.15.46:0/2677173511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fe491e0 msgr2=0x55da6fee26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.988+0000 7fd37b419b80 1 --2- 172.21.15.46:0/2677173511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fe491e0 0x55da6fee26c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fd3580099c0 tx=0x7fd35801e000 comp rx=0 tx=0).stop 2024-05-09T04:57:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.988+0000 7fd37b419b80 1 -- 172.21.15.46:0/2677173511 shutdown_connections 2024-05-09T04:57:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.988+0000 7fd37b419b80 1 --2- 172.21.15.46:0/2677173511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 0x55da6fedcc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.988+0000 7fd37b419b80 1 --2- 172.21.15.46:0/2677173511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fe491e0 0x55da6fee26c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.988+0000 7fd37b419b80 1 --2- 172.21.15.46:0/2677173511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fd5f520 0x55da6fee2c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.988+0000 7fd37b419b80 1 -- 172.21.15.46:0/2677173511 >> 172.21.15.46:0/2677173511 conn(0x55da6fdbf720 msgr2=0x55da6fed76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.989+0000 7fd37b419b80 1 -- 172.21.15.46:0/2677173511 shutdown_connections 2024-05-09T04:57:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.989+0000 7fd37b419b80 1 -- 172.21.15.46:0/2677173511 wait complete. 2024-05-09T04:57:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.989+0000 7fd37b419b80 1 -- start start 2024-05-09T04:57:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.989+0000 7fd37b419b80 1 Processor -- start 2024-05-09T04:57:26.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.989+0000 7fd37b419b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fd5f520 0x55da6feec070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.989+0000 7fd37b419b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fe491e0 0x55da6feec5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.989+0000 7fd37b419b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 0x55da6fee6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.989+0000 7fd37b419b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55da6fc7ac20 con 0x55da6fe491e0 2024-05-09T04:57:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.989+0000 7fd37b419b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55da6fdb91d0 con 0x55da6fd5f520 2024-05-09T04:57:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.989+0000 7fd37b419b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55da6fedd5b0 con 0x55da6fedc880 2024-05-09T04:57:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.989+0000 7fd37b419b80 10 client.0 _ll_register_callbacks cb 0x55da6fc70740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd365d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fe491e0 0x55da6feec5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd365d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fe491e0 0x55da6feec5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58878/0 (socket says 172.21.15.46:58878) 2024-05-09T04:57:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd365d9b700 1 -- 172.21.15.46:0/36686540 learned_addr learned my addr 172.21.15.46:0/36686540 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd36659c700 1 --2- 172.21.15.46:0/36686540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fd5f520 0x55da6feec070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd366d9d700 1 --2- 172.21.15.46:0/36686540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 0x55da6fee6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd365d9b700 1 -- 172.21.15.46:0/36686540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 msgr2=0x55da6fee6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd365d9b700 1 --2- 172.21.15.46:0/36686540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 0x55da6fee6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd365d9b700 1 -- 172.21.15.46:0/36686540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fd5f520 msgr2=0x55da6feec070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:26.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd365d9b700 1 --2- 172.21.15.46:0/36686540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fd5f520 0x55da6feec070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd365d9b700 1 -- 172.21.15.46:0/36686540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da6fc7a510 con 0x55da6fe491e0 2024-05-09T04:57:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd365d9b700 1 --2- 172.21.15.46:0/36686540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fe491e0 0x55da6feec5b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0a9730 tx=0x7fd35c0a9f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98f5697dcbee60dd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd34dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd34dffb700 1 -- 172.21.15.46:0/36686540 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd35c020040 con 0x55da6fe491e0 2024-05-09T04:57:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd34dffb700 1 -- 172.21.15.46:0/36686540 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fd35c0a9c00 con 0x55da6fe491e0 2024-05-09T04:57:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd37b419b80 1 -- 172.21.15.46:0/36686540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3580055d0 con 0x55da6fe491e0 2024-05-09T04:57:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd34dffb700 1 -- 172.21.15.46:0/36686540 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd35c0b2600 con 0x55da6fe491e0 2024-05-09T04:57:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd37b419b80 10 client.16845 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.990+0000 7fd37b419b80 1 -- 172.21.15.46:0/36686540 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55da6fef3c00 con 0x55da6fe491e0 2024-05-09T04:57:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.991+0000 7fd34dffb700 1 -- 172.21.15.46:0/36686540 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3580055d0 con 0x55da6fe491e0 2024-05-09T04:57:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.991+0000 7fd34dffb700 1 -- 172.21.15.46:0/36686540 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd35c090c30 con 0x55da6fe491e0 2024-05-09T04:57:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.991+0000 7fd37b419b80 10 client.16845 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:26.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.991+0000 7fd37b419b80 10 client.16845 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:57:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.991+0000 7fd37b419b80 20 client.16845 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.991+0000 7fd37b419b80 1 --2- 172.21.15.46:0/36686540 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55da6fef60d0 0x55da6fef84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.991+0000 7fd37b419b80 4 client.16845 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:57:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.991+0000 7fd37b419b80 1 -- 172.21.15.46:0/36686540 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da6fe0ca50 con 0x55da6fef60d0 2024-05-09T04:57:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.991+0000 7fd36659c700 1 --2- 172.21.15.46:0/36686540 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55da6fef60d0 0x55da6fef84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.992+0000 7fd36659c700 1 --2- 172.21.15.46:0/36686540 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55da6fef60d0 0x55da6fef84f0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5208510977e0a3fd server_cookie=dbfafb58052e2192 in_seq=0 out_seq=0 2024-05-09T04:57:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.992+0000 7fd34dffb700 10 client.16845 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.992+0000 7fd34dffb700 1 -- 172.21.15.46:0/36686540 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55da6fe0ca50 con 0x55da6fef60d0 2024-05-09T04:57:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.992+0000 7fd34dffb700 10 client.16845 handle_command_reply: tid=0 2024-05-09T04:57:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.992+0000 7fd37b419b80 1 client.16845 shutdown 2024-05-09T04:57:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.993+0000 7fd37b419b80 1 -- 172.21.15.46:0/36686540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fe491e0 msgr2=0x55da6feec5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.993+0000 7fd37b419b80 1 --2- 172.21.15.46:0/36686540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fe491e0 0x55da6feec5b0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fd35c0a9730 tx=0x7fd35c0a9f30 comp rx=0 tx=0).stop 2024-05-09T04:57:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.993+0000 7fd37b419b80 1 -- 172.21.15.46:0/36686540 shutdown_connections 2024-05-09T04:57:26.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.993+0000 7fd37b419b80 1 --2- 172.21.15.46:0/36686540 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 0x55da6fee6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.993+0000 7fd37b419b80 1 --2- 172.21.15.46:0/36686540 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55da6fef60d0 0x55da6fef84f0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.993+0000 7fd37b419b80 1 --2- 172.21.15.46:0/36686540 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fd5f520 0x55da6feec070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.993+0000 7fd37b419b80 1 --2- 172.21.15.46:0/36686540 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fe491e0 0x55da6feec5b0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.993+0000 7fd37b419b80 1 -- 172.21.15.46:0/36686540 >> 172.21.15.46:0/36686540 conn(0x55da6fdbf720 msgr2=0x55da6fed7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.993+0000 7fd34dffb700 0 client.16845 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.993+0000 7fd37b419b80 1 -- 172.21.15.46:0/36686540 shutdown_connections 2024-05-09T04:57:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.993+0000 7fd37b419b80 1 -- 172.21.15.46:0/36686540 wait complete. 2024-05-09T04:57:26.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:26.993+0000 7fd37b419b80 20 client.16845 trim_cache size 0 max 16384 2024-05-09T04:57:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.014+0000 7fd37b419b80 1 Processor -- start 2024-05-09T04:57:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.014+0000 7fd37b419b80 1 -- start start 2024-05-09T04:57:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.014+0000 7fd37b419b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fd5f520 0x55da6fee0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.014+0000 7fd37b419b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fe491e0 0x55da6fee1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.014+0000 7fd37b419b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fedc880 0x55da6fee5470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.014+0000 7fd37b419b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681b37f0 con 0x55da6fedc880 2024-05-09T04:57:27.015 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.014+0000 7fd37b419b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681b1080 con 0x55da6fd5f520 2024-05-09T04:57:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.014+0000 7fd37b419b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681aeed0 con 0x55da6fe491e0 2024-05-09T04:57:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.014+0000 7fd366d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fedc880 0x55da6fee5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.014+0000 7fd365d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fe491e0 0x55da6fee1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.014+0000 7fd366d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fedc880 0x55da6fee5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58888/0 (socket says 172.21.15.46:58888) 2024-05-09T04:57:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.014+0000 7fd366d9d700 1 -- 172.21.15.46:0/737207612 learned_addr learned my addr 172.21.15.46:0/737207612 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.014+0000 7fd36659c700 1 --2- 172.21.15.46:0/737207612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fd5f520 0x55da6fee0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.015+0000 7fd366d9d700 1 -- 172.21.15.46:0/737207612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fe491e0 msgr2=0x55da6fee1bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.015+0000 7fd366d9d700 1 --2- 172.21.15.46:0/737207612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fe491e0 0x55da6fee1bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.015+0000 7fd366d9d700 1 -- 172.21.15.46:0/737207612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fd5f520 msgr2=0x55da6fee0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.015+0000 7fd366d9d700 1 --2- 172.21.15.46:0/737207612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fd5f520 0x55da6fee0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.015+0000 7fd366d9d700 1 -- 172.21.15.46:0/737207612 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da6fee2d00 con 0x55da6fedc880 2024-05-09T04:57:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.015+0000 7fd365d9b700 1 --2- 172.21.15.46:0/737207612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fe491e0 0x55da6fee1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.015+0000 7fd366d9d700 1 --2- 172.21.15.46:0/737207612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fedc880 0x55da6fee5470 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd360009e10 tx=0x7fd360009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2ebc54e7e218c9f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:27.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.015+0000 7fd34ffff700 1 -- 172.21.15.46:0/737207612 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36001a020 con 0x55da6fedc880 2024-05-09T04:57:27.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.015+0000 7fd34ffff700 1 -- 172.21.15.46:0/737207612 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd360016b10 con 0x55da6fedc880 2024-05-09T04:57:27.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.015+0000 7fd34ffff700 1 -- 172.21.15.46:0/737207612 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd360015b00 con 0x55da6fedc880 2024-05-09T04:57:27.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.016+0000 7fd37b419b80 1 -- 172.21.15.46:0/737207612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fedc880 msgr2=0x55da6fee5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:27.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.016+0000 7fd37b419b80 1 --2- 172.21.15.46:0/737207612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fedc880 0x55da6fee5470 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fd360009e10 tx=0x7fd360009040 comp rx=0 tx=0).stop 2024-05-09T04:57:27.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.016+0000 7fd37b419b80 1 -- 172.21.15.46:0/737207612 shutdown_connections 2024-05-09T04:57:27.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.016+0000 7fd37b419b80 1 --2- 172.21.15.46:0/737207612 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fedc880 0x55da6fee5470 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.016+0000 7fd37b419b80 1 --2- 172.21.15.46:0/737207612 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fd5f520 0x55da6fee0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.016+0000 7fd37b419b80 1 --2- 172.21.15.46:0/737207612 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fe491e0 0x55da6fee1bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.016+0000 7fd37b419b80 1 -- 172.21.15.46:0/737207612 >> 172.21.15.46:0/737207612 conn(0x55da6fdbf720 msgr2=0x55da6fee4d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:27.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.016+0000 7fd37b419b80 1 -- 172.21.15.46:0/737207612 shutdown_connections 2024-05-09T04:57:27.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.016+0000 7fd37b419b80 1 -- 172.21.15.46:0/737207612 wait complete. 2024-05-09T04:57:27.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.016+0000 7fd37b419b80 1 -- start start 2024-05-09T04:57:27.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd37b419b80 1 Processor -- start 2024-05-09T04:57:27.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd37b419b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fd5f520 0x55da6fef2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:27.020 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd37b419b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fe491e0 0x55da6fef08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd37b419b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 0x55da6fef0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd37b419b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681af540 con 0x55da6fd5f520 2024-05-09T04:57:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd37b419b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681b09b0 con 0x55da6fe491e0 2024-05-09T04:57:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd37b419b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd3681afc10 con 0x55da6fedc880 2024-05-09T04:57:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd37b419b80 10 client.0 _ll_register_callbacks cb 0x55da6fc70740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd36659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fd5f520 0x55da6fef2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd36659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fd5f520 0x55da6fef2550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58900/0 (socket says 172.21.15.46:58900) 2024-05-09T04:57:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd36659c700 1 -- 172.21.15.46:0/3257281643 learned_addr learned my addr 172.21.15.46:0/3257281643 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd365d9b700 1 --2- 172.21.15.46:0/3257281643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fe491e0 0x55da6fef08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd366d9d700 1 --2- 172.21.15.46:0/3257281643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 0x55da6fef0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd36659c700 1 -- 172.21.15.46:0/3257281643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 msgr2=0x55da6fef0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd36659c700 1 --2- 172.21.15.46:0/3257281643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 0x55da6fef0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd36659c700 1 -- 172.21.15.46:0/3257281643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fe491e0 msgr2=0x55da6fef08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd36659c700 1 --2- 172.21.15.46:0/3257281643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fe491e0 0x55da6fef08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.021 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.017+0000 7fd36659c700 1 -- 172.21.15.46:0/3257281643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da6fef1390 con 0x55da6fd5f520 2024-05-09T04:57:27.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.018+0000 7fd36659c700 1 --2- 172.21.15.46:0/3257281643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fd5f520 0x55da6fef2550 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd35800f4d0 tx=0x7fd35800f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b2430a35d5c248 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:27.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.018+0000 7fd34dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:27.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.018+0000 7fd34dffb700 1 -- 172.21.15.46:0/3257281643 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd358040030 con 0x55da6fd5f520 2024-05-09T04:57:27.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.018+0000 7fd34dffb700 1 -- 172.21.15.46:0/3257281643 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fd358034040 con 0x55da6fd5f520 2024-05-09T04:57:27.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.018+0000 7fd37b419b80 1 -- 172.21.15.46:0/3257281643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd360016b10 con 0x55da6fd5f520 2024-05-09T04:57:27.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.018+0000 7fd34dffb700 1 -- 172.21.15.46:0/3257281643 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd35800b6d0 con 0x55da6fd5f520 2024-05-09T04:57:27.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.018+0000 7fd37b419b80 10 client.16851 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:27.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.018+0000 7fd37b419b80 1 -- 172.21.15.46:0/3257281643 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55da6ff615b0 con 0x55da6fd5f520 2024-05-09T04:57:27.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.018+0000 7fd34dffb700 1 -- 172.21.15.46:0/3257281643 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd360016b10 con 0x55da6fd5f520 2024-05-09T04:57:27.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.018+0000 7fd34dffb700 1 -- 172.21.15.46:0/3257281643 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd358022070 con 0x55da6fd5f520 2024-05-09T04:57:27.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.019+0000 7fd37b419b80 10 client.16851 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:27.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.019+0000 7fd37b419b80 10 client.16851 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:57:27.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.019+0000 7fd37b419b80 20 client.16851 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:27.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.019+0000 7fd37b419b80 1 --2- 172.21.15.46:0/3257281643 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55da6ff61830 0x55da6ff63c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:27.022 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.019+0000 7fd37b419b80 4 client.16851 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:57:27.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.019+0000 7fd37b419b80 1 -- 172.21.15.46:0/3257281643 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55da6fee68b0 con 0x55da6ff61830 2024-05-09T04:57:27.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.019+0000 7fd365d9b700 1 --2- 172.21.15.46:0/3257281643 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55da6ff61830 0x55da6ff63c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:27.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.019+0000 7fd34dffb700 10 client.16851 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:27.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.019+0000 7fd365d9b700 1 --2- 172.21.15.46:0/3257281643 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55da6ff61830 0x55da6ff63c50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dda57bc84030d54d server_cookie=e54743cdbe520fc in_seq=0 out_seq=0 2024-05-09T04:57:27.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.020+0000 7fd34dffb700 1 -- 172.21.15.46:0/3257281643 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55da6fee68b0 con 0x55da6ff61830 2024-05-09T04:57:27.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.020+0000 7fd34dffb700 10 client.16851 handle_command_reply: tid=0 2024-05-09T04:57:27.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.020+0000 7fd37b419b80 1 client.16851 shutdown 2024-05-09T04:57:27.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.021+0000 7fd37b419b80 1 -- 172.21.15.46:0/3257281643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fd5f520 msgr2=0x55da6fef2550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:27.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.021+0000 7fd37b419b80 1 --2- 172.21.15.46:0/3257281643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fd5f520 0x55da6fef2550 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fd35800f4d0 tx=0x7fd35800f5a0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.021+0000 7fd37b419b80 1 -- 172.21.15.46:0/3257281643 shutdown_connections 2024-05-09T04:57:27.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.021+0000 7fd37b419b80 1 --2- 172.21.15.46:0/3257281643 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55da6fedc880 0x55da6fef0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.021+0000 7fd37b419b80 1 --2- 172.21.15.46:0/3257281643 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55da6fd5f520 0x55da6fef2550 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.021+0000 7fd37b419b80 1 --2- 172.21.15.46:0/3257281643 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55da6ff61830 0x55da6ff63c50 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.021+0000 7fd37b419b80 1 --2- 172.21.15.46:0/3257281643 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55da6fe491e0 0x55da6fef08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.021+0000 7fd37b419b80 1 -- 172.21.15.46:0/3257281643 >> 172.21.15.46:0/3257281643 conn(0x55da6fdbf720 msgr2=0x55da6fee1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:27.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.021+0000 7fd34dffb700 0 client.16851 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:27.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.021+0000 7fd37b419b80 1 -- 172.21.15.46:0/3257281643 shutdown_connections 2024-05-09T04:57:27.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.021+0000 7fd37b419b80 1 -- 172.21.15.46:0/3257281643 wait complete. 2024-05-09T04:57:27.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.021+0000 7fd37b419b80 20 client.16851 trim_cache size 0 max 16384 2024-05-09T04:57:27.025 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:57:27.025 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:57:27.026 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "2a5c5df7-53fa-4ddd-89a6-ac37b3e350d2", 2024-05-09T04:57:27.026 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:57:27.026 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:57:27.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.022+0000 7fd36759e700 1 -- 172.21.15.46:0/2787227696 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd3500b29f0 msgr2=0x7fd3500b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:27.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.022+0000 7fd36759e700 1 --2- 172.21.15.46:0/2787227696 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd3500b29f0 0x7fd3500b4f00 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fd358009990 tx=0x7fd3580036e0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.022+0000 7fd36759e700 1 -- 172.21.15.46:0/2787227696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd368107c20 msgr2=0x7fd36819d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:27.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.022+0000 7fd36759e700 1 --2- 172.21.15.46:0/2787227696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd368107c20 0x7fd36819d3c0 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fd35c00d980 tx=0x7fd35c00dc90 comp rx=0 tx=0).stop 2024-05-09T04:57:27.026 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.022+0000 7fd36759e700 1 -- 172.21.15.46:0/2787227696 shutdown_connections 2024-05-09T04:57:27.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.022+0000 7fd36759e700 1 --2- 172.21.15.46:0/2787227696 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd3500b29f0 0x7fd3500b4f00 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.022+0000 7fd36759e700 1 --2- 172.21.15.46:0/2787227696 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd368108570 0x7fd3681a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.022+0000 7fd36759e700 1 --2- 172.21.15.46:0/2787227696 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd3681032f0 0x7fd36819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.022+0000 7fd36759e700 1 --2- 172.21.15.46:0/2787227696 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd368107c20 0x7fd36819d3c0 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:27.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.022+0000 7fd36759e700 1 -- 172.21.15.46:0/2787227696 >> 172.21.15.46:0/2787227696 conn(0x7fd368078b50 msgr2=0x7fd36810fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:27.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.022+0000 7fd36759e700 1 -- 172.21.15.46:0/2787227696 shutdown_connections 2024-05-09T04:57:27.027 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:27.022+0000 7fd36759e700 1 -- 172.21.15.46:0/2787227696 wait complete. 2024-05-09T04:57:27.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:26 smithi110 ceph-mon[34904]: pgmap v193: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:26 smithi072 ceph-mon[34088]: pgmap v193: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:27.813 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:57:28.068 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:57:28.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:28 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:57:28.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:28 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:57:28.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:28 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:57:28.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:28 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:57:28.781 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:28 smithi046 ceph-mon[30136]: pgmap v194: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:57:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:28 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:57:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:28 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:57:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:28 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:57:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:28 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:57:29.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:28 smithi110 ceph-mon[34904]: pgmap v194: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:57:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:28 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:57:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:28 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:57:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:28 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:57:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:28 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:57:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:28 smithi072 ceph-mon[34088]: pgmap v194: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:57:29.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.709+0000 7f13d734c700 1 -- 172.21.15.46:0/956829866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d0108760 msgr2=0x7f13d01100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:29.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.709+0000 7f13d734c700 1 --2- 172.21.15.46:0/956829866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d0108760 0x7f13d01100c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f13c800b0d0 tx=0x7f13c800b3e0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.709+0000 7f13d734c700 1 -- 172.21.15.46:0/956829866 shutdown_connections 2024-05-09T04:57:29.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.709+0000 7f13d734c700 1 --2- 172.21.15.46:0/956829866 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d0108760 0x7f13d01100c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.709+0000 7f13d734c700 1 --2- 172.21.15.46:0/956829866 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13d01034e0 0x7f13d01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.709+0000 7f13d734c700 1 --2- 172.21.15.46:0/956829866 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13d0107e10 0x7f13d0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.709+0000 7f13d734c700 1 -- 172.21.15.46:0/956829866 >> 172.21.15.46:0/956829866 conn(0x7f13d00fef30 msgr2=0x7f13d0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:29.710 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.709+0000 7f13d734c700 1 -- 172.21.15.46:0/956829866 shutdown_connections 2024-05-09T04:57:29.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.709+0000 7f13d734c700 1 -- 172.21.15.46:0/956829866 wait complete. 2024-05-09T04:57:29.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.710+0000 7f13d734c700 1 Processor -- start 2024-05-09T04:57:29.711 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.710+0000 7f13d734c700 1 -- start start 2024-05-09T04:57:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.710+0000 7f13d734c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13d01034e0 0x7f13d0199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.710+0000 7f13d734c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d0107e10 0x7f13d019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.710+0000 7f13d734c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13d0108760 0x7f13d019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.710+0000 7f13d734c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13d0113390 con 0x7f13d0107e10 2024-05-09T04:57:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.710+0000 7f13d734c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f13d0113210 con 0x7f13d01034e0 2024-05-09T04:57:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.710+0000 7f13d734c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f13d0113510 con 0x7f13d0108760 2024-05-09T04:57:29.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.710+0000 7f13d48e7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d0107e10 0x7f13d019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.710+0000 7f13d48e7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d0107e10 0x7f13d019a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58914/0 (socket says 172.21.15.46:58914) 2024-05-09T04:57:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.710+0000 7f13d48e7700 1 -- 172.21.15.46:0/3020937966 learned_addr learned my addr 172.21.15.46:0/3020937966 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.710+0000 7f13d50e8700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13d01034e0 0x7f13d0199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.710+0000 7f13d58e9700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13d0108760 0x7f13d019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.711+0000 7f13d48e7700 1 -- 172.21.15.46:0/3020937966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13d0108760 msgr2=0x7f13d019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.711+0000 7f13d48e7700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13d0108760 0x7f13d019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.711+0000 7f13d48e7700 1 -- 172.21.15.46:0/3020937966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13d01034e0 msgr2=0x7f13d0199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.711+0000 7f13d48e7700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13d01034e0 0x7f13d0199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.711+0000 7f13d48e7700 1 -- 172.21.15.46:0/3020937966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13d01a9200 con 0x7f13d0107e10 2024-05-09T04:57:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.711+0000 7f13d50e8700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13d01034e0 0x7f13d0199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:57:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.711+0000 7f13d58e9700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13d0108760 0x7f13d019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:57:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.711+0000 7f13d48e7700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d0107e10 0x7f13d019a040 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f13c000d950 tx=0x7f13c000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.711+0000 7f13c67fc700 1 -- 172.21.15.46:0/3020937966 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0016070 con 0x7f13d0107e10 2024-05-09T04:57:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.711+0000 7f13d734c700 1 -- 172.21.15.46:0/3020937966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13d01a94f0 con 0x7f13d0107e10 2024-05-09T04:57:29.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.711+0000 7f13c67fc700 1 -- 172.21.15.46:0/3020937966 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c000de30 con 0x7f13d0107e10 2024-05-09T04:57:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.711+0000 7f13c67fc700 1 -- 172.21.15.46:0/3020937966 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c0005390 con 0x7f13d0107e10 2024-05-09T04:57:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.711+0000 7f13d734c700 1 -- 172.21.15.46:0/3020937966 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13d01a9a80 con 0x7f13d0107e10 2024-05-09T04:57:29.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.714+0000 7f13c67fc700 1 -- 172.21.15.46:0/3020937966 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f13c0005530 con 0x7f13d0107e10 2024-05-09T04:57:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.715+0000 7f13e5d62b80 1 Processor -- start 2024-05-09T04:57:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.715+0000 7f13e5d62b80 1 -- start start 2024-05-09T04:57:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.715+0000 7f13e5d62b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5008e3d00 0x55d5009ec240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.715+0000 7f13e5d62b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 0x55d5009ec780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.715+0000 7f13e5d62b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5009e6400 0x55d5009e6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.715+0000 7f13e5d62b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d5009090a0 con 0x55d5008e3d00 2024-05-09T04:57:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.715+0000 7f13e5d62b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d5008ae9a0 con 0x55d5009e6400 2024-05-09T04:57:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.715+0000 7f13e5d62b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d500905060 con 0x55d5008694a0 2024-05-09T04:57:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.715+0000 7f13c67fc700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f13bc0b29f0 0x7f13bc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.715+0000 7f13d58e9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5008e3d00 0x55d5009ec240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.716+0000 7f13d58e9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5008e3d00 0x55d5009ec240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58926/0 (socket says 172.21.15.46:58926) 2024-05-09T04:57:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.716+0000 7f13d50e8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 0x55d5009ec780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.716 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.716+0000 7f13d58e9700 1 -- 172.21.15.46:0/4277479629 learned_addr learned my addr 172.21.15.46:0/4277479629 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:29.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.716+0000 7f13c67fc700 1 -- 172.21.15.46:0/3020937966 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f13c00e1d70 con 0x7f13d0107e10 2024-05-09T04:57:29.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.716+0000 7f13d58e9700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f13bc0b29f0 0x7f13bc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.717 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.716+0000 7f13d48e7700 1 --2- 172.21.15.46:0/4277479629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5009e6400 0x55d5009e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.716+0000 7f13d50e8700 1 -- 172.21.15.46:0/4277479629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5009e6400 msgr2=0x55d5009e6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.716+0000 7f13d50e8700 1 --2- 172.21.15.46:0/4277479629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5009e6400 0x55d5009e6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.716+0000 7f13d50e8700 1 -- 172.21.15.46:0/4277479629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5008e3d00 msgr2=0x55d5009ec240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.716+0000 7f13d50e8700 1 --2- 172.21.15.46:0/4277479629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5008e3d00 0x55d5009ec240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.716+0000 7f13d50e8700 1 -- 172.21.15.46:0/4277479629 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d50089f500 con 0x55d5008694a0 2024-05-09T04:57:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.716+0000 7f13d58e9700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f13bc0b29f0 0x7f13bc0b4f00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f13c8005bd0 tx=0x7f13c8009780 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.716+0000 7f13d50e8700 1 --2- 172.21.15.46:0/4277479629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 0x55d5009ec780 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00bcd0 tx=0x7f13cc00f360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e334e25c3d223dc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.716+0000 7f13bb7fe700 1 -- 172.21.15.46:0/4277479629 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc00fbe0 con 0x55d5008694a0 2024-05-09T04:57:29.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.717+0000 7f13bb7fe700 1 -- 172.21.15.46:0/4277479629 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13cc00fd80 con 0x55d5008694a0 2024-05-09T04:57:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.717+0000 7f13bb7fe700 1 -- 172.21.15.46:0/4277479629 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13cc01b650 con 0x55d5008694a0 2024-05-09T04:57:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.717+0000 7f13e5d62b80 1 -- 172.21.15.46:0/4277479629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 msgr2=0x55d5009ec780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.717+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/4277479629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 0x55d5009ec780 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00bcd0 tx=0x7f13cc00f360 comp rx=0 tx=0).stop 2024-05-09T04:57:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.717+0000 7f13e5d62b80 1 -- 172.21.15.46:0/4277479629 shutdown_connections 2024-05-09T04:57:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.717+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/4277479629 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5009e6400 0x55d5009e6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.717+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/4277479629 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5008e3d00 0x55d5009ec240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.718+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/4277479629 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 0x55d5009ec780 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.718+0000 7f13e5d62b80 1 -- 172.21.15.46:0/4277479629 >> 172.21.15.46:0/4277479629 conn(0x55d50086cf70 msgr2=0x55d5009e1220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.718+0000 7f13e5d62b80 1 -- 172.21.15.46:0/4277479629 shutdown_connections 2024-05-09T04:57:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.718+0000 7f13e5d62b80 1 -- 172.21.15.46:0/4277479629 wait complete. 2024-05-09T04:57:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.718+0000 7f13e5d62b80 1 -- start start 2024-05-09T04:57:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.718+0000 7f13e5d62b80 1 Processor -- start 2024-05-09T04:57:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13e5d62b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 0x55d5009efff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13e5d62b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008e3d00 0x55d5009f0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13e5d62b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5009e6400 0x55d5009f0a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13e5d62b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d5009e7160 con 0x55d5009e6400 2024-05-09T04:57:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13e5d62b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d5008a3b20 con 0x55d5008e3d00 2024-05-09T04:57:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13e5d62b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d5009f3020 con 0x55d5008694a0 2024-05-09T04:57:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13e5d62b80 10 client.0 _ll_register_callbacks cb 0x55d50077a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13d58e9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5009e6400 0x55d5009f0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13d58e9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5009e6400 0x55d5009f0a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58936/0 (socket says 172.21.15.46:58936) 2024-05-09T04:57:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13d58e9700 1 -- 172.21.15.46:0/399372538 learned_addr learned my addr 172.21.15.46:0/399372538 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13d48e7700 1 --2- 172.21.15.46:0/399372538 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008e3d00 0x55d5009f0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13d50e8700 1 --2- 172.21.15.46:0/399372538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 0x55d5009efff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13d58e9700 1 -- 172.21.15.46:0/399372538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 msgr2=0x55d5009efff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13d58e9700 1 --2- 172.21.15.46:0/399372538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 0x55d5009efff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13d58e9700 1 -- 172.21.15.46:0/399372538 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008e3d00 msgr2=0x55d5009f0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13d58e9700 1 --2- 172.21.15.46:0/399372538 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008e3d00 0x55d5009f0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13d58e9700 1 -- 172.21.15.46:0/399372538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d500784490 con 0x55d5009e6400 2024-05-09T04:57:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13d48e7700 1 --2- 172.21.15.46:0/399372538 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008e3d00 0x55d5009f0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:57:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13d58e9700 1 --2- 172.21.15.46:0/399372538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5009e6400 0x55d5009f0a70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f13c800b0d0 tx=0x7f13c8005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d974f105b469c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.719+0000 7f13b8ff9700 1 -- 172.21.15.46:0/399372538 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8030030 con 0x55d5009e6400 2024-05-09T04:57:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.720+0000 7f13b8ff9700 1 -- 172.21.15.46:0/399372538 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f13c8026040 con 0x55d5009e6400 2024-05-09T04:57:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.720+0000 7f13e5d62b80 1 -- 172.21.15.46:0/399372538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13cc00fd80 con 0x55d5009e6400 2024-05-09T04:57:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.720+0000 7f13b8ff9700 1 -- 172.21.15.46:0/399372538 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f13c80091f0 con 0x55d5009e6400 2024-05-09T04:57:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.720+0000 7f13e5d62b80 10 client.16863 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.720+0000 7f13e5d62b80 1 -- 172.21.15.46:0/399372538 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d5009fda00 con 0x55d5009e6400 2024-05-09T04:57:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.720+0000 7f13b8ff9700 1 -- 172.21.15.46:0/399372538 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13cc00fd80 con 0x55d5009e6400 2024-05-09T04:57:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.720+0000 7f13b8ff9700 1 -- 172.21.15.46:0/399372538 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f13c8011070 con 0x55d5009e6400 2024-05-09T04:57:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.720+0000 7f13e5d62b80 10 client.16863 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.720+0000 7f13e5d62b80 10 client.16863 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:57:29.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.720+0000 7f13e5d62b80 20 client.16863 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:29.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.720+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/399372538 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500a000e0 0x55d500a02570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.720+0000 7f13e5d62b80 4 client.16863 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:57:29.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.720+0000 7f13e5d62b80 1 -- 172.21.15.46:0/399372538 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d500916a80 con 0x55d500a000e0 2024-05-09T04:57:29.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.721+0000 7f13d50e8700 1 --2- 172.21.15.46:0/399372538 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500a000e0 0x55d500a02570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.721+0000 7f13b8ff9700 10 client.16863 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:29.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.721+0000 7f13d50e8700 1 --2- 172.21.15.46:0/399372538 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500a000e0 0x55d500a02570 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b4209f7fc30f60f5 server_cookie=967a41630ecef583 in_seq=0 out_seq=0 2024-05-09T04:57:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.722+0000 7f13b8ff9700 1 -- 172.21.15.46:0/399372538 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d500916a80 con 0x55d500a000e0 2024-05-09T04:57:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.722+0000 7f13b8ff9700 10 client.16863 handle_command_reply: tid=0 2024-05-09T04:57:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.722+0000 7f13e5d62b80 1 client.16863 shutdown 2024-05-09T04:57:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.722+0000 7f13e5d62b80 1 -- 172.21.15.46:0/399372538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5009e6400 msgr2=0x55d5009f0a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.722+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/399372538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5009e6400 0x55d5009f0a70 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f13c800b0d0 tx=0x7f13c8005aa0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.722+0000 7f13e5d62b80 1 -- 172.21.15.46:0/399372538 shutdown_connections 2024-05-09T04:57:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.722+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/399372538 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5009e6400 0x55d5009f0a70 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.722+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/399372538 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 0x55d5009efff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.722+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/399372538 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500a000e0 0x55d500a02570 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.722+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/399372538 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008e3d00 0x55d5009f0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.722+0000 7f13b8ff9700 0 client.16863 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:29.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.722+0000 7f13e5d62b80 1 -- 172.21.15.46:0/399372538 >> 172.21.15.46:0/399372538 conn(0x55d50086cf70 msgr2=0x55d5009daff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:29.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.722+0000 7f13e5d62b80 1 -- 172.21.15.46:0/399372538 shutdown_connections 2024-05-09T04:57:29.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.722+0000 7f13e5d62b80 1 -- 172.21.15.46:0/399372538 wait complete. 2024-05-09T04:57:29.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.722+0000 7f13e5d62b80 20 client.16863 trim_cache size 0 max 16384 2024-05-09T04:57:29.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.742+0000 7f13e5d62b80 1 Processor -- start 2024-05-09T04:57:29.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13e5d62b80 1 -- start start 2024-05-09T04:57:29.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13e5d62b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008694a0 0x55d5009eefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13e5d62b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5008e3d00 0x55d5009f08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13e5d62b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5009e6400 0x55d5009db700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13e5d62b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13b400ff40 con 0x55d5008e3d00 2024-05-09T04:57:29.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13e5d62b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f13b400d7d0 con 0x55d5008694a0 2024-05-09T04:57:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13e5d62b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f13b400b620 con 0x55d5009e6400 2024-05-09T04:57:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13d48e7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5008e3d00 0x55d5009f08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13d48e7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5008e3d00 0x55d5009f08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58940/0 (socket says 172.21.15.46:58940) 2024-05-09T04:57:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13d48e7700 1 -- 172.21.15.46:0/3163912450 learned_addr learned my addr 172.21.15.46:0/3163912450 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13d58e9700 1 --2- 172.21.15.46:0/3163912450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5009e6400 0x55d5009db700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13d50e8700 1 --2- 172.21.15.46:0/3163912450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008694a0 0x55d5009eefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13d48e7700 1 -- 172.21.15.46:0/3163912450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5009e6400 msgr2=0x55d5009db700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13d48e7700 1 --2- 172.21.15.46:0/3163912450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5009e6400 0x55d5009db700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13d48e7700 1 -- 172.21.15.46:0/3163912450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008694a0 msgr2=0x55d5009eefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13d48e7700 1 --2- 172.21.15.46:0/3163912450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008694a0 0x55d5009eefb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13d48e7700 1 -- 172.21.15.46:0/3163912450 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5009e2aa0 con 0x55d5008e3d00 2024-05-09T04:57:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.743+0000 7f13d50e8700 1 --2- 172.21.15.46:0/3163912450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008694a0 0x55d5009eefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:57:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.744+0000 7f13d48e7700 1 --2- 172.21.15.46:0/3163912450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5008e3d00 0x55d5009f08b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f13c000bec0 tx=0x7f13c00a96d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=110db7e97313f924 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:29.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.744+0000 7f13baffd700 1 -- 172.21.15.46:0/3163912450 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c00df8d0 con 0x55d5008e3d00 2024-05-09T04:57:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.744+0000 7f13baffd700 1 -- 172.21.15.46:0/3163912450 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c00a8ae0 con 0x55d5008e3d00 2024-05-09T04:57:29.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.744+0000 7f13baffd700 1 -- 172.21.15.46:0/3163912450 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c00a9490 con 0x55d5008e3d00 2024-05-09T04:57:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.744+0000 7f13e5d62b80 1 -- 172.21.15.46:0/3163912450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5008e3d00 msgr2=0x55d5009f08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.744+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/3163912450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5008e3d00 0x55d5009f08b0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f13c000bec0 tx=0x7f13c00a96d0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.745+0000 7f13e5d62b80 1 -- 172.21.15.46:0/3163912450 shutdown_connections 2024-05-09T04:57:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.745+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/3163912450 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5009e6400 0x55d5009db700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.745+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/3163912450 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008694a0 0x55d5009eefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.745+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/3163912450 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5008e3d00 0x55d5009f08b0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.745+0000 7f13e5d62b80 1 -- 172.21.15.46:0/3163912450 >> 172.21.15.46:0/3163912450 conn(0x55d50086cf70 msgr2=0x55d5009daff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.745+0000 7f13e5d62b80 1 -- 172.21.15.46:0/3163912450 shutdown_connections 2024-05-09T04:57:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.745+0000 7f13e5d62b80 1 -- 172.21.15.46:0/3163912450 wait complete. 2024-05-09T04:57:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.745+0000 7f13e5d62b80 1 -- start start 2024-05-09T04:57:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.745+0000 7f13e5d62b80 1 Processor -- start 2024-05-09T04:57:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13e5d62b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 0x55d5009f8360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13e5d62b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008e3d00 0x55d5009f8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13e5d62b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5009e6400 0x55d5009f3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13e5d62b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f13b400bc90 con 0x55d5009e6400 2024-05-09T04:57:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13e5d62b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f13b400d100 con 0x55d5008e3d00 2024-05-09T04:57:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13e5d62b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f13b400c360 con 0x55d5008694a0 2024-05-09T04:57:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13e5d62b80 10 client.0 _ll_register_callbacks cb 0x55d50077a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:57:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13d58e9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5009e6400 0x55d5009f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13d58e9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5009e6400 0x55d5009f3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:58956/0 (socket says 172.21.15.46:58956) 2024-05-09T04:57:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13d58e9700 1 -- 172.21.15.46:0/1862874238 learned_addr learned my addr 172.21.15.46:0/1862874238 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:57:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13d50e8700 1 --2- 172.21.15.46:0/1862874238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 0x55d5009f8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13d48e7700 1 --2- 172.21.15.46:0/1862874238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008e3d00 0x55d5009f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13d58e9700 1 -- 172.21.15.46:0/1862874238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 msgr2=0x55d5009f8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13d58e9700 1 --2- 172.21.15.46:0/1862874238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 0x55d5009f8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13d58e9700 1 -- 172.21.15.46:0/1862874238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008e3d00 msgr2=0x55d5009f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:57:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13d58e9700 1 --2- 172.21.15.46:0/1862874238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008e3d00 0x55d5009f8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13d58e9700 1 -- 172.21.15.46:0/1862874238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5009f40a0 con 0x55d5009e6400 2024-05-09T04:57:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13d58e9700 1 --2- 172.21.15.46:0/1862874238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5009e6400 0x55d5009f3900 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f13c800d010 tx=0x7f13c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f61887eab0a85980 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:57:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:57:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13b8ff9700 1 -- 172.21.15.46:0/1862874238 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13c8030050 con 0x55d5009e6400 2024-05-09T04:57:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13b8ff9700 1 -- 172.21.15.46:0/1862874238 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7f13c800f020 con 0x55d5009e6400 2024-05-09T04:57:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.746+0000 7f13e5d62b80 1 -- 172.21.15.46:0/1862874238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13c00a8ae0 con 0x55d5009e6400 2024-05-09T04:57:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.747+0000 7f13b8ff9700 1 -- 172.21.15.46:0/1862874238 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f13c804d050 con 0x55d5009e6400 2024-05-09T04:57:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.747+0000 7f13e5d62b80 10 client.16869 fetch_fsmap learned FSMap version 25 2024-05-09T04:57:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.747+0000 7f13e5d62b80 1 -- 172.21.15.46:0/1862874238 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d500a6a3d0 con 0x55d5009e6400 2024-05-09T04:57:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.747+0000 7f13b8ff9700 1 -- 172.21.15.46:0/1862874238 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13c00a8ae0 con 0x55d5009e6400 2024-05-09T04:57:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.747+0000 7f13b8ff9700 1 -- 172.21.15.46:0/1862874238 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f13c8011040 con 0x55d5009e6400 2024-05-09T04:57:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.747+0000 7f13e5d62b80 10 client.16869 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:57:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.747+0000 7f13e5d62b80 10 client.16869 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:57:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.747+0000 7f13e5d62b80 20 client.16869 populate_metadata read hostname 'smithi046' 2024-05-09T04:57:29.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.747+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/1862874238 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500a6aa40 0x55d500a6ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:57:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.747+0000 7f13e5d62b80 4 client.16869 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:57:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.747+0000 7f13e5d62b80 1 -- 172.21.15.46:0/1862874238 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d500910b20 con 0x55d500a6aa40 2024-05-09T04:57:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.748+0000 7f13d50e8700 1 --2- 172.21.15.46:0/1862874238 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500a6aa40 0x55d500a6ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:57:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.748+0000 7f13d50e8700 1 --2- 172.21.15.46:0/1862874238 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500a6aa40 0x55d500a6ce60 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15dbcf6cccaf1495 server_cookie=138b6cec7f0ad790 in_seq=0 out_seq=0 2024-05-09T04:57:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.748+0000 7f13b8ff9700 10 client.16869 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.748+0000 7f13b8ff9700 1 -- 172.21.15.46:0/1862874238 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d500910b20 con 0x55d500a6aa40 2024-05-09T04:57:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.748+0000 7f13b8ff9700 10 client.16869 handle_command_reply: tid=0 2024-05-09T04:57:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.748+0000 7f13e5d62b80 1 client.16869 shutdown 2024-05-09T04:57:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.748+0000 7f13e5d62b80 1 -- 172.21.15.46:0/1862874238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5009e6400 msgr2=0x55d5009f3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.748+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/1862874238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5009e6400 0x55d5009f3900 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f13c800d010 tx=0x7f13c8005830 comp rx=0 tx=0).stop 2024-05-09T04:57:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.748+0000 7f13e5d62b80 1 -- 172.21.15.46:0/1862874238 shutdown_connections 2024-05-09T04:57:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.748+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/1862874238 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d5009e6400 0x55d5009f3900 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.748+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/1862874238 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d5008694a0 0x55d5009f8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.748+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/1862874238 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d500a6aa40 0x55d500a6ce60 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.748+0000 7f13e5d62b80 1 --2- 172.21.15.46:0/1862874238 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d5008e3d00 0x55d5009f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.748+0000 7f13b8ff9700 0 client.16869 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:57:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.748+0000 7f13e5d62b80 1 -- 172.21.15.46:0/1862874238 >> 172.21.15.46:0/1862874238 conn(0x55d50086cf70 msgr2=0x55d5009ea460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.749+0000 7f13e5d62b80 1 -- 172.21.15.46:0/1862874238 shutdown_connections 2024-05-09T04:57:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.749+0000 7f13e5d62b80 1 -- 172.21.15.46:0/1862874238 wait complete. 2024-05-09T04:57:29.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.749+0000 7f13e5d62b80 20 client.16869 trim_cache size 0 max 16384 2024-05-09T04:57:29.754 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:57:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T04:57:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-09T04:57:29.754 INFO:teuthology.orchestra.run.smithi046.stdout: "2a5c5df7-53fa-4ddd-89a6-ac37b3e350d2": { 2024-05-09T04:57:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-09T04:57:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "2a5c5df7-53fa-4ddd-89a6-ac37b3e350d2", 2024-05-09T04:57:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-09T04:57:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T04:57:29.755 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T04:57:29.755 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:57:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.749+0000 7f13d734c700 1 -- 172.21.15.46:0/3020937966 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f13bc0b29f0 msgr2=0x7f13bc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:29.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.749+0000 7f13d734c700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f13bc0b29f0 0x7f13bc0b4f00 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f13c8005bd0 tx=0x7f13c8009780 comp rx=0 tx=0).stop 2024-05-09T04:57:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.749+0000 7f13d734c700 1 -- 172.21.15.46:0/3020937966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d0107e10 msgr2=0x7f13d019a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:57:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.749+0000 7f13d734c700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d0107e10 0x7f13d019a040 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f13c000d950 tx=0x7f13c000dd10 comp rx=0 tx=0).stop 2024-05-09T04:57:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.749+0000 7f13d734c700 1 -- 172.21.15.46:0/3020937966 shutdown_connections 2024-05-09T04:57:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.749+0000 7f13d734c700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f13bc0b29f0 0x7f13bc0b4f00 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.749+0000 7f13d734c700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f13d0108760 0x7f13d019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.749+0000 7f13d734c700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13d01034e0 0x7f13d0199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.749+0000 7f13d734c700 1 --2- 172.21.15.46:0/3020937966 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f13d0107e10 0x7f13d019a040 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:57:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.749+0000 7f13d734c700 1 -- 172.21.15.46:0/3020937966 >> 172.21.15.46:0/3020937966 conn(0x7f13d00fef30 msgr2=0x7f13d0107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:57:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.749+0000 7f13d734c700 1 -- 172.21.15.46:0/3020937966 shutdown_connections 2024-05-09T04:57:29.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:57:29.749+0000 7f13d734c700 1 -- 172.21.15.46:0/3020937966 wait complete. 2024-05-09T04:57:30.644 INFO:tasks.cephfs.filesystem:scrub status for tag:2a5c5df7-53fa-4ddd-89a6-ac37b3e350d2 - {'path': '/', 'tag': '2a5c5df7-53fa-4ddd-89a6-ac37b3e350d2', 'options': 'recursive,force'} 2024-05-09T04:57:31.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:30 smithi110 ceph-mon[34904]: pgmap v195: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:31.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:30 smithi046 ceph-mon[30136]: pgmap v195: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:31.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:30 smithi072 ceph-mon[34088]: pgmap v195: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:32.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:31 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:57:32.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:31 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:57:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:31 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:57:33.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:32 smithi110 ceph-mon[34904]: pgmap v196: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-05-09T04:57:33.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:32 smithi046 ceph-mon[30136]: pgmap v196: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-05-09T04:57:33.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:32 smithi072 ceph-mon[34088]: pgmap v196: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-05-09T04:57:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:34 smithi110 ceph-mon[34904]: pgmap v197: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T04:57:35.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:34 smithi046 ceph-mon[30136]: pgmap v197: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T04:57:35.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:34 smithi072 ceph-mon[34088]: pgmap v197: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T04:57:37.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:36 smithi110 ceph-mon[34904]: pgmap v198: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:57:37.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:36 smithi046 ceph-mon[30136]: pgmap v198: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:57:37.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:36 smithi072 ceph-mon[34088]: pgmap v198: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:57:39.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:38 smithi110 ceph-mon[34904]: pgmap v199: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T04:57:39.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:38 smithi046 ceph-mon[30136]: pgmap v199: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T04:57:39.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:38 smithi072 ceph-mon[34088]: pgmap v199: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T04:57:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:40 smithi110 ceph-mon[34904]: pgmap v200: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:57:41.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:40 smithi046 ceph-mon[30136]: pgmap v200: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:57:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:40 smithi072 ceph-mon[34088]: pgmap v200: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:57:43.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:42 smithi110 ceph-mon[34904]: pgmap v201: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T04:57:43.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:42 smithi046 ceph-mon[30136]: pgmap v201: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T04:57:43.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:42 smithi072 ceph-mon[34088]: pgmap v201: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T04:57:45.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:44 smithi046 ceph-mon[30136]: pgmap v202: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:57:45.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:44 smithi072 ceph-mon[34088]: pgmap v202: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:57:45.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:44 smithi110 ceph-mon[34904]: pgmap v202: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:57:47.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:46 smithi046 ceph-mon[30136]: pgmap v203: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:46 smithi072 ceph-mon[34088]: pgmap v203: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:47.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:46 smithi110 ceph-mon[34904]: pgmap v203: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:49.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:48 smithi046 ceph-mon[30136]: pgmap v204: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:57:49.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:48 smithi072 ceph-mon[34088]: pgmap v204: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:57:49.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:48 smithi110 ceph-mon[34904]: pgmap v204: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:57:51.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:50 smithi046 ceph-mon[30136]: pgmap v205: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:51.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:50 smithi072 ceph-mon[34088]: pgmap v205: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:51.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:50 smithi110 ceph-mon[34904]: pgmap v205: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:53.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:52 smithi046 ceph-mon[30136]: pgmap v206: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:57:53.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:52 smithi072 ceph-mon[34088]: pgmap v206: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:57:53.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:52 smithi110 ceph-mon[34904]: pgmap v206: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:57:55.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:54 smithi046 ceph-mon[30136]: pgmap v207: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:57:55.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:54 smithi072 ceph-mon[34088]: pgmap v207: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:57:55.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:54 smithi110 ceph-mon[34904]: pgmap v207: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:57:57.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:56 smithi046 ceph-mon[30136]: pgmap v208: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:56 smithi072 ceph-mon[34088]: pgmap v208: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:57.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:56 smithi110 ceph-mon[34904]: pgmap v208: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:57:59.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:58 smithi072 ceph-mon[34088]: pgmap v209: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:57:59.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:58 smithi110 ceph-mon[34904]: pgmap v209: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:57:59.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:58 smithi046 ceph-mon[30136]: pgmap v209: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:58:00.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:57:59 smithi110 ceph-mon[34904]: pgmap v210: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:58:00.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:57:59 smithi046 ceph-mon[30136]: pgmap v210: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:58:00.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:57:59 smithi072 ceph-mon[34088]: pgmap v210: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:58:00.645 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:58:00.927 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.733+0000 7fdd87917700 1 -- 172.21.15.46:0/1354713692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd801045e0 msgr2=0x7fdd801049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.733+0000 7fdd87917700 1 --2- 172.21.15.46:0/1354713692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd801045e0 0x7fdd801049f0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00b0d0 tx=0x7fdd7c00b3e0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.733+0000 7fdd87917700 1 -- 172.21.15.46:0/1354713692 shutdown_connections 2024-05-09T04:58:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.733+0000 7fdd87917700 1 --2- 172.21.15.46:0/1354713692 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd80109860 0x7fdd801111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.735 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.733+0000 7fdd87917700 1 --2- 172.21.15.46:0/1354713692 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd801045e0 0x7fdd801049f0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.733+0000 7fdd87917700 1 --2- 172.21.15.46:0/1354713692 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd80108f10 0x7fdd80109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.733+0000 7fdd87917700 1 -- 172.21.15.46:0/1354713692 >> 172.21.15.46:0/1354713692 conn(0x7fdd800ffed0 msgr2=0x7fdd80102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.733+0000 7fdd87917700 1 -- 172.21.15.46:0/1354713692 shutdown_connections 2024-05-09T04:58:02.736 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.733+0000 7fdd87917700 1 -- 172.21.15.46:0/1354713692 wait complete. 2024-05-09T04:58:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.734+0000 7fdd87917700 1 Processor -- start 2024-05-09T04:58:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.734+0000 7fdd87917700 1 -- start start 2024-05-09T04:58:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd87917700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd801045e0 0x7fdd8019ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd87917700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80108f10 0x7fdd8019b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd87917700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd80109860 0x7fdd8019f630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd87917700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdd80114490 con 0x7fdd80108f10 2024-05-09T04:58:02.738 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd87917700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdd80114310 con 0x7fdd80109860 2024-05-09T04:58:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd87917700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdd80114610 con 0x7fdd801045e0 2024-05-09T04:58:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd84eb2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80108f10 0x7fdd8019b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd84eb2700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80108f10 0x7fdd8019b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60514/0 (socket says 172.21.15.46:60514) 2024-05-09T04:58:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd84eb2700 1 -- 172.21.15.46:0/3084743776 learned_addr learned my addr 172.21.15.46:0/3084743776 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd85eb4700 1 --2- 172.21.15.46:0/3084743776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd80109860 0x7fdd8019f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd856b3700 1 --2- 172.21.15.46:0/3084743776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd801045e0 0x7fdd8019ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd84eb2700 1 -- 172.21.15.46:0/3084743776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd801045e0 msgr2=0x7fdd8019ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd84eb2700 1 --2- 172.21.15.46:0/3084743776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd801045e0 0x7fdd8019ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd84eb2700 1 -- 172.21.15.46:0/3084743776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd80109860 msgr2=0x7fdd8019f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd84eb2700 1 --2- 172.21.15.46:0/3084743776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd80109860 0x7fdd8019f630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd84eb2700 1 -- 172.21.15.46:0/3084743776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd8019fc20 con 0x7fdd80108f10 2024-05-09T04:58:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd856b3700 1 --2- 172.21.15.46:0/3084743776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd801045e0 0x7fdd8019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:58:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.735+0000 7fdd84eb2700 1 --2- 172.21.15.46:0/3084743776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80108f10 0x7fdd8019b3a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c9e0 tx=0x7fdd7000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.736+0000 7fdd767fc700 1 -- 172.21.15.46:0/3084743776 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70007aa0 con 0x7fdd80108f10 2024-05-09T04:58:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.736+0000 7fdd87917700 1 -- 172.21.15.46:0/3084743776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd8019ff10 con 0x7fdd80108f10 2024-05-09T04:58:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.736+0000 7fdd767fc700 1 -- 172.21.15.46:0/3084743776 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd70007c40 con 0x7fdd80108f10 2024-05-09T04:58:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.736+0000 7fdd767fc700 1 -- 172.21.15.46:0/3084743776 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd70018660 con 0x7fdd80108f10 2024-05-09T04:58:02.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.736+0000 7fdd87917700 1 -- 172.21.15.46:0/3084743776 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd801a78e0 con 0x7fdd80108f10 2024-05-09T04:58:02.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.738+0000 7fdd767fc700 1 -- 172.21.15.46:0/3084743776 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdd70018800 con 0x7fdd80108f10 2024-05-09T04:58:02.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.739+0000 7fdd767fc700 1 --2- 172.21.15.46:0/3084743776 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdd6c0b2a70 0x7fdd6c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.739+0000 7fdd767fc700 1 -- 172.21.15.46:0/3084743776 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fdd700e2620 con 0x7fdd80108f10 2024-05-09T04:58:02.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.739+0000 7fdd856b3700 1 --2- 172.21.15.46:0/3084743776 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdd6c0b2a70 0x7fdd6c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.740+0000 7fdd856b3700 1 --2- 172.21.15.46:0/3084743776 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdd6c0b2a70 0x7fdd6c0b4f80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c000f80 tx=0x7fdd7c00b040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:02.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.740+0000 7fdd9632db80 1 Processor -- start 2024-05-09T04:58:02.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.740+0000 7fdd9632db80 1 -- start start 2024-05-09T04:58:02.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.740+0000 7fdd9632db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c086d00 0x55578c18f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.740+0000 7fdd9632db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c00c4a0 0x55578c18f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.740+0000 7fdd9632db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c189400 0x55578c189810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.740+0000 7fdd9632db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55578c0ac0a0 con 0x55578c086d00 2024-05-09T04:58:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.740+0000 7fdd9632db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55578c0519a0 con 0x55578c189400 2024-05-09T04:58:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.740+0000 7fdd9632db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55578c0a8060 con 0x55578c00c4a0 2024-05-09T04:58:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.740+0000 7fdd856b3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c086d00 0x55578c18f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.740+0000 7fdd856b3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c086d00 0x55578c18f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60530/0 (socket says 172.21.15.46:60530) 2024-05-09T04:58:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.740+0000 7fdd856b3700 1 -- 172.21.15.46:0/2774736779 learned_addr learned my addr 172.21.15.46:0/2774736779 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.741+0000 7fdd84eb2700 1 --2- 172.21.15.46:0/2774736779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c00c4a0 0x55578c18f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.741+0000 7fdd85eb4700 1 --2- 172.21.15.46:0/2774736779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c189400 0x55578c189810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.741+0000 7fdd856b3700 1 -- 172.21.15.46:0/2774736779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c00c4a0 msgr2=0x55578c18f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.741+0000 7fdd856b3700 1 --2- 172.21.15.46:0/2774736779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c00c4a0 0x55578c18f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.741+0000 7fdd856b3700 1 -- 172.21.15.46:0/2774736779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c189400 msgr2=0x55578c189810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:02.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.741+0000 7fdd856b3700 1 --2- 172.21.15.46:0/2774736779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c189400 0x55578c189810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.741+0000 7fdd856b3700 1 -- 172.21.15.46:0/2774736779 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55578c042500 con 0x55578c086d00 2024-05-09T04:58:02.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.741+0000 7fdd85eb4700 1 --2- 172.21.15.46:0/2774736779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c189400 0x55578c189810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:58:02.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.741+0000 7fdd856b3700 1 --2- 172.21.15.46:0/2774736779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c086d00 0x55578c18f240 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c015000 tx=0x7fdd7c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=657f2abee9a386eb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:02.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.741+0000 7fdd6b7fe700 1 -- 172.21.15.46:0/2774736779 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c02c070 con 0x55578c086d00 2024-05-09T04:58:02.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.741+0000 7fdd6b7fe700 1 -- 172.21.15.46:0/2774736779 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd7c022070 con 0x55578c086d00 2024-05-09T04:58:02.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.741+0000 7fdd6b7fe700 1 -- 172.21.15.46:0/2774736779 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c01ccc0 con 0x55578c086d00 2024-05-09T04:58:02.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.742+0000 7fdd9632db80 1 -- 172.21.15.46:0/2774736779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c086d00 msgr2=0x55578c18f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:02.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.742+0000 7fdd9632db80 1 --2- 172.21.15.46:0/2774736779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c086d00 0x55578c18f240 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c015000 tx=0x7fdd7c01c000 comp rx=0 tx=0).stop 2024-05-09T04:58:02.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.742+0000 7fdd9632db80 1 -- 172.21.15.46:0/2774736779 shutdown_connections 2024-05-09T04:58:02.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.742+0000 7fdd9632db80 1 --2- 172.21.15.46:0/2774736779 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c189400 0x55578c189810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.742+0000 7fdd9632db80 1 --2- 172.21.15.46:0/2774736779 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c086d00 0x55578c18f240 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.742+0000 7fdd9632db80 1 --2- 172.21.15.46:0/2774736779 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c00c4a0 0x55578c18f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.742+0000 7fdd9632db80 1 -- 172.21.15.46:0/2774736779 >> 172.21.15.46:0/2774736779 conn(0x55578c00ff70 msgr2=0x55578c184220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.742+0000 7fdd9632db80 1 -- 172.21.15.46:0/2774736779 shutdown_connections 2024-05-09T04:58:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.742+0000 7fdd9632db80 1 -- 172.21.15.46:0/2774736779 wait complete. 2024-05-09T04:58:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.743+0000 7fdd9632db80 1 -- start start 2024-05-09T04:58:02.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.743+0000 7fdd9632db80 1 Processor -- start 2024-05-09T04:58:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.743+0000 7fdd9632db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c00c4a0 0x55578c193080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.743+0000 7fdd9632db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c086d00 0x55578c1935c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.743+0000 7fdd9632db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c189400 0x55578c193b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.743+0000 7fdd9632db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55578c18a100 con 0x55578c00c4a0 2024-05-09T04:58:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.743+0000 7fdd9632db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55578c046b20 con 0x55578c086d00 2024-05-09T04:58:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.743+0000 7fdd9632db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55578c19f1d0 con 0x55578c189400 2024-05-09T04:58:02.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.743+0000 7fdd9632db80 10 client.0 _ll_register_callbacks cb 0x55578bf1d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.743+0000 7fdd856b3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c00c4a0 0x55578c193080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.743+0000 7fdd856b3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c00c4a0 0x55578c193080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60536/0 (socket says 172.21.15.46:60536) 2024-05-09T04:58:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.743+0000 7fdd856b3700 1 -- 172.21.15.46:0/3513042185 learned_addr learned my addr 172.21.15.46:0/3513042185 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd85eb4700 1 --2- 172.21.15.46:0/3513042185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c189400 0x55578c193b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd84eb2700 1 --2- 172.21.15.46:0/3513042185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c086d00 0x55578c1935c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd856b3700 1 -- 172.21.15.46:0/3513042185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c189400 msgr2=0x55578c193b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd856b3700 1 --2- 172.21.15.46:0/3513042185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c189400 0x55578c193b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd856b3700 1 -- 172.21.15.46:0/3513042185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c086d00 msgr2=0x55578c1935c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd856b3700 1 --2- 172.21.15.46:0/3513042185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c086d00 0x55578c1935c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd856b3700 1 -- 172.21.15.46:0/3513042185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55578bf27490 con 0x55578c00c4a0 2024-05-09T04:58:02.752 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd85eb4700 1 --2- 172.21.15.46:0/3513042185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c189400 0x55578c193b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:58:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd84eb2700 1 --2- 172.21.15.46:0/3513042185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c086d00 0x55578c1935c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:58:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd856b3700 1 --2- 172.21.15.46:0/3513042185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c00c4a0 0x55578c193080 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00d010 tx=0x7fdd7c009300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d6d28211d188716 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd68ff9700 1 -- 172.21.15.46:0/3513042185 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c038030 con 0x55578c00c4a0 2024-05-09T04:58:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd68ff9700 1 -- 172.21.15.46:0/3513042185 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fdd7c01c900 con 0x55578c00c4a0 2024-05-09T04:58:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd9632db80 1 -- 172.21.15.46:0/3513042185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd7c022070 con 0x55578c00c4a0 2024-05-09T04:58:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd68ff9700 1 -- 172.21.15.46:0/3513042185 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdd7c02c040 con 0x55578c00c4a0 2024-05-09T04:58:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd9632db80 10 client.16881 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.744+0000 7fdd9632db80 1 -- 172.21.15.46:0/3513042185 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55578c1a0a90 con 0x55578c00c4a0 2024-05-09T04:58:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.745+0000 7fdd68ff9700 1 -- 172.21.15.46:0/3513042185 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd7c022070 con 0x55578c00c4a0 2024-05-09T04:58:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.745+0000 7fdd68ff9700 1 -- 172.21.15.46:0/3513042185 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fdd7c027070 con 0x55578c00c4a0 2024-05-09T04:58:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.745+0000 7fdd9632db80 10 client.16881 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:02.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.745+0000 7fdd9632db80 10 client.16881 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:58:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.745+0000 7fdd9632db80 20 client.16881 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.745+0000 7fdd9632db80 1 --2- 172.21.15.46:0/3513042185 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55578c1a2f60 0x55578c1a5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.745+0000 7fdd9632db80 4 client.16881 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:58:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.745+0000 7fdd9632db80 1 -- 172.21.15.46:0/3513042185 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55578c0b9a80 con 0x55578c1a2f60 2024-05-09T04:58:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.745+0000 7fdd84eb2700 1 --2- 172.21.15.46:0/3513042185 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55578c1a2f60 0x55578c1a5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.746+0000 7fdd84eb2700 1 --2- 172.21.15.46:0/3513042185 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55578c1a2f60 0x55578c1a5380 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=764d5258437ed8ad server_cookie=8a8e2e4ccecce147 in_seq=0 out_seq=0 2024-05-09T04:58:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.746+0000 7fdd68ff9700 10 client.16881 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.746+0000 7fdd68ff9700 1 -- 172.21.15.46:0/3513042185 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55578c0b9a80 con 0x55578c1a2f60 2024-05-09T04:58:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.746+0000 7fdd68ff9700 10 client.16881 handle_command_reply: tid=0 2024-05-09T04:58:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.746+0000 7fdd9632db80 1 client.16881 shutdown 2024-05-09T04:58:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.747+0000 7fdd9632db80 1 -- 172.21.15.46:0/3513042185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c00c4a0 msgr2=0x55578c193080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.747+0000 7fdd9632db80 1 --2- 172.21.15.46:0/3513042185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c00c4a0 0x55578c193080 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c00d010 tx=0x7fdd7c009300 comp rx=0 tx=0).stop 2024-05-09T04:58:02.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.747+0000 7fdd9632db80 1 -- 172.21.15.46:0/3513042185 shutdown_connections 2024-05-09T04:58:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.747+0000 7fdd9632db80 1 --2- 172.21.15.46:0/3513042185 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c189400 0x55578c193b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.747+0000 7fdd9632db80 1 --2- 172.21.15.46:0/3513042185 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c00c4a0 0x55578c193080 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.747+0000 7fdd9632db80 1 --2- 172.21.15.46:0/3513042185 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55578c1a2f60 0x55578c1a5380 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.747+0000 7fdd9632db80 1 --2- 172.21.15.46:0/3513042185 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c086d00 0x55578c1935c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.747+0000 7fdd9632db80 1 -- 172.21.15.46:0/3513042185 >> 172.21.15.46:0/3513042185 conn(0x55578c00ff70 msgr2=0x55578c17dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.747+0000 7fdd68ff9700 0 client.16881 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.747+0000 7fdd9632db80 1 -- 172.21.15.46:0/3513042185 shutdown_connections 2024-05-09T04:58:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.747+0000 7fdd9632db80 1 -- 172.21.15.46:0/3513042185 wait complete. 2024-05-09T04:58:02.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.747+0000 7fdd9632db80 20 client.16881 trim_cache size 0 max 16384 2024-05-09T04:58:02.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd9632db80 1 Processor -- start 2024-05-09T04:58:02.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd9632db80 1 -- start start 2024-05-09T04:58:02.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd9632db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c00c4a0 0x55578c1923a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd9632db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c086d00 0x55578c1954d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd9632db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c189400 0x55578c193940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd9632db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdd801b1630 con 0x55578c189400 2024-05-09T04:58:02.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd9632db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdd801aeec0 con 0x55578c00c4a0 2024-05-09T04:58:02.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd9632db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdd801acd10 con 0x55578c086d00 2024-05-09T04:58:02.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd85eb4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c189400 0x55578c193940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd856b3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c00c4a0 0x55578c1923a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd85eb4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c189400 0x55578c193940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60540/0 (socket says 172.21.15.46:60540) 2024-05-09T04:58:02.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd856b3700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c00c4a0 0x55578c1923a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:48852/0 (socket says 172.21.15.46:48852) 2024-05-09T04:58:02.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd85eb4700 1 -- 172.21.15.46:0/3752918144 learned_addr learned my addr 172.21.15.46:0/3752918144 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:02.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd84eb2700 1 --2- 172.21.15.46:0/3752918144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c086d00 0x55578c1954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd85eb4700 1 -- 172.21.15.46:0/3752918144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c086d00 msgr2=0x55578c1954d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:02.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd85eb4700 1 --2- 172.21.15.46:0/3752918144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c086d00 0x55578c1954d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd85eb4700 1 -- 172.21.15.46:0/3752918144 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c00c4a0 msgr2=0x55578c1923a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:02.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd85eb4700 1 --2- 172.21.15.46:0/3752918144 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c00c4a0 0x55578c1923a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd85eb4700 1 -- 172.21.15.46:0/3752918144 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55578c1809d0 con 0x55578c189400 2024-05-09T04:58:02.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.768+0000 7fdd856b3700 1 --2- 172.21.15.46:0/3752918144 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c00c4a0 0x55578c1923a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:58:02.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.769+0000 7fdd85eb4700 1 --2- 172.21.15.46:0/3752918144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c189400 0x55578c193940 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800aba0 tx=0x7fdd7800aeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8dbf56a9e4ee38 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:02.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.769+0000 7fdd6affd700 1 -- 172.21.15.46:0/3752918144 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd78012d90 con 0x55578c189400 2024-05-09T04:58:02.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.769+0000 7fdd6affd700 1 -- 172.21.15.46:0/3752918144 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd78004ce0 con 0x55578c189400 2024-05-09T04:58:02.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.769+0000 7fdd6affd700 1 -- 172.21.15.46:0/3752918144 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7800f670 con 0x55578c189400 2024-05-09T04:58:02.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.770+0000 7fdd9632db80 1 -- 172.21.15.46:0/3752918144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c189400 msgr2=0x55578c193940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:02.776 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.770+0000 7fdd9632db80 1 --2- 172.21.15.46:0/3752918144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c189400 0x55578c193940 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fdd7800aba0 tx=0x7fdd7800aeb0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.770+0000 7fdd9632db80 1 -- 172.21.15.46:0/3752918144 shutdown_connections 2024-05-09T04:58:02.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.770+0000 7fdd9632db80 1 --2- 172.21.15.46:0/3752918144 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c189400 0x55578c193940 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.770+0000 7fdd9632db80 1 --2- 172.21.15.46:0/3752918144 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c00c4a0 0x55578c1923a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.770+0000 7fdd9632db80 1 --2- 172.21.15.46:0/3752918144 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c086d00 0x55578c1954d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.770+0000 7fdd9632db80 1 -- 172.21.15.46:0/3752918144 >> 172.21.15.46:0/3752918144 conn(0x55578c00ff70 msgr2=0x55578c17dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:02.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.770+0000 7fdd9632db80 1 -- 172.21.15.46:0/3752918144 shutdown_connections 2024-05-09T04:58:02.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.770+0000 7fdd9632db80 1 -- 172.21.15.46:0/3752918144 wait complete. 2024-05-09T04:58:02.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.770+0000 7fdd9632db80 1 -- start start 2024-05-09T04:58:02.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.770+0000 7fdd9632db80 1 Processor -- start 2024-05-09T04:58:02.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd9632db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c00c4a0 0x55578c1a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd9632db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c086d00 0x55578c1a54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd9632db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c189400 0x55578c20b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd9632db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdd801ad380 con 0x55578c00c4a0 2024-05-09T04:58:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd9632db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdd801ae7f0 con 0x55578c189400 2024-05-09T04:58:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd9632db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdd801ada50 con 0x55578c086d00 2024-05-09T04:58:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd9632db80 10 client.0 _ll_register_callbacks cb 0x55578bf1d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd856b3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c00c4a0 0x55578c1a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd856b3700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c00c4a0 0x55578c1a4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:60542/0 (socket says 172.21.15.46:60542) 2024-05-09T04:58:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd856b3700 1 -- 172.21.15.46:0/2318141714 learned_addr learned my addr 172.21.15.46:0/2318141714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd85eb4700 1 --2- 172.21.15.46:0/2318141714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c189400 0x55578c20b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd84eb2700 1 --2- 172.21.15.46:0/2318141714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c086d00 0x55578c1a54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd856b3700 1 -- 172.21.15.46:0/2318141714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c086d00 msgr2=0x55578c1a54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd856b3700 1 --2- 172.21.15.46:0/2318141714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c086d00 0x55578c1a54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd856b3700 1 -- 172.21.15.46:0/2318141714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c189400 msgr2=0x55578c20b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd856b3700 1 --2- 172.21.15.46:0/2318141714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c189400 0x55578c20b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd856b3700 1 -- 172.21.15.46:0/2318141714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55578c1a5cf0 con 0x55578c00c4a0 2024-05-09T04:58:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.771+0000 7fdd84eb2700 1 --2- 172.21.15.46:0/2318141714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c086d00 0x55578c1a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:58:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.772+0000 7fdd856b3700 1 --2- 172.21.15.46:0/2318141714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c00c4a0 0x55578c1a4f70 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009470 tx=0x7fdd7c008b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd723052b1d06f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.772+0000 7fdd68ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.772+0000 7fdd68ff9700 1 -- 172.21.15.46:0/2318141714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd7c049020 con 0x55578c00c4a0 2024-05-09T04:58:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.772+0000 7fdd68ff9700 1 -- 172.21.15.46:0/2318141714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fdd7c036070 con 0x55578c00c4a0 2024-05-09T04:58:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.772+0000 7fdd9632db80 1 -- 172.21.15.46:0/2318141714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdd78004ce0 con 0x55578c00c4a0 2024-05-09T04:58:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.772+0000 7fdd68ff9700 1 -- 172.21.15.46:0/2318141714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdd7c054050 con 0x55578c00c4a0 2024-05-09T04:58:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.772+0000 7fdd9632db80 10 client.16887 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.772+0000 7fdd9632db80 1 -- 172.21.15.46:0/2318141714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55578c20c9c0 con 0x55578c00c4a0 2024-05-09T04:58:02.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.772+0000 7fdd68ff9700 1 -- 172.21.15.46:0/2318141714 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd78004ce0 con 0x55578c00c4a0 2024-05-09T04:58:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.772+0000 7fdd68ff9700 1 -- 172.21.15.46:0/2318141714 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fdd7c027040 con 0x55578c00c4a0 2024-05-09T04:58:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.773+0000 7fdd9632db80 10 client.16887 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.773+0000 7fdd9632db80 10 client.16887 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:58:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.773+0000 7fdd9632db80 20 client.16887 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.773+0000 7fdd9632db80 1 --2- 172.21.15.46:0/2318141714 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55578c20d060 0x55578c20f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.773+0000 7fdd9632db80 4 client.16887 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:58:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.773+0000 7fdd9632db80 1 -- 172.21.15.46:0/2318141714 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55578c0b3b20 con 0x55578c20d060 2024-05-09T04:58:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.773+0000 7fdd84eb2700 1 --2- 172.21.15.46:0/2318141714 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55578c20d060 0x55578c20f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.773+0000 7fdd68ff9700 10 client.16887 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.773+0000 7fdd84eb2700 1 --2- 172.21.15.46:0/2318141714 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55578c20d060 0x55578c20f480 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e805d5540df8538 server_cookie=fd8476e51125a506 in_seq=0 out_seq=0 2024-05-09T04:58:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.774+0000 7fdd68ff9700 1 -- 172.21.15.46:0/2318141714 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55578c0b3b20 con 0x55578c20d060 2024-05-09T04:58:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.774+0000 7fdd68ff9700 10 client.16887 handle_command_reply: tid=0 2024-05-09T04:58:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.774+0000 7fdd9632db80 1 client.16887 shutdown 2024-05-09T04:58:02.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.774+0000 7fdd9632db80 1 -- 172.21.15.46:0/2318141714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c00c4a0 msgr2=0x55578c1a4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:02.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.774+0000 7fdd9632db80 1 --2- 172.21.15.46:0/2318141714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c00c4a0 0x55578c1a4f70 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c009470 tx=0x7fdd7c008b10 comp rx=0 tx=0).stop 2024-05-09T04:58:02.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.774+0000 7fdd9632db80 1 -- 172.21.15.46:0/2318141714 shutdown_connections 2024-05-09T04:58:02.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.774+0000 7fdd9632db80 1 --2- 172.21.15.46:0/2318141714 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55578c20d060 0x55578c20f480 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.774+0000 7fdd9632db80 1 --2- 172.21.15.46:0/2318141714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55578c189400 0x55578c20b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.774+0000 7fdd9632db80 1 --2- 172.21.15.46:0/2318141714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55578c00c4a0 0x55578c1a4f70 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.774+0000 7fdd9632db80 1 --2- 172.21.15.46:0/2318141714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55578c086d00 0x55578c1a54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.774+0000 7fdd9632db80 1 -- 172.21.15.46:0/2318141714 >> 172.21.15.46:0/2318141714 conn(0x55578c00ff70 msgr2=0x55578c192100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:02.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.774+0000 7fdd68ff9700 0 client.16887 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:02.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.774+0000 7fdd9632db80 1 -- 172.21.15.46:0/2318141714 shutdown_connections 2024-05-09T04:58:02.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.774+0000 7fdd9632db80 1 -- 172.21.15.46:0/2318141714 wait complete. 2024-05-09T04:58:02.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.774+0000 7fdd9632db80 20 client.16887 trim_cache size 0 max 16384 2024-05-09T04:58:02.782 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:58:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:58:02.782 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:58:02.782 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:58:02.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.775+0000 7fdd87917700 1 -- 172.21.15.46:0/3084743776 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdd6c0b2a70 msgr2=0x7fdd6c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:02.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.775+0000 7fdd87917700 1 --2- 172.21.15.46:0/3084743776 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdd6c0b2a70 0x7fdd6c0b4f80 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdd7c000f80 tx=0x7fdd7c00b040 comp rx=0 tx=0).stop 2024-05-09T04:58:02.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.775+0000 7fdd87917700 1 -- 172.21.15.46:0/3084743776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80108f10 msgr2=0x7fdd8019b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:02.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.775+0000 7fdd87917700 1 --2- 172.21.15.46:0/3084743776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80108f10 0x7fdd8019b3a0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c9e0 tx=0x7fdd7000ccf0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.775+0000 7fdd87917700 1 -- 172.21.15.46:0/3084743776 shutdown_connections 2024-05-09T04:58:02.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.775+0000 7fdd87917700 1 --2- 172.21.15.46:0/3084743776 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdd6c0b2a70 0x7fdd6c0b4f80 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.775+0000 7fdd87917700 1 --2- 172.21.15.46:0/3084743776 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdd80109860 0x7fdd8019f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.775+0000 7fdd87917700 1 --2- 172.21.15.46:0/3084743776 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdd801045e0 0x7fdd8019ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.775+0000 7fdd87917700 1 --2- 172.21.15.46:0/3084743776 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdd80108f10 0x7fdd8019b3a0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:02.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.775+0000 7fdd87917700 1 -- 172.21.15.46:0/3084743776 >> 172.21.15.46:0/3084743776 conn(0x7fdd800ffed0 msgr2=0x7fdd80108190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:02.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.775+0000 7fdd87917700 1 -- 172.21.15.46:0/3084743776 shutdown_connections 2024-05-09T04:58:02.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:02.775+0000 7fdd87917700 1 -- 172.21.15.46:0/3084743776 wait complete. 2024-05-09T04:58:03.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:02 smithi110 ceph-mon[34904]: pgmap v211: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-09T04:58:03.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:02 smithi046 ceph-mon[30136]: pgmap v211: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-09T04:58:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:02 smithi072 ceph-mon[34088]: pgmap v211: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-09T04:58:03.675 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:58:03.675 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:58:03.940 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:05.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:04 smithi110 ceph-mon[34904]: pgmap v212: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T04:58:05.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:04 smithi046 ceph-mon[30136]: pgmap v212: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T04:58:05.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:04 smithi072 ceph-mon[34088]: pgmap v212: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T04:58:05.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.718+0000 7fc236d82700 1 -- 172.21.15.46:0/2845452416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2301034c0 msgr2=0x7fc2301038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:05.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.718+0000 7fc236d82700 1 --2- 172.21.15.46:0/2845452416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2301034c0 0x7fc2301038d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc2240099c0 tx=0x7fc224009cd0 comp rx=0 tx=0).stop 2024-05-09T04:58:05.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.718+0000 7fc236d82700 1 -- 172.21.15.46:0/2845452416 shutdown_connections 2024-05-09T04:58:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.718+0000 7fc236d82700 1 --2- 172.21.15.46:0/2845452416 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc230108740 0x7fc2301100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.718+0000 7fc236d82700 1 --2- 172.21.15.46:0/2845452416 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc2301034c0 0x7fc2301038d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.718+0000 7fc236d82700 1 --2- 172.21.15.46:0/2845452416 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc230107df0 0x7fc230108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.718+0000 7fc236d82700 1 -- 172.21.15.46:0/2845452416 >> 172.21.15.46:0/2845452416 conn(0x7fc2300fedf0 msgr2=0x7fc230101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.718+0000 7fc236d82700 1 -- 172.21.15.46:0/2845452416 shutdown_connections 2024-05-09T04:58:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.718+0000 7fc236d82700 1 -- 172.21.15.46:0/2845452416 wait complete. 2024-05-09T04:58:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.719+0000 7fc236d82700 1 Processor -- start 2024-05-09T04:58:05.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.719+0000 7fc236d82700 1 -- start start 2024-05-09T04:58:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc236d82700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2301034c0 0x7fc230100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc236d82700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc230107df0 0x7fc2300ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc236d82700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc230108740 0x7fc2300ff8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc236d82700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc230113370 con 0x7fc230108740 2024-05-09T04:58:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc236d82700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301131f0 con 0x7fc230107df0 2024-05-09T04:58:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc236d82700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc2301134f0 con 0x7fc2301034c0 2024-05-09T04:58:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc22ffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc230107df0 0x7fc2300ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc23531f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc230108740 0x7fc2300ff8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc22ffff700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc230107df0 0x7fc2300ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:54894/0 (socket says 172.21.15.46:54894) 2024-05-09T04:58:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc22ffff700 1 -- 172.21.15.46:0/56398294 learned_addr learned my addr 172.21.15.46:0/56398294 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:05.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc23531f700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc230108740 0x7fc2300ff8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50022/0 (socket says 172.21.15.46:50022) 2024-05-09T04:58:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc234b1e700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2301034c0 0x7fc230100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc22ffff700 1 -- 172.21.15.46:0/56398294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2301034c0 msgr2=0x7fc230100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc22ffff700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2301034c0 0x7fc230100e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc22ffff700 1 -- 172.21.15.46:0/56398294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc230108740 msgr2=0x7fc2300ff8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc22ffff700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc230108740 0x7fc2300ff8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc22ffff700 1 -- 172.21.15.46:0/56398294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2301000e0 con 0x7fc230107df0 2024-05-09T04:58:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc234b1e700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2301034c0 0x7fc230100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc23531f700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc230108740 0x7fc2300ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:58:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.720+0000 7fc22ffff700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc230107df0 0x7fc2300ff2d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00d980 tx=0x7fc21c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.721+0000 7fc22dffb700 1 -- 172.21.15.46:0/56398294 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c017070 con 0x7fc230107df0 2024-05-09T04:58:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.721+0000 7fc236d82700 1 -- 172.21.15.46:0/56398294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2301ad880 con 0x7fc230107df0 2024-05-09T04:58:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.721+0000 7fc22dffb700 1 -- 172.21.15.46:0/56398294 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc21c009de0 con 0x7fc230107df0 2024-05-09T04:58:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.721+0000 7fc22dffb700 1 -- 172.21.15.46:0/56398294 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c00f3c0 con 0x7fc230107df0 2024-05-09T04:58:05.724 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.721+0000 7fc236d82700 1 -- 172.21.15.46:0/56398294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2301adde0 con 0x7fc230107df0 2024-05-09T04:58:05.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.722+0000 7fc236d82700 1 -- 172.21.15.46:0/56398294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc230104970 con 0x7fc230107df0 2024-05-09T04:58:05.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.724+0000 7fc22dffb700 1 -- 172.21.15.46:0/56398294 <== mon.2 v2:172.21.15.72:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc21c009930 con 0x7fc230107df0 2024-05-09T04:58:05.725 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.724+0000 7fc22dffb700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc2200b29f0 0x7fc2200b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:05.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.725+0000 7fc234b1e700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc2200b29f0 0x7fc2200b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:05.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.725+0000 7fc22dffb700 1 -- 172.21.15.46:0/56398294 <== mon.2 v2:172.21.15.72:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8556+0+0 (secure 0 0 0) 0x7fc21c0e2170 con 0x7fc230107df0 2024-05-09T04:58:05.726 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.725+0000 7fc234b1e700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc2200b29f0 0x7fc2200b4f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc224000c00 tx=0x7fc22401a040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:05.730 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.729+0000 7fc22dffb700 1 -- 172.21.15.46:0/56398294 <== mon.2 v2:172.21.15.72:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc21c0a9ad0 con 0x7fc230107df0 2024-05-09T04:58:05.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.993+0000 7fc236d82700 1 -- 172.21.15.46:0/56398294 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc230100800 con 0x7fc230107df0 2024-05-09T04:58:05.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.994+0000 7fc22dffb700 1 -- 172.21.15.46:0/56398294 <== mon.2 v2:172.21.15.72:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7fc21c0a93d0 con 0x7fc230107df0 2024-05-09T04:58:05.995 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:58:05.995 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:58:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.996+0000 7fc236d82700 1 -- 172.21.15.46:0/56398294 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc2200b29f0 msgr2=0x7fc2200b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.996+0000 7fc236d82700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc2200b29f0 0x7fc2200b4f00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc224000c00 tx=0x7fc22401a040 comp rx=0 tx=0).stop 2024-05-09T04:58:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.996+0000 7fc236d82700 1 -- 172.21.15.46:0/56398294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc230107df0 msgr2=0x7fc2300ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.996+0000 7fc236d82700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc230107df0 0x7fc2300ff2d0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00d980 tx=0x7fc21c00dc90 comp rx=0 tx=0).stop 2024-05-09T04:58:05.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.996+0000 7fc236d82700 1 -- 172.21.15.46:0/56398294 shutdown_connections 2024-05-09T04:58:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.996+0000 7fc236d82700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc2200b29f0 0x7fc2200b4f00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.996+0000 7fc236d82700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc230108740 0x7fc2300ff8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.996+0000 7fc236d82700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc2301034c0 0x7fc230100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.996+0000 7fc236d82700 1 --2- 172.21.15.46:0/56398294 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc230107df0 0x7fc2300ff2d0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.996+0000 7fc236d82700 1 -- 172.21.15.46:0/56398294 >> 172.21.15.46:0/56398294 conn(0x7fc2300fedf0 msgr2=0x7fc230075ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.996+0000 7fc236d82700 1 -- 172.21.15.46:0/56398294 shutdown_connections 2024-05-09T04:58:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:05.996+0000 7fc236d82700 1 -- 172.21.15.46:0/56398294 wait complete. 2024-05-09T04:58:05.998 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:58:06.799 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:58:07.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:06 smithi110 ceph-mon[34904]: pgmap v213: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T04:58:07.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:06 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/56398294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:58:07.066 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:07.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:06 smithi046 ceph-mon[30136]: pgmap v213: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T04:58:07.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:06 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/56398294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:58:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:06 smithi072 ceph-mon[34088]: pgmap v213: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T04:58:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:06 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/56398294' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:58:08.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:07 smithi110 ceph-mon[34904]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T04:58:08.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:07 smithi046 ceph-mon[30136]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T04:58:08.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:07 smithi072 ceph-mon[34088]: osdmap e86: 12 total, 12 up, 12 in 2024-05-09T04:58:08.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.845+0000 7f49f1b90700 1 -- 172.21.15.46:0/286020351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec108f30 msgr2=0x7f49ec109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:08.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.845+0000 7f49f1b90700 1 --2- 172.21.15.46:0/286020351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec108f30 0x7f49ec109340 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f49e00099c0 tx=0x7f49e0009cd0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.845+0000 7f49f1b90700 1 -- 172.21.15.46:0/286020351 shutdown_connections 2024-05-09T04:58:08.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.845+0000 7f49f1b90700 1 --2- 172.21.15.46:0/286020351 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49ec109880 0x7f49ec1111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.845+0000 7f49f1b90700 1 --2- 172.21.15.46:0/286020351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49ec104650 0x7f49ec104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.845+0000 7f49f1b90700 1 --2- 172.21.15.46:0/286020351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec108f30 0x7f49ec109340 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.845+0000 7f49f1b90700 1 -- 172.21.15.46:0/286020351 >> 172.21.15.46:0/286020351 conn(0x7f49ec079f00 msgr2=0x7f49ec07a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:08.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.845+0000 7f49f1b90700 1 -- 172.21.15.46:0/286020351 shutdown_connections 2024-05-09T04:58:08.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.845+0000 7f49f1b90700 1 -- 172.21.15.46:0/286020351 wait complete. 2024-05-09T04:58:08.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.846+0000 7f49f1b90700 1 Processor -- start 2024-05-09T04:58:08.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.846+0000 7f49f1b90700 1 -- start start 2024-05-09T04:58:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49f1b90700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49ec104650 0x7f49ec075ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49f1b90700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49ec108f30 0x7f49ec076000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49f1b90700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec109880 0x7f49ec076540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49f1b90700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec114500 con 0x7f49ec109880 2024-05-09T04:58:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49f1b90700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec114380 con 0x7f49ec108f30 2024-05-09T04:58:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49f1b90700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec114680 con 0x7f49ec104650 2024-05-09T04:58:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec109880 0x7f49ec076540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec109880 0x7f49ec076540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50052/0 (socket says 172.21.15.46:50052) 2024-05-09T04:58:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49ebfff700 1 -- 172.21.15.46:0/3397518400 learned_addr learned my addr 172.21.15.46:0/3397518400 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49eaffd700 1 --2- 172.21.15.46:0/3397518400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49ec108f30 0x7f49ec076000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49eb7fe700 1 --2- 172.21.15.46:0/3397518400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49ec104650 0x7f49ec075ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49ebfff700 1 -- 172.21.15.46:0/3397518400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49ec104650 msgr2=0x7f49ec075ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:08.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49ebfff700 1 --2- 172.21.15.46:0/3397518400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49ec104650 0x7f49ec075ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49ebfff700 1 -- 172.21.15.46:0/3397518400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49ec108f30 msgr2=0x7f49ec076000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:08.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49ebfff700 1 --2- 172.21.15.46:0/3397518400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49ec108f30 0x7f49ec076000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49ebfff700 1 -- 172.21.15.46:0/3397518400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ec1a9330 con 0x7f49ec109880 2024-05-09T04:58:08.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.847+0000 7f49ebfff700 1 --2- 172.21.15.46:0/3397518400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec109880 0x7f49ec076540 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f49e400ada0 tx=0x7f49e400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:08.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.848+0000 7f49e8ff9700 1 -- 172.21.15.46:0/3397518400 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4017020 con 0x7f49ec109880 2024-05-09T04:58:08.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.848+0000 7f49f1b90700 1 -- 172.21.15.46:0/3397518400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49ec1a9620 con 0x7f49ec109880 2024-05-09T04:58:08.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.848+0000 7f49e8ff9700 1 -- 172.21.15.46:0/3397518400 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e400cd80 con 0x7f49ec109880 2024-05-09T04:58:08.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.848+0000 7f49f1b90700 1 -- 172.21.15.46:0/3397518400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49ec1a9ae0 con 0x7f49ec109880 2024-05-09T04:58:08.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.848+0000 7f49e8ff9700 1 -- 172.21.15.46:0/3397518400 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e4005240 con 0x7f49ec109880 2024-05-09T04:58:08.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.850+0000 7f49e8ff9700 1 -- 172.21.15.46:0/3397518400 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f49e40054a0 con 0x7f49ec109880 2024-05-09T04:58:08.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.851+0000 7f49e8ff9700 1 --2- 172.21.15.46:0/3397518400 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49d40b29f0 0x7f49d40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.851+0000 7f49eb7fe700 1 --2- 172.21.15.46:0/3397518400 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49d40b29f0 0x7f49d40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.852 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.851+0000 7f49e8ff9700 1 -- 172.21.15.46:0/3397518400 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8644+0+0 (secure 0 0 0) 0x7f49e40e1d80 con 0x7f49ec109880 2024-05-09T04:58:08.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.852+0000 7f4a005a6b80 1 Processor -- start 2024-05-09T04:58:08.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.852+0000 7f4a005a6b80 1 -- start start 2024-05-09T04:58:08.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.852+0000 7f49eb7fe700 1 --2- 172.21.15.46:0/3397518400 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49d40b29f0 0x7f49d40b4f00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009a10 tx=0x7f49dc009190 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:08.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.852+0000 7f4a005a6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2cc2d00 0x55eee2dcb240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.852+0000 7f4a005a6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2c484a0 0x55eee2dcb780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.852+0000 7f4a005a6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2dc5400 0x55eee2dc5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.852+0000 7f4a005a6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55eee2ce80a0 con 0x55eee2c484a0 2024-05-09T04:58:08.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.852+0000 7f4a005a6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55eee2c8d9a0 con 0x55eee2dc5400 2024-05-09T04:58:08.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.852+0000 7f4a005a6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eee2ce4060 con 0x55eee2cc2d00 2024-05-09T04:58:08.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.852+0000 7f49eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2c484a0 0x55eee2dcb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.852+0000 7f49eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2c484a0 0x55eee2dcb780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50054/0 (socket says 172.21.15.46:50054) 2024-05-09T04:58:08.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.852+0000 7f49eaffd700 1 -- 172.21.15.46:0/1885574379 learned_addr learned my addr 172.21.15.46:0/1885574379 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:08.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.852+0000 7f49eb7fe700 1 --2- 172.21.15.46:0/1885574379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2cc2d00 0x55eee2dcb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.853+0000 7f49ebfff700 1 --2- 172.21.15.46:0/1885574379 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2dc5400 0x55eee2dc5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.853+0000 7f49eaffd700 1 -- 172.21.15.46:0/1885574379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2cc2d00 msgr2=0x55eee2dcb240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:08.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.853+0000 7f49eaffd700 1 --2- 172.21.15.46:0/1885574379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2cc2d00 0x55eee2dcb240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.853+0000 7f49eaffd700 1 -- 172.21.15.46:0/1885574379 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2dc5400 msgr2=0x55eee2dc5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:08.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.853+0000 7f49eaffd700 1 --2- 172.21.15.46:0/1885574379 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2dc5400 0x55eee2dc5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.853+0000 7f49eaffd700 1 -- 172.21.15.46:0/1885574379 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee2c7e500 con 0x55eee2c484a0 2024-05-09T04:58:08.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.853+0000 7f49eb7fe700 1 --2- 172.21.15.46:0/1885574379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2cc2d00 0x55eee2dcb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:08.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.853+0000 7f49eaffd700 1 --2- 172.21.15.46:0/1885574379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2c484a0 0x55eee2dcb780 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f49ec06b4c0 tx=0x7f49e0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=512705bded393cc3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:08.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.853+0000 7f49d1ffb700 1 -- 172.21.15.46:0/1885574379 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0003cb0 con 0x55eee2c484a0 2024-05-09T04:58:08.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.853+0000 7f49d1ffb700 1 -- 172.21.15.46:0/1885574379 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e0003e50 con 0x55eee2c484a0 2024-05-09T04:58:08.856 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.853+0000 7f49d1ffb700 1 -- 172.21.15.46:0/1885574379 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e000f460 con 0x55eee2c484a0 2024-05-09T04:58:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.854+0000 7f4a005a6b80 1 -- 172.21.15.46:0/1885574379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2c484a0 msgr2=0x55eee2dcb780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.854+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/1885574379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2c484a0 0x55eee2dcb780 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f49ec06b4c0 tx=0x7f49e0003910 comp rx=0 tx=0).stop 2024-05-09T04:58:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.854+0000 7f4a005a6b80 1 -- 172.21.15.46:0/1885574379 shutdown_connections 2024-05-09T04:58:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.854+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/1885574379 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2dc5400 0x55eee2dc5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.854+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/1885574379 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2cc2d00 0x55eee2dcb240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.854+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/1885574379 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2c484a0 0x55eee2dcb780 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.854+0000 7f4a005a6b80 1 -- 172.21.15.46:0/1885574379 >> 172.21.15.46:0/1885574379 conn(0x55eee2c4bf70 msgr2=0x55eee2dc0220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.854+0000 7f4a005a6b80 1 -- 172.21.15.46:0/1885574379 shutdown_connections 2024-05-09T04:58:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.854+0000 7f4a005a6b80 1 -- 172.21.15.46:0/1885574379 wait complete. 2024-05-09T04:58:08.859 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.855+0000 7f4a005a6b80 1 -- start start 2024-05-09T04:58:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.855+0000 7f4a005a6b80 1 Processor -- start 2024-05-09T04:58:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.855+0000 7f4a005a6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 0x55eee2dcefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.855+0000 7f4a005a6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2cc2d00 0x55eee2dcf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.855+0000 7f4a005a6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2dc5400 0x55eee2dcfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.855+0000 7f4a005a6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55eee2dc60a0 con 0x55eee2cc2d00 2024-05-09T04:58:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.855+0000 7f4a005a6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55eee2c82b20 con 0x55eee2dc5400 2024-05-09T04:58:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.855+0000 7f4a005a6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eee2dc6220 con 0x55eee2c484a0 2024-05-09T04:58:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.855+0000 7f4a005a6b80 10 client.0 _ll_register_callbacks cb 0x55eee2b596c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.855+0000 7f49eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2cc2d00 0x55eee2dcf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.855+0000 7f49eaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2cc2d00 0x55eee2dcf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50070/0 (socket says 172.21.15.46:50070) 2024-05-09T04:58:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.855+0000 7f49eaffd700 1 -- 172.21.15.46:0/3459470475 learned_addr learned my addr 172.21.15.46:0/3459470475 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.855+0000 7f49ebfff700 1 --2- 172.21.15.46:0/3459470475 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2dc5400 0x55eee2dcfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.855+0000 7f49eb7fe700 1 --2- 172.21.15.46:0/3459470475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 0x55eee2dcefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.856+0000 7f49eaffd700 1 -- 172.21.15.46:0/3459470475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 msgr2=0x55eee2dcefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:08.860 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.856+0000 7f49eaffd700 1 --2- 172.21.15.46:0/3459470475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 0x55eee2dcefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.856+0000 7f49eaffd700 1 -- 172.21.15.46:0/3459470475 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2dc5400 msgr2=0x55eee2dcfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.856+0000 7f49eaffd700 1 --2- 172.21.15.46:0/3459470475 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2dc5400 0x55eee2dcfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.856+0000 7f49eaffd700 1 -- 172.21.15.46:0/3459470475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eee2b63490 con 0x55eee2cc2d00 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.856+0000 7f49eaffd700 1 --2- 172.21.15.46:0/3459470475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2cc2d00 0x55eee2dcf510 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f49e000b580 tx=0x7f49e0003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a6dfc726c1e992 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.856+0000 7f49caffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.856+0000 7f49caffd700 1 -- 172.21.15.46:0/3459470475 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e0031020 con 0x55eee2cc2d00 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.856+0000 7f49caffd700 1 -- 172.21.15.46:0/3459470475 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8644+0+0 (secure 0 0 0) 0x7f49e0005550 con 0x55eee2cc2d00 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.856+0000 7f4a005a6b80 1 -- 172.21.15.46:0/3459470475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e0003e50 con 0x55eee2cc2d00 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.856+0000 7f49caffd700 1 -- 172.21.15.46:0/3459470475 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f49e003c050 con 0x55eee2cc2d00 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.856+0000 7f4a005a6b80 10 client.16905 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.856+0000 7f4a005a6b80 1 -- 172.21.15.46:0/3459470475 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eee2ddc900 con 0x55eee2cc2d00 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.857+0000 7f49caffd700 1 -- 172.21.15.46:0/3459470475 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e0003e50 con 0x55eee2cc2d00 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.857+0000 7f49caffd700 1 -- 172.21.15.46:0/3459470475 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f49e000f8a0 con 0x55eee2cc2d00 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.857+0000 7f4a005a6b80 10 client.16905 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.857+0000 7f4a005a6b80 10 client.16905 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:58:08.861 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.857+0000 7f4a005a6b80 20 client.16905 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:08.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.857+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/3459470475 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eee2ddedd0 0x55eee2de11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.857+0000 7f4a005a6b80 4 client.16905 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:58:08.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.857+0000 7f4a005a6b80 1 -- 172.21.15.46:0/3459470475 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eee2cf5a80 con 0x55eee2ddedd0 2024-05-09T04:58:08.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.857+0000 7f49eb7fe700 1 --2- 172.21.15.46:0/3459470475 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eee2ddedd0 0x55eee2de11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.858+0000 7f49caffd700 10 client.16905 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:58:08.862 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.858+0000 7f49eb7fe700 1 --2- 172.21.15.46:0/3459470475 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eee2ddedd0 0x55eee2de11f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b52e572d1604010b server_cookie=1d179f167f3cdf7f in_seq=0 out_seq=0 2024-05-09T04:58:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.858+0000 7f49caffd700 1 -- 172.21.15.46:0/3459470475 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eee2cf5a80 con 0x55eee2ddedd0 2024-05-09T04:58:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.858+0000 7f49caffd700 10 client.16905 handle_command_reply: tid=0 2024-05-09T04:58:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.858+0000 7f4a005a6b80 1 client.16905 shutdown 2024-05-09T04:58:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.859+0000 7f4a005a6b80 1 -- 172.21.15.46:0/3459470475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2cc2d00 msgr2=0x55eee2dcf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.859+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/3459470475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2cc2d00 0x55eee2dcf510 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f49e000b580 tx=0x7f49e0003d50 comp rx=0 tx=0).stop 2024-05-09T04:58:08.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.859+0000 7f4a005a6b80 1 -- 172.21.15.46:0/3459470475 shutdown_connections 2024-05-09T04:58:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.859+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/3459470475 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2dc5400 0x55eee2dcfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.859+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/3459470475 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eee2ddedd0 0x55eee2de11f0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.859+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/3459470475 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 0x55eee2dcefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.859+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/3459470475 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2cc2d00 0x55eee2dcf510 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.859+0000 7f49caffd700 0 client.16905 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:58:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.859+0000 7f4a005a6b80 1 -- 172.21.15.46:0/3459470475 >> 172.21.15.46:0/3459470475 conn(0x55eee2c4bf70 msgr2=0x55eee2db9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.859+0000 7f4a005a6b80 1 -- 172.21.15.46:0/3459470475 shutdown_connections 2024-05-09T04:58:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.859+0000 7f4a005a6b80 1 -- 172.21.15.46:0/3459470475 wait complete. 2024-05-09T04:58:08.864 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.859+0000 7f4a005a6b80 20 client.16905 trim_cache size 0 max 16384 2024-05-09T04:58:08.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.874+0000 7f4a005a6b80 1 Processor -- start 2024-05-09T04:58:08.876 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.874+0000 7f4a005a6b80 1 -- start start 2024-05-09T04:58:08.876 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:08 smithi046 ceph-mon[30136]: pgmap v215: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T04:58:08.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f4a005a6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 0x55eee2dce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f4a005a6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2cc2d00 0x55eee2ddb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f4a005a6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2dc5400 0x55eee2dc5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f4a005a6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec1b3b50 con 0x55eee2dc5400 2024-05-09T04:58:08.878 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f4a005a6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec1b13e0 con 0x55eee2cc2d00 2024-05-09T04:58:08.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f4a005a6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec1af250 con 0x55eee2c484a0 2024-05-09T04:58:08.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f49ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2dc5400 0x55eee2dc5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f49ebfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2dc5400 0x55eee2dc5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50072/0 (socket says 172.21.15.46:50072) 2024-05-09T04:58:08.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f49ebfff700 1 -- 172.21.15.46:0/3430266658 learned_addr learned my addr 172.21.15.46:0/3430266658 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:08.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f49eaffd700 1 --2- 172.21.15.46:0/3430266658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2cc2d00 0x55eee2ddb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f49eb7fe700 1 --2- 172.21.15.46:0/3430266658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 0x55eee2dce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f49ebfff700 1 -- 172.21.15.46:0/3430266658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 msgr2=0x55eee2dce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:08.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f49ebfff700 1 --2- 172.21.15.46:0/3430266658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 0x55eee2dce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f49ebfff700 1 -- 172.21.15.46:0/3430266658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2cc2d00 msgr2=0x55eee2ddb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:08.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f49ebfff700 1 --2- 172.21.15.46:0/3430266658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2cc2d00 0x55eee2ddb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.875+0000 7f49ebfff700 1 -- 172.21.15.46:0/3430266658 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eee2dc0f40 con 0x55eee2dc5400 2024-05-09T04:58:08.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.876+0000 7f49ebfff700 1 --2- 172.21.15.46:0/3430266658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2dc5400 0x55eee2dc5810 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f49e40a50f0 tx=0x7f49e40a8c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc140c6aa055c4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:08.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.876+0000 7f49d17fa700 1 -- 172.21.15.46:0/3430266658 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40ad420 con 0x55eee2dc5400 2024-05-09T04:58:08.879 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.876+0000 7f49d17fa700 1 -- 172.21.15.46:0/3430266658 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e40b1e40 con 0x55eee2dc5400 2024-05-09T04:58:08.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.876+0000 7f49d17fa700 1 -- 172.21.15.46:0/3430266658 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49e40b19b0 con 0x55eee2dc5400 2024-05-09T04:58:08.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.877+0000 7f4a005a6b80 1 -- 172.21.15.46:0/3430266658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2dc5400 msgr2=0x55eee2dc5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:08.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.877+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/3430266658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2dc5400 0x55eee2dc5810 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f49e40a50f0 tx=0x7f49e40a8c50 comp rx=0 tx=0).stop 2024-05-09T04:58:08.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.877+0000 7f4a005a6b80 1 -- 172.21.15.46:0/3430266658 shutdown_connections 2024-05-09T04:58:08.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.877+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/3430266658 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2dc5400 0x55eee2dc5810 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.877+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/3430266658 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 0x55eee2dce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.877+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/3430266658 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2cc2d00 0x55eee2ddb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.877+0000 7f4a005a6b80 1 -- 172.21.15.46:0/3430266658 >> 172.21.15.46:0/3430266658 conn(0x55eee2c4bf70 msgr2=0x55eee2db9ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:08.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.877+0000 7f4a005a6b80 1 -- 172.21.15.46:0/3430266658 shutdown_connections 2024-05-09T04:58:08.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.877+0000 7f4a005a6b80 1 -- 172.21.15.46:0/3430266658 wait complete. 2024-05-09T04:58:08.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.877+0000 7f4a005a6b80 1 -- start start 2024-05-09T04:58:08.880 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.877+0000 7f4a005a6b80 1 Processor -- start 2024-05-09T04:58:08.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f4a005a6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 0x55eee2de1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.883 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f4a005a6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2cc2d00 0x55eee2de1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f4a005a6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2dc5400 0x55eee2e47310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f4a005a6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec1af8a0 con 0x55eee2cc2d00 2024-05-09T04:58:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f4a005a6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec1b0d10 con 0x55eee2dc5400 2024-05-09T04:58:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f4a005a6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f49ec1aff70 con 0x55eee2c484a0 2024-05-09T04:58:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f49eb7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 0x55eee2de1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f49eb7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 0x55eee2de1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:55400/0 (socket says 172.21.15.46:55400) 2024-05-09T04:58:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f49eb7fe700 1 -- 172.21.15.46:0/1716565112 learned_addr learned my addr 172.21.15.46:0/1716565112 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f49eaffd700 1 --2- 172.21.15.46:0/1716565112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2cc2d00 0x55eee2de1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f49ebfff700 1 --2- 172.21.15.46:0/1716565112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2dc5400 0x55eee2e47310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f4a005a6b80 10 client.0 _ll_register_callbacks cb 0x55eee2b596c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f49eb7fe700 1 -- 172.21.15.46:0/1716565112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2dc5400 msgr2=0x55eee2e47310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f49eb7fe700 1 --2- 172.21.15.46:0/1716565112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2dc5400 0x55eee2e47310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f49eb7fe700 1 -- 172.21.15.46:0/1716565112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2cc2d00 msgr2=0x55eee2de1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f49eb7fe700 1 --2- 172.21.15.46:0/1716565112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2cc2d00 0x55eee2de1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.884 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.878+0000 7f49eb7fe700 1 -- 172.21.15.46:0/1716565112 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eee2e47850 con 0x55eee2c484a0 2024-05-09T04:58:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.879+0000 7f49eb7fe700 1 --2- 172.21.15.46:0/1716565112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 0x55eee2de1450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0097f0 tx=0x7f49dc004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=24d43eecc18ddef0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.879+0000 7f49caffd700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:58:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.879+0000 7f49caffd700 1 -- 172.21.15.46:0/1716565112 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc031020 con 0x55eee2c484a0 2024-05-09T04:58:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.879+0000 7f49caffd700 1 -- 172.21.15.46:0/1716565112 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8644+0+0 (secure 0 0 0) 0x7f49dc0159f0 con 0x55eee2c484a0 2024-05-09T04:58:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.879+0000 7f4a005a6b80 1 -- 172.21.15.46:0/1716565112 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f49e40b1e40 con 0x55eee2c484a0 2024-05-09T04:58:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.879+0000 7f49caffd700 1 -- 172.21.15.46:0/1716565112 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f49dc03c050 con 0x55eee2c484a0 2024-05-09T04:58:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.879+0000 7f4a005a6b80 10 client.26347 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.879+0000 7f4a005a6b80 1 -- 172.21.15.46:0/1716565112 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eee2e48990 con 0x55eee2c484a0 2024-05-09T04:58:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.879+0000 7f49caffd700 1 -- 172.21.15.46:0/1716565112 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49e40b1e40 con 0x55eee2c484a0 2024-05-09T04:58:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.880+0000 7f49caffd700 1 -- 172.21.15.46:0/1716565112 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f49dc022430 con 0x55eee2c484a0 2024-05-09T04:58:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.880+0000 7f4a005a6b80 10 client.26347 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.880+0000 7f4a005a6b80 10 client.26347 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:58:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.880+0000 7f4a005a6b80 20 client.26347 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.880+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/1716565112 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eee2e490f0 0x55eee2e4b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:08.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.880+0000 7f4a005a6b80 4 client.26347 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:58:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.880+0000 7f4a005a6b80 1 -- 172.21.15.46:0/1716565112 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eee2cefb20 con 0x55eee2e490f0 2024-05-09T04:58:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.880+0000 7f49eaffd700 1 --2- 172.21.15.46:0/1716565112 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eee2e490f0 0x55eee2e4b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.880+0000 7f49eaffd700 1 --2- 172.21.15.46:0/1716565112 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eee2e490f0 0x55eee2e4b510 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8a46837f0dae5917 server_cookie=f755c4d76288fca2 in_seq=0 out_seq=0 2024-05-09T04:58:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.880+0000 7f49caffd700 10 client.26347 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:58:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.881+0000 7f49caffd700 1 -- 172.21.15.46:0/1716565112 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eee2cefb20 con 0x55eee2e490f0 2024-05-09T04:58:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.881+0000 7f49caffd700 10 client.26347 handle_command_reply: tid=0 2024-05-09T04:58:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.881+0000 7f4a005a6b80 1 client.26347 shutdown 2024-05-09T04:58:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.881+0000 7f4a005a6b80 1 -- 172.21.15.46:0/1716565112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 msgr2=0x55eee2de1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.881+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/1716565112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 0x55eee2de1450 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0097f0 tx=0x7f49dc004750 comp rx=0 tx=0).stop 2024-05-09T04:58:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.881+0000 7f4a005a6b80 1 -- 172.21.15.46:0/1716565112 shutdown_connections 2024-05-09T04:58:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.881+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/1716565112 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eee2dc5400 0x55eee2e47310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.881+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/1716565112 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eee2e490f0 0x55eee2e4b510 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.881+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/1716565112 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eee2c484a0 0x55eee2de1450 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.881+0000 7f4a005a6b80 1 --2- 172.21.15.46:0/1716565112 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eee2cc2d00 0x55eee2de1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.881+0000 7f49caffd700 0 client.26347 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:58:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.881+0000 7f4a005a6b80 1 -- 172.21.15.46:0/1716565112 >> 172.21.15.46:0/1716565112 conn(0x55eee2c4bf70 msgr2=0x55eee2dce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.881+0000 7f4a005a6b80 1 -- 172.21.15.46:0/1716565112 shutdown_connections 2024-05-09T04:58:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.881+0000 7f4a005a6b80 1 -- 172.21.15.46:0/1716565112 wait complete. 2024-05-09T04:58:08.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.881+0000 7f4a005a6b80 20 client.26347 trim_cache size 0 max 16384 2024-05-09T04:58:08.887 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:58:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.882+0000 7f49f1b90700 1 -- 172.21.15.46:0/3397518400 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49d40b29f0 msgr2=0x7f49d40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.882+0000 7f49f1b90700 1 --2- 172.21.15.46:0/3397518400 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49d40b29f0 0x7f49d40b4f00 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f49dc009a10 tx=0x7f49dc009190 comp rx=0 tx=0).stop 2024-05-09T04:58:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.882+0000 7f49f1b90700 1 -- 172.21.15.46:0/3397518400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec109880 msgr2=0x7f49ec076540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.882+0000 7f49f1b90700 1 --2- 172.21.15.46:0/3397518400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec109880 0x7f49ec076540 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f49e400ada0 tx=0x7f49e400c330 comp rx=0 tx=0).stop 2024-05-09T04:58:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.882+0000 7f49f1b90700 1 -- 172.21.15.46:0/3397518400 shutdown_connections 2024-05-09T04:58:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.882+0000 7f49f1b90700 1 --2- 172.21.15.46:0/3397518400 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49d40b29f0 0x7f49d40b4f00 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.882+0000 7f49f1b90700 1 --2- 172.21.15.46:0/3397518400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49ec109880 0x7f49ec076540 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.882+0000 7f49f1b90700 1 --2- 172.21.15.46:0/3397518400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49ec104650 0x7f49ec075ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.882+0000 7f49f1b90700 1 --2- 172.21.15.46:0/3397518400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49ec108f30 0x7f49ec076000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.882+0000 7f49f1b90700 1 -- 172.21.15.46:0/3397518400 >> 172.21.15.46:0/3397518400 conn(0x7f49ec079f00 msgr2=0x7f49ec1020b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.882+0000 7f49f1b90700 1 -- 172.21.15.46:0/3397518400 shutdown_connections 2024-05-09T04:58:08.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:08.882+0000 7f49f1b90700 1 -- 172.21.15.46:0/3397518400 wait complete. 2024-05-09T04:58:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:08 smithi110 ceph-mon[34904]: pgmap v215: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T04:58:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:08 smithi072 ceph-mon[34088]: pgmap v215: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T04:58:09.653 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:58:09.913 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:10 smithi110 ceph-mon[34904]: pgmap v216: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T04:58:11.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:10 smithi046 ceph-mon[30136]: pgmap v216: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T04:58:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:10 smithi072 ceph-mon[34088]: pgmap v216: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T04:58:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.639+0000 7fe275dde700 1 -- 172.21.15.46:0/2627395439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe270105500 msgr2=0x7fe270105910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.639+0000 7fe275dde700 1 --2- 172.21.15.46:0/2627395439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe270105500 0x7fe270105910 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fe26000b0a0 tx=0x7fe26000b3b0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.639+0000 7fe275dde700 1 -- 172.21.15.46:0/2627395439 shutdown_connections 2024-05-09T04:58:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.639+0000 7fe275dde700 1 --2- 172.21.15.46:0/2627395439 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe27010e8e0 0x7fe270110dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.639+0000 7fe275dde700 1 --2- 172.21.15.46:0/2627395439 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe270105500 0x7fe270105910 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.639+0000 7fe275dde700 1 --2- 172.21.15.46:0/2627395439 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe270105e50 0x7fe27010e3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.639+0000 7fe275dde700 1 -- 172.21.15.46:0/2627395439 >> 172.21.15.46:0/2627395439 conn(0x7fe2700fedc0 msgr2=0x7fe2701011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:11.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.639+0000 7fe275dde700 1 -- 172.21.15.46:0/2627395439 shutdown_connections 2024-05-09T04:58:11.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.639+0000 7fe275dde700 1 -- 172.21.15.46:0/2627395439 wait complete. 2024-05-09T04:58:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.640+0000 7fe275dde700 1 Processor -- start 2024-05-09T04:58:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.640+0000 7fe275dde700 1 -- start start 2024-05-09T04:58:11.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.640+0000 7fe275dde700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe270105500 0x7fe27019ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.640+0000 7fe275dde700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe270105e50 0x7fe27019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.640+0000 7fe275dde700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27010e8e0 0x7fe27019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.640+0000 7fe275dde700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe270114430 con 0x7fe27010e8e0 2024-05-09T04:58:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.640+0000 7fe275dde700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe2701142b0 con 0x7fe270105500 2024-05-09T04:58:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.640+0000 7fe275dde700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2701145b0 con 0x7fe270105e50 2024-05-09T04:58:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe26ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27010e8e0 0x7fe27019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe26ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27010e8e0 0x7fe27019f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50084/0 (socket says 172.21.15.46:50084) 2024-05-09T04:58:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe26ffff700 1 -- 172.21.15.46:0/1949264955 learned_addr learned my addr 172.21.15.46:0/1949264955 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe266ffd700 1 --2- 172.21.15.46:0/1949264955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe270105e50 0x7fe27019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe26f7fe700 1 --2- 172.21.15.46:0/1949264955 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe270105500 0x7fe27019ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe26ffff700 1 -- 172.21.15.46:0/1949264955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe270105e50 msgr2=0x7fe27019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe26ffff700 1 --2- 172.21.15.46:0/1949264955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe270105e50 0x7fe27019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe26ffff700 1 -- 172.21.15.46:0/1949264955 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe270105500 msgr2=0x7fe27019ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:11.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe26ffff700 1 --2- 172.21.15.46:0/1949264955 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe270105500 0x7fe27019ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe26ffff700 1 -- 172.21.15.46:0/1949264955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe27019fc50 con 0x7fe27010e8e0 2024-05-09T04:58:11.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe26ffff700 1 --2- 172.21.15.46:0/1949264955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27010e8e0 0x7fe27019f5b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe26800d980 tx=0x7fe26800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:11.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe26d7fa700 1 -- 172.21.15.46:0/1949264955 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe268017070 con 0x7fe27010e8e0 2024-05-09T04:58:11.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe275dde700 1 -- 172.21.15.46:0/1949264955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe27019ff40 con 0x7fe27010e8e0 2024-05-09T04:58:11.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe26d7fa700 1 -- 172.21.15.46:0/1949264955 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe268009df0 con 0x7fe27010e8e0 2024-05-09T04:58:11.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe26d7fa700 1 -- 172.21.15.46:0/1949264955 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2680053a0 con 0x7fe27010e8e0 2024-05-09T04:58:11.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.641+0000 7fe275dde700 1 -- 172.21.15.46:0/1949264955 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2701a7860 con 0x7fe27010e8e0 2024-05-09T04:58:11.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.644+0000 7fe26d7fa700 1 -- 172.21.15.46:0/1949264955 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe268004750 con 0x7fe27010e8e0 2024-05-09T04:58:11.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.645+0000 7fe26d7fa700 1 --2- 172.21.15.46:0/1949264955 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe25c0b29f0 0x7fe25c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.645+0000 7fe2847f4b80 1 Processor -- start 2024-05-09T04:58:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.645+0000 7fe26d7fa700 1 -- 172.21.15.46:0/1949264955 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8644+0+0 (secure 0 0 0) 0x7fe2680e2640 con 0x7fe27010e8e0 2024-05-09T04:58:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.645+0000 7fe26f7fe700 1 --2- 172.21.15.46:0/1949264955 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe25c0b29f0 0x7fe25c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.645+0000 7fe2847f4b80 1 -- start start 2024-05-09T04:58:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe2847f4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdcdd3160 0x556cdce6ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe2847f4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcce94a0 0x556cdce6b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe2847f4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdce6f400 0x556cdce6f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe2847f4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556cdcd88ff0 con 0x556cdce6f400 2024-05-09T04:58:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe2847f4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556cdcd277f0 con 0x556cdcdd3160 2024-05-09T04:58:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe2847f4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556cdcd84fb0 con 0x556cdcce94a0 2024-05-09T04:58:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe26f7fe700 1 --2- 172.21.15.46:0/1949264955 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe25c0b29f0 0x7fe25c0b4f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe260006130 tx=0x7fe2600060a0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:11.648 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe26ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdce6f400 0x556cdce6f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe26ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdce6f400 0x556cdce6f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50098/0 (socket says 172.21.15.46:50098) 2024-05-09T04:58:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe26f7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdcdd3160 0x556cdce6ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe26ffff700 1 -- 172.21.15.46:0/2953308557 learned_addr learned my addr 172.21.15.46:0/2953308557 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe266ffd700 1 --2- 172.21.15.46:0/2953308557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcce94a0 0x556cdce6b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe26f7fe700 1 -- 172.21.15.46:0/2953308557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcce94a0 msgr2=0x556cdce6b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe26f7fe700 1 --2- 172.21.15.46:0/2953308557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcce94a0 0x556cdce6b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe26f7fe700 1 -- 172.21.15.46:0/2953308557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdce6f400 msgr2=0x556cdce6f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe26f7fe700 1 --2- 172.21.15.46:0/2953308557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdce6f400 0x556cdce6f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.646+0000 7fe26f7fe700 1 -- 172.21.15.46:0/2953308557 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cdcd1f370 con 0x556cdcdd3160 2024-05-09T04:58:11.649 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.647+0000 7fe26f7fe700 1 --2- 172.21.15.46:0/2953308557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdcdd3160 0x556cdce6ad50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe260009fd0 tx=0x7fe26001c000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d1e85aca93a9af1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:11.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.647+0000 7fe265ffb700 1 -- 172.21.15.46:0/2953308557 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe260007a20 con 0x556cdcdd3160 2024-05-09T04:58:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.647+0000 7fe265ffb700 1 -- 172.21.15.46:0/2953308557 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe26002d070 con 0x556cdcdd3160 2024-05-09T04:58:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.647+0000 7fe265ffb700 1 -- 172.21.15.46:0/2953308557 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe260005040 con 0x556cdcdd3160 2024-05-09T04:58:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.648+0000 7fe2847f4b80 1 -- 172.21.15.46:0/2953308557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdcdd3160 msgr2=0x556cdce6ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.648+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/2953308557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdcdd3160 0x556cdce6ad50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe260009fd0 tx=0x7fe26001c000 comp rx=0 tx=0).stop 2024-05-09T04:58:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.648+0000 7fe2847f4b80 1 -- 172.21.15.46:0/2953308557 shutdown_connections 2024-05-09T04:58:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.648+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/2953308557 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdce6f400 0x556cdce6f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.648+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/2953308557 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdcdd3160 0x556cdce6ad50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.648+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/2953308557 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcce94a0 0x556cdce6b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.648+0000 7fe2847f4b80 1 -- 172.21.15.46:0/2953308557 >> 172.21.15.46:0/2953308557 conn(0x556cdcd496a0 msgr2=0x556cdce61400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.648+0000 7fe2847f4b80 1 -- 172.21.15.46:0/2953308557 shutdown_connections 2024-05-09T04:58:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.648+0000 7fe2847f4b80 1 -- 172.21.15.46:0/2953308557 wait complete. 2024-05-09T04:58:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.648+0000 7fe2847f4b80 1 -- start start 2024-05-09T04:58:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe2847f4b80 1 Processor -- start 2024-05-09T04:58:11.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe2847f4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcce94a0 0x556cdce74650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe2847f4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcdd3160 0x556cdce74b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe2847f4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 0x556cdce750d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe2847f4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x556cdcc04ba0 con 0x556cdcdd3160 2024-05-09T04:58:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe2847f4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x556cdcd43150 con 0x556cdce6f400 2024-05-09T04:58:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe2847f4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x556cdce70150 con 0x556cdcce94a0 2024-05-09T04:58:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe2847f4b80 10 client.0 _ll_register_callbacks cb 0x556cdcbfa6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe266ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcdd3160 0x556cdce74b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe266ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcdd3160 0x556cdce74b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50112/0 (socket says 172.21.15.46:50112) 2024-05-09T04:58:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe266ffd700 1 -- 172.21.15.46:0/3095973722 learned_addr learned my addr 172.21.15.46:0/3095973722 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe26f7fe700 1 --2- 172.21.15.46:0/3095973722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcce94a0 0x556cdce74650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe26ffff700 1 --2- 172.21.15.46:0/3095973722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 0x556cdce750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe266ffd700 1 -- 172.21.15.46:0/3095973722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcce94a0 msgr2=0x556cdce74650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe266ffd700 1 --2- 172.21.15.46:0/3095973722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcce94a0 0x556cdce74650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe266ffd700 1 -- 172.21.15.46:0/3095973722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 msgr2=0x556cdce750d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe266ffd700 1 --2- 172.21.15.46:0/3095973722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 0x556cdce750d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.649+0000 7fe266ffd700 1 -- 172.21.15.46:0/3095973722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cdcc04490 con 0x556cdcdd3160 2024-05-09T04:58:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.650+0000 7fe266ffd700 1 --2- 172.21.15.46:0/3095973722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcdd3160 0x556cdce74b90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe25800bfd0 tx=0x7fe258012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167c788531a77fcf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.650+0000 7fe24effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.650+0000 7fe24effd700 1 -- 172.21.15.46:0/3095973722 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe258004440 con 0x556cdcdd3160 2024-05-09T04:58:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.650+0000 7fe24effd700 1 -- 172.21.15.46:0/3095973722 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8644+0+0 (secure 0 0 0) 0x7fe258003680 con 0x556cdcdd3160 2024-05-09T04:58:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.650+0000 7fe2847f4b80 1 -- 172.21.15.46:0/3095973722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe26002d070 con 0x556cdcdd3160 2024-05-09T04:58:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.650+0000 7fe24effd700 1 -- 172.21.15.46:0/3095973722 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe258003880 con 0x556cdcdd3160 2024-05-09T04:58:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.650+0000 7fe2847f4b80 10 client.16923 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.650+0000 7fe2847f4b80 1 -- 172.21.15.46:0/3095973722 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556cdce81f50 con 0x556cdcdd3160 2024-05-09T04:58:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.650+0000 7fe24effd700 1 -- 172.21.15.46:0/3095973722 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe26002d070 con 0x556cdcdd3160 2024-05-09T04:58:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.650+0000 7fe24effd700 1 -- 172.21.15.46:0/3095973722 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fe258021070 con 0x556cdcdd3160 2024-05-09T04:58:11.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.651+0000 7fe2847f4b80 10 client.16923 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.651+0000 7fe2847f4b80 10 client.16923 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:58:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.651+0000 7fe2847f4b80 20 client.16923 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.651+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/3095973722 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x556cdce84590 0x556cdce869d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.651+0000 7fe2847f4b80 4 client.16923 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:58:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.651+0000 7fe2847f4b80 1 -- 172.21.15.46:0/3095973722 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556cdcd969d0 con 0x556cdce84590 2024-05-09T04:58:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.651+0000 7fe26f7fe700 1 --2- 172.21.15.46:0/3095973722 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x556cdce84590 0x556cdce869d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.651+0000 7fe26f7fe700 1 --2- 172.21.15.46:0/3095973722 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x556cdce84590 0x556cdce869d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2d079901e7b3c5b server_cookie=19df8db4e02439d in_seq=0 out_seq=0 2024-05-09T04:58:11.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.651+0000 7fe24effd700 10 client.16923 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:58:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.652+0000 7fe24effd700 1 -- 172.21.15.46:0/3095973722 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556cdcd969d0 con 0x556cdce84590 2024-05-09T04:58:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.652+0000 7fe24effd700 10 client.16923 handle_command_reply: tid=0 2024-05-09T04:58:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.652+0000 7fe2847f4b80 1 client.16923 shutdown 2024-05-09T04:58:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.652+0000 7fe2847f4b80 1 -- 172.21.15.46:0/3095973722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcdd3160 msgr2=0x556cdce74b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.652+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/3095973722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcdd3160 0x556cdce74b90 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe25800bfd0 tx=0x7fe258012730 comp rx=0 tx=0).stop 2024-05-09T04:58:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.652+0000 7fe2847f4b80 1 -- 172.21.15.46:0/3095973722 shutdown_connections 2024-05-09T04:58:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.652+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/3095973722 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x556cdce84590 0x556cdce869d0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.652+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/3095973722 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 0x556cdce750d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.652+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/3095973722 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcce94a0 0x556cdce74650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.652+0000 7fe24effd700 0 client.16923 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:58:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.652+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/3095973722 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcdd3160 0x556cdce74b90 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.652+0000 7fe2847f4b80 1 -- 172.21.15.46:0/3095973722 >> 172.21.15.46:0/3095973722 conn(0x556cdcd496a0 msgr2=0x556cdce611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.653+0000 7fe2847f4b80 1 -- 172.21.15.46:0/3095973722 shutdown_connections 2024-05-09T04:58:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.653+0000 7fe2847f4b80 1 -- 172.21.15.46:0/3095973722 wait complete. 2024-05-09T04:58:11.658 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.653+0000 7fe2847f4b80 20 client.16923 trim_cache size 0 max 16384 2024-05-09T04:58:11.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe2847f4b80 1 Processor -- start 2024-05-09T04:58:11.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe2847f4b80 1 -- start start 2024-05-09T04:58:11.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe2847f4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcce94a0 0x556cdce660c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe2847f4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcdd3160 0x556cdce74f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe2847f4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 0x556cdce80110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe2847f4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe2701af650 con 0x556cdcce94a0 2024-05-09T04:58:11.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe2847f4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe2701acee0 con 0x556cdce6f400 2024-05-09T04:58:11.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe2847f4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2701aad50 con 0x556cdcdd3160 2024-05-09T04:58:11.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe26f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcce94a0 0x556cdce660c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe26f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcce94a0 0x556cdce660c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50128/0 (socket says 172.21.15.46:50128) 2024-05-09T04:58:11.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe26f7fe700 1 -- 172.21.15.46:0/4091121095 learned_addr learned my addr 172.21.15.46:0/4091121095 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:11.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe26ffff700 1 --2- 172.21.15.46:0/4091121095 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 0x556cdce80110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe266ffd700 1 --2- 172.21.15.46:0/4091121095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcdd3160 0x556cdce74f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe26f7fe700 1 -- 172.21.15.46:0/4091121095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcdd3160 msgr2=0x556cdce74f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:11.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe26f7fe700 1 --2- 172.21.15.46:0/4091121095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcdd3160 0x556cdce74f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe26f7fe700 1 -- 172.21.15.46:0/4091121095 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 msgr2=0x556cdce80110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:11.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.668+0000 7fe26f7fe700 1 --2- 172.21.15.46:0/4091121095 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 0x556cdce80110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.669+0000 7fe26f7fe700 1 -- 172.21.15.46:0/4091121095 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556cdce72bc0 con 0x556cdcce94a0 2024-05-09T04:58:11.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.669+0000 7fe26f7fe700 1 --2- 172.21.15.46:0/4091121095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcce94a0 0x556cdce660c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe26000b830 tx=0x7fe260004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e0ec3169e1b4543 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:11.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.669+0000 7fe2657fa700 1 -- 172.21.15.46:0/4091121095 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe260039070 con 0x556cdcce94a0 2024-05-09T04:58:11.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.669+0000 7fe2657fa700 1 -- 172.21.15.46:0/4091121095 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe26002d040 con 0x556cdcce94a0 2024-05-09T04:58:11.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.669+0000 7fe2657fa700 1 -- 172.21.15.46:0/4091121095 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe26003b570 con 0x556cdcce94a0 2024-05-09T04:58:11.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.670+0000 7fe2847f4b80 1 -- 172.21.15.46:0/4091121095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcce94a0 msgr2=0x556cdce660c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:11.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.670+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/4091121095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcce94a0 0x556cdce660c0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe26000b830 tx=0x7fe260004f10 comp rx=0 tx=0).stop 2024-05-09T04:58:11.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.670+0000 7fe2847f4b80 1 -- 172.21.15.46:0/4091121095 shutdown_connections 2024-05-09T04:58:11.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.670+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/4091121095 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 0x556cdce80110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.670+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/4091121095 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcce94a0 0x556cdce660c0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.670+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/4091121095 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcdd3160 0x556cdce74f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.670+0000 7fe2847f4b80 1 -- 172.21.15.46:0/4091121095 >> 172.21.15.46:0/4091121095 conn(0x556cdcd496a0 msgr2=0x556cdce735b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.670+0000 7fe2847f4b80 1 -- 172.21.15.46:0/4091121095 shutdown_connections 2024-05-09T04:58:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.670+0000 7fe2847f4b80 1 -- 172.21.15.46:0/4091121095 wait complete. 2024-05-09T04:58:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.670+0000 7fe2847f4b80 1 -- start start 2024-05-09T04:58:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.670+0000 7fe2847f4b80 1 Processor -- start 2024-05-09T04:58:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe2847f4b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcce94a0 0x556cdce7cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe2847f4b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcdd3160 0x556cdce77e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe2847f4b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 0x556cdce784c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe2847f4b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe2701ab3a0 con 0x556cdcdd3160 2024-05-09T04:58:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe2847f4b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe2701ac810 con 0x556cdce6f400 2024-05-09T04:58:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe2847f4b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2701aba70 con 0x556cdcce94a0 2024-05-09T04:58:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe2847f4b80 10 client.0 _ll_register_callbacks cb 0x556cdcbfa6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe266ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcdd3160 0x556cdce77e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe266ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcdd3160 0x556cdce77e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50136/0 (socket says 172.21.15.46:50136) 2024-05-09T04:58:11.675 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe266ffd700 1 -- 172.21.15.46:0/2369741768 learned_addr learned my addr 172.21.15.46:0/2369741768 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe26ffff700 1 --2- 172.21.15.46:0/2369741768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 0x556cdce784c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe26f7fe700 1 --2- 172.21.15.46:0/2369741768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcce94a0 0x556cdce7cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe266ffd700 1 -- 172.21.15.46:0/2369741768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcce94a0 msgr2=0x556cdce7cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe266ffd700 1 --2- 172.21.15.46:0/2369741768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcce94a0 0x556cdce7cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe266ffd700 1 -- 172.21.15.46:0/2369741768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 msgr2=0x556cdce784c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe266ffd700 1 --2- 172.21.15.46:0/2369741768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 0x556cdce784c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe266ffd700 1 -- 172.21.15.46:0/2369741768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556cdce78c40 con 0x556cdcdd3160 2024-05-09T04:58:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.671+0000 7fe26ffff700 1 --2- 172.21.15.46:0/2369741768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 0x556cdce784c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe266ffd700 1 --2- 172.21.15.46:0/2369741768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcdd3160 0x556cdce77e70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe25800bcf0 tx=0x7fe258005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ff0b34d4756dc18 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe24effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe24effd700 1 -- 172.21.15.46:0/2369741768 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe25802f020 con 0x556cdcdd3160 2024-05-09T04:58:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe24effd700 1 -- 172.21.15.46:0/2369741768 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8644+0+0 (secure 0 0 0) 0x7fe258003e30 con 0x556cdcdd3160 2024-05-09T04:58:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe2847f4b80 1 -- 172.21.15.46:0/2369741768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe26002d040 con 0x556cdcdd3160 2024-05-09T04:58:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe24effd700 1 -- 172.21.15.46:0/2369741768 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe25803a050 con 0x556cdcdd3160 2024-05-09T04:58:11.676 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe2847f4b80 10 client.16929 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe2847f4b80 1 -- 172.21.15.46:0/2369741768 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x556cdceeed60 con 0x556cdcdd3160 2024-05-09T04:58:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe24effd700 1 -- 172.21.15.46:0/2369741768 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe26002d040 con 0x556cdcdd3160 2024-05-09T04:58:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe24effd700 1 -- 172.21.15.46:0/2369741768 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fe258021040 con 0x556cdcdd3160 2024-05-09T04:58:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe2847f4b80 10 client.16929 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe2847f4b80 10 client.16929 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:58:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe2847f4b80 20 client.16929 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/2369741768 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x556cdceef4a0 0x556cdcef18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe2847f4b80 4 client.16929 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:58:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.672+0000 7fe2847f4b80 1 -- 172.21.15.46:0/2369741768 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556cdcd90a70 con 0x556cdceef4a0 2024-05-09T04:58:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.673+0000 7fe26f7fe700 1 --2- 172.21.15.46:0/2369741768 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x556cdceef4a0 0x556cdcef18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.673+0000 7fe26f7fe700 1 --2- 172.21.15.46:0/2369741768 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x556cdceef4a0 0x556cdcef18c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b885c4484a1ea0ec server_cookie=783cb8b9541615b3 in_seq=0 out_seq=0 2024-05-09T04:58:11.677 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.673+0000 7fe24effd700 10 client.16929 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:58:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.673+0000 7fe24effd700 1 -- 172.21.15.46:0/2369741768 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556cdcd90a70 con 0x556cdceef4a0 2024-05-09T04:58:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.673+0000 7fe24effd700 10 client.16929 handle_command_reply: tid=0 2024-05-09T04:58:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.673+0000 7fe2847f4b80 1 client.16929 shutdown 2024-05-09T04:58:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe2847f4b80 1 -- 172.21.15.46:0/2369741768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcdd3160 msgr2=0x556cdce77e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/2369741768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcdd3160 0x556cdce77e70 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fe25800bcf0 tx=0x7fe258005d50 comp rx=0 tx=0).stop 2024-05-09T04:58:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe2847f4b80 1 -- 172.21.15.46:0/2369741768 shutdown_connections 2024-05-09T04:58:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/2369741768 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x556cdceef4a0 0x556cdcef18c0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/2369741768 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x556cdce6f400 0x556cdce784c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/2369741768 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x556cdcce94a0 0x556cdce7cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe24effd700 0 client.16929 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:58:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe2847f4b80 1 --2- 172.21.15.46:0/2369741768 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x556cdcdd3160 0x556cdce77e70 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe2847f4b80 1 -- 172.21.15.46:0/2369741768 >> 172.21.15.46:0/2369741768 conn(0x556cdcd496a0 msgr2=0x556cdce61310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe2847f4b80 1 -- 172.21.15.46:0/2369741768 shutdown_connections 2024-05-09T04:58:11.679 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe2847f4b80 1 -- 172.21.15.46:0/2369741768 wait complete. 2024-05-09T04:58:11.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe2847f4b80 20 client.16929 trim_cache size 0 max 16384 2024-05-09T04:58:11.680 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:58:11.680 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe275dde700 1 -- 172.21.15.46:0/1949264955 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe25c0b29f0 msgr2=0x7fe25c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe275dde700 1 --2- 172.21.15.46:0/1949264955 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe25c0b29f0 0x7fe25c0b4f00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe260006130 tx=0x7fe2600060a0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe275dde700 1 -- 172.21.15.46:0/1949264955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27010e8e0 msgr2=0x7fe27019f5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.674+0000 7fe275dde700 1 --2- 172.21.15.46:0/1949264955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27010e8e0 0x7fe27019f5b0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fe26800d980 tx=0x7fe26800dd40 comp rx=0 tx=0).stop 2024-05-09T04:58:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.675+0000 7fe275dde700 1 -- 172.21.15.46:0/1949264955 shutdown_connections 2024-05-09T04:58:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.675+0000 7fe275dde700 1 --2- 172.21.15.46:0/1949264955 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe25c0b29f0 0x7fe25c0b4f00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.675+0000 7fe275dde700 1 --2- 172.21.15.46:0/1949264955 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe27010e8e0 0x7fe27019f5b0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.675+0000 7fe275dde700 1 --2- 172.21.15.46:0/1949264955 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe270105500 0x7fe27019ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.675+0000 7fe275dde700 1 --2- 172.21.15.46:0/1949264955 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe270105e50 0x7fe27019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.675+0000 7fe275dde700 1 -- 172.21.15.46:0/1949264955 >> 172.21.15.46:0/1949264955 conn(0x7fe2700fedc0 msgr2=0x7fe2700ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.675+0000 7fe275dde700 1 -- 172.21.15.46:0/1949264955 shutdown_connections 2024-05-09T04:58:11.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:11.675+0000 7fe275dde700 1 -- 172.21.15.46:0/1949264955 wait complete. 2024-05-09T04:58:12.581 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:58:12.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:12 smithi046 ceph-mon[30136]: pgmap v217: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-05-09T04:58:12.840 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:13.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:12 smithi110 ceph-mon[34904]: pgmap v217: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-05-09T04:58:13.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:12 smithi072 ceph-mon[34088]: pgmap v217: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-05-09T04:58:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.538+0000 7f7c6e6c8700 1 -- 172.21.15.46:0/724031351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c681092f0 msgr2=0x7f7c68109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.538+0000 7f7c6e6c8700 1 --2- 172.21.15.46:0/724031351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c681092f0 0x7f7c68109700 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009990 tx=0x7f7c58009ca0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.538+0000 7f7c6e6c8700 1 -- 172.21.15.46:0/724031351 shutdown_connections 2024-05-09T04:58:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.538+0000 7f7c6e6c8700 1 --2- 172.21.15.46:0/724031351 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c681030d0 0x7f7c68111160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.538+0000 7f7c6e6c8700 1 --2- 172.21.15.46:0/724031351 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c681092f0 0x7f7c68109700 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.538+0000 7f7c6e6c8700 1 --2- 172.21.15.46:0/724031351 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c68102780 0x7f7c68102b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.538+0000 7f7c6e6c8700 1 -- 172.21.15.46:0/724031351 >> 172.21.15.46:0/724031351 conn(0x7f7c680fe530 msgr2=0x7f7c68100980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.538+0000 7f7c6e6c8700 1 -- 172.21.15.46:0/724031351 shutdown_connections 2024-05-09T04:58:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.538+0000 7f7c6e6c8700 1 -- 172.21.15.46:0/724031351 wait complete. 2024-05-09T04:58:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.539+0000 7f7c6e6c8700 1 Processor -- start 2024-05-09T04:58:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.539+0000 7f7c6e6c8700 1 -- start start 2024-05-09T04:58:14.542 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.539+0000 7f7c6e6c8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c68102780 0x7f7c6819d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.539+0000 7f7c6e6c8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c681030d0 0x7f7c6819d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.539+0000 7f7c6e6c8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c681092f0 0x7f7c681a1940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.539+0000 7f7c6e6c8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68114580 con 0x7f7c681092f0 2024-05-09T04:58:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.539+0000 7f7c6e6c8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68114400 con 0x7f7c681030d0 2024-05-09T04:58:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.539+0000 7f7c6e6c8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7c68114700 con 0x7f7c68102780 2024-05-09T04:58:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.539+0000 7f7c6cc65700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c681092f0 0x7f7c681a1940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.539+0000 7f7c67fff700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c68102780 0x7f7c6819d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.539+0000 7f7c6cc65700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c681092f0 0x7f7c681a1940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35544/0 (socket says 172.21.15.46:35544) 2024-05-09T04:58:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.539+0000 7f7c6cc65700 1 -- 172.21.15.46:0/1017120385 learned_addr learned my addr 172.21.15.46:0/1017120385 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.539+0000 7f7c5ffff700 1 --2- 172.21.15.46:0/1017120385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c681030d0 0x7f7c6819d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.540+0000 7f7c67fff700 1 -- 172.21.15.46:0/1017120385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c681030d0 msgr2=0x7f7c6819d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.540+0000 7f7c67fff700 1 --2- 172.21.15.46:0/1017120385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c681030d0 0x7f7c6819d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.540+0000 7f7c67fff700 1 -- 172.21.15.46:0/1017120385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c681092f0 msgr2=0x7f7c681a1940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:14.543 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.540+0000 7f7c67fff700 1 --2- 172.21.15.46:0/1017120385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c681092f0 0x7f7c681a1940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.540+0000 7f7c67fff700 1 -- 172.21.15.46:0/1017120385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c681a1fe0 con 0x7f7c68102780 2024-05-09T04:58:14.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.540+0000 7f7c6cc65700 1 --2- 172.21.15.46:0/1017120385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c681092f0 0x7f7c681a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:58:14.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.540+0000 7f7c67fff700 1 --2- 172.21.15.46:0/1017120385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c68102780 0x7f7c6819d110 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7c58000c00 tx=0x7f7c580047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:14.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.540+0000 7f7c65ffb700 1 -- 172.21.15.46:0/1017120385 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58027070 con 0x7f7c68102780 2024-05-09T04:58:14.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.540+0000 7f7c6e6c8700 1 -- 172.21.15.46:0/1017120385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c681a2270 con 0x7f7c68102780 2024-05-09T04:58:14.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.540+0000 7f7c65ffb700 1 -- 172.21.15.46:0/1017120385 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c58003f20 con 0x7f7c68102780 2024-05-09T04:58:14.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.540+0000 7f7c65ffb700 1 -- 172.21.15.46:0/1017120385 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c58022b30 con 0x7f7c68102780 2024-05-09T04:58:14.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.540+0000 7f7c6e6c8700 1 -- 172.21.15.46:0/1017120385 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c681a2770 con 0x7f7c68102780 2024-05-09T04:58:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.543+0000 7f7c65ffb700 1 -- 172.21.15.46:0/1017120385 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7c5802a070 con 0x7f7c68102780 2024-05-09T04:58:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.544+0000 7f7c65ffb700 1 --2- 172.21.15.46:0/1017120385 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7c540b29f0 0x7f7c540b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.544+0000 7f7c7d0deb80 1 Processor -- start 2024-05-09T04:58:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.544+0000 7f7c5ffff700 1 --2- 172.21.15.46:0/1017120385 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7c540b29f0 0x7f7c540b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.544+0000 7f7c65ffb700 1 -- 172.21.15.46:0/1017120385 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8644+0+0 (secure 0 0 0) 0x7f7c580e3600 con 0x7f7c68102780 2024-05-09T04:58:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.544+0000 7f7c7d0deb80 1 -- start start 2024-05-09T04:58:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.544+0000 7f7c7d0deb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 0x5577db376240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.544+0000 7f7c7d0deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db1f34a0 0x5577db376780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.544+0000 7f7c7d0deb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db370400 0x5577db370810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.544+0000 7f7c7d0deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5577db2930a0 con 0x5577db1f34a0 2024-05-09T04:58:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.544+0000 7f7c7d0deb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5577db2389a0 con 0x5577db26dd00 2024-05-09T04:58:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.544+0000 7f7c7d0deb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5577db28f060 con 0x5577db370400 2024-05-09T04:58:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.545+0000 7f7c5ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db1f34a0 0x5577db376780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.545+0000 7f7c5ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db1f34a0 0x5577db376780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35554/0 (socket says 172.21.15.46:35554) 2024-05-09T04:58:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.545+0000 7f7c5ffff700 1 -- 172.21.15.46:0/3679720231 learned_addr learned my addr 172.21.15.46:0/3679720231 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.545+0000 7f7c67fff700 1 --2- 172.21.15.46:0/3679720231 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 0x5577db376240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.545+0000 7f7c5ffff700 1 --2- 172.21.15.46:0/1017120385 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7c540b29f0 0x7f7c540b4f00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c68111630 tx=0x7f7c5000b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.545+0000 7f7c6cc65700 1 --2- 172.21.15.46:0/3679720231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db370400 0x5577db370810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.545+0000 7f7c5ffff700 1 -- 172.21.15.46:0/3679720231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db370400 msgr2=0x5577db370810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.545+0000 7f7c5ffff700 1 --2- 172.21.15.46:0/3679720231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db370400 0x5577db370810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.545+0000 7f7c5ffff700 1 -- 172.21.15.46:0/3679720231 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 msgr2=0x5577db376240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.545+0000 7f7c5ffff700 1 --2- 172.21.15.46:0/3679720231 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 0x5577db376240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.545+0000 7f7c5ffff700 1 -- 172.21.15.46:0/3679720231 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577db229500 con 0x5577db1f34a0 2024-05-09T04:58:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.545+0000 7f7c5ffff700 1 --2- 172.21.15.46:0/3679720231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db1f34a0 0x5577db376780 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f7c50010f40 tx=0x7f7c50013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20963bf6cc1d1d7a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.545+0000 7f7c5e7fc700 1 -- 172.21.15.46:0/3679720231 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5000be60 con 0x5577db1f34a0 2024-05-09T04:58:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.545+0000 7f7c5e7fc700 1 -- 172.21.15.46:0/3679720231 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c50011db0 con 0x5577db1f34a0 2024-05-09T04:58:14.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.545+0000 7f7c5e7fc700 1 -- 172.21.15.46:0/3679720231 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c5001d6b0 con 0x5577db1f34a0 2024-05-09T04:58:14.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.546+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/3679720231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db1f34a0 msgr2=0x5577db376780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.546+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/3679720231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db1f34a0 0x5577db376780 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f7c50010f40 tx=0x7f7c50013000 comp rx=0 tx=0).stop 2024-05-09T04:58:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.546+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/3679720231 shutdown_connections 2024-05-09T04:58:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.546+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/3679720231 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db370400 0x5577db370810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.546+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/3679720231 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 0x5577db376240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.546+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/3679720231 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db1f34a0 0x5577db376780 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.546+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/3679720231 >> 172.21.15.46:0/3679720231 conn(0x5577db1f6f70 msgr2=0x5577db36b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.546+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/3679720231 shutdown_connections 2024-05-09T04:58:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.546+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/3679720231 wait complete. 2024-05-09T04:58:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.547+0000 7f7c7d0deb80 1 -- start start 2024-05-09T04:58:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.547+0000 7f7c7d0deb80 1 Processor -- start 2024-05-09T04:58:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.547+0000 7f7c7d0deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db1f34a0 0x5577db37a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.547+0000 7f7c7d0deb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 0x5577db37a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.547+0000 7f7c7d0deb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db370400 0x5577db37aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.547+0000 7f7c7d0deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5577db371160 con 0x5577db1f34a0 2024-05-09T04:58:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.547+0000 7f7c7d0deb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5577db22db20 con 0x5577db26dd00 2024-05-09T04:58:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.547+0000 7f7c7d0deb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5577db37d020 con 0x5577db370400 2024-05-09T04:58:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.547+0000 7f7c7d0deb80 10 client.0 _ll_register_callbacks cb 0x5577db1046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c67fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db1f34a0 0x5577db37a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c67fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db1f34a0 0x5577db37a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35558/0 (socket says 172.21.15.46:35558) 2024-05-09T04:58:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c67fff700 1 -- 172.21.15.46:0/951549671 learned_addr learned my addr 172.21.15.46:0/951549671 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c6cc65700 1 --2- 172.21.15.46:0/951549671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db370400 0x5577db37aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c5ffff700 1 --2- 172.21.15.46:0/951549671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 0x5577db37a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c67fff700 1 -- 172.21.15.46:0/951549671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db370400 msgr2=0x5577db37aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c67fff700 1 --2- 172.21.15.46:0/951549671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db370400 0x5577db37aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c67fff700 1 -- 172.21.15.46:0/951549671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 msgr2=0x5577db37a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c67fff700 1 --2- 172.21.15.46:0/951549671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 0x5577db37a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c67fff700 1 -- 172.21.15.46:0/951549671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577db10e490 con 0x5577db1f34a0 2024-05-09T04:58:14.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c6cc65700 1 --2- 172.21.15.46:0/951549671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db370400 0x5577db37aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:58:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c67fff700 1 --2- 172.21.15.46:0/951549671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db1f34a0 0x5577db37a050 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009990 tx=0x7f7c580aa130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25892a4e0a4cce59 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c477fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c477fe700 1 -- 172.21.15.46:0/951549671 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c580aa890 con 0x5577db1f34a0 2024-05-09T04:58:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c477fe700 1 -- 172.21.15.46:0/951549671 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8644+0+0 (secure 0 0 0) 0x7f7c580040d0 con 0x5577db1f34a0 2024-05-09T04:58:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.548+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/951549671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c50011db0 con 0x5577db1f34a0 2024-05-09T04:58:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.549+0000 7f7c477fe700 1 -- 172.21.15.46:0/951549671 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7c580aec40 con 0x5577db1f34a0 2024-05-09T04:58:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.549+0000 7f7c7d0deb80 10 client.16941 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.549+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/951549671 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5577db387a30 con 0x5577db1f34a0 2024-05-09T04:58:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.549+0000 7f7c477fe700 1 -- 172.21.15.46:0/951549671 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c50011db0 con 0x5577db1f34a0 2024-05-09T04:58:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.549+0000 7f7c477fe700 1 -- 172.21.15.46:0/951549671 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7c580b3260 con 0x5577db1f34a0 2024-05-09T04:58:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.549+0000 7f7c7d0deb80 10 client.16941 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.549+0000 7f7c7d0deb80 10 client.16941 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:58:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.549+0000 7f7c7d0deb80 20 client.16941 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.549+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/951549671 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5577db389fa0 0x5577db38c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.549+0000 7f7c7d0deb80 4 client.16941 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:58:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.549+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/951549671 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577db2a0a80 con 0x5577db389fa0 2024-05-09T04:58:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.549+0000 7f7c5ffff700 1 --2- 172.21.15.46:0/951549671 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5577db389fa0 0x5577db38c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.550+0000 7f7c477fe700 10 client.16941 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:14.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.550+0000 7f7c5ffff700 1 --2- 172.21.15.46:0/951549671 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5577db389fa0 0x5577db38c450 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7caff2aa140b2cee server_cookie=c6bad4180c8cf346 in_seq=0 out_seq=0 2024-05-09T04:58:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.551+0000 7f7c477fe700 1 -- 172.21.15.46:0/951549671 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5577db2a0a80 con 0x5577db389fa0 2024-05-09T04:58:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.551+0000 7f7c477fe700 10 client.16941 handle_command_reply: tid=0 2024-05-09T04:58:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.551+0000 7f7c7d0deb80 1 client.16941 shutdown 2024-05-09T04:58:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.551+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/951549671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db1f34a0 msgr2=0x5577db37a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.551+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/951549671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db1f34a0 0x5577db37a050 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f7c58009990 tx=0x7f7c580aa130 comp rx=0 tx=0).stop 2024-05-09T04:58:14.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.551+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/951549671 shutdown_connections 2024-05-09T04:58:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.551+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/951549671 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db370400 0x5577db37aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.551+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/951549671 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db1f34a0 0x5577db37a050 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.551+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/951549671 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5577db389fa0 0x5577db38c450 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.551+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/951549671 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 0x5577db37a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.551+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/951549671 >> 172.21.15.46:0/951549671 conn(0x5577db1f6f70 msgr2=0x5577db364ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.551+0000 7f7c477fe700 0 client.16941 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.551+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/951549671 shutdown_connections 2024-05-09T04:58:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.551+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/951549671 wait complete. 2024-05-09T04:58:14.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.551+0000 7f7c7d0deb80 20 client.16941 trim_cache size 0 max 16384 2024-05-09T04:58:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c7d0deb80 1 Processor -- start 2024-05-09T04:58:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c7d0deb80 1 -- start start 2024-05-09T04:58:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c7d0deb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db1f34a0 0x5577db37a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c7d0deb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db26dd00 0x5577db379050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c7d0deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db370400 0x5577db364ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.569 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c7d0deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681af610 con 0x5577db370400 2024-05-09T04:58:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c7d0deb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681acea0 con 0x5577db1f34a0 2024-05-09T04:58:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c7d0deb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681aae00 con 0x5577db26dd00 2024-05-09T04:58:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c6cc65700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db370400 0x5577db364ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c6cc65700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db370400 0x5577db364ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35572/0 (socket says 172.21.15.46:35572) 2024-05-09T04:58:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c6cc65700 1 -- 172.21.15.46:0/2495501328 learned_addr learned my addr 172.21.15.46:0/2495501328 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c5ffff700 1 --2- 172.21.15.46:0/2495501328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db26dd00 0x5577db379050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c67fff700 1 --2- 172.21.15.46:0/2495501328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db1f34a0 0x5577db37a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c6cc65700 1 -- 172.21.15.46:0/2495501328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db26dd00 msgr2=0x5577db379050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c6cc65700 1 --2- 172.21.15.46:0/2495501328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db26dd00 0x5577db379050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c6cc65700 1 -- 172.21.15.46:0/2495501328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db1f34a0 msgr2=0x5577db37a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c6cc65700 1 --2- 172.21.15.46:0/2495501328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db1f34a0 0x5577db37a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.570 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.567+0000 7f7c6cc65700 1 -- 172.21.15.46:0/2495501328 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577db371600 con 0x5577db370400 2024-05-09T04:58:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.568+0000 7f7c6cc65700 1 --2- 172.21.15.46:0/2495501328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db370400 0x5577db364ff0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7c60007cf0 tx=0x7f7c600091b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d98662c6a569df6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.568+0000 7f7c5dffb700 1 -- 172.21.15.46:0/2495501328 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6001b070 con 0x5577db370400 2024-05-09T04:58:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.568+0000 7f7c5dffb700 1 -- 172.21.15.46:0/2495501328 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c60015bb0 con 0x5577db370400 2024-05-09T04:58:14.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.568+0000 7f7c5dffb700 1 -- 172.21.15.46:0/2495501328 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c60017670 con 0x5577db370400 2024-05-09T04:58:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.569+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/2495501328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db370400 msgr2=0x5577db364ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.569+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/2495501328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db370400 0x5577db364ff0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7c60007cf0 tx=0x7f7c600091b0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.569+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/2495501328 shutdown_connections 2024-05-09T04:58:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.569+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/2495501328 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db370400 0x5577db364ff0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.569+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/2495501328 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db1f34a0 0x5577db37a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.569+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/2495501328 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db26dd00 0x5577db379050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.569+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/2495501328 >> 172.21.15.46:0/2495501328 conn(0x5577db1f6f70 msgr2=0x5577db378940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.569+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/2495501328 shutdown_connections 2024-05-09T04:58:14.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.569+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/2495501328 wait complete. 2024-05-09T04:58:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.569+0000 7f7c7d0deb80 1 -- start start 2024-05-09T04:58:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.569+0000 7f7c7d0deb80 1 Processor -- start 2024-05-09T04:58:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c7d0deb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db1f34a0 0x5577db3820b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c7d0deb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 0x5577db3825f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c7d0deb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db370400 0x5577db379900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c7d0deb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681ab380 con 0x5577db370400 2024-05-09T04:58:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c7d0deb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681ac7d0 con 0x5577db26dd00 2024-05-09T04:58:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c7d0deb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7c681aba30 con 0x5577db1f34a0 2024-05-09T04:58:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c7d0deb80 10 client.0 _ll_register_callbacks cb 0x5577db1046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c6cc65700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db370400 0x5577db379900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c6cc65700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db370400 0x5577db379900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35576/0 (socket says 172.21.15.46:35576) 2024-05-09T04:58:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c6cc65700 1 -- 172.21.15.46:0/3537965273 learned_addr learned my addr 172.21.15.46:0/3537965273 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c5ffff700 1 --2- 172.21.15.46:0/3537965273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 0x5577db3825f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c67fff700 1 --2- 172.21.15.46:0/3537965273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db1f34a0 0x5577db3820b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c6cc65700 1 -- 172.21.15.46:0/3537965273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db1f34a0 msgr2=0x5577db3820b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c6cc65700 1 --2- 172.21.15.46:0/3537965273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db1f34a0 0x5577db3820b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c6cc65700 1 -- 172.21.15.46:0/3537965273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 msgr2=0x5577db3825f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c6cc65700 1 --2- 172.21.15.46:0/3537965273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 0x5577db3825f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c6cc65700 1 -- 172.21.15.46:0/3537965273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577db38c9e0 con 0x5577db370400 2024-05-09T04:58:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c5ffff700 1 --2- 172.21.15.46:0/3537965273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 0x5577db3825f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:58:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c6cc65700 1 --2- 172.21.15.46:0/3537965273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db370400 0x5577db379900 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7c60000f80 tx=0x7f7c60015f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba1a35ddb526f842 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.570+0000 7f7c477fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.571+0000 7f7c477fe700 1 -- 172.21.15.46:0/3537965273 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c6001b040 con 0x5577db370400 2024-05-09T04:58:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.571+0000 7f7c477fe700 1 -- 172.21.15.46:0/3537965273 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8644+0+0 (secure 0 0 0) 0x7f7c60004b50 con 0x5577db370400 2024-05-09T04:58:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.571+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/3537965273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c60015bb0 con 0x5577db370400 2024-05-09T04:58:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.571+0000 7f7c477fe700 1 -- 172.21.15.46:0/3537965273 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7c60004d50 con 0x5577db370400 2024-05-09T04:58:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.571+0000 7f7c7d0deb80 10 client.16947 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.571+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/3537965273 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5577db364cc0 con 0x5577db370400 2024-05-09T04:58:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.571+0000 7f7c477fe700 1 -- 172.21.15.46:0/3537965273 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c60015bb0 con 0x5577db370400 2024-05-09T04:58:14.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.571+0000 7f7c477fe700 1 -- 172.21.15.46:0/3537965273 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7c60005540 con 0x5577db370400 2024-05-09T04:58:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.571+0000 7f7c7d0deb80 10 client.16947 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.571+0000 7f7c7d0deb80 10 client.16947 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:58:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.571+0000 7f7c7d0deb80 20 client.16947 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.571+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/3537965273 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5577db3f4180 0x5577db3f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.571+0000 7f7c7d0deb80 4 client.16947 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:58:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.571+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/3537965273 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577db29ab20 con 0x5577db3f4180 2024-05-09T04:58:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.572+0000 7f7c67fff700 1 --2- 172.21.15.46:0/3537965273 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5577db3f4180 0x5577db3f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.572+0000 7f7c67fff700 1 --2- 172.21.15.46:0/3537965273 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5577db3f4180 0x5577db3f65a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=659c94167f0c027a server_cookie=91c5931e3d678363 in_seq=0 out_seq=0 2024-05-09T04:58:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.572+0000 7f7c477fe700 10 client.16947 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.572+0000 7f7c477fe700 1 -- 172.21.15.46:0/3537965273 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577db29ab20 con 0x5577db3f4180 2024-05-09T04:58:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.572+0000 7f7c477fe700 10 client.16947 handle_command_reply: tid=0 2024-05-09T04:58:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.572+0000 7f7c7d0deb80 1 client.16947 shutdown 2024-05-09T04:58:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.572+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/3537965273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db370400 msgr2=0x5577db379900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.572+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/3537965273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db370400 0x5577db379900 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7c60000f80 tx=0x7f7c60015f90 comp rx=0 tx=0).stop 2024-05-09T04:58:14.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.572+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/3537965273 shutdown_connections 2024-05-09T04:58:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.572+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/3537965273 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5577db370400 0x5577db379900 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.572+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/3537965273 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5577db1f34a0 0x5577db3820b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.572+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/3537965273 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5577db3f4180 0x5577db3f65a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.572+0000 7f7c7d0deb80 1 --2- 172.21.15.46:0/3537965273 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5577db26dd00 0x5577db3825f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.572+0000 7f7c477fe700 0 client.16947 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.572+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/3537965273 >> 172.21.15.46:0/3537965273 conn(0x5577db1f6f70 msgr2=0x5577db364ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.573+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/3537965273 shutdown_connections 2024-05-09T04:58:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.573+0000 7f7c7d0deb80 1 -- 172.21.15.46:0/3537965273 wait complete. 2024-05-09T04:58:14.579 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.573+0000 7f7c7d0deb80 20 client.16947 trim_cache size 0 max 16384 2024-05-09T04:58:14.579 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:58:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.573+0000 7f7c6e6c8700 1 -- 172.21.15.46:0/1017120385 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7c540b29f0 msgr2=0x7f7c540b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.573+0000 7f7c6e6c8700 1 --2- 172.21.15.46:0/1017120385 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7c540b29f0 0x7f7c540b4f00 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c68111630 tx=0x7f7c5000b410 comp rx=0 tx=0).stop 2024-05-09T04:58:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.573+0000 7f7c6e6c8700 1 -- 172.21.15.46:0/1017120385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c68102780 msgr2=0x7f7c6819d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.573+0000 7f7c6e6c8700 1 --2- 172.21.15.46:0/1017120385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c68102780 0x7f7c6819d110 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7c58000c00 tx=0x7f7c580047b0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.573+0000 7f7c6e6c8700 1 -- 172.21.15.46:0/1017120385 shutdown_connections 2024-05-09T04:58:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.573+0000 7f7c6e6c8700 1 --2- 172.21.15.46:0/1017120385 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7c540b29f0 0x7f7c540b4f00 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.573+0000 7f7c6e6c8700 1 --2- 172.21.15.46:0/1017120385 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7c681092f0 0x7f7c681a1940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.580 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.573+0000 7f7c6e6c8700 1 --2- 172.21.15.46:0/1017120385 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7c68102780 0x7f7c6819d110 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.573+0000 7f7c6e6c8700 1 --2- 172.21.15.46:0/1017120385 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7c681030d0 0x7f7c6819d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.573+0000 7f7c6e6c8700 1 -- 172.21.15.46:0/1017120385 >> 172.21.15.46:0/1017120385 conn(0x7f7c680fe530 msgr2=0x7f7c6810fb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.573+0000 7f7c6e6c8700 1 -- 172.21.15.46:0/1017120385 shutdown_connections 2024-05-09T04:58:14.581 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:14.573+0000 7f7c6e6c8700 1 -- 172.21.15.46:0/1017120385 wait complete. 2024-05-09T04:58:14.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:14 smithi046 ceph-mon[30136]: pgmap v218: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T04:58:15.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:14 smithi110 ceph-mon[34904]: pgmap v218: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T04:58:15.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:14 smithi072 ceph-mon[34088]: pgmap v218: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T04:58:16.299 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:58:16.299 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:58:16.567 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:17.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:16 smithi046 ceph-mon[30136]: pgmap v219: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T04:58:17.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:16 smithi046 ceph-mon[30136]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T04:58:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:16 smithi110 ceph-mon[34904]: pgmap v219: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T04:58:17.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:16 smithi110 ceph-mon[34904]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T04:58:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:16 smithi072 ceph-mon[34088]: pgmap v219: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T04:58:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:16 smithi072 ceph-mon[34088]: osdmap e87: 12 total, 12 up, 12 in 2024-05-09T04:58:18.341 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.340+0000 7f54b562d700 1 -- 172.21.15.46:0/3927952481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b0105dd0 msgr2=0x7f54b010e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:18.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.340+0000 7f54b562d700 1 --2- 172.21.15.46:0/3927952481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b0105dd0 0x7f54b010e320 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f54a400b0a0 tx=0x7f54a400b3b0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.340+0000 7f54b562d700 1 -- 172.21.15.46:0/3927952481 shutdown_connections 2024-05-09T04:58:18.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.340+0000 7f54b562d700 1 --2- 172.21.15.46:0/3927952481 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54b010e8f0 0x7f54b0110dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.340+0000 7f54b562d700 1 --2- 172.21.15.46:0/3927952481 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54b0105480 0x7f54b0105890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.340+0000 7f54b562d700 1 --2- 172.21.15.46:0/3927952481 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b0105dd0 0x7f54b010e320 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.340+0000 7f54b562d700 1 -- 172.21.15.46:0/3927952481 >> 172.21.15.46:0/3927952481 conn(0x7f54b00fef30 msgr2=0x7f54b0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:18.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.340+0000 7f54b562d700 1 -- 172.21.15.46:0/3927952481 shutdown_connections 2024-05-09T04:58:18.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.340+0000 7f54b562d700 1 -- 172.21.15.46:0/3927952481 wait complete. 2024-05-09T04:58:18.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.341+0000 7f54b562d700 1 Processor -- start 2024-05-09T04:58:18.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.341+0000 7f54b562d700 1 -- start start 2024-05-09T04:58:18.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.341+0000 7f54b562d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54b0105480 0x7f54b01a2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.341+0000 7f54b562d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b0105dd0 0x7f54b01a2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.341+0000 7f54b562d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54b010e8f0 0x7f54b01ad1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.341+0000 7f54b562d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f54b0114450 con 0x7f54b0105dd0 2024-05-09T04:58:18.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.341+0000 7f54b562d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01142d0 con 0x7f54b0105480 2024-05-09T04:58:18.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.341+0000 7f54b562d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01145d0 con 0x7f54b010e8f0 2024-05-09T04:58:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.341+0000 7f54ae7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b0105dd0 0x7f54b01a2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.341+0000 7f54ae7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b0105dd0 0x7f54b01a2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35592/0 (socket says 172.21.15.46:35592) 2024-05-09T04:58:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.341+0000 7f54ae7fc700 1 -- 172.21.15.46:0/4188374471 learned_addr learned my addr 172.21.15.46:0/4188374471 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.342+0000 7f54aeffd700 1 --2- 172.21.15.46:0/4188374471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54b0105480 0x7f54b01a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.342+0000 7f54af7fe700 1 --2- 172.21.15.46:0/4188374471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54b010e8f0 0x7f54b01ad1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.342+0000 7f54ae7fc700 1 -- 172.21.15.46:0/4188374471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54b010e8f0 msgr2=0x7f54b01ad1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.342+0000 7f54ae7fc700 1 --2- 172.21.15.46:0/4188374471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54b010e8f0 0x7f54b01ad1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.342+0000 7f54ae7fc700 1 -- 172.21.15.46:0/4188374471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54b0105480 msgr2=0x7f54b01a2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.342+0000 7f54ae7fc700 1 --2- 172.21.15.46:0/4188374471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54b0105480 0x7f54b01a2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.342+0000 7f54ae7fc700 1 -- 172.21.15.46:0/4188374471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54b01ad6f0 con 0x7f54b0105dd0 2024-05-09T04:58:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.342+0000 7f54aeffd700 1 --2- 172.21.15.46:0/4188374471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54b0105480 0x7f54b01a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.342+0000 7f54ae7fc700 1 --2- 172.21.15.46:0/4188374471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b0105dd0 0x7f54b01a2670 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f54a400b070 tx=0x7f54a4004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:18.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.342+0000 7f5497fff700 1 -- 172.21.15.46:0/4188374471 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4021070 con 0x7f54b0105dd0 2024-05-09T04:58:18.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.342+0000 7f5497fff700 1 -- 172.21.15.46:0/4188374471 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a4007e30 con 0x7f54b0105dd0 2024-05-09T04:58:18.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.342+0000 7f54b562d700 1 -- 172.21.15.46:0/4188374471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54b01ad8c0 con 0x7f54b0105dd0 2024-05-09T04:58:18.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.342+0000 7f5497fff700 1 -- 172.21.15.46:0/4188374471 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a4004840 con 0x7f54b0105dd0 2024-05-09T04:58:18.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.342+0000 7f54b562d700 1 -- 172.21.15.46:0/4188374471 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54b01add10 con 0x7f54b0105dd0 2024-05-09T04:58:18.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.346+0000 7f5497fff700 1 -- 172.21.15.46:0/4188374471 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f54a401d450 con 0x7f54b0105dd0 2024-05-09T04:58:18.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.347+0000 7f5497fff700 1 --2- 172.21.15.46:0/4188374471 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f54980b2a70 0x7f54980b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.347+0000 7f54c4043b80 1 Processor -- start 2024-05-09T04:58:18.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.347+0000 7f54aeffd700 1 --2- 172.21.15.46:0/4188374471 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f54980b2a70 0x7f54980b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.347+0000 7f54c4043b80 1 -- start start 2024-05-09T04:58:18.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.347+0000 7f5497fff700 1 -- 172.21.15.46:0/4188374471 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f54a400f020 con 0x7f54b0105dd0 2024-05-09T04:58:18.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.347+0000 7f54c4043b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22c26d80 0x557a22d2f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.347+0000 7f54c4043b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22bac520 0x557a22d2f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.347+0000 7f54c4043b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22d29480 0x557a22d29890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.347+0000 7f54c4043b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557a22c4c120 con 0x557a22d29480 2024-05-09T04:58:18.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.347+0000 7f54c4043b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557a22bf1a20 con 0x557a22c26d80 2024-05-09T04:58:18.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.347+0000 7f54c4043b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557a22c480e0 con 0x557a22bac520 2024-05-09T04:58:18.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54af7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22d29480 0x557a22d29890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54af7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22d29480 0x557a22d29890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35596/0 (socket says 172.21.15.46:35596) 2024-05-09T04:58:18.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54af7fe700 1 -- 172.21.15.46:0/2215868488 learned_addr learned my addr 172.21.15.46:0/2215868488 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:18.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54aeffd700 1 --2- 172.21.15.46:0/4188374471 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f54980b2a70 0x7f54980b4f80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54a0005950 tx=0x7f54a00058e0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:18.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54ae7fc700 1 --2- 172.21.15.46:0/2215868488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22bac520 0x557a22d2f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54aeffd700 1 --2- 172.21.15.46:0/2215868488 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22c26d80 0x557a22d2f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54af7fe700 1 -- 172.21.15.46:0/2215868488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22bac520 msgr2=0x557a22d2f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54af7fe700 1 --2- 172.21.15.46:0/2215868488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22bac520 0x557a22d2f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54af7fe700 1 -- 172.21.15.46:0/2215868488 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22c26d80 msgr2=0x557a22d2f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54af7fe700 1 --2- 172.21.15.46:0/2215868488 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22c26d80 0x557a22d2f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54af7fe700 1 -- 172.21.15.46:0/2215868488 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a22be2580 con 0x557a22d29480 2024-05-09T04:58:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54ae7fc700 1 --2- 172.21.15.46:0/2215868488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22bac520 0x557a22d2f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54af7fe700 1 --2- 172.21.15.46:0/2215868488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22d29480 0x557a22d29890 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f54a800ac30 tx=0x7f54a800af40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819c4765efb7e454 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54957fa700 1 -- 172.21.15.46:0/2215868488 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a800fe90 con 0x557a22d29480 2024-05-09T04:58:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54957fa700 1 -- 172.21.15.46:0/2215868488 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a8004ce0 con 0x557a22d29480 2024-05-09T04:58:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.348+0000 7f54957fa700 1 -- 172.21.15.46:0/2215868488 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a80186a0 con 0x557a22d29480 2024-05-09T04:58:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.349+0000 7f54c4043b80 1 -- 172.21.15.46:0/2215868488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22d29480 msgr2=0x557a22d29890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.349+0000 7f54c4043b80 1 --2- 172.21.15.46:0/2215868488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22d29480 0x557a22d29890 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f54a800ac30 tx=0x7f54a800af40 comp rx=0 tx=0).stop 2024-05-09T04:58:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.349+0000 7f54c4043b80 1 -- 172.21.15.46:0/2215868488 shutdown_connections 2024-05-09T04:58:18.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.349+0000 7f54c4043b80 1 --2- 172.21.15.46:0/2215868488 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22d29480 0x557a22d29890 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.349+0000 7f54c4043b80 1 --2- 172.21.15.46:0/2215868488 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22c26d80 0x557a22d2f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.349+0000 7f54c4043b80 1 --2- 172.21.15.46:0/2215868488 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22bac520 0x557a22d2f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.349+0000 7f54c4043b80 1 -- 172.21.15.46:0/2215868488 >> 172.21.15.46:0/2215868488 conn(0x557a22bafff0 msgr2=0x557a22d242a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.349+0000 7f54c4043b80 1 -- 172.21.15.46:0/2215868488 shutdown_connections 2024-05-09T04:58:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.349+0000 7f54c4043b80 1 -- 172.21.15.46:0/2215868488 wait complete. 2024-05-09T04:58:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.350+0000 7f54c4043b80 1 -- start start 2024-05-09T04:58:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.350+0000 7f54c4043b80 1 Processor -- start 2024-05-09T04:58:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.350+0000 7f54c4043b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22bac520 0x557a22d33170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.350+0000 7f54c4043b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22c26d80 0x557a22d336b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.350+0000 7f54c4043b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22d29480 0x557a22d33bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.350+0000 7f54c4043b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x557a22d2a1e0 con 0x557a22c26d80 2024-05-09T04:58:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.350+0000 7f54c4043b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x557a22be6ba0 con 0x557a22bac520 2024-05-09T04:58:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.350+0000 7f54c4043b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x557a22d360a0 con 0x557a22d29480 2024-05-09T04:58:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.350+0000 7f54c4043b80 10 client.0 _ll_register_callbacks cb 0x557a22abd740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:18.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.350+0000 7f54ae7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22c26d80 0x557a22d336b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.350+0000 7f54ae7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22c26d80 0x557a22d336b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35604/0 (socket says 172.21.15.46:35604) 2024-05-09T04:58:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.350+0000 7f54ae7fc700 1 -- 172.21.15.46:0/1133493117 learned_addr learned my addr 172.21.15.46:0/1133493117 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f54af7fe700 1 --2- 172.21.15.46:0/1133493117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22d29480 0x557a22d33bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f54aeffd700 1 --2- 172.21.15.46:0/1133493117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22bac520 0x557a22d33170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f54ae7fc700 1 -- 172.21.15.46:0/1133493117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22d29480 msgr2=0x557a22d33bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f54ae7fc700 1 --2- 172.21.15.46:0/1133493117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22d29480 0x557a22d33bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f54ae7fc700 1 -- 172.21.15.46:0/1133493117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22bac520 msgr2=0x557a22d33170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f54ae7fc700 1 --2- 172.21.15.46:0/1133493117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22bac520 0x557a22d33170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f54ae7fc700 1 -- 172.21.15.46:0/1133493117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a22ac7510 con 0x557a22c26d80 2024-05-09T04:58:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f54ae7fc700 1 --2- 172.21.15.46:0/1133493117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22c26d80 0x557a22d336b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f54a40095c0 tx=0x7f54a40aa640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af24dcbe7da00b69 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f5492ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f5492ff5700 1 -- 172.21.15.46:0/1133493117 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a40aaed0 con 0x557a22c26d80 2024-05-09T04:58:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f54c4043b80 1 -- 172.21.15.46:0/1133493117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a8004ce0 con 0x557a22c26d80 2024-05-09T04:58:18.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f5492ff5700 1 -- 172.21.15.46:0/1133493117 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f54a40ab340 con 0x557a22c26d80 2024-05-09T04:58:18.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f5492ff5700 1 -- 172.21.15.46:0/1133493117 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f54a40af280 con 0x557a22c26d80 2024-05-09T04:58:18.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f54c4043b80 10 client.16959 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:18.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f54c4043b80 1 -- 172.21.15.46:0/1133493117 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557a22d40b30 con 0x557a22c26d80 2024-05-09T04:58:18.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.351+0000 7f5492ff5700 1 -- 172.21.15.46:0/1133493117 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a8004ce0 con 0x557a22c26d80 2024-05-09T04:58:18.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.352+0000 7f5492ff5700 1 -- 172.21.15.46:0/1133493117 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f54a4011070 con 0x557a22c26d80 2024-05-09T04:58:18.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.352+0000 7f54c4043b80 10 client.16959 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:18.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.352+0000 7f54c4043b80 10 client.16959 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:58:18.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.352+0000 7f54c4043b80 20 client.16959 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:18.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.352+0000 7f54c4043b80 1 --2- 172.21.15.46:0/1133493117 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557a22d43190 0x557a22d455e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.352+0000 7f54c4043b80 4 client.16959 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:58:18.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.352+0000 7f54c4043b80 1 -- 172.21.15.46:0/1133493117 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a22c59b00 con 0x557a22d43190 2024-05-09T04:58:18.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.352+0000 7f54aeffd700 1 --2- 172.21.15.46:0/1133493117 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557a22d43190 0x557a22d455e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.352+0000 7f5492ff5700 10 client.16959 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:18.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.352+0000 7f54aeffd700 1 --2- 172.21.15.46:0/1133493117 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557a22d43190 0x557a22d455e0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39f24b9e3cbcb60e server_cookie=eac7eb762b4b3512 in_seq=0 out_seq=0 2024-05-09T04:58:18.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.353+0000 7f5492ff5700 1 -- 172.21.15.46:0/1133493117 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x557a22c59b00 con 0x557a22d43190 2024-05-09T04:58:18.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.353+0000 7f5492ff5700 10 client.16959 handle_command_reply: tid=0 2024-05-09T04:58:18.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.353+0000 7f54c4043b80 1 client.16959 shutdown 2024-05-09T04:58:18.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.353+0000 7f54c4043b80 1 -- 172.21.15.46:0/1133493117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22c26d80 msgr2=0x557a22d336b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:18.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.353+0000 7f54c4043b80 1 --2- 172.21.15.46:0/1133493117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22c26d80 0x557a22d336b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f54a40095c0 tx=0x7f54a40aa640 comp rx=0 tx=0).stop 2024-05-09T04:58:18.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.353+0000 7f54c4043b80 1 -- 172.21.15.46:0/1133493117 shutdown_connections 2024-05-09T04:58:18.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.353+0000 7f54c4043b80 1 --2- 172.21.15.46:0/1133493117 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22d29480 0x557a22d33bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.358 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.353+0000 7f54c4043b80 1 --2- 172.21.15.46:0/1133493117 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557a22d43190 0x557a22d455e0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.353+0000 7f54c4043b80 1 --2- 172.21.15.46:0/1133493117 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22bac520 0x557a22d33170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.353+0000 7f54c4043b80 1 --2- 172.21.15.46:0/1133493117 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22c26d80 0x557a22d336b0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.353+0000 7f54c4043b80 1 -- 172.21.15.46:0/1133493117 >> 172.21.15.46:0/1133493117 conn(0x557a22bafff0 msgr2=0x557a22d1e070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:18.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.353+0000 7f5492ff5700 0 client.16959 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:18.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.354+0000 7f54c4043b80 1 -- 172.21.15.46:0/1133493117 shutdown_connections 2024-05-09T04:58:18.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.354+0000 7f54c4043b80 1 -- 172.21.15.46:0/1133493117 wait complete. 2024-05-09T04:58:18.359 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.354+0000 7f54c4043b80 20 client.16959 trim_cache size 0 max 16384 2024-05-09T04:58:18.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.374+0000 7f54c4043b80 1 Processor -- start 2024-05-09T04:58:18.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.374+0000 7f54c4043b80 1 -- start start 2024-05-09T04:58:18.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54c4043b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22bac520 0x557a22d35610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54c4043b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22c26d80 0x557a22d30d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54c4043b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22d29480 0x557a22d28ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54c4043b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01b7d70 con 0x557a22bac520 2024-05-09T04:58:18.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54c4043b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01b5600 con 0x557a22d29480 2024-05-09T04:58:18.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54c4043b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01b3470 con 0x557a22c26d80 2024-05-09T04:58:18.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54aeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22bac520 0x557a22d35610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54aeffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22bac520 0x557a22d35610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35610/0 (socket says 172.21.15.46:35610) 2024-05-09T04:58:18.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54aeffd700 1 -- 172.21.15.46:0/1309581773 learned_addr learned my addr 172.21.15.46:0/1309581773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:18.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54ae7fc700 1 --2- 172.21.15.46:0/1309581773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22c26d80 0x557a22d30d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54aeffd700 1 -- 172.21.15.46:0/1309581773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22c26d80 msgr2=0x557a22d30d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54aeffd700 1 --2- 172.21.15.46:0/1309581773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22c26d80 0x557a22d30d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54aeffd700 1 -- 172.21.15.46:0/1309581773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22d29480 msgr2=0x557a22d28ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54af7fe700 1 --2- 172.21.15.46:0/1309581773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22d29480 0x557a22d28ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54aeffd700 1 --2- 172.21.15.46:0/1309581773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22d29480 0x557a22d28ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54aeffd700 1 -- 172.21.15.46:0/1309581773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a22d247e0 con 0x557a22bac520 2024-05-09T04:58:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54ae7fc700 1 --2- 172.21.15.46:0/1309581773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22c26d80 0x557a22d30d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54af7fe700 1 --2- 172.21.15.46:0/1309581773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22d29480 0x557a22d28ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:58:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.375+0000 7f54aeffd700 1 --2- 172.21.15.46:0/1309581773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22bac520 0x557a22d35610 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f54a000bfe0 tx=0x7f54a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e1e028a8ffb4452 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.376+0000 7f5494ff9700 1 -- 172.21.15.46:0/1309581773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a002e070 con 0x557a22bac520 2024-05-09T04:58:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.376+0000 7f5494ff9700 1 -- 172.21.15.46:0/1309581773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a0020e30 con 0x557a22bac520 2024-05-09T04:58:18.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.376+0000 7f5494ff9700 1 -- 172.21.15.46:0/1309581773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a0011a80 con 0x557a22bac520 2024-05-09T04:58:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.376+0000 7f54c4043b80 1 -- 172.21.15.46:0/1309581773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22bac520 msgr2=0x557a22d35610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.376+0000 7f54c4043b80 1 --2- 172.21.15.46:0/1309581773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22bac520 0x557a22d35610 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f54a000bfe0 tx=0x7f54a0014000 comp rx=0 tx=0).stop 2024-05-09T04:58:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.376+0000 7f54c4043b80 1 -- 172.21.15.46:0/1309581773 shutdown_connections 2024-05-09T04:58:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.376+0000 7f54c4043b80 1 --2- 172.21.15.46:0/1309581773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22d29480 0x557a22d28ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.377+0000 7f54c4043b80 1 --2- 172.21.15.46:0/1309581773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22bac520 0x557a22d35610 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.377+0000 7f54c4043b80 1 --2- 172.21.15.46:0/1309581773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22c26d80 0x557a22d30d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.377+0000 7f54c4043b80 1 -- 172.21.15.46:0/1309581773 >> 172.21.15.46:0/1309581773 conn(0x557a22bafff0 msgr2=0x557a22d1e6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.377+0000 7f54c4043b80 1 -- 172.21.15.46:0/1309581773 shutdown_connections 2024-05-09T04:58:18.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.377+0000 7f54c4043b80 1 -- 172.21.15.46:0/1309581773 wait complete. 2024-05-09T04:58:18.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.377+0000 7f54c4043b80 1 -- start start 2024-05-09T04:58:18.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.377+0000 7f54c4043b80 1 Processor -- start 2024-05-09T04:58:18.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.377+0000 7f54c4043b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22bac520 0x557a22d39550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.377+0000 7f54c4043b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22c26d80 0x557a22d39a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.377+0000 7f54c4043b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22d29480 0x557a22dac980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.377+0000 7f54c4043b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01b3ac0 con 0x557a22d29480 2024-05-09T04:58:18.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.377+0000 7f54c4043b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01b4f30 con 0x557a22c26d80 2024-05-09T04:58:18.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.377+0000 7f54c4043b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f54b01b4190 con 0x557a22bac520 2024-05-09T04:58:18.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f54c4043b80 10 client.0 _ll_register_callbacks cb 0x557a22abd740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:18.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f54af7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22d29480 0x557a22dac980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f54af7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22d29480 0x557a22dac980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35626/0 (socket says 172.21.15.46:35626) 2024-05-09T04:58:18.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f54af7fe700 1 -- 172.21.15.46:0/2039966359 learned_addr learned my addr 172.21.15.46:0/2039966359 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:18.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f54aeffd700 1 --2- 172.21.15.46:0/2039966359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22bac520 0x557a22d39550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f54ae7fc700 1 --2- 172.21.15.46:0/2039966359 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22c26d80 0x557a22d39a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f54af7fe700 1 -- 172.21.15.46:0/2039966359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22bac520 msgr2=0x557a22d39550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:18.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f54af7fe700 1 --2- 172.21.15.46:0/2039966359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22bac520 0x557a22d39550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f54af7fe700 1 -- 172.21.15.46:0/2039966359 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22c26d80 msgr2=0x557a22d39a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:18.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f54af7fe700 1 --2- 172.21.15.46:0/2039966359 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22c26d80 0x557a22d39a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f54af7fe700 1 -- 172.21.15.46:0/2039966359 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a22dacec0 con 0x557a22d29480 2024-05-09T04:58:18.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f54aeffd700 1 --2- 172.21.15.46:0/2039966359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22bac520 0x557a22d39550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:18.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f54af7fe700 1 --2- 172.21.15.46:0/2039966359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22d29480 0x557a22dac980 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f54a801a040 tx=0x7f54a8005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67b76d82d849094 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:18.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f5492ff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:18.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f5492ff5700 1 -- 172.21.15.46:0/2039966359 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54a8027070 con 0x557a22d29480 2024-05-09T04:58:18.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f5492ff5700 1 -- 172.21.15.46:0/2039966359 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f54a8003880 con 0x557a22d29480 2024-05-09T04:58:18.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f54c4043b80 1 -- 172.21.15.46:0/2039966359 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54a0020e30 con 0x557a22d29480 2024-05-09T04:58:18.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f5492ff5700 1 -- 172.21.15.46:0/2039966359 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f54a8033050 con 0x557a22d29480 2024-05-09T04:58:18.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.378+0000 7f54c4043b80 10 client.16965 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.379+0000 7f54c4043b80 1 -- 172.21.15.46:0/2039966359 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x557a22dae030 con 0x557a22d29480 2024-05-09T04:58:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.379+0000 7f5492ff5700 1 -- 172.21.15.46:0/2039966359 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54a0020e30 con 0x557a22d29480 2024-05-09T04:58:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.379+0000 7f5492ff5700 1 -- 172.21.15.46:0/2039966359 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f54a8023b60 con 0x557a22d29480 2024-05-09T04:58:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.379+0000 7f54c4043b80 10 client.16965 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.379+0000 7f54c4043b80 10 client.16965 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:58:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.379+0000 7f54c4043b80 20 client.16965 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.379+0000 7f54c4043b80 1 --2- 172.21.15.46:0/2039966359 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557a22dae490 0x557a22db08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.379+0000 7f54c4043b80 4 client.16965 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:58:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.379+0000 7f54c4043b80 1 -- 172.21.15.46:0/2039966359 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557a22c53ba0 con 0x557a22dae490 2024-05-09T04:58:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.379+0000 7f54aeffd700 1 --2- 172.21.15.46:0/2039966359 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557a22dae490 0x557a22db08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.380+0000 7f54aeffd700 1 --2- 172.21.15.46:0/2039966359 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557a22dae490 0x557a22db08b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ab0f75695b8fa9a7 server_cookie=2c5ced2960397b4d in_seq=0 out_seq=0 2024-05-09T04:58:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.380+0000 7f5492ff5700 10 client.16965 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:18.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.381+0000 7f5492ff5700 1 -- 172.21.15.46:0/2039966359 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557a22c53ba0 con 0x557a22dae490 2024-05-09T04:58:18.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.381+0000 7f5492ff5700 10 client.16965 handle_command_reply: tid=0 2024-05-09T04:58:18.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.381+0000 7f54c4043b80 1 client.16965 shutdown 2024-05-09T04:58:18.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.381+0000 7f54c4043b80 1 -- 172.21.15.46:0/2039966359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22d29480 msgr2=0x557a22dac980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:18.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.381+0000 7f54c4043b80 1 --2- 172.21.15.46:0/2039966359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22d29480 0x557a22dac980 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f54a801a040 tx=0x7f54a8005400 comp rx=0 tx=0).stop 2024-05-09T04:58:18.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.381+0000 7f54c4043b80 1 -- 172.21.15.46:0/2039966359 shutdown_connections 2024-05-09T04:58:18.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.381+0000 7f54c4043b80 1 --2- 172.21.15.46:0/2039966359 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x557a22d29480 0x557a22dac980 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.381+0000 7f54c4043b80 1 --2- 172.21.15.46:0/2039966359 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x557a22bac520 0x557a22d39550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.381+0000 7f54c4043b80 1 --2- 172.21.15.46:0/2039966359 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x557a22dae490 0x557a22db08b0 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.381+0000 7f54c4043b80 1 --2- 172.21.15.46:0/2039966359 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x557a22c26d80 0x557a22d39a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.381+0000 7f5492ff5700 0 client.16965 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:18.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.381+0000 7f54c4043b80 1 -- 172.21.15.46:0/2039966359 >> 172.21.15.46:0/2039966359 conn(0x557a22bafff0 msgr2=0x557a22d2cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:18.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.381+0000 7f54c4043b80 1 -- 172.21.15.46:0/2039966359 shutdown_connections 2024-05-09T04:58:18.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.381+0000 7f54c4043b80 1 -- 172.21.15.46:0/2039966359 wait complete. 2024-05-09T04:58:18.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.381+0000 7f54c4043b80 20 client.16965 trim_cache size 0 max 16384 2024-05-09T04:58:18.388 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:58:18.388 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:58:18.388 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "9873c28e-17e5-4103-b9ad-33e7135e028d", 2024-05-09T04:58:18.388 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:58:18.388 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:58:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.382+0000 7f54b562d700 1 -- 172.21.15.46:0/4188374471 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f54980b2a70 msgr2=0x7f54980b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.382+0000 7f54b562d700 1 --2- 172.21.15.46:0/4188374471 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f54980b2a70 0x7f54980b4f80 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f54a0005950 tx=0x7f54a00058e0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.382+0000 7f54b562d700 1 -- 172.21.15.46:0/4188374471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b0105dd0 msgr2=0x7f54b01a2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.382+0000 7f54b562d700 1 --2- 172.21.15.46:0/4188374471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b0105dd0 0x7f54b01a2670 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f54a400b070 tx=0x7f54a4004d00 comp rx=0 tx=0).stop 2024-05-09T04:58:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.382+0000 7f54b562d700 1 -- 172.21.15.46:0/4188374471 shutdown_connections 2024-05-09T04:58:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.382+0000 7f54b562d700 1 --2- 172.21.15.46:0/4188374471 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f54980b2a70 0x7f54980b4f80 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.382+0000 7f54b562d700 1 --2- 172.21.15.46:0/4188374471 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f54b010e8f0 0x7f54b01ad1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.382+0000 7f54b562d700 1 --2- 172.21.15.46:0/4188374471 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f54b0105480 0x7f54b01a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.382+0000 7f54b562d700 1 --2- 172.21.15.46:0/4188374471 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f54b0105dd0 0x7f54b01a2670 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:18.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.382+0000 7f54b562d700 1 -- 172.21.15.46:0/4188374471 >> 172.21.15.46:0/4188374471 conn(0x7f54b00fef30 msgr2=0x7f54b0100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:18.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.382+0000 7f54b562d700 1 -- 172.21.15.46:0/4188374471 shutdown_connections 2024-05-09T04:58:18.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:18.382+0000 7f54b562d700 1 -- 172.21.15.46:0/4188374471 wait complete. 2024-05-09T04:58:19.057 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:18 smithi046 ceph-mon[30136]: pgmap v221: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T04:58:19.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:18 smithi110 ceph-mon[34904]: pgmap v221: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T04:58:19.088 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:58:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:18 smithi072 ceph-mon[34088]: pgmap v221: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T04:58:19.341 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:20.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:19 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:58:20.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:19 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:58:20.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:19 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:58:20.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:19 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:58:20.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:19 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:58:20.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:19 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:58:20.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:19 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:58:20.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:19 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:58:20.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:19 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:58:20.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:19 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:58:20.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:19 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:58:20.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:19 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:58:21.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:20 smithi046 ceph-mon[30136]: pgmap v222: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T04:58:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:20 smithi072 ceph-mon[34088]: pgmap v222: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T04:58:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.141+0000 7f2f65e25700 1 -- 172.21.15.46:0/3663267401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f60109860 msgr2=0x7f2f601111c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.141+0000 7f2f65e25700 1 --2- 172.21.15.46:0/3663267401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f60109860 0x7f2f601111c0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800b0d0 tx=0x7f2f5800b3e0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.141+0000 7f2f65e25700 1 -- 172.21.15.46:0/3663267401 shutdown_connections 2024-05-09T04:58:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.141+0000 7f2f65e25700 1 --2- 172.21.15.46:0/3663267401 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f60109860 0x7f2f601111c0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.141+0000 7f2f65e25700 1 --2- 172.21.15.46:0/3663267401 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f601045e0 0x7f2f601049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.141+0000 7f2f65e25700 1 --2- 172.21.15.46:0/3663267401 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f60108f10 0x7f2f60109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.141+0000 7f2f65e25700 1 -- 172.21.15.46:0/3663267401 >> 172.21.15.46:0/3663267401 conn(0x7f2f600ffed0 msgr2=0x7f2f60102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.141+0000 7f2f65e25700 1 -- 172.21.15.46:0/3663267401 shutdown_connections 2024-05-09T04:58:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.141+0000 7f2f65e25700 1 -- 172.21.15.46:0/3663267401 wait complete. 2024-05-09T04:58:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.142+0000 7f2f65e25700 1 Processor -- start 2024-05-09T04:58:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.142+0000 7f2f65e25700 1 -- start start 2024-05-09T04:58:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.142+0000 7f2f65e25700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f601045e0 0x7f2f6019ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.142+0000 7f2f65e25700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f60108f10 0x7f2f6019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.142+0000 7f2f65e25700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f60109860 0x7f2f6019f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.142+0000 7f2f65e25700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2f60114490 con 0x7f2f60109860 2024-05-09T04:58:21.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.142+0000 7f2f65e25700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2f60114310 con 0x7f2f60108f10 2024-05-09T04:58:21.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.142+0000 7f2f65e25700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2f60114610 con 0x7f2f601045e0 2024-05-09T04:58:21.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.142+0000 7f2f5ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f60109860 0x7f2f6019f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.142+0000 7f2f5f7fe700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f601045e0 0x7f2f6019ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.142+0000 7f2f5ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f60109860 0x7f2f6019f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35640/0 (socket says 172.21.15.46:35640) 2024-05-09T04:58:21.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.142+0000 7f2f5ffff700 1 -- 172.21.15.46:0/3189200390 learned_addr learned my addr 172.21.15.46:0/3189200390 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:21.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.143+0000 7f2f5effd700 1 --2- 172.21.15.46:0/3189200390 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f60108f10 0x7f2f6019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.143+0000 7f2f5f7fe700 1 -- 172.21.15.46:0/3189200390 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f60108f10 msgr2=0x7f2f6019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:21.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.143+0000 7f2f5f7fe700 1 --2- 172.21.15.46:0/3189200390 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f60108f10 0x7f2f6019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.143+0000 7f2f5f7fe700 1 -- 172.21.15.46:0/3189200390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f60109860 msgr2=0x7f2f6019f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:21.145 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.143+0000 7f2f5f7fe700 1 --2- 172.21.15.46:0/3189200390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f60109860 0x7f2f6019f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.143+0000 7f2f5f7fe700 1 -- 172.21.15.46:0/3189200390 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6019fc50 con 0x7f2f601045e0 2024-05-09T04:58:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.143+0000 7f2f5effd700 1 --2- 172.21.15.46:0/3189200390 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f60108f10 0x7f2f6019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.143+0000 7f2f5ffff700 1 --2- 172.21.15.46:0/3189200390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f60109860 0x7f2f6019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:58:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.143+0000 7f2f5f7fe700 1 --2- 172.21.15.46:0/3189200390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f601045e0 0x7f2f6019ad80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000d750 tx=0x7f2f5000da60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.143+0000 7f2f5cff9700 1 -- 172.21.15.46:0/3189200390 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f50016070 con 0x7f2f601045e0 2024-05-09T04:58:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.143+0000 7f2f65e25700 1 -- 172.21.15.46:0/3189200390 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6019ff40 con 0x7f2f601045e0 2024-05-09T04:58:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.143+0000 7f2f65e25700 1 -- 172.21.15.46:0/3189200390 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f601a7860 con 0x7f2f601045e0 2024-05-09T04:58:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.143+0000 7f2f5cff9700 1 -- 172.21.15.46:0/3189200390 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f5000f690 con 0x7f2f601045e0 2024-05-09T04:58:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.143+0000 7f2f5cff9700 1 -- 172.21.15.46:0/3189200390 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f500038a0 con 0x7f2f601045e0 2024-05-09T04:58:21.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.146+0000 7f2f5cff9700 1 -- 172.21.15.46:0/3189200390 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2f50003aa0 con 0x7f2f601045e0 2024-05-09T04:58:21.148 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.147+0000 7f2f7483bb80 1 Processor -- start 2024-05-09T04:58:21.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.147+0000 7f2f7483bb80 1 -- start start 2024-05-09T04:58:21.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.147+0000 7f2f5cff9700 1 --2- 172.21.15.46:0/3189200390 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f2f480b29f0 0x7f2f480b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.147+0000 7f2f5ffff700 1 --2- 172.21.15.46:0/3189200390 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f2f480b29f0 0x7f2f480b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.147+0000 7f2f5cff9700 1 -- 172.21.15.46:0/3189200390 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f2f500e2410 con 0x7f2f601045e0 2024-05-09T04:58:21.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.147+0000 7f2f7483bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eec19d00 0x5574eed20ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.147+0000 7f2f7483bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eeb9f4a0 0x5574eed21030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.147+0000 7f2f7483bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eed251e0 0x5574eed255f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.147+0000 7f2f7483bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5574eec3f0a0 con 0x5574eeb9f4a0 2024-05-09T04:58:21.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.147+0000 7f2f7483bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5574eebe49a0 con 0x5574eec19d00 2024-05-09T04:58:21.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.147+0000 7f2f7483bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5574eec3b060 con 0x5574eed251e0 2024-05-09T04:58:21.149 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.147+0000 7f2f5effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eeb9f4a0 0x5574eed21030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.148+0000 7f2f5effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eeb9f4a0 0x5574eed21030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35646/0 (socket says 172.21.15.46:35646) 2024-05-09T04:58:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.148+0000 7f2f5effd700 1 -- 172.21.15.46:0/4028616239 learned_addr learned my addr 172.21.15.46:0/4028616239 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.148+0000 7f2f5f7fe700 1 --2- 172.21.15.46:0/4028616239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eec19d00 0x5574eed20ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.148+0000 7f2f5ffff700 1 --2- 172.21.15.46:0/4028616239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eed251e0 0x5574eed255f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.148+0000 7f2f5effd700 1 -- 172.21.15.46:0/4028616239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eed251e0 msgr2=0x5574eed255f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.148+0000 7f2f5effd700 1 --2- 172.21.15.46:0/4028616239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eed251e0 0x5574eed255f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.148+0000 7f2f5effd700 1 -- 172.21.15.46:0/4028616239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eec19d00 msgr2=0x5574eed20ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.148+0000 7f2f5effd700 1 --2- 172.21.15.46:0/4028616239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eec19d00 0x5574eed20ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.148+0000 7f2f5effd700 1 -- 172.21.15.46:0/4028616239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574eebd5500 con 0x5574eeb9f4a0 2024-05-09T04:58:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.148+0000 7f2f5ffff700 1 --2- 172.21.15.46:0/3189200390 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f2f480b29f0 0x7f2f480b4f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800bfd0 tx=0x7f2f5800b040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.148+0000 7f2f5effd700 1 --2- 172.21.15.46:0/4028616239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eeb9f4a0 0x5574eed21030 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f2f54009d20 tx=0x7f2f540106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01f16f571bf5760 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.148+0000 7f2f45ffb700 1 -- 172.21.15.46:0/4028616239 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54010e10 con 0x5574eeb9f4a0 2024-05-09T04:58:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.148+0000 7f2f45ffb700 1 -- 172.21.15.46:0/4028616239 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f540044d0 con 0x5574eeb9f4a0 2024-05-09T04:58:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.148+0000 7f2f45ffb700 1 -- 172.21.15.46:0/4028616239 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54011440 con 0x5574eeb9f4a0 2024-05-09T04:58:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.149+0000 7f2f7483bb80 1 -- 172.21.15.46:0/4028616239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eeb9f4a0 msgr2=0x5574eed21030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.149+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/4028616239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eeb9f4a0 0x5574eed21030 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f2f54009d20 tx=0x7f2f540106b0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.149+0000 7f2f7483bb80 1 -- 172.21.15.46:0/4028616239 shutdown_connections 2024-05-09T04:58:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.149+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/4028616239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eed251e0 0x5574eed255f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.149+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/4028616239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eec19d00 0x5574eed20ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.149+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/4028616239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eeb9f4a0 0x5574eed21030 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.149+0000 7f2f7483bb80 1 -- 172.21.15.46:0/4028616239 >> 172.21.15.46:0/4028616239 conn(0x5574eeba2f70 msgr2=0x5574eed17220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.149+0000 7f2f7483bb80 1 -- 172.21.15.46:0/4028616239 shutdown_connections 2024-05-09T04:58:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.149+0000 7f2f7483bb80 1 -- 172.21.15.46:0/4028616239 wait complete. 2024-05-09T04:58:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.150+0000 7f2f7483bb80 1 -- start start 2024-05-09T04:58:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.150+0000 7f2f7483bb80 1 Processor -- start 2024-05-09T04:58:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.150+0000 7f2f7483bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eeb9f4a0 0x5574eed2a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.150+0000 7f2f7483bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eec19d00 0x5574eed2a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.150+0000 7f2f7483bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eed251e0 0x5574eed2aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.150+0000 7f2f7483bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5574eed25f00 con 0x5574eec19d00 2024-05-09T04:58:21.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.150+0000 7f2f7483bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5574eebd9b20 con 0x5574eed251e0 2024-05-09T04:58:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.150+0000 7f2f7483bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5574eed2d470 con 0x5574eeb9f4a0 2024-05-09T04:58:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.150+0000 7f2f7483bb80 10 client.0 _ll_register_callbacks cb 0x5574eeab06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.150+0000 7f2f5effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eec19d00 0x5574eed2a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.150+0000 7f2f5effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eec19d00 0x5574eed2a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35662/0 (socket says 172.21.15.46:35662) 2024-05-09T04:58:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.150+0000 7f2f5effd700 1 -- 172.21.15.46:0/1109094985 learned_addr learned my addr 172.21.15.46:0/1109094985 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.150+0000 7f2f5f7fe700 1 --2- 172.21.15.46:0/1109094985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eeb9f4a0 0x5574eed2a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.151+0000 7f2f5ffff700 1 --2- 172.21.15.46:0/1109094985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eed251e0 0x5574eed2aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.151+0000 7f2f5effd700 1 -- 172.21.15.46:0/1109094985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eeb9f4a0 msgr2=0x5574eed2a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.151+0000 7f2f5effd700 1 --2- 172.21.15.46:0/1109094985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eeb9f4a0 0x5574eed2a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.151+0000 7f2f5effd700 1 -- 172.21.15.46:0/1109094985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eed251e0 msgr2=0x5574eed2aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.151+0000 7f2f5effd700 1 --2- 172.21.15.46:0/1109094985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eed251e0 0x5574eed2aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.151+0000 7f2f5effd700 1 -- 172.21.15.46:0/1109094985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574eeaba490 con 0x5574eec19d00 2024-05-09T04:58:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.151+0000 7f2f5effd700 1 --2- 172.21.15.46:0/1109094985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eec19d00 0x5574eed2a960 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2f540046e0 tx=0x7f2f54009e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b640150b7c9d49 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.151+0000 7f2f3effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.151+0000 7f2f3effd700 1 -- 172.21.15.46:0/1109094985 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f54029060 con 0x5574eec19d00 2024-05-09T04:58:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.151+0000 7f2f7483bb80 1 -- 172.21.15.46:0/1109094985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f540044d0 con 0x5574eec19d00 2024-05-09T04:58:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.151+0000 7f2f3effd700 1 -- 172.21.15.46:0/1109094985 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f2f54005250 con 0x5574eec19d00 2024-05-09T04:58:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.152+0000 7f2f3effd700 1 -- 172.21.15.46:0/1109094985 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2f54005450 con 0x5574eec19d00 2024-05-09T04:58:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.152+0000 7f2f7483bb80 10 client.16977 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.152+0000 7f2f7483bb80 1 -- 172.21.15.46:0/1109094985 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5574eed37e00 con 0x5574eec19d00 2024-05-09T04:58:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.152+0000 7f2f3effd700 1 -- 172.21.15.46:0/1109094985 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f540044d0 con 0x5574eec19d00 2024-05-09T04:58:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.152+0000 7f2f3effd700 1 -- 172.21.15.46:0/1109094985 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2f54003e10 con 0x5574eec19d00 2024-05-09T04:58:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.152+0000 7f2f7483bb80 10 client.16977 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:21.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.152+0000 7f2f7483bb80 10 client.16977 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:58:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.152+0000 7f2f7483bb80 20 client.16977 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.152+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/1109094985 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5574eed3a2d0 0x5574eed3c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.152+0000 7f2f7483bb80 4 client.16977 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:58:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.152+0000 7f2f7483bb80 1 -- 172.21.15.46:0/1109094985 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574eec4ca80 con 0x5574eed3a2d0 2024-05-09T04:58:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.152+0000 7f2f5f7fe700 1 --2- 172.21.15.46:0/1109094985 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5574eed3a2d0 0x5574eed3c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.153+0000 7f2f3effd700 10 client.16977 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.153+0000 7f2f5f7fe700 1 --2- 172.21.15.46:0/1109094985 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5574eed3a2d0 0x5574eed3c6f0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=946c0a05b05e5b86 server_cookie=250471a87972bd7a in_seq=0 out_seq=0 2024-05-09T04:58:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.153+0000 7f2f3effd700 1 -- 172.21.15.46:0/1109094985 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5574eec4ca80 con 0x5574eed3a2d0 2024-05-09T04:58:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.153+0000 7f2f3effd700 10 client.16977 handle_command_reply: tid=0 2024-05-09T04:58:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.153+0000 7f2f7483bb80 1 client.16977 shutdown 2024-05-09T04:58:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.154+0000 7f2f7483bb80 1 -- 172.21.15.46:0/1109094985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eec19d00 msgr2=0x5574eed2a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.154+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/1109094985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eec19d00 0x5574eed2a960 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f2f540046e0 tx=0x7f2f54009e70 comp rx=0 tx=0).stop 2024-05-09T04:58:21.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.154+0000 7f2f7483bb80 1 -- 172.21.15.46:0/1109094985 shutdown_connections 2024-05-09T04:58:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.154+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/1109094985 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5574eed3a2d0 0x5574eed3c6f0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.154+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/1109094985 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eed251e0 0x5574eed2aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.154+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/1109094985 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eeb9f4a0 0x5574eed2a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.154+0000 7f2f3effd700 0 client.16977 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.154+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/1109094985 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eec19d00 0x5574eed2a960 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.154+0000 7f2f7483bb80 1 -- 172.21.15.46:0/1109094985 >> 172.21.15.46:0/1109094985 conn(0x5574eeba2f70 msgr2=0x5574eed10ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.154+0000 7f2f7483bb80 1 -- 172.21.15.46:0/1109094985 shutdown_connections 2024-05-09T04:58:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.154+0000 7f2f7483bb80 1 -- 172.21.15.46:0/1109094985 wait complete. 2024-05-09T04:58:21.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.154+0000 7f2f7483bb80 20 client.16977 trim_cache size 0 max 16384 2024-05-09T04:58:21.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.174+0000 7f2f7483bb80 1 Processor -- start 2024-05-09T04:58:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.174+0000 7f2f7483bb80 1 -- start start 2024-05-09T04:58:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.174+0000 7f2f7483bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eeb9f4a0 0x5574eed11a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.174+0000 7f2f7483bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eec19d00 0x5574eed29710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f7483bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eed251e0 0x5574eed365b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f7483bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c00ffd0 con 0x5574eeb9f4a0 2024-05-09T04:58:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f7483bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c00d860 con 0x5574eec19d00 2024-05-09T04:58:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f7483bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c00b6d0 con 0x5574eed251e0 2024-05-09T04:58:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f5f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eeb9f4a0 0x5574eed11a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f5f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eeb9f4a0 0x5574eed11a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35664/0 (socket says 172.21.15.46:35664) 2024-05-09T04:58:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f5f7fe700 1 -- 172.21.15.46:0/3667459551 learned_addr learned my addr 172.21.15.46:0/3667459551 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:21.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f5effd700 1 --2- 172.21.15.46:0/3667459551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eec19d00 0x5574eed29710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f5f7fe700 1 -- 172.21.15.46:0/3667459551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eed251e0 msgr2=0x5574eed365b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f5ffff700 1 --2- 172.21.15.46:0/3667459551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eed251e0 0x5574eed365b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f5f7fe700 1 --2- 172.21.15.46:0/3667459551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eed251e0 0x5574eed365b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f5f7fe700 1 -- 172.21.15.46:0/3667459551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eec19d00 msgr2=0x5574eed29710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f5f7fe700 1 --2- 172.21.15.46:0/3667459551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eec19d00 0x5574eed29710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f5f7fe700 1 -- 172.21.15.46:0/3667459551 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574eed28410 con 0x5574eeb9f4a0 2024-05-09T04:58:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f5effd700 1 --2- 172.21.15.46:0/3667459551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eec19d00 0x5574eed29710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:58:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f5ffff700 1 --2- 172.21.15.46:0/3667459551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eed251e0 0x5574eed365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:58:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f5f7fe700 1 --2- 172.21.15.46:0/3667459551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eeb9f4a0 0x5574eed11a20 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2f500a95d0 tx=0x7f2f50091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bac8359c717713e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f457fa700 1 -- 172.21.15.46:0/3667459551 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f500b24a0 con 0x5574eeb9f4a0 2024-05-09T04:58:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f457fa700 1 -- 172.21.15.46:0/3667459551 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f50003d90 con 0x5574eeb9f4a0 2024-05-09T04:58:21.183 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.175+0000 7f2f457fa700 1 -- 172.21.15.46:0/3667459551 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f500b2270 con 0x5574eeb9f4a0 2024-05-09T04:58:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.176+0000 7f2f7483bb80 1 -- 172.21.15.46:0/3667459551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eeb9f4a0 msgr2=0x5574eed11a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.176+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/3667459551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eeb9f4a0 0x5574eed11a20 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f2f500a95d0 tx=0x7f2f50091040 comp rx=0 tx=0).stop 2024-05-09T04:58:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.176+0000 7f2f7483bb80 1 -- 172.21.15.46:0/3667459551 shutdown_connections 2024-05-09T04:58:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.176+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/3667459551 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eed251e0 0x5574eed365b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.176+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/3667459551 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eeb9f4a0 0x5574eed11a20 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.176+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/3667459551 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eec19d00 0x5574eed29710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.176+0000 7f2f7483bb80 1 -- 172.21.15.46:0/3667459551 >> 172.21.15.46:0/3667459551 conn(0x5574eeba2f70 msgr2=0x5574eed1b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.176+0000 7f2f7483bb80 1 -- 172.21.15.46:0/3667459551 shutdown_connections 2024-05-09T04:58:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.176+0000 7f2f7483bb80 1 -- 172.21.15.46:0/3667459551 wait complete. 2024-05-09T04:58:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.177+0000 7f2f7483bb80 1 -- start start 2024-05-09T04:58:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.177+0000 7f2f7483bb80 1 Processor -- start 2024-05-09T04:58:21.184 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.177+0000 7f2f7483bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eeb9f4a0 0x5574eed3b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.177+0000 7f2f7483bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eec19d00 0x5574eed3b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.177+0000 7f2f7483bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eed251e0 0x5574eed3bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.177+0000 7f2f7483bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c00bd20 con 0x5574eed251e0 2024-05-09T04:58:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.177+0000 7f2f7483bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c00d190 con 0x5574eeb9f4a0 2024-05-09T04:58:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.177+0000 7f2f7483bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4c00c3f0 con 0x5574eec19d00 2024-05-09T04:58:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.177+0000 7f2f7483bb80 10 client.0 _ll_register_callbacks cb 0x5574eeab06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f5ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eed251e0 0x5574eed3bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f5ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eed251e0 0x5574eed3bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:35678/0 (socket says 172.21.15.46:35678) 2024-05-09T04:58:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f5ffff700 1 -- 172.21.15.46:0/2966468720 learned_addr learned my addr 172.21.15.46:0/2966468720 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f5effd700 1 --2- 172.21.15.46:0/2966468720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eec19d00 0x5574eed3b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f5f7fe700 1 --2- 172.21.15.46:0/2966468720 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eeb9f4a0 0x5574eed3b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.185 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f5ffff700 1 -- 172.21.15.46:0/2966468720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eec19d00 msgr2=0x5574eed3b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f5ffff700 1 --2- 172.21.15.46:0/2966468720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eec19d00 0x5574eed3b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f5ffff700 1 -- 172.21.15.46:0/2966468720 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eeb9f4a0 msgr2=0x5574eed3b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f5ffff700 1 --2- 172.21.15.46:0/2966468720 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eeb9f4a0 0x5574eed3b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f5ffff700 1 -- 172.21.15.46:0/2966468720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574eeda3940 con 0x5574eed251e0 2024-05-09T04:58:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f5effd700 1 --2- 172.21.15.46:0/2966468720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eec19d00 0x5574eed3b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:58:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f5ffff700 1 --2- 172.21.15.46:0/2966468720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eed251e0 0x5574eed3bef0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2f58009fd0 tx=0x7f2f5800b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a065e56c85aed3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f3effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f3effd700 1 -- 172.21.15.46:0/2966468720 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f58029440 con 0x5574eed251e0 2024-05-09T04:58:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f3effd700 1 -- 172.21.15.46:0/2966468720 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f2f580295e0 con 0x5574eed251e0 2024-05-09T04:58:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.178+0000 7f2f7483bb80 1 -- 172.21.15.46:0/2966468720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f50003d90 con 0x5574eed251e0 2024-05-09T04:58:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.179+0000 7f2f3effd700 1 -- 172.21.15.46:0/2966468720 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2f580297e0 con 0x5574eed251e0 2024-05-09T04:58:21.186 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.179+0000 7f2f7483bb80 10 client.16983 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.179+0000 7f2f7483bb80 1 -- 172.21.15.46:0/2966468720 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5574eeda4a90 con 0x5574eed251e0 2024-05-09T04:58:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.179+0000 7f2f3effd700 1 -- 172.21.15.46:0/2966468720 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f50003d90 con 0x5574eed251e0 2024-05-09T04:58:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.179+0000 7f2f3effd700 1 -- 172.21.15.46:0/2966468720 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2f58011070 con 0x5574eed251e0 2024-05-09T04:58:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.179+0000 7f2f7483bb80 10 client.16983 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.179+0000 7f2f7483bb80 10 client.16983 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:58:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.179+0000 7f2f7483bb80 20 client.16983 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.179+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/2966468720 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5574eeda51f0 0x5574eeda7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.179+0000 7f2f7483bb80 4 client.16983 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:58:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.179+0000 7f2f7483bb80 1 -- 172.21.15.46:0/2966468720 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574eec46b20 con 0x5574eeda51f0 2024-05-09T04:58:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.179+0000 7f2f5f7fe700 1 --2- 172.21.15.46:0/2966468720 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5574eeda51f0 0x5574eeda7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f5f7fe700 1 --2- 172.21.15.46:0/2966468720 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5574eeda51f0 0x5574eeda7610 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6eb52ec4b78cd47 server_cookie=94ac1fb60dfdb0a7 in_seq=0 out_seq=0 2024-05-09T04:58:21.187 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f3effd700 10 client.16983 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f3effd700 1 -- 172.21.15.46:0/2966468720 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5574eec46b20 con 0x5574eeda51f0 2024-05-09T04:58:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f3effd700 10 client.16983 handle_command_reply: tid=0 2024-05-09T04:58:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f7483bb80 1 client.16983 shutdown 2024-05-09T04:58:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f7483bb80 1 -- 172.21.15.46:0/2966468720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eed251e0 msgr2=0x5574eed3bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/2966468720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eed251e0 0x5574eed3bef0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f2f58009fd0 tx=0x7f2f5800b3e0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f7483bb80 1 -- 172.21.15.46:0/2966468720 shutdown_connections 2024-05-09T04:58:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/2966468720 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5574eed251e0 0x5574eed3bef0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/2966468720 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5574eeda51f0 0x5574eeda7610 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/2966468720 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5574eeb9f4a0 0x5574eed3b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f7483bb80 1 --2- 172.21.15.46:0/2966468720 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5574eec19d00 0x5574eed3b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f7483bb80 1 -- 172.21.15.46:0/2966468720 >> 172.21.15.46:0/2966468720 conn(0x5574eeba2f70 msgr2=0x5574eed11780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f3effd700 0 client.16983 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:21.188 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f7483bb80 1 -- 172.21.15.46:0/2966468720 shutdown_connections 2024-05-09T04:58:21.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f7483bb80 1 -- 172.21.15.46:0/2966468720 wait complete. 2024-05-09T04:58:21.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.180+0000 7f2f7483bb80 20 client.16983 trim_cache size 0 max 16384 2024-05-09T04:58:21.189 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:58:21.189 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:58:21.189 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:58:21.189 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:58:21.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.181+0000 7f2f65e25700 1 -- 172.21.15.46:0/3189200390 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f2f480b29f0 msgr2=0x7f2f480b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:21.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.181+0000 7f2f65e25700 1 --2- 172.21.15.46:0/3189200390 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f2f480b29f0 0x7f2f480b4f00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f2f5800bfd0 tx=0x7f2f5800b040 comp rx=0 tx=0).stop 2024-05-09T04:58:21.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.181+0000 7f2f65e25700 1 -- 172.21.15.46:0/3189200390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f601045e0 msgr2=0x7f2f6019ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:21.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.181+0000 7f2f65e25700 1 --2- 172.21.15.46:0/3189200390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f601045e0 0x7f2f6019ad80 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2f5000d750 tx=0x7f2f5000da60 comp rx=0 tx=0).stop 2024-05-09T04:58:21.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.181+0000 7f2f65e25700 1 -- 172.21.15.46:0/3189200390 shutdown_connections 2024-05-09T04:58:21.190 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.181+0000 7f2f65e25700 1 --2- 172.21.15.46:0/3189200390 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f2f480b29f0 0x7f2f480b4f00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.181+0000 7f2f65e25700 1 --2- 172.21.15.46:0/3189200390 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2f60109860 0x7f2f6019f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.181+0000 7f2f65e25700 1 --2- 172.21.15.46:0/3189200390 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2f601045e0 0x7f2f6019ad80 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.181+0000 7f2f65e25700 1 --2- 172.21.15.46:0/3189200390 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2f60108f10 0x7f2f6019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.181+0000 7f2f65e25700 1 -- 172.21.15.46:0/3189200390 >> 172.21.15.46:0/3189200390 conn(0x7f2f600ffed0 msgr2=0x7f2f60108650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.181+0000 7f2f65e25700 1 -- 172.21.15.46:0/3189200390 shutdown_connections 2024-05-09T04:58:21.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:21.181+0000 7f2f65e25700 1 -- 172.21.15.46:0/3189200390 wait complete. 2024-05-09T04:58:21.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:20 smithi110 ceph-mon[34904]: pgmap v222: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T04:58:22.028 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:58:22.028 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:58:22.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:21 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:58:22.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:21 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:58:22.290 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:22.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:21 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:58:23.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:22 smithi046 ceph-mon[30136]: pgmap v223: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6 op/s 2024-05-09T04:58:23.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:22 smithi072 ceph-mon[34088]: pgmap v223: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6 op/s 2024-05-09T04:58:23.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:22 smithi110 ceph-mon[34904]: pgmap v223: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 6 op/s 2024-05-09T04:58:24.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:23 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:58:24.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:23 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:58:24.055 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:23 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:58:24.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:23 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:58:24.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:23 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:58:24.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:23 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:58:24.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.171+0000 7f1364a11700 1 -- 172.21.15.46:0/2364367939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1360107df0 msgr2=0x7f1360108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:24.171 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.171+0000 7f1364a11700 1 --2- 172.21.15.46:0/2364367939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1360107df0 0x7f1360108200 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f13540099c0 tx=0x7f1354009cd0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.171+0000 7f1364a11700 1 -- 172.21.15.46:0/2364367939 shutdown_connections 2024-05-09T04:58:24.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.171+0000 7f1364a11700 1 --2- 172.21.15.46:0/2364367939 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1360108740 0x7f13601100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.171+0000 7f1364a11700 1 --2- 172.21.15.46:0/2364367939 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13601034c0 0x7f13601038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.171+0000 7f1364a11700 1 --2- 172.21.15.46:0/2364367939 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1360107df0 0x7f1360108200 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.171+0000 7f1364a11700 1 -- 172.21.15.46:0/2364367939 >> 172.21.15.46:0/2364367939 conn(0x7f13600fedc0 msgr2=0x7f13601011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:24.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.171+0000 7f1364a11700 1 -- 172.21.15.46:0/2364367939 shutdown_connections 2024-05-09T04:58:24.172 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.171+0000 7f1364a11700 1 -- 172.21.15.46:0/2364367939 wait complete. 2024-05-09T04:58:24.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.172+0000 7f1364a11700 1 Processor -- start 2024-05-09T04:58:24.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.172+0000 7f1364a11700 1 -- start start 2024-05-09T04:58:24.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.172+0000 7f1364a11700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13601034c0 0x7f136019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:24.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.172+0000 7f1364a11700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1360107df0 0x7f136019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:24.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.172+0000 7f1364a11700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1360108740 0x7f136019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:24.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.172+0000 7f1364a11700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1360113350 con 0x7f1360108740 2024-05-09T04:58:24.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.172+0000 7f1364a11700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f13601131d0 con 0x7f13601034c0 2024-05-09T04:58:24.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.172+0000 7f1364a11700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f13601134d0 con 0x7f1360107df0 2024-05-09T04:58:24.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.172+0000 7f135ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1360108740 0x7f136019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:24.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.173+0000 7f135ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1360108740 0x7f136019c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55704/0 (socket says 172.21.15.46:55704) 2024-05-09T04:58:24.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.173+0000 7f135ed9d700 1 -- 172.21.15.46:0/2723949446 learned_addr learned my addr 172.21.15.46:0/2723949446 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:24.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.173+0000 7f135e59c700 1 --2- 172.21.15.46:0/2723949446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13601034c0 0x7f136019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:24.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.173+0000 7f135dd9b700 1 --2- 172.21.15.46:0/2723949446 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1360107df0 0x7f136019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:24.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.173+0000 7f135ed9d700 1 -- 172.21.15.46:0/2723949446 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1360107df0 msgr2=0x7f136019c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:24.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.173+0000 7f135ed9d700 1 --2- 172.21.15.46:0/2723949446 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1360107df0 0x7f136019c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.173+0000 7f135ed9d700 1 -- 172.21.15.46:0/2723949446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13601034c0 msgr2=0x7f136019bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:24.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.173+0000 7f135ed9d700 1 --2- 172.21.15.46:0/2723949446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13601034c0 0x7f136019bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.173+0000 7f135ed9d700 1 -- 172.21.15.46:0/2723949446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13601a6f80 con 0x7f1360108740 2024-05-09T04:58:24.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.173+0000 7f135ed9d700 1 --2- 172.21.15.46:0/2723949446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1360108740 0x7f136019c6d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f135800ef30 tx=0x7f135800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:24.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.173+0000 7f134f7fe700 1 -- 172.21.15.46:0/2723949446 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13580092e0 con 0x7f1360108740 2024-05-09T04:58:24.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.173+0000 7f1364a11700 1 -- 172.21.15.46:0/2723949446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13601a7210 con 0x7f1360108740 2024-05-09T04:58:24.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.173+0000 7f134f7fe700 1 -- 172.21.15.46:0/2723949446 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1358004bb0 con 0x7f1360108740 2024-05-09T04:58:24.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.173+0000 7f134f7fe700 1 -- 172.21.15.46:0/2723949446 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1358004020 con 0x7f1360108740 2024-05-09T04:58:24.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.173+0000 7f1364a11700 1 -- 172.21.15.46:0/2723949446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13601a7720 con 0x7f1360108740 2024-05-09T04:58:24.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.175+0000 7f1364a11700 1 -- 172.21.15.46:0/2723949446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13600512c0 con 0x7f1360108740 2024-05-09T04:58:24.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.176+0000 7f134f7fe700 1 -- 172.21.15.46:0/2723949446 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f135801b030 con 0x7f1360108740 2024-05-09T04:58:24.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.177+0000 7f134f7fe700 1 --2- 172.21.15.46:0/2723949446 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f13480b29f0 0x7f13480b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:24.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.177+0000 7f135e59c700 1 --2- 172.21.15.46:0/2723949446 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f13480b29f0 0x7f13480b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:24.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.177+0000 7f134f7fe700 1 -- 172.21.15.46:0/2723949446 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f13580e2130 con 0x7f1360108740 2024-05-09T04:58:24.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.177+0000 7f135e59c700 1 --2- 172.21.15.46:0/2723949446 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f13480b29f0 0x7f13480b4f00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1350009c60 tx=0x7f1350009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:24.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.182+0000 7f134f7fe700 1 -- 172.21.15.46:0/2723949446 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1358022020 con 0x7f1360108740 2024-05-09T04:58:24.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:23 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:58:24.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:23 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:58:24.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:23 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:58:24.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.442+0000 7f1364a11700 1 -- 172.21.15.46:0/2723949446 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f136002e5f0 con 0x7f1360108740 2024-05-09T04:58:24.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.443+0000 7f134f7fe700 1 -- 172.21.15.46:0/2723949446 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7f1358010040 con 0x7f1360108740 2024-05-09T04:58:24.444 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:58:24.444 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:58:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.446+0000 7f1364a11700 1 -- 172.21.15.46:0/2723949446 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f13480b29f0 msgr2=0x7f13480b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.446+0000 7f1364a11700 1 --2- 172.21.15.46:0/2723949446 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f13480b29f0 0x7f13480b4f00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1350009c60 tx=0x7f1350009260 comp rx=0 tx=0).stop 2024-05-09T04:58:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.446+0000 7f1364a11700 1 -- 172.21.15.46:0/2723949446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1360108740 msgr2=0x7f136019c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.446+0000 7f1364a11700 1 --2- 172.21.15.46:0/2723949446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1360108740 0x7f136019c6d0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f135800ef30 tx=0x7f135800c3a0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.446+0000 7f1364a11700 1 -- 172.21.15.46:0/2723949446 shutdown_connections 2024-05-09T04:58:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.446+0000 7f1364a11700 1 --2- 172.21.15.46:0/2723949446 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f13480b29f0 0x7f13480b4f00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.446+0000 7f1364a11700 1 --2- 172.21.15.46:0/2723949446 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1360108740 0x7f136019c6d0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.446+0000 7f1364a11700 1 --2- 172.21.15.46:0/2723949446 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f13601034c0 0x7f136019bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.446 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.446+0000 7f1364a11700 1 --2- 172.21.15.46:0/2723949446 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1360107df0 0x7f136019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:24.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.446+0000 7f1364a11700 1 -- 172.21.15.46:0/2723949446 >> 172.21.15.46:0/2723949446 conn(0x7f13600fedc0 msgr2=0x7f13601096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:24.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.446+0000 7f1364a11700 1 -- 172.21.15.46:0/2723949446 shutdown_connections 2024-05-09T04:58:24.447 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:24.446+0000 7f1364a11700 1 -- 172.21.15.46:0/2723949446 wait complete. 2024-05-09T04:58:24.447 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:58:25.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:24 smithi046 ceph-mon[30136]: pgmap v224: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T04:58:25.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:24 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:58:25.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:24 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2723949446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:58:25.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:24 smithi072 ceph-mon[34088]: pgmap v224: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T04:58:25.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:24 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:58:25.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:24 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2723949446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:58:25.266 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:58:25.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:24 smithi110 ceph-mon[34904]: pgmap v224: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T04:58:25.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:24 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:58:25.323 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:24 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2723949446' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:58:25.577 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:26.967 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:26 smithi046 ceph-mon[30136]: pgmap v225: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T04:58:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:26 smithi072 ceph-mon[34088]: pgmap v225: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T04:58:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:26 smithi110 ceph-mon[34904]: pgmap v225: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T04:58:27.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.402+0000 7f93773eb700 1 -- 172.21.15.46:0/634208867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9370107df0 msgr2=0x7f9370108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:27.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.402+0000 7f93773eb700 1 --2- 172.21.15.46:0/634208867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9370107df0 0x7f9370108200 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f93600099c0 tx=0x7f9360009cd0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.402+0000 7f93773eb700 1 -- 172.21.15.46:0/634208867 shutdown_connections 2024-05-09T04:58:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.402+0000 7f93773eb700 1 --2- 172.21.15.46:0/634208867 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9370108740 0x7f93701100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.402+0000 7f93773eb700 1 --2- 172.21.15.46:0/634208867 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f93701034c0 0x7f93701038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.402+0000 7f93773eb700 1 --2- 172.21.15.46:0/634208867 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9370107df0 0x7f9370108200 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.402+0000 7f93773eb700 1 -- 172.21.15.46:0/634208867 >> 172.21.15.46:0/634208867 conn(0x7f93700fedf0 msgr2=0x7f9370101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.402+0000 7f93773eb700 1 -- 172.21.15.46:0/634208867 shutdown_connections 2024-05-09T04:58:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.402+0000 7f93773eb700 1 -- 172.21.15.46:0/634208867 wait complete. 2024-05-09T04:58:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.402+0000 7f93773eb700 1 Processor -- start 2024-05-09T04:58:27.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f93773eb700 1 -- start start 2024-05-09T04:58:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f93773eb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93701034c0 0x7f9370199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f93773eb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9370107df0 0x7f937019a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f93773eb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9370108740 0x7f937019a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f93773eb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9370113370 con 0x7f93701034c0 2024-05-09T04:58:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f93773eb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f93701131f0 con 0x7f9370107df0 2024-05-09T04:58:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f93773eb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f93701134f0 con 0x7f9370108740 2024-05-09T04:58:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f9375187700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93701034c0 0x7f9370199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f9375187700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93701034c0 0x7f9370199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55732/0 (socket says 172.21.15.46:55732) 2024-05-09T04:58:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f9375187700 1 -- 172.21.15.46:0/3768511772 learned_addr learned my addr 172.21.15.46:0/3768511772 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:27.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f9374986700 1 --2- 172.21.15.46:0/3768511772 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9370107df0 0x7f937019a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f9375988700 1 --2- 172.21.15.46:0/3768511772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9370108740 0x7f937019a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f9375187700 1 -- 172.21.15.46:0/3768511772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9370108740 msgr2=0x7f937019a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:27.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f9375187700 1 --2- 172.21.15.46:0/3768511772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9370108740 0x7f937019a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f9375187700 1 -- 172.21.15.46:0/3768511772 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9370107df0 msgr2=0x7f937019a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:27.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f9375187700 1 --2- 172.21.15.46:0/3768511772 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9370107df0 0x7f937019a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f9375187700 1 -- 172.21.15.46:0/3768511772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93701a9180 con 0x7f93701034c0 2024-05-09T04:58:27.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.403+0000 7f9375988700 1 --2- 172.21.15.46:0/3768511772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9370108740 0x7f937019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:27.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.404+0000 7f9375187700 1 --2- 172.21.15.46:0/3768511772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93701034c0 0x7f9370199b00 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f936c007f40 tx=0x7f936c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:27.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.404+0000 7f93667fc700 1 -- 172.21.15.46:0/3768511772 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c004bb0 con 0x7f93701034c0 2024-05-09T04:58:27.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.404+0000 7f93667fc700 1 -- 172.21.15.46:0/3768511772 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936c004d50 con 0x7f93701034c0 2024-05-09T04:58:27.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.404+0000 7f93773eb700 1 -- 172.21.15.46:0/3768511772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93701a9470 con 0x7f93701034c0 2024-05-09T04:58:27.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.404+0000 7f93667fc700 1 -- 172.21.15.46:0/3768511772 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c005a00 con 0x7f93701034c0 2024-05-09T04:58:27.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.404+0000 7f93773eb700 1 -- 172.21.15.46:0/3768511772 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93701a9930 con 0x7f93701034c0 2024-05-09T04:58:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.406+0000 7f93667fc700 1 -- 172.21.15.46:0/3768511772 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f936c01a020 con 0x7f93701034c0 2024-05-09T04:58:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.407+0000 7f93667fc700 1 --2- 172.21.15.46:0/3768511772 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f935c0b29f0 0x7f935c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.407+0000 7f9374986700 1 --2- 172.21.15.46:0/3768511772 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f935c0b29f0 0x7f935c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.407+0000 7f93667fc700 1 -- 172.21.15.46:0/3768511772 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f936c0e1e60 con 0x7f93701034c0 2024-05-09T04:58:27.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.408+0000 7f9385e01b80 1 Processor -- start 2024-05-09T04:58:27.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.408+0000 7f9385e01b80 1 -- start start 2024-05-09T04:58:27.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.408+0000 7f9374986700 1 --2- 172.21.15.46:0/3768511772 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f935c0b29f0 0x7f935c0b4f00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9360006010 tx=0x7f9360000bc0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:27.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.408+0000 7f9385e01b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba86d00 0x56470bb8f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.408+0000 7f9385e01b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba0c4a0 0x56470bb8f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.408+0000 7f9385e01b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470bb89400 0x56470bb89810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.408+0000 7f9385e01b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56470baac0a0 con 0x56470ba86d00 2024-05-09T04:58:27.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.408+0000 7f9385e01b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56470ba519a0 con 0x56470ba0c4a0 2024-05-09T04:58:27.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.408+0000 7f9385e01b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56470baa8060 con 0x56470bb89400 2024-05-09T04:58:27.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.408+0000 7f9375187700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba86d00 0x56470bb8f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.408+0000 7f9375187700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba86d00 0x56470bb8f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55746/0 (socket says 172.21.15.46:55746) 2024-05-09T04:58:27.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.408+0000 7f9375187700 1 -- 172.21.15.46:0/3040198233 learned_addr learned my addr 172.21.15.46:0/3040198233 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:27.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.408+0000 7f9374986700 1 --2- 172.21.15.46:0/3040198233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba0c4a0 0x56470bb8f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.408+0000 7f9375988700 1 --2- 172.21.15.46:0/3040198233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470bb89400 0x56470bb89810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.409+0000 7f9375187700 1 -- 172.21.15.46:0/3040198233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470bb89400 msgr2=0x56470bb89810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:27.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.409+0000 7f9375187700 1 --2- 172.21.15.46:0/3040198233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470bb89400 0x56470bb89810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.409+0000 7f9375187700 1 -- 172.21.15.46:0/3040198233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba0c4a0 msgr2=0x56470bb8f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:27.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.409+0000 7f9375187700 1 --2- 172.21.15.46:0/3040198233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba0c4a0 0x56470bb8f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.409+0000 7f9375187700 1 -- 172.21.15.46:0/3040198233 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56470ba42500 con 0x56470ba86d00 2024-05-09T04:58:27.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.409+0000 7f9374986700 1 --2- 172.21.15.46:0/3040198233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba0c4a0 0x56470bb8f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:27.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.409+0000 7f9375187700 1 --2- 172.21.15.46:0/3040198233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba86d00 0x56470bb8f240 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f936c0a8df0 tx=0x7f936c0a8c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d382a03efe5b96 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:27.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.409+0000 7f935b7fe700 1 -- 172.21.15.46:0/3040198233 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c0b1f50 con 0x56470ba86d00 2024-05-09T04:58:27.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.409+0000 7f935b7fe700 1 -- 172.21.15.46:0/3040198233 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936c0ad170 con 0x56470ba86d00 2024-05-09T04:58:27.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.409+0000 7f935b7fe700 1 -- 172.21.15.46:0/3040198233 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c0b1ac0 con 0x56470ba86d00 2024-05-09T04:58:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.410+0000 7f9385e01b80 1 -- 172.21.15.46:0/3040198233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba86d00 msgr2=0x56470bb8f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.410+0000 7f9385e01b80 1 --2- 172.21.15.46:0/3040198233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba86d00 0x56470bb8f240 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f936c0a8df0 tx=0x7f936c0a8c80 comp rx=0 tx=0).stop 2024-05-09T04:58:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.410+0000 7f9385e01b80 1 -- 172.21.15.46:0/3040198233 shutdown_connections 2024-05-09T04:58:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.410+0000 7f9385e01b80 1 --2- 172.21.15.46:0/3040198233 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470bb89400 0x56470bb89810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.410+0000 7f9385e01b80 1 --2- 172.21.15.46:0/3040198233 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba86d00 0x56470bb8f240 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.410+0000 7f9385e01b80 1 --2- 172.21.15.46:0/3040198233 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba0c4a0 0x56470bb8f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.410+0000 7f9385e01b80 1 -- 172.21.15.46:0/3040198233 >> 172.21.15.46:0/3040198233 conn(0x56470ba0ff70 msgr2=0x56470bb84220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.410+0000 7f9385e01b80 1 -- 172.21.15.46:0/3040198233 shutdown_connections 2024-05-09T04:58:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.410+0000 7f9385e01b80 1 -- 172.21.15.46:0/3040198233 wait complete. 2024-05-09T04:58:27.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.410+0000 7f9385e01b80 1 -- start start 2024-05-09T04:58:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.410+0000 7f9385e01b80 1 Processor -- start 2024-05-09T04:58:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9385e01b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba0c4a0 0x56470bb93080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9385e01b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470ba86d00 0x56470bb935c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9385e01b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470bb89400 0x56470bb93b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9385e01b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56470bb8a100 con 0x56470ba0c4a0 2024-05-09T04:58:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9385e01b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56470ba46b20 con 0x56470bb89400 2024-05-09T04:58:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9385e01b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56470bb9f1d0 con 0x56470ba86d00 2024-05-09T04:58:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9385e01b80 10 client.0 _ll_register_callbacks cb 0x56470b91d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9375187700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba0c4a0 0x56470bb93080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9375187700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba0c4a0 0x56470bb93080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55752/0 (socket says 172.21.15.46:55752) 2024-05-09T04:58:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9375187700 1 -- 172.21.15.46:0/2776804909 learned_addr learned my addr 172.21.15.46:0/2776804909 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9375988700 1 --2- 172.21.15.46:0/2776804909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470bb89400 0x56470bb93b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9374986700 1 --2- 172.21.15.46:0/2776804909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470ba86d00 0x56470bb935c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9375187700 1 -- 172.21.15.46:0/2776804909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470ba86d00 msgr2=0x56470bb935c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:27.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9375187700 1 --2- 172.21.15.46:0/2776804909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470ba86d00 0x56470bb935c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9375187700 1 -- 172.21.15.46:0/2776804909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470bb89400 msgr2=0x56470bb93b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9375187700 1 --2- 172.21.15.46:0/2776804909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470bb89400 0x56470bb93b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.411+0000 7f9375187700 1 -- 172.21.15.46:0/2776804909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56470b927490 con 0x56470ba0c4a0 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.412+0000 7f9375187700 1 --2- 172.21.15.46:0/2776804909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba0c4a0 0x56470bb93080 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f936c004b80 tx=0x7f936c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a15cbfbb1b15471 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.412+0000 7f9358ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.412+0000 7f9358ff9700 1 -- 172.21.15.46:0/2776804909 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f936c0c3060 con 0x56470ba0c4a0 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.412+0000 7f9385e01b80 1 -- 172.21.15.46:0/2776804909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936c0ad170 con 0x56470ba0c4a0 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.412+0000 7f9358ff9700 1 -- 172.21.15.46:0/2776804909 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f936c0b1cd0 con 0x56470ba0c4a0 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.412+0000 7f9358ff9700 1 -- 172.21.15.46:0/2776804909 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f936c0b1f50 con 0x56470ba0c4a0 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.412+0000 7f9385e01b80 10 client.17001 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.412+0000 7f9385e01b80 1 -- 172.21.15.46:0/2776804909 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56470bba0b20 con 0x56470ba0c4a0 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.412+0000 7f9358ff9700 1 -- 172.21.15.46:0/2776804909 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936c0ad170 con 0x56470ba0c4a0 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.412+0000 7f9358ff9700 1 -- 172.21.15.46:0/2776804909 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f936c0a7dc0 con 0x56470ba0c4a0 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.412+0000 7f9385e01b80 10 client.17001 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.412+0000 7f9385e01b80 10 client.17001 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:58:27.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.412+0000 7f9385e01b80 20 client.17001 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.413+0000 7f9385e01b80 1 --2- 172.21.15.46:0/2776804909 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x56470bba2ff0 0x56470bba5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.413+0000 7f9385e01b80 4 client.17001 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:58:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.413+0000 7f9385e01b80 1 -- 172.21.15.46:0/2776804909 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56470bab9a80 con 0x56470bba2ff0 2024-05-09T04:58:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.413+0000 7f9374986700 1 --2- 172.21.15.46:0/2776804909 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x56470bba2ff0 0x56470bba5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.413+0000 7f9358ff9700 10 client.17001 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:58:27.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.413+0000 7f9374986700 1 --2- 172.21.15.46:0/2776804909 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x56470bba2ff0 0x56470bba5410 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9a2ec4f2805075f server_cookie=1cd6999926a2521b in_seq=0 out_seq=0 2024-05-09T04:58:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.414+0000 7f9358ff9700 1 -- 172.21.15.46:0/2776804909 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56470bab9a80 con 0x56470bba2ff0 2024-05-09T04:58:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.414+0000 7f9358ff9700 10 client.17001 handle_command_reply: tid=0 2024-05-09T04:58:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.414+0000 7f9385e01b80 1 client.17001 shutdown 2024-05-09T04:58:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.414+0000 7f9385e01b80 1 -- 172.21.15.46:0/2776804909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba0c4a0 msgr2=0x56470bb93080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.414+0000 7f9385e01b80 1 --2- 172.21.15.46:0/2776804909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba0c4a0 0x56470bb93080 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f936c004b80 tx=0x7f936c004ad0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.414+0000 7f9385e01b80 1 -- 172.21.15.46:0/2776804909 shutdown_connections 2024-05-09T04:58:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.414+0000 7f9385e01b80 1 --2- 172.21.15.46:0/2776804909 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470bb89400 0x56470bb93b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.418 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.414+0000 7f9385e01b80 1 --2- 172.21.15.46:0/2776804909 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba0c4a0 0x56470bb93080 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.414+0000 7f9385e01b80 1 --2- 172.21.15.46:0/2776804909 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x56470bba2ff0 0x56470bba5410 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.414+0000 7f9385e01b80 1 --2- 172.21.15.46:0/2776804909 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470ba86d00 0x56470bb935c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.414+0000 7f9385e01b80 1 -- 172.21.15.46:0/2776804909 >> 172.21.15.46:0/2776804909 conn(0x56470ba0ff70 msgr2=0x56470bb7dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.414+0000 7f9358ff9700 0 client.17001 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:58:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.415+0000 7f9385e01b80 1 -- 172.21.15.46:0/2776804909 shutdown_connections 2024-05-09T04:58:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.415+0000 7f9385e01b80 1 -- 172.21.15.46:0/2776804909 wait complete. 2024-05-09T04:58:27.419 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.415+0000 7f9385e01b80 20 client.17001 trim_cache size 0 max 16384 2024-05-09T04:58:27.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.430+0000 7f9385e01b80 1 Processor -- start 2024-05-09T04:58:27.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.430+0000 7f9385e01b80 1 -- start start 2024-05-09T04:58:27.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.430+0000 7f9385e01b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba0c4a0 0x56470bb93940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.430+0000 7f9385e01b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba86d00 0x56470bb91970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.430+0000 7f9385e01b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470bb89400 0x56470bb91eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.430+0000 7f9385e01b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f93701b3a80 con 0x56470ba86d00 2024-05-09T04:58:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.430+0000 7f9385e01b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f93701b1310 con 0x56470ba0c4a0 2024-05-09T04:58:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.430+0000 7f9385e01b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f93701af180 con 0x56470bb89400 2024-05-09T04:58:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.430+0000 7f9374986700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba86d00 0x56470bb91970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.430+0000 7f9374986700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba86d00 0x56470bb91970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55766/0 (socket says 172.21.15.46:55766) 2024-05-09T04:58:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.430+0000 7f9374986700 1 -- 172.21.15.46:0/1869236469 learned_addr learned my addr 172.21.15.46:0/1869236469 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.431+0000 7f9375988700 1 --2- 172.21.15.46:0/1869236469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470bb89400 0x56470bb91eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.431+0000 7f9375187700 1 --2- 172.21.15.46:0/1869236469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba0c4a0 0x56470bb93940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.431+0000 7f9374986700 1 -- 172.21.15.46:0/1869236469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470bb89400 msgr2=0x56470bb91eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.431+0000 7f9374986700 1 --2- 172.21.15.46:0/1869236469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470bb89400 0x56470bb91eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.431+0000 7f9374986700 1 -- 172.21.15.46:0/1869236469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba0c4a0 msgr2=0x56470bb93940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.431+0000 7f9374986700 1 --2- 172.21.15.46:0/1869236469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba0c4a0 0x56470bb93940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.431+0000 7f9374986700 1 -- 172.21.15.46:0/1869236469 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56470bb809d0 con 0x56470ba86d00 2024-05-09T04:58:27.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.431+0000 7f9375988700 1 --2- 172.21.15.46:0/1869236469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470bb89400 0x56470bb91eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.431+0000 7f9374986700 1 --2- 172.21.15.46:0/1869236469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba86d00 0x56470bb91970 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9360004010 tx=0x7f9360021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11dbea16325b4054 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.431+0000 7f935affd700 1 -- 172.21.15.46:0/1869236469 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9360039070 con 0x56470ba86d00 2024-05-09T04:58:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.431+0000 7f935affd700 1 -- 172.21.15.46:0/1869236469 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936000be40 con 0x56470ba86d00 2024-05-09T04:58:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.431+0000 7f935affd700 1 -- 172.21.15.46:0/1869236469 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93600330d0 con 0x56470ba86d00 2024-05-09T04:58:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.432+0000 7f9385e01b80 1 -- 172.21.15.46:0/1869236469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba86d00 msgr2=0x56470bb91970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.432+0000 7f9385e01b80 1 --2- 172.21.15.46:0/1869236469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba86d00 0x56470bb91970 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f9360004010 tx=0x7f9360021000 comp rx=0 tx=0).stop 2024-05-09T04:58:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.432+0000 7f9385e01b80 1 -- 172.21.15.46:0/1869236469 shutdown_connections 2024-05-09T04:58:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.432+0000 7f9385e01b80 1 --2- 172.21.15.46:0/1869236469 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470bb89400 0x56470bb91eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.432+0000 7f9385e01b80 1 --2- 172.21.15.46:0/1869236469 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba0c4a0 0x56470bb93940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.432+0000 7f9385e01b80 1 --2- 172.21.15.46:0/1869236469 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470ba86d00 0x56470bb91970 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.432+0000 7f9385e01b80 1 -- 172.21.15.46:0/1869236469 >> 172.21.15.46:0/1869236469 conn(0x56470ba0ff70 msgr2=0x56470bb8e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.432+0000 7f9385e01b80 1 -- 172.21.15.46:0/1869236469 shutdown_connections 2024-05-09T04:58:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.432+0000 7f9385e01b80 1 -- 172.21.15.46:0/1869236469 wait complete. 2024-05-09T04:58:27.434 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.432+0000 7f9385e01b80 1 -- start start 2024-05-09T04:58:27.435 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.432+0000 7f9385e01b80 1 Processor -- start 2024-05-09T04:58:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9385e01b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470ba0c4a0 0x56470bba8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9385e01b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba86d00 0x56470bba4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9385e01b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470bb89400 0x56470bba4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9385e01b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f93701af7d0 con 0x56470bb89400 2024-05-09T04:58:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9385e01b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f93701b0c40 con 0x56470ba86d00 2024-05-09T04:58:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9385e01b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f93701afea0 con 0x56470ba0c4a0 2024-05-09T04:58:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9385e01b80 10 client.0 _ll_register_callbacks cb 0x56470bb84880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:27.438 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9375988700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470bb89400 0x56470bba4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9375988700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470bb89400 0x56470bba4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55772/0 (socket says 172.21.15.46:55772) 2024-05-09T04:58:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9375988700 1 -- 172.21.15.46:0/1801688806 learned_addr learned my addr 172.21.15.46:0/1801688806 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9375187700 1 --2- 172.21.15.46:0/1801688806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470ba0c4a0 0x56470bba8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9374986700 1 --2- 172.21.15.46:0/1801688806 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba86d00 0x56470bba4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9375988700 1 -- 172.21.15.46:0/1801688806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470ba0c4a0 msgr2=0x56470bba8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9375988700 1 --2- 172.21.15.46:0/1801688806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470ba0c4a0 0x56470bba8400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9375988700 1 -- 172.21.15.46:0/1801688806 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba86d00 msgr2=0x56470bba4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9375988700 1 --2- 172.21.15.46:0/1801688806 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba86d00 0x56470bba4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9375988700 1 -- 172.21.15.46:0/1801688806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56470bba52a0 con 0x56470bb89400 2024-05-09T04:58:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.433+0000 7f9374986700 1 --2- 172.21.15.46:0/1801688806 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba86d00 0x56470bba4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.434+0000 7f9375988700 1 --2- 172.21.15.46:0/1801688806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470bb89400 0x56470bba4cb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9368011a50 tx=0x7f9368011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17df35051ea73c95 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.434+0000 7f9358ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.434+0000 7f9358ff9700 1 -- 172.21.15.46:0/1801688806 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9368020060 con 0x56470bb89400 2024-05-09T04:58:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.434+0000 7f9358ff9700 1 -- 172.21.15.46:0/1801688806 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f9368010cb0 con 0x56470bb89400 2024-05-09T04:58:27.439 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.434+0000 7f9385e01b80 1 -- 172.21.15.46:0/1801688806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f936000be40 con 0x56470bb89400 2024-05-09T04:58:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.434+0000 7f9358ff9700 1 -- 172.21.15.46:0/1801688806 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9368023400 con 0x56470bb89400 2024-05-09T04:58:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.434+0000 7f9385e01b80 10 client.17007 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.434+0000 7f9385e01b80 1 -- 172.21.15.46:0/1801688806 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56470bc0c300 con 0x56470bb89400 2024-05-09T04:58:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.434+0000 7f9358ff9700 1 -- 172.21.15.46:0/1801688806 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f936000be40 con 0x56470bb89400 2024-05-09T04:58:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.434+0000 7f9358ff9700 1 -- 172.21.15.46:0/1801688806 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f936801a070 con 0x56470bb89400 2024-05-09T04:58:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.434+0000 7f9385e01b80 10 client.17007 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.434+0000 7f9385e01b80 10 client.17007 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:58:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.434+0000 7f9385e01b80 20 client.17007 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.434+0000 7f9385e01b80 1 --2- 172.21.15.46:0/1801688806 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x56470bc0ca10 0x56470bc0ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.435+0000 7f9385e01b80 4 client.17007 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:58:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.435+0000 7f9385e01b80 1 -- 172.21.15.46:0/1801688806 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56470bab3b20 con 0x56470bc0ca10 2024-05-09T04:58:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.435+0000 7f9375187700 1 --2- 172.21.15.46:0/1801688806 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x56470bc0ca10 0x56470bc0ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.435+0000 7f9375187700 1 --2- 172.21.15.46:0/1801688806 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x56470bc0ca10 0x56470bc0ee30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c78f79d01ae3f6c server_cookie=43fc007c1b18e42d in_seq=0 out_seq=0 2024-05-09T04:58:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.435+0000 7f9358ff9700 10 client.17007 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:58:27.440 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.435+0000 7f9358ff9700 1 -- 172.21.15.46:0/1801688806 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56470bab3b20 con 0x56470bc0ca10 2024-05-09T04:58:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.435+0000 7f9358ff9700 10 client.17007 handle_command_reply: tid=0 2024-05-09T04:58:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.435+0000 7f9385e01b80 1 client.17007 shutdown 2024-05-09T04:58:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f9385e01b80 1 -- 172.21.15.46:0/1801688806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470bb89400 msgr2=0x56470bba4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f9385e01b80 1 --2- 172.21.15.46:0/1801688806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470bb89400 0x56470bba4cb0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f9368011a50 tx=0x7f9368011d60 comp rx=0 tx=0).stop 2024-05-09T04:58:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f9385e01b80 1 -- 172.21.15.46:0/1801688806 shutdown_connections 2024-05-09T04:58:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f9385e01b80 1 --2- 172.21.15.46:0/1801688806 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56470bb89400 0x56470bba4cb0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f9385e01b80 1 --2- 172.21.15.46:0/1801688806 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x56470bc0ca10 0x56470bc0ee30 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f9385e01b80 1 --2- 172.21.15.46:0/1801688806 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56470ba0c4a0 0x56470bba8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f9385e01b80 1 --2- 172.21.15.46:0/1801688806 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56470ba86d00 0x56470bba4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f9385e01b80 1 -- 172.21.15.46:0/1801688806 >> 172.21.15.46:0/1801688806 conn(0x56470ba0ff70 msgr2=0x56470bb8e190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f9358ff9700 0 client.17007 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:58:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f9385e01b80 1 -- 172.21.15.46:0/1801688806 shutdown_connections 2024-05-09T04:58:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f9385e01b80 1 -- 172.21.15.46:0/1801688806 wait complete. 2024-05-09T04:58:27.441 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f9385e01b80 20 client.17007 trim_cache size 0 max 16384 2024-05-09T04:58:27.442 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:58:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f93773eb700 1 -- 172.21.15.46:0/3768511772 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f935c0b29f0 msgr2=0x7f935c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f93773eb700 1 --2- 172.21.15.46:0/3768511772 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f935c0b29f0 0x7f935c0b4f00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9360006010 tx=0x7f9360000bc0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.442 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f93773eb700 1 -- 172.21.15.46:0/3768511772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93701034c0 msgr2=0x7f9370199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.436+0000 7f93773eb700 1 --2- 172.21.15.46:0/3768511772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93701034c0 0x7f9370199b00 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f936c007f40 tx=0x7f936c00d3b0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.437+0000 7f93773eb700 1 -- 172.21.15.46:0/3768511772 shutdown_connections 2024-05-09T04:58:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.437+0000 7f93773eb700 1 --2- 172.21.15.46:0/3768511772 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f935c0b29f0 0x7f935c0b4f00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.437+0000 7f93773eb700 1 --2- 172.21.15.46:0/3768511772 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9370108740 0x7f937019a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.437+0000 7f93773eb700 1 --2- 172.21.15.46:0/3768511772 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f93701034c0 0x7f9370199b00 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.437+0000 7f93773eb700 1 --2- 172.21.15.46:0/3768511772 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9370107df0 0x7f937019a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.437+0000 7f93773eb700 1 -- 172.21.15.46:0/3768511772 >> 172.21.15.46:0/3768511772 conn(0x7f93700fedf0 msgr2=0x7f9370101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.437+0000 7f93773eb700 1 -- 172.21.15.46:0/3768511772 shutdown_connections 2024-05-09T04:58:27.443 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:27.437+0000 7f93773eb700 1 -- 172.21.15.46:0/3768511772 wait complete. 2024-05-09T04:58:28.152 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:58:28.405 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:29.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:28 smithi046 ceph-mon[30136]: pgmap v226: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2024-05-09T04:58:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:28 smithi072 ceph-mon[34088]: pgmap v226: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2024-05-09T04:58:29.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:28 smithi110 ceph-mon[34904]: pgmap v226: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 8 op/s 2024-05-09T04:58:30.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:29 smithi046 ceph-mon[30136]: pgmap v227: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:58:30.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:29 smithi110 ceph-mon[34904]: pgmap v227: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:58:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.368+0000 7f7165cc6700 1 -- 172.21.15.46:0/255124661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71601034c0 msgr2=0x7f71601038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.368+0000 7f7165cc6700 1 --2- 172.21.15.46:0/255124661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71601034c0 0x7f71601038d0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f71500099c0 tx=0x7f7150009cd0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.368+0000 7f7165cc6700 1 -- 172.21.15.46:0/255124661 shutdown_connections 2024-05-09T04:58:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.368+0000 7f7165cc6700 1 --2- 172.21.15.46:0/255124661 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7160108740 0x7f71601100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.368+0000 7f7165cc6700 1 --2- 172.21.15.46:0/255124661 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71601034c0 0x7f71601038d0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.368+0000 7f7165cc6700 1 --2- 172.21.15.46:0/255124661 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7160107df0 0x7f7160108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.368+0000 7f7165cc6700 1 -- 172.21.15.46:0/255124661 >> 172.21.15.46:0/255124661 conn(0x7f71600fedf0 msgr2=0x7f7160101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:30.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.368+0000 7f7165cc6700 1 -- 172.21.15.46:0/255124661 shutdown_connections 2024-05-09T04:58:30.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.369+0000 7f7165cc6700 1 -- 172.21.15.46:0/255124661 wait complete. 2024-05-09T04:58:30.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.369+0000 7f7165cc6700 1 Processor -- start 2024-05-09T04:58:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.369+0000 7f7165cc6700 1 -- start start 2024-05-09T04:58:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f7165cc6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71601034c0 0x7f71600ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f7165cc6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7160107df0 0x7f71600ff810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f7165cc6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7160108740 0x7f71600ffd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f7165cc6700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7160113370 con 0x7f71601034c0 2024-05-09T04:58:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f7165cc6700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f71601131f0 con 0x7f7160108740 2024-05-09T04:58:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f7165cc6700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71601134f0 con 0x7f7160107df0 2024-05-09T04:58:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f715f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71601034c0 0x7f71600ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f715f7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71601034c0 0x7f71600ff2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55792/0 (socket says 172.21.15.46:55792) 2024-05-09T04:58:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f715f7fe700 1 -- 172.21.15.46:0/2518676204 learned_addr learned my addr 172.21.15.46:0/2518676204 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:30.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f715ffff700 1 --2- 172.21.15.46:0/2518676204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7160108740 0x7f71600ffd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f715effd700 1 --2- 172.21.15.46:0/2518676204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7160107df0 0x7f71600ff810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f715f7fe700 1 -- 172.21.15.46:0/2518676204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7160107df0 msgr2=0x7f71600ff810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f715f7fe700 1 --2- 172.21.15.46:0/2518676204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7160107df0 0x7f71600ff810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f715f7fe700 1 -- 172.21.15.46:0/2518676204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7160108740 msgr2=0x7f71600ffd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f715f7fe700 1 --2- 172.21.15.46:0/2518676204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7160108740 0x7f71600ffd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f715f7fe700 1 -- 172.21.15.46:0/2518676204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71601a9200 con 0x7f71601034c0 2024-05-09T04:58:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.370+0000 7f715f7fe700 1 --2- 172.21.15.46:0/2518676204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71601034c0 0x7f71600ff2d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f71500098e0 tx=0x7f71500057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.371+0000 7f715cff9700 1 -- 172.21.15.46:0/2518676204 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150027070 con 0x7f71601034c0 2024-05-09T04:58:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.371+0000 7f715cff9700 1 -- 172.21.15.46:0/2518676204 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7150003f20 con 0x7f71601034c0 2024-05-09T04:58:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.371+0000 7f7165cc6700 1 -- 172.21.15.46:0/2518676204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71601a9490 con 0x7f71601034c0 2024-05-09T04:58:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.371+0000 7f715cff9700 1 -- 172.21.15.46:0/2518676204 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7150022da0 con 0x7f71601034c0 2024-05-09T04:58:30.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.371+0000 7f7165cc6700 1 -- 172.21.15.46:0/2518676204 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71601a9940 con 0x7f71601034c0 2024-05-09T04:58:30.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.373+0000 7f715cff9700 1 -- 172.21.15.46:0/2518676204 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f715002a070 con 0x7f71601034c0 2024-05-09T04:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.374+0000 7f715cff9700 1 --2- 172.21.15.46:0/2518676204 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f71480b2a70 0x7f71480b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.374+0000 7f71746dcb80 1 Processor -- start 2024-05-09T04:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.374+0000 7f71746dcb80 1 -- start start 2024-05-09T04:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.374+0000 7f715effd700 1 --2- 172.21.15.46:0/2518676204 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f71480b2a70 0x7f71480b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f715cff9700 1 -- 172.21.15.46:0/2518676204 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f71500e33d0 con 0x7f71601034c0 2024-05-09T04:58:30.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f71746dcb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e3370160 0x5576e34094f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f71746dcb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e32864a0 0x5576e3409a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f71746dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e34036a0 0x5576e3403ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f71746dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5576e3325ff0 con 0x5576e34036a0 2024-05-09T04:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f71746dcb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5576e32c47f0 con 0x5576e32864a0 2024-05-09T04:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f71746dcb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5576e3321fb0 con 0x5576e3370160 2024-05-09T04:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f715ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e34036a0 0x5576e3403ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f715ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e34036a0 0x5576e3403ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55804/0 (socket says 172.21.15.46:55804) 2024-05-09T04:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f715ffff700 1 -- 172.21.15.46:0/2429370406 learned_addr learned my addr 172.21.15.46:0/2429370406 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f715effd700 1 --2- 172.21.15.46:0/2518676204 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f71480b2a70 0x7f71480b4f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7154005fd0 tx=0x7f7154005f60 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f715f7fe700 1 --2- 172.21.15.46:0/2429370406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e3370160 0x5576e34094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f715effd700 1 --2- 172.21.15.46:0/2429370406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e32864a0 0x5576e3409a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f715ffff700 1 -- 172.21.15.46:0/2429370406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e3370160 msgr2=0x5576e34094f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:30.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f715ffff700 1 --2- 172.21.15.46:0/2429370406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e3370160 0x5576e34094f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f715ffff700 1 -- 172.21.15.46:0/2429370406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e32864a0 msgr2=0x5576e3409a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f715ffff700 1 --2- 172.21.15.46:0/2429370406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e32864a0 0x5576e3409a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.375+0000 7f715ffff700 1 -- 172.21.15.46:0/2429370406 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576e32bc370 con 0x5576e34036a0 2024-05-09T04:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.376+0000 7f715ffff700 1 --2- 172.21.15.46:0/2429370406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e34036a0 0x5576e3403ad0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7158003f30 tx=0x7f7158009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c02631d516a2401 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.376+0000 7f7145ffb700 1 -- 172.21.15.46:0/2429370406 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158004740 con 0x5576e34036a0 2024-05-09T04:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.376+0000 7f7145ffb700 1 -- 172.21.15.46:0/2429370406 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f715801a070 con 0x5576e34036a0 2024-05-09T04:58:30.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.376+0000 7f7145ffb700 1 -- 172.21.15.46:0/2429370406 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158015680 con 0x5576e34036a0 2024-05-09T04:58:30.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.377+0000 7f71746dcb80 1 -- 172.21.15.46:0/2429370406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e34036a0 msgr2=0x5576e3403ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:30.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.377+0000 7f71746dcb80 1 --2- 172.21.15.46:0/2429370406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e34036a0 0x5576e3403ad0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f7158003f30 tx=0x7f7158009c20 comp rx=0 tx=0).stop 2024-05-09T04:58:30.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.377+0000 7f71746dcb80 1 -- 172.21.15.46:0/2429370406 shutdown_connections 2024-05-09T04:58:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.377+0000 7f71746dcb80 1 --2- 172.21.15.46:0/2429370406 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e34036a0 0x5576e3403ad0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.377+0000 7f71746dcb80 1 --2- 172.21.15.46:0/2429370406 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e3370160 0x5576e34094f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.377+0000 7f71746dcb80 1 --2- 172.21.15.46:0/2429370406 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e32864a0 0x5576e3409a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.377+0000 7f71746dcb80 1 -- 172.21.15.46:0/2429370406 >> 172.21.15.46:0/2429370406 conn(0x5576e32e66a0 msgr2=0x5576e33fe400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.377+0000 7f71746dcb80 1 -- 172.21.15.46:0/2429370406 shutdown_connections 2024-05-09T04:58:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.377+0000 7f71746dcb80 1 -- 172.21.15.46:0/2429370406 wait complete. 2024-05-09T04:58:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.377+0000 7f71746dcb80 1 -- start start 2024-05-09T04:58:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.377+0000 7f71746dcb80 1 Processor -- start 2024-05-09T04:58:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f71746dcb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e32864a0 0x5576e340d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f71746dcb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e3370160 0x5576e340d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f71746dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e34036a0 0x5576e340dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f71746dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5576e31a1ba0 con 0x5576e34036a0 2024-05-09T04:58:30.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f71746dcb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5576e32e0150 con 0x5576e32864a0 2024-05-09T04:58:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f71746dcb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5576e34101d0 con 0x5576e3370160 2024-05-09T04:58:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f71746dcb80 10 client.0 _ll_register_callbacks cb 0x5576e31976c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f715ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e34036a0 0x5576e340dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f715ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e34036a0 0x5576e340dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55814/0 (socket says 172.21.15.46:55814) 2024-05-09T04:58:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f715ffff700 1 -- 172.21.15.46:0/1815693213 learned_addr learned my addr 172.21.15.46:0/1815693213 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f715f7fe700 1 --2- 172.21.15.46:0/1815693213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e32864a0 0x5576e340d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f715effd700 1 --2- 172.21.15.46:0/1815693213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e3370160 0x5576e340d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f715ffff700 1 -- 172.21.15.46:0/1815693213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e3370160 msgr2=0x5576e340d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f715ffff700 1 --2- 172.21.15.46:0/1815693213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e3370160 0x5576e340d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f715ffff700 1 -- 172.21.15.46:0/1815693213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e32864a0 msgr2=0x5576e340d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f715ffff700 1 --2- 172.21.15.46:0/1815693213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e32864a0 0x5576e340d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f715ffff700 1 -- 172.21.15.46:0/1815693213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576e31a1490 con 0x5576e34036a0 2024-05-09T04:58:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.378+0000 7f715effd700 1 --2- 172.21.15.46:0/1815693213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e3370160 0x5576e340d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.379+0000 7f715ffff700 1 --2- 172.21.15.46:0/1815693213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e34036a0 0x5576e340dc70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7158000f80 tx=0x7f7158007860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7e32c9e47a7a82 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.379+0000 7f713effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.379+0000 7f713effd700 1 -- 172.21.15.46:0/1815693213 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7158015890 con 0x5576e34036a0 2024-05-09T04:58:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.379+0000 7f713effd700 1 -- 172.21.15.46:0/1815693213 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f7158015a30 con 0x5576e34036a0 2024-05-09T04:58:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.379+0000 7f71746dcb80 1 -- 172.21.15.46:0/1815693213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f715801a070 con 0x5576e34036a0 2024-05-09T04:58:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.379+0000 7f713effd700 1 -- 172.21.15.46:0/1815693213 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7158015c30 con 0x5576e34036a0 2024-05-09T04:58:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.379+0000 7f71746dcb80 10 client.17019 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.379+0000 7f71746dcb80 1 -- 172.21.15.46:0/1815693213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5576e341ac90 con 0x5576e34036a0 2024-05-09T04:58:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.379+0000 7f713effd700 1 -- 172.21.15.46:0/1815693213 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f715801a070 con 0x5576e34036a0 2024-05-09T04:58:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.379+0000 7f713effd700 1 -- 172.21.15.46:0/1815693213 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f715801e3f0 con 0x5576e34036a0 2024-05-09T04:58:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.379+0000 7f71746dcb80 10 client.17019 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:30.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.379+0000 7f71746dcb80 10 client.17019 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:58:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.379+0000 7f71746dcb80 20 client.17019 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.379+0000 7f71746dcb80 1 --2- 172.21.15.46:0/1815693213 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5576e341d1c0 0x5576e341f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.380+0000 7f71746dcb80 4 client.17019 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:58:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.380+0000 7f71746dcb80 1 -- 172.21.15.46:0/1815693213 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576e33339d0 con 0x5576e341d1c0 2024-05-09T04:58:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.380+0000 7f715f7fe700 1 --2- 172.21.15.46:0/1815693213 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5576e341d1c0 0x5576e341f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.380+0000 7f713effd700 10 client.17019 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:58:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.380+0000 7f715f7fe700 1 --2- 172.21.15.46:0/1815693213 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5576e341d1c0 0x5576e341f5e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fadc2596aae2f61b server_cookie=e0d4fb107a0d7b61 in_seq=0 out_seq=0 2024-05-09T04:58:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.381+0000 7f713effd700 1 -- 172.21.15.46:0/1815693213 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5576e33339d0 con 0x5576e341d1c0 2024-05-09T04:58:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.381+0000 7f713effd700 10 client.17019 handle_command_reply: tid=0 2024-05-09T04:58:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.381+0000 7f71746dcb80 1 client.17019 shutdown 2024-05-09T04:58:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.381+0000 7f71746dcb80 1 -- 172.21.15.46:0/1815693213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e34036a0 msgr2=0x5576e340dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.381+0000 7f71746dcb80 1 --2- 172.21.15.46:0/1815693213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e34036a0 0x5576e340dc70 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f7158000f80 tx=0x7f7158007860 comp rx=0 tx=0).stop 2024-05-09T04:58:30.386 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.381+0000 7f71746dcb80 1 -- 172.21.15.46:0/1815693213 shutdown_connections 2024-05-09T04:58:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.381+0000 7f71746dcb80 1 --2- 172.21.15.46:0/1815693213 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5576e341d1c0 0x5576e341f5e0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.381+0000 7f713effd700 0 client.17019 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:58:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.381+0000 7f71746dcb80 1 --2- 172.21.15.46:0/1815693213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e34036a0 0x5576e340dc70 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.381+0000 7f71746dcb80 1 --2- 172.21.15.46:0/1815693213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e32864a0 0x5576e340d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.381+0000 7f71746dcb80 1 --2- 172.21.15.46:0/1815693213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e3370160 0x5576e340d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.381+0000 7f71746dcb80 1 -- 172.21.15.46:0/1815693213 >> 172.21.15.46:0/1815693213 conn(0x5576e32e66a0 msgr2=0x5576e33fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.381+0000 7f71746dcb80 1 -- 172.21.15.46:0/1815693213 shutdown_connections 2024-05-09T04:58:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.381+0000 7f71746dcb80 1 -- 172.21.15.46:0/1815693213 wait complete. 2024-05-09T04:58:30.387 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.382+0000 7f71746dcb80 20 client.17019 trim_cache size 0 max 16384 2024-05-09T04:58:30.388 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:29 smithi072 ceph-mon[34088]: pgmap v227: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:58:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.396+0000 7f71746dcb80 1 Processor -- start 2024-05-09T04:58:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f71746dcb80 1 -- start start 2024-05-09T04:58:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f71746dcb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e32864a0 0x5576e340c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f71746dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3370160 0x5576e34077c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f71746dcb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e34036a0 0x5576e3419430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f71746dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f71601b3910 con 0x5576e3370160 2024-05-09T04:58:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f71746dcb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f71601b11a0 con 0x5576e34036a0 2024-05-09T04:58:30.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f71746dcb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71601af010 con 0x5576e32864a0 2024-05-09T04:58:30.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f715effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3370160 0x5576e34077c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f715effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3370160 0x5576e34077c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55816/0 (socket says 172.21.15.46:55816) 2024-05-09T04:58:30.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f715effd700 1 -- 172.21.15.46:0/2038786900 learned_addr learned my addr 172.21.15.46:0/2038786900 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:30.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f715ffff700 1 --2- 172.21.15.46:0/2038786900 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e34036a0 0x5576e3419430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f715f7fe700 1 --2- 172.21.15.46:0/2038786900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e32864a0 0x5576e340c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f715effd700 1 -- 172.21.15.46:0/2038786900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e32864a0 msgr2=0x5576e340c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:30.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f715effd700 1 --2- 172.21.15.46:0/2038786900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e32864a0 0x5576e340c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f715effd700 1 -- 172.21.15.46:0/2038786900 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e34036a0 msgr2=0x5576e3419430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:30.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f715effd700 1 --2- 172.21.15.46:0/2038786900 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e34036a0 0x5576e3419430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f715effd700 1 -- 172.21.15.46:0/2038786900 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576e34100e0 con 0x5576e3370160 2024-05-09T04:58:30.399 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f715f7fe700 1 --2- 172.21.15.46:0/2038786900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e32864a0 0x5576e340c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:58:30.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.397+0000 7f715effd700 1 --2- 172.21.15.46:0/2038786900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3370160 0x5576e34077c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7160109ac0 tx=0x7f7154012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ef4d9773e5aca server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:30.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.398+0000 7f71457fa700 1 -- 172.21.15.46:0/2038786900 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154011b50 con 0x5576e3370160 2024-05-09T04:58:30.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.398+0000 7f71457fa700 1 -- 172.21.15.46:0/2038786900 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7154024070 con 0x5576e3370160 2024-05-09T04:58:30.400 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.398+0000 7f71457fa700 1 -- 172.21.15.46:0/2038786900 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71540206a0 con 0x5576e3370160 2024-05-09T04:58:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.398+0000 7f71746dcb80 1 -- 172.21.15.46:0/2038786900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3370160 msgr2=0x5576e34077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.398+0000 7f71746dcb80 1 --2- 172.21.15.46:0/2038786900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3370160 0x5576e34077c0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f7160109ac0 tx=0x7f7154012000 comp rx=0 tx=0).stop 2024-05-09T04:58:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.398+0000 7f71746dcb80 1 -- 172.21.15.46:0/2038786900 shutdown_connections 2024-05-09T04:58:30.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.398+0000 7f71746dcb80 1 --2- 172.21.15.46:0/2038786900 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e34036a0 0x5576e3419430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.398+0000 7f71746dcb80 1 --2- 172.21.15.46:0/2038786900 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e32864a0 0x5576e340c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.398+0000 7f71746dcb80 1 --2- 172.21.15.46:0/2038786900 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3370160 0x5576e34077c0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.399+0000 7f71746dcb80 1 -- 172.21.15.46:0/2038786900 >> 172.21.15.46:0/2038786900 conn(0x5576e32e66a0 msgr2=0x5576e3408860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.399+0000 7f71746dcb80 1 -- 172.21.15.46:0/2038786900 shutdown_connections 2024-05-09T04:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.399+0000 7f71746dcb80 1 -- 172.21.15.46:0/2038786900 wait complete. 2024-05-09T04:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.399+0000 7f71746dcb80 1 -- start start 2024-05-09T04:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.399+0000 7f71746dcb80 1 Processor -- start 2024-05-09T04:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.399+0000 7f71746dcb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e32864a0 0x5576e341d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.399+0000 7f71746dcb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3370160 0x5576e341def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.399+0000 7f71746dcb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e34036a0 0x5576e34856f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f71746dcb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f71601af660 con 0x5576e3370160 2024-05-09T04:58:30.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f71746dcb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f71601b0ad0 con 0x5576e32864a0 2024-05-09T04:58:30.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f71746dcb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f71601afd30 con 0x5576e34036a0 2024-05-09T04:58:30.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f71746dcb80 10 client.0 _ll_register_callbacks cb 0x5576e31976c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:30.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f715effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3370160 0x5576e341def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f715effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3370160 0x5576e341def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55820/0 (socket says 172.21.15.46:55820) 2024-05-09T04:58:30.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f715effd700 1 -- 172.21.15.46:0/1896310697 learned_addr learned my addr 172.21.15.46:0/1896310697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:30.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f715f7fe700 1 --2- 172.21.15.46:0/1896310697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e32864a0 0x5576e341d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f715ffff700 1 --2- 172.21.15.46:0/1896310697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e34036a0 0x5576e34856f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f715effd700 1 -- 172.21.15.46:0/1896310697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e34036a0 msgr2=0x5576e34856f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:30.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f715effd700 1 --2- 172.21.15.46:0/1896310697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e34036a0 0x5576e34856f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f715effd700 1 -- 172.21.15.46:0/1896310697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e32864a0 msgr2=0x5576e341d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:30.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f715effd700 1 --2- 172.21.15.46:0/1896310697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e32864a0 0x5576e341d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f715effd700 1 -- 172.21.15.46:0/1896310697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576e3485c30 con 0x5576e3370160 2024-05-09T04:58:30.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f715ffff700 1 --2- 172.21.15.46:0/1896310697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e34036a0 0x5576e34856f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:58:30.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f715effd700 1 --2- 172.21.15.46:0/1896310697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3370160 0x5576e341def0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7154029cf0 tx=0x7f7154029750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170800edc0ca6813 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:30.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f713effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:30.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f713effd700 1 -- 172.21.15.46:0/1896310697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7154037060 con 0x5576e3370160 2024-05-09T04:58:30.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f71746dcb80 1 -- 172.21.15.46:0/1896310697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7154024070 con 0x5576e3370160 2024-05-09T04:58:30.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.400+0000 7f713effd700 1 -- 172.21.15.46:0/1896310697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f7154011c20 con 0x5576e3370160 2024-05-09T04:58:30.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.401+0000 7f713effd700 1 -- 172.21.15.46:0/1896310697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7154005050 con 0x5576e3370160 2024-05-09T04:58:30.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.401+0000 7f71746dcb80 10 client.17025 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:30.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.401+0000 7f71746dcb80 1 -- 172.21.15.46:0/1896310697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5576e3486de0 con 0x5576e3370160 2024-05-09T04:58:30.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.401+0000 7f713effd700 1 -- 172.21.15.46:0/1896310697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7154024070 con 0x5576e3370160 2024-05-09T04:58:30.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.401+0000 7f713effd700 1 -- 172.21.15.46:0/1896310697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7154004cb0 con 0x5576e3370160 2024-05-09T04:58:30.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.401+0000 7f71746dcb80 10 client.17025 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:30.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.401+0000 7f71746dcb80 10 client.17025 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:58:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.401+0000 7f71746dcb80 20 client.17025 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.401+0000 7f71746dcb80 1 --2- 172.21.15.46:0/1896310697 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5576e341d1c0 0x5576e3489460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.401+0000 7f71746dcb80 4 client.17025 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:58:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.401+0000 7f71746dcb80 1 -- 172.21.15.46:0/1896310697 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576e332da70 con 0x5576e341d1c0 2024-05-09T04:58:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.401+0000 7f715f7fe700 1 --2- 172.21.15.46:0/1896310697 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5576e341d1c0 0x5576e3489460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f713effd700 10 client.17025 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:58:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f715f7fe700 1 --2- 172.21.15.46:0/1896310697 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5576e341d1c0 0x5576e3489460 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9e4d9d5979f6b64b server_cookie=237c23a4481fe11a in_seq=0 out_seq=0 2024-05-09T04:58:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f713effd700 1 -- 172.21.15.46:0/1896310697 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576e332da70 con 0x5576e341d1c0 2024-05-09T04:58:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f713effd700 10 client.17025 handle_command_reply: tid=0 2024-05-09T04:58:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f71746dcb80 1 client.17025 shutdown 2024-05-09T04:58:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f71746dcb80 1 -- 172.21.15.46:0/1896310697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3370160 msgr2=0x5576e341def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f71746dcb80 1 --2- 172.21.15.46:0/1896310697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3370160 0x5576e341def0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f7154029cf0 tx=0x7f7154029750 comp rx=0 tx=0).stop 2024-05-09T04:58:30.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f71746dcb80 1 -- 172.21.15.46:0/1896310697 shutdown_connections 2024-05-09T04:58:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f71746dcb80 1 --2- 172.21.15.46:0/1896310697 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x5576e341d1c0 0x5576e3489460 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f71746dcb80 1 --2- 172.21.15.46:0/1896310697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5576e34036a0 0x5576e34856f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f71746dcb80 1 --2- 172.21.15.46:0/1896310697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5576e32864a0 0x5576e341d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f713effd700 0 client.17025 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:58:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f71746dcb80 1 --2- 172.21.15.46:0/1896310697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5576e3370160 0x5576e341def0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f71746dcb80 1 -- 172.21.15.46:0/1896310697 >> 172.21.15.46:0/1896310697 conn(0x5576e32e66a0 msgr2=0x5576e3406e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f71746dcb80 1 -- 172.21.15.46:0/1896310697 shutdown_connections 2024-05-09T04:58:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.402+0000 7f71746dcb80 1 -- 172.21.15.46:0/1896310697 wait complete. 2024-05-09T04:58:30.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.403+0000 7f71746dcb80 20 client.17025 trim_cache size 0 max 16384 2024-05-09T04:58:30.409 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:58:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.403+0000 7f7165cc6700 1 -- 172.21.15.46:0/2518676204 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f71480b2a70 msgr2=0x7f71480b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.403+0000 7f7165cc6700 1 --2- 172.21.15.46:0/2518676204 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f71480b2a70 0x7f71480b4f80 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7154005fd0 tx=0x7f7154005f60 comp rx=0 tx=0).stop 2024-05-09T04:58:30.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.403+0000 7f7165cc6700 1 -- 172.21.15.46:0/2518676204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71601034c0 msgr2=0x7f71600ff2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.403+0000 7f7165cc6700 1 --2- 172.21.15.46:0/2518676204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71601034c0 0x7f71600ff2d0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f71500098e0 tx=0x7f71500057c0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.403+0000 7f7165cc6700 1 -- 172.21.15.46:0/2518676204 shutdown_connections 2024-05-09T04:58:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.403+0000 7f7165cc6700 1 --2- 172.21.15.46:0/2518676204 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f71480b2a70 0x7f71480b4f80 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.403+0000 7f7165cc6700 1 --2- 172.21.15.46:0/2518676204 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7160108740 0x7f71600ffd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.403+0000 7f7165cc6700 1 --2- 172.21.15.46:0/2518676204 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f71601034c0 0x7f71600ff2d0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.403+0000 7f7165cc6700 1 --2- 172.21.15.46:0/2518676204 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7160107df0 0x7f71600ff810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.403+0000 7f7165cc6700 1 -- 172.21.15.46:0/2518676204 >> 172.21.15.46:0/2518676204 conn(0x7f71600fedf0 msgr2=0x7f7160075d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.403+0000 7f7165cc6700 1 -- 172.21.15.46:0/2518676204 shutdown_connections 2024-05-09T04:58:30.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:30.403+0000 7f7165cc6700 1 -- 172.21.15.46:0/2518676204 wait complete. 2024-05-09T04:58:31.160 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:58:31.413 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:32.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:32 smithi046 ceph-mon[30136]: pgmap v228: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T04:58:33.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:32 smithi110 ceph-mon[34904]: pgmap v228: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T04:58:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.086+0000 7f9b8c639700 1 -- 172.21.15.46:0/1248331201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b841034c0 msgr2=0x7f9b841038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:33.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.086+0000 7f9b8c639700 1 --2- 172.21.15.46:0/1248331201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b841034c0 0x7f9b841038d0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f9b800099c0 tx=0x7f9b80009cd0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.086+0000 7f9b8c639700 1 -- 172.21.15.46:0/1248331201 shutdown_connections 2024-05-09T04:58:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.086+0000 7f9b8c639700 1 --2- 172.21.15.46:0/1248331201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b84108740 0x7f9b841100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.086+0000 7f9b8c639700 1 --2- 172.21.15.46:0/1248331201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b841034c0 0x7f9b841038d0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.086+0000 7f9b8c639700 1 --2- 172.21.15.46:0/1248331201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b84107df0 0x7f9b84108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.086+0000 7f9b8c639700 1 -- 172.21.15.46:0/1248331201 >> 172.21.15.46:0/1248331201 conn(0x7f9b840fedf0 msgr2=0x7f9b84101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.087+0000 7f9b8c639700 1 -- 172.21.15.46:0/1248331201 shutdown_connections 2024-05-09T04:58:33.088 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.087+0000 7f9b8c639700 1 -- 172.21.15.46:0/1248331201 wait complete. 2024-05-09T04:58:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.087+0000 7f9b8c639700 1 Processor -- start 2024-05-09T04:58:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.088+0000 7f9b8c639700 1 -- start start 2024-05-09T04:58:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.088+0000 7f9b8c639700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b841034c0 0x7f9b8419bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.088+0000 7f9b8c639700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b84107df0 0x7f9b8419c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.088+0000 7f9b8c639700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b84108740 0x7f9b841a6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.088+0000 7f9b8c639700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b84113370 con 0x7f9b84108740 2024-05-09T04:58:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.088+0000 7f9b8c639700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841131f0 con 0x7f9b84107df0 2024-05-09T04:58:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.088+0000 7f9b8c639700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841134f0 con 0x7f9b841034c0 2024-05-09T04:58:33.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.088+0000 7f9b89bd4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b84107df0 0x7f9b8419c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.088+0000 7f9b89bd4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b84107df0 0x7f9b8419c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:56752/0 (socket says 172.21.15.46:56752) 2024-05-09T04:58:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.088+0000 7f9b8abd6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b84108740 0x7f9b841a6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.088+0000 7f9b89bd4700 1 -- 172.21.15.46:0/731597812 learned_addr learned my addr 172.21.15.46:0/731597812 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.088+0000 7f9b8a3d5700 1 --2- 172.21.15.46:0/731597812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b841034c0 0x7f9b8419bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.089+0000 7f9b8abd6700 1 -- 172.21.15.46:0/731597812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b841034c0 msgr2=0x7f9b8419bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.089+0000 7f9b8abd6700 1 --2- 172.21.15.46:0/731597812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b841034c0 0x7f9b8419bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.089+0000 7f9b8abd6700 1 -- 172.21.15.46:0/731597812 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b84107df0 msgr2=0x7f9b8419c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.089+0000 7f9b8abd6700 1 --2- 172.21.15.46:0/731597812 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b84107df0 0x7f9b8419c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.089+0000 7f9b8abd6700 1 -- 172.21.15.46:0/731597812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b8419c970 con 0x7f9b84108740 2024-05-09T04:58:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.089+0000 7f9b8a3d5700 1 --2- 172.21.15.46:0/731597812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b841034c0 0x7f9b8419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.089+0000 7f9b8abd6700 1 --2- 172.21.15.46:0/731597812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b84108740 0x7f9b841a6c90 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c007f10 tx=0x7f9b7c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.089+0000 7f9b7b7fe700 1 -- 172.21.15.46:0/731597812 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c00ddd0 con 0x7f9b84108740 2024-05-09T04:58:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.089+0000 7f9b7b7fe700 1 -- 172.21.15.46:0/731597812 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b7c009e20 con 0x7f9b84108740 2024-05-09T04:58:33.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.089+0000 7f9b7b7fe700 1 -- 172.21.15.46:0/731597812 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c015400 con 0x7f9b84108740 2024-05-09T04:58:33.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.089+0000 7f9b8c639700 1 -- 172.21.15.46:0/731597812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b841a7230 con 0x7f9b84108740 2024-05-09T04:58:33.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.089+0000 7f9b8c639700 1 -- 172.21.15.46:0/731597812 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b841a7720 con 0x7f9b84108740 2024-05-09T04:58:33.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.092+0000 7f9b7b7fe700 1 -- 172.21.15.46:0/731597812 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9b7c005020 con 0x7f9b84108740 2024-05-09T04:58:33.093 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.093+0000 7f9b7b7fe700 1 --2- 172.21.15.46:0/731597812 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9b700b2a70 0x7f9b700b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.093+0000 7f9b8a3d5700 1 --2- 172.21.15.46:0/731597812 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9b700b2a70 0x7f9b700b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.093+0000 7f9b7b7fe700 1 -- 172.21.15.46:0/731597812 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f9b7c0e1ff0 con 0x7f9b84108740 2024-05-09T04:58:33.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.093+0000 7f9b9b04fb80 1 Processor -- start 2024-05-09T04:58:33.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.094+0000 7f9b9b04fb80 1 -- start start 2024-05-09T04:58:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.094+0000 7f9b8a3d5700 1 --2- 172.21.15.46:0/731597812 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9b700b2a70 0x7f9b700b4f80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9b80000c00 tx=0x7f9b8001a040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.094+0000 7f9b9b04fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e210160 0x56321e2a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.094+0000 7f9b9b04fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e1264a0 0x56321e2a9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.094+0000 7f9b9b04fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e2a36a0 0x56321e2a3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.094+0000 7f9b9b04fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56321e1c5ff0 con 0x56321e1264a0 2024-05-09T04:58:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.094+0000 7f9b9b04fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56321e1647f0 con 0x56321e210160 2024-05-09T04:58:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.094+0000 7f9b9b04fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56321e1c1fb0 con 0x56321e2a36a0 2024-05-09T04:58:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.094+0000 7f9b89bd4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e1264a0 0x56321e2a9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.094+0000 7f9b89bd4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e1264a0 0x56321e2a9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55854/0 (socket says 172.21.15.46:55854) 2024-05-09T04:58:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.094+0000 7f9b8abd6700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e2a36a0 0x56321e2a3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.094+0000 7f9b89bd4700 1 -- 172.21.15.46:0/3233805911 learned_addr learned my addr 172.21.15.46:0/3233805911 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.094+0000 7f9b8a3d5700 1 --2- 172.21.15.46:0/3233805911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e210160 0x56321e2a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.094+0000 7f9b89bd4700 1 -- 172.21.15.46:0/3233805911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e2a36a0 msgr2=0x56321e2a3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.095+0000 7f9b89bd4700 1 --2- 172.21.15.46:0/3233805911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e2a36a0 0x56321e2a3ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.095+0000 7f9b89bd4700 1 -- 172.21.15.46:0/3233805911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e210160 msgr2=0x56321e2a94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.095+0000 7f9b89bd4700 1 --2- 172.21.15.46:0/3233805911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e210160 0x56321e2a94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.095+0000 7f9b89bd4700 1 -- 172.21.15.46:0/3233805911 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321e15c370 con 0x56321e1264a0 2024-05-09T04:58:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.095+0000 7f9b89bd4700 1 --2- 172.21.15.46:0/3233805911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e1264a0 0x56321e2a9a30 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400adb0 tx=0x7f9b7400f6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c2b80282728863a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.095+0000 7f9b78ff9700 1 -- 172.21.15.46:0/3233805911 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7401d070 con 0x56321e1264a0 2024-05-09T04:58:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.095+0000 7f9b78ff9700 1 -- 172.21.15.46:0/3233805911 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b740188f0 con 0x56321e1264a0 2024-05-09T04:58:33.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.095+0000 7f9b78ff9700 1 -- 172.21.15.46:0/3233805911 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b740108a0 con 0x56321e1264a0 2024-05-09T04:58:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.096+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/3233805911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e1264a0 msgr2=0x56321e2a9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.096+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/3233805911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e1264a0 0x56321e2a9a30 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400adb0 tx=0x7f9b7400f6a0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.096+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/3233805911 shutdown_connections 2024-05-09T04:58:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.096+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/3233805911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e2a36a0 0x56321e2a3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.096+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/3233805911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e210160 0x56321e2a94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.096+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/3233805911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e1264a0 0x56321e2a9a30 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.096+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/3233805911 >> 172.21.15.46:0/3233805911 conn(0x56321e1866a0 msgr2=0x56321e29e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.096+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/3233805911 shutdown_connections 2024-05-09T04:58:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.096+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/3233805911 wait complete. 2024-05-09T04:58:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.096+0000 7f9b9b04fb80 1 -- start start 2024-05-09T04:58:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.096+0000 7f9b9b04fb80 1 Processor -- start 2024-05-09T04:58:33.100 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b9b04fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e1264a0 0x56321e2ad210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b9b04fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e210160 0x56321e2ad750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b9b04fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e2a36a0 0x56321e2adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b9b04fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56321e041ba0 con 0x56321e1264a0 2024-05-09T04:58:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b9b04fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56321e180150 con 0x56321e2a36a0 2024-05-09T04:58:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b9b04fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56321e2b01d0 con 0x56321e210160 2024-05-09T04:58:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b9b04fb80 10 client.0 _ll_register_callbacks cb 0x56321e0376c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b8a3d5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e1264a0 0x56321e2ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b8a3d5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e1264a0 0x56321e2ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55860/0 (socket says 172.21.15.46:55860) 2024-05-09T04:58:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b8a3d5700 1 -- 172.21.15.46:0/2164922697 learned_addr learned my addr 172.21.15.46:0/2164922697 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b8abd6700 1 --2- 172.21.15.46:0/2164922697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e2a36a0 0x56321e2adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b89bd4700 1 --2- 172.21.15.46:0/2164922697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e210160 0x56321e2ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b8a3d5700 1 -- 172.21.15.46:0/2164922697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e210160 msgr2=0x56321e2ad750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b8a3d5700 1 --2- 172.21.15.46:0/2164922697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e210160 0x56321e2ad750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.101 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b8a3d5700 1 -- 172.21.15.46:0/2164922697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e2a36a0 msgr2=0x56321e2adc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b8a3d5700 1 --2- 172.21.15.46:0/2164922697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e2a36a0 0x56321e2adc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.097+0000 7f9b8a3d5700 1 -- 172.21.15.46:0/2164922697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321e041490 con 0x56321e1264a0 2024-05-09T04:58:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.098+0000 7f9b8a3d5700 1 --2- 172.21.15.46:0/2164922697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e1264a0 0x56321e2ad210 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9b800098e0 tx=0x7f9b8001e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a030eb335698dba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.098+0000 7f9b6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.098+0000 7f9b6dffb700 1 -- 172.21.15.46:0/2164922697 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b80003c70 con 0x56321e1264a0 2024-05-09T04:58:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.098+0000 7f9b6dffb700 1 -- 172.21.15.46:0/2164922697 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f9b80005030 con 0x56321e1264a0 2024-05-09T04:58:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.098+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2164922697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b740188f0 con 0x56321e1264a0 2024-05-09T04:58:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.098+0000 7f9b6dffb700 1 -- 172.21.15.46:0/2164922697 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9b80003e10 con 0x56321e1264a0 2024-05-09T04:58:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.098+0000 7f9b9b04fb80 10 client.17037 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.098+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2164922697 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56321e2bac30 con 0x56321e1264a0 2024-05-09T04:58:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.099+0000 7f9b6dffb700 1 -- 172.21.15.46:0/2164922697 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b740188f0 con 0x56321e1264a0 2024-05-09T04:58:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.099+0000 7f9b6dffb700 1 -- 172.21.15.46:0/2164922697 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9b80022070 con 0x56321e1264a0 2024-05-09T04:58:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.099+0000 7f9b9b04fb80 10 client.17037 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:33.102 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.099+0000 7f9b9b04fb80 10 client.17037 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:58:33.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.099+0000 7f9b9b04fb80 20 client.17037 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:33.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.099+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/2164922697 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56321e2bd270 0x56321e2bf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.099+0000 7f9b9b04fb80 4 client.17037 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:58:33.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.099+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2164922697 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56321e1d39d0 con 0x56321e2bd270 2024-05-09T04:58:33.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.099+0000 7f9b89bd4700 1 --2- 172.21.15.46:0/2164922697 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56321e2bd270 0x56321e2bf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.100+0000 7f9b89bd4700 1 --2- 172.21.15.46:0/2164922697 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56321e2bd270 0x56321e2bf690 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=654534542fc9cc67 server_cookie=992874dd7981892c in_seq=0 out_seq=0 2024-05-09T04:58:33.103 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.100+0000 7f9b6dffb700 10 client.17037 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.100+0000 7f9b6dffb700 1 -- 172.21.15.46:0/2164922697 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56321e1d39d0 con 0x56321e2bd270 2024-05-09T04:58:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.100+0000 7f9b6dffb700 10 client.17037 handle_command_reply: tid=0 2024-05-09T04:58:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.100+0000 7f9b9b04fb80 1 client.17037 shutdown 2024-05-09T04:58:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.101+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2164922697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e1264a0 msgr2=0x56321e2ad210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.101+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/2164922697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e1264a0 0x56321e2ad210 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f9b800098e0 tx=0x7f9b8001e000 comp rx=0 tx=0).stop 2024-05-09T04:58:33.104 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.101+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2164922697 shutdown_connections 2024-05-09T04:58:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.101+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/2164922697 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56321e2bd270 0x56321e2bf690 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.101+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/2164922697 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e2a36a0 0x56321e2adc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.101+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/2164922697 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e1264a0 0x56321e2ad210 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.101+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/2164922697 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e210160 0x56321e2ad750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.101+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2164922697 >> 172.21.15.46:0/2164922697 conn(0x56321e1866a0 msgr2=0x56321e29e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.101+0000 7f9b6dffb700 0 client.17037 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.101+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2164922697 shutdown_connections 2024-05-09T04:58:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.101+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2164922697 wait complete. 2024-05-09T04:58:33.105 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.101+0000 7f9b9b04fb80 20 client.17037 trim_cache size 0 max 16384 2024-05-09T04:58:33.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.116+0000 7f9b9b04fb80 1 Processor -- start 2024-05-09T04:58:33.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.116+0000 7f9b9b04fb80 1 -- start start 2024-05-09T04:58:33.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.116+0000 7f9b9b04fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e1264a0 0x56321e2a8950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.116+0000 7f9b9b04fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e210160 0x56321e2adad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.116+0000 7f9b9b04fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e2a36a0 0x56321e2a6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.117 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.116+0000 7f9b9b04fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841b16c0 con 0x56321e2a36a0 2024-05-09T04:58:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.116+0000 7f9b9b04fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841aef50 con 0x56321e1264a0 2024-05-09T04:58:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.116+0000 7f9b9b04fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841acdc0 con 0x56321e210160 2024-05-09T04:58:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.116+0000 7f9b8abd6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e2a36a0 0x56321e2a6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.116+0000 7f9b8abd6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e2a36a0 0x56321e2a6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55876/0 (socket says 172.21.15.46:55876) 2024-05-09T04:58:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.117+0000 7f9b8abd6700 1 -- 172.21.15.46:0/914953062 learned_addr learned my addr 172.21.15.46:0/914953062 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.117+0000 7f9b8abd6700 1 -- 172.21.15.46:0/914953062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e210160 msgr2=0x56321e2adad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.117+0000 7f9b89bd4700 1 --2- 172.21.15.46:0/914953062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e210160 0x56321e2adad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.117+0000 7f9b8a3d5700 1 --2- 172.21.15.46:0/914953062 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e1264a0 0x56321e2a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.117+0000 7f9b8abd6700 1 --2- 172.21.15.46:0/914953062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e210160 0x56321e2adad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.117+0000 7f9b8abd6700 1 -- 172.21.15.46:0/914953062 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e1264a0 msgr2=0x56321e2a8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.117+0000 7f9b8abd6700 1 --2- 172.21.15.46:0/914953062 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e1264a0 0x56321e2a8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.117+0000 7f9b8abd6700 1 -- 172.21.15.46:0/914953062 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56321e29fd00 con 0x56321e2a36a0 2024-05-09T04:58:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.117+0000 7f9b89bd4700 1 --2- 172.21.15.46:0/914953062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e210160 0x56321e2adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:58:33.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.117+0000 7f9b8abd6700 1 --2- 172.21.15.46:0/914953062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e2a36a0 0x56321e2a6e00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0df710 tx=0x7f9b7c0a92f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88d471aa43d2fe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:33.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.117+0000 7f9b6ffff700 1 -- 172.21.15.46:0/914953062 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0c2070 con 0x56321e2a36a0 2024-05-09T04:58:33.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.117+0000 7f9b6ffff700 1 -- 172.21.15.46:0/914953062 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b7c0a5830 con 0x56321e2a36a0 2024-05-09T04:58:33.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.117+0000 7f9b6ffff700 1 -- 172.21.15.46:0/914953062 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0a94a0 con 0x56321e2a36a0 2024-05-09T04:58:33.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.118+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/914953062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e2a36a0 msgr2=0x56321e2a6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:33.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.118+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/914953062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e2a36a0 0x56321e2a6e00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0df710 tx=0x7f9b7c0a92f0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.118+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/914953062 shutdown_connections 2024-05-09T04:58:33.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.118+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/914953062 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e2a36a0 0x56321e2a6e00 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.118+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/914953062 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e1264a0 0x56321e2a8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.118+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/914953062 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e210160 0x56321e2adad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.118+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/914953062 >> 172.21.15.46:0/914953062 conn(0x56321e1866a0 msgr2=0x56321e2a8240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.118+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/914953062 shutdown_connections 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.118+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/914953062 wait complete. 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.118+0000 7f9b9b04fb80 1 -- start start 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b9b04fb80 1 Processor -- start 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b9b04fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e1264a0 0x56321e2b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b9b04fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e210160 0x56321e2b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b9b04fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e2a36a0 0x56321e2c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b9b04fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841ad410 con 0x56321e2a36a0 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b9b04fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841ae880 con 0x56321e210160 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b9b04fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9b841adae0 con 0x56321e1264a0 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b9b04fb80 10 client.0 _ll_register_callbacks cb 0x56321e0376c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b8abd6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e2a36a0 0x56321e2c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b8abd6700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e2a36a0 0x56321e2c3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:55890/0 (socket says 172.21.15.46:55890) 2024-05-09T04:58:33.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b8abd6700 1 -- 172.21.15.46:0/2243001405 learned_addr learned my addr 172.21.15.46:0/2243001405 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b89bd4700 1 --2- 172.21.15.46:0/2243001405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e210160 0x56321e2b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b8a3d5700 1 --2- 172.21.15.46:0/2243001405 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e1264a0 0x56321e2b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b8abd6700 1 -- 172.21.15.46:0/2243001405 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e1264a0 msgr2=0x56321e2b54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b8abd6700 1 --2- 172.21.15.46:0/2243001405 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e1264a0 0x56321e2b54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b8abd6700 1 -- 172.21.15.46:0/2243001405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e210160 msgr2=0x56321e2b5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b8abd6700 1 --2- 172.21.15.46:0/2243001405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e210160 0x56321e2b5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b8abd6700 1 -- 172.21.15.46:0/2243001405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56321e2c4650 con 0x56321e2a36a0 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.119+0000 7f9b89bd4700 1 --2- 172.21.15.46:0/2243001405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e210160 0x56321e2b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b8abd6700 1 --2- 172.21.15.46:0/2243001405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e2a36a0 0x56321e2c3f00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0ad7b0 tx=0x7f9b7c0a9140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ae2b20b86e6c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b6dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b6dffb700 1 -- 172.21.15.46:0/2243001405 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b7c0c2070 con 0x56321e2a36a0 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b6dffb700 1 -- 172.21.15.46:0/2243001405 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f9b7c0a94a0 con 0x56321e2a36a0 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2243001405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b7c0a5830 con 0x56321e2a36a0 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b6dffb700 1 -- 172.21.15.46:0/2243001405 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9b7c0a96e0 con 0x56321e2a36a0 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b9b04fb80 10 client.17043 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:33.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2243001405 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56321e327350 con 0x56321e2a36a0 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b6dffb700 1 -- 172.21.15.46:0/2243001405 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b7c0a5830 con 0x56321e2a36a0 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b6dffb700 1 -- 172.21.15.46:0/2243001405 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9b7c0ad250 con 0x56321e2a36a0 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b9b04fb80 10 client.17043 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b9b04fb80 10 client.17043 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b9b04fb80 20 client.17043 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/2243001405 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56321e327980 0x56321e329da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b9b04fb80 4 client.17043 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.120+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2243001405 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56321e1cda70 con 0x56321e327980 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.121+0000 7f9b8a3d5700 1 --2- 172.21.15.46:0/2243001405 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56321e327980 0x56321e329da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.121+0000 7f9b8a3d5700 1 --2- 172.21.15.46:0/2243001405 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56321e327980 0x56321e329da0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3661655a0f051518 server_cookie=e2635d04c955eb27 in_seq=0 out_seq=0 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.121+0000 7f9b6dffb700 10 client.17043 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.121+0000 7f9b6dffb700 1 -- 172.21.15.46:0/2243001405 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56321e1cda70 con 0x56321e327980 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.121+0000 7f9b6dffb700 10 client.17043 handle_command_reply: tid=0 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.121+0000 7f9b9b04fb80 1 client.17043 shutdown 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2243001405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e2a36a0 msgr2=0x56321e2c3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:33.125 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/2243001405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e2a36a0 0x56321e2c3f00 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c0ad7b0 tx=0x7f9b7c0a9140 comp rx=0 tx=0).stop 2024-05-09T04:58:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2243001405 shutdown_connections 2024-05-09T04:58:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/2243001405 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56321e2a36a0 0x56321e2c3f00 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/2243001405 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56321e1264a0 0x56321e2b54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/2243001405 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56321e327980 0x56321e329da0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b6dffb700 0 client.17043 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b9b04fb80 1 --2- 172.21.15.46:0/2243001405 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56321e210160 0x56321e2b5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2243001405 >> 172.21.15.46:0/2243001405 conn(0x56321e1866a0 msgr2=0x56321e2a6e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2243001405 shutdown_connections 2024-05-09T04:58:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b9b04fb80 1 -- 172.21.15.46:0/2243001405 wait complete. 2024-05-09T04:58:33.126 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b9b04fb80 20 client.17043 trim_cache size 0 max 16384 2024-05-09T04:58:33.126 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:58:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b8c639700 1 -- 172.21.15.46:0/731597812 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9b700b2a70 msgr2=0x7f9b700b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b8c639700 1 --2- 172.21.15.46:0/731597812 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9b700b2a70 0x7f9b700b4f80 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9b80000c00 tx=0x7f9b8001a040 comp rx=0 tx=0).stop 2024-05-09T04:58:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b8c639700 1 -- 172.21.15.46:0/731597812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b84108740 msgr2=0x7f9b841a6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b8c639700 1 --2- 172.21.15.46:0/731597812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b84108740 0x7f9b841a6c90 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c007f10 tx=0x7f9b7c00d3b0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b8c639700 1 -- 172.21.15.46:0/731597812 shutdown_connections 2024-05-09T04:58:33.127 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b8c639700 1 --2- 172.21.15.46:0/731597812 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9b700b2a70 0x7f9b700b4f80 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b8c639700 1 --2- 172.21.15.46:0/731597812 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9b84108740 0x7f9b841a6c90 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b8c639700 1 --2- 172.21.15.46:0/731597812 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9b841034c0 0x7f9b8419bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b8c639700 1 --2- 172.21.15.46:0/731597812 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9b84107df0 0x7f9b8419c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:33.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.122+0000 7f9b8c639700 1 -- 172.21.15.46:0/731597812 >> 172.21.15.46:0/731597812 conn(0x7f9b840fedf0 msgr2=0x7f9b84101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:33.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.123+0000 7f9b8c639700 1 -- 172.21.15.46:0/731597812 shutdown_connections 2024-05-09T04:58:33.128 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:33.123+0000 7f9b8c639700 1 -- 172.21.15.46:0/731597812 wait complete. 2024-05-09T04:58:33.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:32 smithi072 ceph-mon[34088]: pgmap v228: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T04:58:34.821 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:58:34.821 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:58:35.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:34 smithi110 ceph-mon[34904]: pgmap v229: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:58:35.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:34 smithi046 ceph-mon[30136]: pgmap v229: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:58:35.087 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:35.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:34 smithi072 ceph-mon[34088]: pgmap v229: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:58:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.740+0000 7f7d2136b700 1 -- 172.21.15.46:0/2086485908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1c107e10 msgr2=0x7f7d1c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.740+0000 7f7d2136b700 1 --2- 172.21.15.46:0/2086485908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1c107e10 0x7f7d1c108220 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f7d100099c0 tx=0x7f7d10009cd0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.740+0000 7f7d2136b700 1 -- 172.21.15.46:0/2086485908 shutdown_connections 2024-05-09T04:58:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.740+0000 7f7d2136b700 1 --2- 172.21.15.46:0/2086485908 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1c108760 0x7f7d1c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.740+0000 7f7d2136b700 1 --2- 172.21.15.46:0/2086485908 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d1c1034e0 0x7f7d1c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.740+0000 7f7d2136b700 1 --2- 172.21.15.46:0/2086485908 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1c107e10 0x7f7d1c108220 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.740+0000 7f7d2136b700 1 -- 172.21.15.46:0/2086485908 >> 172.21.15.46:0/2086485908 conn(0x7f7d1c0fef30 msgr2=0x7f7d1c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.740+0000 7f7d2136b700 1 -- 172.21.15.46:0/2086485908 shutdown_connections 2024-05-09T04:58:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.740+0000 7f7d2136b700 1 -- 172.21.15.46:0/2086485908 wait complete. 2024-05-09T04:58:36.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.741+0000 7f7d2136b700 1 Processor -- start 2024-05-09T04:58:36.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d2136b700 1 -- start start 2024-05-09T04:58:36.743 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d2136b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1c1034e0 0x7f7d1c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d2136b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1c107e10 0x7f7d1c19c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d2136b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d1c108760 0x7f7d1c19c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d2136b700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c113370 con 0x7f7d1c107e10 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d2136b700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c1131f0 con 0x7f7d1c1034e0 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d2136b700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c1134f0 con 0x7f7d1c108760 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d1a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1c107e10 0x7f7d1c19c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d1affd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1c1034e0 0x7f7d1c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d1affd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1c1034e0 0x7f7d1c19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:34444/0 (socket says 172.21.15.46:34444) 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d1affd700 1 -- 172.21.15.46:0/3083049132 learned_addr learned my addr 172.21.15.46:0/3083049132 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d1b7fe700 1 --2- 172.21.15.46:0/3083049132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d1c108760 0x7f7d1c19c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d1a7fc700 1 -- 172.21.15.46:0/3083049132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d1c108760 msgr2=0x7f7d1c19c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d1a7fc700 1 --2- 172.21.15.46:0/3083049132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d1c108760 0x7f7d1c19c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d1a7fc700 1 -- 172.21.15.46:0/3083049132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1c1034e0 msgr2=0x7f7d1c19bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d1a7fc700 1 --2- 172.21.15.46:0/3083049132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1c1034e0 0x7f7d1c19bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.742+0000 7f7d1a7fc700 1 -- 172.21.15.46:0/3083049132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d1c1a7080 con 0x7f7d1c107e10 2024-05-09T04:58:36.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.743+0000 7f7d1a7fc700 1 --2- 172.21.15.46:0/3083049132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1c107e10 0x7f7d1c19c250 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000b580 tx=0x7f7d100047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.743+0000 7f7d03fff700 1 -- 172.21.15.46:0/3083049132 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10027070 con 0x7f7d1c107e10 2024-05-09T04:58:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.743+0000 7f7d03fff700 1 -- 172.21.15.46:0/3083049132 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d10003f20 con 0x7f7d1c107e10 2024-05-09T04:58:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.743+0000 7f7d03fff700 1 -- 172.21.15.46:0/3083049132 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10022bc0 con 0x7f7d1c107e10 2024-05-09T04:58:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.743+0000 7f7d2136b700 1 -- 172.21.15.46:0/3083049132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d1c199b00 con 0x7f7d1c107e10 2024-05-09T04:58:36.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.743+0000 7f7d2136b700 1 -- 172.21.15.46:0/3083049132 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d1c199f60 con 0x7f7d1c107e10 2024-05-09T04:58:36.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.746+0000 7f7d03fff700 1 -- 172.21.15.46:0/3083049132 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7d1002a070 con 0x7f7d1c107e10 2024-05-09T04:58:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.747+0000 7f7d03fff700 1 --2- 172.21.15.46:0/3083049132 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7d040b6e00 0x7f7d040b9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.747+0000 7f7d2fd81b80 1 Processor -- start 2024-05-09T04:58:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.747+0000 7f7d1affd700 1 --2- 172.21.15.46:0/3083049132 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7d040b6e00 0x7f7d040b9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.747+0000 7f7d2fd81b80 1 -- start start 2024-05-09T04:58:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.747+0000 7f7d03fff700 1 -- 172.21.15.46:0/3083049132 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f7d100e34a0 con 0x7f7d1c107e10 2024-05-09T04:58:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d2fd81b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e90d80 0x55b734f992c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d2fd81b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734e16520 0x55b734f99800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d2fd81b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734f93480 0x55b734f93890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d2fd81b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b734eb6120 con 0x55b734f93480 2024-05-09T04:58:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d2fd81b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55b734e5ba20 con 0x55b734e16520 2024-05-09T04:58:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d2fd81b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b734eb20e0 con 0x55b734e90d80 2024-05-09T04:58:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d1affd700 1 --2- 172.21.15.46:0/3083049132 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7d040b6e00 0x7f7d040b9310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0094f0 tx=0x7f7d0c009040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d1b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734f93480 0x55b734f93890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d1b7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734f93480 0x55b734f93890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56518/0 (socket says 172.21.15.46:56518) 2024-05-09T04:58:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d1b7fe700 1 -- 172.21.15.46:0/131657758 learned_addr learned my addr 172.21.15.46:0/131657758 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d1a7fc700 1 --2- 172.21.15.46:0/131657758 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734e16520 0x55b734f99800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d1affd700 1 --2- 172.21.15.46:0/131657758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e90d80 0x55b734f992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d1b7fe700 1 -- 172.21.15.46:0/131657758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e90d80 msgr2=0x55b734f992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d1b7fe700 1 --2- 172.21.15.46:0/131657758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e90d80 0x55b734f992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d1b7fe700 1 -- 172.21.15.46:0/131657758 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734e16520 msgr2=0x55b734f99800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d1b7fe700 1 --2- 172.21.15.46:0/131657758 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734e16520 0x55b734f99800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d1b7fe700 1 -- 172.21.15.46:0/131657758 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b734e4c580 con 0x55b734f93480 2024-05-09T04:58:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d1affd700 1 --2- 172.21.15.46:0/131657758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e90d80 0x55b734f992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.748+0000 7f7d1b7fe700 1 --2- 172.21.15.46:0/131657758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734f93480 0x55b734f93890 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7d14004660 tx=0x7f7d1400b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40ce175a31871716 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.749+0000 7f7d017fa700 1 -- 172.21.15.46:0/131657758 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d14004a30 con 0x55b734f93480 2024-05-09T04:58:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.749+0000 7f7d017fa700 1 -- 172.21.15.46:0/131657758 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d14015070 con 0x55b734f93480 2024-05-09T04:58:36.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.749+0000 7f7d017fa700 1 -- 172.21.15.46:0/131657758 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d140038e0 con 0x55b734f93480 2024-05-09T04:58:36.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.750+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/131657758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734f93480 msgr2=0x55b734f93890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:36.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.750+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/131657758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734f93480 0x55b734f93890 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f7d14004660 tx=0x7f7d1400b970 comp rx=0 tx=0).stop 2024-05-09T04:58:36.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.750+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/131657758 shutdown_connections 2024-05-09T04:58:36.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.750+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/131657758 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734f93480 0x55b734f93890 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.750+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/131657758 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e90d80 0x55b734f992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.750+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/131657758 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734e16520 0x55b734f99800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.753 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.750+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/131657758 >> 172.21.15.46:0/131657758 conn(0x55b734e19ff0 msgr2=0x55b734f8e2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.750+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/131657758 shutdown_connections 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.750+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/131657758 wait complete. 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.750+0000 7f7d2fd81b80 1 -- start start 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.750+0000 7f7d2fd81b80 1 Processor -- start 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d2fd81b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e16520 0x55b734f9d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d2fd81b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e90d80 0x55b734f9d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d2fd81b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734f93480 0x55b734f9db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d2fd81b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b734f941e0 con 0x55b734e16520 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d2fd81b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55b734e50ba0 con 0x55b734f93480 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d2fd81b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b734fa00a0 con 0x55b734e90d80 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d2fd81b80 10 client.0 _ll_register_callbacks cb 0x55b734d27740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d1affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e16520 0x55b734f9d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d1affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e16520 0x55b734f9d110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56532/0 (socket says 172.21.15.46:56532) 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d1affd700 1 -- 172.21.15.46:0/2355816400 learned_addr learned my addr 172.21.15.46:0/2355816400 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d1a7fc700 1 --2- 172.21.15.46:0/2355816400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e90d80 0x55b734f9d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d1b7fe700 1 --2- 172.21.15.46:0/2355816400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734f93480 0x55b734f9db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d1affd700 1 -- 172.21.15.46:0/2355816400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e90d80 msgr2=0x55b734f9d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d1affd700 1 --2- 172.21.15.46:0/2355816400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e90d80 0x55b734f9d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d1affd700 1 -- 172.21.15.46:0/2355816400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734f93480 msgr2=0x55b734f9db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d1affd700 1 --2- 172.21.15.46:0/2355816400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734f93480 0x55b734f9db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d1affd700 1 -- 172.21.15.46:0/2355816400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b734d31510 con 0x55b734e16520 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d1affd700 1 --2- 172.21.15.46:0/2355816400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e16520 0x55b734f9d110 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c015e30 tx=0x7f7d0c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16c5e5b459c01cb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7cfeff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7cfeff5700 1 -- 172.21.15.46:0/2355816400 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c015e60 con 0x55b734e16520 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/2355816400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d14015070 con 0x55b734e16520 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.751+0000 7f7cfeff5700 1 -- 172.21.15.46:0/2355816400 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f7d0c022420 con 0x55b734e16520 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.752+0000 7f7cfeff5700 1 -- 172.21.15.46:0/2355816400 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d0c016e20 con 0x55b734e16520 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.752+0000 7f7d2fd81b80 10 client.17055 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.752+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/2355816400 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b734faab90 con 0x55b734e16520 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.752+0000 7f7cfeff5700 1 -- 172.21.15.46:0/2355816400 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d14015070 con 0x55b734e16520 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.753+0000 7f7cfeff5700 1 -- 172.21.15.46:0/2355816400 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7d0c02b720 con 0x55b734e16520 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.753+0000 7f7d2fd81b80 10 client.17055 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:36.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.753+0000 7f7d2fd81b80 10 client.17055 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:58:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.753+0000 7f7d2fd81b80 20 client.17055 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.753+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/2355816400 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b734fad120 0x55b734faf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.753+0000 7f7d2fd81b80 4 client.17055 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:58:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.753+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/2355816400 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b734ec3b00 con 0x55b734fad120 2024-05-09T04:58:36.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.753+0000 7f7d1a7fc700 1 --2- 172.21.15.46:0/2355816400 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b734fad120 0x55b734faf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.753+0000 7f7cfeff5700 10 client.17055 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.753+0000 7f7d1a7fc700 1 --2- 172.21.15.46:0/2355816400 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b734fad120 0x55b734faf540 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2b4acab9c987092 server_cookie=d5f1f700295a1397 in_seq=0 out_seq=0 2024-05-09T04:58:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.754+0000 7f7cfeff5700 1 -- 172.21.15.46:0/2355816400 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b734ec3b00 con 0x55b734fad120 2024-05-09T04:58:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.754+0000 7f7cfeff5700 10 client.17055 handle_command_reply: tid=0 2024-05-09T04:58:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.754+0000 7f7d2fd81b80 1 client.17055 shutdown 2024-05-09T04:58:36.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.754+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/2355816400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e16520 msgr2=0x55b734f9d110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.754+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/2355816400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e16520 0x55b734f9d110 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c015e30 tx=0x7f7d0c004b40 comp rx=0 tx=0).stop 2024-05-09T04:58:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.754+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/2355816400 shutdown_connections 2024-05-09T04:58:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.754+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/2355816400 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b734fad120 0x55b734faf540 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.754+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/2355816400 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734f93480 0x55b734f9db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.754+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/2355816400 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e16520 0x55b734f9d110 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.754+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/2355816400 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e90d80 0x55b734f9d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.754+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/2355816400 >> 172.21.15.46:0/2355816400 conn(0x55b734e19ff0 msgr2=0x55b734f88070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.754+0000 7f7cfeff5700 0 client.17055 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.755+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/2355816400 shutdown_connections 2024-05-09T04:58:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.755+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/2355816400 wait complete. 2024-05-09T04:58:36.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.755+0000 7f7d2fd81b80 20 client.17055 trim_cache size 0 max 16384 2024-05-09T04:58:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.775+0000 7f7d2fd81b80 1 Processor -- start 2024-05-09T04:58:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.775+0000 7f7d2fd81b80 1 -- start start 2024-05-09T04:58:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.775+0000 7f7d2fd81b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e16520 0x55b734f88aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d2fd81b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e90d80 0x55b734f98870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d2fd81b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734f93480 0x55b734f9d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d2fd81b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c1af680 con 0x55b734e90d80 2024-05-09T04:58:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d2fd81b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c1acf10 con 0x55b734f93480 2024-05-09T04:58:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d2fd81b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c1aad80 con 0x55b734e16520 2024-05-09T04:58:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d1a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e90d80 0x55b734f98870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d1a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e90d80 0x55b734f98870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56544/0 (socket says 172.21.15.46:56544) 2024-05-09T04:58:36.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d1a7fc700 1 -- 172.21.15.46:0/2954655676 learned_addr learned my addr 172.21.15.46:0/2954655676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d1affd700 1 --2- 172.21.15.46:0/2954655676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e16520 0x55b734f88aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d1b7fe700 1 --2- 172.21.15.46:0/2954655676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734f93480 0x55b734f9d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d1a7fc700 1 -- 172.21.15.46:0/2954655676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e16520 msgr2=0x55b734f88aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d1a7fc700 1 --2- 172.21.15.46:0/2954655676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e16520 0x55b734f88aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d1a7fc700 1 -- 172.21.15.46:0/2954655676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734f93480 msgr2=0x55b734f9d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d1a7fc700 1 --2- 172.21.15.46:0/2954655676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734f93480 0x55b734f9d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d1a7fc700 1 -- 172.21.15.46:0/2954655676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b734f8aa50 con 0x55b734e90d80 2024-05-09T04:58:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.776+0000 7f7d1a7fc700 1 --2- 172.21.15.46:0/2954655676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e90d80 0x55b734f98870 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7d100a79d0 tx=0x7f7d10009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd7273c69457c02a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.777+0000 7f7d00ff9700 1 -- 172.21.15.46:0/2954655676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d100aae60 con 0x55b734e90d80 2024-05-09T04:58:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.777+0000 7f7d00ff9700 1 -- 172.21.15.46:0/2954655676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d100aa760 con 0x55b734e90d80 2024-05-09T04:58:36.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.777+0000 7f7d00ff9700 1 -- 172.21.15.46:0/2954655676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d10022470 con 0x55b734e90d80 2024-05-09T04:58:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.777+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/2954655676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e90d80 msgr2=0x55b734f98870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.778+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/2954655676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e90d80 0x55b734f98870 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f7d100a79d0 tx=0x7f7d10009310 comp rx=0 tx=0).stop 2024-05-09T04:58:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.778+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/2954655676 shutdown_connections 2024-05-09T04:58:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.778+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/2954655676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734f93480 0x55b734f9d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.778+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/2954655676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734e16520 0x55b734f88aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.778+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/2954655676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e90d80 0x55b734f98870 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.778+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/2954655676 >> 172.21.15.46:0/2954655676 conn(0x55b734e19ff0 msgr2=0x55b734f92470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:36.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.778+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/2954655676 shutdown_connections 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.778+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/2954655676 wait complete. 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.778+0000 7f7d2fd81b80 1 -- start start 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.778+0000 7f7d2fd81b80 1 Processor -- start 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d2fd81b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734e16520 0x55b734fa52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d2fd81b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e90d80 0x55b734fa57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d2fd81b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734f93480 0x55b734fa2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d2fd81b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c1ab3d0 con 0x55b734e90d80 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d2fd81b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c1ac840 con 0x55b734e16520 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d2fd81b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7d1c1abaa0 con 0x55b734f93480 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d2fd81b80 10 client.0 _ll_register_callbacks cb 0x55b734d27740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d1a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e90d80 0x55b734fa57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d1affd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734e16520 0x55b734fa52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d1a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e90d80 0x55b734fa57e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56548/0 (socket says 172.21.15.46:56548) 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d1affd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734e16520 0x55b734fa52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:34474/0 (socket says 172.21.15.46:34474) 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d1a7fc700 1 -- 172.21.15.46:0/36383569 learned_addr learned my addr 172.21.15.46:0/36383569 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:36.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d1b7fe700 1 --2- 172.21.15.46:0/36383569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734f93480 0x55b734fa2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d1a7fc700 1 -- 172.21.15.46:0/36383569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734f93480 msgr2=0x55b734fa2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d1a7fc700 1 --2- 172.21.15.46:0/36383569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734f93480 0x55b734fa2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d1a7fc700 1 -- 172.21.15.46:0/36383569 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734e16520 msgr2=0x55b734fa52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d1a7fc700 1 --2- 172.21.15.46:0/36383569 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734e16520 0x55b734fa52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d1a7fc700 1 -- 172.21.15.46:0/36383569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b734fa27a0 con 0x55b734e90d80 2024-05-09T04:58:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d1affd700 1 --2- 172.21.15.46:0/36383569 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734e16520 0x55b734fa52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:58:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7d1a7fc700 1 --2- 172.21.15.46:0/36383569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e90d80 0x55b734fa57e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009fd0 tx=0x7f7d100aa5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4b1490173b5c07 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:36.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.779+0000 7f7cfeff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:36.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.782+0000 7f7cfeff5700 1 -- 172.21.15.46:0/36383569 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d100aae60 con 0x55b734e90d80 2024-05-09T04:58:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.782+0000 7f7cfeff5700 1 -- 172.21.15.46:0/36383569 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f7d100a6d50 con 0x55b734e90d80 2024-05-09T04:58:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.782+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/36383569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d100aa760 con 0x55b734e90d80 2024-05-09T04:58:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.782+0000 7f7cfeff5700 1 -- 172.21.15.46:0/36383569 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7d100c8070 con 0x55b734e90d80 2024-05-09T04:58:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.782+0000 7f7d2fd81b80 10 client.17061 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.782+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/36383569 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b735018340 con 0x55b734e90d80 2024-05-09T04:58:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.782+0000 7f7cfeff5700 1 -- 172.21.15.46:0/36383569 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d100aa760 con 0x55b734e90d80 2024-05-09T04:58:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.782+0000 7f7cfeff5700 1 -- 172.21.15.46:0/36383569 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7d100921b0 con 0x55b734e90d80 2024-05-09T04:58:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.782+0000 7f7d2fd81b80 10 client.17061 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.782+0000 7f7d2fd81b80 10 client.17061 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:58:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.782+0000 7f7d2fd81b80 20 client.17061 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.782+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/36383569 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b735018a80 0x55b73501aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.782+0000 7f7d2fd81b80 4 client.17061 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:58:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.782+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/36383569 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b734ebdba0 con 0x55b735018a80 2024-05-09T04:58:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.782+0000 7f7d1affd700 1 --2- 172.21.15.46:0/36383569 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b735018a80 0x55b73501aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:36.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.783+0000 7f7d1affd700 1 --2- 172.21.15.46:0/36383569 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b735018a80 0x55b73501aea0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=837219d6e2368de0 server_cookie=f2a90fbea7f0d39c in_seq=0 out_seq=0 2024-05-09T04:58:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.783+0000 7f7cfeff5700 10 client.17061 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:36.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.784+0000 7f7cfeff5700 1 -- 172.21.15.46:0/36383569 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b734ebdba0 con 0x55b735018a80 2024-05-09T04:58:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.784+0000 7f7cfeff5700 10 client.17061 handle_command_reply: tid=0 2024-05-09T04:58:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.784+0000 7f7d2fd81b80 1 client.17061 shutdown 2024-05-09T04:58:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.784+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/36383569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e90d80 msgr2=0x55b734fa57e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.784+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/36383569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e90d80 0x55b734fa57e0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f7d10009fd0 tx=0x7f7d100aa5c0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.785+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/36383569 shutdown_connections 2024-05-09T04:58:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.785+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/36383569 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b734f93480 0x55b734fa2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.785+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/36383569 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b735018a80 0x55b73501aea0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.785+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/36383569 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b734e16520 0x55b734fa52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.785+0000 7f7d2fd81b80 1 --2- 172.21.15.46:0/36383569 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b734e90d80 0x55b734fa57e0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.785+0000 7f7cfeff5700 0 client.17061 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.785+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/36383569 >> 172.21.15.46:0/36383569 conn(0x55b734e19ff0 msgr2=0x55b734f88800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.785+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/36383569 shutdown_connections 2024-05-09T04:58:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.785+0000 7f7d2fd81b80 1 -- 172.21.15.46:0/36383569 wait complete. 2024-05-09T04:58:36.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.785+0000 7f7d2fd81b80 20 client.17061 trim_cache size 0 max 16384 2024-05-09T04:58:36.787 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:58:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:58:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "9b7f376e-94a6-478e-b927-974a97904afb", 2024-05-09T04:58:36.787 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:58:36.787 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:58:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.786+0000 7f7d2136b700 1 -- 172.21.15.46:0/3083049132 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7d040b6e00 msgr2=0x7f7d040b9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.786+0000 7f7d2136b700 1 --2- 172.21.15.46:0/3083049132 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7d040b6e00 0x7f7d040b9310 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c0094f0 tx=0x7f7d0c009040 comp rx=0 tx=0).stop 2024-05-09T04:58:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.786+0000 7f7d2136b700 1 -- 172.21.15.46:0/3083049132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1c107e10 msgr2=0x7f7d1c19c250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.786+0000 7f7d2136b700 1 --2- 172.21.15.46:0/3083049132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1c107e10 0x7f7d1c19c250 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f7d1000b580 tx=0x7f7d100047b0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.786+0000 7f7d2136b700 1 -- 172.21.15.46:0/3083049132 shutdown_connections 2024-05-09T04:58:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.786+0000 7f7d2136b700 1 --2- 172.21.15.46:0/3083049132 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7d040b6e00 0x7f7d040b9310 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.786+0000 7f7d2136b700 1 --2- 172.21.15.46:0/3083049132 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7d1c108760 0x7f7d1c19c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.786+0000 7f7d2136b700 1 --2- 172.21.15.46:0/3083049132 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7d1c1034e0 0x7f7d1c19bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.786+0000 7f7d2136b700 1 --2- 172.21.15.46:0/3083049132 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7d1c107e10 0x7f7d1c19c250 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:36.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.786+0000 7f7d2136b700 1 -- 172.21.15.46:0/3083049132 >> 172.21.15.46:0/3083049132 conn(0x7f7d1c0fef30 msgr2=0x7f7d1c1075d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.786+0000 7f7d2136b700 1 -- 172.21.15.46:0/3083049132 shutdown_connections 2024-05-09T04:58:36.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:36.786+0000 7f7d2136b700 1 -- 172.21.15.46:0/3083049132 wait complete. 2024-05-09T04:58:37.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:36 smithi110 ceph-mon[34904]: pgmap v230: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:58:37.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:36 smithi046 ceph-mon[30136]: pgmap v230: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:58:37.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:36 smithi072 ceph-mon[34088]: pgmap v230: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:58:37.558 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:58:37.818 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:58:38.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:37 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:58:38.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:37 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:58:38.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:37 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:58:38.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:37 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:58:38.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:37 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:58:38.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:37 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:58:38.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:37 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:58:38.067 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:37 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:58:38.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:37 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:58:38.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:37 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:58:38.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:37 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:58:38.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:37 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:58:38.975 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:38 smithi046 ceph-mon[30136]: pgmap v231: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:58:39.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:38 smithi110 ceph-mon[34904]: pgmap v231: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:58:39.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:38 smithi072 ceph-mon[34088]: pgmap v231: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:58:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.582+0000 7f862f480700 1 -- 172.21.15.46:0/834865911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8628107e30 msgr2=0x7f8628108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.582+0000 7f862f480700 1 --2- 172.21.15.46:0/834865911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8628107e30 0x7f8628108240 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f86180099c0 tx=0x7f8618009cd0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.582+0000 7f862f480700 1 -- 172.21.15.46:0/834865911 shutdown_connections 2024-05-09T04:58:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.582+0000 7f862f480700 1 --2- 172.21.15.46:0/834865911 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8628108780 0x7f86281100e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.582+0000 7f862f480700 1 --2- 172.21.15.46:0/834865911 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8628103550 0x7f8628103960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.582+0000 7f862f480700 1 --2- 172.21.15.46:0/834865911 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8628107e30 0x7f8628108240 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.582+0000 7f862f480700 1 -- 172.21.15.46:0/834865911 >> 172.21.15.46:0/834865911 conn(0x7f8628078e00 msgr2=0x7f8628079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.582+0000 7f862f480700 1 -- 172.21.15.46:0/834865911 shutdown_connections 2024-05-09T04:58:39.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.582+0000 7f862f480700 1 -- 172.21.15.46:0/834865911 wait complete. 2024-05-09T04:58:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.583+0000 7f862f480700 1 Processor -- start 2024-05-09T04:58:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.583+0000 7f862f480700 1 -- start start 2024-05-09T04:58:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.583+0000 7f862f480700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8628103550 0x7f862819aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.583+0000 7f862f480700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8628107e30 0x7f862819b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.583+0000 7f862f480700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8628108780 0x7f862819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f862f480700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8628113450 con 0x7f8628103550 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f862f480700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f86281132d0 con 0x7f8628107e30 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f862f480700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f86281135d0 con 0x7f8628108780 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f862d21c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8628103550 0x7f862819aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f862d21c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8628103550 0x7f862819aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56566/0 (socket says 172.21.15.46:56566) 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f862d21c700 1 -- 172.21.15.46:0/4064710502 learned_addr learned my addr 172.21.15.46:0/4064710502 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f862ca1b700 1 --2- 172.21.15.46:0/4064710502 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8628107e30 0x7f862819b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f862d21c700 1 -- 172.21.15.46:0/4064710502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8628108780 msgr2=0x7f862819f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f862d21c700 1 --2- 172.21.15.46:0/4064710502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8628108780 0x7f862819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f862d21c700 1 -- 172.21.15.46:0/4064710502 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8628107e30 msgr2=0x7f862819b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f862d21c700 1 --2- 172.21.15.46:0/4064710502 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8628107e30 0x7f862819b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f862d21c700 1 -- 172.21.15.46:0/4064710502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f862819fd80 con 0x7f8628103550 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f862d21c700 1 --2- 172.21.15.46:0/4064710502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8628103550 0x7f862819aeb0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f862400d950 tx=0x7f862400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f861e7fc700 1 -- 172.21.15.46:0/4064710502 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624016070 con 0x7f8628103550 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f862f480700 1 -- 172.21.15.46:0/4064710502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86281a0070 con 0x7f8628103550 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f861e7fc700 1 -- 172.21.15.46:0/4064710502 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8624009df0 con 0x7f8628103550 2024-05-09T04:58:39.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f861e7fc700 1 -- 172.21.15.46:0/4064710502 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862400f3c0 con 0x7f8628103550 2024-05-09T04:58:39.587 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.584+0000 7f862f480700 1 -- 172.21.15.46:0/4064710502 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86281a7990 con 0x7f8628103550 2024-05-09T04:58:39.588 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.588+0000 7f861e7fc700 1 -- 172.21.15.46:0/4064710502 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8624019070 con 0x7f8628103550 2024-05-09T04:58:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.589+0000 7f863de96b80 1 Processor -- start 2024-05-09T04:58:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.589+0000 7f861e7fc700 1 --2- 172.21.15.46:0/4064710502 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f86140b29f0 0x7f86140b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.589+0000 7f863de96b80 1 -- start start 2024-05-09T04:58:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.589+0000 7f862da1d700 1 --2- 172.21.15.46:0/4064710502 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f86140b29f0 0x7f86140b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.589+0000 7f861e7fc700 1 -- 172.21.15.46:0/4064710502 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f86240e22e0 con 0x7f8628103550 2024-05-09T04:58:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f863de96b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd281b160 0x55abd28b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f863de96b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd27314a0 0x55abd28b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f863de96b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd28b7400 0x55abd28b7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f863de96b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55abd27d0ff0 con 0x55abd27314a0 2024-05-09T04:58:39.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f863de96b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55abd276f7f0 con 0x55abd28b7400 2024-05-09T04:58:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f863de96b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55abd27ccfb0 con 0x55abd281b160 2024-05-09T04:58:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f862ca1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd27314a0 0x55abd28b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f862ca1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd27314a0 0x55abd28b32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56572/0 (socket says 172.21.15.46:56572) 2024-05-09T04:58:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f862ca1b700 1 -- 172.21.15.46:0/2757005082 learned_addr learned my addr 172.21.15.46:0/2757005082 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f862da1d700 1 --2- 172.21.15.46:0/4064710502 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f86140b29f0 0x7f86140b4f00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f862000aba0 tx=0x7f8620009250 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f862d21c700 1 --2- 172.21.15.46:0/2757005082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd281b160 0x55abd28b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f862da1d700 1 --2- 172.21.15.46:0/2757005082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd28b7400 0x55abd28b7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f862ca1b700 1 -- 172.21.15.46:0/2757005082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd281b160 msgr2=0x55abd28b2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f862ca1b700 1 --2- 172.21.15.46:0/2757005082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd281b160 0x55abd28b2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f862ca1b700 1 -- 172.21.15.46:0/2757005082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd28b7400 msgr2=0x55abd28b7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f862ca1b700 1 --2- 172.21.15.46:0/2757005082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd28b7400 0x55abd28b7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f862ca1b700 1 -- 172.21.15.46:0/2757005082 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd2767370 con 0x55abd27314a0 2024-05-09T04:58:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f862da1d700 1 --2- 172.21.15.46:0/2757005082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd28b7400 0x55abd28b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.590+0000 7f862ca1b700 1 --2- 172.21.15.46:0/2757005082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd27314a0 0x55abd28b32f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8618000c00 tx=0x7f8618003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=261f93ed2dd3b3e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:39.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.591+0000 7f86137fe700 1 -- 172.21.15.46:0/2757005082 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618003c10 con 0x55abd27314a0 2024-05-09T04:58:39.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.591+0000 7f86137fe700 1 -- 172.21.15.46:0/2757005082 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8618003db0 con 0x55abd27314a0 2024-05-09T04:58:39.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.591+0000 7f86137fe700 1 -- 172.21.15.46:0/2757005082 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861800be90 con 0x55abd27314a0 2024-05-09T04:58:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.591+0000 7f863de96b80 1 -- 172.21.15.46:0/2757005082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd27314a0 msgr2=0x55abd28b32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.591+0000 7f863de96b80 1 --2- 172.21.15.46:0/2757005082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd27314a0 0x55abd28b32f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8618000c00 tx=0x7f8618003760 comp rx=0 tx=0).stop 2024-05-09T04:58:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.591+0000 7f863de96b80 1 -- 172.21.15.46:0/2757005082 shutdown_connections 2024-05-09T04:58:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.591+0000 7f863de96b80 1 --2- 172.21.15.46:0/2757005082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd28b7400 0x55abd28b7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.592+0000 7f863de96b80 1 --2- 172.21.15.46:0/2757005082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd281b160 0x55abd28b2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.592+0000 7f863de96b80 1 --2- 172.21.15.46:0/2757005082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd27314a0 0x55abd28b32f0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.592+0000 7f863de96b80 1 -- 172.21.15.46:0/2757005082 >> 172.21.15.46:0/2757005082 conn(0x55abd27916a0 msgr2=0x55abd28a9400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.592+0000 7f863de96b80 1 -- 172.21.15.46:0/2757005082 shutdown_connections 2024-05-09T04:58:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.592+0000 7f863de96b80 1 -- 172.21.15.46:0/2757005082 wait complete. 2024-05-09T04:58:39.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.592+0000 7f863de96b80 1 -- start start 2024-05-09T04:58:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.592+0000 7f863de96b80 1 Processor -- start 2024-05-09T04:58:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f863de96b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd27314a0 0x55abd28c21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f863de96b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd281b160 0x55abd28c2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f863de96b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd28b7400 0x55abd28bc5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f863de96b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55abd264cba0 con 0x55abd28b7400 2024-05-09T04:58:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f863de96b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55abd278b150 con 0x55abd27314a0 2024-05-09T04:58:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f863de96b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55abd28b80f0 con 0x55abd281b160 2024-05-09T04:58:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f863de96b80 10 client.0 _ll_register_callbacks cb 0x55abd26426c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f862da1d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd28b7400 0x55abd28bc5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f862da1d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd28b7400 0x55abd28bc5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56586/0 (socket says 172.21.15.46:56586) 2024-05-09T04:58:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f862da1d700 1 -- 172.21.15.46:0/3745903131 learned_addr learned my addr 172.21.15.46:0/3745903131 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f862d21c700 1 --2- 172.21.15.46:0/3745903131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd27314a0 0x55abd28c21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f862ca1b700 1 --2- 172.21.15.46:0/3745903131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd281b160 0x55abd28c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f862da1d700 1 -- 172.21.15.46:0/3745903131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd281b160 msgr2=0x55abd28c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f862da1d700 1 --2- 172.21.15.46:0/3745903131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd281b160 0x55abd28c2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f862da1d700 1 -- 172.21.15.46:0/3745903131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd27314a0 msgr2=0x55abd28c21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f862da1d700 1 --2- 172.21.15.46:0/3745903131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd27314a0 0x55abd28c21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f862da1d700 1 -- 172.21.15.46:0/3745903131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd264c490 con 0x55abd28b7400 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f862d21c700 1 --2- 172.21.15.46:0/3745903131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd27314a0 0x55abd28c21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f862da1d700 1 --2- 172.21.15.46:0/3745903131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd28b7400 0x55abd28bc5e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f862000db00 tx=0x7f8620004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edc423aa18f6132a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f8610ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f8610ff9700 1 -- 172.21.15.46:0/3745903131 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f862001ab10 con 0x55abd28b7400 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f8610ff9700 1 -- 172.21.15.46:0/3745903131 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f862001acb0 con 0x55abd28b7400 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.593+0000 7f863de96b80 1 -- 172.21.15.46:0/3745903131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8618003db0 con 0x55abd28b7400 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.594+0000 7f8610ff9700 1 -- 172.21.15.46:0/3745903131 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f8620024430 con 0x55abd28b7400 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.594+0000 7f863de96b80 10 client.17073 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.594+0000 7f863de96b80 1 -- 172.21.15.46:0/3745903131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55abd28c9eb0 con 0x55abd28b7400 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.594+0000 7f8610ff9700 1 -- 172.21.15.46:0/3745903131 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8618003db0 con 0x55abd28b7400 2024-05-09T04:58:39.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.594+0000 7f8610ff9700 1 -- 172.21.15.46:0/3745903131 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f862001b5c0 con 0x55abd28b7400 2024-05-09T04:58:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.594+0000 7f863de96b80 10 client.17073 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.594+0000 7f863de96b80 10 client.17073 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:58:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.594+0000 7f863de96b80 20 client.17073 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.594+0000 7f863de96b80 1 --2- 172.21.15.46:0/3745903131 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55abd28cc590 0x55abd28ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.594+0000 7f863de96b80 4 client.17073 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:58:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.594+0000 7f863de96b80 1 -- 172.21.15.46:0/3745903131 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55abd27de9d0 con 0x55abd28cc590 2024-05-09T04:58:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.594+0000 7f862d21c700 1 --2- 172.21.15.46:0/3745903131 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55abd28cc590 0x55abd28ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.595+0000 7f862d21c700 1 --2- 172.21.15.46:0/3745903131 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55abd28cc590 0x55abd28ce9d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8c635f736ecbf4d server_cookie=45d0e5b613efd6e in_seq=0 out_seq=0 2024-05-09T04:58:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.595+0000 7f8610ff9700 10 client.17073 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.596+0000 7f8610ff9700 1 -- 172.21.15.46:0/3745903131 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55abd27de9d0 con 0x55abd28cc590 2024-05-09T04:58:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.596+0000 7f8610ff9700 10 client.17073 handle_command_reply: tid=0 2024-05-09T04:58:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.596+0000 7f863de96b80 1 client.17073 shutdown 2024-05-09T04:58:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.596+0000 7f863de96b80 1 -- 172.21.15.46:0/3745903131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd28b7400 msgr2=0x55abd28bc5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:39.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.596+0000 7f863de96b80 1 --2- 172.21.15.46:0/3745903131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd28b7400 0x55abd28bc5e0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f862000db00 tx=0x7f8620004c50 comp rx=0 tx=0).stop 2024-05-09T04:58:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.596+0000 7f863de96b80 1 -- 172.21.15.46:0/3745903131 shutdown_connections 2024-05-09T04:58:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.596+0000 7f863de96b80 1 --2- 172.21.15.46:0/3745903131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd28b7400 0x55abd28bc5e0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.596+0000 7f863de96b80 1 --2- 172.21.15.46:0/3745903131 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55abd28cc590 0x55abd28ce9d0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.596+0000 7f863de96b80 1 --2- 172.21.15.46:0/3745903131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd27314a0 0x55abd28c21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.596+0000 7f863de96b80 1 --2- 172.21.15.46:0/3745903131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd281b160 0x55abd28c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.596+0000 7f8610ff9700 0 client.17073 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.596+0000 7f863de96b80 1 -- 172.21.15.46:0/3745903131 >> 172.21.15.46:0/3745903131 conn(0x55abd27916a0 msgr2=0x55abd28a91c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.596+0000 7f863de96b80 1 -- 172.21.15.46:0/3745903131 shutdown_connections 2024-05-09T04:58:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.596+0000 7f863de96b80 1 -- 172.21.15.46:0/3745903131 wait complete. 2024-05-09T04:58:39.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.596+0000 7f863de96b80 20 client.17073 trim_cache size 0 max 16384 2024-05-09T04:58:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.616+0000 7f863de96b80 1 Processor -- start 2024-05-09T04:58:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f863de96b80 1 -- start start 2024-05-09T04:58:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f863de96b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd27314a0 0x55abd28bb260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f863de96b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd281b160 0x55abd28b6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f863de96b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd28b7400 0x55abd28a8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f863de96b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f86281af6f0 con 0x55abd27314a0 2024-05-09T04:58:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f863de96b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f86281acf80 con 0x55abd281b160 2024-05-09T04:58:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f863de96b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f86281aadf0 con 0x55abd28b7400 2024-05-09T04:58:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f862d21c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd27314a0 0x55abd28bb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f862d21c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd27314a0 0x55abd28bb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56590/0 (socket says 172.21.15.46:56590) 2024-05-09T04:58:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f862d21c700 1 -- 172.21.15.46:0/307895246 learned_addr learned my addr 172.21.15.46:0/307895246 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:39.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f862da1d700 1 --2- 172.21.15.46:0/307895246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd28b7400 0x55abd28a8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f862ca1b700 1 --2- 172.21.15.46:0/307895246 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd281b160 0x55abd28b6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f862d21c700 1 -- 172.21.15.46:0/307895246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd28b7400 msgr2=0x55abd28a8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f862d21c700 1 --2- 172.21.15.46:0/307895246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd28b7400 0x55abd28a8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f862d21c700 1 -- 172.21.15.46:0/307895246 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd281b160 msgr2=0x55abd28b6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f862d21c700 1 --2- 172.21.15.46:0/307895246 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd281b160 0x55abd28b6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f862d21c700 1 -- 172.21.15.46:0/307895246 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55abd28aac70 con 0x55abd27314a0 2024-05-09T04:58:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.617+0000 7f862d21c700 1 --2- 172.21.15.46:0/307895246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd27314a0 0x55abd28bb260 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f862400b2f0 tx=0x7f8624005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39b60539d34aef43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.618+0000 7f8612ffd700 1 -- 172.21.15.46:0/307895246 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8624020040 con 0x55abd27314a0 2024-05-09T04:58:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.618+0000 7f8612ffd700 1 -- 172.21.15.46:0/307895246 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86240c3070 con 0x55abd27314a0 2024-05-09T04:58:39.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.618+0000 7f8612ffd700 1 -- 172.21.15.46:0/307895246 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86240a99d0 con 0x55abd27314a0 2024-05-09T04:58:39.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.618+0000 7f863de96b80 1 -- 172.21.15.46:0/307895246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd27314a0 msgr2=0x55abd28bb260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:39.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.618+0000 7f863de96b80 1 --2- 172.21.15.46:0/307895246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd27314a0 0x55abd28bb260 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f862400b2f0 tx=0x7f8624005910 comp rx=0 tx=0).stop 2024-05-09T04:58:39.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.619+0000 7f863de96b80 1 -- 172.21.15.46:0/307895246 shutdown_connections 2024-05-09T04:58:39.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.619+0000 7f863de96b80 1 --2- 172.21.15.46:0/307895246 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd28b7400 0x55abd28a8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.619+0000 7f863de96b80 1 --2- 172.21.15.46:0/307895246 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd27314a0 0x55abd28bb260 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.619+0000 7f863de96b80 1 --2- 172.21.15.46:0/307895246 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd281b160 0x55abd28b6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.619+0000 7f863de96b80 1 -- 172.21.15.46:0/307895246 >> 172.21.15.46:0/307895246 conn(0x55abd27916a0 msgr2=0x55abd28a88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:39.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.619+0000 7f863de96b80 1 -- 172.21.15.46:0/307895246 shutdown_connections 2024-05-09T04:58:39.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.619+0000 7f863de96b80 1 -- 172.21.15.46:0/307895246 wait complete. 2024-05-09T04:58:39.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.619+0000 7f863de96b80 1 -- start start 2024-05-09T04:58:39.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.619+0000 7f863de96b80 1 Processor -- start 2024-05-09T04:58:39.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f863de96b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd27314a0 0x55abd28c88d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f863de96b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd281b160 0x55abd28c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f863de96b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd28b7400 0x55abd28c71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f863de96b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f86281ab440 con 0x55abd281b160 2024-05-09T04:58:39.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f863de96b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f86281ac8b0 con 0x55abd28b7400 2024-05-09T04:58:39.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f863de96b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f86281abb10 con 0x55abd27314a0 2024-05-09T04:58:39.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f863de96b80 10 client.0 _ll_register_callbacks cb 0x55abd26426c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:58:39.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f862ca1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd281b160 0x55abd28c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f862ca1b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd281b160 0x55abd28c6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:56604/0 (socket says 172.21.15.46:56604) 2024-05-09T04:58:39.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f862ca1b700 1 -- 172.21.15.46:0/205717 learned_addr learned my addr 172.21.15.46:0/205717 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f862d21c700 1 --2- 172.21.15.46:0/205717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd27314a0 0x55abd28c88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f862da1d700 1 --2- 172.21.15.46:0/205717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd28b7400 0x55abd28c71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f862ca1b700 1 -- 172.21.15.46:0/205717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd27314a0 msgr2=0x55abd28c88d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f862ca1b700 1 --2- 172.21.15.46:0/205717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd27314a0 0x55abd28c88d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f862ca1b700 1 -- 172.21.15.46:0/205717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd28b7400 msgr2=0x55abd28c71a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f862ca1b700 1 --2- 172.21.15.46:0/205717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd28b7400 0x55abd28c71a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f862ca1b700 1 -- 172.21.15.46:0/205717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55abd28c76e0 con 0x55abd281b160 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f862da1d700 1 --2- 172.21.15.46:0/205717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd28b7400 0x55abd28c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f862d21c700 1 --2- 172.21.15.46:0/205717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd27314a0 0x55abd28c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f862ca1b700 1 --2- 172.21.15.46:0/205717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd281b160 0x55abd28c6c60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f8618000c00 tx=0x7f8618003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349130eefe032b4e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.620+0000 7f8610ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.621+0000 7f8610ff9700 1 -- 172.21.15.46:0/205717 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8618030030 con 0x55abd281b160 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.621+0000 7f8610ff9700 1 -- 172.21.15.46:0/205717 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8600+0+0 (secure 0 0 0) 0x7f86180052e0 con 0x55abd281b160 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.621+0000 7f863de96b80 1 -- 172.21.15.46:0/205717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86240c3070 con 0x55abd281b160 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.621+0000 7f8610ff9700 1 -- 172.21.15.46:0/205717 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f861800b540 con 0x55abd281b160 2024-05-09T04:58:39.625 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.621+0000 7f863de96b80 10 client.17079 fetch_fsmap learned FSMap version 25 2024-05-09T04:58:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.621+0000 7f863de96b80 1 -- 172.21.15.46:0/205717 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55abd2937990 con 0x55abd281b160 2024-05-09T04:58:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.621+0000 7f8610ff9700 1 -- 172.21.15.46:0/205717 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86240c3070 con 0x55abd281b160 2024-05-09T04:58:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.621+0000 7f8610ff9700 1 -- 172.21.15.46:0/205717 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f86180374c0 con 0x55abd281b160 2024-05-09T04:58:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.621+0000 7f863de96b80 10 client.17079 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:58:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.621+0000 7f863de96b80 10 client.17079 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:58:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.621+0000 7f863de96b80 20 client.17079 populate_metadata read hostname 'smithi046' 2024-05-09T04:58:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.621+0000 7f863de96b80 1 --2- 172.21.15.46:0/205717 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55abd2937b60 0x55abd2939f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:58:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.621+0000 7f863de96b80 4 client.17079 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:58:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.621+0000 7f863de96b80 1 -- 172.21.15.46:0/205717 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55abd27d8a70 con 0x55abd2937b60 2024-05-09T04:58:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.622+0000 7f862d21c700 1 --2- 172.21.15.46:0/205717 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55abd2937b60 0x55abd2939f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:58:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.622+0000 7f8610ff9700 10 client.17079 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.622+0000 7f862d21c700 1 --2- 172.21.15.46:0/205717 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55abd2937b60 0x55abd2939f80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff966711d83bd982 server_cookie=cf3b333cb2582a0b in_seq=0 out_seq=0 2024-05-09T04:58:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.622+0000 7f8610ff9700 1 -- 172.21.15.46:0/205717 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55abd27d8a70 con 0x55abd2937b60 2024-05-09T04:58:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.622+0000 7f8610ff9700 10 client.17079 handle_command_reply: tid=0 2024-05-09T04:58:39.626 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.622+0000 7f863de96b80 1 client.17079 shutdown 2024-05-09T04:58:39.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.622+0000 7f863de96b80 1 -- 172.21.15.46:0/205717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd281b160 msgr2=0x55abd28c6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:39.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.622+0000 7f863de96b80 1 --2- 172.21.15.46:0/205717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd281b160 0x55abd28c6c60 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f8618000c00 tx=0x7f8618003cf0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.622+0000 7f863de96b80 1 -- 172.21.15.46:0/205717 shutdown_connections 2024-05-09T04:58:39.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.622+0000 7f863de96b80 1 --2- 172.21.15.46:0/205717 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55abd2937b60 0x55abd2939f80 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.622+0000 7f863de96b80 1 --2- 172.21.15.46:0/205717 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55abd28b7400 0x55abd28c71a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.622+0000 7f863de96b80 1 --2- 172.21.15.46:0/205717 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55abd27314a0 0x55abd28c88d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.622+0000 7f8610ff9700 0 client.17079 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:58:39.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.622+0000 7f863de96b80 1 --2- 172.21.15.46:0/205717 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55abd281b160 0x55abd28c6c60 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.622+0000 7f863de96b80 1 -- 172.21.15.46:0/205717 >> 172.21.15.46:0/205717 conn(0x55abd27916a0 msgr2=0x55abd28b69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:39.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.623+0000 7f863de96b80 1 -- 172.21.15.46:0/205717 shutdown_connections 2024-05-09T04:58:39.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.623+0000 7f863de96b80 1 -- 172.21.15.46:0/205717 wait complete. 2024-05-09T04:58:39.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.623+0000 7f863de96b80 20 client.17079 trim_cache size 0 max 16384 2024-05-09T04:58:39.628 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:58:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T04:58:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-09T04:58:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "9b7f376e-94a6-478e-b927-974a97904afb": { 2024-05-09T04:58:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-09T04:58:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "9b7f376e-94a6-478e-b927-974a97904afb", 2024-05-09T04:58:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-09T04:58:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T04:58:39.628 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T04:58:39.628 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:58:39.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.623+0000 7f862f480700 1 -- 172.21.15.46:0/4064710502 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f86140b29f0 msgr2=0x7f86140b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:39.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.623+0000 7f862f480700 1 --2- 172.21.15.46:0/4064710502 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f86140b29f0 0x7f86140b4f00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f862000aba0 tx=0x7f8620009250 comp rx=0 tx=0).stop 2024-05-09T04:58:39.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.623+0000 7f862f480700 1 -- 172.21.15.46:0/4064710502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8628103550 msgr2=0x7f862819aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:58:39.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.623+0000 7f862f480700 1 --2- 172.21.15.46:0/4064710502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8628103550 0x7f862819aeb0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f862400d950 tx=0x7f862400dc60 comp rx=0 tx=0).stop 2024-05-09T04:58:39.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.624+0000 7f862f480700 1 -- 172.21.15.46:0/4064710502 shutdown_connections 2024-05-09T04:58:39.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.624+0000 7f862f480700 1 --2- 172.21.15.46:0/4064710502 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f86140b29f0 0x7f86140b4f00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.624+0000 7f862f480700 1 --2- 172.21.15.46:0/4064710502 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8628108780 0x7f862819f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.624+0000 7f862f480700 1 --2- 172.21.15.46:0/4064710502 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8628103550 0x7f862819aeb0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.624+0000 7f862f480700 1 --2- 172.21.15.46:0/4064710502 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8628107e30 0x7f862819b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:58:39.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.624+0000 7f862f480700 1 -- 172.21.15.46:0/4064710502 >> 172.21.15.46:0/4064710502 conn(0x7f8628078e00 msgr2=0x7f8628075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:58:39.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.624+0000 7f862f480700 1 -- 172.21.15.46:0/4064710502 shutdown_connections 2024-05-09T04:58:39.630 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:58:39.624+0000 7f862f480700 1 -- 172.21.15.46:0/4064710502 wait complete. 2024-05-09T04:58:40.386 INFO:tasks.cephfs.filesystem:scrub status for tag:9b7f376e-94a6-478e-b927-974a97904afb - {'path': '/', 'tag': '9b7f376e-94a6-478e-b927-974a97904afb', 'options': 'recursive,force'} 2024-05-09T04:58:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:40 smithi110 ceph-mon[34904]: pgmap v232: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:58:41.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:40 smithi046 ceph-mon[30136]: pgmap v232: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:58:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:40 smithi072 ceph-mon[34088]: pgmap v232: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:58:42.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:41 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:58:42.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:41 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:58:42.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:41 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:58:43.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:42 smithi110 ceph-mon[34904]: pgmap v233: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-05-09T04:58:43.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:42 smithi046 ceph-mon[30136]: pgmap v233: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-05-09T04:58:43.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:42 smithi072 ceph-mon[34088]: pgmap v233: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-05-09T04:58:45.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:44 smithi110 ceph-mon[34904]: pgmap v234: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T04:58:45.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:44 smithi046 ceph-mon[30136]: pgmap v234: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T04:58:45.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:44 smithi072 ceph-mon[34088]: pgmap v234: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T04:58:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:46 smithi110 ceph-mon[34904]: pgmap v235: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:58:47.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:46 smithi046 ceph-mon[30136]: pgmap v235: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:58:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:46 smithi072 ceph-mon[34088]: pgmap v235: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:58:49.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:48 smithi110 ceph-mon[34904]: pgmap v236: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-05-09T04:58:49.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:48 smithi046 ceph-mon[30136]: pgmap v236: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-05-09T04:58:49.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:48 smithi072 ceph-mon[34088]: pgmap v236: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 8 op/s 2024-05-09T04:58:51.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:50 smithi046 ceph-mon[30136]: pgmap v237: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-05-09T04:58:51.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:50 smithi072 ceph-mon[34088]: pgmap v237: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-05-09T04:58:51.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:50 smithi110 ceph-mon[34904]: pgmap v237: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 5 op/s 2024-05-09T04:58:53.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:52 smithi046 ceph-mon[30136]: pgmap v238: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T04:58:53.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:52 smithi072 ceph-mon[34088]: pgmap v238: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T04:58:53.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:52 smithi110 ceph-mon[34904]: pgmap v238: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T04:58:55.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:54 smithi046 ceph-mon[30136]: pgmap v239: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:58:55.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:54 smithi072 ceph-mon[34088]: pgmap v239: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:58:55.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:54 smithi110 ceph-mon[34904]: pgmap v239: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:58:57.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:56 smithi046 ceph-mon[30136]: pgmap v240: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-09T04:58:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:56 smithi072 ceph-mon[34088]: pgmap v240: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-09T04:58:57.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:56 smithi110 ceph-mon[34904]: pgmap v240: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2024-05-09T04:58:59.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:58:58 smithi046 ceph-mon[30136]: pgmap v241: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:58:59.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:58:58 smithi072 ceph-mon[34088]: pgmap v241: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:58:59.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:58:58 smithi110 ceph-mon[34904]: pgmap v241: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:59:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:00 smithi046 ceph-mon[30136]: pgmap v242: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:59:01.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:00 smithi072 ceph-mon[34088]: pgmap v242: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:59:01.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:00 smithi110 ceph-mon[34904]: pgmap v242: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:59:02.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:02 smithi072 ceph-mon[34088]: pgmap v243: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-09T04:59:02.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:02 smithi110 ceph-mon[34904]: pgmap v243: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-09T04:59:02.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:02 smithi046 ceph-mon[30136]: pgmap v243: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-09T04:59:05.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:04 smithi110 ceph-mon[34904]: pgmap v244: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T04:59:05.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:04 smithi046 ceph-mon[30136]: pgmap v244: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T04:59:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:04 smithi072 ceph-mon[34088]: pgmap v244: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T04:59:07.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:06 smithi110 ceph-mon[34904]: pgmap v245: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T04:59:07.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:06 smithi046 ceph-mon[30136]: pgmap v245: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T04:59:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:06 smithi072 ceph-mon[34088]: pgmap v245: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T04:59:08.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:07 smithi110 ceph-mon[34904]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T04:59:08.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:07 smithi046 ceph-mon[30136]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T04:59:08.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:07 smithi072 ceph-mon[34088]: osdmap e88: 12 total, 12 up, 12 in 2024-05-09T04:59:09.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:08 smithi110 ceph-mon[34904]: pgmap v247: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T04:59:09.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:08 smithi046 ceph-mon[30136]: pgmap v247: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T04:59:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:08 smithi072 ceph-mon[34088]: pgmap v247: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T04:59:10.387 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:59:10.652 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:11.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:10 smithi046 ceph-mon[30136]: pgmap v248: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T04:59:11.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:10 smithi110 ceph-mon[34904]: pgmap v248: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T04:59:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:10 smithi072 ceph-mon[34088]: pgmap v248: 97 pgs: 97 active+clean; 11 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T04:59:12.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.467+0000 7fdb5c730700 1 -- 172.21.15.46:0/3623371314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb54102ae0 msgr2=0x7fdb54102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:12.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.467+0000 7fdb5c730700 1 --2- 172.21.15.46:0/3623371314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb54102ae0 0x7fdb54102ef0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fdb500099c0 tx=0x7fdb50009cd0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.467+0000 7fdb5c730700 1 -- 172.21.15.46:0/3623371314 shutdown_connections 2024-05-09T04:59:12.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.467+0000 7fdb5c730700 1 --2- 172.21.15.46:0/3623371314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb54109850 0x7fdb541111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.467+0000 7fdb5c730700 1 --2- 172.21.15.46:0/3623371314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb54102ae0 0x7fdb54102ef0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.467+0000 7fdb5c730700 1 --2- 172.21.15.46:0/3623371314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb54108f00 0x7fdb54109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.467+0000 7fdb5c730700 1 -- 172.21.15.46:0/3623371314 >> 172.21.15.46:0/3623371314 conn(0x7fdb540fe4e0 msgr2=0x7fdb54100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:12.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.467+0000 7fdb5c730700 1 -- 172.21.15.46:0/3623371314 shutdown_connections 2024-05-09T04:59:12.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.467+0000 7fdb5c730700 1 -- 172.21.15.46:0/3623371314 wait complete. 2024-05-09T04:59:12.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.468+0000 7fdb5c730700 1 Processor -- start 2024-05-09T04:59:12.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.468+0000 7fdb5c730700 1 -- start start 2024-05-09T04:59:12.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.468+0000 7fdb5c730700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb54102ae0 0x7fdb5419d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.468+0000 7fdb5c730700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb54108f00 0x7fdb5419d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.468+0000 7fdb5c730700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb54109850 0x7fdb541a1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.468+0000 7fdb5c730700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54114480 con 0x7fdb54108f00 2024-05-09T04:59:12.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.468+0000 7fdb5c730700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54114300 con 0x7fdb54102ae0 2024-05-09T04:59:12.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.468+0000 7fdb5c730700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb54114600 con 0x7fdb54109850 2024-05-09T04:59:12.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb59ccb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb54108f00 0x7fdb5419d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb59ccb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb54108f00 0x7fdb5419d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52230/0 (socket says 172.21.15.46:52230) 2024-05-09T04:59:12.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb59ccb700 1 -- 172.21.15.46:0/3465134479 learned_addr learned my addr 172.21.15.46:0/3465134479 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:12.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb5a4cc700 1 --2- 172.21.15.46:0/3465134479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb54102ae0 0x7fdb5419d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb5accd700 1 --2- 172.21.15.46:0/3465134479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb54109850 0x7fdb541a1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb59ccb700 1 -- 172.21.15.46:0/3465134479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb54109850 msgr2=0x7fdb541a1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:12.473 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb59ccb700 1 --2- 172.21.15.46:0/3465134479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb54109850 0x7fdb541a1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb59ccb700 1 -- 172.21.15.46:0/3465134479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb54102ae0 msgr2=0x7fdb5419d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:12.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb59ccb700 1 --2- 172.21.15.46:0/3465134479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb54102ae0 0x7fdb5419d0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb59ccb700 1 -- 172.21.15.46:0/3465134479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb541a1ef0 con 0x7fdb54108f00 2024-05-09T04:59:12.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb5accd700 1 --2- 172.21.15.46:0/3465134479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb54109850 0x7fdb541a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:59:12.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb59ccb700 1 --2- 172.21.15.46:0/3465134479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb54108f00 0x7fdb5419d670 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400c9e0 tx=0x7fdb4400cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:12.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb4b7fe700 1 -- 172.21.15.46:0/3465134479 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44007aa0 con 0x7fdb54108f00 2024-05-09T04:59:12.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb5c730700 1 -- 172.21.15.46:0/3465134479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb541a21e0 con 0x7fdb54108f00 2024-05-09T04:59:12.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb4b7fe700 1 -- 172.21.15.46:0/3465134479 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb44007c40 con 0x7fdb54108f00 2024-05-09T04:59:12.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.469+0000 7fdb4b7fe700 1 -- 172.21.15.46:0/3465134479 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb44018660 con 0x7fdb54108f00 2024-05-09T04:59:12.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.470+0000 7fdb5c730700 1 -- 172.21.15.46:0/3465134479 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb541a2720 con 0x7fdb54108f00 2024-05-09T04:59:12.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.473+0000 7fdb4b7fe700 1 -- 172.21.15.46:0/3465134479 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdb44018800 con 0x7fdb54108f00 2024-05-09T04:59:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.473+0000 7fdb4b7fe700 1 --2- 172.21.15.46:0/3465134479 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdb400b29f0 0x7fdb400b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.473+0000 7fdb6b146b80 1 Processor -- start 2024-05-09T04:59:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.473+0000 7fdb5a4cc700 1 --2- 172.21.15.46:0/3465134479 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdb400b29f0 0x7fdb400b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.474+0000 7fdb4b7fe700 1 -- 172.21.15.46:0/3465134479 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8644+0+0 (secure 0 0 0) 0x7fdb440e2640 con 0x7fdb54108f00 2024-05-09T04:59:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.474+0000 7fdb6b146b80 1 -- start start 2024-05-09T04:59:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.474+0000 7fdb5a4cc700 1 --2- 172.21.15.46:0/3465134479 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdb400b29f0 0x7fdb400b4f00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009750 tx=0x7fdb5001a040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.474+0000 7fdb6b146b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb357d00 0x5583cb460240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.474+0000 7fdb6b146b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb2dd4a0 0x5583cb460780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.474+0000 7fdb6b146b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb45a400 0x5583cb45a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.474+0000 7fdb6b146b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5583cb37d0a0 con 0x5583cb2dd4a0 2024-05-09T04:59:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.474+0000 7fdb6b146b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5583cb3229a0 con 0x5583cb45a400 2024-05-09T04:59:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.474+0000 7fdb6b146b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5583cb379060 con 0x5583cb357d00 2024-05-09T04:59:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.474+0000 7fdb59ccb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb2dd4a0 0x5583cb460780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.474+0000 7fdb59ccb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb2dd4a0 0x5583cb460780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52232/0 (socket says 172.21.15.46:52232) 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.474+0000 7fdb59ccb700 1 -- 172.21.15.46:0/804871861 learned_addr learned my addr 172.21.15.46:0/804871861 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.474+0000 7fdb5a4cc700 1 --2- 172.21.15.46:0/804871861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb357d00 0x5583cb460240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.474+0000 7fdb5accd700 1 --2- 172.21.15.46:0/804871861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb45a400 0x5583cb45a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.474+0000 7fdb59ccb700 1 -- 172.21.15.46:0/804871861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb357d00 msgr2=0x5583cb460240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.475+0000 7fdb59ccb700 1 --2- 172.21.15.46:0/804871861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb357d00 0x5583cb460240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.475+0000 7fdb59ccb700 1 -- 172.21.15.46:0/804871861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb45a400 msgr2=0x5583cb45a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.475+0000 7fdb59ccb700 1 --2- 172.21.15.46:0/804871861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb45a400 0x5583cb45a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.475+0000 7fdb59ccb700 1 -- 172.21.15.46:0/804871861 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583cb313500 con 0x5583cb2dd4a0 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.475+0000 7fdb59ccb700 1 --2- 172.21.15.46:0/804871861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb2dd4a0 0x5583cb460780 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fdb440b0fd0 tx=0x7fdb440a9430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96d2683d445277d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.475+0000 7fdb48ff9700 1 -- 172.21.15.46:0/804871861 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb440a98a0 con 0x5583cb2dd4a0 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.475+0000 7fdb48ff9700 1 -- 172.21.15.46:0/804871861 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb440adc50 con 0x5583cb2dd4a0 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.475+0000 7fdb48ff9700 1 -- 172.21.15.46:0/804871861 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb440a9d60 con 0x5583cb2dd4a0 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.476+0000 7fdb6b146b80 1 -- 172.21.15.46:0/804871861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb2dd4a0 msgr2=0x5583cb460780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.476+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/804871861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb2dd4a0 0x5583cb460780 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fdb440b0fd0 tx=0x7fdb440a9430 comp rx=0 tx=0).stop 2024-05-09T04:59:12.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.476+0000 7fdb6b146b80 1 -- 172.21.15.46:0/804871861 shutdown_connections 2024-05-09T04:59:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.476+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/804871861 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb45a400 0x5583cb45a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.476+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/804871861 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb357d00 0x5583cb460240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.476+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/804871861 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb2dd4a0 0x5583cb460780 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.476+0000 7fdb6b146b80 1 -- 172.21.15.46:0/804871861 >> 172.21.15.46:0/804871861 conn(0x5583cb2e0f70 msgr2=0x5583cb455220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.476+0000 7fdb6b146b80 1 -- 172.21.15.46:0/804871861 shutdown_connections 2024-05-09T04:59:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.476+0000 7fdb6b146b80 1 -- 172.21.15.46:0/804871861 wait complete. 2024-05-09T04:59:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.476+0000 7fdb6b146b80 1 -- start start 2024-05-09T04:59:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb6b146b80 1 Processor -- start 2024-05-09T04:59:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb6b146b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb2dd4a0 0x5583cb463ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb6b146b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb357d00 0x5583cb464530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb6b146b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb45a400 0x5583cb464a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb6b146b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5583cb45b160 con 0x5583cb45a400 2024-05-09T04:59:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb6b146b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5583cb317b20 con 0x5583cb2dd4a0 2024-05-09T04:59:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb6b146b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5583cb467020 con 0x5583cb357d00 2024-05-09T04:59:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb6b146b80 10 client.0 _ll_register_callbacks cb 0x5583cb1ee6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb5accd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb45a400 0x5583cb464a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb5accd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb45a400 0x5583cb464a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52240/0 (socket says 172.21.15.46:52240) 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb5accd700 1 -- 172.21.15.46:0/216929522 learned_addr learned my addr 172.21.15.46:0/216929522 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb59ccb700 1 --2- 172.21.15.46:0/216929522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb357d00 0x5583cb464530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb5a4cc700 1 --2- 172.21.15.46:0/216929522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb2dd4a0 0x5583cb463ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb5accd700 1 -- 172.21.15.46:0/216929522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb357d00 msgr2=0x5583cb464530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb5accd700 1 --2- 172.21.15.46:0/216929522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb357d00 0x5583cb464530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb5accd700 1 -- 172.21.15.46:0/216929522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb2dd4a0 msgr2=0x5583cb463ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb5accd700 1 --2- 172.21.15.46:0/216929522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb2dd4a0 0x5583cb463ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.477+0000 7fdb5accd700 1 -- 172.21.15.46:0/216929522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583cb1f8490 con 0x5583cb45a400 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.478+0000 7fdb5accd700 1 --2- 172.21.15.46:0/216929522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb45a400 0x5583cb464a70 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c008f50 tx=0x7fdb4c010430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d0540b913c9724 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.478+0000 7fdb3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.478+0000 7fdb3dffb700 1 -- 172.21.15.46:0/216929522 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c010bd0 con 0x5583cb45a400 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.478+0000 7fdb3dffb700 1 -- 172.21.15.46:0/216929522 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8644+0+0 (secure 0 0 0) 0x7fdb4c010d70 con 0x5583cb45a400 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.478+0000 7fdb6b146b80 1 -- 172.21.15.46:0/216929522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb440adc50 con 0x5583cb45a400 2024-05-09T04:59:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.478+0000 7fdb3dffb700 1 -- 172.21.15.46:0/216929522 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdb4c024050 con 0x5583cb45a400 2024-05-09T04:59:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.478+0000 7fdb6b146b80 10 client.17091 fetch_fsmap learned FSMap version 25 2024-05-09T04:59:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.478+0000 7fdb6b146b80 1 -- 172.21.15.46:0/216929522 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5583cb471a20 con 0x5583cb45a400 2024-05-09T04:59:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.478+0000 7fdb3dffb700 1 -- 172.21.15.46:0/216929522 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb440adc50 con 0x5583cb45a400 2024-05-09T04:59:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.478+0000 7fdb3dffb700 1 -- 172.21.15.46:0/216929522 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fdb4c015460 con 0x5583cb45a400 2024-05-09T04:59:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.478+0000 7fdb6b146b80 10 client.17091 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:59:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.478+0000 7fdb6b146b80 10 client.17091 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:59:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.479+0000 7fdb6b146b80 20 client.17091 populate_metadata read hostname 'smithi046' 2024-05-09T04:59:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.479+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/216929522 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583cb474060 0x5583cb476480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.479+0000 7fdb6b146b80 4 client.17091 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:59:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.479+0000 7fdb6b146b80 1 -- 172.21.15.46:0/216929522 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583cb38aa80 con 0x5583cb474060 2024-05-09T04:59:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.479+0000 7fdb5a4cc700 1 --2- 172.21.15.46:0/216929522 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583cb474060 0x5583cb476480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.479+0000 7fdb5a4cc700 1 --2- 172.21.15.46:0/216929522 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583cb474060 0x5583cb476480 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0efd8f8215f067e server_cookie=6b971f2d6f0fc in_seq=0 out_seq=0 2024-05-09T04:59:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.479+0000 7fdb3dffb700 10 client.17091 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.480+0000 7fdb3dffb700 1 -- 172.21.15.46:0/216929522 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5583cb38aa80 con 0x5583cb474060 2024-05-09T04:59:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.480+0000 7fdb3dffb700 10 client.17091 handle_command_reply: tid=0 2024-05-09T04:59:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.480+0000 7fdb6b146b80 1 client.17091 shutdown 2024-05-09T04:59:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.480+0000 7fdb6b146b80 1 -- 172.21.15.46:0/216929522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb45a400 msgr2=0x5583cb464a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.480+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/216929522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb45a400 0x5583cb464a70 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c008f50 tx=0x7fdb4c010430 comp rx=0 tx=0).stop 2024-05-09T04:59:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.480+0000 7fdb6b146b80 1 -- 172.21.15.46:0/216929522 shutdown_connections 2024-05-09T04:59:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.480+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/216929522 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb45a400 0x5583cb464a70 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.480+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/216929522 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583cb474060 0x5583cb476480 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.481+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/216929522 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb2dd4a0 0x5583cb463ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.481+0000 7fdb3dffb700 0 client.17091 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.481+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/216929522 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb357d00 0x5583cb464530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.481+0000 7fdb6b146b80 1 -- 172.21.15.46:0/216929522 >> 172.21.15.46:0/216929522 conn(0x5583cb2e0f70 msgr2=0x5583cb44eff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.481+0000 7fdb6b146b80 1 -- 172.21.15.46:0/216929522 shutdown_connections 2024-05-09T04:59:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.481+0000 7fdb6b146b80 1 -- 172.21.15.46:0/216929522 wait complete. 2024-05-09T04:59:12.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.481+0000 7fdb6b146b80 20 client.17091 trim_cache size 0 max 16384 2024-05-09T04:59:12.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.501+0000 7fdb6b146b80 1 Processor -- start 2024-05-09T04:59:12.502 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.501+0000 7fdb6b146b80 1 -- start start 2024-05-09T04:59:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.501+0000 7fdb6b146b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb2dd4a0 0x5583cb44fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.501+0000 7fdb6b146b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb357d00 0x5583cb46fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.501+0000 7fdb6b146b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb45a400 0x5583cb47bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.501+0000 7fdb6b146b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541b3b50 con 0x5583cb357d00 2024-05-09T04:59:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.501+0000 7fdb6b146b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541b13e0 con 0x5583cb2dd4a0 2024-05-09T04:59:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.501+0000 7fdb6b146b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541af250 con 0x5583cb45a400 2024-05-09T04:59:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.501+0000 7fdb59ccb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb357d00 0x5583cb46fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.502+0000 7fdb59ccb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb357d00 0x5583cb46fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52250/0 (socket says 172.21.15.46:52250) 2024-05-09T04:59:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.502+0000 7fdb59ccb700 1 -- 172.21.15.46:0/1211130638 learned_addr learned my addr 172.21.15.46:0/1211130638 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.502+0000 7fdb5accd700 1 --2- 172.21.15.46:0/1211130638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb45a400 0x5583cb47bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.502+0000 7fdb5a4cc700 1 --2- 172.21.15.46:0/1211130638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb2dd4a0 0x5583cb44fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.502+0000 7fdb59ccb700 1 -- 172.21.15.46:0/1211130638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb45a400 msgr2=0x5583cb47bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.502+0000 7fdb59ccb700 1 --2- 172.21.15.46:0/1211130638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb45a400 0x5583cb47bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.502+0000 7fdb59ccb700 1 -- 172.21.15.46:0/1211130638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb2dd4a0 msgr2=0x5583cb44fa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:12.503 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.502+0000 7fdb59ccb700 1 --2- 172.21.15.46:0/1211130638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb2dd4a0 0x5583cb44fa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.502+0000 7fdb59ccb700 1 -- 172.21.15.46:0/1211130638 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583cb463dc0 con 0x5583cb357d00 2024-05-09T04:59:12.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.502+0000 7fdb59ccb700 1 --2- 172.21.15.46:0/1211130638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb357d00 0x5583cb46fdd0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400c9b0 tx=0x7fdb440cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b69aedc4108217b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:12.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.502+0000 7fdb3ffff700 1 -- 172.21.15.46:0/1211130638 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb440c2020 con 0x5583cb357d00 2024-05-09T04:59:12.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.502+0000 7fdb3ffff700 1 -- 172.21.15.46:0/1211130638 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb440b2700 con 0x5583cb357d00 2024-05-09T04:59:12.504 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.502+0000 7fdb3ffff700 1 -- 172.21.15.46:0/1211130638 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb440aae30 con 0x5583cb357d00 2024-05-09T04:59:12.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.503+0000 7fdb6b146b80 1 -- 172.21.15.46:0/1211130638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb357d00 msgr2=0x5583cb46fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:12.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.503+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/1211130638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb357d00 0x5583cb46fdd0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400c9b0 tx=0x7fdb440cc040 comp rx=0 tx=0).stop 2024-05-09T04:59:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.503+0000 7fdb6b146b80 1 -- 172.21.15.46:0/1211130638 shutdown_connections 2024-05-09T04:59:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.503+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/1211130638 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb45a400 0x5583cb47bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.503+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/1211130638 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb2dd4a0 0x5583cb44fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.503+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/1211130638 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb357d00 0x5583cb46fdd0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.503+0000 7fdb6b146b80 1 -- 172.21.15.46:0/1211130638 >> 172.21.15.46:0/1211130638 conn(0x5583cb2e0f70 msgr2=0x5583cb462f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.503+0000 7fdb6b146b80 1 -- 172.21.15.46:0/1211130638 shutdown_connections 2024-05-09T04:59:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.503+0000 7fdb6b146b80 1 -- 172.21.15.46:0/1211130638 wait complete. 2024-05-09T04:59:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.503+0000 7fdb6b146b80 1 -- start start 2024-05-09T04:59:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb6b146b80 1 Processor -- start 2024-05-09T04:59:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb6b146b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb2dd4a0 0x5583cb47b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb6b146b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb357d00 0x5583cb47bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb6b146b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb45a400 0x5583cb47c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb6b146b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541af8a0 con 0x5583cb45a400 2024-05-09T04:59:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb6b146b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541b0d10 con 0x5583cb2dd4a0 2024-05-09T04:59:12.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb6b146b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdb541aff70 con 0x5583cb357d00 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb6b146b80 10 client.0 _ll_register_callbacks cb 0x5583cb1ee6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb5accd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb45a400 0x5583cb47c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb5accd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb45a400 0x5583cb47c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:52256/0 (socket says 172.21.15.46:52256) 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb5accd700 1 -- 172.21.15.46:0/1410958820 learned_addr learned my addr 172.21.15.46:0/1410958820 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb5a4cc700 1 --2- 172.21.15.46:0/1410958820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb2dd4a0 0x5583cb47b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb59ccb700 1 --2- 172.21.15.46:0/1410958820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb357d00 0x5583cb47bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb5accd700 1 -- 172.21.15.46:0/1410958820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb357d00 msgr2=0x5583cb47bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb5accd700 1 --2- 172.21.15.46:0/1410958820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb357d00 0x5583cb47bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb5accd700 1 -- 172.21.15.46:0/1410958820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb2dd4a0 msgr2=0x5583cb47b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb5accd700 1 --2- 172.21.15.46:0/1410958820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb2dd4a0 0x5583cb47b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.504+0000 7fdb5accd700 1 -- 172.21.15.46:0/1410958820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583cb4dd9d0 con 0x5583cb45a400 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.505+0000 7fdb5accd700 1 --2- 172.21.15.46:0/1410958820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb45a400 0x5583cb47c180 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c008f50 tx=0x7fdb4c015da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d05e40bff6645db9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.505+0000 7fdb3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.505+0000 7fdb3dffb700 1 -- 172.21.15.46:0/1410958820 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c010bd0 con 0x5583cb45a400 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.505+0000 7fdb3dffb700 1 -- 172.21.15.46:0/1410958820 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8644+0+0 (secure 0 0 0) 0x7fdb4c010d70 con 0x5583cb45a400 2024-05-09T04:59:12.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.505+0000 7fdb6b146b80 1 -- 172.21.15.46:0/1410958820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5583cb384b20 con 0x5583cb45a400 2024-05-09T04:59:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.505+0000 7fdb3dffb700 1 -- 172.21.15.46:0/1410958820 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fdb4c0153d0 con 0x5583cb45a400 2024-05-09T04:59:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.505+0000 7fdb6b146b80 10 client.17097 fetch_fsmap learned FSMap version 25 2024-05-09T04:59:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.505+0000 7fdb6b146b80 1 -- 172.21.15.46:0/1410958820 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5583cb4dec20 con 0x5583cb45a400 2024-05-09T04:59:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.505+0000 7fdb3dffb700 1 -- 172.21.15.46:0/1410958820 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5583cb384b20 con 0x5583cb45a400 2024-05-09T04:59:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.505+0000 7fdb3dffb700 1 -- 172.21.15.46:0/1410958820 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fdb4c003a00 con 0x5583cb45a400 2024-05-09T04:59:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.505+0000 7fdb6b146b80 10 client.17097 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:59:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.505+0000 7fdb6b146b80 10 client.17097 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:59:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.505+0000 7fdb6b146b80 20 client.17097 populate_metadata read hostname 'smithi046' 2024-05-09T04:59:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.506+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/1410958820 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583cb4df320 0x5583cb4e1740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.506+0000 7fdb6b146b80 4 client.17097 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:59:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.506+0000 7fdb6b146b80 1 -- 172.21.15.46:0/1410958820 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5583cb1f8970 con 0x5583cb4df320 2024-05-09T04:59:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.506+0000 7fdb5a4cc700 1 --2- 172.21.15.46:0/1410958820 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583cb4df320 0x5583cb4e1740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.506+0000 7fdb5a4cc700 1 --2- 172.21.15.46:0/1410958820 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583cb4df320 0x5583cb4e1740 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22c23339ec59e212 server_cookie=dacede569b963b15 in_seq=0 out_seq=0 2024-05-09T04:59:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.506+0000 7fdb3dffb700 10 client.17097 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:12.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.506+0000 7fdb3dffb700 1 -- 172.21.15.46:0/1410958820 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5583cb1f8970 con 0x5583cb4df320 2024-05-09T04:59:12.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb3dffb700 10 client.17097 handle_command_reply: tid=0 2024-05-09T04:59:12.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb6b146b80 1 client.17097 shutdown 2024-05-09T04:59:12.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb6b146b80 1 -- 172.21.15.46:0/1410958820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb45a400 msgr2=0x5583cb47c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:12.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/1410958820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb45a400 0x5583cb47c180 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c008f50 tx=0x7fdb4c015da0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb6b146b80 1 -- 172.21.15.46:0/1410958820 shutdown_connections 2024-05-09T04:59:12.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/1410958820 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5583cb45a400 0x5583cb47c180 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/1410958820 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5583cb4df320 0x5583cb4e1740 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/1410958820 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5583cb2dd4a0 0x5583cb47b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb6b146b80 1 --2- 172.21.15.46:0/1410958820 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5583cb357d00 0x5583cb47bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb6b146b80 1 -- 172.21.15.46:0/1410958820 >> 172.21.15.46:0/1410958820 conn(0x5583cb2e0f70 msgr2=0x5583cb45dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:12.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb3dffb700 0 client.17097 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:12.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb6b146b80 1 -- 172.21.15.46:0/1410958820 shutdown_connections 2024-05-09T04:59:12.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb6b146b80 1 -- 172.21.15.46:0/1410958820 wait complete. 2024-05-09T04:59:12.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb6b146b80 20 client.17097 trim_cache size 0 max 16384 2024-05-09T04:59:12.512 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:59:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T04:59:12.512 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T04:59:12.512 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:59:12.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb5c730700 1 -- 172.21.15.46:0/3465134479 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdb400b29f0 msgr2=0x7fdb400b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:12.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb5c730700 1 --2- 172.21.15.46:0/3465134479 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdb400b29f0 0x7fdb400b4f00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009750 tx=0x7fdb5001a040 comp rx=0 tx=0).stop 2024-05-09T04:59:12.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb5c730700 1 -- 172.21.15.46:0/3465134479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb54108f00 msgr2=0x7fdb5419d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:12.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb5c730700 1 --2- 172.21.15.46:0/3465134479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb54108f00 0x7fdb5419d670 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fdb4400c9e0 tx=0x7fdb4400cda0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb5c730700 1 -- 172.21.15.46:0/3465134479 shutdown_connections 2024-05-09T04:59:12.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb5c730700 1 --2- 172.21.15.46:0/3465134479 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdb400b29f0 0x7fdb400b4f00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb5c730700 1 --2- 172.21.15.46:0/3465134479 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdb54109850 0x7fdb541a1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb5c730700 1 --2- 172.21.15.46:0/3465134479 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdb54102ae0 0x7fdb5419d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb5c730700 1 --2- 172.21.15.46:0/3465134479 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdb54108f00 0x7fdb5419d670 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:12.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.507+0000 7fdb5c730700 1 -- 172.21.15.46:0/3465134479 >> 172.21.15.46:0/3465134479 conn(0x7fdb540fe4e0 msgr2=0x7fdb5410a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:12.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.508+0000 7fdb5c730700 1 -- 172.21.15.46:0/3465134479 shutdown_connections 2024-05-09T04:59:12.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:12.508+0000 7fdb5c730700 1 -- 172.21.15.46:0/3465134479 wait complete. 2024-05-09T04:59:13.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:12 smithi110 ceph-mon[34904]: pgmap v249: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-05-09T04:59:13.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:12 smithi046 ceph-mon[30136]: pgmap v249: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-05-09T04:59:13.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:12 smithi072 ceph-mon[34088]: pgmap v249: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-05-09T04:59:13.363 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T04:59:13.363 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T04:59:13.623 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:15.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:14 smithi110 ceph-mon[34904]: pgmap v250: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T04:59:15.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:14 smithi046 ceph-mon[30136]: pgmap v250: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T04:59:15.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:14 smithi072 ceph-mon[34088]: pgmap v250: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T04:59:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.360+0000 7f4e38b1f700 1 -- 172.21.15.46:0/1730709082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e34108740 msgr2=0x7f4e341100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.360+0000 7f4e38b1f700 1 --2- 172.21.15.46:0/1730709082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e34108740 0x7f4e341100a0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c0099c0 tx=0x7f4e2c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:59:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.361+0000 7f4e38b1f700 1 -- 172.21.15.46:0/1730709082 shutdown_connections 2024-05-09T04:59:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.361+0000 7f4e38b1f700 1 --2- 172.21.15.46:0/1730709082 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e34108740 0x7f4e341100a0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.361+0000 7f4e38b1f700 1 --2- 172.21.15.46:0/1730709082 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e341034c0 0x7f4e341038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.361+0000 7f4e38b1f700 1 --2- 172.21.15.46:0/1730709082 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e34107df0 0x7f4e34108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.361+0000 7f4e38b1f700 1 -- 172.21.15.46:0/1730709082 >> 172.21.15.46:0/1730709082 conn(0x7f4e340fedc0 msgr2=0x7f4e341011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.361+0000 7f4e38b1f700 1 -- 172.21.15.46:0/1730709082 shutdown_connections 2024-05-09T04:59:15.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.361+0000 7f4e38b1f700 1 -- 172.21.15.46:0/1730709082 wait complete. 2024-05-09T04:59:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.362+0000 7f4e38b1f700 1 Processor -- start 2024-05-09T04:59:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.362+0000 7f4e38b1f700 1 -- start start 2024-05-09T04:59:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e38b1f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e341034c0 0x7f4e3419ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e38b1f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e34107df0 0x7f4e3419b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:15.363 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e38b1f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e34108740 0x7f4e3419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e38b1f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f4e341133f0 con 0x7f4e341034c0 2024-05-09T04:59:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e38b1f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f4e34113270 con 0x7f4e34107df0 2024-05-09T04:59:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e38b1f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f4e34113570 con 0x7f4e34108740 2024-05-09T04:59:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e3259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e341034c0 0x7f4e3419ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e3259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e341034c0 0x7f4e3419ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36420/0 (socket says 172.21.15.46:36420) 2024-05-09T04:59:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e3259c700 1 -- 172.21.15.46:0/678082945 learned_addr learned my addr 172.21.15.46:0/678082945 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e31d9b700 1 --2- 172.21.15.46:0/678082945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e34107df0 0x7f4e3419b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e32d9d700 1 --2- 172.21.15.46:0/678082945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e34108740 0x7f4e3419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e3259c700 1 -- 172.21.15.46:0/678082945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e34108740 msgr2=0x7f4e3419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e3259c700 1 --2- 172.21.15.46:0/678082945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e34108740 0x7f4e3419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e3259c700 1 -- 172.21.15.46:0/678082945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e34107df0 msgr2=0x7f4e3419b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e3259c700 1 --2- 172.21.15.46:0/678082945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e34107df0 0x7f4e3419b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e3259c700 1 -- 172.21.15.46:0/678082945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3419fd00 con 0x7f4e341034c0 2024-05-09T04:59:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e3259c700 1 --2- 172.21.15.46:0/678082945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e341034c0 0x7f4e3419ae30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400ba50 tx=0x7f4e2400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:15.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.363+0000 7f4e237fe700 1 -- 172.21.15.46:0/678082945 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e24019070 con 0x7f4e341034c0 2024-05-09T04:59:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.364+0000 7f4e38b1f700 1 -- 172.21.15.46:0/678082945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e3419fff0 con 0x7f4e341034c0 2024-05-09T04:59:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.364+0000 7f4e237fe700 1 -- 172.21.15.46:0/678082945 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e24014440 con 0x7f4e341034c0 2024-05-09T04:59:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.364+0000 7f4e237fe700 1 -- 172.21.15.46:0/678082945 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e2400ce60 con 0x7f4e341034c0 2024-05-09T04:59:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.364+0000 7f4e38b1f700 1 -- 172.21.15.46:0/678082945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e341a7910 con 0x7f4e341034c0 2024-05-09T04:59:15.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.365+0000 7f4e38b1f700 1 -- 172.21.15.46:0/678082945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e340512c0 con 0x7f4e341034c0 2024-05-09T04:59:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.367+0000 7f4e237fe700 1 -- 172.21.15.46:0/678082945 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4e2401c070 con 0x7f4e341034c0 2024-05-09T04:59:15.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.368+0000 7f4e237fe700 1 --2- 172.21.15.46:0/678082945 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4e1c0b29f0 0x7f4e1c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.368+0000 7f4e31d9b700 1 --2- 172.21.15.46:0/678082945 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4e1c0b29f0 0x7f4e1c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.368+0000 7f4e237fe700 1 -- 172.21.15.46:0/678082945 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8644+0+0 (secure 0 0 0) 0x7f4e240e1eb0 con 0x7f4e341034c0 2024-05-09T04:59:15.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.369+0000 7f4e31d9b700 1 --2- 172.21.15.46:0/678082945 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4e1c0b29f0 0x7f4e1c0b4f00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4e34109890 tx=0x7f4e2800b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:15.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.372+0000 7f4e237fe700 1 -- 172.21.15.46:0/678082945 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4e240a98c0 con 0x7f4e341034c0 2024-05-09T04:59:15.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.633+0000 7f4e38b1f700 1 -- 172.21.15.46:0/678082945 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4e3402e5f0 con 0x7f4e341034c0 2024-05-09T04:59:15.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.634+0000 7f4e237fe700 1 -- 172.21.15.46:0/678082945 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7f4e240a91c0 con 0x7f4e341034c0 2024-05-09T04:59:15.634 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T04:59:15.635 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T04:59:15.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.636+0000 7f4e38b1f700 1 -- 172.21.15.46:0/678082945 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4e1c0b29f0 msgr2=0x7f4e1c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:15.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.636+0000 7f4e38b1f700 1 --2- 172.21.15.46:0/678082945 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4e1c0b29f0 0x7f4e1c0b4f00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f4e34109890 tx=0x7f4e2800b410 comp rx=0 tx=0).stop 2024-05-09T04:59:15.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.636+0000 7f4e38b1f700 1 -- 172.21.15.46:0/678082945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e341034c0 msgr2=0x7f4e3419ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:15.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.636+0000 7f4e38b1f700 1 --2- 172.21.15.46:0/678082945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e341034c0 0x7f4e3419ae30 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f4e2400ba50 tx=0x7f4e2400bd60 comp rx=0 tx=0).stop 2024-05-09T04:59:15.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.636+0000 7f4e38b1f700 1 -- 172.21.15.46:0/678082945 shutdown_connections 2024-05-09T04:59:15.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.636+0000 7f4e38b1f700 1 --2- 172.21.15.46:0/678082945 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f4e1c0b29f0 0x7f4e1c0b4f00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:15.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.636+0000 7f4e38b1f700 1 --2- 172.21.15.46:0/678082945 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f4e34108740 0x7f4e3419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:15.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.636+0000 7f4e38b1f700 1 --2- 172.21.15.46:0/678082945 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f4e341034c0 0x7f4e3419ae30 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:15.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.636+0000 7f4e38b1f700 1 --2- 172.21.15.46:0/678082945 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f4e34107df0 0x7f4e3419b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:15.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.636+0000 7f4e38b1f700 1 -- 172.21.15.46:0/678082945 >> 172.21.15.46:0/678082945 conn(0x7f4e340fedc0 msgr2=0x7f4e3406b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:15.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.636+0000 7f4e38b1f700 1 -- 172.21.15.46:0/678082945 shutdown_connections 2024-05-09T04:59:15.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:15.636+0000 7f4e38b1f700 1 -- 172.21.15.46:0/678082945 wait complete. 2024-05-09T04:59:15.638 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T04:59:15.941 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:15 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/678082945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:59:16.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:15 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/678082945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:59:16.244 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T04:59:16.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:15 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/678082945' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T04:59:16.498 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:16.886 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:16 smithi046 ceph-mon[30136]: pgmap v251: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T04:59:16.887 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:16 smithi046 ceph-mon[30136]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T04:59:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:16 smithi072 ceph-mon[34088]: pgmap v251: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T04:59:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:16 smithi072 ceph-mon[34088]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T04:59:17.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:16 smithi110 ceph-mon[34904]: pgmap v251: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T04:59:17.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:16 smithi110 ceph-mon[34904]: osdmap e89: 12 total, 12 up, 12 in 2024-05-09T04:59:18.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.281+0000 7f110cd15700 1 -- 172.21.15.46:0/2773403243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11081034c0 msgr2=0x7f11081038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:18.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.281+0000 7f110cd15700 1 --2- 172.21.15.46:0/2773403243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11081034c0 0x7f11081038d0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f10f80099c0 tx=0x7f10f8009cd0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.281+0000 7f110cd15700 1 -- 172.21.15.46:0/2773403243 shutdown_connections 2024-05-09T04:59:18.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.281+0000 7f110cd15700 1 --2- 172.21.15.46:0/2773403243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1108108740 0x7f11081100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.281+0000 7f110cd15700 1 --2- 172.21.15.46:0/2773403243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11081034c0 0x7f11081038d0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.281+0000 7f110cd15700 1 --2- 172.21.15.46:0/2773403243 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1108107df0 0x7f1108108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.281+0000 7f110cd15700 1 -- 172.21.15.46:0/2773403243 >> 172.21.15.46:0/2773403243 conn(0x7f11080fedf0 msgr2=0x7f1108101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:18.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.281+0000 7f110cd15700 1 -- 172.21.15.46:0/2773403243 shutdown_connections 2024-05-09T04:59:18.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.281+0000 7f110cd15700 1 -- 172.21.15.46:0/2773403243 wait complete. 2024-05-09T04:59:18.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.282+0000 7f110cd15700 1 Processor -- start 2024-05-09T04:59:18.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.282+0000 7f110cd15700 1 -- start start 2024-05-09T04:59:18.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.282+0000 7f110cd15700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11081034c0 0x7f110819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.282+0000 7f110cd15700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1108107df0 0x7f110819c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.282+0000 7f110cd15700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1108108740 0x7f110819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.282+0000 7f110cd15700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f11081133c0 con 0x7f11081034c0 2024-05-09T04:59:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.282+0000 7f110cd15700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1108113240 con 0x7f1108107df0 2024-05-09T04:59:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.282+0000 7f110cd15700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1108113540 con 0x7f1108108740 2024-05-09T04:59:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.282+0000 7f110659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11081034c0 0x7f110819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.282+0000 7f110659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11081034c0 0x7f110819bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36434/0 (socket says 172.21.15.46:36434) 2024-05-09T04:59:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.282+0000 7f110659c700 1 -- 172.21.15.46:0/3730069723 learned_addr learned my addr 172.21.15.46:0/3730069723 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.282+0000 7f1106d9d700 1 --2- 172.21.15.46:0/3730069723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1108108740 0x7f110819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.283+0000 7f1105d9b700 1 --2- 172.21.15.46:0/3730069723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1108107df0 0x7f110819c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.283+0000 7f110659c700 1 -- 172.21.15.46:0/3730069723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1108108740 msgr2=0x7f110819c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.283+0000 7f110659c700 1 --2- 172.21.15.46:0/3730069723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1108108740 0x7f110819c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.283+0000 7f110659c700 1 -- 172.21.15.46:0/3730069723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1108107df0 msgr2=0x7f110819c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:18.285 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.283+0000 7f110659c700 1 --2- 172.21.15.46:0/3730069723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1108107df0 0x7f110819c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.283+0000 7f110659c700 1 -- 172.21.15.46:0/3730069723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11081a6fd0 con 0x7f11081034c0 2024-05-09T04:59:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.283+0000 7f110659c700 1 --2- 172.21.15.46:0/3730069723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11081034c0 0x7f110819bc80 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009990 tx=0x7f10f80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.283+0000 7f10f77fe700 1 -- 172.21.15.46:0/3730069723 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f8026070 con 0x7f11081034c0 2024-05-09T04:59:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.283+0000 7f10f77fe700 1 -- 172.21.15.46:0/3730069723 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10f8003f20 con 0x7f11081034c0 2024-05-09T04:59:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.283+0000 7f110cd15700 1 -- 172.21.15.46:0/3730069723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11081a7260 con 0x7f11081034c0 2024-05-09T04:59:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.283+0000 7f10f77fe700 1 -- 172.21.15.46:0/3730069723 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f800fcc0 con 0x7f11081034c0 2024-05-09T04:59:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.283+0000 7f110cd15700 1 -- 172.21.15.46:0/3730069723 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11081a76b0 con 0x7f11081034c0 2024-05-09T04:59:18.286 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.286+0000 7f10f77fe700 1 -- 172.21.15.46:0/3730069723 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f10f8029070 con 0x7f11081034c0 2024-05-09T04:59:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.287+0000 7f111b72bb80 1 Processor -- start 2024-05-09T04:59:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.287+0000 7f111b72bb80 1 -- start start 2024-05-09T04:59:18.287 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.287+0000 7f10f77fe700 1 --2- 172.21.15.46:0/3730069723 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f10f00b29f0 0x7f10f00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.287+0000 7f111b72bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771b90d00 0x555771c99240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.287+0000 7f1106d9d700 1 --2- 172.21.15.46:0/3730069723 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f10f00b29f0 0x7f10f00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.287+0000 7f111b72bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 0x555771c99780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.287+0000 7f111b72bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771c93400 0x555771c93810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.287+0000 7f111b72bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555771bb60a0 con 0x555771c93400 2024-05-09T04:59:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.287+0000 7f111b72bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x555771b5b9a0 con 0x555771b164a0 2024-05-09T04:59:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.287+0000 7f10f77fe700 1 -- 172.21.15.46:0/3730069723 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f10f80e2ff0 con 0x7f11081034c0 2024-05-09T04:59:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.287+0000 7f111b72bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555771bb2060 con 0x555771b90d00 2024-05-09T04:59:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.288+0000 7f1106d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771c93400 0x555771c93810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.288+0000 7f1106d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771c93400 0x555771c93810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36438/0 (socket says 172.21.15.46:36438) 2024-05-09T04:59:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.288+0000 7f1106d9d700 1 -- 172.21.15.46:0/2276571567 learned_addr learned my addr 172.21.15.46:0/2276571567 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:18.289 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.288+0000 7f1105d9b700 1 --2- 172.21.15.46:0/2276571567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 0x555771c99780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.288+0000 7f110659c700 1 --2- 172.21.15.46:0/2276571567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771b90d00 0x555771c99240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.288+0000 7f1106d9d700 1 --2- 172.21.15.46:0/3730069723 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f10f00b29f0 0x7f10f00b4f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1100009c70 tx=0x7f1100004320 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.288+0000 7f110659c700 1 -- 172.21.15.46:0/2276571567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 msgr2=0x555771c99780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.288+0000 7f110659c700 1 --2- 172.21.15.46:0/2276571567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 0x555771c99780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.288+0000 7f110659c700 1 -- 172.21.15.46:0/2276571567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771c93400 msgr2=0x555771c93810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.288+0000 7f110659c700 1 --2- 172.21.15.46:0/2276571567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771c93400 0x555771c93810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.288+0000 7f110659c700 1 -- 172.21.15.46:0/2276571567 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555771b4c500 con 0x555771b90d00 2024-05-09T04:59:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.288+0000 7f1105d9b700 1 --2- 172.21.15.46:0/2276571567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 0x555771c99780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:59:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.288+0000 7f110659c700 1 --2- 172.21.15.46:0/2276571567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771b90d00 0x555771c99240 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f10f80e06d0 tx=0x7f10f8009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3afc5efc318f6a4b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.288+0000 7f10f4ff9700 1 -- 172.21.15.46:0/2276571567 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f80aa2f0 con 0x555771b90d00 2024-05-09T04:59:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.289+0000 7f10f4ff9700 1 -- 172.21.15.46:0/2276571567 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10f80ae6a0 con 0x555771b90d00 2024-05-09T04:59:18.290 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.289+0000 7f10f4ff9700 1 -- 172.21.15.46:0/2276571567 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f80b2e90 con 0x555771b90d00 2024-05-09T04:59:18.294 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.289+0000 7f111b72bb80 1 -- 172.21.15.46:0/2276571567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771b90d00 msgr2=0x555771c99240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.289+0000 7f111b72bb80 1 --2- 172.21.15.46:0/2276571567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771b90d00 0x555771c99240 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f10f80e06d0 tx=0x7f10f8009cd0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.289+0000 7f111b72bb80 1 -- 172.21.15.46:0/2276571567 shutdown_connections 2024-05-09T04:59:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.289+0000 7f111b72bb80 1 --2- 172.21.15.46:0/2276571567 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771c93400 0x555771c93810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.289+0000 7f111b72bb80 1 --2- 172.21.15.46:0/2276571567 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771b90d00 0x555771c99240 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.289+0000 7f111b72bb80 1 --2- 172.21.15.46:0/2276571567 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 0x555771c99780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.289+0000 7f111b72bb80 1 -- 172.21.15.46:0/2276571567 >> 172.21.15.46:0/2276571567 conn(0x555771b19f70 msgr2=0x555771c8e220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.290+0000 7f111b72bb80 1 -- 172.21.15.46:0/2276571567 shutdown_connections 2024-05-09T04:59:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.290+0000 7f111b72bb80 1 -- 172.21.15.46:0/2276571567 wait complete. 2024-05-09T04:59:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.290+0000 7f111b72bb80 1 -- start start 2024-05-09T04:59:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.290+0000 7f111b72bb80 1 Processor -- start 2024-05-09T04:59:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.290+0000 7f111b72bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771b164a0 0x555771ca2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.290+0000 7f111b72bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b90d00 0x555771ca3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.290+0000 7f111b72bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771c93400 0x555771c9cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.290+0000 7f111b72bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555771bb2060 con 0x555771b164a0 2024-05-09T04:59:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.290+0000 7f111b72bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x555771b50b20 con 0x555771b90d00 2024-05-09T04:59:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.290+0000 7f111b72bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555771c93fb0 con 0x555771c93400 2024-05-09T04:59:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f110659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771b164a0 0x555771ca2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f111b72bb80 10 client.0 _ll_register_callbacks cb 0x555771a276c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:59:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f110659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771b164a0 0x555771ca2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36448/0 (socket says 172.21.15.46:36448) 2024-05-09T04:59:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f110659c700 1 -- 172.21.15.46:0/3971673815 learned_addr learned my addr 172.21.15.46:0/3971673815 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f1106d9d700 1 --2- 172.21.15.46:0/3971673815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771c93400 0x555771c9cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f1105d9b700 1 --2- 172.21.15.46:0/3971673815 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b90d00 0x555771ca3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f110659c700 1 -- 172.21.15.46:0/3971673815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771c93400 msgr2=0x555771c9cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f110659c700 1 --2- 172.21.15.46:0/3971673815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771c93400 0x555771c9cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f110659c700 1 -- 172.21.15.46:0/3971673815 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b90d00 msgr2=0x555771ca3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f110659c700 1 --2- 172.21.15.46:0/3971673815 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b90d00 0x555771ca3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f110659c700 1 -- 172.21.15.46:0/3971673815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555771a31490 con 0x555771b164a0 2024-05-09T04:59:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f110659c700 1 --2- 172.21.15.46:0/3971673815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771b164a0 0x555771ca2bd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f10f800c010 tx=0x7f10f80aa780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b87c556a73a8fab6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f10edffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:59:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f10edffb700 1 -- 172.21.15.46:0/3971673815 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10f80c4060 con 0x555771b164a0 2024-05-09T04:59:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f10edffb700 1 -- 172.21.15.46:0/3971673815 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f10f80b2e40 con 0x555771b164a0 2024-05-09T04:59:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.291+0000 7f111b72bb80 1 -- 172.21.15.46:0/3971673815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10f80ae6a0 con 0x555771b164a0 2024-05-09T04:59:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.292+0000 7f10edffb700 1 -- 172.21.15.46:0/3971673815 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f10f80aa2f0 con 0x555771b164a0 2024-05-09T04:59:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.292+0000 7f111b72bb80 10 client.17115 fetch_fsmap learned FSMap version 25 2024-05-09T04:59:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.292+0000 7f111b72bb80 1 -- 172.21.15.46:0/3971673815 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555771caa7f0 con 0x555771b164a0 2024-05-09T04:59:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.292+0000 7f10edffb700 1 -- 172.21.15.46:0/3971673815 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10f80ae6a0 con 0x555771b164a0 2024-05-09T04:59:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.292+0000 7f10edffb700 1 -- 172.21.15.46:0/3971673815 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f10f80ae300 con 0x555771b164a0 2024-05-09T04:59:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.292+0000 7f111b72bb80 10 client.17115 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:59:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.292+0000 7f111b72bb80 10 client.17115 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:59:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.292+0000 7f111b72bb80 20 client.17115 populate_metadata read hostname 'smithi046' 2024-05-09T04:59:18.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.292+0000 7f111b72bb80 1 --2- 172.21.15.46:0/3971673815 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x555771caccc0 0x555771caf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.292+0000 7f111b72bb80 4 client.17115 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:59:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.292+0000 7f111b72bb80 1 -- 172.21.15.46:0/3971673815 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555771bc3a80 con 0x555771caccc0 2024-05-09T04:59:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.292+0000 7f1105d9b700 1 --2- 172.21.15.46:0/3971673815 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x555771caccc0 0x555771caf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.293+0000 7f10edffb700 10 client.17115 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:59:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.293+0000 7f1105d9b700 1 --2- 172.21.15.46:0/3971673815 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x555771caccc0 0x555771caf0e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df8a8f848582013c server_cookie=fe61a13bfdbe2669 in_seq=0 out_seq=0 2024-05-09T04:59:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.294+0000 7f10edffb700 1 -- 172.21.15.46:0/3971673815 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555771bc3a80 con 0x555771caccc0 2024-05-09T04:59:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.294+0000 7f10edffb700 10 client.17115 handle_command_reply: tid=0 2024-05-09T04:59:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.294+0000 7f111b72bb80 1 client.17115 shutdown 2024-05-09T04:59:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.294+0000 7f111b72bb80 1 -- 172.21.15.46:0/3971673815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771b164a0 msgr2=0x555771ca2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.294+0000 7f111b72bb80 1 --2- 172.21.15.46:0/3971673815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771b164a0 0x555771ca2bd0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f10f800c010 tx=0x7f10f80aa780 comp rx=0 tx=0).stop 2024-05-09T04:59:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.294+0000 7f111b72bb80 1 -- 172.21.15.46:0/3971673815 shutdown_connections 2024-05-09T04:59:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.294+0000 7f111b72bb80 1 --2- 172.21.15.46:0/3971673815 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x555771caccc0 0x555771caf0e0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.294+0000 7f111b72bb80 1 --2- 172.21.15.46:0/3971673815 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771c93400 0x555771c9cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.294+0000 7f111b72bb80 1 --2- 172.21.15.46:0/3971673815 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771b164a0 0x555771ca2bd0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.298 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.294+0000 7f111b72bb80 1 --2- 172.21.15.46:0/3971673815 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b90d00 0x555771ca3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.294+0000 7f111b72bb80 1 -- 172.21.15.46:0/3971673815 >> 172.21.15.46:0/3971673815 conn(0x555771b19f70 msgr2=0x555771c87ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.294+0000 7f10edffb700 0 client.17115 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:59:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.294+0000 7f111b72bb80 1 -- 172.21.15.46:0/3971673815 shutdown_connections 2024-05-09T04:59:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.294+0000 7f111b72bb80 1 -- 172.21.15.46:0/3971673815 wait complete. 2024-05-09T04:59:18.299 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.294+0000 7f111b72bb80 20 client.17115 trim_cache size 0 max 16384 2024-05-09T04:59:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.309+0000 7f111b72bb80 1 Processor -- start 2024-05-09T04:59:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.309+0000 7f111b72bb80 1 -- start start 2024-05-09T04:59:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f111b72bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 0x555771c987d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f111b72bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771b90d00 0x555771c9c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f111b72bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771c93400 0x555771c9cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f111b72bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f11081af4f0 con 0x555771b90d00 2024-05-09T04:59:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f111b72bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f11081acd80 con 0x555771b164a0 2024-05-09T04:59:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f111b72bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f11081aabf0 con 0x555771c93400 2024-05-09T04:59:18.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f1105d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771b90d00 0x555771c9c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f1105d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771b90d00 0x555771c9c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36454/0 (socket says 172.21.15.46:36454) 2024-05-09T04:59:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f1105d9b700 1 -- 172.21.15.46:0/3466533617 learned_addr learned my addr 172.21.15.46:0/3466533617 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f110659c700 1 --2- 172.21.15.46:0/3466533617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 0x555771c987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f1105d9b700 1 -- 172.21.15.46:0/3466533617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771c93400 msgr2=0x555771c9cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f1106d9d700 1 --2- 172.21.15.46:0/3466533617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771c93400 0x555771c9cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f1105d9b700 1 --2- 172.21.15.46:0/3466533617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771c93400 0x555771c9cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f1105d9b700 1 -- 172.21.15.46:0/3466533617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 msgr2=0x555771c987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f1105d9b700 1 --2- 172.21.15.46:0/3466533617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 0x555771c987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f1105d9b700 1 -- 172.21.15.46:0/3466533617 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555771c8a9d0 con 0x555771b90d00 2024-05-09T04:59:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f1106d9d700 1 --2- 172.21.15.46:0/3466533617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771c93400 0x555771c9cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:59:18.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f110659c700 1 --2- 172.21.15.46:0/3466533617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 0x555771c987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:59:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.310+0000 7f1105d9b700 1 --2- 172.21.15.46:0/3466533617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771b90d00 0x555771c9c880 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f10fc013010 tx=0x7f10fc00ab20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a9efc3f6d3f8fa server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.311+0000 7f10effff700 1 -- 172.21.15.46:0/3466533617 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc004a30 con 0x555771b90d00 2024-05-09T04:59:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.311+0000 7f10effff700 1 -- 172.21.15.46:0/3466533617 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10fc0054d0 con 0x555771b90d00 2024-05-09T04:59:18.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.311+0000 7f10effff700 1 -- 172.21.15.46:0/3466533617 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10fc022780 con 0x555771b90d00 2024-05-09T04:59:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.311+0000 7f111b72bb80 1 -- 172.21.15.46:0/3466533617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771b90d00 msgr2=0x555771c9c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.311+0000 7f111b72bb80 1 --2- 172.21.15.46:0/3466533617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771b90d00 0x555771c9c880 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f10fc013010 tx=0x7f10fc00ab20 comp rx=0 tx=0).stop 2024-05-09T04:59:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.311+0000 7f111b72bb80 1 -- 172.21.15.46:0/3466533617 shutdown_connections 2024-05-09T04:59:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.311+0000 7f111b72bb80 1 --2- 172.21.15.46:0/3466533617 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771c93400 0x555771c9cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.311+0000 7f111b72bb80 1 --2- 172.21.15.46:0/3466533617 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 0x555771c987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.311+0000 7f111b72bb80 1 --2- 172.21.15.46:0/3466533617 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771b90d00 0x555771c9c880 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.311+0000 7f111b72bb80 1 -- 172.21.15.46:0/3466533617 >> 172.21.15.46:0/3466533617 conn(0x555771b19f70 msgr2=0x555771c980c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.311+0000 7f111b72bb80 1 -- 172.21.15.46:0/3466533617 shutdown_connections 2024-05-09T04:59:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.312+0000 7f111b72bb80 1 -- 172.21.15.46:0/3466533617 wait complete. 2024-05-09T04:59:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.312+0000 7f111b72bb80 1 -- start start 2024-05-09T04:59:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.312+0000 7f111b72bb80 1 Processor -- start 2024-05-09T04:59:18.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.312+0000 7f111b72bb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 0x555771ca8f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.312+0000 7f111b72bb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771b90d00 0x555771ca6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.312+0000 7f111b72bb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771c93400 0x555771ca6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.312+0000 7f111b72bb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f11081ab240 con 0x555771c93400 2024-05-09T04:59:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.312+0000 7f111b72bb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f11081ac6b0 con 0x555771b164a0 2024-05-09T04:59:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.312+0000 7f111b72bb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f11081ab910 con 0x555771b90d00 2024-05-09T04:59:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.312+0000 7f111b72bb80 10 client.0 _ll_register_callbacks cb 0x555771a276c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:59:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.312+0000 7f1106d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771c93400 0x555771ca6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.312+0000 7f1106d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771c93400 0x555771ca6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36462/0 (socket says 172.21.15.46:36462) 2024-05-09T04:59:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.312+0000 7f1106d9d700 1 -- 172.21.15.46:0/36820096 learned_addr learned my addr 172.21.15.46:0/36820096 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.312+0000 7f1105d9b700 1 --2- 172.21.15.46:0/36820096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771b90d00 0x555771ca6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.313+0000 7f110659c700 1 --2- 172.21.15.46:0/36820096 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 0x555771ca8f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.313+0000 7f1106d9d700 1 -- 172.21.15.46:0/36820096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771b90d00 msgr2=0x555771ca6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.313+0000 7f1106d9d700 1 --2- 172.21.15.46:0/36820096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771b90d00 0x555771ca6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.318 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.313+0000 7f1106d9d700 1 -- 172.21.15.46:0/36820096 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 msgr2=0x555771ca8f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.313+0000 7f1106d9d700 1 --2- 172.21.15.46:0/36820096 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 0x555771ca8f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.313+0000 7f1106d9d700 1 -- 172.21.15.46:0/36820096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555771ca6d80 con 0x555771c93400 2024-05-09T04:59:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.313+0000 7f110659c700 1 --2- 172.21.15.46:0/36820096 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 0x555771ca8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:59:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.313+0000 7f1106d9d700 1 --2- 172.21.15.46:0/36820096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771c93400 0x555771ca6840 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1100008cd0 tx=0x7f1100004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57d7f0bde49e31a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.313+0000 7f10edffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:59:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.313+0000 7f10edffb700 1 -- 172.21.15.46:0/36820096 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1100029070 con 0x555771c93400 2024-05-09T04:59:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.313+0000 7f10edffb700 1 -- 172.21.15.46:0/36820096 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f1100004db0 con 0x555771c93400 2024-05-09T04:59:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.313+0000 7f111b72bb80 1 -- 172.21.15.46:0/36820096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10fc0054d0 con 0x555771c93400 2024-05-09T04:59:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.313+0000 7f10edffb700 1 -- 172.21.15.46:0/36820096 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f1100034050 con 0x555771c93400 2024-05-09T04:59:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.313+0000 7f111b72bb80 10 client.17121 fetch_fsmap learned FSMap version 25 2024-05-09T04:59:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.313+0000 7f111b72bb80 1 -- 172.21.15.46:0/36820096 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555771d16d70 con 0x555771c93400 2024-05-09T04:59:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.314+0000 7f10edffb700 1 -- 172.21.15.46:0/36820096 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10fc0054d0 con 0x555771c93400 2024-05-09T04:59:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.314+0000 7f10edffb700 1 -- 172.21.15.46:0/36820096 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f1100018040 con 0x555771c93400 2024-05-09T04:59:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.314+0000 7f111b72bb80 10 client.17121 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:59:18.319 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.314+0000 7f111b72bb80 10 client.17121 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T04:59:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.314+0000 7f111b72bb80 20 client.17121 populate_metadata read hostname 'smithi046' 2024-05-09T04:59:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.314+0000 7f111b72bb80 1 --2- 172.21.15.46:0/36820096 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x555771d17480 0x555771d198a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.314+0000 7f111b72bb80 4 client.17121 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:59:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.314+0000 7f111b72bb80 1 -- 172.21.15.46:0/36820096 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555771bbdb20 con 0x555771d17480 2024-05-09T04:59:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.314+0000 7f110659c700 1 --2- 172.21.15.46:0/36820096 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x555771d17480 0x555771d198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.314+0000 7f110659c700 1 --2- 172.21.15.46:0/36820096 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x555771d17480 0x555771d198a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=382e239ed99165e3 server_cookie=8f22728a809459e6 in_seq=0 out_seq=0 2024-05-09T04:59:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.314+0000 7f10edffb700 10 client.17121 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T04:59:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.315+0000 7f10edffb700 1 -- 172.21.15.46:0/36820096 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555771bbdb20 con 0x555771d17480 2024-05-09T04:59:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.315+0000 7f10edffb700 10 client.17121 handle_command_reply: tid=0 2024-05-09T04:59:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.315+0000 7f111b72bb80 1 client.17121 shutdown 2024-05-09T04:59:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.315+0000 7f111b72bb80 1 -- 172.21.15.46:0/36820096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771c93400 msgr2=0x555771ca6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.315+0000 7f111b72bb80 1 --2- 172.21.15.46:0/36820096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771c93400 0x555771ca6840 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1100008cd0 tx=0x7f1100004b40 comp rx=0 tx=0).stop 2024-05-09T04:59:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.315+0000 7f111b72bb80 1 -- 172.21.15.46:0/36820096 shutdown_connections 2024-05-09T04:59:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.315+0000 7f111b72bb80 1 --2- 172.21.15.46:0/36820096 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555771c93400 0x555771ca6840 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.320 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.315+0000 7f111b72bb80 1 --2- 172.21.15.46:0/36820096 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555771b164a0 0x555771ca8f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.315+0000 7f111b72bb80 1 --2- 172.21.15.46:0/36820096 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x555771d17480 0x555771d198a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.315+0000 7f111b72bb80 1 --2- 172.21.15.46:0/36820096 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555771b90d00 0x555771ca6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.315+0000 7f111b72bb80 1 -- 172.21.15.46:0/36820096 >> 172.21.15.46:0/36820096 conn(0x555771b19f70 msgr2=0x555771c96cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.315+0000 7f10edffb700 0 client.17121 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T04:59:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.315+0000 7f111b72bb80 1 -- 172.21.15.46:0/36820096 shutdown_connections 2024-05-09T04:59:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.315+0000 7f111b72bb80 1 -- 172.21.15.46:0/36820096 wait complete. 2024-05-09T04:59:18.321 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.315+0000 7f111b72bb80 20 client.17121 trim_cache size 0 max 16384 2024-05-09T04:59:18.321 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:59:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.316+0000 7f110cd15700 1 -- 172.21.15.46:0/3730069723 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f10f00b29f0 msgr2=0x7f10f00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.316+0000 7f110cd15700 1 --2- 172.21.15.46:0/3730069723 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f10f00b29f0 0x7f10f00b4f00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f1100009c70 tx=0x7f1100004320 comp rx=0 tx=0).stop 2024-05-09T04:59:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.316+0000 7f110cd15700 1 -- 172.21.15.46:0/3730069723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11081034c0 msgr2=0x7f110819bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.316+0000 7f110cd15700 1 --2- 172.21.15.46:0/3730069723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11081034c0 0x7f110819bc80 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f10f8009990 tx=0x7f10f80047b0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.316+0000 7f110cd15700 1 -- 172.21.15.46:0/3730069723 shutdown_connections 2024-05-09T04:59:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.316+0000 7f110cd15700 1 --2- 172.21.15.46:0/3730069723 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f10f00b29f0 0x7f10f00b4f00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.316+0000 7f110cd15700 1 --2- 172.21.15.46:0/3730069723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1108108740 0x7f110819c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.316+0000 7f110cd15700 1 --2- 172.21.15.46:0/3730069723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f11081034c0 0x7f110819bc80 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.322 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.316+0000 7f110cd15700 1 --2- 172.21.15.46:0/3730069723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1108107df0 0x7f110819c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.316+0000 7f110cd15700 1 -- 172.21.15.46:0/3730069723 >> 172.21.15.46:0/3730069723 conn(0x7f11080fedf0 msgr2=0x7f1108107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.316+0000 7f110cd15700 1 -- 172.21.15.46:0/3730069723 shutdown_connections 2024-05-09T04:59:18.323 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:18.316+0000 7f110cd15700 1 -- 172.21.15.46:0/3730069723 wait complete. 2024-05-09T04:59:19.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:18 smithi046 ceph-mon[30136]: pgmap v253: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T04:59:19.058 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T04:59:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:18 smithi072 ceph-mon[34088]: pgmap v253: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T04:59:19.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:18 smithi110 ceph-mon[34904]: pgmap v253: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T04:59:19.312 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:21.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:20 smithi046 ceph-mon[30136]: pgmap v254: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T04:59:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:20 smithi072 ceph-mon[34088]: pgmap v254: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T04:59:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.137+0000 7fca4ce57700 1 -- 172.21.15.46:0/27528084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca48107e10 msgr2=0x7fca48108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.137+0000 7fca4ce57700 1 --2- 172.21.15.46:0/27528084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca48107e10 0x7fca48108220 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fca300099c0 tx=0x7fca30009cd0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.138+0000 7fca4ce57700 1 -- 172.21.15.46:0/27528084 shutdown_connections 2024-05-09T04:59:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.138+0000 7fca4ce57700 1 --2- 172.21.15.46:0/27528084 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca48108760 0x7fca481100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.138+0000 7fca4ce57700 1 --2- 172.21.15.46:0/27528084 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca481034e0 0x7fca481038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.138+0000 7fca4ce57700 1 --2- 172.21.15.46:0/27528084 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca48107e10 0x7fca48108220 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.138+0000 7fca4ce57700 1 -- 172.21.15.46:0/27528084 >> 172.21.15.46:0/27528084 conn(0x7fca480fef30 msgr2=0x7fca48101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.138+0000 7fca4ce57700 1 -- 172.21.15.46:0/27528084 shutdown_connections 2024-05-09T04:59:21.138 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.138+0000 7fca4ce57700 1 -- 172.21.15.46:0/27528084 wait complete. 2024-05-09T04:59:21.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.138+0000 7fca4ce57700 1 Processor -- start 2024-05-09T04:59:21.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca4ce57700 1 -- start start 2024-05-09T04:59:21.140 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca4ce57700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca481034e0 0x7fca4819bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca4ce57700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca48107e10 0x7fca4819c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca4ce57700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca48108760 0x7fca481a6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca4ce57700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fca481133c0 con 0x7fca481034e0 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca4ce57700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fca48113240 con 0x7fca48108760 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca4ce57700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fca48113540 con 0x7fca48107e10 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca4659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca481034e0 0x7fca4819bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca4659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca481034e0 0x7fca4819bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36482/0 (socket says 172.21.15.46:36482) 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca4659c700 1 -- 172.21.15.46:0/2104070212 learned_addr learned my addr 172.21.15.46:0/2104070212 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca46d9d700 1 --2- 172.21.15.46:0/2104070212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca48108760 0x7fca481a6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca45d9b700 1 --2- 172.21.15.46:0/2104070212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca48107e10 0x7fca4819c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca4659c700 1 -- 172.21.15.46:0/2104070212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca48107e10 msgr2=0x7fca4819c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca4659c700 1 --2- 172.21.15.46:0/2104070212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca48107e10 0x7fca4819c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca4659c700 1 -- 172.21.15.46:0/2104070212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca48108760 msgr2=0x7fca481a6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca4659c700 1 --2- 172.21.15.46:0/2104070212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca48108760 0x7fca481a6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.139+0000 7fca4659c700 1 -- 172.21.15.46:0/2104070212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca4819cad0 con 0x7fca481034e0 2024-05-09T04:59:21.141 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.140+0000 7fca4659c700 1 --2- 172.21.15.46:0/2104070212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca481034e0 0x7fca4819bce0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fca3800ef10 tx=0x7fca3800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.140+0000 7fca3f7fe700 1 -- 172.21.15.46:0/2104070212 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca3800c9c0 con 0x7fca481034e0 2024-05-09T04:59:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.140+0000 7fca3f7fe700 1 -- 172.21.15.46:0/2104070212 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca38019070 con 0x7fca481034e0 2024-05-09T04:59:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.140+0000 7fca4ce57700 1 -- 172.21.15.46:0/2104070212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca481a7360 con 0x7fca481034e0 2024-05-09T04:59:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.140+0000 7fca3f7fe700 1 -- 172.21.15.46:0/2104070212 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca38014480 con 0x7fca481034e0 2024-05-09T04:59:21.142 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.140+0000 7fca4ce57700 1 -- 172.21.15.46:0/2104070212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca481a7870 con 0x7fca481034e0 2024-05-09T04:59:21.143 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.143+0000 7fca3f7fe700 1 -- 172.21.15.46:0/2104070212 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fca38012050 con 0x7fca481034e0 2024-05-09T04:59:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.144+0000 7fca3f7fe700 1 --2- 172.21.15.46:0/2104070212 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fca340b2a70 0x7fca340b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.144+0000 7fca5b86db80 1 Processor -- start 2024-05-09T04:59:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.144+0000 7fca45d9b700 1 --2- 172.21.15.46:0/2104070212 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fca340b2a70 0x7fca340b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.144+0000 7fca5b86db80 1 -- start start 2024-05-09T04:59:21.144 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.144+0000 7fca3f7fe700 1 -- 172.21.15.46:0/2104070212 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7fca380e25f0 con 0x7fca481034e0 2024-05-09T04:59:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca5b86db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d600160 0x555b2d6994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca5b86db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d5164a0 0x555b2d699a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca5b86db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 0x555b2d693ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca5b86db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555b2d5b5ff0 con 0x555b2d600160 2024-05-09T04:59:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca5b86db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x555b2d5547f0 con 0x555b2d5164a0 2024-05-09T04:59:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca5b86db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555b2d5b1fb0 con 0x555b2d6936a0 2024-05-09T04:59:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca45d9b700 1 --2- 172.21.15.46:0/2104070212 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fca340b2a70 0x7fca340b4f80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fca30000c00 tx=0x7fca3001a040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca4659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d600160 0x555b2d6994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.146 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca4659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d600160 0x555b2d6994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36488/0 (socket says 172.21.15.46:36488) 2024-05-09T04:59:21.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca4659c700 1 -- 172.21.15.46:0/902744131 learned_addr learned my addr 172.21.15.46:0/902744131 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:21.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca45d9b700 1 --2- 172.21.15.46:0/902744131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d5164a0 0x555b2d699a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca46d9d700 1 --2- 172.21.15.46:0/902744131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 0x555b2d693ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca4659c700 1 -- 172.21.15.46:0/902744131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 msgr2=0x555b2d693ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:21.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca4659c700 1 --2- 172.21.15.46:0/902744131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 0x555b2d693ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca4659c700 1 -- 172.21.15.46:0/902744131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d5164a0 msgr2=0x555b2d699a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:21.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca4659c700 1 --2- 172.21.15.46:0/902744131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d5164a0 0x555b2d699a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca4659c700 1 -- 172.21.15.46:0/902744131 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b2d54c370 con 0x555b2d600160 2024-05-09T04:59:21.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca46d9d700 1 --2- 172.21.15.46:0/902744131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 0x555b2d693ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:59:21.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.145+0000 7fca4659c700 1 --2- 172.21.15.46:0/902744131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d600160 0x555b2d6994f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fca380090a0 tx=0x7fca380a9170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5330d03086a65b73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:21.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.146+0000 7fca3cff9700 1 -- 172.21.15.46:0/902744131 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca380a9330 con 0x555b2d600160 2024-05-09T04:59:21.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.146+0000 7fca3cff9700 1 -- 172.21.15.46:0/902744131 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca380a9f20 con 0x555b2d600160 2024-05-09T04:59:21.147 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.146+0000 7fca3cff9700 1 -- 172.21.15.46:0/902744131 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca380a9a80 con 0x555b2d600160 2024-05-09T04:59:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.146+0000 7fca5b86db80 1 -- 172.21.15.46:0/902744131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d600160 msgr2=0x555b2d6994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.146+0000 7fca5b86db80 1 --2- 172.21.15.46:0/902744131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d600160 0x555b2d6994f0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fca380090a0 tx=0x7fca380a9170 comp rx=0 tx=0).stop 2024-05-09T04:59:21.150 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.147+0000 7fca5b86db80 1 -- 172.21.15.46:0/902744131 shutdown_connections 2024-05-09T04:59:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.147+0000 7fca5b86db80 1 --2- 172.21.15.46:0/902744131 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 0x555b2d693ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.147+0000 7fca5b86db80 1 --2- 172.21.15.46:0/902744131 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d600160 0x555b2d6994f0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.147+0000 7fca5b86db80 1 --2- 172.21.15.46:0/902744131 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d5164a0 0x555b2d699a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.147+0000 7fca5b86db80 1 -- 172.21.15.46:0/902744131 >> 172.21.15.46:0/902744131 conn(0x555b2d5766a0 msgr2=0x555b2d68e400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.147+0000 7fca5b86db80 1 -- 172.21.15.46:0/902744131 shutdown_connections 2024-05-09T04:59:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.147+0000 7fca5b86db80 1 -- 172.21.15.46:0/902744131 wait complete. 2024-05-09T04:59:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.147+0000 7fca5b86db80 1 -- start start 2024-05-09T04:59:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.147+0000 7fca5b86db80 1 Processor -- start 2024-05-09T04:59:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.147+0000 7fca5b86db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d5164a0 0x555b2d69d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.147+0000 7fca5b86db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 0x555b2d69d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.147+0000 7fca5b86db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d6936a0 0x555b2d69dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca5b86db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x555b2d431ba0 con 0x555b2d6936a0 2024-05-09T04:59:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca5b86db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x555b2d570150 con 0x555b2d600160 2024-05-09T04:59:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca5b86db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x555b2d6a01d0 con 0x555b2d5164a0 2024-05-09T04:59:21.151 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca5b86db80 10 client.0 _ll_register_callbacks cb 0x555b2d4276c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca46d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d6936a0 0x555b2d69dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca46d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d6936a0 0x555b2d69dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36500/0 (socket says 172.21.15.46:36500) 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca46d9d700 1 -- 172.21.15.46:0/447374453 learned_addr learned my addr 172.21.15.46:0/447374453 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca4659c700 1 --2- 172.21.15.46:0/447374453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d5164a0 0x555b2d69d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca45d9b700 1 --2- 172.21.15.46:0/447374453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 0x555b2d69d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca46d9d700 1 -- 172.21.15.46:0/447374453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d5164a0 msgr2=0x555b2d69d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca46d9d700 1 --2- 172.21.15.46:0/447374453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d5164a0 0x555b2d69d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca46d9d700 1 -- 172.21.15.46:0/447374453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 msgr2=0x555b2d69d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca46d9d700 1 --2- 172.21.15.46:0/447374453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 0x555b2d69d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca46d9d700 1 -- 172.21.15.46:0/447374453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b2d431490 con 0x555b2d6936a0 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca45d9b700 1 --2- 172.21.15.46:0/447374453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 0x555b2d69d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca46d9d700 1 --2- 172.21.15.46:0/447374453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d6936a0 0x555b2d69dd00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fca40011740 tx=0x7fca40011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d44d3c5ee4551f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca2dffb700 1 -- 172.21.15.46:0/447374453 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca40012980 con 0x555b2d6936a0 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca2dffb700 1 -- 172.21.15.46:0/447374453 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7fca40012b20 con 0x555b2d6936a0 2024-05-09T04:59:21.152 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.148+0000 7fca5b86db80 1 -- 172.21.15.46:0/447374453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca380a9f20 con 0x555b2d6936a0 2024-05-09T04:59:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.149+0000 7fca2dffb700 1 -- 172.21.15.46:0/447374453 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca40012d20 con 0x555b2d6936a0 2024-05-09T04:59:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.149+0000 7fca5b86db80 10 client.17133 fetch_fsmap learned FSMap version 25 2024-05-09T04:59:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.149+0000 7fca5b86db80 1 -- 172.21.15.46:0/447374453 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555b2d6af040 con 0x555b2d6936a0 2024-05-09T04:59:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.149+0000 7fca2dffb700 1 -- 172.21.15.46:0/447374453 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca380a9f20 con 0x555b2d6936a0 2024-05-09T04:59:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.149+0000 7fca2dffb700 1 -- 172.21.15.46:0/447374453 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fca4001a3f0 con 0x555b2d6936a0 2024-05-09T04:59:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.149+0000 7fca5b86db80 10 client.17133 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:59:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.149+0000 7fca5b86db80 10 client.17133 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:59:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.149+0000 7fca5b86db80 20 client.17133 populate_metadata read hostname 'smithi046' 2024-05-09T04:59:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.149+0000 7fca5b86db80 1 --2- 172.21.15.46:0/447374453 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x555b2d6b1680 0x555b2d6b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.149+0000 7fca5b86db80 4 client.17133 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:59:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.149+0000 7fca5b86db80 1 -- 172.21.15.46:0/447374453 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555b2d5c39d0 con 0x555b2d6b1680 2024-05-09T04:59:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.149+0000 7fca4659c700 1 --2- 172.21.15.46:0/447374453 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x555b2d6b1680 0x555b2d6b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.150+0000 7fca4659c700 1 --2- 172.21.15.46:0/447374453 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x555b2d6b1680 0x555b2d6b3b10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48d3ca776dd4fe58 server_cookie=f1f018f74c31a859 in_seq=0 out_seq=0 2024-05-09T04:59:21.153 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.150+0000 7fca2dffb700 10 client.17133 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:59:21.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.151+0000 7fca2dffb700 1 -- 172.21.15.46:0/447374453 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555b2d5c39d0 con 0x555b2d6b1680 2024-05-09T04:59:21.154 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.151+0000 7fca2dffb700 10 client.17133 handle_command_reply: tid=0 2024-05-09T04:59:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.151+0000 7fca5b86db80 1 client.17133 shutdown 2024-05-09T04:59:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.151+0000 7fca5b86db80 1 -- 172.21.15.46:0/447374453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d6936a0 msgr2=0x555b2d69dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.151+0000 7fca5b86db80 1 --2- 172.21.15.46:0/447374453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d6936a0 0x555b2d69dd00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7fca40011740 tx=0x7fca40011a50 comp rx=0 tx=0).stop 2024-05-09T04:59:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.151+0000 7fca5b86db80 1 -- 172.21.15.46:0/447374453 shutdown_connections 2024-05-09T04:59:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.151+0000 7fca5b86db80 1 --2- 172.21.15.46:0/447374453 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x555b2d6b1680 0x555b2d6b3b10 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.151+0000 7fca5b86db80 1 --2- 172.21.15.46:0/447374453 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d6936a0 0x555b2d69dd00 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.151+0000 7fca5b86db80 1 --2- 172.21.15.46:0/447374453 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d5164a0 0x555b2d69d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.151+0000 7fca5b86db80 1 --2- 172.21.15.46:0/447374453 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 0x555b2d69d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.151+0000 7fca5b86db80 1 -- 172.21.15.46:0/447374453 >> 172.21.15.46:0/447374453 conn(0x555b2d5766a0 msgr2=0x555b2d68e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.151+0000 7fca2dffb700 0 client.17133 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:59:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.151+0000 7fca5b86db80 1 -- 172.21.15.46:0/447374453 shutdown_connections 2024-05-09T04:59:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.151+0000 7fca5b86db80 1 -- 172.21.15.46:0/447374453 wait complete. 2024-05-09T04:59:21.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.151+0000 7fca5b86db80 20 client.17133 trim_cache size 0 max 16384 2024-05-09T04:59:21.167 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.166+0000 7fca5b86db80 1 Processor -- start 2024-05-09T04:59:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca5b86db80 1 -- start start 2024-05-09T04:59:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca5b86db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d5164a0 0x555b2d6b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca5b86db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 0x555b2d69ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca5b86db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 0x555b2d69db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca5b86db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fca481af7d0 con 0x555b2d5164a0 2024-05-09T04:59:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca5b86db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fca481ad060 con 0x555b2d600160 2024-05-09T04:59:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca5b86db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fca481aaed0 con 0x555b2d6936a0 2024-05-09T04:59:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca4659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d5164a0 0x555b2d6b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca4659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d5164a0 0x555b2d6b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36516/0 (socket says 172.21.15.46:36516) 2024-05-09T04:59:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca4659c700 1 -- 172.21.15.46:0/2506671382 learned_addr learned my addr 172.21.15.46:0/2506671382 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca45d9b700 1 --2- 172.21.15.46:0/2506671382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 0x555b2d69ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca46d9d700 1 --2- 172.21.15.46:0/2506671382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 0x555b2d69db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca4659c700 1 -- 172.21.15.46:0/2506671382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 msgr2=0x555b2d69db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca4659c700 1 --2- 172.21.15.46:0/2506671382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 0x555b2d69db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca4659c700 1 -- 172.21.15.46:0/2506671382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 msgr2=0x555b2d69ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca4659c700 1 --2- 172.21.15.46:0/2506671382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 0x555b2d69ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca4659c700 1 -- 172.21.15.46:0/2506671382 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555b2d68fd00 con 0x555b2d5164a0 2024-05-09T04:59:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.167+0000 7fca4659c700 1 --2- 172.21.15.46:0/2506671382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d5164a0 0x555b2d6b3d00 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fca380a97b0 tx=0x7fca380a5050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ace09c18ea22a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.168+0000 7fca2ffff700 1 -- 172.21.15.46:0/2506671382 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca380c6070 con 0x555b2d5164a0 2024-05-09T04:59:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.168+0000 7fca2ffff700 1 -- 172.21.15.46:0/2506671382 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca380a9f20 con 0x555b2d5164a0 2024-05-09T04:59:21.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.168+0000 7fca2ffff700 1 -- 172.21.15.46:0/2506671382 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca380c2030 con 0x555b2d5164a0 2024-05-09T04:59:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.168+0000 7fca5b86db80 1 -- 172.21.15.46:0/2506671382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d5164a0 msgr2=0x555b2d6b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.168+0000 7fca5b86db80 1 --2- 172.21.15.46:0/2506671382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d5164a0 0x555b2d6b3d00 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fca380a97b0 tx=0x7fca380a5050 comp rx=0 tx=0).stop 2024-05-09T04:59:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.168+0000 7fca5b86db80 1 -- 172.21.15.46:0/2506671382 shutdown_connections 2024-05-09T04:59:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.168+0000 7fca5b86db80 1 --2- 172.21.15.46:0/2506671382 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 0x555b2d69db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.168+0000 7fca5b86db80 1 --2- 172.21.15.46:0/2506671382 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d5164a0 0x555b2d6b3d00 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.168+0000 7fca5b86db80 1 --2- 172.21.15.46:0/2506671382 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 0x555b2d69ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.168+0000 7fca5b86db80 1 -- 172.21.15.46:0/2506671382 >> 172.21.15.46:0/2506671382 conn(0x555b2d5766a0 msgr2=0x555b2d69bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.169+0000 7fca5b86db80 1 -- 172.21.15.46:0/2506671382 shutdown_connections 2024-05-09T04:59:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.169+0000 7fca5b86db80 1 -- 172.21.15.46:0/2506671382 wait complete. 2024-05-09T04:59:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.169+0000 7fca5b86db80 1 -- start start 2024-05-09T04:59:21.173 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.169+0000 7fca5b86db80 1 Processor -- start 2024-05-09T04:59:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.169+0000 7fca5b86db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d5164a0 0x555b2d6a9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.169+0000 7fca5b86db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 0x555b2d6a47a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.169+0000 7fca5b86db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 0x555b2d6a4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.169+0000 7fca5b86db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fca481ab520 con 0x555b2d5164a0 2024-05-09T04:59:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.169+0000 7fca5b86db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fca481ac990 con 0x555b2d600160 2024-05-09T04:59:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.169+0000 7fca5b86db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fca481abbf0 con 0x555b2d6936a0 2024-05-09T04:59:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca4659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d5164a0 0x555b2d6a9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca5b86db80 10 client.0 _ll_register_callbacks cb 0x555b2d4276c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:59:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca4659c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d5164a0 0x555b2d6a9c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36524/0 (socket says 172.21.15.46:36524) 2024-05-09T04:59:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca4659c700 1 -- 172.21.15.46:0/1930801225 learned_addr learned my addr 172.21.15.46:0/1930801225 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca45d9b700 1 --2- 172.21.15.46:0/1930801225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 0x555b2d6a47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca46d9d700 1 --2- 172.21.15.46:0/1930801225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 0x555b2d6a4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca4659c700 1 -- 172.21.15.46:0/1930801225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 msgr2=0x555b2d6a4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca4659c700 1 --2- 172.21.15.46:0/1930801225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 0x555b2d6a4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.174 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca4659c700 1 -- 172.21.15.46:0/1930801225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 msgr2=0x555b2d6a47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:21.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca4659c700 1 --2- 172.21.15.46:0/1930801225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 0x555b2d6a47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca4659c700 1 -- 172.21.15.46:0/1930801225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555b2d6a5320 con 0x555b2d5164a0 2024-05-09T04:59:21.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca4659c700 1 --2- 172.21.15.46:0/1930801225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d5164a0 0x555b2d6a9c90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fca38007fd0 tx=0x7fca38009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d74983d2d9bda25 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:21.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca2dffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:59:21.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca2dffb700 1 -- 172.21.15.46:0/1930801225 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca380c2070 con 0x555b2d5164a0 2024-05-09T04:59:21.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca2dffb700 1 -- 172.21.15.46:0/1930801225 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7fca380a9c90 con 0x555b2d5164a0 2024-05-09T04:59:21.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca5b86db80 1 -- 172.21.15.46:0/1930801225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca380a9f20 con 0x555b2d5164a0 2024-05-09T04:59:21.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca2dffb700 1 -- 172.21.15.46:0/1930801225 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fca380c6050 con 0x555b2d5164a0 2024-05-09T04:59:21.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca5b86db80 10 client.17139 fetch_fsmap learned FSMap version 25 2024-05-09T04:59:21.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.170+0000 7fca5b86db80 1 -- 172.21.15.46:0/1930801225 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x555b2d71ac90 con 0x555b2d5164a0 2024-05-09T04:59:21.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.171+0000 7fca2dffb700 1 -- 172.21.15.46:0/1930801225 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca380a9f20 con 0x555b2d5164a0 2024-05-09T04:59:21.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.171+0000 7fca2dffb700 1 -- 172.21.15.46:0/1930801225 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fca380a8a80 con 0x555b2d5164a0 2024-05-09T04:59:21.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.171+0000 7fca5b86db80 10 client.17139 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:59:21.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.171+0000 7fca5b86db80 10 client.17139 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T04:59:21.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.171+0000 7fca5b86db80 20 client.17139 populate_metadata read hostname 'smithi046' 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.171+0000 7fca5b86db80 1 --2- 172.21.15.46:0/1930801225 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x555b2d71af10 0x555b2d71d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.171+0000 7fca5b86db80 4 client.17139 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.171+0000 7fca5b86db80 1 -- 172.21.15.46:0/1930801225 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555b2d5bda70 con 0x555b2d71af10 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.171+0000 7fca45d9b700 1 --2- 172.21.15.46:0/1930801225 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x555b2d71af10 0x555b2d71d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca2dffb700 10 client.17139 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca45d9b700 1 --2- 172.21.15.46:0/1930801225 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x555b2d71af10 0x555b2d71d330 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=28693bda880cc969 server_cookie=bb1c118073149dc3 in_seq=0 out_seq=0 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca2dffb700 1 -- 172.21.15.46:0/1930801225 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555b2d5bda70 con 0x555b2d71af10 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca2dffb700 10 client.17139 handle_command_reply: tid=0 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca5b86db80 1 client.17139 shutdown 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca5b86db80 1 -- 172.21.15.46:0/1930801225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d5164a0 msgr2=0x555b2d6a9c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca5b86db80 1 --2- 172.21.15.46:0/1930801225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d5164a0 0x555b2d6a9c90 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7fca38007fd0 tx=0x7fca38009eb0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca5b86db80 1 -- 172.21.15.46:0/1930801225 shutdown_connections 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca5b86db80 1 --2- 172.21.15.46:0/1930801225 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x555b2d71af10 0x555b2d71d330 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca5b86db80 1 --2- 172.21.15.46:0/1930801225 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x555b2d6936a0 0x555b2d6a4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca5b86db80 1 --2- 172.21.15.46:0/1930801225 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x555b2d5164a0 0x555b2d6a9c90 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca2dffb700 0 client.17139 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T04:59:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca5b86db80 1 --2- 172.21.15.46:0/1930801225 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x555b2d600160 0x555b2d6a47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca5b86db80 1 -- 172.21.15.46:0/1930801225 >> 172.21.15.46:0/1930801225 conn(0x555b2d5766a0 msgr2=0x555b2d698a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca5b86db80 1 -- 172.21.15.46:0/1930801225 shutdown_connections 2024-05-09T04:59:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca5b86db80 1 -- 172.21.15.46:0/1930801225 wait complete. 2024-05-09T04:59:21.177 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.172+0000 7fca5b86db80 20 client.17139 trim_cache size 0 max 16384 2024-05-09T04:59:21.177 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:59:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.173+0000 7fca4ce57700 1 -- 172.21.15.46:0/2104070212 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fca340b2a70 msgr2=0x7fca340b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.173+0000 7fca4ce57700 1 --2- 172.21.15.46:0/2104070212 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fca340b2a70 0x7fca340b4f80 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fca30000c00 tx=0x7fca3001a040 comp rx=0 tx=0).stop 2024-05-09T04:59:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.173+0000 7fca4ce57700 1 -- 172.21.15.46:0/2104070212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca481034e0 msgr2=0x7fca4819bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.173+0000 7fca4ce57700 1 --2- 172.21.15.46:0/2104070212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca481034e0 0x7fca4819bce0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fca3800ef10 tx=0x7fca3800c3a0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.173+0000 7fca4ce57700 1 -- 172.21.15.46:0/2104070212 shutdown_connections 2024-05-09T04:59:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.173+0000 7fca4ce57700 1 --2- 172.21.15.46:0/2104070212 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fca340b2a70 0x7fca340b4f80 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.173+0000 7fca4ce57700 1 --2- 172.21.15.46:0/2104070212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fca48108760 0x7fca481a6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.173+0000 7fca4ce57700 1 --2- 172.21.15.46:0/2104070212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fca481034e0 0x7fca4819bce0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.173+0000 7fca4ce57700 1 --2- 172.21.15.46:0/2104070212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fca48107e10 0x7fca4819c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:21.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.173+0000 7fca4ce57700 1 -- 172.21.15.46:0/2104070212 >> 172.21.15.46:0/2104070212 conn(0x7fca480fef30 msgr2=0x7fca481075b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:21.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.173+0000 7fca4ce57700 1 -- 172.21.15.46:0/2104070212 shutdown_connections 2024-05-09T04:59:21.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:21.173+0000 7fca4ce57700 1 -- 172.21.15.46:0/2104070212 wait complete. 2024-05-09T04:59:21.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:20 smithi110 ceph-mon[34904]: pgmap v254: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T04:59:21.884 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T04:59:22.136 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:23.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:22 smithi046 ceph-mon[30136]: pgmap v255: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:59:23.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:22 smithi072 ceph-mon[34088]: pgmap v255: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:59:23.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:22 smithi110 ceph-mon[34904]: pgmap v255: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:59:23.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.953+0000 7f5d21a06700 1 -- 172.21.15.46:0/360990314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d1c10e900 msgr2=0x7f5d1c110dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:23.953 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.953+0000 7f5d21a06700 1 --2- 172.21.15.46:0/360990314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d1c10e900 0x7f5d1c110dc0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5d14009990 tx=0x7f5d14009ca0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.953+0000 7f5d21a06700 1 -- 172.21.15.46:0/360990314 shutdown_connections 2024-05-09T04:59:23.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.953+0000 7f5d21a06700 1 --2- 172.21.15.46:0/360990314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d1c10e900 0x7f5d1c110dc0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.953+0000 7f5d21a06700 1 --2- 172.21.15.46:0/360990314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d1c105520 0x7f5d1c105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.953+0000 7f5d21a06700 1 --2- 172.21.15.46:0/360990314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d1c105e70 0x7f5d1c10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.953+0000 7f5d21a06700 1 -- 172.21.15.46:0/360990314 >> 172.21.15.46:0/360990314 conn(0x7f5d1c0fedc0 msgr2=0x7f5d1c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:23.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.953+0000 7f5d21a06700 1 -- 172.21.15.46:0/360990314 shutdown_connections 2024-05-09T04:59:23.954 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.953+0000 7f5d21a06700 1 -- 172.21.15.46:0/360990314 wait complete. 2024-05-09T04:59:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.953+0000 7f5d21a06700 1 Processor -- start 2024-05-09T04:59:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.954+0000 7f5d21a06700 1 -- start start 2024-05-09T04:59:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.954+0000 7f5d21a06700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d1c105520 0x7f5d1c19ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.954+0000 7f5d21a06700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d1c105e70 0x7f5d1c19b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.954+0000 7f5d21a06700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d1c10e900 0x7f5d1c19f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.954+0000 7f5d21a06700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c114440 con 0x7f5d1c105e70 2024-05-09T04:59:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.954+0000 7f5d21a06700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1142c0 con 0x7f5d1c105520 2024-05-09T04:59:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.954+0000 7f5d21a06700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1145c0 con 0x7f5d1c10e900 2024-05-09T04:59:23.956 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.954+0000 7f5d1a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d1c105e70 0x7f5d1c19b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.954+0000 7f5d1a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d1c105e70 0x7f5d1c19b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50010/0 (socket says 172.21.15.46:50010) 2024-05-09T04:59:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.954+0000 7f5d1a7fc700 1 -- 172.21.15.46:0/4215946333 learned_addr learned my addr 172.21.15.46:0/4215946333 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.954+0000 7f5d1affd700 1 --2- 172.21.15.46:0/4215946333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d1c105520 0x7f5d1c19ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.954+0000 7f5d1b7fe700 1 --2- 172.21.15.46:0/4215946333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d1c10e900 0x7f5d1c19f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.955+0000 7f5d1a7fc700 1 -- 172.21.15.46:0/4215946333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d1c10e900 msgr2=0x7f5d1c19f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.955+0000 7f5d1a7fc700 1 --2- 172.21.15.46:0/4215946333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d1c10e900 0x7f5d1c19f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.955+0000 7f5d1a7fc700 1 -- 172.21.15.46:0/4215946333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d1c105520 msgr2=0x7f5d1c19ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.955+0000 7f5d1a7fc700 1 --2- 172.21.15.46:0/4215946333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d1c105520 0x7f5d1c19ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.955+0000 7f5d1a7fc700 1 -- 172.21.15.46:0/4215946333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1c19fc40 con 0x7f5d1c105e70 2024-05-09T04:59:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.955+0000 7f5d1a7fc700 1 --2- 172.21.15.46:0/4215946333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d1c105e70 0x7f5d1c19b310 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000d950 tx=0x7f5d1000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.955+0000 7f5d20a04700 1 -- 172.21.15.46:0/4215946333 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10016070 con 0x7f5d1c105e70 2024-05-09T04:59:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.955+0000 7f5d21a06700 1 -- 172.21.15.46:0/4215946333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d1c19ff30 con 0x7f5d1c105e70 2024-05-09T04:59:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.955+0000 7f5d20a04700 1 -- 172.21.15.46:0/4215946333 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d1000fd40 con 0x7f5d1c105e70 2024-05-09T04:59:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.955+0000 7f5d20a04700 1 -- 172.21.15.46:0/4215946333 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1000e3b0 con 0x7f5d1c105e70 2024-05-09T04:59:23.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.955+0000 7f5d21a06700 1 -- 172.21.15.46:0/4215946333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d1c1a7850 con 0x7f5d1c105e70 2024-05-09T04:59:23.958 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.958+0000 7f5d20a04700 1 -- 172.21.15.46:0/4215946333 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5d10019070 con 0x7f5d1c105e70 2024-05-09T04:59:23.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.959+0000 7f5d20a04700 1 --2- 172.21.15.46:0/4215946333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5d040b29f0 0x7f5d040b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.959+0000 7f5d1affd700 1 --2- 172.21.15.46:0/4215946333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5d040b29f0 0x7f5d040b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.959+0000 7f5d20a04700 1 -- 172.21.15.46:0/4215946333 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f5d100e2160 con 0x7f5d1c105e70 2024-05-09T04:59:23.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.959+0000 7f5d3041cb80 1 Processor -- start 2024-05-09T04:59:23.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.959+0000 7f5d3041cb80 1 -- start start 2024-05-09T04:59:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d3041cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d3041cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c98463a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d3041cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c984634810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d3041cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c9845570a0 con 0x55c984531d00 2024-05-09T04:59:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d3041cb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c9844fc9a0 con 0x55c984634400 2024-05-09T04:59:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d3041cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c984553060 con 0x55c9844b74a0 2024-05-09T04:59:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d1affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.961 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d1affd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50016/0 (socket says 172.21.15.46:50016) 2024-05-09T04:59:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d1affd700 1 -- 172.21.15.46:0/3874345227 learned_addr learned my addr 172.21.15.46:0/3874345227 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d1affd700 1 --2- 172.21.15.46:0/4215946333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5d040b29f0 0x7f5d040b4f00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009550 tx=0x7f5d0c009cd0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d1b7fe700 1 --2- 172.21.15.46:0/3874345227 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c984634810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d1a7fc700 1 --2- 172.21.15.46:0/3874345227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c98463a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d1affd700 1 -- 172.21.15.46:0/3874345227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 msgr2=0x55c98463a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d1affd700 1 --2- 172.21.15.46:0/3874345227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c98463a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d1affd700 1 -- 172.21.15.46:0/3874345227 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 msgr2=0x55c984634810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d1affd700 1 --2- 172.21.15.46:0/3874345227 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c984634810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d1affd700 1 -- 172.21.15.46:0/3874345227 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c9844ed500 con 0x55c984531d00 2024-05-09T04:59:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d1a7fc700 1 --2- 172.21.15.46:0/3874345227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c98463a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:59:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.960+0000 7f5d1b7fe700 1 --2- 172.21.15.46:0/3874345227 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c984634810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:59:23.962 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.961+0000 7f5d1affd700 1 --2- 172.21.15.46:0/3874345227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463a240 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0047d0 tx=0x7f5d0c01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c16dae2289d3545f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.961+0000 7f5d01ffb700 1 -- 172.21.15.46:0/3874345227 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c026070 con 0x55c984531d00 2024-05-09T04:59:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.961+0000 7f5d01ffb700 1 -- 172.21.15.46:0/3874345227 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d0c0048c0 con 0x55c984531d00 2024-05-09T04:59:23.963 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.961+0000 7f5d01ffb700 1 -- 172.21.15.46:0/3874345227 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c01d040 con 0x55c984531d00 2024-05-09T04:59:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.962+0000 7f5d3041cb80 1 -- 172.21.15.46:0/3874345227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 msgr2=0x55c98463a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.962+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/3874345227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463a240 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c0047d0 tx=0x7f5d0c01b000 comp rx=0 tx=0).stop 2024-05-09T04:59:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.962+0000 7f5d3041cb80 1 -- 172.21.15.46:0/3874345227 shutdown_connections 2024-05-09T04:59:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.962+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/3874345227 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c984634810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.962+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/3874345227 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463a240 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.962+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/3874345227 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c98463a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.962+0000 7f5d3041cb80 1 -- 172.21.15.46:0/3874345227 >> 172.21.15.46:0/3874345227 conn(0x55c9844baf70 msgr2=0x55c98462f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.962+0000 7f5d3041cb80 1 -- 172.21.15.46:0/3874345227 shutdown_connections 2024-05-09T04:59:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.962+0000 7f5d3041cb80 1 -- 172.21.15.46:0/3874345227 wait complete. 2024-05-09T04:59:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.962+0000 7f5d3041cb80 1 -- start start 2024-05-09T04:59:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.963+0000 7f5d3041cb80 1 Processor -- start 2024-05-09T04:59:23.966 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.963+0000 7f5d3041cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c98463dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.963+0000 7f5d3041cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.963+0000 7f5d3041cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c98463ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.963+0000 7f5d3041cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c984635160 con 0x55c984531d00 2024-05-09T04:59:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.963+0000 7f5d3041cb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c9844f1b20 con 0x55c984634400 2024-05-09T04:59:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.963+0000 7f5d3041cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c984641020 con 0x55c9844b74a0 2024-05-09T04:59:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.963+0000 7f5d3041cb80 10 client.0 _ll_register_callbacks cb 0x55c9843c86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:59:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.963+0000 7f5d1a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.963+0000 7f5d1affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c98463dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.963+0000 7f5d1b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c98463ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.963+0000 7f5d1affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c98463dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:39866/0 (socket says 172.21.15.46:39866) 2024-05-09T04:59:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.963+0000 7f5d1a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50026/0 (socket says 172.21.15.46:50026) 2024-05-09T04:59:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.963+0000 7f5d1affd700 1 -- 172.21.15.46:0/494921119 learned_addr learned my addr 172.21.15.46:0/494921119 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.964+0000 7f5d1affd700 1 -- 172.21.15.46:0/494921119 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 msgr2=0x55c98463ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.964+0000 7f5d1affd700 1 --2- 172.21.15.46:0/494921119 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c98463ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.964+0000 7f5d1affd700 1 -- 172.21.15.46:0/494921119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 msgr2=0x55c98463e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.964+0000 7f5d1affd700 1 --2- 172.21.15.46:0/494921119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.964+0000 7f5d1affd700 1 -- 172.21.15.46:0/494921119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c9843d2490 con 0x55c9844b74a0 2024-05-09T04:59:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.964+0000 7f5d1a7fc700 1 --2- 172.21.15.46:0/494921119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:59:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.964+0000 7f5d1affd700 1 --2- 172.21.15.46:0/494921119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c98463dff0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c003cb0 tx=0x7f5d0c004ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95450bb17af39a77 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.964+0000 7f5cfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T04:59:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.964+0000 7f5cfaffd700 1 -- 172.21.15.46:0/494921119 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d0c01d030 con 0x55c9844b74a0 2024-05-09T04:59:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.964+0000 7f5d3041cb80 1 -- 172.21.15.46:0/494921119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d0c0048c0 con 0x55c9844b74a0 2024-05-09T04:59:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.964+0000 7f5cfaffd700 1 -- 172.21.15.46:0/494921119 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f5d0c021e00 con 0x55c9844b74a0 2024-05-09T04:59:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.964+0000 7f5cfaffd700 1 -- 172.21.15.46:0/494921119 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d0c026070 con 0x55c9844b74a0 2024-05-09T04:59:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.965+0000 7f5d3041cb80 10 client.26545 fetch_fsmap learned FSMap version 25 2024-05-09T04:59:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.965+0000 7f5d3041cb80 1 -- 172.21.15.46:0/494921119 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c98464b930 con 0x55c9844b74a0 2024-05-09T04:59:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.965+0000 7f5cfaffd700 1 -- 172.21.15.46:0/494921119 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d0c0048c0 con 0x55c9844b74a0 2024-05-09T04:59:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.965+0000 7f5cfaffd700 1 -- 172.21.15.46:0/494921119 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5d0c020070 con 0x55c9844b74a0 2024-05-09T04:59:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.965+0000 7f5d3041cb80 10 client.26545 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:59:23.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.965+0000 7f5d3041cb80 10 client.26545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:59:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.965+0000 7f5d3041cb80 20 client.26545 populate_metadata read hostname 'smithi046' 2024-05-09T04:59:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.965+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/494921119 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c98464de60 0x55c984650280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.965+0000 7f5d3041cb80 4 client.26545 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:59:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.965+0000 7f5d3041cb80 1 -- 172.21.15.46:0/494921119 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c984564a80 con 0x55c98464de60 2024-05-09T04:59:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.965+0000 7f5d1a7fc700 1 --2- 172.21.15.46:0/494921119 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c98464de60 0x55c984650280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.966+0000 7f5d1a7fc700 1 --2- 172.21.15.46:0/494921119 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c98464de60 0x55c984650280 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c657914924abb60a server_cookie=4fd96dfa5822526a in_seq=0 out_seq=0 2024-05-09T04:59:23.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.966+0000 7f5cfaffd700 10 client.26545 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.967+0000 7f5cfaffd700 1 -- 172.21.15.46:0/494921119 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c984564a80 con 0x55c98464de60 2024-05-09T04:59:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.967+0000 7f5cfaffd700 10 client.26545 handle_command_reply: tid=0 2024-05-09T04:59:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.967+0000 7f5d3041cb80 1 client.26545 shutdown 2024-05-09T04:59:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.967+0000 7f5d3041cb80 1 -- 172.21.15.46:0/494921119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 msgr2=0x55c98463dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.967+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/494921119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c98463dff0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c003cb0 tx=0x7f5d0c004ca0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.967+0000 7f5d3041cb80 1 -- 172.21.15.46:0/494921119 shutdown_connections 2024-05-09T04:59:23.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.967+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/494921119 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c98464de60 0x55c984650280 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.967+0000 7f5cfaffd700 0 client.26545 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:23.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.967+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/494921119 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c98463ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.967+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/494921119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c98463dff0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.967+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/494921119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.967+0000 7f5d3041cb80 1 -- 172.21.15.46:0/494921119 >> 172.21.15.46:0/494921119 conn(0x55c9844baf70 msgr2=0x55c984628ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:23.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.967+0000 7f5d3041cb80 1 -- 172.21.15.46:0/494921119 shutdown_connections 2024-05-09T04:59:23.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.967+0000 7f5d3041cb80 1 -- 172.21.15.46:0/494921119 wait complete. 2024-05-09T04:59:23.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.967+0000 7f5d3041cb80 20 client.26545 trim_cache size 0 max 16384 2024-05-09T04:59:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.983+0000 7f5d3041cb80 1 Processor -- start 2024-05-09T04:59:23.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.983+0000 7f5d3041cb80 1 -- start start 2024-05-09T04:59:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.983+0000 7f5d3041cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c984629a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.983+0000 7f5d3041cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.983+0000 7f5d3041cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c98464a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.983+0000 7f5d3041cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1b18c0 con 0x55c984531d00 2024-05-09T04:59:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.983+0000 7f5d3041cb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1af150 con 0x55c984634400 2024-05-09T04:59:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.983+0000 7f5d3041cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1acfc0 con 0x55c9844b74a0 2024-05-09T04:59:23.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.984+0000 7f5d1a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.984+0000 7f5d1a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50040/0 (socket says 172.21.15.46:50040) 2024-05-09T04:59:23.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.984+0000 7f5d1a7fc700 1 -- 172.21.15.46:0/2253761437 learned_addr learned my addr 172.21.15.46:0/2253761437 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:23.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.984+0000 7f5d1affd700 1 --2- 172.21.15.46:0/2253761437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c984629a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.984+0000 7f5d1b7fe700 1 --2- 172.21.15.46:0/2253761437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c98464a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.984+0000 7f5d1a7fc700 1 -- 172.21.15.46:0/2253761437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 msgr2=0x55c984629a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:23.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.984+0000 7f5d1a7fc700 1 --2- 172.21.15.46:0/2253761437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c984629a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.984+0000 7f5d1a7fc700 1 -- 172.21.15.46:0/2253761437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 msgr2=0x55c98464a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:23.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.984+0000 7f5d1a7fc700 1 --2- 172.21.15.46:0/2253761437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c98464a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.984+0000 7f5d1a7fc700 1 -- 172.21.15.46:0/2253761437 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c98463c560 con 0x55c984531d00 2024-05-09T04:59:23.985 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.984+0000 7f5d1a7fc700 1 --2- 172.21.15.46:0/2253761437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463cf50 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f5d100a5410 tx=0x7f5d10010a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2167b7d63b0e4b6c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:23.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.984+0000 7f5d017fa700 1 -- 172.21.15.46:0/2253761437 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d10020040 con 0x55c984531d00 2024-05-09T04:59:23.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.984+0000 7f5d017fa700 1 -- 172.21.15.46:0/2253761437 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d100ad740 con 0x55c984531d00 2024-05-09T04:59:23.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.984+0000 7f5d017fa700 1 -- 172.21.15.46:0/2253761437 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d100a97f0 con 0x55c984531d00 2024-05-09T04:59:23.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.985+0000 7f5d3041cb80 1 -- 172.21.15.46:0/2253761437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 msgr2=0x55c98463cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:23.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.985+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/2253761437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463cf50 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f5d100a5410 tx=0x7f5d10010a40 comp rx=0 tx=0).stop 2024-05-09T04:59:23.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.985+0000 7f5d3041cb80 1 -- 172.21.15.46:0/2253761437 shutdown_connections 2024-05-09T04:59:23.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.985+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/2253761437 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c98464a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.987 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.985+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/2253761437 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c984629a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.985+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/2253761437 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c98463cf50 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.985+0000 7f5d3041cb80 1 -- 172.21.15.46:0/2253761437 >> 172.21.15.46:0/2253761437 conn(0x55c9844baf70 msgr2=0x55c984637cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.986+0000 7f5d3041cb80 1 -- 172.21.15.46:0/2253761437 shutdown_connections 2024-05-09T04:59:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.986+0000 7f5d3041cb80 1 -- 172.21.15.46:0/2253761437 wait complete. 2024-05-09T04:59:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.986+0000 7f5d3041cb80 1 -- start start 2024-05-09T04:59:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.986+0000 7f5d3041cb80 1 Processor -- start 2024-05-09T04:59:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d3041cb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c9846af700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d3041cb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c9846afc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d3041cb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c9846506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d3041cb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1ad610 con 0x55c984531d00 2024-05-09T04:59:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d3041cb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1aea80 con 0x55c984634400 2024-05-09T04:59:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d3041cb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5d1c1adce0 con 0x55c9844b74a0 2024-05-09T04:59:23.988 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d3041cb80 10 client.0 _ll_register_callbacks cb 0x55c9843c86c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:59:23.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d1affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c9846af700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d1affd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c9846af700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:39886/0 (socket says 172.21.15.46:39886) 2024-05-09T04:59:23.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d1a7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c9846afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d1affd700 1 -- 172.21.15.46:0/2574989370 learned_addr learned my addr 172.21.15.46:0/2574989370 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:23.989 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d1b7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c9846506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d1a7fc700 1 -- 172.21.15.46:0/2574989370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 msgr2=0x55c9846af700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d1a7fc700 1 --2- 172.21.15.46:0/2574989370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c9846af700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d1a7fc700 1 -- 172.21.15.46:0/2574989370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 msgr2=0x55c9846506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d1a7fc700 1 --2- 172.21.15.46:0/2574989370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c9846506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d1a7fc700 1 -- 172.21.15.46:0/2574989370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c984650dc0 con 0x55c984531d00 2024-05-09T04:59:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5d1a7fc700 1 --2- 172.21.15.46:0/2574989370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c9846afc40 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000ffa0 tx=0x7f5d10000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5f6bfc7bd9ca12 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.987+0000 7f5cfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:59:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.988+0000 7f5cfaffd700 1 -- 172.21.15.46:0/2574989370 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1006d020 con 0x55c984531d00 2024-05-09T04:59:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.988+0000 7f5cfaffd700 1 -- 172.21.15.46:0/2574989370 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f5d100a95f0 con 0x55c984531d00 2024-05-09T04:59:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.988+0000 7f5d3041cb80 1 -- 172.21.15.46:0/2574989370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5d100ad740 con 0x55c984531d00 2024-05-09T04:59:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.988+0000 7f5cfaffd700 1 -- 172.21.15.46:0/2574989370 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5d10020030 con 0x55c984531d00 2024-05-09T04:59:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.988+0000 7f5d3041cb80 10 client.17157 fetch_fsmap learned FSMap version 25 2024-05-09T04:59:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.988+0000 7f5d3041cb80 1 -- 172.21.15.46:0/2574989370 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c9846b7da0 con 0x55c984531d00 2024-05-09T04:59:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.988+0000 7f5cfaffd700 1 -- 172.21.15.46:0/2574989370 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d100ad740 con 0x55c984531d00 2024-05-09T04:59:23.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.988+0000 7f5cfaffd700 1 -- 172.21.15.46:0/2574989370 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5d100b1df0 con 0x55c984531d00 2024-05-09T04:59:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.988+0000 7f5d3041cb80 10 client.17157 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:59:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.988+0000 7f5d3041cb80 10 client.17157 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:59:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.988+0000 7f5d3041cb80 20 client.17157 populate_metadata read hostname 'smithi046' 2024-05-09T04:59:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.988+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/2574989370 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c98464de60 0x55c9846ba460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.988+0000 7f5d3041cb80 4 client.17157 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T04:59:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.988+0000 7f5d3041cb80 1 -- 172.21.15.46:0/2574989370 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c98455eb20 con 0x55c98464de60 2024-05-09T04:59:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.989+0000 7f5d1affd700 1 --2- 172.21.15.46:0/2574989370 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c98464de60 0x55c9846ba460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.989+0000 7f5cfaffd700 10 client.17157 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.989+0000 7f5d1affd700 1 --2- 172.21.15.46:0/2574989370 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c98464de60 0x55c9846ba460 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffbf33bd44c2c5bf server_cookie=d109082e8bcf8fd9 in_seq=0 out_seq=0 2024-05-09T04:59:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.989+0000 7f5cfaffd700 1 -- 172.21.15.46:0/2574989370 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c98455eb20 con 0x55c98464de60 2024-05-09T04:59:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.989+0000 7f5cfaffd700 10 client.17157 handle_command_reply: tid=0 2024-05-09T04:59:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.989+0000 7f5d3041cb80 1 client.17157 shutdown 2024-05-09T04:59:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.990+0000 7f5d3041cb80 1 -- 172.21.15.46:0/2574989370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 msgr2=0x55c9846afc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:23.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.990+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/2574989370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c9846afc40 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000ffa0 tx=0x7f5d10000bc0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.990+0000 7f5d3041cb80 1 -- 172.21.15.46:0/2574989370 shutdown_connections 2024-05-09T04:59:23.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.990+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/2574989370 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55c98464de60 0x55c9846ba460 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.990+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/2574989370 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c984634400 0x55c9846506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.990+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/2574989370 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c9844b74a0 0x55c9846af700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.990+0000 7f5d3041cb80 1 --2- 172.21.15.46:0/2574989370 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c984531d00 0x55c9846afc40 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.990+0000 7f5d3041cb80 1 -- 172.21.15.46:0/2574989370 >> 172.21.15.46:0/2574989370 conn(0x55c9844baf70 msgr2=0x55c984629780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:23.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.990+0000 7f5cfaffd700 0 client.17157 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:23.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.990+0000 7f5d3041cb80 1 -- 172.21.15.46:0/2574989370 shutdown_connections 2024-05-09T04:59:23.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.990+0000 7f5d3041cb80 1 -- 172.21.15.46:0/2574989370 wait complete. 2024-05-09T04:59:23.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.990+0000 7f5d3041cb80 20 client.17157 trim_cache size 0 max 16384 2024-05-09T04:59:23.994 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T04:59:23.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.991+0000 7f5d21a06700 1 -- 172.21.15.46:0/4215946333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5d040b29f0 msgr2=0x7f5d040b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.991+0000 7f5d21a06700 1 --2- 172.21.15.46:0/4215946333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5d040b29f0 0x7f5d040b4f00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5d0c009550 tx=0x7f5d0c009cd0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.991+0000 7f5d21a06700 1 -- 172.21.15.46:0/4215946333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d1c105e70 msgr2=0x7f5d1c19b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.991+0000 7f5d21a06700 1 --2- 172.21.15.46:0/4215946333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d1c105e70 0x7f5d1c19b310 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5d1000d950 tx=0x7f5d1000dc60 comp rx=0 tx=0).stop 2024-05-09T04:59:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.991+0000 7f5d21a06700 1 -- 172.21.15.46:0/4215946333 shutdown_connections 2024-05-09T04:59:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.991+0000 7f5d21a06700 1 --2- 172.21.15.46:0/4215946333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5d040b29f0 0x7f5d040b4f00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.991+0000 7f5d21a06700 1 --2- 172.21.15.46:0/4215946333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5d1c10e900 0x7f5d1c19f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.991+0000 7f5d21a06700 1 --2- 172.21.15.46:0/4215946333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5d1c105520 0x7f5d1c19ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.991+0000 7f5d21a06700 1 --2- 172.21.15.46:0/4215946333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5d1c105e70 0x7f5d1c19b310 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.991+0000 7f5d21a06700 1 -- 172.21.15.46:0/4215946333 >> 172.21.15.46:0/4215946333 conn(0x7f5d1c0fedc0 msgr2=0x7f5d1c1084a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.991+0000 7f5d21a06700 1 -- 172.21.15.46:0/4215946333 shutdown_connections 2024-05-09T04:59:23.996 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:23.991+0000 7f5d21a06700 1 -- 172.21.15.46:0/4215946333 wait complete. 2024-05-09T04:59:25.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:24 smithi046 ceph-mon[30136]: pgmap v256: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:59:25.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:24 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:59:25.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:24 smithi072 ceph-mon[34088]: pgmap v256: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:59:25.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:24 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:59:25.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:24 smithi110 ceph-mon[34904]: pgmap v256: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:59:25.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:24 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T04:59:25.844 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T04:59:25.845 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T04:59:26.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:25 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:59:26.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:25 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:59:26.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:25 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:59:26.106 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:26.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:25 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:59:26.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:25 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:59:26.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:25 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:59:26.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:25 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T04:59:26.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:25 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T04:59:26.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:25 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T04:59:27.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:26 smithi046 ceph-mon[30136]: pgmap v257: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:59:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:26 smithi072 ceph-mon[34088]: pgmap v257: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:59:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:26 smithi110 ceph-mon[34904]: pgmap v257: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T04:59:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.966+0000 7ff7f7c40700 1 -- 172.21.15.46:0/202566148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f0109850 msgr2=0x7ff7f01111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.966+0000 7ff7f7c40700 1 --2- 172.21.15.46:0/202566148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f0109850 0x7ff7f01111b0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7ff7e80099c0 tx=0x7ff7e8009cd0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.966+0000 7ff7f7c40700 1 -- 172.21.15.46:0/202566148 shutdown_connections 2024-05-09T04:59:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.966+0000 7ff7f7c40700 1 --2- 172.21.15.46:0/202566148 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f0109850 0x7ff7f01111b0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.967 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.966+0000 7ff7f7c40700 1 --2- 172.21.15.46:0/202566148 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7f01045d0 0x7ff7f01049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.966+0000 7ff7f7c40700 1 --2- 172.21.15.46:0/202566148 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7f0108f00 0x7ff7f0109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.966+0000 7ff7f7c40700 1 -- 172.21.15.46:0/202566148 >> 172.21.15.46:0/202566148 conn(0x7ff7f00ffea0 msgr2=0x7ff7f01022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:27.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.967+0000 7ff7f7c40700 1 -- 172.21.15.46:0/202566148 shutdown_connections 2024-05-09T04:59:27.968 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.967+0000 7ff7f7c40700 1 -- 172.21.15.46:0/202566148 wait complete. 2024-05-09T04:59:27.969 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.967+0000 7ff7f7c40700 1 Processor -- start 2024-05-09T04:59:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f7c40700 1 -- start start 2024-05-09T04:59:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f7c40700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7f01045d0 0x7ff7f019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f7c40700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f0108f00 0x7ff7f019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f7c40700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7f0109850 0x7ff7f019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f7c40700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f01144d0 con 0x7ff7f0108f00 2024-05-09T04:59:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f7c40700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f0114350 con 0x7ff7f0109850 2024-05-09T04:59:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f7c40700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f0114650 con 0x7ff7f01045d0 2024-05-09T04:59:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f51db700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f0108f00 0x7ff7f019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f51db700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f0108f00 0x7ff7f019b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50064/0 (socket says 172.21.15.46:50064) 2024-05-09T04:59:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f51db700 1 -- 172.21.15.46:0/410742662 learned_addr learned my addr 172.21.15.46:0/410742662 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f59dc700 1 --2- 172.21.15.46:0/410742662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7f01045d0 0x7ff7f019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f61dd700 1 --2- 172.21.15.46:0/410742662 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7f0109850 0x7ff7f019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:27.970 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f51db700 1 -- 172.21.15.46:0/410742662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7f01045d0 msgr2=0x7ff7f019ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f51db700 1 --2- 172.21.15.46:0/410742662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7f01045d0 0x7ff7f019ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f51db700 1 -- 172.21.15.46:0/410742662 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7f0109850 msgr2=0x7ff7f019f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f51db700 1 --2- 172.21.15.46:0/410742662 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7f0109850 0x7ff7f019f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f51db700 1 -- 172.21.15.46:0/410742662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7f019fcb0 con 0x7ff7f0108f00 2024-05-09T04:59:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.968+0000 7ff7f61dd700 1 --2- 172.21.15.46:0/410742662 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7f0109850 0x7ff7f019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:59:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.969+0000 7ff7f51db700 1 --2- 172.21.15.46:0/410742662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f0108f00 0x7ff7f019b380 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400e970 tx=0x7ff7e400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.969+0000 7ff7eeffd700 1 -- 172.21.15.46:0/410742662 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e400cbd0 con 0x7ff7f0108f00 2024-05-09T04:59:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.969+0000 7ff7f7c40700 1 -- 172.21.15.46:0/410742662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7f019ffa0 con 0x7ff7f0108f00 2024-05-09T04:59:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.969+0000 7ff7eeffd700 1 -- 172.21.15.46:0/410742662 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e400cd70 con 0x7ff7f0108f00 2024-05-09T04:59:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.969+0000 7ff7eeffd700 1 -- 172.21.15.46:0/410742662 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e4005680 con 0x7ff7f0108f00 2024-05-09T04:59:27.971 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.969+0000 7ff7f7c40700 1 -- 172.21.15.46:0/410742662 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7f01a78c0 con 0x7ff7f0108f00 2024-05-09T04:59:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.971+0000 7ff7eeffd700 1 -- 172.21.15.46:0/410742662 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff7e4004750 con 0x7ff7f0108f00 2024-05-09T04:59:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.972+0000 7ff7eeffd700 1 --2- 172.21.15.46:0/410742662 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff7d80b29f0 0x7ff7d80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.972+0000 7ff7f59dc700 1 --2- 172.21.15.46:0/410742662 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff7d80b29f0 0x7ff7d80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:27.972 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.972+0000 7ff7eeffd700 1 -- 172.21.15.46:0/410742662 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7ff7e40e29f0 con 0x7ff7f0108f00 2024-05-09T04:59:27.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.973+0000 7ff806656b80 1 Processor -- start 2024-05-09T04:59:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.973+0000 7ff7f59dc700 1 --2- 172.21.15.46:0/410742662 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff7d80b29f0 0x7ff7d80b4f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000f540 tx=0x7ff7e000b040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.973+0000 7ff806656b80 1 -- start start 2024-05-09T04:59:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.973+0000 7ff806656b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa4c91e0 0x55ecaa562570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.973+0000 7ff806656b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 0x55ecaa562ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.973+0000 7ff806656b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa55c720 0x55ecaa55cb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.973+0000 7ff806656b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ecaa47f070 con 0x55ecaa55c720 2024-05-09T04:59:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.973+0000 7ff806656b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ecaa41d870 con 0x55ecaa3df520 2024-05-09T04:59:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.973+0000 7ff806656b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ecaa47b030 con 0x55ecaa4c91e0 2024-05-09T04:59:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.973+0000 7ff7f61dd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa55c720 0x55ecaa55cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.973+0000 7ff7f59dc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa4c91e0 0x55ecaa562570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:27.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.973+0000 7ff7f61dd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa55c720 0x55ecaa55cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50078/0 (socket says 172.21.15.46:50078) 2024-05-09T04:59:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.973+0000 7ff7f61dd700 1 -- 172.21.15.46:0/341810545 learned_addr learned my addr 172.21.15.46:0/341810545 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.973+0000 7ff7f51db700 1 --2- 172.21.15.46:0/341810545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 0x55ecaa562ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.974+0000 7ff7f59dc700 1 -- 172.21.15.46:0/341810545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 msgr2=0x55ecaa562ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.974+0000 7ff7f59dc700 1 --2- 172.21.15.46:0/341810545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 0x55ecaa562ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.974+0000 7ff7f59dc700 1 -- 172.21.15.46:0/341810545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa55c720 msgr2=0x55ecaa55cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.974+0000 7ff7f59dc700 1 --2- 172.21.15.46:0/341810545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa55c720 0x55ecaa55cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.974+0000 7ff7f59dc700 1 -- 172.21.15.46:0/341810545 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecaa4153f0 con 0x55ecaa4c91e0 2024-05-09T04:59:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.974+0000 7ff7f61dd700 1 --2- 172.21.15.46:0/341810545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa55c720 0x55ecaa55cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:59:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.974+0000 7ff7f59dc700 1 --2- 172.21.15.46:0/341810545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa4c91e0 0x55ecaa562570 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000fc90 tx=0x7ff7e0014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=972b2ffccc0cbcbd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.974+0000 7ff7d7fff700 1 -- 172.21.15.46:0/341810545 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0027070 con 0x55ecaa4c91e0 2024-05-09T04:59:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.974+0000 7ff7d7fff700 1 -- 172.21.15.46:0/341810545 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e001e040 con 0x55ecaa4c91e0 2024-05-09T04:59:27.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.974+0000 7ff7d7fff700 1 -- 172.21.15.46:0/341810545 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e001d050 con 0x55ecaa4c91e0 2024-05-09T04:59:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.975+0000 7ff806656b80 1 -- 172.21.15.46:0/341810545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa4c91e0 msgr2=0x55ecaa562570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.975+0000 7ff806656b80 1 --2- 172.21.15.46:0/341810545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa4c91e0 0x55ecaa562570 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000fc90 tx=0x7ff7e0014000 comp rx=0 tx=0).stop 2024-05-09T04:59:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.975+0000 7ff806656b80 1 -- 172.21.15.46:0/341810545 shutdown_connections 2024-05-09T04:59:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.975+0000 7ff806656b80 1 --2- 172.21.15.46:0/341810545 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa55c720 0x55ecaa55cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.975+0000 7ff806656b80 1 --2- 172.21.15.46:0/341810545 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa4c91e0 0x55ecaa562570 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.975+0000 7ff806656b80 1 --2- 172.21.15.46:0/341810545 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 0x55ecaa562ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.975+0000 7ff806656b80 1 -- 172.21.15.46:0/341810545 >> 172.21.15.46:0/341810545 conn(0x55ecaa43f720 msgr2=0x55ecaa557480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.975+0000 7ff806656b80 1 -- 172.21.15.46:0/341810545 shutdown_connections 2024-05-09T04:59:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.975+0000 7ff806656b80 1 -- 172.21.15.46:0/341810545 wait complete. 2024-05-09T04:59:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.976+0000 7ff806656b80 1 -- start start 2024-05-09T04:59:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.976+0000 7ff806656b80 1 Processor -- start 2024-05-09T04:59:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.976+0000 7ff806656b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa3df520 0x55ecaa5662e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.976+0000 7ff806656b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa4c91e0 0x55ecaa566820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.976+0000 7ff806656b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa55c720 0x55ecaa566d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.976+0000 7ff806656b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ecaa2fac20 con 0x55ecaa3df520 2024-05-09T04:59:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.976+0000 7ff806656b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ecaa4391d0 con 0x55ecaa4c91e0 2024-05-09T04:59:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.976+0000 7ff806656b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ecaa56d660 con 0x55ecaa55c720 2024-05-09T04:59:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.976+0000 7ff806656b80 10 client.0 _ll_register_callbacks cb 0x55ecaa2f0740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:59:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.976+0000 7ff7f59dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa3df520 0x55ecaa5662e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:27.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.976+0000 7ff7f59dc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa3df520 0x55ecaa5662e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50090/0 (socket says 172.21.15.46:50090) 2024-05-09T04:59:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.976+0000 7ff7f59dc700 1 -- 172.21.15.46:0/3067318943 learned_addr learned my addr 172.21.15.46:0/3067318943 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.977+0000 7ff7f61dd700 1 --2- 172.21.15.46:0/3067318943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa55c720 0x55ecaa566d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.977+0000 7ff7f51db700 1 --2- 172.21.15.46:0/3067318943 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa4c91e0 0x55ecaa566820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.977+0000 7ff7f59dc700 1 -- 172.21.15.46:0/3067318943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa55c720 msgr2=0x55ecaa566d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.977+0000 7ff7f59dc700 1 --2- 172.21.15.46:0/3067318943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa55c720 0x55ecaa566d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.977+0000 7ff7f59dc700 1 -- 172.21.15.46:0/3067318943 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa4c91e0 msgr2=0x55ecaa566820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.977+0000 7ff7f59dc700 1 --2- 172.21.15.46:0/3067318943 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa4c91e0 0x55ecaa566820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.977+0000 7ff7f59dc700 1 -- 172.21.15.46:0/3067318943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecaa2fa510 con 0x55ecaa3df520 2024-05-09T04:59:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.977+0000 7ff7f59dc700 1 --2- 172.21.15.46:0/3067318943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa3df520 0x55ecaa5662e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0008a30 tx=0x7ff7e0005980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fba784b3775daef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.977+0000 7ff7d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:59:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.977+0000 7ff7d57fa700 1 -- 172.21.15.46:0/3067318943 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e0031020 con 0x55ecaa3df520 2024-05-09T04:59:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.977+0000 7ff7d57fa700 1 -- 172.21.15.46:0/3067318943 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7ff7e0008f60 con 0x55ecaa3df520 2024-05-09T04:59:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.977+0000 7ff806656b80 1 -- 172.21.15.46:0/3067318943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e001e040 con 0x55ecaa3df520 2024-05-09T04:59:27.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.977+0000 7ff7d57fa700 1 -- 172.21.15.46:0/3067318943 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff7e001d020 con 0x55ecaa3df520 2024-05-09T04:59:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.977+0000 7ff806656b80 10 client.17169 fetch_fsmap learned FSMap version 25 2024-05-09T04:59:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.977+0000 7ff806656b80 1 -- 172.21.15.46:0/3067318943 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ecaa5780d0 con 0x55ecaa3df520 2024-05-09T04:59:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.978+0000 7ff7d57fa700 1 -- 172.21.15.46:0/3067318943 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e001e040 con 0x55ecaa3df520 2024-05-09T04:59:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.978+0000 7ff7d57fa700 1 -- 172.21.15.46:0/3067318943 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff7e0022440 con 0x55ecaa3df520 2024-05-09T04:59:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.978+0000 7ff806656b80 10 client.17169 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:59:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.978+0000 7ff806656b80 10 client.17169 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:59:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.978+0000 7ff806656b80 20 client.17169 populate_metadata read hostname 'smithi046' 2024-05-09T04:59:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.978+0000 7ff806656b80 1 --2- 172.21.15.46:0/3067318943 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ecaa57a5f0 0x55ecaa57ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.978+0000 7ff806656b80 4 client.17169 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:59:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.978+0000 7ff806656b80 1 -- 172.21.15.46:0/3067318943 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ecaa48ca50 con 0x55ecaa57a5f0 2024-05-09T04:59:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.978+0000 7ff7f51db700 1 --2- 172.21.15.46:0/3067318943 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ecaa57a5f0 0x55ecaa57ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.979+0000 7ff7d57fa700 10 client.17169 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.979+0000 7ff7f51db700 1 --2- 172.21.15.46:0/3067318943 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ecaa57a5f0 0x55ecaa57ca10 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a0096a1b1428848 server_cookie=cedc8feea96e5a0 in_seq=0 out_seq=0 2024-05-09T04:59:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.979+0000 7ff7d57fa700 1 -- 172.21.15.46:0/3067318943 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ecaa48ca50 con 0x55ecaa57a5f0 2024-05-09T04:59:27.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.979+0000 7ff7d57fa700 10 client.17169 handle_command_reply: tid=0 2024-05-09T04:59:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.979+0000 7ff806656b80 1 client.17169 shutdown 2024-05-09T04:59:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.980+0000 7ff806656b80 1 -- 172.21.15.46:0/3067318943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa3df520 msgr2=0x55ecaa5662e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.980+0000 7ff806656b80 1 --2- 172.21.15.46:0/3067318943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa3df520 0x55ecaa5662e0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7ff7e0008a30 tx=0x7ff7e0005980 comp rx=0 tx=0).stop 2024-05-09T04:59:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.980+0000 7ff806656b80 1 -- 172.21.15.46:0/3067318943 shutdown_connections 2024-05-09T04:59:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.980+0000 7ff806656b80 1 --2- 172.21.15.46:0/3067318943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa55c720 0x55ecaa566d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.980+0000 7ff806656b80 1 --2- 172.21.15.46:0/3067318943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa3df520 0x55ecaa5662e0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.980+0000 7ff806656b80 1 --2- 172.21.15.46:0/3067318943 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ecaa57a5f0 0x55ecaa57ca10 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.980+0000 7ff806656b80 1 --2- 172.21.15.46:0/3067318943 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa4c91e0 0x55ecaa566820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.980+0000 7ff7d57fa700 0 client.17169 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.980+0000 7ff806656b80 1 -- 172.21.15.46:0/3067318943 >> 172.21.15.46:0/3067318943 conn(0x55ecaa43f720 msgr2=0x55ecaa557240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.980+0000 7ff806656b80 1 -- 172.21.15.46:0/3067318943 shutdown_connections 2024-05-09T04:59:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.980+0000 7ff806656b80 1 -- 172.21.15.46:0/3067318943 wait complete. 2024-05-09T04:59:27.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:27.980+0000 7ff806656b80 20 client.17169 trim_cache size 0 max 16384 2024-05-09T04:59:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.001+0000 7ff806656b80 1 Processor -- start 2024-05-09T04:59:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.001+0000 7ff806656b80 1 -- start start 2024-05-09T04:59:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.001+0000 7ff806656b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 0x55ecaa55c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.001+0000 7ff806656b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa4c91e0 0x55ecaa565490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.001+0000 7ff806656b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa55c720 0x55ecaa566ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:28.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.001+0000 7ff806656b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f01af770 con 0x55ecaa55c720 2024-05-09T04:59:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.001+0000 7ff806656b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f01ad000 con 0x55ecaa3df520 2024-05-09T04:59:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.001+0000 7ff806656b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f01aae70 con 0x55ecaa4c91e0 2024-05-09T04:59:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.001+0000 7ff7f61dd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa55c720 0x55ecaa566ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.001+0000 7ff7f61dd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa55c720 0x55ecaa566ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50100/0 (socket says 172.21.15.46:50100) 2024-05-09T04:59:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.001+0000 7ff7f61dd700 1 -- 172.21.15.46:0/2287893923 learned_addr learned my addr 172.21.15.46:0/2287893923 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.001+0000 7ff7f59dc700 1 --2- 172.21.15.46:0/2287893923 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 0x55ecaa55c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.002+0000 7ff7f51db700 1 --2- 172.21.15.46:0/2287893923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa4c91e0 0x55ecaa565490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.002+0000 7ff7f61dd700 1 -- 172.21.15.46:0/2287893923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa4c91e0 msgr2=0x55ecaa565490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.002+0000 7ff7f61dd700 1 --2- 172.21.15.46:0/2287893923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa4c91e0 0x55ecaa565490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.002+0000 7ff7f61dd700 1 -- 172.21.15.46:0/2287893923 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 msgr2=0x55ecaa55c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.002+0000 7ff7f61dd700 1 --2- 172.21.15.46:0/2287893923 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 0x55ecaa55c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.003 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.002+0000 7ff7f61dd700 1 -- 172.21.15.46:0/2287893923 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ecaa550fe0 con 0x55ecaa55c720 2024-05-09T04:59:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.002+0000 7ff7f61dd700 1 --2- 172.21.15.46:0/2287893923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa55c720 0x55ecaa566ba0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8005b40 tx=0x7ff7e8005ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9ce611c3193c73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.002+0000 7ff7d77fe700 1 -- 172.21.15.46:0/2287893923 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e8005350 con 0x55ecaa55c720 2024-05-09T04:59:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.002+0000 7ff7d77fe700 1 -- 172.21.15.46:0/2287893923 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e80054f0 con 0x55ecaa55c720 2024-05-09T04:59:28.004 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.002+0000 7ff7d77fe700 1 -- 172.21.15.46:0/2287893923 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e800bd90 con 0x55ecaa55c720 2024-05-09T04:59:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.003+0000 7ff806656b80 1 -- 172.21.15.46:0/2287893923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa55c720 msgr2=0x55ecaa566ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.003+0000 7ff806656b80 1 --2- 172.21.15.46:0/2287893923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa55c720 0x55ecaa566ba0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7ff7e8005b40 tx=0x7ff7e8005ad0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.003+0000 7ff806656b80 1 -- 172.21.15.46:0/2287893923 shutdown_connections 2024-05-09T04:59:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.003+0000 7ff806656b80 1 --2- 172.21.15.46:0/2287893923 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa55c720 0x55ecaa566ba0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.003+0000 7ff806656b80 1 --2- 172.21.15.46:0/2287893923 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 0x55ecaa55c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.003+0000 7ff806656b80 1 --2- 172.21.15.46:0/2287893923 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa4c91e0 0x55ecaa565490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.003+0000 7ff806656b80 1 -- 172.21.15.46:0/2287893923 >> 172.21.15.46:0/2287893923 conn(0x55ecaa43f720 msgr2=0x55ecaa55fe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.003+0000 7ff806656b80 1 -- 172.21.15.46:0/2287893923 shutdown_connections 2024-05-09T04:59:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.003+0000 7ff806656b80 1 -- 172.21.15.46:0/2287893923 wait complete. 2024-05-09T04:59:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.003+0000 7ff806656b80 1 -- start start 2024-05-09T04:59:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.004+0000 7ff806656b80 1 Processor -- start 2024-05-09T04:59:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.004+0000 7ff806656b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 0x55ecaa57fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.004+0000 7ff806656b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa4c91e0 0x55ecaa57d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.004+0000 7ff806656b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa55c720 0x55ecaa57d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.004+0000 7ff806656b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f01ab4c0 con 0x55ecaa4c91e0 2024-05-09T04:59:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.004+0000 7ff806656b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f01ac930 con 0x55ecaa3df520 2024-05-09T04:59:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.004+0000 7ff806656b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff7f01abb90 con 0x55ecaa55c720 2024-05-09T04:59:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.004+0000 7ff806656b80 10 client.0 _ll_register_callbacks cb 0x55ecaa2f0740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:59:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.004+0000 7ff7f51db700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa4c91e0 0x55ecaa57d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:28.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.004+0000 7ff7f51db700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa4c91e0 0x55ecaa57d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50106/0 (socket says 172.21.15.46:50106) 2024-05-09T04:59:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.004+0000 7ff7f51db700 1 -- 172.21.15.46:0/2749681699 learned_addr learned my addr 172.21.15.46:0/2749681699 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.004+0000 7ff7f59dc700 1 --2- 172.21.15.46:0/2749681699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 0x55ecaa57fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.004+0000 7ff7f61dd700 1 --2- 172.21.15.46:0/2749681699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa55c720 0x55ecaa57d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.005+0000 7ff7f51db700 1 -- 172.21.15.46:0/2749681699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa55c720 msgr2=0x55ecaa57d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.005+0000 7ff7f51db700 1 --2- 172.21.15.46:0/2749681699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa55c720 0x55ecaa57d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.005+0000 7ff7f51db700 1 -- 172.21.15.46:0/2749681699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 msgr2=0x55ecaa57fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.005+0000 7ff7f51db700 1 --2- 172.21.15.46:0/2749681699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 0x55ecaa57fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.005+0000 7ff7f51db700 1 -- 172.21.15.46:0/2749681699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ecaa57df40 con 0x55ecaa4c91e0 2024-05-09T04:59:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.005+0000 7ff7f59dc700 1 --2- 172.21.15.46:0/2749681699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 0x55ecaa57fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T04:59:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.005+0000 7ff7f51db700 1 --2- 172.21.15.46:0/2749681699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa4c91e0 0x55ecaa57d120 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40e01a0 tx=0x7ff7e4005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d3b7bda83c6d9c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.005+0000 7ff7d57fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:59:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.005+0000 7ff7d57fa700 1 -- 172.21.15.46:0/2749681699 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7e40c4340 con 0x55ecaa4c91e0 2024-05-09T04:59:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.005+0000 7ff7d57fa700 1 -- 172.21.15.46:0/2749681699 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7ff7e40aa0f0 con 0x55ecaa4c91e0 2024-05-09T04:59:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.005+0000 7ff806656b80 1 -- 172.21.15.46:0/2749681699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff7e80054f0 con 0x55ecaa4c91e0 2024-05-09T04:59:28.010 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.005+0000 7ff7d57fa700 1 -- 172.21.15.46:0/2749681699 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff7e40aa380 con 0x55ecaa4c91e0 2024-05-09T04:59:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.005+0000 7ff806656b80 10 client.17175 fetch_fsmap learned FSMap version 25 2024-05-09T04:59:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.005+0000 7ff806656b80 1 -- 172.21.15.46:0/2749681699 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ecaa5e4bf0 con 0x55ecaa4c91e0 2024-05-09T04:59:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.005+0000 7ff7d57fa700 1 -- 172.21.15.46:0/2749681699 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7e80054f0 con 0x55ecaa4c91e0 2024-05-09T04:59:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.006+0000 7ff7d57fa700 1 -- 172.21.15.46:0/2749681699 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff7e40a55f0 con 0x55ecaa4c91e0 2024-05-09T04:59:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.006+0000 7ff806656b80 10 client.17175 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:59:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.006+0000 7ff806656b80 10 client.17175 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:59:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.006+0000 7ff806656b80 20 client.17175 populate_metadata read hostname 'smithi046' 2024-05-09T04:59:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.006+0000 7ff806656b80 1 --2- 172.21.15.46:0/2749681699 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ecaa5e5280 0x55ecaa5e76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.006+0000 7ff806656b80 4 client.17175 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T04:59:28.011 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.006+0000 7ff806656b80 1 -- 172.21.15.46:0/2749681699 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ecaa486af0 con 0x55ecaa5e5280 2024-05-09T04:59:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.006+0000 7ff7f59dc700 1 --2- 172.21.15.46:0/2749681699 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ecaa5e5280 0x55ecaa5e76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.006+0000 7ff7d57fa700 10 client.17175 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.006+0000 7ff7f59dc700 1 --2- 172.21.15.46:0/2749681699 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ecaa5e5280 0x55ecaa5e76a0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d273665062ba2d21 server_cookie=1046ebd8ff33df9 in_seq=0 out_seq=0 2024-05-09T04:59:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.008+0000 7ff7d57fa700 1 -- 172.21.15.46:0/2749681699 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ecaa486af0 con 0x55ecaa5e5280 2024-05-09T04:59:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.008+0000 7ff7d57fa700 10 client.17175 handle_command_reply: tid=0 2024-05-09T04:59:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.008+0000 7ff806656b80 1 client.17175 shutdown 2024-05-09T04:59:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.008+0000 7ff806656b80 1 -- 172.21.15.46:0/2749681699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa4c91e0 msgr2=0x55ecaa57d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.008+0000 7ff806656b80 1 --2- 172.21.15.46:0/2749681699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa4c91e0 0x55ecaa57d120 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40e01a0 tx=0x7ff7e4005910 comp rx=0 tx=0).stop 2024-05-09T04:59:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.008+0000 7ff806656b80 1 -- 172.21.15.46:0/2749681699 shutdown_connections 2024-05-09T04:59:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.008+0000 7ff806656b80 1 --2- 172.21.15.46:0/2749681699 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ecaa55c720 0x55ecaa57d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.012 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.008+0000 7ff806656b80 1 --2- 172.21.15.46:0/2749681699 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ecaa5e5280 0x55ecaa5e76a0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.008+0000 7ff806656b80 1 --2- 172.21.15.46:0/2749681699 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ecaa3df520 0x55ecaa57fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.008+0000 7ff806656b80 1 --2- 172.21.15.46:0/2749681699 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ecaa4c91e0 0x55ecaa57d120 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.008+0000 7ff7d57fa700 0 client.17175 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.008+0000 7ff806656b80 1 -- 172.21.15.46:0/2749681699 >> 172.21.15.46:0/2749681699 conn(0x55ecaa43f720 msgr2=0x55ecaa55bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.008+0000 7ff806656b80 1 -- 172.21.15.46:0/2749681699 shutdown_connections 2024-05-09T04:59:28.013 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.008+0000 7ff806656b80 1 -- 172.21.15.46:0/2749681699 wait complete. 2024-05-09T04:59:28.014 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.008+0000 7ff806656b80 20 client.17175 trim_cache size 0 max 16384 2024-05-09T04:59:28.014 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:59:28.014 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T04:59:28.014 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "f764d258-e927-4a64-a0fc-a6259cbbbfd4", 2024-05-09T04:59:28.015 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T04:59:28.015 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:59:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.009+0000 7ff7f7c40700 1 -- 172.21.15.46:0/410742662 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff7d80b29f0 msgr2=0x7ff7d80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.009+0000 7ff7f7c40700 1 --2- 172.21.15.46:0/410742662 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff7d80b29f0 0x7ff7d80b4f00 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000f540 tx=0x7ff7e000b040 comp rx=0 tx=0).stop 2024-05-09T04:59:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.009+0000 7ff7f7c40700 1 -- 172.21.15.46:0/410742662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f0108f00 msgr2=0x7ff7f019b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:28.016 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.009+0000 7ff7f7c40700 1 --2- 172.21.15.46:0/410742662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f0108f00 0x7ff7f019b380 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7ff7e400e970 tx=0x7ff7e400ec80 comp rx=0 tx=0).stop 2024-05-09T04:59:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.009+0000 7ff7f7c40700 1 -- 172.21.15.46:0/410742662 shutdown_connections 2024-05-09T04:59:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.009+0000 7ff7f7c40700 1 --2- 172.21.15.46:0/410742662 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff7d80b29f0 0x7ff7d80b4f00 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.009+0000 7ff7f7c40700 1 --2- 172.21.15.46:0/410742662 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff7f0109850 0x7ff7f019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.009+0000 7ff7f7c40700 1 --2- 172.21.15.46:0/410742662 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff7f01045d0 0x7ff7f019ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.009+0000 7ff7f7c40700 1 --2- 172.21.15.46:0/410742662 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff7f0108f00 0x7ff7f019b380 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:28.017 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.009+0000 7ff7f7c40700 1 -- 172.21.15.46:0/410742662 >> 172.21.15.46:0/410742662 conn(0x7ff7f00ffea0 msgr2=0x7ff7f0108670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.009+0000 7ff7f7c40700 1 -- 172.21.15.46:0/410742662 shutdown_connections 2024-05-09T04:59:28.018 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:28.009+0000 7ff7f7c40700 1 -- 172.21.15.46:0/410742662 wait complete. 2024-05-09T04:59:28.815 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T04:59:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T04:59:29.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:28 smithi046 ceph-mon[30136]: pgmap v258: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-09T04:59:29.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:28 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T04:59:29.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:28 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:59:29.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:28 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T04:59:29.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:28 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T04:59:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:28 smithi072 ceph-mon[34088]: pgmap v258: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-09T04:59:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:28 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T04:59:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:28 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:59:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:28 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T04:59:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:28 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T04:59:29.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:28 smithi110 ceph-mon[34904]: pgmap v258: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 7 op/s 2024-05-09T04:59:29.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:28 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T04:59:29.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:28 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:59:29.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:28 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T04:59:29.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:28 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T04:59:30.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.739+0000 7f121a1bb700 1 -- 172.21.15.46:0/2015823244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12141034e0 msgr2=0x7f12141038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.739+0000 7f121a1bb700 1 --2- 172.21.15.46:0/2015823244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12141034e0 0x7f12141038f0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f120400b0d0 tx=0x7f120400b3e0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.740+0000 7f121a1bb700 1 -- 172.21.15.46:0/2015823244 shutdown_connections 2024-05-09T04:59:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.740+0000 7f121a1bb700 1 --2- 172.21.15.46:0/2015823244 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1214108760 0x7f12141100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.740+0000 7f121a1bb700 1 --2- 172.21.15.46:0/2015823244 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f12141034e0 0x7f12141038f0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.740+0000 7f121a1bb700 1 --2- 172.21.15.46:0/2015823244 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f1214107e10 0x7f1214108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.740+0000 7f121a1bb700 1 -- 172.21.15.46:0/2015823244 >> 172.21.15.46:0/2015823244 conn(0x7f12140fef30 msgr2=0x7f1214101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.740+0000 7f121a1bb700 1 -- 172.21.15.46:0/2015823244 shutdown_connections 2024-05-09T04:59:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.740+0000 7f121a1bb700 1 -- 172.21.15.46:0/2015823244 wait complete. 2024-05-09T04:59:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.740+0000 7f121a1bb700 1 Processor -- start 2024-05-09T04:59:30.742 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.741+0000 7f121a1bb700 1 -- start start 2024-05-09T04:59:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.741+0000 7f121a1bb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12141034e0 0x7f121419ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.741+0000 7f121a1bb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1214107e10 0x7f121419b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.741+0000 7f121a1bb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1214108760 0x7f121419f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.741+0000 7f121a1bb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f1214113400 con 0x7f1214108760 2024-05-09T04:59:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.741+0000 7f121a1bb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f1214113280 con 0x7f1214107e10 2024-05-09T04:59:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.741+0000 7f121a1bb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f1214113580 con 0x7f12141034e0 2024-05-09T04:59:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.741+0000 7f1213fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1214108760 0x7f121419f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.741+0000 7f1213fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1214108760 0x7f121419f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50122/0 (socket says 172.21.15.46:50122) 2024-05-09T04:59:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.741+0000 7f1213fff700 1 -- 172.21.15.46:0/1034270700 learned_addr learned my addr 172.21.15.46:0/1034270700 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.741+0000 7f12137fe700 1 --2- 172.21.15.46:0/1034270700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12141034e0 0x7f121419ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.741+0000 7f1212ffd700 1 --2- 172.21.15.46:0/1034270700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1214107e10 0x7f121419b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.742+0000 7f1213fff700 1 -- 172.21.15.46:0/1034270700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12141034e0 msgr2=0x7f121419ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:30.744 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.742+0000 7f1213fff700 1 --2- 172.21.15.46:0/1034270700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12141034e0 0x7f121419ae60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.742+0000 7f1213fff700 1 -- 172.21.15.46:0/1034270700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1214107e10 msgr2=0x7f121419b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.742+0000 7f1213fff700 1 --2- 172.21.15.46:0/1034270700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1214107e10 0x7f121419b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.742+0000 7f1213fff700 1 -- 172.21.15.46:0/1034270700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f121419fd30 con 0x7f1214108760 2024-05-09T04:59:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.742+0000 7f1213fff700 1 --2- 172.21.15.46:0/1034270700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1214108760 0x7f121419f690 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f120c00d980 tx=0x7f120c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.742+0000 7f1210ff9700 1 -- 172.21.15.46:0/1034270700 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c017070 con 0x7f1214108760 2024-05-09T04:59:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.742+0000 7f121a1bb700 1 -- 172.21.15.46:0/1034270700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12141a0020 con 0x7f1214108760 2024-05-09T04:59:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.742+0000 7f1210ff9700 1 -- 172.21.15.46:0/1034270700 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f120c005020 con 0x7f1214108760 2024-05-09T04:59:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.743+0000 7f1210ff9700 1 -- 172.21.15.46:0/1034270700 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c00fa10 con 0x7f1214108760 2024-05-09T04:59:30.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.743+0000 7f121a1bb700 1 -- 172.21.15.46:0/1034270700 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12141a7940 con 0x7f1214108760 2024-05-09T04:59:30.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.746+0000 7f1210ff9700 1 -- 172.21.15.46:0/1034270700 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f120c004a30 con 0x7f1214108760 2024-05-09T04:59:30.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.746+0000 7f1210ff9700 1 --2- 172.21.15.46:0/1034270700 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f12000b29f0 0x7f12000b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.747+0000 7f12137fe700 1 --2- 172.21.15.46:0/1034270700 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f12000b29f0 0x7f12000b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.747+0000 7f1210ff9700 1 -- 172.21.15.46:0/1034270700 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f120c0e24b0 con 0x7f1214108760 2024-05-09T04:59:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.747+0000 7f12137fe700 1 --2- 172.21.15.46:0/1034270700 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f12000b29f0 0x7f12000b4f00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1204009900 tx=0x7f120400b040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.747+0000 7f1228bd1b80 1 Processor -- start 2024-05-09T04:59:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.747+0000 7f1228bd1b80 1 -- start start 2024-05-09T04:59:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1228bd1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb226d00 0x564adb32f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1228bd1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb1ac4a0 0x564adb32f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1228bd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb329400 0x564adb329810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1228bd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564adb24c0a0 con 0x564adb329400 2024-05-09T04:59:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1228bd1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x564adb1f19a0 con 0x564adb226d00 2024-05-09T04:59:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1228bd1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564adb248060 con 0x564adb1ac4a0 2024-05-09T04:59:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1213fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb329400 0x564adb329810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f12137fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb226d00 0x564adb32f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1213fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb329400 0x564adb329810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50138/0 (socket says 172.21.15.46:50138) 2024-05-09T04:59:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1213fff700 1 -- 172.21.15.46:0/2294987191 learned_addr learned my addr 172.21.15.46:0/2294987191 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f12137fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb226d00 0x564adb32f240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:40654/0 (socket says 172.21.15.46:40654) 2024-05-09T04:59:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1212ffd700 1 --2- 172.21.15.46:0/2294987191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb1ac4a0 0x564adb32f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1213fff700 1 -- 172.21.15.46:0/2294987191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb1ac4a0 msgr2=0x564adb32f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1213fff700 1 --2- 172.21.15.46:0/2294987191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb1ac4a0 0x564adb32f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1213fff700 1 -- 172.21.15.46:0/2294987191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb226d00 msgr2=0x564adb32f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1213fff700 1 --2- 172.21.15.46:0/2294987191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb226d00 0x564adb32f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1213fff700 1 -- 172.21.15.46:0/2294987191 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564adb1e2500 con 0x564adb329400 2024-05-09T04:59:30.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.748+0000 7f1213fff700 1 --2- 172.21.15.46:0/2294987191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb329400 0x564adb329810 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f120c0a57c0 tx=0x7f120c0a9ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b03bc8ab8b3207 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:30.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.749+0000 7f1209ffb700 1 -- 172.21.15.46:0/2294987191 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c0a3e60 con 0x564adb329400 2024-05-09T04:59:30.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.749+0000 7f1209ffb700 1 -- 172.21.15.46:0/2294987191 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f120c0b25a0 con 0x564adb329400 2024-05-09T04:59:30.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.749+0000 7f1209ffb700 1 -- 172.21.15.46:0/2294987191 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f120c0a99a0 con 0x564adb329400 2024-05-09T04:59:30.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.749+0000 7f1228bd1b80 1 -- 172.21.15.46:0/2294987191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb329400 msgr2=0x564adb329810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:30.754 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.749+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/2294987191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb329400 0x564adb329810 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f120c0a57c0 tx=0x7f120c0a9ec0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.750+0000 7f1228bd1b80 1 -- 172.21.15.46:0/2294987191 shutdown_connections 2024-05-09T04:59:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.750+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/2294987191 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb329400 0x564adb329810 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.750+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/2294987191 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb226d00 0x564adb32f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.750+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/2294987191 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb1ac4a0 0x564adb32f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.750+0000 7f1228bd1b80 1 -- 172.21.15.46:0/2294987191 >> 172.21.15.46:0/2294987191 conn(0x564adb1aff70 msgr2=0x564adb324220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.750+0000 7f1228bd1b80 1 -- 172.21.15.46:0/2294987191 shutdown_connections 2024-05-09T04:59:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.750+0000 7f1228bd1b80 1 -- 172.21.15.46:0/2294987191 wait complete. 2024-05-09T04:59:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.750+0000 7f1228bd1b80 1 -- start start 2024-05-09T04:59:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.750+0000 7f1228bd1b80 1 Processor -- start 2024-05-09T04:59:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1228bd1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb1ac4a0 0x564adb3331a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1228bd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb226d00 0x564adb3336e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1228bd1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 0x564adb333c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1228bd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564adb32a160 con 0x564adb226d00 2024-05-09T04:59:30.755 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1228bd1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x564adb1e6b20 con 0x564adb1ac4a0 2024-05-09T04:59:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1228bd1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564adb336020 con 0x564adb329400 2024-05-09T04:59:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1228bd1b80 10 client.0 _ll_register_callbacks cb 0x564adb0bd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:59:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1212ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb226d00 0x564adb3336e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1212ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb226d00 0x564adb3336e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50152/0 (socket says 172.21.15.46:50152) 2024-05-09T04:59:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1212ffd700 1 -- 172.21.15.46:0/3435527177 learned_addr learned my addr 172.21.15.46:0/3435527177 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f12137fe700 1 --2- 172.21.15.46:0/3435527177 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb1ac4a0 0x564adb3331a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1213fff700 1 --2- 172.21.15.46:0/3435527177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 0x564adb333c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1212ffd700 1 -- 172.21.15.46:0/3435527177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 msgr2=0x564adb333c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1212ffd700 1 --2- 172.21.15.46:0/3435527177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 0x564adb333c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1212ffd700 1 -- 172.21.15.46:0/3435527177 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb1ac4a0 msgr2=0x564adb3331a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1212ffd700 1 --2- 172.21.15.46:0/3435527177 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb1ac4a0 0x564adb3331a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1212ffd700 1 -- 172.21.15.46:0/3435527177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564adb0c7490 con 0x564adb226d00 2024-05-09T04:59:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1213fff700 1 --2- 172.21.15.46:0/3435527177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 0x564adb333c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T04:59:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f1212ffd700 1 --2- 172.21.15.46:0/3435527177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb226d00 0x564adb3336e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f11fc011570 tx=0x7f11fc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7592a92209315c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:30.756 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.751+0000 7f11f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:59:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.752+0000 7f11f2ffd700 1 -- 172.21.15.46:0/3435527177 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc014650 con 0x564adb226d00 2024-05-09T04:59:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.752+0000 7f11f2ffd700 1 -- 172.21.15.46:0/3435527177 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f11fc0147f0 con 0x564adb226d00 2024-05-09T04:59:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.752+0000 7f1228bd1b80 1 -- 172.21.15.46:0/3435527177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f120c0b25a0 con 0x564adb226d00 2024-05-09T04:59:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.752+0000 7f11f2ffd700 1 -- 172.21.15.46:0/3435527177 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f11fc0149f0 con 0x564adb226d00 2024-05-09T04:59:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.752+0000 7f1228bd1b80 10 client.17187 fetch_fsmap learned FSMap version 25 2024-05-09T04:59:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.752+0000 7f1228bd1b80 1 -- 172.21.15.46:0/3435527177 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564adb340bd0 con 0x564adb226d00 2024-05-09T04:59:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.752+0000 7f11f2ffd700 1 -- 172.21.15.46:0/3435527177 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f120c0b25a0 con 0x564adb226d00 2024-05-09T04:59:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.752+0000 7f11f2ffd700 1 -- 172.21.15.46:0/3435527177 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f11fc025c90 con 0x564adb226d00 2024-05-09T04:59:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.752+0000 7f1228bd1b80 10 client.17187 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:59:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.752+0000 7f1228bd1b80 10 client.17187 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:59:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.752+0000 7f1228bd1b80 20 client.17187 populate_metadata read hostname 'smithi046' 2024-05-09T04:59:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.752+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/3435527177 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564adb343210 0x564adb345630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.752+0000 7f1228bd1b80 4 client.17187 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T04:59:30.757 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.752+0000 7f1228bd1b80 1 -- 172.21.15.46:0/3435527177 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564adb259a80 con 0x564adb343210 2024-05-09T04:59:30.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.753+0000 7f12137fe700 1 --2- 172.21.15.46:0/3435527177 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564adb343210 0x564adb345630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.753+0000 7f12137fe700 1 --2- 172.21.15.46:0/3435527177 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564adb343210 0x564adb345630 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c0a1fe569405950 server_cookie=c1c04fcc6a6fcc32 in_seq=0 out_seq=0 2024-05-09T04:59:30.758 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.753+0000 7f11f2ffd700 10 client.17187 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.754+0000 7f11f2ffd700 1 -- 172.21.15.46:0/3435527177 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564adb259a80 con 0x564adb343210 2024-05-09T04:59:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.754+0000 7f11f2ffd700 10 client.17187 handle_command_reply: tid=0 2024-05-09T04:59:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.754+0000 7f1228bd1b80 1 client.17187 shutdown 2024-05-09T04:59:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.754+0000 7f1228bd1b80 1 -- 172.21.15.46:0/3435527177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb226d00 msgr2=0x564adb3336e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.754+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/3435527177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb226d00 0x564adb3336e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f11fc011570 tx=0x7f11fc009f90 comp rx=0 tx=0).stop 2024-05-09T04:59:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.754+0000 7f1228bd1b80 1 -- 172.21.15.46:0/3435527177 shutdown_connections 2024-05-09T04:59:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.754+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/3435527177 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 0x564adb333c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.754+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/3435527177 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564adb343210 0x564adb345630 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.754+0000 7f11f2ffd700 0 client.17187 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.754+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/3435527177 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb1ac4a0 0x564adb3331a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.754+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/3435527177 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb226d00 0x564adb3336e0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.754+0000 7f1228bd1b80 1 -- 172.21.15.46:0/3435527177 >> 172.21.15.46:0/3435527177 conn(0x564adb1aff70 msgr2=0x564adb31dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.754+0000 7f1228bd1b80 1 -- 172.21.15.46:0/3435527177 shutdown_connections 2024-05-09T04:59:30.759 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.754+0000 7f1228bd1b80 1 -- 172.21.15.46:0/3435527177 wait complete. 2024-05-09T04:59:30.760 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.754+0000 7f1228bd1b80 20 client.17187 trim_cache size 0 max 16384 2024-05-09T04:59:30.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.775+0000 7f1228bd1b80 1 Processor -- start 2024-05-09T04:59:30.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.775+0000 7f1228bd1b80 1 -- start start 2024-05-09T04:59:30.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.775+0000 7f1228bd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb1ac4a0 0x564adb31ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.775+0000 7f1228bd1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb226d00 0x564adb333a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.775+0000 7f1228bd1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 0x564adb33ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.775+0000 7f1228bd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f12141b18d0 con 0x564adb1ac4a0 2024-05-09T04:59:30.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.775+0000 7f1228bd1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f12141af160 con 0x564adb226d00 2024-05-09T04:59:30.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.775+0000 7f1228bd1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f12141acfd0 con 0x564adb329400 2024-05-09T04:59:30.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.775+0000 7f12137fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb1ac4a0 0x564adb31ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.775+0000 7f12137fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb1ac4a0 0x564adb31ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50166/0 (socket says 172.21.15.46:50166) 2024-05-09T04:59:30.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.775+0000 7f12137fe700 1 -- 172.21.15.46:0/4042811087 learned_addr learned my addr 172.21.15.46:0/4042811087 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:30.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.776+0000 7f1212ffd700 1 --2- 172.21.15.46:0/4042811087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb226d00 0x564adb333a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.776+0000 7f1213fff700 1 --2- 172.21.15.46:0/4042811087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 0x564adb33ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.776+0000 7f12137fe700 1 -- 172.21.15.46:0/4042811087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 msgr2=0x564adb33ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:30.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.776+0000 7f12137fe700 1 --2- 172.21.15.46:0/4042811087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 0x564adb33ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.776+0000 7f12137fe700 1 -- 172.21.15.46:0/4042811087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb226d00 msgr2=0x564adb333a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:30.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.776+0000 7f12137fe700 1 --2- 172.21.15.46:0/4042811087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb226d00 0x564adb333a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.776+0000 7f12137fe700 1 -- 172.21.15.46:0/4042811087 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564adb3209d0 con 0x564adb1ac4a0 2024-05-09T04:59:30.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.776+0000 7f1212ffd700 1 --2- 172.21.15.46:0/4042811087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb226d00 0x564adb333a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T04:59:30.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.776+0000 7f12137fe700 1 --2- 172.21.15.46:0/4042811087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb1ac4a0 0x564adb31ea20 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f120400b010 tx=0x7f1204020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ec7cd852431d5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:30.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.776+0000 7f12097fa700 1 -- 172.21.15.46:0/4042811087 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204007ab0 con 0x564adb1ac4a0 2024-05-09T04:59:30.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.776+0000 7f12097fa700 1 -- 172.21.15.46:0/4042811087 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1204039070 con 0x564adb1ac4a0 2024-05-09T04:59:30.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.776+0000 7f12097fa700 1 -- 172.21.15.46:0/4042811087 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204029680 con 0x564adb1ac4a0 2024-05-09T04:59:30.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.777+0000 7f1228bd1b80 1 -- 172.21.15.46:0/4042811087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb1ac4a0 msgr2=0x564adb31ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:30.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.777+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/4042811087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb1ac4a0 0x564adb31ea20 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f120400b010 tx=0x7f1204020000 comp rx=0 tx=0).stop 2024-05-09T04:59:30.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.777+0000 7f1228bd1b80 1 -- 172.21.15.46:0/4042811087 shutdown_connections 2024-05-09T04:59:30.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.777+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/4042811087 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 0x564adb33ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.777+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/4042811087 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb1ac4a0 0x564adb31ea20 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.782 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.777+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/4042811087 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb226d00 0x564adb333a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.777+0000 7f1228bd1b80 1 -- 172.21.15.46:0/4042811087 >> 172.21.15.46:0/4042811087 conn(0x564adb1aff70 msgr2=0x564adb32e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:30.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.777+0000 7f1228bd1b80 1 -- 172.21.15.46:0/4042811087 shutdown_connections 2024-05-09T04:59:30.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.777+0000 7f1228bd1b80 1 -- 172.21.15.46:0/4042811087 wait complete. 2024-05-09T04:59:30.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.777+0000 7f1228bd1b80 1 -- start start 2024-05-09T04:59:30.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1228bd1b80 1 Processor -- start 2024-05-09T04:59:30.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1228bd1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb1ac4a0 0x564adb33e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1228bd1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb226d00 0x564adb33ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1228bd1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 0x564adb33ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1228bd1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f12141ad620 con 0x564adb226d00 2024-05-09T04:59:30.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1228bd1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f12141aea90 con 0x564adb1ac4a0 2024-05-09T04:59:30.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1228bd1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f12141adcf0 con 0x564adb329400 2024-05-09T04:59:30.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1228bd1b80 10 client.0 _ll_register_callbacks cb 0x564adb0bd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T04:59:30.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1212ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb226d00 0x564adb33ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.783 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1212ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb226d00 0x564adb33ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50176/0 (socket says 172.21.15.46:50176) 2024-05-09T04:59:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1212ffd700 1 -- 172.21.15.46:0/1882489226 learned_addr learned my addr 172.21.15.46:0/1882489226 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T04:59:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f12137fe700 1 --2- 172.21.15.46:0/1882489226 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb1ac4a0 0x564adb33e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1213fff700 1 --2- 172.21.15.46:0/1882489226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 0x564adb33ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1212ffd700 1 -- 172.21.15.46:0/1882489226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 msgr2=0x564adb33ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1212ffd700 1 --2- 172.21.15.46:0/1882489226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 0x564adb33ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1212ffd700 1 -- 172.21.15.46:0/1882489226 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb1ac4a0 msgr2=0x564adb33e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T04:59:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1212ffd700 1 --2- 172.21.15.46:0/1882489226 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb1ac4a0 0x564adb33e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1212ffd700 1 -- 172.21.15.46:0/1882489226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564adb33bc20 con 0x564adb226d00 2024-05-09T04:59:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f1212ffd700 1 --2- 172.21.15.46:0/1882489226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb226d00 0x564adb33ea10 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f11fc01d610 tx=0x7f11fc014e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301a776ef2c507ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T04:59:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.778+0000 7f11f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T04:59:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.779+0000 7f11f2ffd700 1 -- 172.21.15.46:0/1882489226 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11fc032060 con 0x564adb226d00 2024-05-09T04:59:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.779+0000 7f11f2ffd700 1 -- 172.21.15.46:0/1882489226 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f11fc01db60 con 0x564adb226d00 2024-05-09T04:59:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.779+0000 7f1228bd1b80 1 -- 172.21.15.46:0/1882489226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1204039070 con 0x564adb226d00 2024-05-09T04:59:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.779+0000 7f11f2ffd700 1 -- 172.21.15.46:0/1882489226 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f11fc01dd80 con 0x564adb226d00 2024-05-09T04:59:30.784 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.779+0000 7f1228bd1b80 10 client.17193 fetch_fsmap learned FSMap version 25 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.779+0000 7f1228bd1b80 1 -- 172.21.15.46:0/1882489226 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564adb3ad9f0 con 0x564adb226d00 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.779+0000 7f11f2ffd700 1 -- 172.21.15.46:0/1882489226 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1204039070 con 0x564adb226d00 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.779+0000 7f11f2ffd700 1 -- 172.21.15.46:0/1882489226 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f11fc035410 con 0x564adb226d00 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.779+0000 7f1228bd1b80 10 client.17193 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.779+0000 7f1228bd1b80 10 client.17193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.779+0000 7f1228bd1b80 20 client.17193 populate_metadata read hostname 'smithi046' 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.779+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/1882489226 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564adb343210 0x564adb3b0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.779+0000 7f1228bd1b80 4 client.17193 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.779+0000 7f1228bd1b80 1 -- 172.21.15.46:0/1882489226 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564adb253b20 con 0x564adb343210 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.780+0000 7f12137fe700 1 --2- 172.21.15.46:0/1882489226 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564adb343210 0x564adb3b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.780+0000 7f11f2ffd700 10 client.17193 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.780+0000 7f12137fe700 1 --2- 172.21.15.46:0/1882489226 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564adb343210 0x564adb3b0140 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b737a3b86edf85f5 server_cookie=b27156969414cf3b in_seq=0 out_seq=0 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.780+0000 7f11f2ffd700 1 -- 172.21.15.46:0/1882489226 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x564adb253b20 con 0x564adb343210 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.780+0000 7f11f2ffd700 10 client.17193 handle_command_reply: tid=0 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.780+0000 7f1228bd1b80 1 client.17193 shutdown 2024-05-09T04:59:30.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.780+0000 7f1228bd1b80 1 -- 172.21.15.46:0/1882489226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb226d00 msgr2=0x564adb33ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.780+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/1882489226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb226d00 0x564adb33ea10 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f11fc01d610 tx=0x7f11fc014e70 comp rx=0 tx=0).stop 2024-05-09T04:59:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.780+0000 7f1228bd1b80 1 -- 172.21.15.46:0/1882489226 shutdown_connections 2024-05-09T04:59:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.780+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/1882489226 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564adb329400 0x564adb33ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/1882489226 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x564adb343210 0x564adb3b0140 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/1882489226 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564adb1ac4a0 0x564adb33e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f1228bd1b80 1 --2- 172.21.15.46:0/1882489226 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564adb226d00 0x564adb33ea10 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f11f2ffd700 0 client.17193 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T04:59:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f1228bd1b80 1 -- 172.21.15.46:0/1882489226 >> 172.21.15.46:0/1882489226 conn(0x564adb1aff70 msgr2=0x564adb32e240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f1228bd1b80 1 -- 172.21.15.46:0/1882489226 shutdown_connections 2024-05-09T04:59:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f1228bd1b80 1 -- 172.21.15.46:0/1882489226 wait complete. 2024-05-09T04:59:30.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f1228bd1b80 20 client.17193 trim_cache size 0 max 16384 2024-05-09T04:59:30.787 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T04:59:30.787 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T04:59:30.787 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-09T04:59:30.787 INFO:teuthology.orchestra.run.smithi046.stdout: "f764d258-e927-4a64-a0fc-a6259cbbbfd4": { 2024-05-09T04:59:30.787 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-09T04:59:30.787 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "f764d258-e927-4a64-a0fc-a6259cbbbfd4", 2024-05-09T04:59:30.787 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-09T04:59:30.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T04:59:30.787 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T04:59:30.787 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T04:59:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f121a1bb700 1 -- 172.21.15.46:0/1034270700 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f12000b29f0 msgr2=0x7f12000b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f121a1bb700 1 --2- 172.21.15.46:0/1034270700 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f12000b29f0 0x7f12000b4f00 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1204009900 tx=0x7f120400b040 comp rx=0 tx=0).stop 2024-05-09T04:59:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f121a1bb700 1 -- 172.21.15.46:0/1034270700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1214108760 msgr2=0x7f121419f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T04:59:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f121a1bb700 1 --2- 172.21.15.46:0/1034270700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1214108760 0x7f121419f690 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f120c00d980 tx=0x7f120c00dc90 comp rx=0 tx=0).stop 2024-05-09T04:59:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f121a1bb700 1 -- 172.21.15.46:0/1034270700 shutdown_connections 2024-05-09T04:59:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f121a1bb700 1 --2- 172.21.15.46:0/1034270700 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f12000b29f0 0x7f12000b4f00 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f121a1bb700 1 --2- 172.21.15.46:0/1034270700 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f1214108760 0x7f121419f690 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f121a1bb700 1 --2- 172.21.15.46:0/1034270700 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f12141034e0 0x7f121419ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f121a1bb700 1 --2- 172.21.15.46:0/1034270700 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f1214107e10 0x7f121419b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T04:59:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f121a1bb700 1 -- 172.21.15.46:0/1034270700 >> 172.21.15.46:0/1034270700 conn(0x7f12140fef30 msgr2=0x7f121406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T04:59:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.781+0000 7f121a1bb700 1 -- 172.21.15.46:0/1034270700 shutdown_connections 2024-05-09T04:59:30.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T04:59:30.782+0000 7f121a1bb700 1 -- 172.21.15.46:0/1034270700 wait complete. 2024-05-09T04:59:31.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:30 smithi046 ceph-mon[30136]: pgmap v259: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:59:31.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:30 smithi072 ceph-mon[34088]: pgmap v259: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:59:31.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:30 smithi110 ceph-mon[34904]: pgmap v259: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:59:31.592 INFO:tasks.cephfs.filesystem:scrub status for tag:f764d258-e927-4a64-a0fc-a6259cbbbfd4 - {'path': '/', 'tag': 'f764d258-e927-4a64-a0fc-a6259cbbbfd4', 'options': 'recursive,force'} 2024-05-09T04:59:32.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:31 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T04:59:32.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:31 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T04:59:32.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:31 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T04:59:33.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:32 smithi072 ceph-mon[34088]: pgmap v260: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-05-09T04:59:33.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:32 smithi110 ceph-mon[34904]: pgmap v260: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-05-09T04:59:33.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:32 smithi046 ceph-mon[30136]: pgmap v260: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-05-09T04:59:35.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:34 smithi072 ceph-mon[34088]: pgmap v261: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T04:59:35.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:34 smithi110 ceph-mon[34904]: pgmap v261: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T04:59:35.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:34 smithi046 ceph-mon[30136]: pgmap v261: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T04:59:36.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:35 smithi072 ceph-mon[34088]: pgmap v262: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:59:36.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:35 smithi110 ceph-mon[34904]: pgmap v262: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:59:36.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:35 smithi046 ceph-mon[30136]: pgmap v262: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:59:39.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:38 smithi110 ceph-mon[34904]: pgmap v263: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T04:59:39.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:38 smithi046 ceph-mon[30136]: pgmap v263: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T04:59:39.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:38 smithi072 ceph-mon[34088]: pgmap v263: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T04:59:41.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:40 smithi110 ceph-mon[34904]: pgmap v264: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:59:41.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:40 smithi046 ceph-mon[30136]: pgmap v264: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:59:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:40 smithi072 ceph-mon[34088]: pgmap v264: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T04:59:43.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:42 smithi110 ceph-mon[34904]: pgmap v265: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T04:59:43.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:42 smithi046 ceph-mon[30136]: pgmap v265: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T04:59:43.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:42 smithi072 ceph-mon[34088]: pgmap v265: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T04:59:45.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:44 smithi110 ceph-mon[34904]: pgmap v266: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:59:45.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:44 smithi046 ceph-mon[30136]: pgmap v266: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:59:45.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:44 smithi072 ceph-mon[34088]: pgmap v266: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T04:59:47.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:46 smithi110 ceph-mon[34904]: pgmap v267: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:59:47.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:46 smithi046 ceph-mon[30136]: pgmap v267: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:59:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:46 smithi072 ceph-mon[34088]: pgmap v267: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:59:49.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:48 smithi110 ceph-mon[34904]: pgmap v268: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:59:49.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:48 smithi046 ceph-mon[30136]: pgmap v268: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:59:49.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:48 smithi072 ceph-mon[34088]: pgmap v268: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:59:51.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:50 smithi046 ceph-mon[30136]: pgmap v269: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:59:51.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:50 smithi072 ceph-mon[34088]: pgmap v269: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:59:51.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:50 smithi110 ceph-mon[34904]: pgmap v269: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:59:53.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:52 smithi046 ceph-mon[30136]: pgmap v270: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:59:53.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:52 smithi072 ceph-mon[34088]: pgmap v270: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:59:53.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:52 smithi110 ceph-mon[34904]: pgmap v270: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T04:59:55.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:54 smithi046 ceph-mon[30136]: pgmap v271: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:59:55.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:54 smithi072 ceph-mon[34088]: pgmap v271: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:59:55.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:54 smithi110 ceph-mon[34904]: pgmap v271: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T04:59:57.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:56 smithi046 ceph-mon[30136]: pgmap v272: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:59:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:56 smithi072 ceph-mon[34088]: pgmap v272: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:59:57.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:56 smithi110 ceph-mon[34904]: pgmap v272: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T04:59:59.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 04:59:58 smithi046 ceph-mon[30136]: pgmap v273: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:59:59.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 04:59:58 smithi072 ceph-mon[34088]: pgmap v273: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T04:59:59.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 04:59:58 smithi110 ceph-mon[34904]: pgmap v273: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:00:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:00 smithi046 ceph-mon[30136]: pgmap v274: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:00:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:00 smithi046 ceph-mon[30136]: overall HEALTH_OK 2024-05-09T05:00:01.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:00 smithi072 ceph-mon[34088]: pgmap v274: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:00:01.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:00 smithi072 ceph-mon[34088]: overall HEALTH_OK 2024-05-09T05:00:01.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:00 smithi110 ceph-mon[34904]: pgmap v274: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:00:01.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:00 smithi110 ceph-mon[34904]: overall HEALTH_OK 2024-05-09T05:00:01.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:00:01.860 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:03.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:02 smithi046 ceph-mon[30136]: pgmap v275: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-09T05:00:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:02 smithi072 ceph-mon[34088]: pgmap v275: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-09T05:00:03.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:02 smithi110 ceph-mon[34904]: pgmap v275: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 23 KiB/s wr, 9 op/s 2024-05-09T05:00:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.703+0000 7f33d611d700 1 -- 172.21.15.46:0/3049299230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d01045e0 msgr2=0x7f33d01049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.704+0000 7f33d611d700 1 --2- 172.21.15.46:0/3049299230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d01045e0 0x7f33d01049f0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f33c00099c0 tx=0x7f33c0009cd0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.704+0000 7f33d611d700 1 -- 172.21.15.46:0/3049299230 shutdown_connections 2024-05-09T05:00:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.704+0000 7f33d611d700 1 --2- 172.21.15.46:0/3049299230 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f33d0109860 0x7f33d01111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.704 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.704+0000 7f33d611d700 1 --2- 172.21.15.46:0/3049299230 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d01045e0 0x7f33d01049f0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.704+0000 7f33d611d700 1 --2- 172.21.15.46:0/3049299230 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33d0108f10 0x7f33d0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.704+0000 7f33d611d700 1 -- 172.21.15.46:0/3049299230 >> 172.21.15.46:0/3049299230 conn(0x7f33d00ffe70 msgr2=0x7f33d01022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.704+0000 7f33d611d700 1 -- 172.21.15.46:0/3049299230 shutdown_connections 2024-05-09T05:00:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.705+0000 7f33d611d700 1 -- 172.21.15.46:0/3049299230 wait complete. 2024-05-09T05:00:03.705 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.705+0000 7f33d611d700 1 Processor -- start 2024-05-09T05:00:03.706 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.705+0000 7f33d611d700 1 -- start start 2024-05-09T05:00:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33d611d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33d01045e0 0x7f33d019ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33d611d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f33d0108f10 0x7f33d019b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.707 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33d611d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d0109860 0x7f33d019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33d611d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33d01144e0 con 0x7f33d0109860 2024-05-09T05:00:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33d611d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f33d0114360 con 0x7f33d0108f10 2024-05-09T05:00:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33d611d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33d0114660 con 0x7f33d01045e0 2024-05-09T05:00:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33cffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d0109860 0x7f33d019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33cffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d0109860 0x7f33d019f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:46996/0 (socket says 172.21.15.46:46996) 2024-05-09T05:00:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33cffff700 1 -- 172.21.15.46:0/3954716104 learned_addr learned my addr 172.21.15.46:0/3954716104 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33ceffd700 1 --2- 172.21.15.46:0/3954716104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f33d0108f10 0x7f33d019b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33cf7fe700 1 --2- 172.21.15.46:0/3954716104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33d01045e0 0x7f33d019ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33cffff700 1 -- 172.21.15.46:0/3954716104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33d01045e0 msgr2=0x7f33d019ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33cffff700 1 --2- 172.21.15.46:0/3954716104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33d01045e0 0x7f33d019ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33cffff700 1 -- 172.21.15.46:0/3954716104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f33d0108f10 msgr2=0x7f33d019b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33cffff700 1 --2- 172.21.15.46:0/3954716104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f33d0108f10 0x7f33d019b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.706+0000 7f33cffff700 1 -- 172.21.15.46:0/3954716104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33d019fd40 con 0x7f33d0109860 2024-05-09T05:00:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.707+0000 7f33cffff700 1 --2- 172.21.15.46:0/3954716104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d0109860 0x7f33d019f6a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f33c800ef30 tx=0x7f33c800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:03.708 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.707+0000 7f33ccff9700 1 -- 172.21.15.46:0/3954716104 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c80092e0 con 0x7f33d0109860 2024-05-09T05:00:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.707+0000 7f33d611d700 1 -- 172.21.15.46:0/3954716104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33d019ffd0 con 0x7f33d0109860 2024-05-09T05:00:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.707+0000 7f33ccff9700 1 -- 172.21.15.46:0/3954716104 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c8004bb0 con 0x7f33d0109860 2024-05-09T05:00:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.707+0000 7f33ccff9700 1 -- 172.21.15.46:0/3954716104 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c8004020 con 0x7f33d0109860 2024-05-09T05:00:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.707+0000 7f33d611d700 1 -- 172.21.15.46:0/3954716104 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33d01a04c0 con 0x7f33d0109860 2024-05-09T05:00:03.709 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.709+0000 7f33ccff9700 1 -- 172.21.15.46:0/3954716104 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f33c801b030 con 0x7f33d0109860 2024-05-09T05:00:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.710+0000 7f33ccff9700 1 --2- 172.21.15.46:0/3954716104 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f33b80b29f0 0x7f33b80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.710+0000 7f33cf7fe700 1 --2- 172.21.15.46:0/3954716104 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f33b80b29f0 0x7f33b80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.710+0000 7f33ccff9700 1 -- 172.21.15.46:0/3954716104 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f33c80e2120 con 0x7f33d0109860 2024-05-09T05:00:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.710+0000 7f33e4b33b80 1 Processor -- start 2024-05-09T05:00:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33e4b33b80 1 -- start start 2024-05-09T05:00:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33cf7fe700 1 --2- 172.21.15.46:0/3954716104 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f33b80b29f0 0x7f33b80b4f00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f33c0003760 tx=0x7f33c001a040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:03.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33e4b33b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 0x55594d7cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33e4b33b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d64c4a0 0x55594d7ce030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33e4b33b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d7d21e0 0x55594d7d25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33e4b33b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55594d6ec0a0 con 0x55594d64c4a0 2024-05-09T05:00:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33e4b33b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55594d6919a0 con 0x55594d7d21e0 2024-05-09T05:00:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33e4b33b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55594d6e8060 con 0x55594d6c6d00 2024-05-09T05:00:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33ceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d64c4a0 0x55594d7ce030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33ceffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d64c4a0 0x55594d7ce030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47004/0 (socket says 172.21.15.46:47004) 2024-05-09T05:00:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33ceffd700 1 -- 172.21.15.46:0/259730979 learned_addr learned my addr 172.21.15.46:0/259730979 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33cffff700 1 --2- 172.21.15.46:0/259730979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d7d21e0 0x55594d7d25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33cf7fe700 1 --2- 172.21.15.46:0/259730979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 0x55594d7cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33ceffd700 1 -- 172.21.15.46:0/259730979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 msgr2=0x55594d7cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33ceffd700 1 --2- 172.21.15.46:0/259730979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 0x55594d7cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33ceffd700 1 -- 172.21.15.46:0/259730979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d7d21e0 msgr2=0x55594d7d25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:03.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33ceffd700 1 --2- 172.21.15.46:0/259730979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d7d21e0 0x55594d7d25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33ceffd700 1 -- 172.21.15.46:0/259730979 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55594d682500 con 0x55594d64c4a0 2024-05-09T05:00:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.711+0000 7f33cffff700 1 --2- 172.21.15.46:0/259730979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d7d21e0 0x55594d7d25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.712+0000 7f33ceffd700 1 --2- 172.21.15.46:0/259730979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d64c4a0 0x55594d7ce030 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f33c400bcf0 tx=0x7f33c40103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0094c5024ea10e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.712+0000 7f33b5ffb700 1 -- 172.21.15.46:0/259730979 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4010dc0 con 0x55594d64c4a0 2024-05-09T05:00:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.712+0000 7f33b5ffb700 1 -- 172.21.15.46:0/259730979 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c40044e0 con 0x55594d64c4a0 2024-05-09T05:00:03.714 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.712+0000 7f33b5ffb700 1 -- 172.21.15.46:0/259730979 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4005280 con 0x55594d64c4a0 2024-05-09T05:00:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.713+0000 7f33e4b33b80 1 -- 172.21.15.46:0/259730979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d64c4a0 msgr2=0x55594d7ce030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.713+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/259730979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d64c4a0 0x55594d7ce030 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f33c400bcf0 tx=0x7f33c40103d0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.713+0000 7f33e4b33b80 1 -- 172.21.15.46:0/259730979 shutdown_connections 2024-05-09T05:00:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.713+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/259730979 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d7d21e0 0x55594d7d25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.713+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/259730979 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 0x55594d7cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.718 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.713+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/259730979 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d64c4a0 0x55594d7ce030 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.713+0000 7f33e4b33b80 1 -- 172.21.15.46:0/259730979 >> 172.21.15.46:0/259730979 conn(0x55594d64ff70 msgr2=0x55594d7c4220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.713+0000 7f33e4b33b80 1 -- 172.21.15.46:0/259730979 shutdown_connections 2024-05-09T05:00:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.713+0000 7f33e4b33b80 1 -- 172.21.15.46:0/259730979 wait complete. 2024-05-09T05:00:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.713+0000 7f33e4b33b80 1 -- start start 2024-05-09T05:00:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.713+0000 7f33e4b33b80 1 Processor -- start 2024-05-09T05:00:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33e4b33b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d64c4a0 0x55594d7d73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33e4b33b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 0x55594d7d7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33e4b33b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d7d21e0 0x55594d7d7e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33e4b33b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55594d7d2f00 con 0x55594d7d21e0 2024-05-09T05:00:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33e4b33b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55594d686b20 con 0x55594d64c4a0 2024-05-09T05:00:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33e4b33b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55594d7da470 con 0x55594d6c6d00 2024-05-09T05:00:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33e4b33b80 10 client.0 _ll_register_callbacks cb 0x55594d55d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33cffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d7d21e0 0x55594d7d7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33cffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d7d21e0 0x55594d7d7e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47010/0 (socket says 172.21.15.46:47010) 2024-05-09T05:00:03.719 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33cffff700 1 -- 172.21.15.46:0/1489165708 learned_addr learned my addr 172.21.15.46:0/1489165708 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33cf7fe700 1 --2- 172.21.15.46:0/1489165708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d64c4a0 0x55594d7d73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33ceffd700 1 --2- 172.21.15.46:0/1489165708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 0x55594d7d7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33cffff700 1 -- 172.21.15.46:0/1489165708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 msgr2=0x55594d7d7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33cffff700 1 --2- 172.21.15.46:0/1489165708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 0x55594d7d7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33cffff700 1 -- 172.21.15.46:0/1489165708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d64c4a0 msgr2=0x55594d7d73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33cffff700 1 --2- 172.21.15.46:0/1489165708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d64c4a0 0x55594d7d73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33cffff700 1 -- 172.21.15.46:0/1489165708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55594d567490 con 0x55594d7d21e0 2024-05-09T05:00:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33ceffd700 1 --2- 172.21.15.46:0/1489165708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 0x55594d7d7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:00:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33cf7fe700 1 --2- 172.21.15.46:0/1489165708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d64c4a0 0x55594d7d73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33cffff700 1 --2- 172.21.15.46:0/1489165708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d7d21e0 0x55594d7d7e40 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f33c80a90e0 tx=0x7f33c80a9920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbef14282a87f97 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.714+0000 7f33aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.715+0000 7f33aeffd700 1 -- 172.21.15.46:0/1489165708 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c80c3060 con 0x55594d7d21e0 2024-05-09T05:00:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.715+0000 7f33aeffd700 1 -- 172.21.15.46:0/1489165708 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f33c80a8850 con 0x55594d7d21e0 2024-05-09T05:00:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.715+0000 7f33e4b33b80 1 -- 172.21.15.46:0/1489165708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33c40044e0 con 0x55594d7d21e0 2024-05-09T05:00:03.720 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.715+0000 7f33aeffd700 1 -- 172.21.15.46:0/1489165708 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f33c80a9ae0 con 0x55594d7d21e0 2024-05-09T05:00:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.715+0000 7f33e4b33b80 10 client.17205 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.715+0000 7f33e4b33b80 1 -- 172.21.15.46:0/1489165708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55594d7e4db0 con 0x55594d7d21e0 2024-05-09T05:00:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.715+0000 7f33aeffd700 1 -- 172.21.15.46:0/1489165708 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c40044e0 con 0x55594d7d21e0 2024-05-09T05:00:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.715+0000 7f33aeffd700 1 -- 172.21.15.46:0/1489165708 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f33c80a4de0 con 0x55594d7d21e0 2024-05-09T05:00:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.715+0000 7f33e4b33b80 10 client.17205 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.715+0000 7f33e4b33b80 10 client.17205 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.715+0000 7f33e4b33b80 20 client.17205 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.715+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/1489165708 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55594d7e73f0 0x55594d7e9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.715+0000 7f33e4b33b80 4 client.17205 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:00:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.715+0000 7f33e4b33b80 1 -- 172.21.15.46:0/1489165708 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55594d6f9a80 con 0x55594d7e73f0 2024-05-09T05:00:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.716+0000 7f33cf7fe700 1 --2- 172.21.15.46:0/1489165708 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55594d7e73f0 0x55594d7e9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.716+0000 7f33cf7fe700 1 --2- 172.21.15.46:0/1489165708 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55594d7e73f0 0x55594d7e9810 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f27747c9203a65f2 server_cookie=4788c6e9ab544bf8 in_seq=0 out_seq=0 2024-05-09T05:00:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.716+0000 7f33aeffd700 10 client.17205 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:03.721 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.717+0000 7f33aeffd700 1 -- 172.21.15.46:0/1489165708 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55594d6f9a80 con 0x55594d7e73f0 2024-05-09T05:00:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.717+0000 7f33aeffd700 10 client.17205 handle_command_reply: tid=0 2024-05-09T05:00:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.717+0000 7f33e4b33b80 1 client.17205 shutdown 2024-05-09T05:00:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.717+0000 7f33e4b33b80 1 -- 172.21.15.46:0/1489165708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d7d21e0 msgr2=0x55594d7d7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.717+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/1489165708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d7d21e0 0x55594d7d7e40 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f33c80a90e0 tx=0x7f33c80a9920 comp rx=0 tx=0).stop 2024-05-09T05:00:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.717+0000 7f33e4b33b80 1 -- 172.21.15.46:0/1489165708 shutdown_connections 2024-05-09T05:00:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.717+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/1489165708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d7d21e0 0x55594d7d7e40 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.717+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/1489165708 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55594d7e73f0 0x55594d7e9810 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.717+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/1489165708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d64c4a0 0x55594d7d73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.717+0000 7f33aeffd700 0 client.17205 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.717+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/1489165708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 0x55594d7d7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.717+0000 7f33e4b33b80 1 -- 172.21.15.46:0/1489165708 >> 172.21.15.46:0/1489165708 conn(0x55594d64ff70 msgr2=0x55594d7bdff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:03.722 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.717+0000 7f33e4b33b80 1 -- 172.21.15.46:0/1489165708 shutdown_connections 2024-05-09T05:00:03.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.717+0000 7f33e4b33b80 1 -- 172.21.15.46:0/1489165708 wait complete. 2024-05-09T05:00:03.723 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.717+0000 7f33e4b33b80 20 client.17205 trim_cache size 0 max 16384 2024-05-09T05:00:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.738+0000 7f33e4b33b80 1 Processor -- start 2024-05-09T05:00:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.738+0000 7f33e4b33b80 1 -- start start 2024-05-09T05:00:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.738+0000 7f33e4b33b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d64c4a0 0x55594d7d1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.738+0000 7f33e4b33b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d6c6d00 0x55594d7d7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.739 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.738+0000 7f33e4b33b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d7d21e0 0x55594d7e3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.738+0000 7f33e4b33b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33d01af710 con 0x55594d64c4a0 2024-05-09T05:00:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.738+0000 7f33e4b33b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f33d01acfa0 con 0x55594d6c6d00 2024-05-09T05:00:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.738+0000 7f33e4b33b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33d01aae10 con 0x55594d7d21e0 2024-05-09T05:00:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.738+0000 7f33cf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d64c4a0 0x55594d7d1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.738+0000 7f33cf7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d64c4a0 0x55594d7d1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47014/0 (socket says 172.21.15.46:47014) 2024-05-09T05:00:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.738+0000 7f33cf7fe700 1 -- 172.21.15.46:0/35777644 learned_addr learned my addr 172.21.15.46:0/35777644 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.738+0000 7f33ceffd700 1 --2- 172.21.15.46:0/35777644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d6c6d00 0x55594d7d7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.739+0000 7f33cffff700 1 --2- 172.21.15.46:0/35777644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d7d21e0 0x55594d7e3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.739+0000 7f33cf7fe700 1 -- 172.21.15.46:0/35777644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d7d21e0 msgr2=0x55594d7e3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.739+0000 7f33cf7fe700 1 --2- 172.21.15.46:0/35777644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d7d21e0 0x55594d7e3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.739+0000 7f33cf7fe700 1 -- 172.21.15.46:0/35777644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d6c6d00 msgr2=0x55594d7d7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.739+0000 7f33cf7fe700 1 --2- 172.21.15.46:0/35777644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d6c6d00 0x55594d7d7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.739+0000 7f33cf7fe700 1 -- 172.21.15.46:0/35777644 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55594d7c51b0 con 0x55594d64c4a0 2024-05-09T05:00:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.739+0000 7f33cf7fe700 1 --2- 172.21.15.46:0/35777644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d64c4a0 0x55594d7d1a40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f33c00099c0 tx=0x7f33c0009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33201f68375f9af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:03.740 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.739+0000 7f33b57fa700 1 -- 172.21.15.46:0/35777644 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0037070 con 0x55594d64c4a0 2024-05-09T05:00:03.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.739+0000 7f33b57fa700 1 -- 172.21.15.46:0/35777644 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c0004d60 con 0x55594d64c4a0 2024-05-09T05:00:03.741 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.739+0000 7f33b57fa700 1 -- 172.21.15.46:0/35777644 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c000be50 con 0x55594d64c4a0 2024-05-09T05:00:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.740+0000 7f33e4b33b80 1 -- 172.21.15.46:0/35777644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d64c4a0 msgr2=0x55594d7d1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.740+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/35777644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d64c4a0 0x55594d7d1a40 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f33c00099c0 tx=0x7f33c0009cd0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.740+0000 7f33e4b33b80 1 -- 172.21.15.46:0/35777644 shutdown_connections 2024-05-09T05:00:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.740+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/35777644 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d7d21e0 0x55594d7e3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.740+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/35777644 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d64c4a0 0x55594d7d1a40 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.740+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/35777644 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d6c6d00 0x55594d7d7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.740+0000 7f33e4b33b80 1 -- 172.21.15.46:0/35777644 >> 172.21.15.46:0/35777644 conn(0x55594d64ff70 msgr2=0x55594d7be360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.740+0000 7f33e4b33b80 1 -- 172.21.15.46:0/35777644 shutdown_connections 2024-05-09T05:00:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.740+0000 7f33e4b33b80 1 -- 172.21.15.46:0/35777644 wait complete. 2024-05-09T05:00:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.740+0000 7f33e4b33b80 1 -- start start 2024-05-09T05:00:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.740+0000 7f33e4b33b80 1 Processor -- start 2024-05-09T05:00:03.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33e4b33b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d64c4a0 0x55594d7df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33e4b33b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 0x55594d7df920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33e4b33b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d7d21e0 0x55594d7dc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33e4b33b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f33d01ab460 con 0x55594d7d21e0 2024-05-09T05:00:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33e4b33b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f33d01ac8d0 con 0x55594d64c4a0 2024-05-09T05:00:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33e4b33b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f33d01abb30 con 0x55594d6c6d00 2024-05-09T05:00:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33cf7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d64c4a0 0x55594d7df3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33e4b33b80 10 client.0 _ll_register_callbacks cb 0x55594d55d6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33cf7fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d64c4a0 0x55594d7df3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:37120/0 (socket says 172.21.15.46:37120) 2024-05-09T05:00:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33cf7fe700 1 -- 172.21.15.46:0/3577591749 learned_addr learned my addr 172.21.15.46:0/3577591749 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33cffff700 1 --2- 172.21.15.46:0/3577591749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d7d21e0 0x55594d7dc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33ceffd700 1 --2- 172.21.15.46:0/3577591749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 0x55594d7df920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33cf7fe700 1 -- 172.21.15.46:0/3577591749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 msgr2=0x55594d7df920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33cf7fe700 1 --2- 172.21.15.46:0/3577591749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 0x55594d7df920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.746 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33cf7fe700 1 -- 172.21.15.46:0/3577591749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d7d21e0 msgr2=0x55594d7dc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33cf7fe700 1 --2- 172.21.15.46:0/3577591749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d7d21e0 0x55594d7dc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33cf7fe700 1 -- 172.21.15.46:0/3577591749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55594d7dfe90 con 0x55594d64c4a0 2024-05-09T05:00:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.741+0000 7f33cffff700 1 --2- 172.21.15.46:0/3577591749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d7d21e0 0x55594d7dc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:00:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.742+0000 7f33cf7fe700 1 --2- 172.21.15.46:0/3577591749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d64c4a0 0x55594d7df3e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f33c000b950 tx=0x7f33c0005190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a53afb2ef1985f43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.742+0000 7f33aeffd700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T05:00:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.742+0000 7f33aeffd700 1 -- 172.21.15.46:0/3577591749 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c0037040 con 0x55594d64c4a0 2024-05-09T05:00:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.742+0000 7f33aeffd700 1 -- 172.21.15.46:0/3577591749 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f33c000ba50 con 0x55594d64c4a0 2024-05-09T05:00:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.742+0000 7f33e4b33b80 1 -- 172.21.15.46:0/3577591749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f33c0004d60 con 0x55594d64c4a0 2024-05-09T05:00:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.742+0000 7f33aeffd700 1 -- 172.21.15.46:0/3577591749 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f33c000bc50 con 0x55594d64c4a0 2024-05-09T05:00:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.742+0000 7f33e4b33b80 10 client.26729 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.742+0000 7f33e4b33b80 1 -- 172.21.15.46:0/3577591749 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55594d851fc0 con 0x55594d64c4a0 2024-05-09T05:00:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.742+0000 7f33aeffd700 1 -- 172.21.15.46:0/3577591749 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c0004d60 con 0x55594d64c4a0 2024-05-09T05:00:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.743+0000 7f33aeffd700 1 -- 172.21.15.46:0/3577591749 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f33c0045030 con 0x55594d64c4a0 2024-05-09T05:00:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.743+0000 7f33e4b33b80 10 client.26729 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:03.747 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.743+0000 7f33e4b33b80 10 client.26729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:03.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.743+0000 7f33e4b33b80 20 client.26729 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:03.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.743+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/3577591749 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55594d852640 0x55594d854a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:03.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.743+0000 7f33e4b33b80 4 client.26729 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:00:03.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.743+0000 7f33e4b33b80 1 -- 172.21.15.46:0/3577591749 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55594d6f3b20 con 0x55594d852640 2024-05-09T05:00:03.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.743+0000 7f33ceffd700 1 --2- 172.21.15.46:0/3577591749 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55594d852640 0x55594d854a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:03.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.743+0000 7f33ceffd700 1 --2- 172.21.15.46:0/3577591749 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55594d852640 0x55594d854a60 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a745da5418ecd10d server_cookie=80fc92f188365e29 in_seq=0 out_seq=0 2024-05-09T05:00:03.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.743+0000 7f33aeffd700 10 client.26729 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:03.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.744+0000 7f33aeffd700 1 -- 172.21.15.46:0/3577591749 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55594d6f3b20 con 0x55594d852640 2024-05-09T05:00:03.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.744+0000 7f33aeffd700 10 client.26729 handle_command_reply: tid=0 2024-05-09T05:00:03.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.744+0000 7f33e4b33b80 1 client.26729 shutdown 2024-05-09T05:00:03.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.744+0000 7f33e4b33b80 1 -- 172.21.15.46:0/3577591749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d64c4a0 msgr2=0x55594d7df3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:03.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.744+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/3577591749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d64c4a0 0x55594d7df3e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f33c000b950 tx=0x7f33c0005190 comp rx=0 tx=0).stop 2024-05-09T05:00:03.748 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.744+0000 7f33e4b33b80 1 -- 172.21.15.46:0/3577591749 shutdown_connections 2024-05-09T05:00:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.744+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/3577591749 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55594d7d21e0 0x55594d7dc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.744+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/3577591749 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55594d852640 0x55594d854a60 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.744+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/3577591749 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55594d64c4a0 0x55594d7df3e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.744+0000 7f33e4b33b80 1 --2- 172.21.15.46:0/3577591749 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55594d6c6d00 0x55594d7df920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.744+0000 7f33aeffd700 0 client.26729 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.744+0000 7f33e4b33b80 1 -- 172.21.15.46:0/3577591749 >> 172.21.15.46:0/3577591749 conn(0x55594d64ff70 msgr2=0x55594d7d5b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.744+0000 7f33e4b33b80 1 -- 172.21.15.46:0/3577591749 shutdown_connections 2024-05-09T05:00:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.744+0000 7f33e4b33b80 1 -- 172.21.15.46:0/3577591749 wait complete. 2024-05-09T05:00:03.749 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.744+0000 7f33e4b33b80 20 client.26729 trim_cache size 0 max 16384 2024-05-09T05:00:03.749 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T05:00:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T05:00:03.750 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T05:00:03.750 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T05:00:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.745+0000 7f33d611d700 1 -- 172.21.15.46:0/3954716104 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f33b80b29f0 msgr2=0x7f33b80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:03.750 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.745+0000 7f33d611d700 1 --2- 172.21.15.46:0/3954716104 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f33b80b29f0 0x7f33b80b4f00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f33c0003760 tx=0x7f33c001a040 comp rx=0 tx=0).stop 2024-05-09T05:00:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.745+0000 7f33d611d700 1 -- 172.21.15.46:0/3954716104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d0109860 msgr2=0x7f33d019f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.745+0000 7f33d611d700 1 --2- 172.21.15.46:0/3954716104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d0109860 0x7f33d019f6a0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f33c800ef30 tx=0x7f33c800c3a0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.745+0000 7f33d611d700 1 -- 172.21.15.46:0/3954716104 shutdown_connections 2024-05-09T05:00:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.745+0000 7f33d611d700 1 --2- 172.21.15.46:0/3954716104 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f33b80b29f0 0x7f33b80b4f00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.745+0000 7f33d611d700 1 --2- 172.21.15.46:0/3954716104 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f33d0109860 0x7f33d019f6a0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.745+0000 7f33d611d700 1 --2- 172.21.15.46:0/3954716104 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f33d01045e0 0x7f33d019ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.745+0000 7f33d611d700 1 --2- 172.21.15.46:0/3954716104 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f33d0108f10 0x7f33d019b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.745+0000 7f33d611d700 1 -- 172.21.15.46:0/3954716104 >> 172.21.15.46:0/3954716104 conn(0x7f33d00ffe70 msgr2=0x7f33d01071c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.745+0000 7f33d611d700 1 -- 172.21.15.46:0/3954716104 shutdown_connections 2024-05-09T05:00:03.751 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:03.745+0000 7f33d611d700 1 -- 172.21.15.46:0/3954716104 wait complete. 2024-05-09T05:00:04.554 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T05:00:04.554 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:00:04.808 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:05.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:04 smithi046 ceph-mon[30136]: pgmap v276: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T05:00:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:04 smithi072 ceph-mon[34088]: pgmap v276: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T05:00:05.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:04 smithi110 ceph-mon[34904]: pgmap v276: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T05:00:06.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.530+0000 7f224b775700 1 -- 172.21.15.46:0/2827503193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2244076e40 msgr2=0x7f2244077250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.530+0000 7f224b775700 1 --2- 172.21.15.46:0/2827503193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2244076e40 0x7f2244077250 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2240009990 tx=0x7f2240009ca0 comp rx=0 tx=0).stop 2024-05-09T05:00:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.530+0000 7f224b775700 1 -- 172.21.15.46:0/2827503193 shutdown_connections 2024-05-09T05:00:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.530+0000 7f224b775700 1 --2- 172.21.15.46:0/2827503193 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2244110b90 0x7f2244112ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.530+0000 7f224b775700 1 --2- 172.21.15.46:0/2827503193 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2244076e40 0x7f2244077250 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.530+0000 7f224b775700 1 --2- 172.21.15.46:0/2827503193 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2244077790 0x7f2244110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.530+0000 7f224b775700 1 -- 172.21.15.46:0/2827503193 >> 172.21.15.46:0/2827503193 conn(0x7f22440ffef0 msgr2=0x7f2244102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.530+0000 7f224b775700 1 -- 172.21.15.46:0/2827503193 shutdown_connections 2024-05-09T05:00:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.530+0000 7f224b775700 1 -- 172.21.15.46:0/2827503193 wait complete. 2024-05-09T05:00:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.531+0000 7f224b775700 1 Processor -- start 2024-05-09T05:00:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.531+0000 7f224b775700 1 -- start start 2024-05-09T05:00:06.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.531+0000 7f224b775700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2244076e40 0x7f224419d080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.531+0000 7f224b775700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2244077790 0x7f224419d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.531+0000 7f224b775700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2244110b90 0x7f22441a18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.531+0000 7f224b775700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f22441166d0 con 0x7f2244076e40 2024-05-09T05:00:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.531+0000 7f224b775700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2244116550 con 0x7f2244077790 2024-05-09T05:00:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.531+0000 7f224b775700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2244116850 con 0x7f2244110b90 2024-05-09T05:00:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.532+0000 7f2249511700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2244076e40 0x7f224419d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.532+0000 7f2249511700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2244076e40 0x7f224419d080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47028/0 (socket says 172.21.15.46:47028) 2024-05-09T05:00:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.532+0000 7f2249511700 1 -- 172.21.15.46:0/1284618278 learned_addr learned my addr 172.21.15.46:0/1284618278 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.532+0000 7f2248d10700 1 --2- 172.21.15.46:0/1284618278 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2244077790 0x7f224419d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.532+0000 7f2249511700 1 -- 172.21.15.46:0/1284618278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2244110b90 msgr2=0x7f22441a18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.532+0000 7f2249511700 1 --2- 172.21.15.46:0/1284618278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2244110b90 0x7f22441a18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:06.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.532+0000 7f2249511700 1 -- 172.21.15.46:0/1284618278 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2244077790 msgr2=0x7f224419d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.532+0000 7f2249511700 1 --2- 172.21.15.46:0/1284618278 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2244077790 0x7f224419d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.532+0000 7f2249511700 1 -- 172.21.15.46:0/1284618278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22441a1f50 con 0x7f2244076e40 2024-05-09T05:00:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.532+0000 7f2249511700 1 --2- 172.21.15.46:0/1284618278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2244076e40 0x7f224419d080 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2240005910 tx=0x7f22400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.532+0000 7f223a7fc700 1 -- 172.21.15.46:0/1284618278 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240027070 con 0x7f2244076e40 2024-05-09T05:00:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.532+0000 7f224b775700 1 -- 172.21.15.46:0/1284618278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22441a21e0 con 0x7f2244076e40 2024-05-09T05:00:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.532+0000 7f223a7fc700 1 -- 172.21.15.46:0/1284618278 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2240003f20 con 0x7f2244076e40 2024-05-09T05:00:06.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.532+0000 7f223a7fc700 1 -- 172.21.15.46:0/1284618278 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2240022d00 con 0x7f2244076e40 2024-05-09T05:00:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.533+0000 7f224b775700 1 -- 172.21.15.46:0/1284618278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22441a26c0 con 0x7f2244076e40 2024-05-09T05:00:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.534+0000 7f224b775700 1 -- 172.21.15.46:0/1284618278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22440512c0 con 0x7f2244076e40 2024-05-09T05:00:06.536 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.536+0000 7f223a7fc700 1 -- 172.21.15.46:0/1284618278 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f224002a070 con 0x7f2244076e40 2024-05-09T05:00:06.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.537+0000 7f223a7fc700 1 --2- 172.21.15.46:0/1284618278 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22300b29f0 0x7f22300b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:06.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.537+0000 7f2248d10700 1 --2- 172.21.15.46:0/1284618278 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22300b29f0 0x7f22300b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:06.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.537+0000 7f223a7fc700 1 -- 172.21.15.46:0/1284618278 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f22400e39f0 con 0x7f2244076e40 2024-05-09T05:00:06.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.538+0000 7f2248d10700 1 --2- 172.21.15.46:0/1284618278 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22300b29f0 0x7f22300b4f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f22441033f0 tx=0x7f22340073d0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:06.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.541+0000 7f223a7fc700 1 -- 172.21.15.46:0/1284618278 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2240031080 con 0x7f2244076e40 2024-05-09T05:00:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.802+0000 7f224b775700 1 -- 172.21.15.46:0/1284618278 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f224402e5f0 con 0x7f2244076e40 2024-05-09T05:00:06.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.803+0000 7f223a7fc700 1 -- 172.21.15.46:0/1284618278 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7f2240013070 con 0x7f2244076e40 2024-05-09T05:00:06.804 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T05:00:06.804 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T05:00:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.805+0000 7f224b775700 1 -- 172.21.15.46:0/1284618278 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22300b29f0 msgr2=0x7f22300b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.805+0000 7f224b775700 1 --2- 172.21.15.46:0/1284618278 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22300b29f0 0x7f22300b4f00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f22441033f0 tx=0x7f22340073d0 comp rx=0 tx=0).stop 2024-05-09T05:00:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.806+0000 7f224b775700 1 -- 172.21.15.46:0/1284618278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2244076e40 msgr2=0x7f224419d080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.806+0000 7f224b775700 1 --2- 172.21.15.46:0/1284618278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2244076e40 0x7f224419d080 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2240005910 tx=0x7f22400047b0 comp rx=0 tx=0).stop 2024-05-09T05:00:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.806+0000 7f224b775700 1 -- 172.21.15.46:0/1284618278 shutdown_connections 2024-05-09T05:00:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.806+0000 7f224b775700 1 --2- 172.21.15.46:0/1284618278 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22300b29f0 0x7f22300b4f00 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.806+0000 7f224b775700 1 --2- 172.21.15.46:0/1284618278 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2244110b90 0x7f22441a18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:06.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.806+0000 7f224b775700 1 --2- 172.21.15.46:0/1284618278 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2244076e40 0x7f224419d080 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:06.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.806+0000 7f224b775700 1 --2- 172.21.15.46:0/1284618278 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2244077790 0x7f224419d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:06.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.806+0000 7f224b775700 1 -- 172.21.15.46:0/1284618278 >> 172.21.15.46:0/1284618278 conn(0x7f22440ffef0 msgr2=0x7f2244100540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:06.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.806+0000 7f224b775700 1 -- 172.21.15.46:0/1284618278 shutdown_connections 2024-05-09T05:00:06.807 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:06.806+0000 7f224b775700 1 -- 172.21.15.46:0/1284618278 wait complete. 2024-05-09T05:00:06.807 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T05:00:07.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:06 smithi046 ceph-mon[30136]: pgmap v277: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T05:00:07.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:06 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1284618278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:00:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:06 smithi072 ceph-mon[34088]: pgmap v277: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T05:00:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:06 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1284618278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:00:07.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:06 smithi110 ceph-mon[34904]: pgmap v277: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T05:00:07.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:06 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1284618278' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:00:07.475 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T05:00:07.729 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:09.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:08 smithi046 ceph-mon[30136]: pgmap v278: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-05-09T05:00:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:08 smithi072 ceph-mon[34088]: pgmap v278: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-05-09T05:00:09.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:08 smithi110 ceph-mon[34904]: pgmap v278: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-05-09T05:00:09.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.367+0000 7f230be7d700 1 -- 172.21.15.46:0/4107425389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2304108f10 msgr2=0x7f2304109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.367+0000 7f230be7d700 1 --2- 172.21.15.46:0/4107425389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2304108f10 0x7f2304109320 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f22f40099c0 tx=0x7f22f4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.367+0000 7f230be7d700 1 -- 172.21.15.46:0/4107425389 shutdown_connections 2024-05-09T05:00:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.367+0000 7f230be7d700 1 --2- 172.21.15.46:0/4107425389 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2304109860 0x7f23041111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.367+0000 7f230be7d700 1 --2- 172.21.15.46:0/4107425389 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23041045e0 0x7f23041049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.367+0000 7f230be7d700 1 --2- 172.21.15.46:0/4107425389 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2304108f10 0x7f2304109320 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.367+0000 7f230be7d700 1 -- 172.21.15.46:0/4107425389 >> 172.21.15.46:0/4107425389 conn(0x7f23040ffed0 msgr2=0x7f2304102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.367+0000 7f230be7d700 1 -- 172.21.15.46:0/4107425389 shutdown_connections 2024-05-09T05:00:09.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.367+0000 7f230be7d700 1 -- 172.21.15.46:0/4107425389 wait complete. 2024-05-09T05:00:09.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.368+0000 7f230be7d700 1 Processor -- start 2024-05-09T05:00:09.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.368+0000 7f230be7d700 1 -- start start 2024-05-09T05:00:09.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.368+0000 7f230be7d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23041045e0 0x7f2304076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.368+0000 7f230be7d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2304108f10 0x7f2304077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.368+0000 7f230be7d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2304109860 0x7f2304077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.368+0000 7f230be7d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2304114470 con 0x7f2304109860 2024-05-09T05:00:09.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.368+0000 7f230be7d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f23041142f0 con 0x7f23041045e0 2024-05-09T05:00:09.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.368+0000 7f230be7d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f23041145f0 con 0x7f2304108f10 2024-05-09T05:00:09.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.368+0000 7f230a41a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2304109860 0x7f2304077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.368+0000 7f230a41a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2304109860 0x7f2304077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47046/0 (socket says 172.21.15.46:47046) 2024-05-09T05:00:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.368+0000 7f230a41a700 1 -- 172.21.15.46:0/387811333 learned_addr learned my addr 172.21.15.46:0/387811333 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.368+0000 7f2309418700 1 --2- 172.21.15.46:0/387811333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2304108f10 0x7f2304077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.368+0000 7f2309c19700 1 --2- 172.21.15.46:0/387811333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23041045e0 0x7f2304076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.369+0000 7f230a41a700 1 -- 172.21.15.46:0/387811333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2304108f10 msgr2=0x7f2304077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.369+0000 7f230a41a700 1 --2- 172.21.15.46:0/387811333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2304108f10 0x7f2304077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.369+0000 7f230a41a700 1 -- 172.21.15.46:0/387811333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23041045e0 msgr2=0x7f2304076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.369+0000 7f230a41a700 1 --2- 172.21.15.46:0/387811333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23041045e0 0x7f2304076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.369+0000 7f230a41a700 1 -- 172.21.15.46:0/387811333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23041a91a0 con 0x7f2304109860 2024-05-09T05:00:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.369+0000 7f230a41a700 1 --2- 172.21.15.46:0/387811333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2304109860 0x7f2304077650 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f22fc007ee0 tx=0x7f22fc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.369+0000 7f22faffd700 1 -- 172.21.15.46:0/387811333 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc00dbd0 con 0x7f2304109860 2024-05-09T05:00:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.369+0000 7f22faffd700 1 -- 172.21.15.46:0/387811333 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22fc00dd70 con 0x7f2304109860 2024-05-09T05:00:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.369+0000 7f230be7d700 1 -- 172.21.15.46:0/387811333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23041a9490 con 0x7f2304109860 2024-05-09T05:00:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.369+0000 7f22faffd700 1 -- 172.21.15.46:0/387811333 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc005990 con 0x7f2304109860 2024-05-09T05:00:09.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.369+0000 7f230be7d700 1 -- 172.21.15.46:0/387811333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23041a9950 con 0x7f2304109860 2024-05-09T05:00:09.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.372+0000 7f22faffd700 1 -- 172.21.15.46:0/387811333 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f22fc01a020 con 0x7f2304109860 2024-05-09T05:00:09.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.372+0000 7f22faffd700 1 --2- 172.21.15.46:0/387811333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22f00b29f0 0x7f22f00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.373+0000 7f2309c19700 1 --2- 172.21.15.46:0/387811333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22f00b29f0 0x7f22f00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.373+0000 7f22faffd700 1 -- 172.21.15.46:0/387811333 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f22fc0e1da0 con 0x7f2304109860 2024-05-09T05:00:09.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.373+0000 7f231a893b80 1 Processor -- start 2024-05-09T05:00:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.373+0000 7f231a893b80 1 -- start start 2024-05-09T05:00:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.373+0000 7f2309c19700 1 --2- 172.21.15.46:0/387811333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22f00b29f0 0x7f22f00b4f00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2300005950 tx=0x7f23000058e0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.373+0000 7f231a893b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44ebb7160 0x55c44ec504f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.373+0000 7f231a893b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44eacd4a0 0x55c44ec50a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.373+0000 7f231a893b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ec4a6a0 0x55c44ec4aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.373+0000 7f231a893b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c44eb6cff0 con 0x55c44ebb7160 2024-05-09T05:00:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.373+0000 7f231a893b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c44eb0b7f0 con 0x55c44ec4a6a0 2024-05-09T05:00:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.373+0000 7f231a893b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c44eb68fb0 con 0x55c44eacd4a0 2024-05-09T05:00:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.373+0000 7f2309c19700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44ebb7160 0x55c44ec504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.373+0000 7f2309c19700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44ebb7160 0x55c44ec504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47048/0 (socket says 172.21.15.46:47048) 2024-05-09T05:00:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.373+0000 7f2309c19700 1 -- 172.21.15.46:0/391015632 learned_addr learned my addr 172.21.15.46:0/391015632 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:09.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.374+0000 7f2309418700 1 --2- 172.21.15.46:0/391015632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44eacd4a0 0x55c44ec50a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.374+0000 7f230a41a700 1 --2- 172.21.15.46:0/391015632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ec4a6a0 0x55c44ec4aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.374+0000 7f2309c19700 1 -- 172.21.15.46:0/391015632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44eacd4a0 msgr2=0x55c44ec50a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:09.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.374+0000 7f2309c19700 1 --2- 172.21.15.46:0/391015632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44eacd4a0 0x55c44ec50a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.374+0000 7f2309c19700 1 -- 172.21.15.46:0/391015632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ec4a6a0 msgr2=0x55c44ec4aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:09.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.374+0000 7f2309c19700 1 --2- 172.21.15.46:0/391015632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ec4a6a0 0x55c44ec4aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.374+0000 7f2309c19700 1 -- 172.21.15.46:0/391015632 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c44eb03370 con 0x55c44ebb7160 2024-05-09T05:00:09.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.374+0000 7f230a41a700 1 --2- 172.21.15.46:0/391015632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ec4a6a0 0x55c44ec4aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:00:09.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.374+0000 7f2309c19700 1 --2- 172.21.15.46:0/391015632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44ebb7160 0x55c44ec504f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f230000fd70 tx=0x7f2300012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33386d24ae2b3a3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:09.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.374+0000 7f22effff700 1 -- 172.21.15.46:0/391015632 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300022070 con 0x55c44ebb7160 2024-05-09T05:00:09.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.374+0000 7f22effff700 1 -- 172.21.15.46:0/391015632 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f230001e450 con 0x55c44ebb7160 2024-05-09T05:00:09.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.374+0000 7f22effff700 1 -- 172.21.15.46:0/391015632 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f230001d610 con 0x55c44ebb7160 2024-05-09T05:00:09.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.375+0000 7f231a893b80 1 -- 172.21.15.46:0/391015632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44ebb7160 msgr2=0x55c44ec504f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:09.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.375+0000 7f231a893b80 1 --2- 172.21.15.46:0/391015632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44ebb7160 0x55c44ec504f0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f230000fd70 tx=0x7f2300012000 comp rx=0 tx=0).stop 2024-05-09T05:00:09.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.375+0000 7f231a893b80 1 -- 172.21.15.46:0/391015632 shutdown_connections 2024-05-09T05:00:09.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.375+0000 7f231a893b80 1 --2- 172.21.15.46:0/391015632 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ec4a6a0 0x55c44ec4aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.375+0000 7f231a893b80 1 --2- 172.21.15.46:0/391015632 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44ebb7160 0x55c44ec504f0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.375+0000 7f231a893b80 1 --2- 172.21.15.46:0/391015632 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44eacd4a0 0x55c44ec50a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.375+0000 7f231a893b80 1 -- 172.21.15.46:0/391015632 >> 172.21.15.46:0/391015632 conn(0x55c44eb2d6a0 msgr2=0x55c44ec45400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.375+0000 7f231a893b80 1 -- 172.21.15.46:0/391015632 shutdown_connections 2024-05-09T05:00:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.375+0000 7f231a893b80 1 -- 172.21.15.46:0/391015632 wait complete. 2024-05-09T05:00:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.376+0000 7f231a893b80 1 -- start start 2024-05-09T05:00:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.376+0000 7f231a893b80 1 Processor -- start 2024-05-09T05:00:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.376+0000 7f231a893b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44eacd4a0 0x55c44ec54210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.376+0000 7f231a893b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ebb7160 0x55c44ec54750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.376+0000 7f231a893b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44ec4a6a0 0x55c44ec54c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f231a893b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55c44e9e8ba0 con 0x55c44eacd4a0 2024-05-09T05:00:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f231a893b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55c44eb27150 con 0x55c44ebb7160 2024-05-09T05:00:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f231a893b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55c44ec571d0 con 0x55c44ec4a6a0 2024-05-09T05:00:09.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f231a893b80 10 client.0 _ll_register_callbacks cb 0x55c44e9de6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f2309c19700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44eacd4a0 0x55c44ec54210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f2309c19700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44eacd4a0 0x55c44ec54210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47050/0 (socket says 172.21.15.46:47050) 2024-05-09T05:00:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f2309c19700 1 -- 172.21.15.46:0/2195114913 learned_addr learned my addr 172.21.15.46:0/2195114913 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f230a41a700 1 --2- 172.21.15.46:0/2195114913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44ec4a6a0 0x55c44ec54c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f2309418700 1 --2- 172.21.15.46:0/2195114913 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ebb7160 0x55c44ec54750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f2309c19700 1 -- 172.21.15.46:0/2195114913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44ec4a6a0 msgr2=0x55c44ec54c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f2309c19700 1 --2- 172.21.15.46:0/2195114913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44ec4a6a0 0x55c44ec54c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f2309c19700 1 -- 172.21.15.46:0/2195114913 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ebb7160 msgr2=0x55c44ec54750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f2309c19700 1 --2- 172.21.15.46:0/2195114913 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ebb7160 0x55c44ec54750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f2309c19700 1 -- 172.21.15.46:0/2195114913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c44e9e8490 con 0x55c44eacd4a0 2024-05-09T05:00:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f2309418700 1 --2- 172.21.15.46:0/2195114913 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ebb7160 0x55c44ec54750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f2309c19700 1 --2- 172.21.15.46:0/2195114913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44eacd4a0 0x55c44ec54210 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f230000fd40 tx=0x7f2300011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0b306ce703f09c1 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f22ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:09.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f22ed7fa700 1 -- 172.21.15.46:0/2195114913 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300035060 con 0x55c44eacd4a0 2024-05-09T05:00:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f22ed7fa700 1 -- 172.21.15.46:0/2195114913 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f230001ee00 con 0x55c44eacd4a0 2024-05-09T05:00:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.377+0000 7f231a893b80 1 -- 172.21.15.46:0/2195114913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f230001e450 con 0x55c44eacd4a0 2024-05-09T05:00:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.378+0000 7f22ed7fa700 1 -- 172.21.15.46:0/2195114913 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2300022070 con 0x55c44eacd4a0 2024-05-09T05:00:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.378+0000 7f231a893b80 10 client.17229 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.378+0000 7f231a893b80 1 -- 172.21.15.46:0/2195114913 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c44ec61b50 con 0x55c44eacd4a0 2024-05-09T05:00:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.378+0000 7f22ed7fa700 1 -- 172.21.15.46:0/2195114913 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f230001e450 con 0x55c44eacd4a0 2024-05-09T05:00:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.378+0000 7f22ed7fa700 1 -- 172.21.15.46:0/2195114913 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2300004780 con 0x55c44eacd4a0 2024-05-09T05:00:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.378+0000 7f231a893b80 10 client.17229 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.378+0000 7f231a893b80 10 client.17229 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T05:00:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.378+0000 7f231a893b80 20 client.17229 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.378+0000 7f231a893b80 1 --2- 172.21.15.46:0/2195114913 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55c44ec64020 0x55c44ec66440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.378+0000 7f231a893b80 4 client.17229 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:00:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.378+0000 7f231a893b80 1 -- 172.21.15.46:0/2195114913 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c44eb7a9d0 con 0x55c44ec64020 2024-05-09T05:00:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.379+0000 7f2309418700 1 --2- 172.21.15.46:0/2195114913 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55c44ec64020 0x55c44ec66440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.379+0000 7f2309418700 1 --2- 172.21.15.46:0/2195114913 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55c44ec64020 0x55c44ec66440 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d870692e16545fcc server_cookie=8d3bcd166d413c4 in_seq=0 out_seq=0 2024-05-09T05:00:09.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.379+0000 7f22ed7fa700 10 client.17229 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T05:00:09.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.380+0000 7f22ed7fa700 1 -- 172.21.15.46:0/2195114913 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c44eb7a9d0 con 0x55c44ec64020 2024-05-09T05:00:09.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.380+0000 7f22ed7fa700 10 client.17229 handle_command_reply: tid=0 2024-05-09T05:00:09.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.380+0000 7f231a893b80 1 client.17229 shutdown 2024-05-09T05:00:09.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.380+0000 7f231a893b80 1 -- 172.21.15.46:0/2195114913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44eacd4a0 msgr2=0x55c44ec54210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:09.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.380+0000 7f231a893b80 1 --2- 172.21.15.46:0/2195114913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44eacd4a0 0x55c44ec54210 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f230000fd40 tx=0x7f2300011d60 comp rx=0 tx=0).stop 2024-05-09T05:00:09.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.380+0000 7f231a893b80 1 -- 172.21.15.46:0/2195114913 shutdown_connections 2024-05-09T05:00:09.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.380+0000 7f231a893b80 1 --2- 172.21.15.46:0/2195114913 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55c44ec64020 0x55c44ec66440 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.380+0000 7f231a893b80 1 --2- 172.21.15.46:0/2195114913 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44ec4a6a0 0x55c44ec54c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.380+0000 7f231a893b80 1 --2- 172.21.15.46:0/2195114913 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44eacd4a0 0x55c44ec54210 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.380+0000 7f231a893b80 1 --2- 172.21.15.46:0/2195114913 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ebb7160 0x55c44ec54750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.380+0000 7f22ed7fa700 0 client.17229 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T05:00:09.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.380+0000 7f231a893b80 1 -- 172.21.15.46:0/2195114913 >> 172.21.15.46:0/2195114913 conn(0x55c44eb2d6a0 msgr2=0x55c44ec451c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:09.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.380+0000 7f231a893b80 1 -- 172.21.15.46:0/2195114913 shutdown_connections 2024-05-09T05:00:09.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.380+0000 7f231a893b80 1 -- 172.21.15.46:0/2195114913 wait complete. 2024-05-09T05:00:09.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.380+0000 7f231a893b80 20 client.17229 trim_cache size 0 max 16384 2024-05-09T05:00:09.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f231a893b80 1 Processor -- start 2024-05-09T05:00:09.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f231a893b80 1 -- start start 2024-05-09T05:00:09.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f231a893b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44eacd4a0 0x55c44ec60310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f231a893b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44ebb7160 0x55c44ec49690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f231a893b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ec4a6a0 0x55c44ec49bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f231a893b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f23041b3a60 con 0x55c44eacd4a0 2024-05-09T05:00:09.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f231a893b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f23041b12f0 con 0x55c44ec4a6a0 2024-05-09T05:00:09.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f231a893b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f23041af160 con 0x55c44ebb7160 2024-05-09T05:00:09.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f2309c19700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44eacd4a0 0x55c44ec60310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f2309c19700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44eacd4a0 0x55c44ec60310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47056/0 (socket says 172.21.15.46:47056) 2024-05-09T05:00:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f2309c19700 1 -- 172.21.15.46:0/3347616480 learned_addr learned my addr 172.21.15.46:0/3347616480 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f230a41a700 1 --2- 172.21.15.46:0/3347616480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ec4a6a0 0x55c44ec49bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f2309418700 1 --2- 172.21.15.46:0/3347616480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44ebb7160 0x55c44ec49690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f2309c19700 1 -- 172.21.15.46:0/3347616480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44ebb7160 msgr2=0x55c44ec49690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f2309c19700 1 --2- 172.21.15.46:0/3347616480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44ebb7160 0x55c44ec49690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f2309c19700 1 -- 172.21.15.46:0/3347616480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ec4a6a0 msgr2=0x55c44ec49bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f2309c19700 1 --2- 172.21.15.46:0/3347616480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ec4a6a0 0x55c44ec49bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f2309c19700 1 -- 172.21.15.46:0/3347616480 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c44ec527a0 con 0x55c44eacd4a0 2024-05-09T05:00:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.396+0000 7f230a41a700 1 --2- 172.21.15.46:0/3347616480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ec4a6a0 0x55c44ec49bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.397+0000 7f2309c19700 1 --2- 172.21.15.46:0/3347616480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44eacd4a0 0x55c44ec60310 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f230003a040 tx=0x7f2300027450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b854c6d02a40f40a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.397+0000 7f22ef7fe700 1 -- 172.21.15.46:0/3347616480 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300035070 con 0x55c44eacd4a0 2024-05-09T05:00:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.397+0000 7f22ef7fe700 1 -- 172.21.15.46:0/3347616480 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f230000b440 con 0x55c44eacd4a0 2024-05-09T05:00:09.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.397+0000 7f22ef7fe700 1 -- 172.21.15.46:0/3347616480 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2300015040 con 0x55c44eacd4a0 2024-05-09T05:00:09.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.398+0000 7f231a893b80 1 -- 172.21.15.46:0/3347616480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44eacd4a0 msgr2=0x55c44ec60310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.398+0000 7f231a893b80 1 --2- 172.21.15.46:0/3347616480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44eacd4a0 0x55c44ec60310 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f230003a040 tx=0x7f2300027450 comp rx=0 tx=0).stop 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.398+0000 7f231a893b80 1 -- 172.21.15.46:0/3347616480 shutdown_connections 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.398+0000 7f231a893b80 1 --2- 172.21.15.46:0/3347616480 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ec4a6a0 0x55c44ec49bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.398+0000 7f231a893b80 1 --2- 172.21.15.46:0/3347616480 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44eacd4a0 0x55c44ec60310 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.398+0000 7f231a893b80 1 --2- 172.21.15.46:0/3347616480 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44ebb7160 0x55c44ec49690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.398+0000 7f231a893b80 1 -- 172.21.15.46:0/3347616480 >> 172.21.15.46:0/3347616480 conn(0x55c44eb2d6a0 msgr2=0x55c44ec52b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.398+0000 7f231a893b80 1 -- 172.21.15.46:0/3347616480 shutdown_connections 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.398+0000 7f231a893b80 1 -- 172.21.15.46:0/3347616480 wait complete. 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.398+0000 7f231a893b80 1 -- start start 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.398+0000 7f231a893b80 1 Processor -- start 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f231a893b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44eacd4a0 0x55c44ec66440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f231a893b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ebb7160 0x55c44ec66980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f231a893b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44ec4a6a0 0x55c44eccc560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f231a893b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f23041af7b0 con 0x55c44ec4a6a0 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f231a893b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f23041b0c20 con 0x55c44ebb7160 2024-05-09T05:00:09.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f231a893b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f23041afe80 con 0x55c44eacd4a0 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f231a893b80 10 client.0 _ll_register_callbacks cb 0x55c44e9de6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f230a41a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44ec4a6a0 0x55c44eccc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f2309c19700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44eacd4a0 0x55c44ec66440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f230a41a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44ec4a6a0 0x55c44eccc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47070/0 (socket says 172.21.15.46:47070) 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f230a41a700 1 -- 172.21.15.46:0/4151905627 learned_addr learned my addr 172.21.15.46:0/4151905627 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f2309c19700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44eacd4a0 0x55c44ec66440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:36864/0 (socket says 172.21.15.46:36864) 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f2309418700 1 --2- 172.21.15.46:0/4151905627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ebb7160 0x55c44ec66980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f230a41a700 1 -- 172.21.15.46:0/4151905627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44eacd4a0 msgr2=0x55c44ec66440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f230a41a700 1 --2- 172.21.15.46:0/4151905627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44eacd4a0 0x55c44ec66440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f230a41a700 1 -- 172.21.15.46:0/4151905627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ebb7160 msgr2=0x55c44ec66980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f230a41a700 1 --2- 172.21.15.46:0/4151905627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ebb7160 0x55c44ec66980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f230a41a700 1 -- 172.21.15.46:0/4151905627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c44ec67260 con 0x55c44ec4a6a0 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f230a41a700 1 --2- 172.21.15.46:0/4151905627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44ec4a6a0 0x55c44eccc560 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0a8d30 tx=0x7f22fc004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5612741952bffac2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f22ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.399+0000 7f22ed7fa700 1 -- 172.21.15.46:0/4151905627 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22fc0a9730 con 0x55c44ec4a6a0 2024-05-09T05:00:09.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.400+0000 7f231a893b80 1 -- 172.21.15.46:0/4151905627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f230000b440 con 0x55c44ec4a6a0 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.400+0000 7f22ed7fa700 1 -- 172.21.15.46:0/4151905627 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8600+0+0 (secure 0 0 0) 0x7f22fc0a9290 con 0x55c44ec4a6a0 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.400+0000 7f22ed7fa700 1 -- 172.21.15.46:0/4151905627 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f22fc0a9030 con 0x55c44ec4a6a0 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.400+0000 7f231a893b80 10 client.17235 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.400+0000 7f231a893b80 1 -- 172.21.15.46:0/4151905627 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55c44eccdbc0 con 0x55c44ec4a6a0 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.400+0000 7f22ed7fa700 1 -- 172.21.15.46:0/4151905627 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f230000b440 con 0x55c44ec4a6a0 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.400+0000 7f22ed7fa700 1 -- 172.21.15.46:0/4151905627 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f22fc0a8290 con 0x55c44ec4a6a0 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.400+0000 7f231a893b80 10 client.17235 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.400+0000 7f231a893b80 10 client.17235 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.400+0000 7f231a893b80 20 client.17235 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.400+0000 7f231a893b80 1 --2- 172.21.15.46:0/4151905627 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55c44ecce220 0x55c44ecd0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.400+0000 7f231a893b80 4 client.17235 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.400+0000 7f231a893b80 1 -- 172.21.15.46:0/4151905627 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c44eb74a70 con 0x55c44ecce220 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.400+0000 7f2309c19700 1 --2- 172.21.15.46:0/4151905627 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55c44ecce220 0x55c44ecd0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f22ed7fa700 10 client.17235 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f2309c19700 1 --2- 172.21.15.46:0/4151905627 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55c44ecce220 0x55c44ecd0640 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42e3b58a5bb96319 server_cookie=3f219e4173b4077d in_seq=0 out_seq=0 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f22ed7fa700 1 -- 172.21.15.46:0/4151905627 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c44eb74a70 con 0x55c44ecce220 2024-05-09T05:00:09.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f22ed7fa700 10 client.17235 handle_command_reply: tid=0 2024-05-09T05:00:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f231a893b80 1 client.17235 shutdown 2024-05-09T05:00:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f231a893b80 1 -- 172.21.15.46:0/4151905627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44ec4a6a0 msgr2=0x55c44eccc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f231a893b80 1 --2- 172.21.15.46:0/4151905627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44ec4a6a0 0x55c44eccc560 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f22fc0a8d30 tx=0x7f22fc004d50 comp rx=0 tx=0).stop 2024-05-09T05:00:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f231a893b80 1 -- 172.21.15.46:0/4151905627 shutdown_connections 2024-05-09T05:00:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f231a893b80 1 --2- 172.21.15.46:0/4151905627 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55c44ec4a6a0 0x55c44eccc560 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f231a893b80 1 --2- 172.21.15.46:0/4151905627 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55c44ecce220 0x55c44ecd0640 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f231a893b80 1 --2- 172.21.15.46:0/4151905627 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55c44eacd4a0 0x55c44ec66440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f231a893b80 1 --2- 172.21.15.46:0/4151905627 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55c44ebb7160 0x55c44ec66980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f22ed7fa700 0 client.17235 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T05:00:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f231a893b80 1 -- 172.21.15.46:0/4151905627 >> 172.21.15.46:0/4151905627 conn(0x55c44eb2d6a0 msgr2=0x55c44ec49690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f231a893b80 1 -- 172.21.15.46:0/4151905627 shutdown_connections 2024-05-09T05:00:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.401+0000 7f231a893b80 1 -- 172.21.15.46:0/4151905627 wait complete. 2024-05-09T05:00:09.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.402+0000 7f231a893b80 20 client.17235 trim_cache size 0 max 16384 2024-05-09T05:00:09.407 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T05:00:09.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.402+0000 7f230be7d700 1 -- 172.21.15.46:0/387811333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22f00b29f0 msgr2=0x7f22f00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:09.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.402+0000 7f230be7d700 1 --2- 172.21.15.46:0/387811333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22f00b29f0 0x7f22f00b4f00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2300005950 tx=0x7f23000058e0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.402+0000 7f230be7d700 1 -- 172.21.15.46:0/387811333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2304109860 msgr2=0x7f2304077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:09.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.402+0000 7f230be7d700 1 --2- 172.21.15.46:0/387811333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2304109860 0x7f2304077650 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f22fc007ee0 tx=0x7f22fc00d3b0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.402+0000 7f230be7d700 1 -- 172.21.15.46:0/387811333 shutdown_connections 2024-05-09T05:00:09.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.402+0000 7f230be7d700 1 --2- 172.21.15.46:0/387811333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22f00b29f0 0x7f22f00b4f00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.402+0000 7f230be7d700 1 --2- 172.21.15.46:0/387811333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2304109860 0x7f2304077650 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.402+0000 7f230be7d700 1 --2- 172.21.15.46:0/387811333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23041045e0 0x7f2304076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.402+0000 7f230be7d700 1 --2- 172.21.15.46:0/387811333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2304108f10 0x7f2304077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:09.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.402+0000 7f230be7d700 1 -- 172.21.15.46:0/387811333 >> 172.21.15.46:0/387811333 conn(0x7f23040ffed0 msgr2=0x7f230410a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:09.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.402+0000 7f230be7d700 1 -- 172.21.15.46:0/387811333 shutdown_connections 2024-05-09T05:00:09.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:09.402+0000 7f230be7d700 1 -- 172.21.15.46:0/387811333 wait complete. 2024-05-09T05:00:10.254 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T05:00:10.510 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:10.885 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:10 smithi046 ceph-mon[30136]: pgmap v279: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T05:00:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:10 smithi072 ceph-mon[34088]: pgmap v279: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T05:00:11.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:10 smithi110 ceph-mon[34904]: pgmap v279: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T05:00:12.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.457+0000 7f7440b5c700 1 -- 172.21.15.46:0/957017201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f743c06b820 msgr2=0x7f743c1102a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:12.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.457+0000 7f7440b5c700 1 --2- 172.21.15.46:0/957017201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f743c06b820 0x7f743c1102a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f743400f400 tx=0x7f743400f710 comp rx=0 tx=0).stop 2024-05-09T05:00:12.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.457+0000 7f7440b5c700 1 -- 172.21.15.46:0/957017201 shutdown_connections 2024-05-09T05:00:12.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.457+0000 7f7440b5c700 1 --2- 172.21.15.46:0/957017201 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f743c06b820 0x7f743c1102a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.458 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.457+0000 7f7440b5c700 1 --2- 172.21.15.46:0/957017201 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f743c10b020 0x7f743c10b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.457+0000 7f7440b5c700 1 --2- 172.21.15.46:0/957017201 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f743c10f950 0x7f743c10fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.457+0000 7f7440b5c700 1 -- 172.21.15.46:0/957017201 >> 172.21.15.46:0/957017201 conn(0x7f743c06f990 msgr2=0x7f743c06fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:12.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.458+0000 7f7440b5c700 1 -- 172.21.15.46:0/957017201 shutdown_connections 2024-05-09T05:00:12.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.458+0000 7f7440b5c700 1 -- 172.21.15.46:0/957017201 wait complete. 2024-05-09T05:00:12.459 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.459+0000 7f7440b5c700 1 Processor -- start 2024-05-09T05:00:12.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.460+0000 7f7440b5c700 1 -- start start 2024-05-09T05:00:12.460 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.460+0000 7f7440b5c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f743c06b820 0x7f743c1a1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.461+0000 7f7440b5c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f743c10b020 0x7f743c1a19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.461+0000 7f7440b5c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f743c10f950 0x7f743c1a5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.461 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.461+0000 7f7439d9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f743c10b020 0x7f743c1a19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.461+0000 7f7439d9b700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f743c10b020 0x7f743c1a19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:37204/0 (socket says 172.21.15.46:37204) 2024-05-09T05:00:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.461+0000 7f743a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f743c06b820 0x7f743c1a1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.461+0000 7f743a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f743c06b820 0x7f743c1a1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:36880/0 (socket says 172.21.15.46:36880) 2024-05-09T05:00:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.461+0000 7f7439d9b700 1 -- 172.21.15.46:0/2043847499 learned_addr learned my addr 172.21.15.46:0/2043847499 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.461+0000 7f7440b5c700 1 -- 172.21.15.46:0/2043847499 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f743c11ab20 con 0x7f743c10f950 2024-05-09T05:00:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.461+0000 7f7440b5c700 1 -- 172.21.15.46:0/2043847499 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f743c11a9a0 con 0x7f743c10b020 2024-05-09T05:00:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.461+0000 7f7440b5c700 1 -- 172.21.15.46:0/2043847499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f743c11aca0 con 0x7f743c06b820 2024-05-09T05:00:12.462 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.462+0000 7f743ad9d700 1 --2- 172.21.15.46:0/2043847499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f743c10f950 0x7f743c1a5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.462+0000 7f743a59c700 1 -- 172.21.15.46:0/2043847499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f743c10b020 msgr2=0x7f743c1a19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.462+0000 7f743a59c700 1 --2- 172.21.15.46:0/2043847499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f743c10b020 0x7f743c1a19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.462+0000 7f743a59c700 1 -- 172.21.15.46:0/2043847499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f743c10f950 msgr2=0x7f743c1a5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.462+0000 7f743a59c700 1 --2- 172.21.15.46:0/2043847499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f743c10f950 0x7f743c1a5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.462+0000 7f743a59c700 1 -- 172.21.15.46:0/2043847499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c1a6300 con 0x7f743c06b820 2024-05-09T05:00:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.462+0000 7f743ad9d700 1 --2- 172.21.15.46:0/2043847499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f743c10f950 0x7f743c1a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:00:12.463 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.462+0000 7f743a59c700 1 --2- 172.21.15.46:0/2043847499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f743c06b820 0x7f743c1a1430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f742c00d950 tx=0x7f742c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:12.464 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.464+0000 7f742b7fe700 1 -- 172.21.15.46:0/2043847499 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c016070 con 0x7f743c06b820 2024-05-09T05:00:12.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.464+0000 7f742b7fe700 1 -- 172.21.15.46:0/2043847499 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f742c00fd40 con 0x7f743c06b820 2024-05-09T05:00:12.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.464+0000 7f742b7fe700 1 -- 172.21.15.46:0/2043847499 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f742c00e3b0 con 0x7f743c06b820 2024-05-09T05:00:12.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.465+0000 7f7440b5c700 1 -- 172.21.15.46:0/2043847499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743c1a65f0 con 0x7f743c06b820 2024-05-09T05:00:12.466 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.465+0000 7f7440b5c700 1 -- 172.21.15.46:0/2043847499 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743c1adf10 con 0x7f743c06b820 2024-05-09T05:00:12.472 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.472+0000 7f742b7fe700 1 -- 172.21.15.46:0/2043847499 <== mon.1 v2:172.21.15.110:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f742c019070 con 0x7f743c06b820 2024-05-09T05:00:12.474 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.474+0000 7f742b7fe700 1 --2- 172.21.15.46:0/2043847499 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f74240b2c00 0x7f74240b5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.475+0000 7f742b7fe700 1 -- 172.21.15.46:0/2043847499 <== mon.1 v2:172.21.15.110:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8644+0+0 (secure 0 0 0) 0x7f742c0e2000 con 0x7f743c06b820 2024-05-09T05:00:12.475 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.475+0000 7f7439d9b700 1 --2- 172.21.15.46:0/2043847499 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f74240b2c00 0x7f74240b5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:12.476 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.476+0000 7f7439d9b700 1 --2- 172.21.15.46:0/2043847499 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f74240b2c00 0x7f74240b5110 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f743c10cdb0 tx=0x7f743000b480 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:12.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.477+0000 7f744f572b80 1 Processor -- start 2024-05-09T05:00:12.477 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.478+0000 7f744f572b80 1 -- start start 2024-05-09T05:00:12.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.478+0000 7f744f572b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bc6ed00 0x55e77bd7e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.478+0000 7f744f572b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bbf44a0 0x55e77bd7ee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.478+0000 7f744f572b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 0x55e77bd71810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.478+0000 7f744f572b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e77bc940a0 con 0x55e77bbf44a0 2024-05-09T05:00:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.478+0000 7f744f572b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e77bc399a0 con 0x55e77bd71400 2024-05-09T05:00:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.478+0000 7f744f572b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e77bc90060 con 0x55e77bc6ed00 2024-05-09T05:00:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.478+0000 7f7439d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bbf44a0 0x55e77bd7ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.479+0000 7f7439d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bbf44a0 0x55e77bd7ee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47098/0 (socket says 172.21.15.46:47098) 2024-05-09T05:00:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.479+0000 7f7439d9b700 1 -- 172.21.15.46:0/1869065847 learned_addr learned my addr 172.21.15.46:0/1869065847 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:12.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.479+0000 7f743a59c700 1 --2- 172.21.15.46:0/1869065847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bc6ed00 0x55e77bd7e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.480+0000 7f7439d9b700 1 -- 172.21.15.46:0/1869065847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bc6ed00 msgr2=0x55e77bd7e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.480+0000 7f7439d9b700 1 --2- 172.21.15.46:0/1869065847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bc6ed00 0x55e77bd7e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.480+0000 7f7439d9b700 1 -- 172.21.15.46:0/1869065847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 msgr2=0x55e77bd71810 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:00:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.480+0000 7f7439d9b700 1 --2- 172.21.15.46:0/1869065847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 0x55e77bd71810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.480+0000 7f7439d9b700 1 -- 172.21.15.46:0/1869065847 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e77bc2a500 con 0x55e77bbf44a0 2024-05-09T05:00:12.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.481+0000 7f7439d9b700 1 --2- 172.21.15.46:0/1869065847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bbf44a0 0x55e77bd7ee00 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7430003b10 tx=0x7f7430013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7024c15939bc7a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.482+0000 7f7428ff9700 1 -- 172.21.15.46:0/1869065847 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430011ad0 con 0x55e77bbf44a0 2024-05-09T05:00:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.482+0000 7f7428ff9700 1 -- 172.21.15.46:0/1869065847 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7430011c70 con 0x55e77bbf44a0 2024-05-09T05:00:12.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.483+0000 7f7428ff9700 1 -- 172.21.15.46:0/1869065847 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743001c610 con 0x55e77bbf44a0 2024-05-09T05:00:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.483+0000 7f744f572b80 1 -- 172.21.15.46:0/1869065847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bbf44a0 msgr2=0x55e77bd7ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:12.483 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.483+0000 7f744f572b80 1 --2- 172.21.15.46:0/1869065847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bbf44a0 0x55e77bd7ee00 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f7430003b10 tx=0x7f7430013000 comp rx=0 tx=0).stop 2024-05-09T05:00:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.484+0000 7f744f572b80 1 -- 172.21.15.46:0/1869065847 shutdown_connections 2024-05-09T05:00:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.484+0000 7f744f572b80 1 --2- 172.21.15.46:0/1869065847 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 0x55e77bd71810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.484+0000 7f744f572b80 1 --2- 172.21.15.46:0/1869065847 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bc6ed00 0x55e77bd7e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.484+0000 7f744f572b80 1 --2- 172.21.15.46:0/1869065847 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bbf44a0 0x55e77bd7ee00 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.484+0000 7f744f572b80 1 -- 172.21.15.46:0/1869065847 >> 172.21.15.46:0/1869065847 conn(0x55e77bbf7f70 msgr2=0x55e77bd6c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.484+0000 7f744f572b80 1 -- 172.21.15.46:0/1869065847 shutdown_connections 2024-05-09T05:00:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.484+0000 7f744f572b80 1 -- 172.21.15.46:0/1869065847 wait complete. 2024-05-09T05:00:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.485+0000 7f744f572b80 1 -- start start 2024-05-09T05:00:12.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.485+0000 7f744f572b80 1 Processor -- start 2024-05-09T05:00:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.486+0000 7f744f572b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bbf44a0 0x55e77bd83780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.486+0000 7f744f572b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bc6ed00 0x55e77bd83cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.486+0000 7f744f572b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bd71400 0x55e77bd84200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.486+0000 7f744f572b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55e77bd721c0 con 0x55e77bd71400 2024-05-09T05:00:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.486+0000 7f744f572b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55e77bc2eb20 con 0x55e77bbf44a0 2024-05-09T05:00:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.486+0000 7f744f572b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55e77bd86870 con 0x55e77bc6ed00 2024-05-09T05:00:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.486+0000 7f744f572b80 10 client.0 _ll_register_callbacks cb 0x55e77bb056c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.486+0000 7f743ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bd71400 0x55e77bd84200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.486+0000 7f743ad9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bd71400 0x55e77bd84200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47112/0 (socket says 172.21.15.46:47112) 2024-05-09T05:00:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.486+0000 7f743ad9d700 1 -- 172.21.15.46:0/2704437213 learned_addr learned my addr 172.21.15.46:0/2704437213 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:12.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.486+0000 7f743a59c700 1 --2- 172.21.15.46:0/2704437213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bbf44a0 0x55e77bd83780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.487+0000 7f743ad9d700 1 -- 172.21.15.46:0/2704437213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bc6ed00 msgr2=0x55e77bd83cc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:00:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.487+0000 7f743ad9d700 1 --2- 172.21.15.46:0/2704437213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bc6ed00 0x55e77bd83cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.488 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.487+0000 7f743ad9d700 1 -- 172.21.15.46:0/2704437213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bbf44a0 msgr2=0x55e77bd83780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.487+0000 7f743ad9d700 1 --2- 172.21.15.46:0/2704437213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bbf44a0 0x55e77bd83780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.487+0000 7f743ad9d700 1 -- 172.21.15.46:0/2704437213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e77bb0f490 con 0x55e77bd71400 2024-05-09T05:00:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.487+0000 7f743ad9d700 1 --2- 172.21.15.46:0/2704437213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bd71400 0x55e77bd84200 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f743400d7e0 tx=0x7f743400d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24c81f3089cf7e8c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.487+0000 7f7421ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.488+0000 7f7421ffb700 1 -- 172.21.15.46:0/2704437213 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743401b040 con 0x55e77bd71400 2024-05-09T05:00:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.488+0000 7f744f572b80 1 -- 172.21.15.46:0/2704437213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7430011c70 con 0x55e77bd71400 2024-05-09T05:00:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.488+0000 7f7421ffb700 1 -- 172.21.15.46:0/2704437213 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8644+0+0 (secure 0 0 0) 0x7f7434004c50 con 0x55e77bd71400 2024-05-09T05:00:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.488+0000 7f7421ffb700 1 -- 172.21.15.46:0/2704437213 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7434004e50 con 0x55e77bd71400 2024-05-09T05:00:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.488+0000 7f744f572b80 10 client.17247 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:12.489 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.488+0000 7f744f572b80 1 -- 172.21.15.46:0/2704437213 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e77bd911e0 con 0x55e77bd71400 2024-05-09T05:00:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.489+0000 7f7421ffb700 1 -- 172.21.15.46:0/2704437213 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7430011c70 con 0x55e77bd71400 2024-05-09T05:00:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.489+0000 7f7421ffb700 1 -- 172.21.15.46:0/2704437213 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f7434015030 con 0x55e77bd71400 2024-05-09T05:00:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.489+0000 7f744f572b80 10 client.17247 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.489+0000 7f744f572b80 10 client.17247 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T05:00:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.489+0000 7f744f572b80 20 client.17247 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.489+0000 7f744f572b80 1 --2- 172.21.15.46:0/2704437213 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e77bd93960 0x55e77bd95e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.489+0000 7f744f572b80 4 client.17247 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:00:12.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.489+0000 7f744f572b80 1 -- 172.21.15.46:0/2704437213 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e77bca1a80 con 0x55e77bd93960 2024-05-09T05:00:12.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.490+0000 7f743a59c700 1 --2- 172.21.15.46:0/2704437213 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e77bd93960 0x55e77bd95e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:12.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.492+0000 7f743a59c700 1 --2- 172.21.15.46:0/2704437213 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e77bd93960 0x55e77bd95e70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ae5942059298a61e server_cookie=39a8eb3f1b363d4d in_seq=0 out_seq=0 2024-05-09T05:00:12.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.492+0000 7f7421ffb700 10 client.17247 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T05:00:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.494+0000 7f7421ffb700 1 -- 172.21.15.46:0/2704437213 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e77bca1a80 con 0x55e77bd93960 2024-05-09T05:00:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.494+0000 7f7421ffb700 10 client.17247 handle_command_reply: tid=0 2024-05-09T05:00:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.494+0000 7f744f572b80 1 client.17247 shutdown 2024-05-09T05:00:12.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.494+0000 7f744f572b80 1 -- 172.21.15.46:0/2704437213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bd71400 msgr2=0x55e77bd84200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:12.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.494+0000 7f744f572b80 1 --2- 172.21.15.46:0/2704437213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bd71400 0x55e77bd84200 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f743400d7e0 tx=0x7f743400d8c0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.496 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.496+0000 7f744f572b80 1 -- 172.21.15.46:0/2704437213 shutdown_connections 2024-05-09T05:00:12.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.496+0000 7f744f572b80 1 --2- 172.21.15.46:0/2704437213 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e77bd93960 0x55e77bd95e70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.496+0000 7f744f572b80 1 --2- 172.21.15.46:0/2704437213 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bd71400 0x55e77bd84200 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.496+0000 7f744f572b80 1 --2- 172.21.15.46:0/2704437213 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bbf44a0 0x55e77bd83780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.496+0000 7f744f572b80 1 --2- 172.21.15.46:0/2704437213 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bc6ed00 0x55e77bd83cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.496+0000 7f744f572b80 1 -- 172.21.15.46:0/2704437213 >> 172.21.15.46:0/2704437213 conn(0x55e77bbf7f70 msgr2=0x55e77bd703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:12.497 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.497+0000 7f7421ffb700 0 client.17247 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T05:00:12.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.498+0000 7f744f572b80 1 -- 172.21.15.46:0/2704437213 shutdown_connections 2024-05-09T05:00:12.498 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.498+0000 7f744f572b80 1 -- 172.21.15.46:0/2704437213 wait complete. 2024-05-09T05:00:12.499 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.499+0000 7f744f572b80 20 client.17247 trim_cache size 0 max 16384 2024-05-09T05:00:12.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.530+0000 7f744f572b80 1 Processor -- start 2024-05-09T05:00:12.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.531+0000 7f744f572b80 1 -- start start 2024-05-09T05:00:12.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.532+0000 7f744f572b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bbf44a0 0x55e77bd84040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.532+0000 7f744f572b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bc6ed00 0x55e77bd66700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.533+0000 7f744f572b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 0x55e77bd8f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.533+0000 7f743a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bbf44a0 0x55e77bd84040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.533+0000 7f743a59c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bbf44a0 0x55e77bd84040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:36918/0 (socket says 172.21.15.46:36918) 2024-05-09T05:00:12.533 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.533+0000 7f743a59c700 1 -- 172.21.15.46:0/2137402500 learned_addr learned my addr 172.21.15.46:0/2137402500 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.533+0000 7f7439d9b700 1 --2- 172.21.15.46:0/2137402500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bc6ed00 0x55e77bd66700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.533+0000 7f743ad9d700 1 --2- 172.21.15.46:0/2137402500 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 0x55e77bd8f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.533+0000 7f744f572b80 1 -- 172.21.15.46:0/2137402500 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1b8080 con 0x55e77bc6ed00 2024-05-09T05:00:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.533+0000 7f744f572b80 1 -- 172.21.15.46:0/2137402500 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1b5910 con 0x55e77bd71400 2024-05-09T05:00:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.533+0000 7f744f572b80 1 -- 172.21.15.46:0/2137402500 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1b3780 con 0x55e77bbf44a0 2024-05-09T05:00:12.534 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.534+0000 7f743ad9d700 1 -- 172.21.15.46:0/2137402500 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bbf44a0 msgr2=0x55e77bd84040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.534+0000 7f743ad9d700 1 --2- 172.21.15.46:0/2137402500 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bbf44a0 0x55e77bd84040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.534+0000 7f743ad9d700 1 -- 172.21.15.46:0/2137402500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bc6ed00 msgr2=0x55e77bd66700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.534+0000 7f743ad9d700 1 --2- 172.21.15.46:0/2137402500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bc6ed00 0x55e77bd66700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.534+0000 7f743ad9d700 1 -- 172.21.15.46:0/2137402500 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e77bd72250 con 0x55e77bd71400 2024-05-09T05:00:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.534+0000 7f743ad9d700 1 --2- 172.21.15.46:0/2137402500 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 0x55e77bd8f740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f743400dc20 tx=0x7f743400bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d41f44b7d1376f43 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.535+0000 7f7423fff700 1 -- 172.21.15.46:0/2137402500 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434005c60 con 0x55e77bd71400 2024-05-09T05:00:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.535+0000 7f7423fff700 1 -- 172.21.15.46:0/2137402500 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7434005e00 con 0x55e77bd71400 2024-05-09T05:00:12.535 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.535+0000 7f7423fff700 1 -- 172.21.15.46:0/2137402500 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434020a90 con 0x55e77bd71400 2024-05-09T05:00:12.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.537+0000 7f744f572b80 1 -- 172.21.15.46:0/2137402500 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 msgr2=0x55e77bd8f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:12.537 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.537+0000 7f744f572b80 1 --2- 172.21.15.46:0/2137402500 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 0x55e77bd8f740 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f743400dc20 tx=0x7f743400bf90 comp rx=0 tx=0).stop 2024-05-09T05:00:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.538+0000 7f744f572b80 1 -- 172.21.15.46:0/2137402500 shutdown_connections 2024-05-09T05:00:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.538+0000 7f744f572b80 1 --2- 172.21.15.46:0/2137402500 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 0x55e77bd8f740 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.538+0000 7f744f572b80 1 --2- 172.21.15.46:0/2137402500 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bbf44a0 0x55e77bd84040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.538 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.538+0000 7f744f572b80 1 --2- 172.21.15.46:0/2137402500 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bc6ed00 0x55e77bd66700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.539 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.538+0000 7f744f572b80 1 -- 172.21.15.46:0/2137402500 >> 172.21.15.46:0/2137402500 conn(0x55e77bbf7f70 msgr2=0x55e77bd65ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:12.540 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.540+0000 7f744f572b80 1 -- 172.21.15.46:0/2137402500 shutdown_connections 2024-05-09T05:00:12.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.540+0000 7f744f572b80 1 -- 172.21.15.46:0/2137402500 wait complete. 2024-05-09T05:00:12.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.541+0000 7f744f572b80 1 -- start start 2024-05-09T05:00:12.541 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.541+0000 7f744f572b80 1 Processor -- start 2024-05-09T05:00:12.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.543+0000 7f744f572b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bbf44a0 0x55e77bd8bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.543+0000 7f744f572b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bc6ed00 0x55e77bd800c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.543+0000 7f744f572b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 0x55e77bd80600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.544+0000 7f744f572b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1b3dd0 con 0x55e77bc6ed00 2024-05-09T05:00:12.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.544+0000 7f744f572b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1b5240 con 0x55e77bd71400 2024-05-09T05:00:12.544 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.544+0000 7f744f572b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1b44a0 con 0x55e77bbf44a0 2024-05-09T05:00:12.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.544+0000 7f743ad9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 0x55e77bd80600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:12.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.544+0000 7f744f572b80 10 client.0 _ll_register_callbacks cb 0x55e77bb056c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:12.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.544+0000 7f743ad9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 0x55e77bd80600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:37234/0 (socket says 172.21.15.46:37234) 2024-05-09T05:00:12.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.544+0000 7f743ad9d700 1 -- 172.21.15.46:0/299225171 learned_addr learned my addr 172.21.15.46:0/299225171 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:12.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.545+0000 7f7439d9b700 1 --2- 172.21.15.46:0/299225171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bc6ed00 0x55e77bd800c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:12.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.545+0000 7f743ad9d700 1 -- 172.21.15.46:0/299225171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bbf44a0 msgr2=0x55e77bd8bcc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:00:12.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.545+0000 7f743ad9d700 1 --2- 172.21.15.46:0/299225171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bbf44a0 0x55e77bd8bcc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.545+0000 7f743ad9d700 1 -- 172.21.15.46:0/299225171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bc6ed00 msgr2=0x55e77bd800c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:12.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.545+0000 7f743ad9d700 1 --2- 172.21.15.46:0/299225171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bc6ed00 0x55e77bd800c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.545+0000 7f743ad9d700 1 -- 172.21.15.46:0/299225171 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e77bd80b40 con 0x55e77bd71400 2024-05-09T05:00:12.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.545+0000 7f7439d9b700 1 --2- 172.21.15.46:0/299225171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bc6ed00 0x55e77bd800c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:12.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.546+0000 7f743ad9d700 1 --2- 172.21.15.46:0/299225171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 0x55e77bd80600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f743400f210 tx=0x7f743400b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8fd6d8b50501a268 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:12.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.546+0000 7f7421ffb700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T05:00:12.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.547+0000 7f7421ffb700 1 -- 172.21.15.46:0/299225171 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7434035060 con 0x55e77bd71400 2024-05-09T05:00:12.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.547+0000 7f7421ffb700 1 -- 172.21.15.46:0/299225171 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8644+0+0 (secure 0 0 0) 0x7f7434020800 con 0x55e77bd71400 2024-05-09T05:00:12.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.547+0000 7f744f572b80 1 -- 172.21.15.46:0/299225171 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7434005e00 con 0x55e77bd71400 2024-05-09T05:00:12.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.547+0000 7f7421ffb700 1 -- 172.21.15.46:0/299225171 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f7434005c60 con 0x55e77bd71400 2024-05-09T05:00:12.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.548+0000 7f744f572b80 10 client.26765 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:12.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.548+0000 7f744f572b80 1 -- 172.21.15.46:0/299225171 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e77bdfdd20 con 0x55e77bd71400 2024-05-09T05:00:12.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.549+0000 7f7421ffb700 1 -- 172.21.15.46:0/299225171 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7434005e00 con 0x55e77bd71400 2024-05-09T05:00:12.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.549+0000 7f7421ffb700 1 -- 172.21.15.46:0/299225171 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f743402f070 con 0x55e77bd71400 2024-05-09T05:00:12.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.551+0000 7f744f572b80 10 client.26765 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:12.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.551+0000 7f744f572b80 10 client.26765 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T05:00:12.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.551+0000 7f744f572b80 20 client.26765 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:12.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.551+0000 7f744f572b80 1 --2- 172.21.15.46:0/299225171 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e77bdfe000 0x55e77be00450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:12.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.551+0000 7f744f572b80 4 client.26765 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:00:12.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.551+0000 7f744f572b80 1 -- 172.21.15.46:0/299225171 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e77bc9bb20 con 0x55e77bdfe000 2024-05-09T05:00:12.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.553+0000 7f743a59c700 1 --2- 172.21.15.46:0/299225171 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e77bdfe000 0x55e77be00450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:12.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.555+0000 7f743a59c700 1 --2- 172.21.15.46:0/299225171 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e77bdfe000 0x55e77be00450 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=de3cf681efc73c07 server_cookie=24ebad7945e5b925 in_seq=0 out_seq=0 2024-05-09T05:00:12.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.555+0000 7f7421ffb700 10 client.26765 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T05:00:12.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.556+0000 7f7421ffb700 1 -- 172.21.15.46:0/299225171 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e77bc9bb20 con 0x55e77bdfe000 2024-05-09T05:00:12.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.556+0000 7f7421ffb700 10 client.26765 handle_command_reply: tid=0 2024-05-09T05:00:12.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.557+0000 7f744f572b80 1 client.26765 shutdown 2024-05-09T05:00:12.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.559+0000 7f744f572b80 1 -- 172.21.15.46:0/299225171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 msgr2=0x55e77bd80600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:12.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.559+0000 7f744f572b80 1 --2- 172.21.15.46:0/299225171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 0x55e77bd80600 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f743400f210 tx=0x7f743400b460 comp rx=0 tx=0).stop 2024-05-09T05:00:12.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.559+0000 7f744f572b80 1 -- 172.21.15.46:0/299225171 shutdown_connections 2024-05-09T05:00:12.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.559+0000 7f744f572b80 1 --2- 172.21.15.46:0/299225171 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55e77bdfe000 0x55e77be00450 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.559+0000 7f744f572b80 1 --2- 172.21.15.46:0/299225171 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55e77bd71400 0x55e77bd80600 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.559+0000 7f744f572b80 1 --2- 172.21.15.46:0/299225171 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55e77bbf44a0 0x55e77bd8bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.559+0000 7f744f572b80 1 --2- 172.21.15.46:0/299225171 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55e77bc6ed00 0x55e77bd800c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.559+0000 7f744f572b80 1 -- 172.21.15.46:0/299225171 >> 172.21.15.46:0/299225171 conn(0x55e77bbf7f70 msgr2=0x55e77bd703f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:12.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.559+0000 7f7421ffb700 0 client.26765 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T05:00:12.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.560+0000 7f744f572b80 1 -- 172.21.15.46:0/299225171 shutdown_connections 2024-05-09T05:00:12.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.560+0000 7f744f572b80 1 -- 172.21.15.46:0/299225171 wait complete. 2024-05-09T05:00:12.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.560+0000 7f744f572b80 20 client.26765 trim_cache size 0 max 16384 2024-05-09T05:00:12.561 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T05:00:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.561+0000 7f7440b5c700 1 -- 172.21.15.46:0/2043847499 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f74240b2c00 msgr2=0x7f74240b5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:12.561 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.561+0000 7f7440b5c700 1 --2- 172.21.15.46:0/2043847499 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f74240b2c00 0x7f74240b5110 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f743c10cdb0 tx=0x7f743000b480 comp rx=0 tx=0).stop 2024-05-09T05:00:12.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.561+0000 7f7440b5c700 1 -- 172.21.15.46:0/2043847499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f743c06b820 msgr2=0x7f743c1a1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:12.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.561+0000 7f7440b5c700 1 --2- 172.21.15.46:0/2043847499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f743c06b820 0x7f743c1a1430 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f742c00d950 tx=0x7f742c00dc60 comp rx=0 tx=0).stop 2024-05-09T05:00:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.562+0000 7f7440b5c700 1 -- 172.21.15.46:0/2043847499 shutdown_connections 2024-05-09T05:00:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.562+0000 7f7440b5c700 1 --2- 172.21.15.46:0/2043847499 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f74240b2c00 0x7f74240b5110 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.562+0000 7f7440b5c700 1 --2- 172.21.15.46:0/2043847499 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f743c10f950 0x7f743c1a5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.562+0000 7f7440b5c700 1 --2- 172.21.15.46:0/2043847499 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f743c06b820 0x7f743c1a1430 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.562+0000 7f7440b5c700 1 --2- 172.21.15.46:0/2043847499 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f743c10b020 0x7f743c1a19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.562+0000 7f7440b5c700 1 -- 172.21.15.46:0/2043847499 >> 172.21.15.46:0/2043847499 conn(0x7f743c06f990 msgr2=0x7f743c116730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.563+0000 7f7440b5c700 1 -- 172.21.15.46:0/2043847499 shutdown_connections 2024-05-09T05:00:12.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:12.563+0000 7f7440b5c700 1 -- 172.21.15.46:0/2043847499 wait complete. 2024-05-09T05:00:13.108 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:12 smithi110 ceph-mon[34904]: pgmap v280: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-05-09T05:00:13.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:12 smithi110 ceph-mon[34904]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T05:00:13.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:12 smithi072 ceph-mon[34088]: pgmap v280: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-05-09T05:00:13.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:12 smithi072 ceph-mon[34088]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T05:00:13.305 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:12 smithi046 ceph-mon[30136]: pgmap v280: 97 pgs: 97 active+clean; 10 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 24 op/s 2024-05-09T05:00:13.305 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:12 smithi046 ceph-mon[30136]: osdmap e90: 12 total, 12 up, 12 in 2024-05-09T05:00:13.331 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T05:00:13.580 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:15.115 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:14 smithi046 ceph-mon[30136]: pgmap v282: 97 pgs: 4 active+clean+snaptrim, 82 active+clean, 11 active+clean+snaptrim_wait; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:00:15.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:14 smithi072 ceph-mon[34088]: pgmap v282: 97 pgs: 4 active+clean+snaptrim, 82 active+clean, 11 active+clean+snaptrim_wait; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:00:15.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:14 smithi110 ceph-mon[34904]: pgmap v282: 97 pgs: 4 active+clean+snaptrim, 82 active+clean, 11 active+clean+snaptrim_wait; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:00:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.477+0000 7f98020fa700 1 -- 172.21.15.46:0/1647662524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc108f30 msgr2=0x7f97fc109340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.477+0000 7f98020fa700 1 --2- 172.21.15.46:0/1647662524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc108f30 0x7f97fc109340 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f97e40099c0 tx=0x7f97e4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.477+0000 7f98020fa700 1 -- 172.21.15.46:0/1647662524 shutdown_connections 2024-05-09T05:00:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.477+0000 7f98020fa700 1 --2- 172.21.15.46:0/1647662524 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97fc109880 0x7f97fc1111e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.478 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.477+0000 7f98020fa700 1 --2- 172.21.15.46:0/1647662524 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97fc104650 0x7f97fc104a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.477+0000 7f98020fa700 1 --2- 172.21.15.46:0/1647662524 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc108f30 0x7f97fc109340 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.477+0000 7f98020fa700 1 -- 172.21.15.46:0/1647662524 >> 172.21.15.46:0/1647662524 conn(0x7f97fc079f00 msgr2=0x7f97fc07a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.477+0000 7f98020fa700 1 -- 172.21.15.46:0/1647662524 shutdown_connections 2024-05-09T05:00:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.477+0000 7f98020fa700 1 -- 172.21.15.46:0/1647662524 wait complete. 2024-05-09T05:00:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.478+0000 7f98020fa700 1 Processor -- start 2024-05-09T05:00:15.479 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.478+0000 7f98020fa700 1 -- start start 2024-05-09T05:00:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f98020fa700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc104650 0x7f97fc19ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.480 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f98020fa700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97fc108f30 0x7f97fc19b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f98020fa700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97fc109880 0x7f97fc19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f98020fa700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1144b0 con 0x7f97fc104650 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f98020fa700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc114330 con 0x7f97fc109880 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f98020fa700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc114630 con 0x7f97fc108f30 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f97fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc104650 0x7f97fc19ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f97fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc104650 0x7f97fc19ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43328/0 (socket says 172.21.15.46:43328) 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f97fb7fe700 1 -- 172.21.15.46:0/3253982689 learned_addr learned my addr 172.21.15.46:0/3253982689 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f97f2ffd700 1 --2- 172.21.15.46:0/3253982689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97fc108f30 0x7f97fc19b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f97fbfff700 1 --2- 172.21.15.46:0/3253982689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97fc109880 0x7f97fc19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f97fb7fe700 1 -- 172.21.15.46:0/3253982689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97fc108f30 msgr2=0x7f97fc19b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f97fb7fe700 1 --2- 172.21.15.46:0/3253982689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97fc108f30 0x7f97fc19b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f97fb7fe700 1 -- 172.21.15.46:0/3253982689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97fc109880 msgr2=0x7f97fc19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f97fb7fe700 1 --2- 172.21.15.46:0/3253982689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97fc109880 0x7f97fc19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f97fb7fe700 1 -- 172.21.15.46:0/3253982689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97fc19fd00 con 0x7f97fc104650 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f97fb7fe700 1 --2- 172.21.15.46:0/3253982689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc104650 0x7f97fc19ae80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00d980 tx=0x7f97ec00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:15.481 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.479+0000 7f97f97fa700 1 -- 172.21.15.46:0/3253982689 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec017070 con 0x7f97fc104650 2024-05-09T05:00:15.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.480+0000 7f97f97fa700 1 -- 172.21.15.46:0/3253982689 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97ec004830 con 0x7f97fc104650 2024-05-09T05:00:15.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.480+0000 7f98020fa700 1 -- 172.21.15.46:0/3253982689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97fc19fff0 con 0x7f97fc104650 2024-05-09T05:00:15.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.480+0000 7f97f97fa700 1 -- 172.21.15.46:0/3253982689 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec005440 con 0x7f97fc104650 2024-05-09T05:00:15.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.480+0000 7f98020fa700 1 -- 172.21.15.46:0/3253982689 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97fc1a7910 con 0x7f97fc104650 2024-05-09T05:00:15.482 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.482+0000 7f97f97fa700 1 -- 172.21.15.46:0/3253982689 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f97ec0055e0 con 0x7f97fc104650 2024-05-09T05:00:15.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.483+0000 7f97f97fa700 1 --2- 172.21.15.46:0/3253982689 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f97e80b29f0 0x7f97e80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f97f2ffd700 1 --2- 172.21.15.46:0/3253982689 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f97e80b29f0 0x7f97e80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f97f97fa700 1 -- 172.21.15.46:0/3253982689 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8644+0+0 (secure 0 0 0) 0x7f97ec0e2650 con 0x7f97fc104650 2024-05-09T05:00:15.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f9810b10b80 1 Processor -- start 2024-05-09T05:00:15.484 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f9810b10b80 1 -- start start 2024-05-09T05:00:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f97f2ffd700 1 --2- 172.21.15.46:0/3253982689 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f97e80b29f0 0x7f97e80b4f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f97e4000c00 tx=0x7f97e401a040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:15.485 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f9810b10b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ad05d00 0x56323ae0e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f9810b10b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ac8b4a0 0x56323ae0e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f9810b10b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ae08400 0x56323ae08810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f9810b10b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56323ad2b0a0 con 0x56323ae08400 2024-05-09T05:00:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f9810b10b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56323acd09a0 con 0x56323ad05d00 2024-05-09T05:00:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f9810b10b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56323ad27060 con 0x56323ac8b4a0 2024-05-09T05:00:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f97fbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ae08400 0x56323ae08810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f97f2ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ac8b4a0 0x56323ae0e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f97fbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ae08400 0x56323ae08810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43342/0 (socket says 172.21.15.46:43342) 2024-05-09T05:00:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f97f2ffd700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ac8b4a0 0x56323ae0e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:34604/0 (socket says 172.21.15.46:34604) 2024-05-09T05:00:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.484+0000 7f97fbfff700 1 -- 172.21.15.46:0/799516691 learned_addr learned my addr 172.21.15.46:0/799516691 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.485+0000 7f97fb7fe700 1 --2- 172.21.15.46:0/799516691 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ad05d00 0x56323ae0e240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.485+0000 7f97fbfff700 1 -- 172.21.15.46:0/799516691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ac8b4a0 msgr2=0x56323ae0e780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.485+0000 7f97fbfff700 1 --2- 172.21.15.46:0/799516691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ac8b4a0 0x56323ae0e780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.486 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.485+0000 7f97fbfff700 1 -- 172.21.15.46:0/799516691 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ad05d00 msgr2=0x56323ae0e240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.485+0000 7f97fbfff700 1 --2- 172.21.15.46:0/799516691 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ad05d00 0x56323ae0e240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.485+0000 7f97fbfff700 1 -- 172.21.15.46:0/799516691 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56323acc1500 con 0x56323ae08400 2024-05-09T05:00:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.485+0000 7f97fbfff700 1 --2- 172.21.15.46:0/799516691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ae08400 0x56323ae08810 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009d20 tx=0x7f97f40077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41182731ad0af22d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.485+0000 7f97f1ffb700 1 -- 172.21.15.46:0/799516691 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4004bb0 con 0x56323ae08400 2024-05-09T05:00:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.485+0000 7f97f1ffb700 1 -- 172.21.15.46:0/799516691 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97f4004d50 con 0x56323ae08400 2024-05-09T05:00:15.487 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.485+0000 7f97f1ffb700 1 -- 172.21.15.46:0/799516691 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4014670 con 0x56323ae08400 2024-05-09T05:00:15.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.486+0000 7f9810b10b80 1 -- 172.21.15.46:0/799516691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ae08400 msgr2=0x56323ae08810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:15.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.486+0000 7f9810b10b80 1 --2- 172.21.15.46:0/799516691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ae08400 0x56323ae08810 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f97f4009d20 tx=0x7f97f40077e0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.486+0000 7f9810b10b80 1 -- 172.21.15.46:0/799516691 shutdown_connections 2024-05-09T05:00:15.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.486+0000 7f9810b10b80 1 --2- 172.21.15.46:0/799516691 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ae08400 0x56323ae08810 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.486+0000 7f9810b10b80 1 --2- 172.21.15.46:0/799516691 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ad05d00 0x56323ae0e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.486+0000 7f9810b10b80 1 --2- 172.21.15.46:0/799516691 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ac8b4a0 0x56323ae0e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.486+0000 7f9810b10b80 1 -- 172.21.15.46:0/799516691 >> 172.21.15.46:0/799516691 conn(0x56323ac8ef70 msgr2=0x56323ae03220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:15.490 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.486+0000 7f9810b10b80 1 -- 172.21.15.46:0/799516691 shutdown_connections 2024-05-09T05:00:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.486+0000 7f9810b10b80 1 -- 172.21.15.46:0/799516691 wait complete. 2024-05-09T05:00:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.487+0000 7f9810b10b80 1 -- start start 2024-05-09T05:00:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.487+0000 7f9810b10b80 1 Processor -- start 2024-05-09T05:00:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.487+0000 7f9810b10b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ac8b4a0 0x56323ae12080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.487+0000 7f9810b10b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ad05d00 0x56323ae125c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.487+0000 7f9810b10b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ae08400 0x56323ae12b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.487+0000 7f9810b10b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56323ae09160 con 0x56323ae08400 2024-05-09T05:00:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.487+0000 7f9810b10b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56323acc5b20 con 0x56323ac8b4a0 2024-05-09T05:00:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.487+0000 7f9810b10b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56323ae15020 con 0x56323ad05d00 2024-05-09T05:00:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.487+0000 7f9810b10b80 10 client.0 _ll_register_callbacks cb 0x56323ab9c6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.487+0000 7f97fbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ae08400 0x56323ae12b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.487+0000 7f97fbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ae08400 0x56323ae12b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43356/0 (socket says 172.21.15.46:43356) 2024-05-09T05:00:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.487+0000 7f97fbfff700 1 -- 172.21.15.46:0/269155323 learned_addr learned my addr 172.21.15.46:0/269155323 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.488+0000 7f97f2ffd700 1 --2- 172.21.15.46:0/269155323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ad05d00 0x56323ae125c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.491 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.488+0000 7f97fb7fe700 1 --2- 172.21.15.46:0/269155323 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ac8b4a0 0x56323ae12080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.488+0000 7f97fbfff700 1 -- 172.21.15.46:0/269155323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ad05d00 msgr2=0x56323ae125c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.488+0000 7f97fbfff700 1 --2- 172.21.15.46:0/269155323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ad05d00 0x56323ae125c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.488+0000 7f97fbfff700 1 -- 172.21.15.46:0/269155323 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ac8b4a0 msgr2=0x56323ae12080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.488+0000 7f97fbfff700 1 --2- 172.21.15.46:0/269155323 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ac8b4a0 0x56323ae12080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.488+0000 7f97fbfff700 1 -- 172.21.15.46:0/269155323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56323aba6490 con 0x56323ae08400 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.488+0000 7f97fbfff700 1 --2- 172.21.15.46:0/269155323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ae08400 0x56323ae12b00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f97f40059b0 tx=0x7f97f40055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b6402f7d8932fb server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.488+0000 7f97daffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.488+0000 7f97daffd700 1 -- 172.21.15.46:0/269155323 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97f4022070 con 0x56323ae08400 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.488+0000 7f97daffd700 1 -- 172.21.15.46:0/269155323 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8644+0+0 (secure 0 0 0) 0x7f97f401e420 con 0x56323ae08400 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.488+0000 7f9810b10b80 1 -- 172.21.15.46:0/269155323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97f4004d50 con 0x56323ae08400 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.488+0000 7f97daffd700 1 -- 172.21.15.46:0/269155323 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f97f402e050 con 0x56323ae08400 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.488+0000 7f9810b10b80 10 client.17265 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.489+0000 7f9810b10b80 1 -- 172.21.15.46:0/269155323 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56323ae1fa60 con 0x56323ae08400 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.489+0000 7f97daffd700 1 -- 172.21.15.46:0/269155323 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97f4004d50 con 0x56323ae08400 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.489+0000 7f97daffd700 1 -- 172.21.15.46:0/269155323 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f97f4014410 con 0x56323ae08400 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.489+0000 7f9810b10b80 10 client.17265 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:15.492 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.489+0000 7f9810b10b80 10 client.17265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:15.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.489+0000 7f9810b10b80 20 client.17265 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:15.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.489+0000 7f9810b10b80 1 --2- 172.21.15.46:0/269155323 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56323ae21f90 0x56323ae243b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.489+0000 7f9810b10b80 4 client.17265 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:00:15.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.489+0000 7f9810b10b80 1 -- 172.21.15.46:0/269155323 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56323ad38a80 con 0x56323ae21f90 2024-05-09T05:00:15.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.489+0000 7f97fb7fe700 1 --2- 172.21.15.46:0/269155323 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56323ae21f90 0x56323ae243b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.490+0000 7f97fb7fe700 1 --2- 172.21.15.46:0/269155323 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56323ae21f90 0x56323ae243b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ca7c680231381b1 server_cookie=976eb328ebb02bba in_seq=0 out_seq=0 2024-05-09T05:00:15.493 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.490+0000 7f97daffd700 10 client.17265 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.490+0000 7f97daffd700 1 -- 172.21.15.46:0/269155323 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56323ad38a80 con 0x56323ae21f90 2024-05-09T05:00:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.490+0000 7f97daffd700 10 client.17265 handle_command_reply: tid=0 2024-05-09T05:00:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.490+0000 7f9810b10b80 1 client.17265 shutdown 2024-05-09T05:00:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.491+0000 7f9810b10b80 1 -- 172.21.15.46:0/269155323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ae08400 msgr2=0x56323ae12b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.491+0000 7f9810b10b80 1 --2- 172.21.15.46:0/269155323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ae08400 0x56323ae12b00 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f97f40059b0 tx=0x7f97f40055f0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.491+0000 7f9810b10b80 1 -- 172.21.15.46:0/269155323 shutdown_connections 2024-05-09T05:00:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.491+0000 7f9810b10b80 1 --2- 172.21.15.46:0/269155323 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ae08400 0x56323ae12b00 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.491+0000 7f9810b10b80 1 --2- 172.21.15.46:0/269155323 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56323ae21f90 0x56323ae243b0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.491+0000 7f9810b10b80 1 --2- 172.21.15.46:0/269155323 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ac8b4a0 0x56323ae12080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.494 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.491+0000 7f9810b10b80 1 --2- 172.21.15.46:0/269155323 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ad05d00 0x56323ae125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.491+0000 7f9810b10b80 1 -- 172.21.15.46:0/269155323 >> 172.21.15.46:0/269155323 conn(0x56323ac8ef70 msgr2=0x56323adfcff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.491+0000 7f97daffd700 0 client.17265 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.491+0000 7f9810b10b80 1 -- 172.21.15.46:0/269155323 shutdown_connections 2024-05-09T05:00:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.491+0000 7f9810b10b80 1 -- 172.21.15.46:0/269155323 wait complete. 2024-05-09T05:00:15.495 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.491+0000 7f9810b10b80 20 client.17265 trim_cache size 0 max 16384 2024-05-09T05:00:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.506+0000 7f9810b10b80 1 Processor -- start 2024-05-09T05:00:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.506+0000 7f9810b10b80 1 -- start start 2024-05-09T05:00:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.506+0000 7f9810b10b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ac8b4a0 0x56323ae0d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.506+0000 7f9810b10b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ad05d00 0x56323ae033d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.506+0000 7f9810b10b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ae08400 0x56323adfda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.506+0000 7f9810b10b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1b18b0 con 0x56323ac8b4a0 2024-05-09T05:00:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.506+0000 7f9810b10b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1af140 con 0x56323ae08400 2024-05-09T05:00:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.506+0000 7f9810b10b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1acfb0 con 0x56323ad05d00 2024-05-09T05:00:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.506+0000 7f97fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ac8b4a0 0x56323ae0d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.507+0000 7f97fb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ac8b4a0 0x56323ae0d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43358/0 (socket says 172.21.15.46:43358) 2024-05-09T05:00:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.507+0000 7f97fb7fe700 1 -- 172.21.15.46:0/47870221 learned_addr learned my addr 172.21.15.46:0/47870221 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.507+0000 7f97fbfff700 1 --2- 172.21.15.46:0/47870221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ae08400 0x56323adfda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.507+0000 7f97f2ffd700 1 --2- 172.21.15.46:0/47870221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ad05d00 0x56323ae033d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.507+0000 7f97fb7fe700 1 -- 172.21.15.46:0/47870221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ad05d00 msgr2=0x56323ae033d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.507+0000 7f97fb7fe700 1 --2- 172.21.15.46:0/47870221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ad05d00 0x56323ae033d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.507+0000 7f97fb7fe700 1 -- 172.21.15.46:0/47870221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ae08400 msgr2=0x56323adfda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.507+0000 7f97fb7fe700 1 --2- 172.21.15.46:0/47870221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ae08400 0x56323adfda20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.507+0000 7f97fb7fe700 1 -- 172.21.15.46:0/47870221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56323adfccc0 con 0x56323ac8b4a0 2024-05-09T05:00:15.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.507+0000 7f97fb7fe700 1 --2- 172.21.15.46:0/47870221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ac8b4a0 0x56323ae0d4d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0a6b20 tx=0x7f97ec091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b692b099d6da132 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.507+0000 7f97f17fa700 1 -- 172.21.15.46:0/47870221 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec0adc60 con 0x56323ac8b4a0 2024-05-09T05:00:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.507+0000 7f97f17fa700 1 -- 172.21.15.46:0/47870221 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97ec0b2710 con 0x56323ac8b4a0 2024-05-09T05:00:15.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.507+0000 7f97f17fa700 1 -- 172.21.15.46:0/47870221 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec0aae40 con 0x56323ac8b4a0 2024-05-09T05:00:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.508+0000 7f9810b10b80 1 -- 172.21.15.46:0/47870221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ac8b4a0 msgr2=0x56323ae0d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:15.512 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.508+0000 7f9810b10b80 1 --2- 172.21.15.46:0/47870221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ac8b4a0 0x56323ae0d4d0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f97ec0a6b20 tx=0x7f97ec091040 comp rx=0 tx=0).stop 2024-05-09T05:00:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.508+0000 7f9810b10b80 1 -- 172.21.15.46:0/47870221 shutdown_connections 2024-05-09T05:00:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.508+0000 7f9810b10b80 1 --2- 172.21.15.46:0/47870221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ae08400 0x56323adfda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.508+0000 7f9810b10b80 1 --2- 172.21.15.46:0/47870221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ac8b4a0 0x56323ae0d4d0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.508+0000 7f9810b10b80 1 --2- 172.21.15.46:0/47870221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ad05d00 0x56323ae033d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.508+0000 7f9810b10b80 1 -- 172.21.15.46:0/47870221 >> 172.21.15.46:0/47870221 conn(0x56323ac8ef70 msgr2=0x56323ae0bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.508+0000 7f9810b10b80 1 -- 172.21.15.46:0/47870221 shutdown_connections 2024-05-09T05:00:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.508+0000 7f9810b10b80 1 -- 172.21.15.46:0/47870221 wait complete. 2024-05-09T05:00:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.508+0000 7f9810b10b80 1 -- start start 2024-05-09T05:00:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f9810b10b80 1 Processor -- start 2024-05-09T05:00:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f9810b10b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ac8b4a0 0x56323ae24000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f9810b10b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ad05d00 0x56323ae24540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f9810b10b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ae08400 0x56323ae8a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f9810b10b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1ad600 con 0x56323ad05d00 2024-05-09T05:00:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f9810b10b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1aea70 con 0x56323ac8b4a0 2024-05-09T05:00:15.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f9810b10b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f97fc1adcd0 con 0x56323ae08400 2024-05-09T05:00:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f9810b10b80 10 client.0 _ll_register_callbacks cb 0x56323ae0dcf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f97f2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ad05d00 0x56323ae24540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f97f2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ad05d00 0x56323ae24540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43372/0 (socket says 172.21.15.46:43372) 2024-05-09T05:00:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f97f2ffd700 1 -- 172.21.15.46:0/442809476 learned_addr learned my addr 172.21.15.46:0/442809476 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f97fb7fe700 1 --2- 172.21.15.46:0/442809476 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ac8b4a0 0x56323ae24000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f97fbfff700 1 --2- 172.21.15.46:0/442809476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ae08400 0x56323ae8a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f97f2ffd700 1 -- 172.21.15.46:0/442809476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ae08400 msgr2=0x56323ae8a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f97f2ffd700 1 --2- 172.21.15.46:0/442809476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ae08400 0x56323ae8a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f97f2ffd700 1 -- 172.21.15.46:0/442809476 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ac8b4a0 msgr2=0x56323ae24000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.509+0000 7f97f2ffd700 1 --2- 172.21.15.46:0/442809476 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ac8b4a0 0x56323ae24000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.510+0000 7f97f2ffd700 1 -- 172.21.15.46:0/442809476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56323ae8aa10 con 0x56323ad05d00 2024-05-09T05:00:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.510+0000 7f97f2ffd700 1 --2- 172.21.15.46:0/442809476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ad05d00 0x56323ae24540 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f97e400b840 tx=0x7f97e4009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ad9bee3cfc8071 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.510+0000 7f97daffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:15.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.510+0000 7f97daffd700 1 -- 172.21.15.46:0/442809476 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4034070 con 0x56323ad05d00 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.510+0000 7f9810b10b80 1 -- 172.21.15.46:0/442809476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97ec0b2710 con 0x56323ad05d00 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.510+0000 7f97daffd700 1 -- 172.21.15.46:0/442809476 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8644+0+0 (secure 0 0 0) 0x7f97e4030850 con 0x56323ad05d00 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.510+0000 7f97daffd700 1 -- 172.21.15.46:0/442809476 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f97e4040050 con 0x56323ad05d00 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.510+0000 7f9810b10b80 10 client.17271 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.510+0000 7f9810b10b80 1 -- 172.21.15.46:0/442809476 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56323ae8bb40 con 0x56323ad05d00 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.510+0000 7f97daffd700 1 -- 172.21.15.46:0/442809476 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97ec0b2710 con 0x56323ad05d00 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.511+0000 7f97daffd700 1 -- 172.21.15.46:0/442809476 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f97e4022b90 con 0x56323ad05d00 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.511+0000 7f9810b10b80 10 client.17271 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.511+0000 7f9810b10b80 10 client.17271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.511+0000 7f9810b10b80 20 client.17271 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.511+0000 7f9810b10b80 1 --2- 172.21.15.46:0/442809476 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56323ae8c1a0 0x56323ae8e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.511+0000 7f9810b10b80 4 client.17271 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.511+0000 7f9810b10b80 1 -- 172.21.15.46:0/442809476 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56323ad32b20 con 0x56323ae8c1a0 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.511+0000 7f97fb7fe700 1 --2- 172.21.15.46:0/442809476 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56323ae8c1a0 0x56323ae8e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.511+0000 7f97fb7fe700 1 --2- 172.21.15.46:0/442809476 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56323ae8c1a0 0x56323ae8e5c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd997732f65ef120 server_cookie=347399dad1264368 in_seq=0 out_seq=0 2024-05-09T05:00:15.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.511+0000 7f97daffd700 10 client.17271 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.512+0000 7f97daffd700 1 -- 172.21.15.46:0/442809476 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56323ad32b20 con 0x56323ae8c1a0 2024-05-09T05:00:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.512+0000 7f97daffd700 10 client.17271 handle_command_reply: tid=0 2024-05-09T05:00:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.512+0000 7f9810b10b80 1 client.17271 shutdown 2024-05-09T05:00:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.512+0000 7f9810b10b80 1 -- 172.21.15.46:0/442809476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ad05d00 msgr2=0x56323ae24540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.512+0000 7f9810b10b80 1 --2- 172.21.15.46:0/442809476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ad05d00 0x56323ae24540 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f97e400b840 tx=0x7f97e4009310 comp rx=0 tx=0).stop 2024-05-09T05:00:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.512+0000 7f9810b10b80 1 -- 172.21.15.46:0/442809476 shutdown_connections 2024-05-09T05:00:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.512+0000 7f9810b10b80 1 --2- 172.21.15.46:0/442809476 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56323ae08400 0x56323ae8a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.512+0000 7f9810b10b80 1 --2- 172.21.15.46:0/442809476 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x56323ae8c1a0 0x56323ae8e5c0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.512+0000 7f9810b10b80 1 --2- 172.21.15.46:0/442809476 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56323ac8b4a0 0x56323ae24000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.512+0000 7f9810b10b80 1 --2- 172.21.15.46:0/442809476 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56323ad05d00 0x56323ae24540 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.512+0000 7f9810b10b80 1 -- 172.21.15.46:0/442809476 >> 172.21.15.46:0/442809476 conn(0x56323ac8ef70 msgr2=0x56323adfd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.512+0000 7f97daffd700 0 client.17271 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.512+0000 7f9810b10b80 1 -- 172.21.15.46:0/442809476 shutdown_connections 2024-05-09T05:00:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.512+0000 7f9810b10b80 1 -- 172.21.15.46:0/442809476 wait complete. 2024-05-09T05:00:15.516 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.512+0000 7f9810b10b80 20 client.17271 trim_cache size 0 max 16384 2024-05-09T05:00:15.517 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T05:00:15.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.513+0000 7f98020fa700 1 -- 172.21.15.46:0/3253982689 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f97e80b29f0 msgr2=0x7f97e80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:15.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.513+0000 7f98020fa700 1 --2- 172.21.15.46:0/3253982689 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f97e80b29f0 0x7f97e80b4f00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f97e4000c00 tx=0x7f97e401a040 comp rx=0 tx=0).stop 2024-05-09T05:00:15.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.513+0000 7f98020fa700 1 -- 172.21.15.46:0/3253982689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc104650 msgr2=0x7f97fc19ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:15.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.513+0000 7f98020fa700 1 --2- 172.21.15.46:0/3253982689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc104650 0x7f97fc19ae80 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00d980 tx=0x7f97ec00dd40 comp rx=0 tx=0).stop 2024-05-09T05:00:15.517 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.513+0000 7f98020fa700 1 -- 172.21.15.46:0/3253982689 shutdown_connections 2024-05-09T05:00:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.513+0000 7f98020fa700 1 --2- 172.21.15.46:0/3253982689 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f97e80b29f0 0x7f97e80b4f00 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.513+0000 7f98020fa700 1 --2- 172.21.15.46:0/3253982689 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f97fc109880 0x7f97fc19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.513+0000 7f98020fa700 1 --2- 172.21.15.46:0/3253982689 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f97fc104650 0x7f97fc19ae80 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.513+0000 7f98020fa700 1 --2- 172.21.15.46:0/3253982689 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f97fc108f30 0x7f97fc19b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.513+0000 7f98020fa700 1 -- 172.21.15.46:0/3253982689 >> 172.21.15.46:0/3253982689 conn(0x7f97fc079f00 msgr2=0x7f97fc102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.513+0000 7f98020fa700 1 -- 172.21.15.46:0/3253982689 shutdown_connections 2024-05-09T05:00:15.518 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:15.513+0000 7f98020fa700 1 -- 172.21.15.46:0/3253982689 wait complete. 2024-05-09T05:00:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:16 smithi072 ceph-mon[34088]: pgmap v283: 97 pgs: 4 active+clean+snaptrim, 82 active+clean, 11 active+clean+snaptrim_wait; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:00:17.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:16 smithi110 ceph-mon[34904]: pgmap v283: 97 pgs: 4 active+clean+snaptrim, 82 active+clean, 11 active+clean+snaptrim_wait; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:00:17.314 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T05:00:17.315 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T05:00:17.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:16 smithi046 ceph-mon[30136]: pgmap v283: 97 pgs: 4 active+clean+snaptrim, 82 active+clean, 11 active+clean+snaptrim_wait; 9.7 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:00:17.579 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:18.998 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:18 smithi046 ceph-mon[30136]: pgmap v284: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-09T05:00:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:18 smithi072 ceph-mon[34088]: pgmap v284: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-09T05:00:19.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:18 smithi110 ceph-mon[34904]: pgmap v284: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-09T05:00:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.366+0000 7f36cbbc4700 1 -- 172.21.15.46:0/2387865166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c4105db0 msgr2=0x7f36c410e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.366+0000 7f36cbbc4700 1 --2- 172.21.15.46:0/2387865166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c4105db0 0x7f36c410e300 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009990 tx=0x7f36b4009ca0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.366+0000 7f36cbbc4700 1 -- 172.21.15.46:0/2387865166 shutdown_connections 2024-05-09T05:00:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.366+0000 7f36cbbc4700 1 --2- 172.21.15.46:0/2387865166 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36c410e8d0 0x7f36c4110db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.366+0000 7f36cbbc4700 1 --2- 172.21.15.46:0/2387865166 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c4105460 0x7f36c4105870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.366+0000 7f36cbbc4700 1 --2- 172.21.15.46:0/2387865166 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c4105db0 0x7f36c410e300 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.366+0000 7f36cbbc4700 1 -- 172.21.15.46:0/2387865166 >> 172.21.15.46:0/2387865166 conn(0x7f36c40fedc0 msgr2=0x7f36c41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:19.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.366+0000 7f36cbbc4700 1 -- 172.21.15.46:0/2387865166 shutdown_connections 2024-05-09T05:00:19.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.367+0000 7f36cbbc4700 1 -- 172.21.15.46:0/2387865166 wait complete. 2024-05-09T05:00:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.367+0000 7f36cbbc4700 1 Processor -- start 2024-05-09T05:00:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.367+0000 7f36cbbc4700 1 -- start start 2024-05-09T05:00:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36cbbc4700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c4105460 0x7f36c419ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36cbbc4700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36c4105db0 0x7f36c419b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36cbbc4700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c410e8d0 0x7f36c419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36cbbc4700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4114500 con 0x7f36c410e8d0 2024-05-09T05:00:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36cbbc4700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4114380 con 0x7f36c4105460 2024-05-09T05:00:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36cbbc4700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36c4114680 con 0x7f36c4105db0 2024-05-09T05:00:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36ca161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c410e8d0 0x7f36c419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36ca161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c410e8d0 0x7f36c419f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43394/0 (socket says 172.21.15.46:43394) 2024-05-09T05:00:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36c9960700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c4105460 0x7f36c419ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36ca161700 1 -- 172.21.15.46:0/1149444912 learned_addr learned my addr 172.21.15.46:0/1149444912 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36c915f700 1 --2- 172.21.15.46:0/1149444912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36c4105db0 0x7f36c419b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36ca161700 1 -- 172.21.15.46:0/1149444912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36c4105db0 msgr2=0x7f36c419b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36ca161700 1 --2- 172.21.15.46:0/1149444912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36c4105db0 0x7f36c419b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36ca161700 1 -- 172.21.15.46:0/1149444912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c4105460 msgr2=0x7f36c419ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36ca161700 1 --2- 172.21.15.46:0/1149444912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c4105460 0x7f36c419ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.368+0000 7f36ca161700 1 -- 172.21.15.46:0/1149444912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36c419fd40 con 0x7f36c410e8d0 2024-05-09T05:00:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.369+0000 7f36ca161700 1 --2- 172.21.15.46:0/1149444912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c410e8d0 0x7f36c419f6a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00b970 tx=0x7f36bc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.369+0000 7f36baffd700 1 -- 172.21.15.46:0/1149444912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc00d860 con 0x7f36c410e8d0 2024-05-09T05:00:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.369+0000 7f36baffd700 1 -- 172.21.15.46:0/1149444912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36bc00e430 con 0x7f36c410e8d0 2024-05-09T05:00:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.369+0000 7f36cbbc4700 1 -- 172.21.15.46:0/1149444912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36c41a0030 con 0x7f36c410e8d0 2024-05-09T05:00:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.369+0000 7f36baffd700 1 -- 172.21.15.46:0/1149444912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc017660 con 0x7f36c410e8d0 2024-05-09T05:00:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.369+0000 7f36cbbc4700 1 -- 172.21.15.46:0/1149444912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36c41a7950 con 0x7f36c410e8d0 2024-05-09T05:00:19.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.371+0000 7f36baffd700 1 -- 172.21.15.46:0/1149444912 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f36bc00da00 con 0x7f36c410e8d0 2024-05-09T05:00:19.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.372+0000 7f36baffd700 1 --2- 172.21.15.46:0/1149444912 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f36b00b29f0 0x7f36b00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.372+0000 7f36c9960700 1 --2- 172.21.15.46:0/1149444912 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f36b00b29f0 0x7f36b00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.372+0000 7f36baffd700 1 -- 172.21.15.46:0/1149444912 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8644+0+0 (secure 0 0 0) 0x7f36bc0e2290 con 0x7f36c410e8d0 2024-05-09T05:00:19.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.373+0000 7f36da5dab80 1 Processor -- start 2024-05-09T05:00:19.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.373+0000 7f36c9960700 1 --2- 172.21.15.46:0/1149444912 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f36b00b29f0 0x7f36b00b4f00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f36c0005950 tx=0x7f36c000b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:19.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.373+0000 7f36da5dab80 1 -- start start 2024-05-09T05:00:19.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.373+0000 7f36da5dab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5825eed80 0x55b5826f5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.373+0000 7f36da5dab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b582574520 0x55b5826f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.373+0000 7f36da5dab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5826fa260 0x55b5826fa670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.373+0000 7f36da5dab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b582614120 con 0x55b5825eed80 2024-05-09T05:00:19.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.373+0000 7f36da5dab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55b5825b9a20 con 0x55b582574520 2024-05-09T05:00:19.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.373+0000 7f36da5dab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b5826100e0 con 0x55b5826fa260 2024-05-09T05:00:19.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.373+0000 7f36c9960700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5825eed80 0x55b5826f5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.373+0000 7f36c9960700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5825eed80 0x55b5826f5b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43408/0 (socket says 172.21.15.46:43408) 2024-05-09T05:00:19.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.373+0000 7f36c9960700 1 -- 172.21.15.46:0/3619657645 learned_addr learned my addr 172.21.15.46:0/3619657645 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:19.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.373+0000 7f36c915f700 1 --2- 172.21.15.46:0/3619657645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b582574520 0x55b5826f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.374+0000 7f36ca161700 1 --2- 172.21.15.46:0/3619657645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5826fa260 0x55b5826fa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.374+0000 7f36c915f700 1 -- 172.21.15.46:0/3619657645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5826fa260 msgr2=0x55b5826fa670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:19.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.374+0000 7f36c915f700 1 --2- 172.21.15.46:0/3619657645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5826fa260 0x55b5826fa670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.374+0000 7f36c915f700 1 -- 172.21.15.46:0/3619657645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5825eed80 msgr2=0x55b5826f5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:19.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.374+0000 7f36c915f700 1 --2- 172.21.15.46:0/3619657645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5825eed80 0x55b5826f5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.374+0000 7f36c915f700 1 -- 172.21.15.46:0/3619657645 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5825aa580 con 0x55b582574520 2024-05-09T05:00:19.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.374+0000 7f36c9960700 1 --2- 172.21.15.46:0/3619657645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5825eed80 0x55b5826f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:00:19.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.374+0000 7f36c915f700 1 --2- 172.21.15.46:0/3619657645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b582574520 0x55b5826f60b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f36b400b5c0 tx=0x7f36b40095c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d3c68bb5d0f9eef server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:19.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.374+0000 7f36affff700 1 -- 172.21.15.46:0/3619657645 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4003a60 con 0x55b582574520 2024-05-09T05:00:19.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.374+0000 7f36affff700 1 -- 172.21.15.46:0/3619657645 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36b4005520 con 0x55b582574520 2024-05-09T05:00:19.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.374+0000 7f36affff700 1 -- 172.21.15.46:0/3619657645 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b400bc10 con 0x55b582574520 2024-05-09T05:00:19.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.375+0000 7f36da5dab80 1 -- 172.21.15.46:0/3619657645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b582574520 msgr2=0x55b5826f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:19.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.375+0000 7f36da5dab80 1 --2- 172.21.15.46:0/3619657645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b582574520 0x55b5826f60b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f36b400b5c0 tx=0x7f36b40095c0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.375+0000 7f36da5dab80 1 -- 172.21.15.46:0/3619657645 shutdown_connections 2024-05-09T05:00:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.375+0000 7f36da5dab80 1 --2- 172.21.15.46:0/3619657645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5826fa260 0x55b5826fa670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.375+0000 7f36da5dab80 1 --2- 172.21.15.46:0/3619657645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5825eed80 0x55b5826f5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.375+0000 7f36da5dab80 1 --2- 172.21.15.46:0/3619657645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b582574520 0x55b5826f60b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.375+0000 7f36da5dab80 1 -- 172.21.15.46:0/3619657645 >> 172.21.15.46:0/3619657645 conn(0x55b582577ff0 msgr2=0x55b5826ec2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.375+0000 7f36da5dab80 1 -- 172.21.15.46:0/3619657645 shutdown_connections 2024-05-09T05:00:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.375+0000 7f36da5dab80 1 -- 172.21.15.46:0/3619657645 wait complete. 2024-05-09T05:00:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.376+0000 7f36da5dab80 1 -- start start 2024-05-09T05:00:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.376+0000 7f36da5dab80 1 Processor -- start 2024-05-09T05:00:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.376+0000 7f36da5dab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b582574520 0x55b5826ff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.376+0000 7f36da5dab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b5825eed80 0x55b5826ff910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.376+0000 7f36da5dab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b5826ffe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.376+0000 7f36da5dab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b5826faec0 con 0x55b5826fa260 2024-05-09T05:00:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.376+0000 7f36da5dab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55b5825aeba0 con 0x55b5825eed80 2024-05-09T05:00:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.376+0000 7f36da5dab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b5826fb040 con 0x55b582574520 2024-05-09T05:00:19.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.376+0000 7f36da5dab80 10 client.0 _ll_register_callbacks cb 0x55b582485740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.376+0000 7f36ca161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b5826ffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.376+0000 7f36ca161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b5826ffe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43418/0 (socket says 172.21.15.46:43418) 2024-05-09T05:00:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.376+0000 7f36ca161700 1 -- 172.21.15.46:0/2805014424 learned_addr learned my addr 172.21.15.46:0/2805014424 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.377+0000 7f36c9960700 1 --2- 172.21.15.46:0/2805014424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b582574520 0x55b5826ff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.377+0000 7f36c915f700 1 --2- 172.21.15.46:0/2805014424 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b5825eed80 0x55b5826ff910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.377+0000 7f36ca161700 1 -- 172.21.15.46:0/2805014424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b582574520 msgr2=0x55b5826ff3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.377+0000 7f36ca161700 1 --2- 172.21.15.46:0/2805014424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b582574520 0x55b5826ff3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.377+0000 7f36ca161700 1 -- 172.21.15.46:0/2805014424 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b5825eed80 msgr2=0x55b5826ff910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.377+0000 7f36ca161700 1 --2- 172.21.15.46:0/2805014424 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b5825eed80 0x55b5826ff910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.377+0000 7f36ca161700 1 -- 172.21.15.46:0/2805014424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b58248f510 con 0x55b5826fa260 2024-05-09T05:00:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.377+0000 7f36ca161700 1 --2- 172.21.15.46:0/2805014424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b5826ffe50 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00b940 tx=0x7f36bc00eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbe38a1fbfa022f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.377+0000 7f36ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.377+0000 7f36ad7fa700 1 -- 172.21.15.46:0/2805014424 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0b2350 con 0x55b5826fa260 2024-05-09T05:00:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.377+0000 7f36ad7fa700 1 -- 172.21.15.46:0/2805014424 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8644+0+0 (secure 0 0 0) 0x7f36bc00dcf0 con 0x55b5826fa260 2024-05-09T05:00:19.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.377+0000 7f36da5dab80 1 -- 172.21.15.46:0/2805014424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36b4005520 con 0x55b5826fa260 2024-05-09T05:00:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.377+0000 7f36ad7fa700 1 -- 172.21.15.46:0/2805014424 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36bc01e2f0 con 0x55b5826fa260 2024-05-09T05:00:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.377+0000 7f36da5dab80 10 client.17283 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.377+0000 7f36da5dab80 1 -- 172.21.15.46:0/2805014424 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b58270cd10 con 0x55b5826fa260 2024-05-09T05:00:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.378+0000 7f36ad7fa700 1 -- 172.21.15.46:0/2805014424 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36b4005520 con 0x55b5826fa260 2024-05-09T05:00:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.378+0000 7f36ad7fa700 1 -- 172.21.15.46:0/2805014424 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f36bc06e070 con 0x55b5826fa260 2024-05-09T05:00:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.378+0000 7f36da5dab80 10 client.17283 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.378+0000 7f36da5dab80 10 client.17283 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.378+0000 7f36da5dab80 20 client.17283 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.378+0000 7f36da5dab80 1 --2- 172.21.15.46:0/2805014424 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b58270f350 0x55b5827117b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.378+0000 7f36da5dab80 4 client.17283 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:00:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.378+0000 7f36da5dab80 1 -- 172.21.15.46:0/2805014424 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b582621b00 con 0x55b58270f350 2024-05-09T05:00:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.378+0000 7f36c9960700 1 --2- 172.21.15.46:0/2805014424 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b58270f350 0x55b5827117b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.379+0000 7f36ad7fa700 10 client.17283 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.379+0000 7f36c9960700 1 --2- 172.21.15.46:0/2805014424 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b58270f350 0x55b5827117b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e48618f62d8cd5a7 server_cookie=8ca58bde5e6fa5a1 in_seq=0 out_seq=0 2024-05-09T05:00:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.379+0000 7f36ad7fa700 1 -- 172.21.15.46:0/2805014424 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b582621b00 con 0x55b58270f350 2024-05-09T05:00:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.379+0000 7f36ad7fa700 10 client.17283 handle_command_reply: tid=0 2024-05-09T05:00:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.379+0000 7f36da5dab80 1 client.17283 shutdown 2024-05-09T05:00:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.380+0000 7f36da5dab80 1 -- 172.21.15.46:0/2805014424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 msgr2=0x55b5826ffe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.380+0000 7f36da5dab80 1 --2- 172.21.15.46:0/2805014424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b5826ffe50 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00b940 tx=0x7f36bc00eb10 comp rx=0 tx=0).stop 2024-05-09T05:00:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.380+0000 7f36da5dab80 1 -- 172.21.15.46:0/2805014424 shutdown_connections 2024-05-09T05:00:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.380+0000 7f36da5dab80 1 --2- 172.21.15.46:0/2805014424 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b5826ffe50 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.380+0000 7f36da5dab80 1 --2- 172.21.15.46:0/2805014424 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b582574520 0x55b5826ff3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.380+0000 7f36da5dab80 1 --2- 172.21.15.46:0/2805014424 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b58270f350 0x55b5827117b0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.380+0000 7f36ad7fa700 0 client.17283 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:19.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.380+0000 7f36da5dab80 1 --2- 172.21.15.46:0/2805014424 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b5825eed80 0x55b5826ff910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.380+0000 7f36da5dab80 1 -- 172.21.15.46:0/2805014424 >> 172.21.15.46:0/2805014424 conn(0x55b582577ff0 msgr2=0x55b5826e6070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.380+0000 7f36da5dab80 1 -- 172.21.15.46:0/2805014424 shutdown_connections 2024-05-09T05:00:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.380+0000 7f36da5dab80 1 -- 172.21.15.46:0/2805014424 wait complete. 2024-05-09T05:00:19.385 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.380+0000 7f36da5dab80 20 client.17283 trim_cache size 0 max 16384 2024-05-09T05:00:19.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36da5dab80 1 Processor -- start 2024-05-09T05:00:19.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36da5dab80 1 -- start start 2024-05-09T05:00:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36da5dab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b582574520 0x55b5826f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36da5dab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5825eed80 0x55b5826e6930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36da5dab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b58270b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36da5dab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41af720 con 0x55b5826fa260 2024-05-09T05:00:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36da5dab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41acfb0 con 0x55b582574520 2024-05-09T05:00:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36da5dab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41aae20 con 0x55b5825eed80 2024-05-09T05:00:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36ca161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b58270b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36ca161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b58270b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43420/0 (socket says 172.21.15.46:43420) 2024-05-09T05:00:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36ca161700 1 -- 172.21.15.46:0/636894258 learned_addr learned my addr 172.21.15.46:0/636894258 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36c9960700 1 --2- 172.21.15.46:0/636894258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b582574520 0x55b5826f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36c915f700 1 --2- 172.21.15.46:0/636894258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5825eed80 0x55b5826e6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36ca161700 1 -- 172.21.15.46:0/636894258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5825eed80 msgr2=0x55b5826e6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:19.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36ca161700 1 --2- 172.21.15.46:0/636894258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5825eed80 0x55b5826e6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36ca161700 1 -- 172.21.15.46:0/636894258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b582574520 msgr2=0x55b5826f0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:19.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.401+0000 7f36ca161700 1 --2- 172.21.15.46:0/636894258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b582574520 0x55b5826f0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.402+0000 7f36ca161700 1 -- 172.21.15.46:0/636894258 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b582701c70 con 0x55b5826fa260 2024-05-09T05:00:19.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.402+0000 7f36c9960700 1 --2- 172.21.15.46:0/636894258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b582574520 0x55b5826f0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:19.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.402+0000 7f36c915f700 1 --2- 172.21.15.46:0/636894258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5825eed80 0x55b5826e6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:00:19.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.402+0000 7f36ca161700 1 --2- 172.21.15.46:0/636894258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b58270b240 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0a9480 tx=0x7f36bc0a54e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f88c11c9887b0d server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:19.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.402+0000 7f36af7fe700 1 -- 172.21.15.46:0/636894258 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0b2350 con 0x55b5826fa260 2024-05-09T05:00:19.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.402+0000 7f36af7fe700 1 -- 172.21.15.46:0/636894258 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36bc01e490 con 0x55b5826fa260 2024-05-09T05:00:19.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.402+0000 7f36af7fe700 1 -- 172.21.15.46:0/636894258 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0b1ec0 con 0x55b5826fa260 2024-05-09T05:00:19.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.403+0000 7f36da5dab80 1 -- 172.21.15.46:0/636894258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 msgr2=0x55b58270b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:19.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.403+0000 7f36da5dab80 1 --2- 172.21.15.46:0/636894258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b58270b240 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f36bc0a9480 tx=0x7f36bc0a54e0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.403+0000 7f36da5dab80 1 -- 172.21.15.46:0/636894258 shutdown_connections 2024-05-09T05:00:19.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.403+0000 7f36da5dab80 1 --2- 172.21.15.46:0/636894258 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b58270b240 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.403+0000 7f36da5dab80 1 --2- 172.21.15.46:0/636894258 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b582574520 0x55b5826f0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.403+0000 7f36da5dab80 1 --2- 172.21.15.46:0/636894258 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b5825eed80 0x55b5826e6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.403+0000 7f36da5dab80 1 -- 172.21.15.46:0/636894258 >> 172.21.15.46:0/636894258 conn(0x55b582577ff0 msgr2=0x55b582700860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.403+0000 7f36da5dab80 1 -- 172.21.15.46:0/636894258 shutdown_connections 2024-05-09T05:00:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.403+0000 7f36da5dab80 1 -- 172.21.15.46:0/636894258 wait complete. 2024-05-09T05:00:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.403+0000 7f36da5dab80 1 -- start start 2024-05-09T05:00:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.403+0000 7f36da5dab80 1 Processor -- start 2024-05-09T05:00:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36da5dab80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b582574520 0x55b582706440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36da5dab80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b5825eed80 0x55b582706980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36da5dab80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b582779b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36da5dab80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41ab470 con 0x55b5826fa260 2024-05-09T05:00:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36da5dab80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41ac8e0 con 0x55b5825eed80 2024-05-09T05:00:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36da5dab80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f36c41abb40 con 0x55b582574520 2024-05-09T05:00:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36da5dab80 10 client.0 _ll_register_callbacks cb 0x55b582485740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:19.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36ca161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b582779b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36ca161700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b582779b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43422/0 (socket says 172.21.15.46:43422) 2024-05-09T05:00:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36ca161700 1 -- 172.21.15.46:0/2776264464 learned_addr learned my addr 172.21.15.46:0/2776264464 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36c915f700 1 --2- 172.21.15.46:0/2776264464 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b5825eed80 0x55b582706980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36c9960700 1 --2- 172.21.15.46:0/2776264464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b582574520 0x55b582706440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36ca161700 1 -- 172.21.15.46:0/2776264464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b582574520 msgr2=0x55b582706440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36ca161700 1 --2- 172.21.15.46:0/2776264464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b582574520 0x55b582706440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36ca161700 1 -- 172.21.15.46:0/2776264464 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b5825eed80 msgr2=0x55b582706980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36ca161700 1 --2- 172.21.15.46:0/2776264464 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b5825eed80 0x55b582706980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36ca161700 1 -- 172.21.15.46:0/2776264464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5827071a0 con 0x55b5826fa260 2024-05-09T05:00:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.404+0000 7f36c9960700 1 --2- 172.21.15.46:0/2776264464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b582574520 0x55b582706440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.405+0000 7f36ca161700 1 --2- 172.21.15.46:0/2776264464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b582779b10 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00bfd0 tx=0x7f36bc0ad960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d44a7e397ebdd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.405+0000 7f36ad7fa700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.405+0000 7f36ad7fa700 1 -- 172.21.15.46:0/2776264464 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0ca340 con 0x55b5826fa260 2024-05-09T05:00:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.405+0000 7f36da5dab80 1 -- 172.21.15.46:0/2776264464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36bc01e490 con 0x55b5826fa260 2024-05-09T05:00:19.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.405+0000 7f36ad7fa700 1 -- 172.21.15.46:0/2776264464 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 8644+0+0 (secure 0 0 0) 0x7f36bc0ca4e0 con 0x55b5826fa260 2024-05-09T05:00:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.405+0000 7f36ad7fa700 1 -- 172.21.15.46:0/2776264464 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f36bc08f050 con 0x55b5826fa260 2024-05-09T05:00:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.405+0000 7f36da5dab80 10 client.17289 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.405+0000 7f36da5dab80 1 -- 172.21.15.46:0/2776264464 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b58277afb0 con 0x55b5826fa260 2024-05-09T05:00:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.405+0000 7f36ad7fa700 1 -- 172.21.15.46:0/2776264464 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36bc01e490 con 0x55b5826fa260 2024-05-09T05:00:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.405+0000 7f36ad7fa700 1 -- 172.21.15.46:0/2776264464 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f36bc06e070 con 0x55b5826fa260 2024-05-09T05:00:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.406+0000 7f36da5dab80 10 client.17289 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.406+0000 7f36da5dab80 10 client.17289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.406+0000 7f36da5dab80 20 client.17289 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.406+0000 7f36da5dab80 1 --2- 172.21.15.46:0/2776264464 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b58277b5f0 0x55b58277da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.406+0000 7f36da5dab80 4 client.17289 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T05:00:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.406+0000 7f36da5dab80 1 -- 172.21.15.46:0/2776264464 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55b58261bba0 con 0x55b58277b5f0 2024-05-09T05:00:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.406+0000 7f36c9960700 1 --2- 172.21.15.46:0/2776264464 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b58277b5f0 0x55b58277da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:19.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.406+0000 7f36c9960700 1 --2- 172.21.15.46:0/2776264464 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b58277b5f0 0x55b58277da10 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=77d7f86f2890a1e8 server_cookie=1b2366167579637e in_seq=0 out_seq=0 2024-05-09T05:00:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.406+0000 7f36ad7fa700 10 client.17289 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.407+0000 7f36ad7fa700 1 -- 172.21.15.46:0/2776264464 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55b58261bba0 con 0x55b58277b5f0 2024-05-09T05:00:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.407+0000 7f36ad7fa700 10 client.17289 handle_command_reply: tid=0 2024-05-09T05:00:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.407+0000 7f36da5dab80 1 client.17289 shutdown 2024-05-09T05:00:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.407+0000 7f36da5dab80 1 -- 172.21.15.46:0/2776264464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 msgr2=0x55b582779b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.407+0000 7f36da5dab80 1 --2- 172.21.15.46:0/2776264464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b582779b10 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00bfd0 tx=0x7f36bc0ad960 comp rx=0 tx=0).stop 2024-05-09T05:00:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.407+0000 7f36da5dab80 1 -- 172.21.15.46:0/2776264464 shutdown_connections 2024-05-09T05:00:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.407+0000 7f36da5dab80 1 --2- 172.21.15.46:0/2776264464 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b5826fa260 0x55b582779b10 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.407+0000 7f36da5dab80 1 --2- 172.21.15.46:0/2776264464 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b582574520 0x55b582706440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.407+0000 7f36da5dab80 1 --2- 172.21.15.46:0/2776264464 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b58277b5f0 0x55b58277da10 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36da5dab80 1 --2- 172.21.15.46:0/2776264464 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b5825eed80 0x55b582706980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36ad7fa700 0 client.17289 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36da5dab80 1 -- 172.21.15.46:0/2776264464 >> 172.21.15.46:0/2776264464 conn(0x55b582577ff0 msgr2=0x55b5826f0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:19.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36da5dab80 1 -- 172.21.15.46:0/2776264464 shutdown_connections 2024-05-09T05:00:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36da5dab80 1 -- 172.21.15.46:0/2776264464 wait complete. 2024-05-09T05:00:19.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36da5dab80 20 client.17289 trim_cache size 0 max 16384 2024-05-09T05:00:19.414 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T05:00:19.414 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T05:00:19.414 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "80d7d88d-f68c-4939-a712-0ce73bd9ebf2", 2024-05-09T05:00:19.414 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T05:00:19.414 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T05:00:19.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36cbbc4700 1 -- 172.21.15.46:0/1149444912 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f36b00b29f0 msgr2=0x7f36b00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:19.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36cbbc4700 1 --2- 172.21.15.46:0/1149444912 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f36b00b29f0 0x7f36b00b4f00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f36c0005950 tx=0x7f36c000b410 comp rx=0 tx=0).stop 2024-05-09T05:00:19.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36cbbc4700 1 -- 172.21.15.46:0/1149444912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c410e8d0 msgr2=0x7f36c419f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:19.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36cbbc4700 1 --2- 172.21.15.46:0/1149444912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c410e8d0 0x7f36c419f6a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f36bc00b970 tx=0x7f36bc00bc80 comp rx=0 tx=0).stop 2024-05-09T05:00:19.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36cbbc4700 1 -- 172.21.15.46:0/1149444912 shutdown_connections 2024-05-09T05:00:19.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36cbbc4700 1 --2- 172.21.15.46:0/1149444912 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f36b00b29f0 0x7f36b00b4f00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36cbbc4700 1 --2- 172.21.15.46:0/1149444912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f36c410e8d0 0x7f36c419f6a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36cbbc4700 1 --2- 172.21.15.46:0/1149444912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f36c4105460 0x7f36c419ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36cbbc4700 1 --2- 172.21.15.46:0/1149444912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f36c4105db0 0x7f36c419b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:19.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36cbbc4700 1 -- 172.21.15.46:0/1149444912 >> 172.21.15.46:0/1149444912 conn(0x7f36c40fedc0 msgr2=0x7f36c40ff3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.408+0000 7f36cbbc4700 1 -- 172.21.15.46:0/1149444912 shutdown_connections 2024-05-09T05:00:19.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:19.409+0000 7f36cbbc4700 1 -- 172.21.15.46:0/1149444912 wait complete. 2024-05-09T05:00:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:20 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T05:00:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:20 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:20 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T05:00:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:20 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:20 smithi072 ceph-mon[34088]: pgmap v285: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-09T05:00:21.166 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:20 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T05:00:21.166 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:20 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:21.166 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:20 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T05:00:21.166 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:20 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:21.166 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:20 smithi046 ceph-mon[30136]: pgmap v285: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-09T05:00:21.201 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:00:21.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:20 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T05:00:21.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:20 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:21.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:20 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T05:00:21.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:20 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:21.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:20 smithi110 ceph-mon[34904]: pgmap v285: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 307 B/s wr, 6 op/s 2024-05-09T05:00:21.509 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:21.953 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:21 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T05:00:21.953 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:21 smithi046 ceph-mon[30136]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T05:00:21.953 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:21 smithi046 ceph-mon[30136]: pgmap v287: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 206 B/s wr, 6 op/s 2024-05-09T05:00:22.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:21 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T05:00:22.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:21 smithi072 ceph-mon[34088]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T05:00:22.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:21 smithi072 ceph-mon[34088]: pgmap v287: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 206 B/s wr, 6 op/s 2024-05-09T05:00:22.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:21 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T05:00:22.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:21 smithi110 ceph-mon[34904]: osdmap e91: 12 total, 12 up, 12 in 2024-05-09T05:00:22.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:21 smithi110 ceph-mon[34904]: pgmap v287: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 206 B/s wr, 6 op/s 2024-05-09T05:00:23.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.188+0000 7f0f9b150700 1 -- 172.21.15.46:0/968584753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f94107e10 msgr2=0x7f0f94108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:23.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.188+0000 7f0f9b150700 1 --2- 172.21.15.46:0/968584753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f94107e10 0x7f0f94108220 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f0f880099c0 tx=0x7f0f88009cd0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.188+0000 7f0f9b150700 1 -- 172.21.15.46:0/968584753 shutdown_connections 2024-05-09T05:00:23.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.188+0000 7f0f9b150700 1 --2- 172.21.15.46:0/968584753 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f94108760 0x7f0f941100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.188+0000 7f0f9b150700 1 --2- 172.21.15.46:0/968584753 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f941034e0 0x7f0f941038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.188+0000 7f0f9b150700 1 --2- 172.21.15.46:0/968584753 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f94107e10 0x7f0f94108220 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.188+0000 7f0f9b150700 1 -- 172.21.15.46:0/968584753 >> 172.21.15.46:0/968584753 conn(0x7f0f940fef30 msgr2=0x7f0f94101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:23.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.188+0000 7f0f9b150700 1 -- 172.21.15.46:0/968584753 shutdown_connections 2024-05-09T05:00:23.189 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.188+0000 7f0f9b150700 1 -- 172.21.15.46:0/968584753 wait complete. 2024-05-09T05:00:23.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.189+0000 7f0f9b150700 1 Processor -- start 2024-05-09T05:00:23.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.189+0000 7f0f9b150700 1 -- start start 2024-05-09T05:00:23.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f9b150700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f941034e0 0x7f0f9419aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.191 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f9b150700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f94107e10 0x7f0f9419b470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f9b150700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f94108760 0x7f0f9419f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f9b150700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f94113430 con 0x7f0f94108760 2024-05-09T05:00:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f9b150700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941132b0 con 0x7f0f941034e0 2024-05-09T05:00:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f9b150700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941135b0 con 0x7f0f94107e10 2024-05-09T05:00:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f996ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f94108760 0x7f0f9419f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f996ed700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f94108760 0x7f0f9419f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43438/0 (socket says 172.21.15.46:43438) 2024-05-09T05:00:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f996ed700 1 -- 172.21.15.46:0/681747066 learned_addr learned my addr 172.21.15.46:0/681747066 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f93fff700 1 --2- 172.21.15.46:0/681747066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f94107e10 0x7f0f9419b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f98eec700 1 --2- 172.21.15.46:0/681747066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f941034e0 0x7f0f9419aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f996ed700 1 -- 172.21.15.46:0/681747066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f94107e10 msgr2=0x7f0f9419b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f996ed700 1 --2- 172.21.15.46:0/681747066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f94107e10 0x7f0f9419b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f996ed700 1 -- 172.21.15.46:0/681747066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f941034e0 msgr2=0x7f0f9419aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f996ed700 1 --2- 172.21.15.46:0/681747066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f941034e0 0x7f0f9419aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f996ed700 1 -- 172.21.15.46:0/681747066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f9419fda0 con 0x7f0f94108760 2024-05-09T05:00:23.192 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.190+0000 7f0f996ed700 1 --2- 172.21.15.46:0/681747066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f94108760 0x7f0f9419f700 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c007ee0 tx=0x7f0f8c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:23.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.191+0000 7f0f91ffb700 1 -- 172.21.15.46:0/681747066 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c004bb0 con 0x7f0f94108760 2024-05-09T05:00:23.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.191+0000 7f0f91ffb700 1 -- 172.21.15.46:0/681747066 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f8c004d50 con 0x7f0f94108760 2024-05-09T05:00:23.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.191+0000 7f0f9b150700 1 -- 172.21.15.46:0/681747066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f941a0090 con 0x7f0f94108760 2024-05-09T05:00:23.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.191+0000 7f0f91ffb700 1 -- 172.21.15.46:0/681747066 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8c005920 con 0x7f0f94108760 2024-05-09T05:00:23.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.191+0000 7f0f9b150700 1 -- 172.21.15.46:0/681747066 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f941a79b0 con 0x7f0f94108760 2024-05-09T05:00:23.193 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.193+0000 7f0f91ffb700 1 -- 172.21.15.46:0/681747066 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0f8c01a020 con 0x7f0f94108760 2024-05-09T05:00:23.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.194+0000 7f0f91ffb700 1 --2- 172.21.15.46:0/681747066 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0f7c0b29f0 0x7f0f7c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.194+0000 7f0f98eec700 1 --2- 172.21.15.46:0/681747066 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0f7c0b29f0 0x7f0f7c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.194 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.194+0000 7f0f91ffb700 1 -- 172.21.15.46:0/681747066 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f0f8c0e1d20 con 0x7f0f94108760 2024-05-09T05:00:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.194+0000 7f0fa9b66b80 1 Processor -- start 2024-05-09T05:00:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0fa9b66b80 1 -- start start 2024-05-09T05:00:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0f98eec700 1 --2- 172.21.15.46:0/681747066 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0f7c0b29f0 0x7f0f7c0b4f00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009a10 tx=0x7f0f84009190 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0fa9b66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d749daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0fa9b66b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d73124a0 0x5609d749e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0fa9b66b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d748f400 0x5609d748f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0fa9b66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5609d73b20a0 con 0x5609d738cd00 2024-05-09T05:00:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0fa9b66b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5609d73579a0 con 0x5609d73124a0 2024-05-09T05:00:23.196 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0fa9b66b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5609d73ae060 con 0x5609d748f400 2024-05-09T05:00:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0f98eec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d749daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0f98eec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d749daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43448/0 (socket says 172.21.15.46:43448) 2024-05-09T05:00:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0f98eec700 1 -- 172.21.15.46:0/1289457287 learned_addr learned my addr 172.21.15.46:0/1289457287 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0f93fff700 1 --2- 172.21.15.46:0/1289457287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d73124a0 0x5609d749e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0f996ed700 1 --2- 172.21.15.46:0/1289457287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d748f400 0x5609d748f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0f98eec700 1 -- 172.21.15.46:0/1289457287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d748f400 msgr2=0x5609d748f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0f98eec700 1 --2- 172.21.15.46:0/1289457287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d748f400 0x5609d748f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0f98eec700 1 -- 172.21.15.46:0/1289457287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d73124a0 msgr2=0x5609d749e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0f98eec700 1 --2- 172.21.15.46:0/1289457287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d73124a0 0x5609d749e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0f98eec700 1 -- 172.21.15.46:0/1289457287 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609d7348500 con 0x5609d738cd00 2024-05-09T05:00:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.195+0000 7f0f93fff700 1 --2- 172.21.15.46:0/1289457287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d73124a0 0x5609d749e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:00:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.196+0000 7f0f98eec700 1 --2- 172.21.15.46:0/1289457287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d749daf0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400ea00 tx=0x7f0f84011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9157032389fb543 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:23.197 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.196+0000 7f0f7affd700 1 -- 172.21.15.46:0/1289457287 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8401b040 con 0x5609d738cd00 2024-05-09T05:00:23.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.196+0000 7f0f7affd700 1 -- 172.21.15.46:0/1289457287 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f84024070 con 0x5609d738cd00 2024-05-09T05:00:23.198 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.196+0000 7f0f7affd700 1 -- 172.21.15.46:0/1289457287 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f840136b0 con 0x5609d738cd00 2024-05-09T05:00:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.197+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/1289457287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 msgr2=0x5609d749daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.197+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/1289457287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d749daf0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400ea00 tx=0x7f0f84011000 comp rx=0 tx=0).stop 2024-05-09T05:00:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.197+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/1289457287 shutdown_connections 2024-05-09T05:00:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.197+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/1289457287 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d748f400 0x5609d748f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.197+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/1289457287 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d749daf0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.197+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/1289457287 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d73124a0 0x5609d749e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.197+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/1289457287 >> 172.21.15.46:0/1289457287 conn(0x5609d7315f70 msgr2=0x5609d748a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:23.204 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.197+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/1289457287 shutdown_connections 2024-05-09T05:00:23.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.197+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/1289457287 wait complete. 2024-05-09T05:00:23.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.197+0000 7f0fa9b66b80 1 -- start start 2024-05-09T05:00:23.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.197+0000 7f0fa9b66b80 1 Processor -- start 2024-05-09T05:00:23.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0fa9b66b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d73124a0 0x5609d74a1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0fa9b66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d74a1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0fa9b66b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d748f400 0x5609d74a22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0fa9b66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5609d7490150 con 0x5609d738cd00 2024-05-09T05:00:23.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0fa9b66b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5609d734cb20 con 0x5609d73124a0 2024-05-09T05:00:23.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0fa9b66b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5609d74a4840 con 0x5609d748f400 2024-05-09T05:00:23.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0fa9b66b80 10 client.0 _ll_register_callbacks cb 0x5609d72236c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:23.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0f93fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d74a1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0f98eec700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d73124a0 0x5609d74a1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.205 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0f98eec700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d73124a0 0x5609d74a1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:53614/0 (socket says 172.21.15.46:53614) 2024-05-09T05:00:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0f98eec700 1 -- 172.21.15.46:0/572812275 learned_addr learned my addr 172.21.15.46:0/572812275 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0f996ed700 1 --2- 172.21.15.46:0/572812275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d748f400 0x5609d74a22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0f98eec700 1 -- 172.21.15.46:0/572812275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d748f400 msgr2=0x5609d74a22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0f98eec700 1 --2- 172.21.15.46:0/572812275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d748f400 0x5609d74a22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0f98eec700 1 -- 172.21.15.46:0/572812275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 msgr2=0x5609d74a1da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.198+0000 7f0f98eec700 1 --2- 172.21.15.46:0/572812275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d74a1da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.199+0000 7f0f98eec700 1 -- 172.21.15.46:0/572812275 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609d722d490 con 0x5609d73124a0 2024-05-09T05:00:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.199+0000 7f0f93fff700 1 --2- 172.21.15.46:0/572812275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d74a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:00:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.199+0000 7f0f98eec700 1 --2- 172.21.15.46:0/572812275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d73124a0 0x5609d74a1860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400ed10 tx=0x7f0f84015990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e3ba34101c93d84 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.199+0000 7f0f73fff700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T05:00:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.199+0000 7f0f73fff700 1 -- 172.21.15.46:0/572812275 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8402d070 con 0x5609d73124a0 2024-05-09T05:00:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.199+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/572812275 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f84024070 con 0x5609d73124a0 2024-05-09T05:00:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.199+0000 7f0f73fff700 1 -- 172.21.15.46:0/572812275 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f0f84029810 con 0x5609d73124a0 2024-05-09T05:00:23.206 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.199+0000 7f0f73fff700 1 -- 172.21.15.46:0/572812275 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0f84029a10 con 0x5609d73124a0 2024-05-09T05:00:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.199+0000 7f0fa9b66b80 10 client.26813 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.199+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/572812275 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5609d74af230 con 0x5609d73124a0 2024-05-09T05:00:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.200+0000 7f0f73fff700 1 -- 172.21.15.46:0/572812275 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f84024070 con 0x5609d73124a0 2024-05-09T05:00:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.200+0000 7f0f73fff700 1 -- 172.21.15.46:0/572812275 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f0f8401e030 con 0x5609d73124a0 2024-05-09T05:00:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.200+0000 7f0fa9b66b80 10 client.26813 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.200+0000 7f0fa9b66b80 10 client.26813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.200+0000 7f0fa9b66b80 20 client.26813 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.200+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/572812275 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5609d74b1870 0x5609d74b3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.200+0000 7f0fa9b66b80 4 client.26813 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:00:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.200+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/572812275 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5609d73bfa80 con 0x5609d74b1870 2024-05-09T05:00:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.200+0000 7f0f93fff700 1 --2- 172.21.15.46:0/572812275 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5609d74b1870 0x5609d74b3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.201+0000 7f0f93fff700 1 --2- 172.21.15.46:0/572812275 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5609d74b1870 0x5609d74b3c90 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95487320b9dd8f36 server_cookie=b88b7b4c386c941a in_seq=0 out_seq=0 2024-05-09T05:00:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.201+0000 7f0f73fff700 10 client.26813 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.201+0000 7f0f73fff700 1 -- 172.21.15.46:0/572812275 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5609d73bfa80 con 0x5609d74b1870 2024-05-09T05:00:23.207 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.201+0000 7f0f73fff700 10 client.26813 handle_command_reply: tid=0 2024-05-09T05:00:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.201+0000 7f0fa9b66b80 1 client.26813 shutdown 2024-05-09T05:00:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.202+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/572812275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d73124a0 msgr2=0x5609d74a1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.202+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/572812275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d73124a0 0x5609d74a1860 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400ed10 tx=0x7f0f84015990 comp rx=0 tx=0).stop 2024-05-09T05:00:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.202+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/572812275 shutdown_connections 2024-05-09T05:00:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.202+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/572812275 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d748f400 0x5609d74a22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.202+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/572812275 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5609d74b1870 0x5609d74b3c90 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.202+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/572812275 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d73124a0 0x5609d74a1860 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.202+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/572812275 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d74a1da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.202+0000 7f0f73fff700 0 client.26813 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.202+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/572812275 >> 172.21.15.46:0/572812275 conn(0x5609d7315f70 msgr2=0x5609d7483ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.202+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/572812275 shutdown_connections 2024-05-09T05:00:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.202+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/572812275 wait complete. 2024-05-09T05:00:23.208 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.202+0000 7f0fa9b66b80 20 client.26813 trim_cache size 0 max 16384 2024-05-09T05:00:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.222+0000 7f0fa9b66b80 1 Processor -- start 2024-05-09T05:00:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.222+0000 7f0fa9b66b80 1 -- start start 2024-05-09T05:00:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0fa9b66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d73124a0 0x5609d748ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0fa9b66b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d738cd00 0x5609d74ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0fa9b66b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d748f400 0x5609d74a2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0fa9b66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941af660 con 0x5609d73124a0 2024-05-09T05:00:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0fa9b66b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941acef0 con 0x5609d748f400 2024-05-09T05:00:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0fa9b66b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941aad60 con 0x5609d738cd00 2024-05-09T05:00:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0f98eec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d73124a0 0x5609d748ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0f98eec700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d73124a0 0x5609d748ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43474/0 (socket says 172.21.15.46:43474) 2024-05-09T05:00:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0f98eec700 1 -- 172.21.15.46:0/2260423590 learned_addr learned my addr 172.21.15.46:0/2260423590 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:23.224 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0f93fff700 1 --2- 172.21.15.46:0/2260423590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d738cd00 0x5609d74ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0f98eec700 1 -- 172.21.15.46:0/2260423590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d738cd00 msgr2=0x5609d74ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0f98eec700 1 --2- 172.21.15.46:0/2260423590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d738cd00 0x5609d74ad9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0f98eec700 1 -- 172.21.15.46:0/2260423590 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d748f400 msgr2=0x5609d74a2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0f996ed700 1 --2- 172.21.15.46:0/2260423590 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d748f400 0x5609d74a2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0f98eec700 1 --2- 172.21.15.46:0/2260423590 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d748f400 0x5609d74a2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0f98eec700 1 -- 172.21.15.46:0/2260423590 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5609d748bea0 con 0x5609d73124a0 2024-05-09T05:00:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0f98eec700 1 --2- 172.21.15.46:0/2260423590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d73124a0 0x5609d748ee20 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400e4b0 tx=0x7f0f84006ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8d7d5f9e0e699a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.223+0000 7f0f7a7fc700 1 -- 172.21.15.46:0/2260423590 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f8402d070 con 0x5609d73124a0 2024-05-09T05:00:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.224+0000 7f0f7a7fc700 1 -- 172.21.15.46:0/2260423590 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f84024040 con 0x5609d73124a0 2024-05-09T05:00:23.225 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.224+0000 7f0f7a7fc700 1 -- 172.21.15.46:0/2260423590 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f84029e10 con 0x5609d73124a0 2024-05-09T05:00:23.229 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.224+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/2260423590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d73124a0 msgr2=0x5609d748ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.224+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/2260423590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d73124a0 0x5609d748ee20 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f0f8400e4b0 tx=0x7f0f84006ed0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.224+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/2260423590 shutdown_connections 2024-05-09T05:00:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.224+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/2260423590 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d748f400 0x5609d74a2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.224+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/2260423590 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d73124a0 0x5609d748ee20 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.224+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/2260423590 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d738cd00 0x5609d74ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.224+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/2260423590 >> 172.21.15.46:0/2260423590 conn(0x5609d7315f70 msgr2=0x5609d749c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.224+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/2260423590 shutdown_connections 2024-05-09T05:00:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.224+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/2260423590 wait complete. 2024-05-09T05:00:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.225+0000 7f0fa9b66b80 1 -- start start 2024-05-09T05:00:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.225+0000 7f0fa9b66b80 1 Processor -- start 2024-05-09T05:00:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.225+0000 7f0fa9b66b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d73124a0 0x5609d74b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.225+0000 7f0fa9b66b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d74b4230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.225+0000 7f0fa9b66b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d748f400 0x5609d7513d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.225+0000 7f0fa9b66b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941ab3b0 con 0x5609d738cd00 2024-05-09T05:00:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0fa9b66b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941ac820 con 0x5609d748f400 2024-05-09T05:00:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0fa9b66b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0f941aba80 con 0x5609d73124a0 2024-05-09T05:00:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0fa9b66b80 10 client.0 _ll_register_callbacks cb 0x5609d72236c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0f93fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d74b4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0f93fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d74b4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:43484/0 (socket says 172.21.15.46:43484) 2024-05-09T05:00:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0f93fff700 1 -- 172.21.15.46:0/3142091713 learned_addr learned my addr 172.21.15.46:0/3142091713 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0f996ed700 1 --2- 172.21.15.46:0/3142091713 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d748f400 0x5609d7513d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0f98eec700 1 --2- 172.21.15.46:0/3142091713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d73124a0 0x5609d74b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0f93fff700 1 -- 172.21.15.46:0/3142091713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d73124a0 msgr2=0x5609d74b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0f93fff700 1 --2- 172.21.15.46:0/3142091713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d73124a0 0x5609d74b3cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0f93fff700 1 -- 172.21.15.46:0/3142091713 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d748f400 msgr2=0x5609d7513d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0f93fff700 1 --2- 172.21.15.46:0/3142091713 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d748f400 0x5609d7513d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0f93fff700 1 -- 172.21.15.46:0/3142091713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609d7514310 con 0x5609d738cd00 2024-05-09T05:00:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0f93fff700 1 --2- 172.21.15.46:0/3142091713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d74b4230 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f0f88005b40 tx=0x7f0f88009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a431e13cfcb0083f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:23.231 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0f73fff700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0f73fff700 1 -- 172.21.15.46:0/3142091713 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f88005510 con 0x5609d738cd00 2024-05-09T05:00:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0f73fff700 1 -- 172.21.15.46:0/3142091713 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f0f8800b730 con 0x5609d738cd00 2024-05-09T05:00:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.226+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/3142091713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0f84024040 con 0x5609d738cd00 2024-05-09T05:00:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.227+0000 7f0f73fff700 1 -- 172.21.15.46:0/3142091713 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0f880056b0 con 0x5609d738cd00 2024-05-09T05:00:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.227+0000 7f0fa9b66b80 10 client.17307 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.227+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/3142091713 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5609d751b620 con 0x5609d738cd00 2024-05-09T05:00:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.227+0000 7f0f73fff700 1 -- 172.21.15.46:0/3142091713 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f84024040 con 0x5609d738cd00 2024-05-09T05:00:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.227+0000 7f0f73fff700 1 -- 172.21.15.46:0/3142091713 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f0f88029760 con 0x5609d738cd00 2024-05-09T05:00:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.227+0000 7f0fa9b66b80 10 client.17307 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.227+0000 7f0fa9b66b80 10 client.17307 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.227+0000 7f0fa9b66b80 20 client.17307 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.227+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/3142091713 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5609d751bd60 0x5609d751e180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.227+0000 7f0fa9b66b80 4 client.17307 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:00:23.232 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.227+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/3142091713 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5609d73b9b20 con 0x5609d751bd60 2024-05-09T05:00:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.227+0000 7f0f98eec700 1 --2- 172.21.15.46:0/3142091713 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5609d751bd60 0x5609d751e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.228+0000 7f0f73fff700 10 client.17307 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.228+0000 7f0f98eec700 1 --2- 172.21.15.46:0/3142091713 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5609d751bd60 0x5609d751e180 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b86784aac6d2a4c6 server_cookie=7128a5808274e58 in_seq=0 out_seq=0 2024-05-09T05:00:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.228+0000 7f0f73fff700 1 -- 172.21.15.46:0/3142091713 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5609d73b9b20 con 0x5609d751bd60 2024-05-09T05:00:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.228+0000 7f0f73fff700 10 client.17307 handle_command_reply: tid=0 2024-05-09T05:00:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.228+0000 7f0fa9b66b80 1 client.17307 shutdown 2024-05-09T05:00:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.228+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/3142091713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 msgr2=0x5609d74b4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.228+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/3142091713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d74b4230 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f0f88005b40 tx=0x7f0f88009310 comp rx=0 tx=0).stop 2024-05-09T05:00:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.228+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/3142091713 shutdown_connections 2024-05-09T05:00:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.228+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/3142091713 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5609d751bd60 0x5609d751e180 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.228+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/3142091713 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5609d748f400 0x5609d7513d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.228+0000 7f0f73fff700 0 client.17307 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.228+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/3142091713 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5609d73124a0 0x5609d74b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.228+0000 7f0fa9b66b80 1 --2- 172.21.15.46:0/3142091713 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5609d738cd00 0x5609d74b4230 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.228+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/3142091713 >> 172.21.15.46:0/3142091713 conn(0x5609d7315f70 msgr2=0x5609d749c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.229+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/3142091713 shutdown_connections 2024-05-09T05:00:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.229+0000 7f0fa9b66b80 1 -- 172.21.15.46:0/3142091713 wait complete. 2024-05-09T05:00:23.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.229+0000 7f0fa9b66b80 20 client.17307 trim_cache size 0 max 16384 2024-05-09T05:00:23.234 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T05:00:23.234 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T05:00:23.234 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T05:00:23.234 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T05:00:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.229+0000 7f0f9b150700 1 -- 172.21.15.46:0/681747066 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0f7c0b29f0 msgr2=0x7f0f7c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.229+0000 7f0f9b150700 1 --2- 172.21.15.46:0/681747066 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0f7c0b29f0 0x7f0f7c0b4f00 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f0f84009a10 tx=0x7f0f84009190 comp rx=0 tx=0).stop 2024-05-09T05:00:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.229+0000 7f0f9b150700 1 -- 172.21.15.46:0/681747066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f94108760 msgr2=0x7f0f9419f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.229+0000 7f0f9b150700 1 --2- 172.21.15.46:0/681747066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f94108760 0x7f0f9419f700 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c007ee0 tx=0x7f0f8c00d3b0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.229+0000 7f0f9b150700 1 -- 172.21.15.46:0/681747066 shutdown_connections 2024-05-09T05:00:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.230+0000 7f0f9b150700 1 --2- 172.21.15.46:0/681747066 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f0f7c0b29f0 0x7f0f7c0b4f00 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.230+0000 7f0f9b150700 1 --2- 172.21.15.46:0/681747066 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0f94108760 0x7f0f9419f700 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.230+0000 7f0f9b150700 1 --2- 172.21.15.46:0/681747066 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0f941034e0 0x7f0f9419aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.230+0000 7f0f9b150700 1 --2- 172.21.15.46:0/681747066 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0f94107e10 0x7f0f9419b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.230+0000 7f0f9b150700 1 -- 172.21.15.46:0/681747066 >> 172.21.15.46:0/681747066 conn(0x7f0f940fef30 msgr2=0x7f0f9406c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:23.235 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.230+0000 7f0f9b150700 1 -- 172.21.15.46:0/681747066 shutdown_connections 2024-05-09T05:00:23.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:23.230+0000 7f0f9b150700 1 -- 172.21.15.46:0/681747066 wait complete. 2024-05-09T05:00:24.319 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T05:00:24.319 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:00:24.577 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:24.954 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:24 smithi046 ceph-mon[30136]: pgmap v288: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-05-09T05:00:25.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:24 smithi110 ceph-mon[34904]: pgmap v288: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-05-09T05:00:25.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:24 smithi072 ceph-mon[34088]: pgmap v288: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-05-09T05:00:26.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:25 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:00:26.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:25 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:00:26.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:25 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:00:26.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.682+0000 7fe396ebb700 1 -- 172.21.15.46:0/2227849592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe390107df0 msgr2=0x7fe390108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:26.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.682+0000 7fe396ebb700 1 --2- 172.21.15.46:0/2227849592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe390107df0 0x7fe390108200 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fe37c0099c0 tx=0x7fe37c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:00:26.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.682+0000 7fe396ebb700 1 -- 172.21.15.46:0/2227849592 shutdown_connections 2024-05-09T05:00:26.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.682+0000 7fe396ebb700 1 --2- 172.21.15.46:0/2227849592 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe390108740 0x7fe3901100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:26.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.682+0000 7fe396ebb700 1 --2- 172.21.15.46:0/2227849592 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe3901034c0 0x7fe3901038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:26.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.682+0000 7fe396ebb700 1 --2- 172.21.15.46:0/2227849592 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe390107df0 0x7fe390108200 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:26.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.682+0000 7fe396ebb700 1 -- 172.21.15.46:0/2227849592 >> 172.21.15.46:0/2227849592 conn(0x7fe3900fedf0 msgr2=0x7fe390101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:26.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.682+0000 7fe396ebb700 1 -- 172.21.15.46:0/2227849592 shutdown_connections 2024-05-09T05:00:26.683 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.682+0000 7fe396ebb700 1 -- 172.21.15.46:0/2227849592 wait complete. 2024-05-09T05:00:26.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.683+0000 7fe396ebb700 1 Processor -- start 2024-05-09T05:00:26.684 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.683+0000 7fe396ebb700 1 -- start start 2024-05-09T05:00:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.683+0000 7fe396ebb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3901034c0 0x7fe390199d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.683+0000 7fe396ebb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe390107df0 0x7fe39019a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.683+0000 7fe396ebb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe390108740 0x7fe39019e550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.683+0000 7fe396ebb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe390113370 con 0x7fe390107df0 2024-05-09T05:00:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.683+0000 7fe396ebb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe3901131f0 con 0x7fe390108740 2024-05-09T05:00:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.683+0000 7fe396ebb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe3901134f0 con 0x7fe3901034c0 2024-05-09T05:00:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.683+0000 7fe38ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe390107df0 0x7fe39019a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.683+0000 7fe38ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe390107df0 0x7fe39019a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36814/0 (socket says 172.21.15.46:36814) 2024-05-09T05:00:26.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.683+0000 7fe38ffff700 1 -- 172.21.15.46:0/449210841 learned_addr learned my addr 172.21.15.46:0/449210841 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.683+0000 7fe394c57700 1 --2- 172.21.15.46:0/449210841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3901034c0 0x7fe390199d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.684+0000 7fe395458700 1 --2- 172.21.15.46:0/449210841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe390108740 0x7fe39019e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.684+0000 7fe38ffff700 1 -- 172.21.15.46:0/449210841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3901034c0 msgr2=0x7fe390199d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.684+0000 7fe38ffff700 1 --2- 172.21.15.46:0/449210841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3901034c0 0x7fe390199d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.684+0000 7fe38ffff700 1 -- 172.21.15.46:0/449210841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe390108740 msgr2=0x7fe39019e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.684+0000 7fe38ffff700 1 --2- 172.21.15.46:0/449210841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe390108740 0x7fe39019e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.684+0000 7fe38ffff700 1 -- 172.21.15.46:0/449210841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe39019ebf0 con 0x7fe390107df0 2024-05-09T05:00:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.684+0000 7fe38ffff700 1 --2- 172.21.15.46:0/449210841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe390107df0 0x7fe39019a2c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe37c006010 tx=0x7fe37c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.684+0000 7fe38dffb700 1 -- 172.21.15.46:0/449210841 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c027070 con 0x7fe390107df0 2024-05-09T05:00:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.684+0000 7fe38dffb700 1 -- 172.21.15.46:0/449210841 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe37c003f20 con 0x7fe390107df0 2024-05-09T05:00:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.684+0000 7fe396ebb700 1 -- 172.21.15.46:0/449210841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe39019ee80 con 0x7fe390107df0 2024-05-09T05:00:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.684+0000 7fe38dffb700 1 -- 172.21.15.46:0/449210841 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe37c022cd0 con 0x7fe390107df0 2024-05-09T05:00:26.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.684+0000 7fe396ebb700 1 -- 172.21.15.46:0/449210841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe39019f360 con 0x7fe390107df0 2024-05-09T05:00:26.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.686+0000 7fe396ebb700 1 -- 172.21.15.46:0/449210841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3900512c0 con 0x7fe390107df0 2024-05-09T05:00:26.687 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.687+0000 7fe38dffb700 1 -- 172.21.15.46:0/449210841 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe37c02a070 con 0x7fe390107df0 2024-05-09T05:00:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.688+0000 7fe38dffb700 1 --2- 172.21.15.46:0/449210841 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe3800b29f0 0x7fe3800b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.688+0000 7fe394c57700 1 --2- 172.21.15.46:0/449210841 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe3800b29f0 0x7fe3800b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.688+0000 7fe38dffb700 1 -- 172.21.15.46:0/449210841 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7fe37c0e3960 con 0x7fe390107df0 2024-05-09T05:00:26.688 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.689+0000 7fe394c57700 1 --2- 172.21.15.46:0/449210841 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe3800b29f0 0x7fe3800b4f00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe384005950 tx=0x7fe38400b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:26.692 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.692+0000 7fe38dffb700 1 -- 172.21.15.46:0/449210841 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe37c031080 con 0x7fe390107df0 2024-05-09T05:00:26.955 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.955+0000 7fe396ebb700 1 -- 172.21.15.46:0/449210841 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe39002e5f0 con 0x7fe390107df0 2024-05-09T05:00:26.956 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:26 smithi046 ceph-mon[30136]: pgmap v289: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-05-09T05:00:26.956 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:26 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:00:26.956 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:26 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:00:26.956 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:26 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:00:26.957 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.956+0000 7fe38dffb700 1 -- 172.21.15.46:0/449210841 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7fe37c013070 con 0x7fe390107df0 2024-05-09T05:00:26.957 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T05:00:26.957 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T05:00:26.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.959+0000 7fe396ebb700 1 -- 172.21.15.46:0/449210841 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe3800b29f0 msgr2=0x7fe3800b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:26.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.959+0000 7fe396ebb700 1 --2- 172.21.15.46:0/449210841 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe3800b29f0 0x7fe3800b4f00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe384005950 tx=0x7fe38400b410 comp rx=0 tx=0).stop 2024-05-09T05:00:26.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.959+0000 7fe396ebb700 1 -- 172.21.15.46:0/449210841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe390107df0 msgr2=0x7fe39019a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:26.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.959+0000 7fe396ebb700 1 --2- 172.21.15.46:0/449210841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe390107df0 0x7fe39019a2c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe37c006010 tx=0x7fe37c0047b0 comp rx=0 tx=0).stop 2024-05-09T05:00:26.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.959+0000 7fe396ebb700 1 -- 172.21.15.46:0/449210841 shutdown_connections 2024-05-09T05:00:26.959 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.959+0000 7fe396ebb700 1 --2- 172.21.15.46:0/449210841 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe3800b29f0 0x7fe3800b4f00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:26.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.959+0000 7fe396ebb700 1 --2- 172.21.15.46:0/449210841 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe390108740 0x7fe39019e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:26.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.959+0000 7fe396ebb700 1 --2- 172.21.15.46:0/449210841 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe3901034c0 0x7fe390199d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:26.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.959+0000 7fe396ebb700 1 --2- 172.21.15.46:0/449210841 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe390107df0 0x7fe39019a2c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:26.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.959+0000 7fe396ebb700 1 -- 172.21.15.46:0/449210841 >> 172.21.15.46:0/449210841 conn(0x7fe3900fedf0 msgr2=0x7fe3901096a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:26.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.959+0000 7fe396ebb700 1 -- 172.21.15.46:0/449210841 shutdown_connections 2024-05-09T05:00:26.960 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:26.959+0000 7fe396ebb700 1 -- 172.21.15.46:0/449210841 wait complete. 2024-05-09T05:00:26.960 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T05:00:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:26 smithi072 ceph-mon[34088]: pgmap v289: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-05-09T05:00:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:26 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:00:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:26 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:00:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:26 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:00:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:26 smithi110 ceph-mon[34904]: pgmap v289: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 204 B/s wr, 7 op/s 2024-05-09T05:00:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:26 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:00:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:26 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:00:27.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:26 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:00:27.676 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T05:00:27.935 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:28.058 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:27 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/449210841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:00:28.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:27 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/449210841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:00:28.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:27 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/449210841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:00:29.052 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:28 smithi046 ceph-mon[30136]: pgmap v290: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T05:00:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:28 smithi072 ceph-mon[34088]: pgmap v290: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T05:00:29.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:28 smithi110 ceph-mon[34904]: pgmap v290: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T05:00:29.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.558+0000 7f76007f0700 1 -- 172.21.15.46:0/2758857824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f8104610 msgr2=0x7f75f8104a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:29.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.558+0000 7f76007f0700 1 --2- 172.21.15.46:0/2758857824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f8104610 0x7f75f8104a20 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f75f40099c0 tx=0x7f75f4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.558+0000 7f76007f0700 1 -- 172.21.15.46:0/2758857824 shutdown_connections 2024-05-09T05:00:29.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.558+0000 7f76007f0700 1 --2- 172.21.15.46:0/2758857824 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75f8109890 0x7f75f81111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.558+0000 7f76007f0700 1 --2- 172.21.15.46:0/2758857824 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f8104610 0x7f75f8104a20 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.558+0000 7f76007f0700 1 --2- 172.21.15.46:0/2758857824 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75f8108f40 0x7f75f8109350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.558+0000 7f76007f0700 1 -- 172.21.15.46:0/2758857824 >> 172.21.15.46:0/2758857824 conn(0x7f75f807af70 msgr2=0x7f75f807b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:29.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.559+0000 7f76007f0700 1 -- 172.21.15.46:0/2758857824 shutdown_connections 2024-05-09T05:00:29.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.559+0000 7f76007f0700 1 -- 172.21.15.46:0/2758857824 wait complete. 2024-05-09T05:00:29.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.559+0000 7f76007f0700 1 Processor -- start 2024-05-09T05:00:29.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.559+0000 7f76007f0700 1 -- start start 2024-05-09T05:00:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f76007f0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75f8104610 0x7f75f819af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f76007f0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f8108f40 0x7f75f819b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f76007f0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75f8109890 0x7f75f819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f76007f0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75f81145b0 con 0x7f75f8108f40 2024-05-09T05:00:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f76007f0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f75f8114430 con 0x7f75f8104610 2024-05-09T05:00:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f76007f0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f75f8114730 con 0x7f75f8109890 2024-05-09T05:00:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f75fdd8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f8108f40 0x7f75f819b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f75fdd8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f8108f40 0x7f75f819b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36830/0 (socket says 172.21.15.46:36830) 2024-05-09T05:00:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f75fdd8b700 1 -- 172.21.15.46:0/3708687740 learned_addr learned my addr 172.21.15.46:0/3708687740 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f75fe58c700 1 --2- 172.21.15.46:0/3708687740 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75f8104610 0x7f75f819af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.562 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f75fed8d700 1 --2- 172.21.15.46:0/3708687740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75f8109890 0x7f75f819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f75fdd8b700 1 -- 172.21.15.46:0/3708687740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75f8109890 msgr2=0x7f75f819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:29.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f75fdd8b700 1 --2- 172.21.15.46:0/3708687740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75f8109890 0x7f75f819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f75fdd8b700 1 -- 172.21.15.46:0/3708687740 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75f8104610 msgr2=0x7f75f819af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:29.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f75fdd8b700 1 --2- 172.21.15.46:0/3708687740 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75f8104610 0x7f75f819af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f75fdd8b700 1 -- 172.21.15.46:0/3708687740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75f819fe10 con 0x7f75f8108f40 2024-05-09T05:00:29.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f75fdd8b700 1 --2- 172.21.15.46:0/3708687740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f8108f40 0x7f75f819b4e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f75e800b500 tx=0x7f75e800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:29.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.560+0000 7f75ef7fe700 1 -- 172.21.15.46:0/3708687740 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8015020 con 0x7f75f8108f40 2024-05-09T05:00:29.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.561+0000 7f75ef7fe700 1 -- 172.21.15.46:0/3708687740 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75e8011ce0 con 0x7f75f8108f40 2024-05-09T05:00:29.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.561+0000 7f76007f0700 1 -- 172.21.15.46:0/3708687740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75f81a0100 con 0x7f75f8108f40 2024-05-09T05:00:29.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.561+0000 7f75ef7fe700 1 -- 172.21.15.46:0/3708687740 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75e8010d40 con 0x7f75f8108f40 2024-05-09T05:00:29.563 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.561+0000 7f76007f0700 1 -- 172.21.15.46:0/3708687740 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75f81a7a20 con 0x7f75f8108f40 2024-05-09T05:00:29.564 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.563+0000 7f75ef7fe700 1 -- 172.21.15.46:0/3708687740 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f75e8018020 con 0x7f75f8108f40 2024-05-09T05:00:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.564+0000 7f75ef7fe700 1 --2- 172.21.15.46:0/3708687740 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f75e40b29f0 0x7f75e40b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.565+0000 7f75fe58c700 1 --2- 172.21.15.46:0/3708687740 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f75e40b29f0 0x7f75e40b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.565+0000 7f75ef7fe700 1 -- 172.21.15.46:0/3708687740 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f75e80e1d10 con 0x7f75f8108f40 2024-05-09T05:00:29.565 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.565+0000 7f760f206b80 1 Processor -- start 2024-05-09T05:00:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.565+0000 7f760f206b80 1 -- start start 2024-05-09T05:00:29.566 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.565+0000 7f75fe58c700 1 --2- 172.21.15.46:0/3708687740 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f75e40b29f0 0x7f75e40b4f00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009990 tx=0x7f75f4000bc0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.565+0000 7f760f206b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb71100d00 0x55eb7120d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.565+0000 7f760f206b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb710864a0 0x55eb7120db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.565+0000 7f760f206b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb71207c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.565+0000 7f760f206b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55eb711260a0 con 0x55eb71207810 2024-05-09T05:00:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.565+0000 7f760f206b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55eb710cb9a0 con 0x55eb71100d00 2024-05-09T05:00:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.565+0000 7f760f206b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eb71122060 con 0x55eb710864a0 2024-05-09T05:00:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.565+0000 7f75fed8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb71207c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.565+0000 7f75fed8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb71207c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36840/0 (socket says 172.21.15.46:36840) 2024-05-09T05:00:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.565+0000 7f75fed8d700 1 -- 172.21.15.46:0/386225217 learned_addr learned my addr 172.21.15.46:0/386225217 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.566+0000 7f75fdd8b700 1 --2- 172.21.15.46:0/386225217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb710864a0 0x55eb7120db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.566+0000 7f75fe58c700 1 --2- 172.21.15.46:0/386225217 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb71100d00 0x55eb7120d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.566+0000 7f75fed8d700 1 -- 172.21.15.46:0/386225217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb710864a0 msgr2=0x55eb7120db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:29.567 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.566+0000 7f75fed8d700 1 --2- 172.21.15.46:0/386225217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb710864a0 0x55eb7120db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.566+0000 7f75fed8d700 1 -- 172.21.15.46:0/386225217 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb71100d00 msgr2=0x55eb7120d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:29.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.566+0000 7f75fed8d700 1 --2- 172.21.15.46:0/386225217 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb71100d00 0x55eb7120d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.566+0000 7f75fed8d700 1 -- 172.21.15.46:0/386225217 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb710bc500 con 0x55eb71207810 2024-05-09T05:00:29.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.566+0000 7f75fed8d700 1 --2- 172.21.15.46:0/386225217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb71207c20 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009f20 tx=0x7f75f0009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c88de7e4992e8e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:29.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.566+0000 7f75ecff9700 1 -- 172.21.15.46:0/386225217 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f001a070 con 0x55eb71207810 2024-05-09T05:00:29.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.566+0000 7f75ecff9700 1 -- 172.21.15.46:0/386225217 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75f0015410 con 0x55eb71207810 2024-05-09T05:00:29.568 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.566+0000 7f75ecff9700 1 -- 172.21.15.46:0/386225217 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f000a600 con 0x55eb71207810 2024-05-09T05:00:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.567+0000 7f760f206b80 1 -- 172.21.15.46:0/386225217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 msgr2=0x55eb71207c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.567+0000 7f760f206b80 1 --2- 172.21.15.46:0/386225217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb71207c20 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f75f0009f20 tx=0x7f75f0009f50 comp rx=0 tx=0).stop 2024-05-09T05:00:29.571 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.567+0000 7f760f206b80 1 -- 172.21.15.46:0/386225217 shutdown_connections 2024-05-09T05:00:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.567+0000 7f760f206b80 1 --2- 172.21.15.46:0/386225217 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb71207c20 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.567+0000 7f760f206b80 1 --2- 172.21.15.46:0/386225217 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb71100d00 0x55eb7120d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.567+0000 7f760f206b80 1 --2- 172.21.15.46:0/386225217 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb710864a0 0x55eb7120db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.567+0000 7f760f206b80 1 -- 172.21.15.46:0/386225217 >> 172.21.15.46:0/386225217 conn(0x55eb71089f70 msgr2=0x55eb711fe220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.567+0000 7f760f206b80 1 -- 172.21.15.46:0/386225217 shutdown_connections 2024-05-09T05:00:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.567+0000 7f760f206b80 1 -- 172.21.15.46:0/386225217 wait complete. 2024-05-09T05:00:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.568+0000 7f760f206b80 1 -- start start 2024-05-09T05:00:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.568+0000 7f760f206b80 1 Processor -- start 2024-05-09T05:00:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.568+0000 7f760f206b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb710864a0 0x55eb71211490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.568+0000 7f760f206b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 0x55eb712119d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.568+0000 7f760f206b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb71207810 0x55eb71211f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.568+0000 7f760f206b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55eb71208570 con 0x55eb710864a0 2024-05-09T05:00:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.568+0000 7f760f206b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55eb710c0b20 con 0x55eb71207810 2024-05-09T05:00:29.572 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.568+0000 7f760f206b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55eb71214430 con 0x55eb71100d00 2024-05-09T05:00:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.568+0000 7f75fe58c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb710864a0 0x55eb71211490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.568+0000 7f75fe58c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb710864a0 0x55eb71211490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36850/0 (socket says 172.21.15.46:36850) 2024-05-09T05:00:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.568+0000 7f75fe58c700 1 -- 172.21.15.46:0/4241448578 learned_addr learned my addr 172.21.15.46:0/4241448578 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.568+0000 7f75fdd8b700 1 --2- 172.21.15.46:0/4241448578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 0x55eb712119d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.568+0000 7f760f206b80 10 client.0 _ll_register_callbacks cb 0x55eb70f976c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.569+0000 7f75fed8d700 1 --2- 172.21.15.46:0/4241448578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb71207810 0x55eb71211f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.569+0000 7f75fe58c700 1 -- 172.21.15.46:0/4241448578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 msgr2=0x55eb712119d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.569+0000 7f75fe58c700 1 --2- 172.21.15.46:0/4241448578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 0x55eb712119d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.569+0000 7f75fe58c700 1 -- 172.21.15.46:0/4241448578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb71207810 msgr2=0x55eb71211f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.569+0000 7f75fe58c700 1 --2- 172.21.15.46:0/4241448578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb71207810 0x55eb71211f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.569+0000 7f75fe58c700 1 -- 172.21.15.46:0/4241448578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb70fa1490 con 0x55eb710864a0 2024-05-09T05:00:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.569+0000 7f75fed8d700 1 --2- 172.21.15.46:0/4241448578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb71207810 0x55eb71211f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.569+0000 7f75fe58c700 1 --2- 172.21.15.46:0/4241448578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb710864a0 0x55eb71211490 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f75f4005780 tx=0x7f75f401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbf53f24d446e3f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:29.573 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.569+0000 7f75e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.569+0000 7f75e1ffb700 1 -- 172.21.15.46:0/4241448578 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f400f660 con 0x55eb710864a0 2024-05-09T05:00:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.569+0000 7f75e1ffb700 1 -- 172.21.15.46:0/4241448578 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f75f400f800 con 0x55eb710864a0 2024-05-09T05:00:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.569+0000 7f760f206b80 1 -- 172.21.15.46:0/4241448578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75f0015410 con 0x55eb710864a0 2024-05-09T05:00:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.569+0000 7f75e1ffb700 1 -- 172.21.15.46:0/4241448578 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75f403d050 con 0x55eb710864a0 2024-05-09T05:00:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.569+0000 7f760f206b80 10 client.17325 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.570+0000 7f760f206b80 1 -- 172.21.15.46:0/4241448578 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eb7121ee50 con 0x55eb710864a0 2024-05-09T05:00:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.570+0000 7f75e1ffb700 1 -- 172.21.15.46:0/4241448578 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75f0015410 con 0x55eb710864a0 2024-05-09T05:00:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.570+0000 7f75e1ffb700 1 -- 172.21.15.46:0/4241448578 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f75f4022070 con 0x55eb710864a0 2024-05-09T05:00:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.570+0000 7f760f206b80 10 client.17325 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.570+0000 7f760f206b80 10 client.17325 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T05:00:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.570+0000 7f760f206b80 20 client.17325 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:29.574 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.570+0000 7f760f206b80 1 --2- 172.21.15.46:0/4241448578 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eb712214e0 0x55eb712239b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.570+0000 7f760f206b80 4 client.17325 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:00:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.570+0000 7f760f206b80 1 -- 172.21.15.46:0/4241448578 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55eb71133a80 con 0x55eb712214e0 2024-05-09T05:00:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.571+0000 7f75fdd8b700 1 --2- 172.21.15.46:0/4241448578 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eb712214e0 0x55eb712239b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.571+0000 7f75fdd8b700 1 --2- 172.21.15.46:0/4241448578 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eb712214e0 0x55eb712239b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=47110ca3c466a141 server_cookie=1b112260178baed0 in_seq=0 out_seq=0 2024-05-09T05:00:29.575 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.571+0000 7f75e1ffb700 10 client.17325 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T05:00:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.572+0000 7f75e1ffb700 1 -- 172.21.15.46:0/4241448578 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55eb71133a80 con 0x55eb712214e0 2024-05-09T05:00:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.572+0000 7f75e1ffb700 10 client.17325 handle_command_reply: tid=0 2024-05-09T05:00:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.572+0000 7f760f206b80 1 client.17325 shutdown 2024-05-09T05:00:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.572+0000 7f760f206b80 1 -- 172.21.15.46:0/4241448578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb710864a0 msgr2=0x55eb71211490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:29.576 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.572+0000 7f760f206b80 1 --2- 172.21.15.46:0/4241448578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb710864a0 0x55eb71211490 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f75f4005780 tx=0x7f75f401a050 comp rx=0 tx=0).stop 2024-05-09T05:00:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.572+0000 7f760f206b80 1 -- 172.21.15.46:0/4241448578 shutdown_connections 2024-05-09T05:00:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.572+0000 7f760f206b80 1 --2- 172.21.15.46:0/4241448578 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb71207810 0x55eb71211f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.572+0000 7f760f206b80 1 --2- 172.21.15.46:0/4241448578 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb710864a0 0x55eb71211490 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.572+0000 7f760f206b80 1 --2- 172.21.15.46:0/4241448578 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eb712214e0 0x55eb712239b0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.572+0000 7f760f206b80 1 --2- 172.21.15.46:0/4241448578 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 0x55eb712119d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.572+0000 7f760f206b80 1 -- 172.21.15.46:0/4241448578 >> 172.21.15.46:0/4241448578 conn(0x55eb71089f70 msgr2=0x55eb711f7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.572+0000 7f75e1ffb700 0 client.17325 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T05:00:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.572+0000 7f760f206b80 1 -- 172.21.15.46:0/4241448578 shutdown_connections 2024-05-09T05:00:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.572+0000 7f760f206b80 1 -- 172.21.15.46:0/4241448578 wait complete. 2024-05-09T05:00:29.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.572+0000 7f760f206b80 20 client.17325 trim_cache size 0 max 16384 2024-05-09T05:00:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.587+0000 7f760f206b80 1 Processor -- start 2024-05-09T05:00:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.587+0000 7f760f206b80 1 -- start start 2024-05-09T05:00:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f760f206b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb710864a0 0x55eb71211d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f760f206b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 0x55eb71210490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f760f206b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb711f7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f760f206b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75f81af7a0 con 0x55eb71207810 2024-05-09T05:00:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f760f206b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f75f81ad030 con 0x55eb710864a0 2024-05-09T05:00:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f760f206b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f75f81aaea0 con 0x55eb71100d00 2024-05-09T05:00:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f75fed8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb711f7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f75fed8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb711f7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36854/0 (socket says 172.21.15.46:36854) 2024-05-09T05:00:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f75fed8d700 1 -- 172.21.15.46:0/2285176587 learned_addr learned my addr 172.21.15.46:0/2285176587 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f75fe58c700 1 --2- 172.21.15.46:0/2285176587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb710864a0 0x55eb71211d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f75fdd8b700 1 --2- 172.21.15.46:0/2285176587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 0x55eb71210490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f75fed8d700 1 -- 172.21.15.46:0/2285176587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 msgr2=0x55eb71210490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f75fed8d700 1 --2- 172.21.15.46:0/2285176587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 0x55eb71210490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f75fed8d700 1 -- 172.21.15.46:0/2285176587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb710864a0 msgr2=0x55eb71211d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f75fed8d700 1 --2- 172.21.15.46:0/2285176587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb710864a0 0x55eb71211d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f75fed8d700 1 -- 172.21.15.46:0/2285176587 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55eb711fa9d0 con 0x55eb71207810 2024-05-09T05:00:29.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f75fdd8b700 1 --2- 172.21.15.46:0/2285176587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 0x55eb71210490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:00:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.588+0000 7f75fe58c700 1 --2- 172.21.15.46:0/2285176587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb710864a0 0x55eb71211d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.589+0000 7f75fed8d700 1 --2- 172.21.15.46:0/2285176587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb711f7ff0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f75f0018040 tx=0x7f75f0015f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667af3f18a60c9ac server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.589+0000 7f75e3fff700 1 -- 172.21.15.46:0/2285176587 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f001a040 con 0x55eb71207810 2024-05-09T05:00:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.589+0000 7f75e3fff700 1 -- 172.21.15.46:0/2285176587 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75f0015410 con 0x55eb71207810 2024-05-09T05:00:29.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.589+0000 7f75e3fff700 1 -- 172.21.15.46:0/2285176587 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f001f6c0 con 0x55eb71207810 2024-05-09T05:00:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 -- 172.21.15.46:0/2285176587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 msgr2=0x55eb711f7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 --2- 172.21.15.46:0/2285176587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb711f7ff0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f75f0018040 tx=0x7f75f0015f70 comp rx=0 tx=0).stop 2024-05-09T05:00:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 -- 172.21.15.46:0/2285176587 shutdown_connections 2024-05-09T05:00:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 --2- 172.21.15.46:0/2285176587 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb711f7ff0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 --2- 172.21.15.46:0/2285176587 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb710864a0 0x55eb71211d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 --2- 172.21.15.46:0/2285176587 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 0x55eb71210490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 -- 172.21.15.46:0/2285176587 >> 172.21.15.46:0/2285176587 conn(0x55eb71089f70 msgr2=0x55eb7120fd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 -- 172.21.15.46:0/2285176587 shutdown_connections 2024-05-09T05:00:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 -- 172.21.15.46:0/2285176587 wait complete. 2024-05-09T05:00:29.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 -- start start 2024-05-09T05:00:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 Processor -- start 2024-05-09T05:00:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb710864a0 0x55eb71219510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 0x55eb71219a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb71210d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f75f81ab4f0 con 0x55eb71207810 2024-05-09T05:00:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f75f81ac960 con 0x55eb710864a0 2024-05-09T05:00:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.590+0000 7f760f206b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f75f81abbc0 con 0x55eb71100d00 2024-05-09T05:00:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f760f206b80 10 client.0 _ll_register_callbacks cb 0x55eb70f976c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f75fed8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb71210d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f75fed8d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb71210d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36870/0 (socket says 172.21.15.46:36870) 2024-05-09T05:00:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f75fed8d700 1 -- 172.21.15.46:0/2058581146 learned_addr learned my addr 172.21.15.46:0/2058581146 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f75fe58c700 1 --2- 172.21.15.46:0/2058581146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb710864a0 0x55eb71219510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f75fdd8b700 1 --2- 172.21.15.46:0/2058581146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 0x55eb71219a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f75fed8d700 1 -- 172.21.15.46:0/2058581146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 msgr2=0x55eb71219a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f75fed8d700 1 --2- 172.21.15.46:0/2058581146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 0x55eb71219a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f75fed8d700 1 -- 172.21.15.46:0/2058581146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb710864a0 msgr2=0x55eb71219510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f75fed8d700 1 --2- 172.21.15.46:0/2058581146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb710864a0 0x55eb71219510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f75fed8d700 1 -- 172.21.15.46:0/2058581146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55eb71208a10 con 0x55eb71207810 2024-05-09T05:00:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f75fed8d700 1 --2- 172.21.15.46:0/2058581146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb71210d40 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f75f0015a90 tx=0x7f75f0015b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f4c0e44f77c558 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f75e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f75e1ffb700 1 -- 172.21.15.46:0/2058581146 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75f002e070 con 0x55eb71207810 2024-05-09T05:00:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f760f206b80 1 -- 172.21.15.46:0/2058581146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f75f0015410 con 0x55eb71207810 2024-05-09T05:00:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.591+0000 7f75e1ffb700 1 -- 172.21.15.46:0/2058581146 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f75f000cc40 con 0x55eb71207810 2024-05-09T05:00:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.592+0000 7f75e1ffb700 1 -- 172.21.15.46:0/2058581146 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f75f001a040 con 0x55eb71207810 2024-05-09T05:00:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.592+0000 7f760f206b80 10 client.17331 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.592+0000 7f760f206b80 1 -- 172.21.15.46:0/2058581146 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55eb7128b190 con 0x55eb71207810 2024-05-09T05:00:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.592+0000 7f75e1ffb700 1 -- 172.21.15.46:0/2058581146 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75f0015410 con 0x55eb71207810 2024-05-09T05:00:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.592+0000 7f75e1ffb700 1 -- 172.21.15.46:0/2058581146 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f75f000a400 con 0x55eb71207810 2024-05-09T05:00:29.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.592+0000 7f760f206b80 10 client.17331 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.592+0000 7f760f206b80 10 client.17331 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T05:00:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.592+0000 7f760f206b80 20 client.17331 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.592+0000 7f760f206b80 1 --2- 172.21.15.46:0/2058581146 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eb7128b610 0x55eb7128da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.592+0000 7f760f206b80 4 client.17331 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:00:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.592+0000 7f760f206b80 1 -- 172.21.15.46:0/2058581146 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55eb7112db20 con 0x55eb7128b610 2024-05-09T05:00:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.592+0000 7f75fe58c700 1 --2- 172.21.15.46:0/2058581146 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eb7128b610 0x55eb7128da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.593+0000 7f75fe58c700 1 --2- 172.21.15.46:0/2058581146 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eb7128b610 0x55eb7128da30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=be3e4a995e30ed34 server_cookie=fc6a0106352e746e in_seq=0 out_seq=0 2024-05-09T05:00:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.593+0000 7f75e1ffb700 10 client.17331 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T05:00:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.593+0000 7f75e1ffb700 1 -- 172.21.15.46:0/2058581146 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55eb7112db20 con 0x55eb7128b610 2024-05-09T05:00:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.593+0000 7f75e1ffb700 10 client.17331 handle_command_reply: tid=0 2024-05-09T05:00:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.593+0000 7f760f206b80 1 client.17331 shutdown 2024-05-09T05:00:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.593+0000 7f760f206b80 1 -- 172.21.15.46:0/2058581146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 msgr2=0x55eb71210d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.593+0000 7f760f206b80 1 --2- 172.21.15.46:0/2058581146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb71210d40 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f75f0015a90 tx=0x7f75f0015b70 comp rx=0 tx=0).stop 2024-05-09T05:00:29.598 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.593+0000 7f760f206b80 1 -- 172.21.15.46:0/2058581146 shutdown_connections 2024-05-09T05:00:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.593+0000 7f760f206b80 1 --2- 172.21.15.46:0/2058581146 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55eb71207810 0x55eb71210d40 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.593+0000 7f760f206b80 1 --2- 172.21.15.46:0/2058581146 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55eb710864a0 0x55eb71219510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.593+0000 7f760f206b80 1 --2- 172.21.15.46:0/2058581146 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x55eb7128b610 0x55eb7128da30 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.593+0000 7f760f206b80 1 --2- 172.21.15.46:0/2058581146 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55eb71100d00 0x55eb71219a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.593+0000 7f75e1ffb700 0 client.17331 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T05:00:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.593+0000 7f760f206b80 1 -- 172.21.15.46:0/2058581146 >> 172.21.15.46:0/2058581146 conn(0x55eb71089f70 msgr2=0x55eb711f7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.593+0000 7f760f206b80 1 -- 172.21.15.46:0/2058581146 shutdown_connections 2024-05-09T05:00:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.594+0000 7f760f206b80 1 -- 172.21.15.46:0/2058581146 wait complete. 2024-05-09T05:00:29.599 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.594+0000 7f760f206b80 20 client.17331 trim_cache size 0 max 16384 2024-05-09T05:00:29.600 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T05:00:29.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.594+0000 7f76007f0700 1 -- 172.21.15.46:0/3708687740 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f75e40b29f0 msgr2=0x7f75e40b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:29.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.594+0000 7f76007f0700 1 --2- 172.21.15.46:0/3708687740 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f75e40b29f0 0x7f75e40b4f00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009990 tx=0x7f75f4000bc0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.600 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.594+0000 7f76007f0700 1 -- 172.21.15.46:0/3708687740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f8108f40 msgr2=0x7f75f819b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.594+0000 7f76007f0700 1 --2- 172.21.15.46:0/3708687740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f8108f40 0x7f75f819b4e0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f75e800b500 tx=0x7f75e800b810 comp rx=0 tx=0).stop 2024-05-09T05:00:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.594+0000 7f76007f0700 1 -- 172.21.15.46:0/3708687740 shutdown_connections 2024-05-09T05:00:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.594+0000 7f76007f0700 1 --2- 172.21.15.46:0/3708687740 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f75e40b29f0 0x7f75e40b4f00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.594+0000 7f76007f0700 1 --2- 172.21.15.46:0/3708687740 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f75f8109890 0x7f75f819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.594+0000 7f76007f0700 1 --2- 172.21.15.46:0/3708687740 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f75f8104610 0x7f75f819af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.594+0000 7f76007f0700 1 --2- 172.21.15.46:0/3708687740 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f75f8108f40 0x7f75f819b4e0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.594+0000 7f76007f0700 1 -- 172.21.15.46:0/3708687740 >> 172.21.15.46:0/3708687740 conn(0x7f75f807af70 msgr2=0x7f75f81019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.594+0000 7f76007f0700 1 -- 172.21.15.46:0/3708687740 shutdown_connections 2024-05-09T05:00:29.601 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:29.594+0000 7f76007f0700 1 -- 172.21.15.46:0/3708687740 wait complete. 2024-05-09T05:00:30.317 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T05:00:30.572 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:31.030 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:30 smithi046 ceph-mon[30136]: pgmap v291: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T05:00:31.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:30 smithi072 ceph-mon[34088]: pgmap v291: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T05:00:31.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:30 smithi110 ceph-mon[34904]: pgmap v291: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 7 op/s 2024-05-09T05:00:32.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.265+0000 7f2519b0c700 1 -- 172.21.15.46:0/2929719919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2514105e40 msgr2=0x7f251410e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:32.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.265+0000 7f2519b0c700 1 --2- 172.21.15.46:0/2929719919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2514105e40 0x7f251410e390 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f2508009990 tx=0x7f2508009ca0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.265+0000 7f2519b0c700 1 -- 172.21.15.46:0/2929719919 shutdown_connections 2024-05-09T05:00:32.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.265+0000 7f2519b0c700 1 --2- 172.21.15.46:0/2929719919 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f251410e920 0x7f2514110e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.265+0000 7f2519b0c700 1 --2- 172.21.15.46:0/2929719919 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f25141054f0 0x7f2514105900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.265+0000 7f2519b0c700 1 --2- 172.21.15.46:0/2929719919 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2514105e40 0x7f251410e390 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.265+0000 7f2519b0c700 1 -- 172.21.15.46:0/2929719919 >> 172.21.15.46:0/2929719919 conn(0x7f2514078e00 msgr2=0x7f2514079240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:32.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.265+0000 7f2519b0c700 1 -- 172.21.15.46:0/2929719919 shutdown_connections 2024-05-09T05:00:32.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.265+0000 7f2519b0c700 1 -- 172.21.15.46:0/2929719919 wait complete. 2024-05-09T05:00:32.266 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.266+0000 7f2519b0c700 1 Processor -- start 2024-05-09T05:00:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.266+0000 7f2519b0c700 1 -- start start 2024-05-09T05:00:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.266+0000 7f2519b0c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25141054f0 0x7f25141031e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.266+0000 7f2519b0c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2514105e40 0x7f2514103720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.268 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.266+0000 7f2519b0c700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f251410e920 0x7f2514103c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.266+0000 7f2519b0c700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2514114480 con 0x7f2514105e40 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.266+0000 7f2519b0c700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2514114300 con 0x7f25141054f0 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.266+0000 7f2519b0c700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2514114600 con 0x7f251410e920 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f2512ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2514105e40 0x7f2514103720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f2512ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2514105e40 0x7f2514103720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36882/0 (socket says 172.21.15.46:36882) 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f2512ffd700 1 -- 172.21.15.46:0/2823469333 learned_addr learned my addr 172.21.15.46:0/2823469333 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f2513fff700 1 --2- 172.21.15.46:0/2823469333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f251410e920 0x7f2514103c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f25137fe700 1 --2- 172.21.15.46:0/2823469333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25141054f0 0x7f25141031e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f2512ffd700 1 -- 172.21.15.46:0/2823469333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f251410e920 msgr2=0x7f2514103c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f2512ffd700 1 --2- 172.21.15.46:0/2823469333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f251410e920 0x7f2514103c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f2512ffd700 1 -- 172.21.15.46:0/2823469333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25141054f0 msgr2=0x7f25141031e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f2512ffd700 1 --2- 172.21.15.46:0/2823469333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25141054f0 0x7f25141031e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f2512ffd700 1 -- 172.21.15.46:0/2823469333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25141a91b0 con 0x7f2514105e40 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f25137fe700 1 --2- 172.21.15.46:0/2823469333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25141054f0 0x7f25141031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f2512ffd700 1 --2- 172.21.15.46:0/2823469333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2514105e40 0x7f2514103720 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2508000c00 tx=0x7f25080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:32.269 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f2510ff9700 1 -- 172.21.15.46:0/2823469333 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508027070 con 0x7f2514105e40 2024-05-09T05:00:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f2510ff9700 1 -- 172.21.15.46:0/2823469333 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2508003fa0 con 0x7f2514105e40 2024-05-09T05:00:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f2519b0c700 1 -- 172.21.15.46:0/2823469333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25141a9440 con 0x7f2514105e40 2024-05-09T05:00:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f2510ff9700 1 -- 172.21.15.46:0/2823469333 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508022d40 con 0x7f2514105e40 2024-05-09T05:00:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.267+0000 7f2519b0c700 1 -- 172.21.15.46:0/2823469333 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25141a9970 con 0x7f2514105e40 2024-05-09T05:00:32.270 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.270+0000 7f2510ff9700 1 -- 172.21.15.46:0/2823469333 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f250802a070 con 0x7f2514105e40 2024-05-09T05:00:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.271+0000 7f2510ff9700 1 --2- 172.21.15.46:0/2823469333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f24fc0b2a70 0x7f24fc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.271+0000 7f25137fe700 1 --2- 172.21.15.46:0/2823469333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f24fc0b2a70 0x7f24fc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.271 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.271+0000 7f2510ff9700 1 -- 172.21.15.46:0/2823469333 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f25080e34c0 con 0x7f2514105e40 2024-05-09T05:00:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.271+0000 7f25137fe700 1 --2- 172.21.15.46:0/2823469333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f24fc0b2a70 0x7f24fc0b4f80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2504009dd0 tx=0x7f2504009040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.271+0000 7f2528522b80 1 Processor -- start 2024-05-09T05:00:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.271+0000 7f2528522b80 1 -- start start 2024-05-09T05:00:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2528522b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315aa7c160 0x56315ab154f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2528522b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab15a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2528522b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315ab0f6a0 0x56315ab0fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2528522b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56315aa31ff0 con 0x56315ab0f6a0 2024-05-09T05:00:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2528522b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56315a9d07f0 con 0x56315a9924a0 2024-05-09T05:00:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2528522b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56315aa2dfb0 con 0x56315aa7c160 2024-05-09T05:00:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2512ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab15a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.273 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2512ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab15a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:35726/0 (socket says 172.21.15.46:35726) 2024-05-09T05:00:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2512ffd700 1 -- 172.21.15.46:0/2007548848 learned_addr learned my addr 172.21.15.46:0/2007548848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f25137fe700 1 --2- 172.21.15.46:0/2007548848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315aa7c160 0x56315ab154f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2513fff700 1 --2- 172.21.15.46:0/2007548848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315ab0f6a0 0x56315ab0fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2512ffd700 1 -- 172.21.15.46:0/2007548848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315aa7c160 msgr2=0x56315ab154f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2512ffd700 1 --2- 172.21.15.46:0/2007548848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315aa7c160 0x56315ab154f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2512ffd700 1 -- 172.21.15.46:0/2007548848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315ab0f6a0 msgr2=0x56315ab0fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2512ffd700 1 --2- 172.21.15.46:0/2007548848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315ab0f6a0 0x56315ab0fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2512ffd700 1 -- 172.21.15.46:0/2007548848 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56315a9c8370 con 0x56315a9924a0 2024-05-09T05:00:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.272+0000 7f2513fff700 1 --2- 172.21.15.46:0/2007548848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315ab0f6a0 0x56315ab0fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.273+0000 7f2512ffd700 1 --2- 172.21.15.46:0/2007548848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab15a30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f25080aa770 tx=0x7f25080aa3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a06426659cb5a21 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.273+0000 7f24f9ffb700 1 -- 172.21.15.46:0/2007548848 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25080b10b0 con 0x56315a9924a0 2024-05-09T05:00:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.273+0000 7f24f9ffb700 1 -- 172.21.15.46:0/2007548848 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25080aa7e0 con 0x56315a9924a0 2024-05-09T05:00:32.274 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.273+0000 7f24f9ffb700 1 -- 172.21.15.46:0/2007548848 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2508093110 con 0x56315a9924a0 2024-05-09T05:00:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.274+0000 7f2528522b80 1 -- 172.21.15.46:0/2007548848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 msgr2=0x56315ab15a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:32.277 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.274+0000 7f2528522b80 1 --2- 172.21.15.46:0/2007548848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab15a30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f25080aa770 tx=0x7f25080aa3c0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.274+0000 7f2528522b80 1 -- 172.21.15.46:0/2007548848 shutdown_connections 2024-05-09T05:00:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.274+0000 7f2528522b80 1 --2- 172.21.15.46:0/2007548848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315ab0f6a0 0x56315ab0fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.274+0000 7f2528522b80 1 --2- 172.21.15.46:0/2007548848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315aa7c160 0x56315ab154f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.274+0000 7f2528522b80 1 --2- 172.21.15.46:0/2007548848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab15a30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.274+0000 7f2528522b80 1 -- 172.21.15.46:0/2007548848 >> 172.21.15.46:0/2007548848 conn(0x56315a9f26a0 msgr2=0x56315ab0a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.274+0000 7f2528522b80 1 -- 172.21.15.46:0/2007548848 shutdown_connections 2024-05-09T05:00:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.274+0000 7f2528522b80 1 -- 172.21.15.46:0/2007548848 wait complete. 2024-05-09T05:00:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.274+0000 7f2528522b80 1 -- start start 2024-05-09T05:00:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f2528522b80 1 Processor -- start 2024-05-09T05:00:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f2528522b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab1eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f2528522b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315aa7c160 0x56315ab1f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f2528522b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315ab0f6a0 0x56315ab18f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f2528522b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x56315a8adba0 con 0x56315ab0f6a0 2024-05-09T05:00:32.278 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f2528522b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x56315a9ec150 con 0x56315a9924a0 2024-05-09T05:00:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f2528522b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x56315a9d07f0 con 0x56315aa7c160 2024-05-09T05:00:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f2528522b80 10 client.0 _ll_register_callbacks cb 0x56315a8a36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f2513fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315ab0f6a0 0x56315ab18f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f25137fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab1eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f2513fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315ab0f6a0 0x56315ab18f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36890/0 (socket says 172.21.15.46:36890) 2024-05-09T05:00:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f2513fff700 1 -- 172.21.15.46:0/1822122314 learned_addr learned my addr 172.21.15.46:0/1822122314 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f25137fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab1eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:35728/0 (socket says 172.21.15.46:35728) 2024-05-09T05:00:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f2512ffd700 1 --2- 172.21.15.46:0/1822122314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315aa7c160 0x56315ab1f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f25137fe700 1 -- 172.21.15.46:0/1822122314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315aa7c160 msgr2=0x56315ab1f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f25137fe700 1 --2- 172.21.15.46:0/1822122314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315aa7c160 0x56315ab1f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f25137fe700 1 -- 172.21.15.46:0/1822122314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315ab0f6a0 msgr2=0x56315ab18f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f25137fe700 1 --2- 172.21.15.46:0/1822122314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315ab0f6a0 0x56315ab18f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f25137fe700 1 -- 172.21.15.46:0/1822122314 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56315a8ad490 con 0x56315a9924a0 2024-05-09T05:00:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.275+0000 7f2513fff700 1 --2- 172.21.15.46:0/1822122314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315ab0f6a0 0x56315ab18f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:00:32.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.276+0000 7f25137fe700 1 --2- 172.21.15.46:0/1822122314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab1eb70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2504009bb0 tx=0x7f2504004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c774c1f3670307f8 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.276+0000 7f24f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T05:00:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.276+0000 7f24f2ffd700 1 -- 172.21.15.46:0/1822122314 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504026030 con 0x56315a9924a0 2024-05-09T05:00:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.276+0000 7f24f2ffd700 1 -- 172.21.15.46:0/1822122314 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f250401f040 con 0x56315a9924a0 2024-05-09T05:00:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.276+0000 7f2528522b80 1 -- 172.21.15.46:0/1822122314 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25080aa7e0 con 0x56315a9924a0 2024-05-09T05:00:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.276+0000 7f24f2ffd700 1 -- 172.21.15.46:0/1822122314 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2504030050 con 0x56315a9924a0 2024-05-09T05:00:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.276+0000 7f2528522b80 10 client.26846 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.276+0000 7f2528522b80 1 -- 172.21.15.46:0/1822122314 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56315ab267f0 con 0x56315a9924a0 2024-05-09T05:00:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.276+0000 7f24f2ffd700 1 -- 172.21.15.46:0/1822122314 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25080aa7e0 con 0x56315a9924a0 2024-05-09T05:00:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.277+0000 7f24f2ffd700 1 -- 172.21.15.46:0/1822122314 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2504004020 con 0x56315a9924a0 2024-05-09T05:00:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.277+0000 7f2528522b80 10 client.26846 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.277+0000 7f2528522b80 10 client.26846 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T05:00:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.277+0000 7f2528522b80 20 client.26846 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.277+0000 7f2528522b80 1 --2- 172.21.15.46:0/1822122314 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x56315ab28e80 0x56315ab2b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.277+0000 7f2528522b80 4 client.26846 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:00:32.280 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.277+0000 7f2528522b80 1 -- 172.21.15.46:0/1822122314 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56315aa3f9d0 con 0x56315ab28e80 2024-05-09T05:00:32.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.277+0000 7f2512ffd700 1 --2- 172.21.15.46:0/1822122314 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x56315ab28e80 0x56315ab2b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.277+0000 7f24f2ffd700 10 client.26846 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T05:00:32.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.278+0000 7f2512ffd700 1 --2- 172.21.15.46:0/1822122314 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x56315ab28e80 0x56315ab2b2a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bd18ace9c9a088cc server_cookie=426fc150e43b5d83 in_seq=0 out_seq=0 2024-05-09T05:00:32.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.278+0000 7f24f2ffd700 1 -- 172.21.15.46:0/1822122314 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56315aa3f9d0 con 0x56315ab28e80 2024-05-09T05:00:32.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.278+0000 7f24f2ffd700 10 client.26846 handle_command_reply: tid=0 2024-05-09T05:00:32.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.278+0000 7f2528522b80 1 client.26846 shutdown 2024-05-09T05:00:32.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.279+0000 7f2528522b80 1 -- 172.21.15.46:0/1822122314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 msgr2=0x56315ab1eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:32.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.279+0000 7f2528522b80 1 --2- 172.21.15.46:0/1822122314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab1eb70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2504009bb0 tx=0x7f2504004ad0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.279+0000 7f2528522b80 1 -- 172.21.15.46:0/1822122314 shutdown_connections 2024-05-09T05:00:32.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.279+0000 7f2528522b80 1 --2- 172.21.15.46:0/1822122314 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x56315ab28e80 0x56315ab2b2a0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.279+0000 7f2528522b80 1 --2- 172.21.15.46:0/1822122314 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315ab0f6a0 0x56315ab18f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.279+0000 7f2528522b80 1 --2- 172.21.15.46:0/1822122314 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab1eb70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.282 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.279+0000 7f2528522b80 1 --2- 172.21.15.46:0/1822122314 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315aa7c160 0x56315ab1f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.279+0000 7f24f2ffd700 0 client.26846 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T05:00:32.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.279+0000 7f2528522b80 1 -- 172.21.15.46:0/1822122314 >> 172.21.15.46:0/1822122314 conn(0x56315a9f26a0 msgr2=0x56315ab0a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:32.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.279+0000 7f2528522b80 1 -- 172.21.15.46:0/1822122314 shutdown_connections 2024-05-09T05:00:32.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.279+0000 7f2528522b80 1 -- 172.21.15.46:0/1822122314 wait complete. 2024-05-09T05:00:32.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.279+0000 7f2528522b80 20 client.26846 trim_cache size 0 max 16384 2024-05-09T05:00:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.294+0000 7f2528522b80 1 Processor -- start 2024-05-09T05:00:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.294+0000 7f2528522b80 1 -- start start 2024-05-09T05:00:32.295 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.294+0000 7f2528522b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315a9924a0 0x56315ab16b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.294+0000 7f2528522b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315aa7c160 0x56315ab30a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.294+0000 7f2528522b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315ab0f6a0 0x56315ab30fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.294+0000 7f2528522b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25141b3a50 con 0x56315a9924a0 2024-05-09T05:00:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.294+0000 7f2528522b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f25141b12e0 con 0x56315aa7c160 2024-05-09T05:00:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.294+0000 7f2528522b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25141af150 con 0x56315ab0f6a0 2024-05-09T05:00:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.294+0000 7f25137fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315a9924a0 0x56315ab16b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.295+0000 7f25137fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315a9924a0 0x56315ab16b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36892/0 (socket says 172.21.15.46:36892) 2024-05-09T05:00:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.295+0000 7f25137fe700 1 -- 172.21.15.46:0/1320911975 learned_addr learned my addr 172.21.15.46:0/1320911975 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.295+0000 7f2512ffd700 1 --2- 172.21.15.46:0/1320911975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315aa7c160 0x56315ab30a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.295+0000 7f2513fff700 1 --2- 172.21.15.46:0/1320911975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315ab0f6a0 0x56315ab30fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.295+0000 7f25137fe700 1 -- 172.21.15.46:0/1320911975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315ab0f6a0 msgr2=0x56315ab30fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.295+0000 7f25137fe700 1 --2- 172.21.15.46:0/1320911975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315ab0f6a0 0x56315ab30fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.295+0000 7f25137fe700 1 -- 172.21.15.46:0/1320911975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315aa7c160 msgr2=0x56315ab30a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.295+0000 7f25137fe700 1 --2- 172.21.15.46:0/1320911975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315aa7c160 0x56315ab30a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.296 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.295+0000 7f25137fe700 1 -- 172.21.15.46:0/1320911975 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56315ab0c150 con 0x56315a9924a0 2024-05-09T05:00:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.295+0000 7f25137fe700 1 --2- 172.21.15.46:0/1320911975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315a9924a0 0x56315ab16b90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2504007e40 tx=0x7f2504016f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25e45ff5511ec5fc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.295+0000 7f24f97fa700 1 -- 172.21.15.46:0/1320911975 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504026040 con 0x56315a9924a0 2024-05-09T05:00:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.295+0000 7f24f97fa700 1 -- 172.21.15.46:0/1320911975 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2504005c00 con 0x56315a9924a0 2024-05-09T05:00:32.297 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.295+0000 7f24f97fa700 1 -- 172.21.15.46:0/1320911975 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2504003c60 con 0x56315a9924a0 2024-05-09T05:00:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.296+0000 7f2528522b80 1 -- 172.21.15.46:0/1320911975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315a9924a0 msgr2=0x56315ab16b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.296+0000 7f2528522b80 1 --2- 172.21.15.46:0/1320911975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315a9924a0 0x56315ab16b90 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f2504007e40 tx=0x7f2504016f90 comp rx=0 tx=0).stop 2024-05-09T05:00:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.296+0000 7f2528522b80 1 -- 172.21.15.46:0/1320911975 shutdown_connections 2024-05-09T05:00:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.296+0000 7f2528522b80 1 --2- 172.21.15.46:0/1320911975 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315ab0f6a0 0x56315ab30fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.296+0000 7f2528522b80 1 --2- 172.21.15.46:0/1320911975 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315a9924a0 0x56315ab16b90 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.296+0000 7f2528522b80 1 --2- 172.21.15.46:0/1320911975 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315aa7c160 0x56315ab30a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.296+0000 7f2528522b80 1 -- 172.21.15.46:0/1320911975 >> 172.21.15.46:0/1320911975 conn(0x56315a9f26a0 msgr2=0x56315ab17880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.296+0000 7f2528522b80 1 -- 172.21.15.46:0/1320911975 shutdown_connections 2024-05-09T05:00:32.301 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.296+0000 7f2528522b80 1 -- 172.21.15.46:0/1320911975 wait complete. 2024-05-09T05:00:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.296+0000 7f2528522b80 1 -- start start 2024-05-09T05:00:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.297+0000 7f2528522b80 1 Processor -- start 2024-05-09T05:00:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.297+0000 7f2528522b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab21c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.297+0000 7f2528522b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315aa7c160 0x56315ab221b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.297+0000 7f2528522b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315ab0f6a0 0x56315ab226f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.297+0000 7f2528522b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f25141af7a0 con 0x56315aa7c160 2024-05-09T05:00:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.297+0000 7f2528522b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f25141b0c10 con 0x56315a9924a0 2024-05-09T05:00:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.297+0000 7f2528522b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f25141afe70 con 0x56315ab0f6a0 2024-05-09T05:00:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.297+0000 7f2528522b80 10 client.0 _ll_register_callbacks cb 0x56315a8a36c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.297+0000 7f2512ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315aa7c160 0x56315ab221b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.297+0000 7f2512ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315aa7c160 0x56315ab221b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:36902/0 (socket says 172.21.15.46:36902) 2024-05-09T05:00:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.297+0000 7f2512ffd700 1 -- 172.21.15.46:0/3693388110 learned_addr learned my addr 172.21.15.46:0/3693388110 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.297+0000 7f2513fff700 1 --2- 172.21.15.46:0/3693388110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315ab0f6a0 0x56315ab226f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.297+0000 7f25137fe700 1 --2- 172.21.15.46:0/3693388110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab21c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.302 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.297+0000 7f2512ffd700 1 -- 172.21.15.46:0/3693388110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315ab0f6a0 msgr2=0x56315ab226f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.297+0000 7f2512ffd700 1 --2- 172.21.15.46:0/3693388110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315ab0f6a0 0x56315ab226f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.298+0000 7f2512ffd700 1 -- 172.21.15.46:0/3693388110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 msgr2=0x56315ab21c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.298+0000 7f2512ffd700 1 --2- 172.21.15.46:0/3693388110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab21c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.298+0000 7f2512ffd700 1 -- 172.21.15.46:0/3693388110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56315ab91680 con 0x56315aa7c160 2024-05-09T05:00:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.298+0000 7f25137fe700 1 --2- 172.21.15.46:0/3693388110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab21c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:00:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.298+0000 7f2512ffd700 1 --2- 172.21.15.46:0/3693388110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315aa7c160 0x56315ab221b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f25080098e0 tx=0x7f2508004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd854c1c7e7d6554 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.298+0000 7f24f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.298+0000 7f24f2ffd700 1 -- 172.21.15.46:0/3693388110 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25080a66a0 con 0x56315aa7c160 2024-05-09T05:00:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.298+0000 7f24f2ffd700 1 -- 172.21.15.46:0/3693388110 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f2508022470 con 0x56315aa7c160 2024-05-09T05:00:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.298+0000 7f2528522b80 1 -- 172.21.15.46:0/3693388110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2504005c00 con 0x56315aa7c160 2024-05-09T05:00:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.298+0000 7f24f2ffd700 1 -- 172.21.15.46:0/3693388110 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f25080b10b0 con 0x56315aa7c160 2024-05-09T05:00:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.298+0000 7f2528522b80 10 client.17349 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.298+0000 7f2528522b80 1 -- 172.21.15.46:0/3693388110 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56315ab928e0 con 0x56315aa7c160 2024-05-09T05:00:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.298+0000 7f24f2ffd700 1 -- 172.21.15.46:0/3693388110 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2504005c00 con 0x56315aa7c160 2024-05-09T05:00:32.303 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.298+0000 7f24f2ffd700 1 -- 172.21.15.46:0/3693388110 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2508093110 con 0x56315aa7c160 2024-05-09T05:00:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.299+0000 7f2528522b80 10 client.17349 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.299+0000 7f2528522b80 10 client.17349 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T05:00:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.299+0000 7f2528522b80 20 client.17349 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.299+0000 7f2528522b80 1 --2- 172.21.15.46:0/3693388110 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x56315ab92de0 0x56315ab95200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.299+0000 7f2528522b80 4 client.17349 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:00:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.299+0000 7f2528522b80 1 -- 172.21.15.46:0/3693388110 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56315aa39a70 con 0x56315ab92de0 2024-05-09T05:00:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.299+0000 7f25137fe700 1 --2- 172.21.15.46:0/3693388110 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x56315ab92de0 0x56315ab95200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.299+0000 7f24f2ffd700 10 client.17349 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T05:00:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.299+0000 7f25137fe700 1 --2- 172.21.15.46:0/3693388110 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x56315ab92de0 0x56315ab95200 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=48a472a92639818 server_cookie=facaf3625939955c in_seq=0 out_seq=0 2024-05-09T05:00:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.300+0000 7f24f2ffd700 1 -- 172.21.15.46:0/3693388110 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56315aa39a70 con 0x56315ab92de0 2024-05-09T05:00:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.300+0000 7f24f2ffd700 10 client.17349 handle_command_reply: tid=0 2024-05-09T05:00:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.300+0000 7f2528522b80 1 client.17349 shutdown 2024-05-09T05:00:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.300+0000 7f2528522b80 1 -- 172.21.15.46:0/3693388110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315aa7c160 msgr2=0x56315ab221b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.300+0000 7f2528522b80 1 --2- 172.21.15.46:0/3693388110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315aa7c160 0x56315ab221b0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f25080098e0 tx=0x7f2508004360 comp rx=0 tx=0).stop 2024-05-09T05:00:32.304 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.300+0000 7f2528522b80 1 -- 172.21.15.46:0/3693388110 shutdown_connections 2024-05-09T05:00:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.300+0000 7f2528522b80 1 --2- 172.21.15.46:0/3693388110 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x56315ab92de0 0x56315ab95200 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.300+0000 7f2528522b80 1 --2- 172.21.15.46:0/3693388110 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x56315ab0f6a0 0x56315ab226f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.300+0000 7f2528522b80 1 --2- 172.21.15.46:0/3693388110 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x56315a9924a0 0x56315ab21c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.300+0000 7f2528522b80 1 --2- 172.21.15.46:0/3693388110 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x56315aa7c160 0x56315ab221b0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.300+0000 7f2528522b80 1 -- 172.21.15.46:0/3693388110 >> 172.21.15.46:0/3693388110 conn(0x56315a9f26a0 msgr2=0x56315ab18740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.300+0000 7f24f2ffd700 0 client.17349 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T05:00:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.300+0000 7f2528522b80 1 -- 172.21.15.46:0/3693388110 shutdown_connections 2024-05-09T05:00:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.300+0000 7f2528522b80 1 -- 172.21.15.46:0/3693388110 wait complete. 2024-05-09T05:00:32.305 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.300+0000 7f2528522b80 20 client.17349 trim_cache size 0 max 16384 2024-05-09T05:00:32.305 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T05:00:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.301+0000 7f2519b0c700 1 -- 172.21.15.46:0/2823469333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f24fc0b2a70 msgr2=0x7f24fc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.301+0000 7f2519b0c700 1 --2- 172.21.15.46:0/2823469333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f24fc0b2a70 0x7f24fc0b4f80 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2504009dd0 tx=0x7f2504009040 comp rx=0 tx=0).stop 2024-05-09T05:00:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.301+0000 7f2519b0c700 1 -- 172.21.15.46:0/2823469333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2514105e40 msgr2=0x7f2514103720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.301+0000 7f2519b0c700 1 --2- 172.21.15.46:0/2823469333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2514105e40 0x7f2514103720 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f2508000c00 tx=0x7f25080047b0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.301+0000 7f2519b0c700 1 -- 172.21.15.46:0/2823469333 shutdown_connections 2024-05-09T05:00:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.301+0000 7f2519b0c700 1 --2- 172.21.15.46:0/2823469333 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f24fc0b2a70 0x7f24fc0b4f80 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.306 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.301+0000 7f2519b0c700 1 --2- 172.21.15.46:0/2823469333 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f251410e920 0x7f2514103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.301+0000 7f2519b0c700 1 --2- 172.21.15.46:0/2823469333 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f25141054f0 0x7f25141031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.301+0000 7f2519b0c700 1 --2- 172.21.15.46:0/2823469333 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2514105e40 0x7f2514103720 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:32.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.301+0000 7f2519b0c700 1 -- 172.21.15.46:0/2823469333 >> 172.21.15.46:0/2823469333 conn(0x7f2514078e00 msgr2=0x7f25141000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:32.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.301+0000 7f2519b0c700 1 -- 172.21.15.46:0/2823469333 shutdown_connections 2024-05-09T05:00:32.307 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:32.301+0000 7f2519b0c700 1 -- 172.21.15.46:0/2823469333 wait complete. 2024-05-09T05:00:33.050 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T05:00:33.076 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:32 smithi046 ceph-mon[30136]: pgmap v292: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-09T05:00:33.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:32 smithi072 ceph-mon[34088]: pgmap v292: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-09T05:00:33.306 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:33.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:32 smithi110 ceph-mon[34904]: pgmap v292: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 9 op/s 2024-05-09T05:00:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.990+0000 7fc3163bc700 1 -- 172.21.15.46:0/1697741138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3101045d0 msgr2=0x7fc3101049e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.990+0000 7fc3163bc700 1 --2- 172.21.15.46:0/1697741138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3101045d0 0x7fc3101049e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fc3000099c0 tx=0x7fc300009cd0 comp rx=0 tx=0).stop 2024-05-09T05:00:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.990+0000 7fc3163bc700 1 -- 172.21.15.46:0/1697741138 shutdown_connections 2024-05-09T05:00:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.990+0000 7fc3163bc700 1 --2- 172.21.15.46:0/1697741138 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc310109850 0x7fc3101111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.990+0000 7fc3163bc700 1 --2- 172.21.15.46:0/1697741138 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc3101045d0 0x7fc3101049e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:34.991 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.990+0000 7fc3163bc700 1 --2- 172.21.15.46:0/1697741138 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc310108f00 0x7fc310109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.990+0000 7fc3163bc700 1 -- 172.21.15.46:0/1697741138 >> 172.21.15.46:0/1697741138 conn(0x7fc3100ffea0 msgr2=0x7fc3101022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.990+0000 7fc3163bc700 1 -- 172.21.15.46:0/1697741138 shutdown_connections 2024-05-09T05:00:34.992 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.990+0000 7fc3163bc700 1 -- 172.21.15.46:0/1697741138 wait complete. 2024-05-09T05:00:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.991+0000 7fc3163bc700 1 Processor -- start 2024-05-09T05:00:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.991+0000 7fc3163bc700 1 -- start start 2024-05-09T05:00:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.991+0000 7fc3163bc700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc3101045d0 0x7fc310076bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:34.993 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.991+0000 7fc3163bc700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc310108f00 0x7fc310077110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.991+0000 7fc3163bc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc310109850 0x7fc310077650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.991+0000 7fc3163bc700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc310114480 con 0x7fc310109850 2024-05-09T05:00:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.991+0000 7fc3163bc700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc310114300 con 0x7fc3101045d0 2024-05-09T05:00:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.991+0000 7fc3163bc700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc310114600 con 0x7fc310108f00 2024-05-09T05:00:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc314959700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc310109850 0x7fc310077650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc314959700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc310109850 0x7fc310077650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51364/0 (socket says 172.21.15.46:51364) 2024-05-09T05:00:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc314959700 1 -- 172.21.15.46:0/2585316239 learned_addr learned my addr 172.21.15.46:0/2585316239 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc30ffff700 1 --2- 172.21.15.46:0/2585316239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc3101045d0 0x7fc310076bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc307fff700 1 --2- 172.21.15.46:0/2585316239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc310108f00 0x7fc310077110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc314959700 1 -- 172.21.15.46:0/2585316239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc310108f00 msgr2=0x7fc310077110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc314959700 1 --2- 172.21.15.46:0/2585316239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc310108f00 0x7fc310077110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc314959700 1 -- 172.21.15.46:0/2585316239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc3101045d0 msgr2=0x7fc310076bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc314959700 1 --2- 172.21.15.46:0/2585316239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc3101045d0 0x7fc310076bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:34.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc314959700 1 -- 172.21.15.46:0/2585316239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3101a9170 con 0x7fc310109850 2024-05-09T05:00:34.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc314959700 1 --2- 172.21.15.46:0/2585316239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc310109850 0x7fc310077650 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc30800ef30 tx=0x7fc30800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:34.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc30dffb700 1 -- 172.21.15.46:0/2585316239 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3080092e0 con 0x7fc310109850 2024-05-09T05:00:34.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc3163bc700 1 -- 172.21.15.46:0/2585316239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3101a9400 con 0x7fc310109850 2024-05-09T05:00:34.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc30dffb700 1 -- 172.21.15.46:0/2585316239 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc308004bb0 con 0x7fc310109850 2024-05-09T05:00:34.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc30dffb700 1 -- 172.21.15.46:0/2585316239 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc308004020 con 0x7fc310109850 2024-05-09T05:00:34.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.992+0000 7fc3163bc700 1 -- 172.21.15.46:0/2585316239 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3101a9940 con 0x7fc310109850 2024-05-09T05:00:34.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.995+0000 7fc30dffb700 1 -- 172.21.15.46:0/2585316239 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc30801b030 con 0x7fc310109850 2024-05-09T05:00:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.996+0000 7fc30dffb700 1 --2- 172.21.15.46:0/2585316239 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc2fc0b29f0 0x7fc2fc0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.996+0000 7fc324dd2b80 1 Processor -- start 2024-05-09T05:00:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.996+0000 7fc324dd2b80 1 -- start start 2024-05-09T05:00:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.996+0000 7fc30ffff700 1 --2- 172.21.15.46:0/2585316239 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc2fc0b29f0 0x7fc2fc0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.996+0000 7fc30dffb700 1 -- 172.21.15.46:0/2585316239 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7fc3080e2110 con 0x7fc310109850 2024-05-09T05:00:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.996+0000 7fc324dd2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ff74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.996+0000 7fc324dd2b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34e744a0 0x55ae34ff7a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc324dd2b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34ff16a0 0x55ae34ff1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:34.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc324dd2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ae34f13ff0 con 0x55ae34f5e160 2024-05-09T05:00:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc324dd2b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ae34eb27f0 con 0x55ae34e744a0 2024-05-09T05:00:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc324dd2b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ae34f0ffb0 con 0x55ae34ff16a0 2024-05-09T05:00:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc30ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ff74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc30ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ff74f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51378/0 (socket says 172.21.15.46:51378) 2024-05-09T05:00:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc30ffff700 1 -- 172.21.15.46:0/1794417706 learned_addr learned my addr 172.21.15.46:0/1794417706 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc307fff700 1 --2- 172.21.15.46:0/1794417706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34e744a0 0x55ae34ff7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc314959700 1 --2- 172.21.15.46:0/1794417706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34ff16a0 0x55ae34ff1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc30ffff700 1 --2- 172.21.15.46:0/2585316239 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc2fc0b29f0 0x7fc2fc0b4f00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc300000c00 tx=0x7fc300005040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc30ffff700 1 -- 172.21.15.46:0/1794417706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34ff16a0 msgr2=0x55ae34ff1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc30ffff700 1 --2- 172.21.15.46:0/1794417706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34ff16a0 0x55ae34ff1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc30ffff700 1 -- 172.21.15.46:0/1794417706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34e744a0 msgr2=0x55ae34ff7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc30ffff700 1 --2- 172.21.15.46:0/1794417706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34e744a0 0x55ae34ff7a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc30ffff700 1 -- 172.21.15.46:0/1794417706 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae34eaa370 con 0x55ae34f5e160 2024-05-09T05:00:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc30ffff700 1 --2- 172.21.15.46:0/1794417706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ff74f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fc300003760 tx=0x7fc30002a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2905668f9f4804c server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:34.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.997+0000 7fc3067fc700 1 -- 172.21.15.46:0/1794417706 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3000038c0 con 0x55ae34f5e160 2024-05-09T05:00:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.998+0000 7fc3067fc700 1 -- 172.21.15.46:0/1794417706 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc30002d070 con 0x55ae34f5e160 2024-05-09T05:00:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.998+0000 7fc3067fc700 1 -- 172.21.15.46:0/1794417706 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc30000f6c0 con 0x55ae34f5e160 2024-05-09T05:00:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.998+0000 7fc324dd2b80 1 -- 172.21.15.46:0/1794417706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 msgr2=0x55ae34ff74f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:35.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.998+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/1794417706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ff74f0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fc300003760 tx=0x7fc30002a000 comp rx=0 tx=0).stop 2024-05-09T05:00:35.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.998+0000 7fc324dd2b80 1 -- 172.21.15.46:0/1794417706 shutdown_connections 2024-05-09T05:00:35.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.998+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/1794417706 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34ff16a0 0x55ae34ff1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.998+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/1794417706 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ff74f0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.998+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/1794417706 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34e744a0 0x55ae34ff7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.001 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.998+0000 7fc324dd2b80 1 -- 172.21.15.46:0/1794417706 >> 172.21.15.46:0/1794417706 conn(0x55ae34ed46a0 msgr2=0x55ae34fec400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:35.001 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:34 smithi046 ceph-mon[30136]: pgmap v293: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T05:00:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.999+0000 7fc324dd2b80 1 -- 172.21.15.46:0/1794417706 shutdown_connections 2024-05-09T05:00:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.999+0000 7fc324dd2b80 1 -- 172.21.15.46:0/1794417706 wait complete. 2024-05-09T05:00:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.999+0000 7fc324dd2b80 1 -- start start 2024-05-09T05:00:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:34.999+0000 7fc324dd2b80 1 Processor -- start 2024-05-09T05:00:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc324dd2b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34e744a0 0x55ae34ffb210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:35.005 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc324dd2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ffb750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc324dd2b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34ff16a0 0x55ae34ffbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc324dd2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ae34d8fba0 con 0x55ae34f5e160 2024-05-09T05:00:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc324dd2b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ae34ece150 con 0x55ae34ff16a0 2024-05-09T05:00:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc324dd2b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ae34ffe1d0 con 0x55ae34e744a0 2024-05-09T05:00:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc324dd2b80 10 client.0 _ll_register_callbacks cb 0x55ae34d856c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc307fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ffb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc307fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ffb750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51388/0 (socket says 172.21.15.46:51388) 2024-05-09T05:00:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc307fff700 1 -- 172.21.15.46:0/3447495421 learned_addr learned my addr 172.21.15.46:0/3447495421 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc314959700 1 --2- 172.21.15.46:0/3447495421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34ff16a0 0x55ae34ffbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc30ffff700 1 --2- 172.21.15.46:0/3447495421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34e744a0 0x55ae34ffb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc307fff700 1 -- 172.21.15.46:0/3447495421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34e744a0 msgr2=0x55ae34ffb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc307fff700 1 --2- 172.21.15.46:0/3447495421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34e744a0 0x55ae34ffb210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.006 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc307fff700 1 -- 172.21.15.46:0/3447495421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34ff16a0 msgr2=0x55ae34ffbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc307fff700 1 --2- 172.21.15.46:0/3447495421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34ff16a0 0x55ae34ffbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.000+0000 7fc307fff700 1 -- 172.21.15.46:0/3447495421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae34d8f490 con 0x55ae34f5e160 2024-05-09T05:00:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.001+0000 7fc307fff700 1 --2- 172.21.15.46:0/3447495421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ffb750 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8012770 tx=0x7fc2f8012a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c85b1b3c8ca41 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.001+0000 7fc2ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.001+0000 7fc2ef7fe700 1 -- 172.21.15.46:0/3447495421 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f800f480 con 0x55ae34f5e160 2024-05-09T05:00:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.001+0000 7fc2ef7fe700 1 -- 172.21.15.46:0/3447495421 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7fc2f800f620 con 0x55ae34f5e160 2024-05-09T05:00:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.001+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3447495421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc30002d070 con 0x55ae34f5e160 2024-05-09T05:00:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.001+0000 7fc2ef7fe700 1 -- 172.21.15.46:0/3447495421 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc2f800f820 con 0x55ae34f5e160 2024-05-09T05:00:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.001+0000 7fc324dd2b80 10 client.17361 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.001+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3447495421 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae35008c50 con 0x55ae34f5e160 2024-05-09T05:00:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.001+0000 7fc2ef7fe700 1 -- 172.21.15.46:0/3447495421 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc30002d070 con 0x55ae34f5e160 2024-05-09T05:00:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.001+0000 7fc2ef7fe700 1 -- 172.21.15.46:0/3447495421 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc2f80109b0 con 0x55ae34f5e160 2024-05-09T05:00:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.001+0000 7fc324dd2b80 10 client.17361 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.001+0000 7fc324dd2b80 10 client.17361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:35.007 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.001+0000 7fc324dd2b80 20 client.17361 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.001+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/3447495421 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae3500b330 0x55ae3500d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.002+0000 7fc324dd2b80 4 client.17361 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:00:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.002+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3447495421 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae34f219d0 con 0x55ae3500b330 2024-05-09T05:00:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.002+0000 7fc30ffff700 1 --2- 172.21.15.46:0/3447495421 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae3500b330 0x55ae3500d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.002+0000 7fc2ef7fe700 10 client.17361 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.002+0000 7fc30ffff700 1 --2- 172.21.15.46:0/3447495421 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae3500b330 0x55ae3500d780 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c112f97a8534ada server_cookie=e4648d517965cdd in_seq=0 out_seq=0 2024-05-09T05:00:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.003+0000 7fc2ef7fe700 1 -- 172.21.15.46:0/3447495421 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ae34f219d0 con 0x55ae3500b330 2024-05-09T05:00:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.003+0000 7fc2ef7fe700 10 client.17361 handle_command_reply: tid=0 2024-05-09T05:00:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.003+0000 7fc324dd2b80 1 client.17361 shutdown 2024-05-09T05:00:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.003+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3447495421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 msgr2=0x55ae34ffb750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.003+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/3447495421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ffb750 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8012770 tx=0x7fc2f8012a80 comp rx=0 tx=0).stop 2024-05-09T05:00:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.003+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3447495421 shutdown_connections 2024-05-09T05:00:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.003+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/3447495421 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae3500b330 0x55ae3500d780 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.008 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.003+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/3447495421 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34ff16a0 0x55ae34ffbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.003+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/3447495421 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34e744a0 0x55ae34ffb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.003+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/3447495421 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ffb750 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.003+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3447495421 >> 172.21.15.46:0/3447495421 conn(0x55ae34ed46a0 msgr2=0x55ae34fec1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.003+0000 7fc2ef7fe700 0 client.17361 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.004+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3447495421 shutdown_connections 2024-05-09T05:00:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.004+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3447495421 wait complete. 2024-05-09T05:00:35.009 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.004+0000 7fc324dd2b80 20 client.17361 trim_cache size 0 max 16384 2024-05-09T05:00:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.021+0000 7fc324dd2b80 1 Processor -- start 2024-05-09T05:00:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.021+0000 7fc324dd2b80 1 -- start start 2024-05-09T05:00:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.021+0000 7fc324dd2b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34e744a0 0x55ae34ff6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc324dd2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ffbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc324dd2b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34ff16a0 0x55ae34ff4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc324dd2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc3101b3920 con 0x55ae34f5e160 2024-05-09T05:00:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc324dd2b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc3101b11b0 con 0x55ae34e744a0 2024-05-09T05:00:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc324dd2b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc3101af020 con 0x55ae34ff16a0 2024-05-09T05:00:35.023 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc307fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ffbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc307fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ffbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51392/0 (socket says 172.21.15.46:51392) 2024-05-09T05:00:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc307fff700 1 -- 172.21.15.46:0/2804107157 learned_addr learned my addr 172.21.15.46:0/2804107157 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc30ffff700 1 --2- 172.21.15.46:0/2804107157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34e744a0 0x55ae34ff6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc314959700 1 --2- 172.21.15.46:0/2804107157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34ff16a0 0x55ae34ff4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc307fff700 1 -- 172.21.15.46:0/2804107157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34ff16a0 msgr2=0x55ae34ff4e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc307fff700 1 --2- 172.21.15.46:0/2804107157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34ff16a0 0x55ae34ff4e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc307fff700 1 -- 172.21.15.46:0/2804107157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34e744a0 msgr2=0x55ae34ff6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc307fff700 1 --2- 172.21.15.46:0/2804107157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34e744a0 0x55ae34ff6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc307fff700 1 -- 172.21.15.46:0/2804107157 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae34ff7050 con 0x55ae34f5e160 2024-05-09T05:00:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc30ffff700 1 --2- 172.21.15.46:0/2804107157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34e744a0 0x55ae34ff6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc307fff700 1 --2- 172.21.15.46:0/2804107157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ffbad0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8010d20 tx=0x7fc2f801c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bf68a0a1de85968 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.022+0000 7fc305ffb700 1 -- 172.21.15.46:0/2804107157 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f800f480 con 0x55ae34f5e160 2024-05-09T05:00:35.024 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.023+0000 7fc305ffb700 1 -- 172.21.15.46:0/2804107157 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f800fe70 con 0x55ae34f5e160 2024-05-09T05:00:35.025 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.023+0000 7fc305ffb700 1 -- 172.21.15.46:0/2804107157 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2f80050d0 con 0x55ae34f5e160 2024-05-09T05:00:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.023+0000 7fc324dd2b80 1 -- 172.21.15.46:0/2804107157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 msgr2=0x55ae34ffbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.023+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/2804107157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ffbad0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fc2f8010d20 tx=0x7fc2f801c630 comp rx=0 tx=0).stop 2024-05-09T05:00:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.023+0000 7fc324dd2b80 1 -- 172.21.15.46:0/2804107157 shutdown_connections 2024-05-09T05:00:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.023+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/2804107157 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34ff16a0 0x55ae34ff4e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.023+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/2804107157 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34e744a0 0x55ae34ff6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.023+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/2804107157 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34f5e160 0x55ae34ffbad0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.023+0000 7fc324dd2b80 1 -- 172.21.15.46:0/2804107157 >> 172.21.15.46:0/2804107157 conn(0x55ae34ed46a0 msgr2=0x55ae34ff6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.024+0000 7fc324dd2b80 1 -- 172.21.15.46:0/2804107157 shutdown_connections 2024-05-09T05:00:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.024+0000 7fc324dd2b80 1 -- 172.21.15.46:0/2804107157 wait complete. 2024-05-09T05:00:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.024+0000 7fc324dd2b80 1 -- start start 2024-05-09T05:00:35.029 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.024+0000 7fc324dd2b80 1 Processor -- start 2024-05-09T05:00:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.024+0000 7fc324dd2b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34e744a0 0x55ae350032b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc324dd2b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34f5e160 0x55ae350037f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc324dd2b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34ff16a0 0x55ae35011f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc324dd2b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc3101af670 con 0x55ae34e744a0 2024-05-09T05:00:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc324dd2b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc3101b0ae0 con 0x55ae34ff16a0 2024-05-09T05:00:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc324dd2b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc3101afd40 con 0x55ae34f5e160 2024-05-09T05:00:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc324dd2b80 10 client.0 _ll_register_callbacks cb 0x55ae34d856c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc30ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34e744a0 0x55ae350032b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc30ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34e744a0 0x55ae350032b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51404/0 (socket says 172.21.15.46:51404) 2024-05-09T05:00:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc30ffff700 1 -- 172.21.15.46:0/3221877863 learned_addr learned my addr 172.21.15.46:0/3221877863 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc314959700 1 --2- 172.21.15.46:0/3221877863 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34ff16a0 0x55ae35011f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc307fff700 1 --2- 172.21.15.46:0/3221877863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34f5e160 0x55ae350037f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc30ffff700 1 -- 172.21.15.46:0/3221877863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34f5e160 msgr2=0x55ae350037f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:35.030 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc30ffff700 1 --2- 172.21.15.46:0/3221877863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34f5e160 0x55ae350037f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc30ffff700 1 -- 172.21.15.46:0/3221877863 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34ff16a0 msgr2=0x55ae35011f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc30ffff700 1 --2- 172.21.15.46:0/3221877863 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34ff16a0 0x55ae35011f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc30ffff700 1 -- 172.21.15.46:0/3221877863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae350138c0 con 0x55ae34e744a0 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc314959700 1 --2- 172.21.15.46:0/3221877863 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34ff16a0 0x55ae35011f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc30ffff700 1 --2- 172.21.15.46:0/3221877863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34e744a0 0x55ae350032b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc300005010 tx=0x7fc30000b600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b27363a0a51472 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.025+0000 7fc2ef7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.026+0000 7fc2ef7fe700 1 -- 172.21.15.46:0/3221877863 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc300045070 con 0x55ae34e744a0 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.026+0000 7fc2ef7fe700 1 -- 172.21.15.46:0/3221877863 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7fc30000b910 con 0x55ae34e744a0 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.026+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3221877863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2f800fe70 con 0x55ae34e744a0 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.026+0000 7fc2ef7fe700 1 -- 172.21.15.46:0/3221877863 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc300051050 con 0x55ae34e744a0 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.026+0000 7fc324dd2b80 10 client.17367 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.026+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3221877863 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ae35074fd0 con 0x55ae34e744a0 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.026+0000 7fc2ef7fe700 1 -- 172.21.15.46:0/3221877863 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2f800fe70 con 0x55ae34e744a0 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.026+0000 7fc2ef7fe700 1 -- 172.21.15.46:0/3221877863 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc300024070 con 0x55ae34e744a0 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.026+0000 7fc324dd2b80 10 client.17367 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:35.031 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.026+0000 7fc324dd2b80 10 client.17367 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.026+0000 7fc324dd2b80 20 client.17367 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.026+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/3221877863 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae35075260 0x55ae35077680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.026+0000 7fc324dd2b80 4 client.17367 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.026+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3221877863 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae34f1ba70 con 0x55ae35075260 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.027+0000 7fc307fff700 1 --2- 172.21.15.46:0/3221877863 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae35075260 0x55ae35077680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.027+0000 7fc307fff700 1 --2- 172.21.15.46:0/3221877863 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae35075260 0x55ae35077680 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=671410f4907bb627 server_cookie=7deae15897377822 in_seq=0 out_seq=0 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.027+0000 7fc2ef7fe700 10 client.17367 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.027+0000 7fc2ef7fe700 1 -- 172.21.15.46:0/3221877863 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae34f1ba70 con 0x55ae35075260 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.027+0000 7fc2ef7fe700 10 client.17367 handle_command_reply: tid=0 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.027+0000 7fc324dd2b80 1 client.17367 shutdown 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.027+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3221877863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34e744a0 msgr2=0x55ae350032b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.027+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/3221877863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34e744a0 0x55ae350032b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fc300005010 tx=0x7fc30000b600 comp rx=0 tx=0).stop 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.027+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3221877863 shutdown_connections 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.027+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/3221877863 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ae35075260 0x55ae35077680 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.027+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/3221877863 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ae34ff16a0 0x55ae35011f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.032 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.027+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/3221877863 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ae34e744a0 0x55ae350032b0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.027+0000 7fc324dd2b80 1 --2- 172.21.15.46:0/3221877863 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ae34f5e160 0x55ae350037f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.027+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3221877863 >> 172.21.15.46:0/3221877863 conn(0x55ae34ed46a0 msgr2=0x55ae34ff4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:35.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.028+0000 7fc2ef7fe700 0 client.17367 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:35.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.028+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3221877863 shutdown_connections 2024-05-09T05:00:35.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.028+0000 7fc324dd2b80 1 -- 172.21.15.46:0/3221877863 wait complete. 2024-05-09T05:00:35.033 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.028+0000 7fc324dd2b80 20 client.17367 trim_cache size 0 max 16384 2024-05-09T05:00:35.033 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T05:00:35.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.028+0000 7fc3163bc700 1 -- 172.21.15.46:0/2585316239 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc2fc0b29f0 msgr2=0x7fc2fc0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:35.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.028+0000 7fc3163bc700 1 --2- 172.21.15.46:0/2585316239 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc2fc0b29f0 0x7fc2fc0b4f00 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc300000c00 tx=0x7fc300005040 comp rx=0 tx=0).stop 2024-05-09T05:00:35.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.028+0000 7fc3163bc700 1 -- 172.21.15.46:0/2585316239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc310109850 msgr2=0x7fc310077650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:35.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.028+0000 7fc3163bc700 1 --2- 172.21.15.46:0/2585316239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc310109850 0x7fc310077650 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fc30800ef30 tx=0x7fc30800c3a0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.028+0000 7fc3163bc700 1 -- 172.21.15.46:0/2585316239 shutdown_connections 2024-05-09T05:00:35.034 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.028+0000 7fc3163bc700 1 --2- 172.21.15.46:0/2585316239 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc2fc0b29f0 0x7fc2fc0b4f00 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.028+0000 7fc3163bc700 1 --2- 172.21.15.46:0/2585316239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc310109850 0x7fc310077650 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.028+0000 7fc3163bc700 1 --2- 172.21.15.46:0/2585316239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc3101045d0 0x7fc310076bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.028+0000 7fc3163bc700 1 --2- 172.21.15.46:0/2585316239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc310108f00 0x7fc310077110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:35.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.028+0000 7fc3163bc700 1 -- 172.21.15.46:0/2585316239 >> 172.21.15.46:0/2585316239 conn(0x7fc3100ffea0 msgr2=0x7fc3101022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:35.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.028+0000 7fc3163bc700 1 -- 172.21.15.46:0/2585316239 shutdown_connections 2024-05-09T05:00:35.035 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:35.029+0000 7fc3163bc700 1 -- 172.21.15.46:0/2585316239 wait complete. 2024-05-09T05:00:35.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:34 smithi072 ceph-mon[34088]: pgmap v293: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T05:00:35.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:34 smithi110 ceph-mon[34904]: pgmap v293: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T05:00:36.717 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T05:00:36.717 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T05:00:36.985 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:37.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:36 smithi046 ceph-mon[30136]: pgmap v294: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:00:37.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:36 smithi072 ceph-mon[34088]: pgmap v294: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:00:37.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:36 smithi110 ceph-mon[34904]: pgmap v294: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:00:38.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.760+0000 7ff1688a7700 1 -- 172.21.15.46:0/4212179984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164108f10 msgr2=0x7ff164109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:38.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.760+0000 7ff1688a7700 1 --2- 172.21.15.46:0/4212179984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164108f10 0x7ff164109320 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0099c0 tx=0x7ff14c009cd0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.761+0000 7ff1688a7700 1 -- 172.21.15.46:0/4212179984 shutdown_connections 2024-05-09T05:00:38.761 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.761+0000 7ff1688a7700 1 --2- 172.21.15.46:0/4212179984 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff164109860 0x7ff1641111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.761+0000 7ff1688a7700 1 --2- 172.21.15.46:0/4212179984 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff1641045e0 0x7ff1641049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.761+0000 7ff1688a7700 1 --2- 172.21.15.46:0/4212179984 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164108f10 0x7ff164109320 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.761+0000 7ff1688a7700 1 -- 172.21.15.46:0/4212179984 >> 172.21.15.46:0/4212179984 conn(0x7ff1640ffe70 msgr2=0x7ff1641022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:38.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.761+0000 7ff1688a7700 1 -- 172.21.15.46:0/4212179984 shutdown_connections 2024-05-09T05:00:38.762 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.761+0000 7ff1688a7700 1 -- 172.21.15.46:0/4212179984 wait complete. 2024-05-09T05:00:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.762+0000 7ff1688a7700 1 Processor -- start 2024-05-09T05:00:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.762+0000 7ff1688a7700 1 -- start start 2024-05-09T05:00:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.762+0000 7ff1688a7700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1641045e0 0x7ff16419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.762+0000 7ff1688a7700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164108f10 0x7ff16419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.762+0000 7ff1688a7700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff164109860 0x7ff16419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.762+0000 7ff1688a7700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641144c0 con 0x7ff164108f10 2024-05-09T05:00:38.770 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.762+0000 7ff1688a7700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff164114340 con 0x7ff164109860 2024-05-09T05:00:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.762+0000 7ff1688a7700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff164114640 con 0x7ff1641045e0 2024-05-09T05:00:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.762+0000 7ff161d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164108f10 0x7ff16419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.762+0000 7ff161d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164108f10 0x7ff16419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51422/0 (socket says 172.21.15.46:51422) 2024-05-09T05:00:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.762+0000 7ff161d9b700 1 -- 172.21.15.46:0/1330817253 learned_addr learned my addr 172.21.15.46:0/1330817253 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.762+0000 7ff16259c700 1 --2- 172.21.15.46:0/1330817253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1641045e0 0x7ff16419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.762+0000 7ff162d9d700 1 --2- 172.21.15.46:0/1330817253 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff164109860 0x7ff16419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.763+0000 7ff161d9b700 1 -- 172.21.15.46:0/1330817253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1641045e0 msgr2=0x7ff16419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.763+0000 7ff161d9b700 1 --2- 172.21.15.46:0/1330817253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1641045e0 0x7ff16419ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.763+0000 7ff161d9b700 1 -- 172.21.15.46:0/1330817253 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff164109860 msgr2=0x7ff16419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.763+0000 7ff161d9b700 1 --2- 172.21.15.46:0/1330817253 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff164109860 0x7ff16419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.763+0000 7ff161d9b700 1 -- 172.21.15.46:0/1330817253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff16419fcb0 con 0x7ff164108f10 2024-05-09T05:00:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.763+0000 7ff162d9d700 1 --2- 172.21.15.46:0/1330817253 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff164109860 0x7ff16419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.763+0000 7ff161d9b700 1 --2- 172.21.15.46:0/1330817253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164108f10 0x7ff16419b430 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff14c006010 tx=0x7ff14c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.763+0000 7ff15b7fe700 1 -- 172.21.15.46:0/1330817253 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c026070 con 0x7ff164108f10 2024-05-09T05:00:38.771 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.763+0000 7ff1688a7700 1 -- 172.21.15.46:0/1330817253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff16419ff40 con 0x7ff164108f10 2024-05-09T05:00:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.763+0000 7ff15b7fe700 1 -- 172.21.15.46:0/1330817253 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff14c003f20 con 0x7ff164108f10 2024-05-09T05:00:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.763+0000 7ff15b7fe700 1 -- 172.21.15.46:0/1330817253 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c00fcc0 con 0x7ff164108f10 2024-05-09T05:00:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.763+0000 7ff1688a7700 1 -- 172.21.15.46:0/1330817253 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1641a0420 con 0x7ff164108f10 2024-05-09T05:00:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.766+0000 7ff15b7fe700 1 -- 172.21.15.46:0/1330817253 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff14c029070 con 0x7ff164108f10 2024-05-09T05:00:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.767+0000 7ff15b7fe700 1 --2- 172.21.15.46:0/1330817253 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff1500b29f0 0x7ff1500b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.767+0000 7ff16259c700 1 --2- 172.21.15.46:0/1330817253 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff1500b29f0 0x7ff1500b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.767+0000 7ff15b7fe700 1 -- 172.21.15.46:0/1330817253 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7ff14c0e2fe0 con 0x7ff164108f10 2024-05-09T05:00:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.767+0000 7ff1772bdb80 1 Processor -- start 2024-05-09T05:00:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.767+0000 7ff16259c700 1 --2- 172.21.15.46:0/1330817253 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff1500b29f0 0x7ff1500b4f00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff154009550 tx=0x7ff1540091b0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.767+0000 7ff1772bdb80 1 -- start start 2024-05-09T05:00:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.767+0000 7ff1772bdb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8af3b1e0 0x55ec8afd4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.767+0000 7ff1772bdb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8ae51520 0x55ec8afd4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.767+0000 7ff1772bdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afceb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.772 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.767+0000 7ff1772bdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec8aef1070 con 0x55ec8afce720 2024-05-09T05:00:38.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.767+0000 7ff1772bdb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ec8ae8f870 con 0x55ec8af3b1e0 2024-05-09T05:00:38.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.767+0000 7ff1772bdb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ec8aeed030 con 0x55ec8ae51520 2024-05-09T05:00:38.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.768+0000 7ff162d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.768+0000 7ff162d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afceb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51434/0 (socket says 172.21.15.46:51434) 2024-05-09T05:00:38.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.768+0000 7ff162d9d700 1 -- 172.21.15.46:0/1788112912 learned_addr learned my addr 172.21.15.46:0/1788112912 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:38.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.768+0000 7ff16259c700 1 --2- 172.21.15.46:0/1788112912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8af3b1e0 0x55ec8afd4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.768+0000 7ff161d9b700 1 --2- 172.21.15.46:0/1788112912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8ae51520 0x55ec8afd4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.768+0000 7ff162d9d700 1 -- 172.21.15.46:0/1788112912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8ae51520 msgr2=0x55ec8afd4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:38.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.768+0000 7ff162d9d700 1 --2- 172.21.15.46:0/1788112912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8ae51520 0x55ec8afd4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.768+0000 7ff162d9d700 1 -- 172.21.15.46:0/1788112912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8af3b1e0 msgr2=0x55ec8afd4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:38.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.768+0000 7ff162d9d700 1 --2- 172.21.15.46:0/1788112912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8af3b1e0 0x55ec8afd4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.768+0000 7ff162d9d700 1 -- 172.21.15.46:0/1788112912 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec8ae873f0 con 0x55ec8afce720 2024-05-09T05:00:38.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.768+0000 7ff162d9d700 1 --2- 172.21.15.46:0/1788112912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afceb50 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00cc70 tx=0x7ff15c00cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e3692c268e0b6d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:38.773 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.768+0000 7ff158ff9700 1 -- 172.21.15.46:0/1788112912 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00ed70 con 0x55ec8afce720 2024-05-09T05:00:38.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.768+0000 7ff158ff9700 1 -- 172.21.15.46:0/1788112912 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff15c0044d0 con 0x55ec8afce720 2024-05-09T05:00:38.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.768+0000 7ff158ff9700 1 -- 172.21.15.46:0/1788112912 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c005240 con 0x55ec8afce720 2024-05-09T05:00:38.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.769+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1788112912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 msgr2=0x55ec8afceb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:38.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.769+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/1788112912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afceb50 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00cc70 tx=0x7ff15c00cf80 comp rx=0 tx=0).stop 2024-05-09T05:00:38.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.769+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1788112912 shutdown_connections 2024-05-09T05:00:38.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.769+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/1788112912 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afceb50 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.769+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/1788112912 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8af3b1e0 0x55ec8afd4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.769+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/1788112912 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8ae51520 0x55ec8afd4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.769+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1788112912 >> 172.21.15.46:0/1788112912 conn(0x55ec8aeb1720 msgr2=0x55ec8afc9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:38.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.769+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1788112912 shutdown_connections 2024-05-09T05:00:38.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.770+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1788112912 wait complete. 2024-05-09T05:00:38.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.770+0000 7ff1772bdb80 1 -- start start 2024-05-09T05:00:38.774 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.770+0000 7ff1772bdb80 1 Processor -- start 2024-05-09T05:00:38.777 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.770+0000 7ff1772bdb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 0x55ec8afd82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.770+0000 7ff1772bdb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8af3b1e0 0x55ec8afd8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.770+0000 7ff1772bdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afd8d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.770+0000 7ff1772bdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ec8ad6cc20 con 0x55ec8afce720 2024-05-09T05:00:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.770+0000 7ff1772bdb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ec8aeab1d0 con 0x55ec8ae51520 2024-05-09T05:00:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.770+0000 7ff1772bdb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ec8afdb250 con 0x55ec8af3b1e0 2024-05-09T05:00:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.770+0000 7ff1772bdb80 10 client.0 _ll_register_callbacks cb 0x55ec8ad62740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff16259c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 0x55ec8afd82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff162d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afd8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff161d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8af3b1e0 0x55ec8afd8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff16259c700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 0x55ec8afd82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:53734/0 (socket says 172.21.15.46:53734) 2024-05-09T05:00:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff16259c700 1 -- 172.21.15.46:0/1034332180 learned_addr learned my addr 172.21.15.46:0/1034332180 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff162d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afd8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51448/0 (socket says 172.21.15.46:51448) 2024-05-09T05:00:38.778 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff16259c700 1 -- 172.21.15.46:0/1034332180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8af3b1e0 msgr2=0x55ec8afd8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff16259c700 1 --2- 172.21.15.46:0/1034332180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8af3b1e0 0x55ec8afd8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff16259c700 1 -- 172.21.15.46:0/1034332180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 msgr2=0x55ec8afd8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff16259c700 1 --2- 172.21.15.46:0/1034332180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afd8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff16259c700 1 -- 172.21.15.46:0/1034332180 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec8ad6c510 con 0x55ec8ae51520 2024-05-09T05:00:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff161d9b700 1 --2- 172.21.15.46:0/1034332180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8af3b1e0 0x55ec8afd8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:00:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff162d9d700 1 --2- 172.21.15.46:0/1034332180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afd8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:00:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff16259c700 1 --2- 172.21.15.46:0/1034332180 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 0x55ec8afd82e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff154007ab0 tx=0x7ff154004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8dd4cc21e3fac4db server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff149ffb700 10 client.0 ms_handle_connect on v2:172.21.15.72:3300/0 2024-05-09T05:00:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff149ffb700 1 -- 172.21.15.46:0/1034332180 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff154026070 con 0x55ec8ae51520 2024-05-09T05:00:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff149ffb700 1 -- 172.21.15.46:0/1034332180 <== mon.2 v2:172.21.15.72:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7ff154015ad0 con 0x55ec8ae51520 2024-05-09T05:00:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.771+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1034332180 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff15c0044d0 con 0x55ec8ae51520 2024-05-09T05:00:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.772+0000 7ff149ffb700 1 -- 172.21.15.46:0/1034332180 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff154032050 con 0x55ec8ae51520 2024-05-09T05:00:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.772+0000 7ff1772bdb80 10 client.26879 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.772+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1034332180 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ec8afe5cc0 con 0x55ec8ae51520 2024-05-09T05:00:38.779 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.772+0000 7ff149ffb700 1 -- 172.21.15.46:0/1034332180 <== mon.2 v2:172.21.15.72:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff15c0044d0 con 0x55ec8ae51520 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.772+0000 7ff149ffb700 1 -- 172.21.15.46:0/1034332180 <== mon.2 v2:172.21.15.72:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff154022c90 con 0x55ec8ae51520 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.772+0000 7ff1772bdb80 10 client.26879 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.772+0000 7ff1772bdb80 10 client.26879 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.772+0000 7ff1772bdb80 20 client.26879 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.772+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/1034332180 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ec8afe82a0 0x55ec8afea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.772+0000 7ff1772bdb80 4 client.26879 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.772+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1034332180 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec8aefea50 con 0x55ec8afe82a0 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.772+0000 7ff161d9b700 1 --2- 172.21.15.46:0/1034332180 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ec8afe82a0 0x55ec8afea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.773+0000 7ff149ffb700 10 client.26879 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.773+0000 7ff161d9b700 1 --2- 172.21.15.46:0/1034332180 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ec8afe82a0 0x55ec8afea720 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e8d5fc9cce3c809 server_cookie=d30fd9659ed730d3 in_seq=0 out_seq=0 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.773+0000 7ff149ffb700 1 -- 172.21.15.46:0/1034332180 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ec8aefea50 con 0x55ec8afe82a0 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.773+0000 7ff149ffb700 10 client.26879 handle_command_reply: tid=0 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.774+0000 7ff1772bdb80 1 client.26879 shutdown 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.774+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1034332180 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 msgr2=0x55ec8afd82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.774+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/1034332180 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 0x55ec8afd82e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff154007ab0 tx=0x7ff154004b40 comp rx=0 tx=0).stop 2024-05-09T05:00:38.780 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.774+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1034332180 shutdown_connections 2024-05-09T05:00:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.774+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/1034332180 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afd8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.774+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/1034332180 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ec8afe82a0 0x55ec8afea720 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.774+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/1034332180 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 0x55ec8afd82e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.774+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/1034332180 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8af3b1e0 0x55ec8afd8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.774+0000 7ff149ffb700 0 client.26879 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.774+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1034332180 >> 172.21.15.46:0/1034332180 conn(0x55ec8aeb1720 msgr2=0x55ec8afc9240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.774+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1034332180 shutdown_connections 2024-05-09T05:00:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.774+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1034332180 wait complete. 2024-05-09T05:00:38.781 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.774+0000 7ff1772bdb80 20 client.26879 trim_cache size 0 max 16384 2024-05-09T05:00:38.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.795+0000 7ff1772bdb80 1 Processor -- start 2024-05-09T05:00:38.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.795+0000 7ff1772bdb80 1 -- start start 2024-05-09T05:00:38.796 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.795+0000 7ff1772bdb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 0x55ec8afd29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.795+0000 7ff1772bdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8af3b1e0 0x55ec8afce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.795+0000 7ff1772bdb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8afce720 0x55ec8afd8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff1772bdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641b17c0 con 0x55ec8af3b1e0 2024-05-09T05:00:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff1772bdb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641af050 con 0x55ec8ae51520 2024-05-09T05:00:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff1772bdb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641acec0 con 0x55ec8afce720 2024-05-09T05:00:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff161d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8af3b1e0 0x55ec8afce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff161d9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8af3b1e0 0x55ec8afce140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51456/0 (socket says 172.21.15.46:51456) 2024-05-09T05:00:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff161d9b700 1 -- 172.21.15.46:0/1796056221 learned_addr learned my addr 172.21.15.46:0/1796056221 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff16259c700 1 --2- 172.21.15.46:0/1796056221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 0x55ec8afd29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff162d9d700 1 --2- 172.21.15.46:0/1796056221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8afce720 0x55ec8afd8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff161d9b700 1 -- 172.21.15.46:0/1796056221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8afce720 msgr2=0x55ec8afd8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff161d9b700 1 --2- 172.21.15.46:0/1796056221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8afce720 0x55ec8afd8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.797 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff161d9b700 1 -- 172.21.15.46:0/1796056221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 msgr2=0x55ec8afd29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff161d9b700 1 --2- 172.21.15.46:0/1796056221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 0x55ec8afd29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff161d9b700 1 -- 172.21.15.46:0/1796056221 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec8afcf0b0 con 0x55ec8af3b1e0 2024-05-09T05:00:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff161d9b700 1 --2- 172.21.15.46:0/1796056221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8af3b1e0 0x55ec8afce140 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0a6340 tx=0x7ff14c003780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7706d54b67210ca9 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff14bfff700 1 -- 172.21.15.46:0/1796056221 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0b3090 con 0x55ec8af3b1e0 2024-05-09T05:00:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff14bfff700 1 -- 172.21.15.46:0/1796056221 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff14c0a6550 con 0x55ec8af3b1e0 2024-05-09T05:00:38.798 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.796+0000 7ff14bfff700 1 -- 172.21.15.46:0/1796056221 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff14c0ae170 con 0x55ec8af3b1e0 2024-05-09T05:00:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.797+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1796056221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8af3b1e0 msgr2=0x55ec8afce140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.797+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/1796056221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8af3b1e0 0x55ec8afce140 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7ff14c0a6340 tx=0x7ff14c003780 comp rx=0 tx=0).stop 2024-05-09T05:00:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.797+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1796056221 shutdown_connections 2024-05-09T05:00:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.797+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/1796056221 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8afce720 0x55ec8afd8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.797+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/1796056221 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 0x55ec8afd29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.797+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/1796056221 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8af3b1e0 0x55ec8afce140 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.801 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.797+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1796056221 >> 172.21.15.46:0/1796056221 conn(0x55ec8aeb1720 msgr2=0x55ec8afc8960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.797+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1796056221 shutdown_connections 2024-05-09T05:00:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.797+0000 7ff1772bdb80 1 -- 172.21.15.46:0/1796056221 wait complete. 2024-05-09T05:00:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.798+0000 7ff1772bdb80 1 -- start start 2024-05-09T05:00:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.798+0000 7ff1772bdb80 1 Processor -- start 2024-05-09T05:00:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.798+0000 7ff1772bdb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 0x55ec8afe03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.798+0000 7ff1772bdb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8af3b1e0 0x55ec8afe0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.798+0000 7ff1772bdb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afdb390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.798+0000 7ff1772bdb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641ad510 con 0x55ec8afce720 2024-05-09T05:00:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.798+0000 7ff1772bdb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641ae980 con 0x55ec8ae51520 2024-05-09T05:00:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.798+0000 7ff1772bdb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff1641adbe0 con 0x55ec8af3b1e0 2024-05-09T05:00:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.798+0000 7ff1772bdb80 10 client.0 _ll_register_callbacks cb 0x55ec8ad62740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.798+0000 7ff162d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afdb390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.802 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.798+0000 7ff161d9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8af3b1e0 0x55ec8afe0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.798+0000 7ff162d9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afdb390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51458/0 (socket says 172.21.15.46:51458) 2024-05-09T05:00:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.798+0000 7ff162d9d700 1 -- 172.21.15.46:0/4280032797 learned_addr learned my addr 172.21.15.46:0/4280032797 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.798+0000 7ff16259c700 1 --2- 172.21.15.46:0/4280032797 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 0x55ec8afe03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.799+0000 7ff162d9d700 1 -- 172.21.15.46:0/4280032797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8af3b1e0 msgr2=0x55ec8afe0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.799+0000 7ff162d9d700 1 --2- 172.21.15.46:0/4280032797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8af3b1e0 0x55ec8afe0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.799+0000 7ff162d9d700 1 -- 172.21.15.46:0/4280032797 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 msgr2=0x55ec8afe03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.799+0000 7ff162d9d700 1 --2- 172.21.15.46:0/4280032797 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 0x55ec8afe03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.799+0000 7ff162d9d700 1 -- 172.21.15.46:0/4280032797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec8afe4b40 con 0x55ec8afce720 2024-05-09T05:00:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.799+0000 7ff161d9b700 1 --2- 172.21.15.46:0/4280032797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8af3b1e0 0x55ec8afe0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:00:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.799+0000 7ff162d9d700 1 --2- 172.21.15.46:0/4280032797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afdb390 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00cc70 tx=0x7ff15c004940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340faad810f4a323 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.799+0000 7ff149ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.799+0000 7ff149ffb700 1 -- 172.21.15.46:0/4280032797 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff15c0050a0 con 0x55ec8afce720 2024-05-09T05:00:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.799+0000 7ff149ffb700 1 -- 172.21.15.46:0/4280032797 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7ff15c005240 con 0x55ec8afce720 2024-05-09T05:00:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.799+0000 7ff1772bdb80 1 -- 172.21.15.46:0/4280032797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff14c0a6550 con 0x55ec8afce720 2024-05-09T05:00:38.803 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.799+0000 7ff149ffb700 1 -- 172.21.15.46:0/4280032797 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff15c036050 con 0x55ec8afce720 2024-05-09T05:00:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.799+0000 7ff1772bdb80 10 client.17385 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.799+0000 7ff1772bdb80 1 -- 172.21.15.46:0/4280032797 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ec8b053070 con 0x55ec8afce720 2024-05-09T05:00:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.799+0000 7ff149ffb700 1 -- 172.21.15.46:0/4280032797 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff14c0a6550 con 0x55ec8afce720 2024-05-09T05:00:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.800+0000 7ff149ffb700 1 -- 172.21.15.46:0/4280032797 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff15c01a930 con 0x55ec8afce720 2024-05-09T05:00:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.800+0000 7ff1772bdb80 10 client.17385 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.800+0000 7ff1772bdb80 10 client.17385 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.800+0000 7ff1772bdb80 20 client.17385 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.800+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/4280032797 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ec8b053630 0x55ec8b055a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.800+0000 7ff1772bdb80 4 client.17385 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T05:00:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.800+0000 7ff1772bdb80 1 -- 172.21.15.46:0/4280032797 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ec8aef8af0 con 0x55ec8b053630 2024-05-09T05:00:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.800+0000 7ff16259c700 1 --2- 172.21.15.46:0/4280032797 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ec8b053630 0x55ec8b055a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.800+0000 7ff149ffb700 10 client.17385 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:38.804 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.800+0000 7ff16259c700 1 --2- 172.21.15.46:0/4280032797 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ec8b053630 0x55ec8b055a50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d3521b8419e9f73 server_cookie=91f1a030d4d58e0 in_seq=0 out_seq=0 2024-05-09T05:00:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.801+0000 7ff149ffb700 1 -- 172.21.15.46:0/4280032797 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ec8aef8af0 con 0x55ec8b053630 2024-05-09T05:00:38.805 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.801+0000 7ff149ffb700 10 client.17385 handle_command_reply: tid=0 2024-05-09T05:00:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.802+0000 7ff1772bdb80 1 client.17385 shutdown 2024-05-09T05:00:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.802+0000 7ff1772bdb80 1 -- 172.21.15.46:0/4280032797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 msgr2=0x55ec8afdb390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.802+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/4280032797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afdb390 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7ff15c00cc70 tx=0x7ff15c004940 comp rx=0 tx=0).stop 2024-05-09T05:00:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.802+0000 7ff1772bdb80 1 -- 172.21.15.46:0/4280032797 shutdown_connections 2024-05-09T05:00:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.802+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/4280032797 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ec8afce720 0x55ec8afdb390 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.802+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/4280032797 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55ec8b053630 0x55ec8b055a50 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.802+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/4280032797 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ec8ae51520 0x55ec8afe03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.802+0000 7ff1772bdb80 1 --2- 172.21.15.46:0/4280032797 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ec8af3b1e0 0x55ec8afe0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.802+0000 7ff1772bdb80 1 -- 172.21.15.46:0/4280032797 >> 172.21.15.46:0/4280032797 conn(0x55ec8aeb1720 msgr2=0x55ec8afc9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.802+0000 7ff149ffb700 0 client.17385 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.802+0000 7ff1772bdb80 1 -- 172.21.15.46:0/4280032797 shutdown_connections 2024-05-09T05:00:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.802+0000 7ff1772bdb80 1 -- 172.21.15.46:0/4280032797 wait complete. 2024-05-09T05:00:38.806 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.802+0000 7ff1772bdb80 20 client.17385 trim_cache size 0 max 16384 2024-05-09T05:00:38.807 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T05:00:38.807 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T05:00:38.807 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "596a7fd9-6afb-4c62-aab9-3d7ec6c23d46", 2024-05-09T05:00:38.807 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T05:00:38.807 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T05:00:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.803+0000 7ff1688a7700 1 -- 172.21.15.46:0/1330817253 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff1500b29f0 msgr2=0x7ff1500b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.803+0000 7ff1688a7700 1 --2- 172.21.15.46:0/1330817253 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff1500b29f0 0x7ff1500b4f00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff154009550 tx=0x7ff1540091b0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.803+0000 7ff1688a7700 1 -- 172.21.15.46:0/1330817253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164108f10 msgr2=0x7ff16419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.803+0000 7ff1688a7700 1 --2- 172.21.15.46:0/1330817253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164108f10 0x7ff16419b430 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7ff14c006010 tx=0x7ff14c0047b0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.803+0000 7ff1688a7700 1 -- 172.21.15.46:0/1330817253 shutdown_connections 2024-05-09T05:00:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.803+0000 7ff1688a7700 1 --2- 172.21.15.46:0/1330817253 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff1500b29f0 0x7ff1500b4f00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.803+0000 7ff1688a7700 1 --2- 172.21.15.46:0/1330817253 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff164109860 0x7ff16419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.803+0000 7ff1688a7700 1 --2- 172.21.15.46:0/1330817253 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff1641045e0 0x7ff16419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.803+0000 7ff1688a7700 1 --2- 172.21.15.46:0/1330817253 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff164108f10 0x7ff16419b430 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.803+0000 7ff1688a7700 1 -- 172.21.15.46:0/1330817253 >> 172.21.15.46:0/1330817253 conn(0x7ff1640ffe70 msgr2=0x7ff164107160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.803+0000 7ff1688a7700 1 -- 172.21.15.46:0/1330817253 shutdown_connections 2024-05-09T05:00:38.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:38.803+0000 7ff1688a7700 1 -- 172.21.15.46:0/1330817253 wait complete. 2024-05-09T05:00:39.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:38 smithi046 ceph-mon[30136]: pgmap v295: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:00:39.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:38 smithi072 ceph-mon[34088]: pgmap v295: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:00:39.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:38 smithi110 ceph-mon[34904]: pgmap v295: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:00:39.532 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:00:39.784 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:40.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:39 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T05:00:40.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:39 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:40.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:39 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T05:00:40.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:39 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:40.196 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:39 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T05:00:40.197 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:39 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:40.197 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:39 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T05:00:40.197 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:39 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:40.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:39 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T05:00:40.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:39 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:40.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:39 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T05:00:40.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:39 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:40.985 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:40 smithi046 ceph-mon[30136]: pgmap v296: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:00:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:40 smithi072 ceph-mon[34088]: pgmap v296: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:00:41.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:40 smithi110 ceph-mon[34904]: pgmap v296: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:00:41.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.619+0000 7f2285a79700 1 -- 172.21.15.46:0/1419593243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22801034e0 msgr2=0x7f22801038f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.619+0000 7f2285a79700 1 --2- 172.21.15.46:0/1419593243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22801034e0 0x7f22801038f0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f227000b0d0 tx=0x7f227000b3e0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.619+0000 7f2285a79700 1 -- 172.21.15.46:0/1419593243 shutdown_connections 2024-05-09T05:00:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.619+0000 7f2285a79700 1 --2- 172.21.15.46:0/1419593243 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2280108760 0x7f22801100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.619+0000 7f2285a79700 1 --2- 172.21.15.46:0/1419593243 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f22801034e0 0x7f22801038f0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.619+0000 7f2285a79700 1 --2- 172.21.15.46:0/1419593243 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f2280107e10 0x7f2280108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.619+0000 7f2285a79700 1 -- 172.21.15.46:0/1419593243 >> 172.21.15.46:0/1419593243 conn(0x7f22800fef30 msgr2=0x7f2280101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.619+0000 7f2285a79700 1 -- 172.21.15.46:0/1419593243 shutdown_connections 2024-05-09T05:00:41.620 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.619+0000 7f2285a79700 1 -- 172.21.15.46:0/1419593243 wait complete. 2024-05-09T05:00:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.620+0000 7f2285a79700 1 Processor -- start 2024-05-09T05:00:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.620+0000 7f2285a79700 1 -- start start 2024-05-09T05:00:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.620+0000 7f2285a79700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22801034e0 0x7f228019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.620+0000 7f2285a79700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2280107e10 0x7f228019c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.620+0000 7f2285a79700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2280108760 0x7f228019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.620+0000 7f2285a79700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f2280113390 con 0x7f2280108760 2024-05-09T05:00:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.620+0000 7f2285a79700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f2280113210 con 0x7f2280107e10 2024-05-09T05:00:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.620+0000 7f2285a79700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f2280113510 con 0x7f22801034e0 2024-05-09T05:00:41.622 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.621+0000 7f227ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2280108760 0x7f228019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.621+0000 7f227ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2280108760 0x7f228019c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51484/0 (socket says 172.21.15.46:51484) 2024-05-09T05:00:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.621+0000 7f227ffff700 1 -- 172.21.15.46:0/2543549619 learned_addr learned my addr 172.21.15.46:0/2543549619 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.621+0000 7f227effd700 1 --2- 172.21.15.46:0/2543549619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2280107e10 0x7f228019c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.621+0000 7f227f7fe700 1 --2- 172.21.15.46:0/2543549619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22801034e0 0x7f228019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.621+0000 7f227ffff700 1 -- 172.21.15.46:0/2543549619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22801034e0 msgr2=0x7f228019bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.621+0000 7f227ffff700 1 --2- 172.21.15.46:0/2543549619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22801034e0 0x7f228019bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.621+0000 7f227ffff700 1 -- 172.21.15.46:0/2543549619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2280107e10 msgr2=0x7f228019c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.621+0000 7f227ffff700 1 --2- 172.21.15.46:0/2543549619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2280107e10 0x7f228019c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.621+0000 7f227ffff700 1 -- 172.21.15.46:0/2543549619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22801a7000 con 0x7f2280108760 2024-05-09T05:00:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.621+0000 7f227f7fe700 1 --2- 172.21.15.46:0/2543549619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22801034e0 0x7f228019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.621+0000 7f227ffff700 1 --2- 172.21.15.46:0/2543549619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2280108760 0x7f228019c750 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f227800d980 tx=0x7f227800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.621+0000 7f227cff9700 1 -- 172.21.15.46:0/2543549619 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278017070 con 0x7f2280108760 2024-05-09T05:00:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.621+0000 7f2285a79700 1 -- 172.21.15.46:0/2543549619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22801a72f0 con 0x7f2280108760 2024-05-09T05:00:41.623 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.621+0000 7f227cff9700 1 -- 172.21.15.46:0/2543549619 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2278005020 con 0x7f2280108760 2024-05-09T05:00:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.622+0000 7f227cff9700 1 -- 172.21.15.46:0/2543549619 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22780099b0 con 0x7f2280108760 2024-05-09T05:00:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.622+0000 7f2285a79700 1 -- 172.21.15.46:0/2543549619 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22801a77e0 con 0x7f2280108760 2024-05-09T05:00:41.624 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.624+0000 7f227cff9700 1 -- 172.21.15.46:0/2543549619 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2278009b90 con 0x7f2280108760 2024-05-09T05:00:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.626+0000 7f227cff9700 1 --2- 172.21.15.46:0/2543549619 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f226c0b29f0 0x7f226c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.626+0000 7f227f7fe700 1 --2- 172.21.15.46:0/2543549619 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f226c0b29f0 0x7f226c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.626+0000 7f229448fb80 1 Processor -- start 2024-05-09T05:00:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.626+0000 7f227cff9700 1 -- 172.21.15.46:0/2543549619 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f22780e2530 con 0x7f2280108760 2024-05-09T05:00:41.627 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.626+0000 7f229448fb80 1 -- start start 2024-05-09T05:00:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.626+0000 7f229448fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561577f9ed00 0x5615780a7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.626+0000 7f227f7fe700 1 --2- 172.21.15.46:0/2543549619 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f226c0b29f0 0x7f226c0b4f00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f227000d010 tx=0x7f227000b040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.626+0000 7f229448fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f244a0 0x5615780a7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.626+0000 7f229448fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5615780a1400 0x5615780a1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.626+0000 7f229448fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x561577fc40a0 con 0x561577f244a0 2024-05-09T05:00:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.626+0000 7f229448fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x561577f699a0 con 0x5615780a1400 2024-05-09T05:00:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.626+0000 7f229448fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x561577fc0060 con 0x561577f9ed00 2024-05-09T05:00:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.626+0000 7f227effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f244a0 0x5615780a7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.626+0000 7f227effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f244a0 0x5615780a7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51488/0 (socket says 172.21.15.46:51488) 2024-05-09T05:00:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.626+0000 7f227effd700 1 -- 172.21.15.46:0/407630219 learned_addr learned my addr 172.21.15.46:0/407630219 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.627+0000 7f227f7fe700 1 --2- 172.21.15.46:0/407630219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561577f9ed00 0x5615780a7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.627+0000 7f227ffff700 1 --2- 172.21.15.46:0/407630219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5615780a1400 0x5615780a1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.627+0000 7f227effd700 1 -- 172.21.15.46:0/407630219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561577f9ed00 msgr2=0x5615780a7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.627+0000 7f227effd700 1 --2- 172.21.15.46:0/407630219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561577f9ed00 0x5615780a7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.628 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.627+0000 7f227effd700 1 -- 172.21.15.46:0/407630219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5615780a1400 msgr2=0x5615780a1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.627+0000 7f227effd700 1 --2- 172.21.15.46:0/407630219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5615780a1400 0x5615780a1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.627+0000 7f227effd700 1 -- 172.21.15.46:0/407630219 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561577f5a500 con 0x561577f244a0 2024-05-09T05:00:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.627+0000 7f227ffff700 1 --2- 172.21.15.46:0/407630219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5615780a1400 0x5615780a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.627+0000 7f227effd700 1 --2- 172.21.15.46:0/407630219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f244a0 0x5615780a7780 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f226800ac40 tx=0x7f226800af50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6729b3e8292a73 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.627+0000 7f2275ffb700 1 -- 172.21.15.46:0/407630219 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226801d070 con 0x561577f244a0 2024-05-09T05:00:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.627+0000 7f2275ffb700 1 -- 172.21.15.46:0/407630219 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f226800fdd0 con 0x561577f244a0 2024-05-09T05:00:41.629 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.627+0000 7f2275ffb700 1 -- 172.21.15.46:0/407630219 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268010850 con 0x561577f244a0 2024-05-09T05:00:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.628+0000 7f229448fb80 1 -- 172.21.15.46:0/407630219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f244a0 msgr2=0x5615780a7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.628+0000 7f229448fb80 1 --2- 172.21.15.46:0/407630219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f244a0 0x5615780a7780 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f226800ac40 tx=0x7f226800af50 comp rx=0 tx=0).stop 2024-05-09T05:00:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.628+0000 7f229448fb80 1 -- 172.21.15.46:0/407630219 shutdown_connections 2024-05-09T05:00:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.628+0000 7f229448fb80 1 --2- 172.21.15.46:0/407630219 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5615780a1400 0x5615780a1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.628+0000 7f229448fb80 1 --2- 172.21.15.46:0/407630219 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561577f9ed00 0x5615780a7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.628+0000 7f229448fb80 1 --2- 172.21.15.46:0/407630219 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f244a0 0x5615780a7780 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.628+0000 7f229448fb80 1 -- 172.21.15.46:0/407630219 >> 172.21.15.46:0/407630219 conn(0x561577f27f70 msgr2=0x56157809c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.628+0000 7f229448fb80 1 -- 172.21.15.46:0/407630219 shutdown_connections 2024-05-09T05:00:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.628+0000 7f229448fb80 1 -- 172.21.15.46:0/407630219 wait complete. 2024-05-09T05:00:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.629+0000 7f229448fb80 1 -- start start 2024-05-09T05:00:41.632 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.629+0000 7f229448fb80 1 Processor -- start 2024-05-09T05:00:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.629+0000 7f229448fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 0x5615780aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.629+0000 7f229448fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f9ed00 0x5615780ab4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.629+0000 7f229448fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615780a1400 0x5615780ab9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.629+0000 7f229448fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5615780a2160 con 0x561577f9ed00 2024-05-09T05:00:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.629+0000 7f229448fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x561577f5eb20 con 0x561577f244a0 2024-05-09T05:00:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.629+0000 7f229448fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5615780ae020 con 0x5615780a1400 2024-05-09T05:00:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.629+0000 7f229448fb80 10 client.0 _ll_register_callbacks cb 0x561577e356c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.629+0000 7f227effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f9ed00 0x5615780ab4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.629+0000 7f227effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f9ed00 0x5615780ab4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51504/0 (socket says 172.21.15.46:51504) 2024-05-09T05:00:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.629+0000 7f227effd700 1 -- 172.21.15.46:0/2594850950 learned_addr learned my addr 172.21.15.46:0/2594850950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.630+0000 7f227f7fe700 1 --2- 172.21.15.46:0/2594850950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 0x5615780aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.630+0000 7f227ffff700 1 --2- 172.21.15.46:0/2594850950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615780a1400 0x5615780ab9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.630+0000 7f227effd700 1 -- 172.21.15.46:0/2594850950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615780a1400 msgr2=0x5615780ab9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.630+0000 7f227effd700 1 --2- 172.21.15.46:0/2594850950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615780a1400 0x5615780ab9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.630+0000 7f227effd700 1 -- 172.21.15.46:0/2594850950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 msgr2=0x5615780aaf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.630+0000 7f227effd700 1 --2- 172.21.15.46:0/2594850950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 0x5615780aaf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.630+0000 7f227effd700 1 -- 172.21.15.46:0/2594850950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561577e3f490 con 0x561577f9ed00 2024-05-09T05:00:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.630+0000 7f227effd700 1 --2- 172.21.15.46:0/2594850950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f9ed00 0x5615780ab4a0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2268005950 tx=0x7f2268005730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d1cbd32ff211f70 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.630+0000 7f225effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.630+0000 7f225effd700 1 -- 172.21.15.46:0/2594850950 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f226801d070 con 0x561577f9ed00 2024-05-09T05:00:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.630+0000 7f229448fb80 1 -- 172.21.15.46:0/2594850950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f226800fdd0 con 0x561577f9ed00 2024-05-09T05:00:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.630+0000 7f225effd700 1 -- 172.21.15.46:0/2594850950 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f2268004960 con 0x561577f9ed00 2024-05-09T05:00:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.630+0000 7f225effd700 1 -- 172.21.15.46:0/2594850950 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f2268030050 con 0x561577f9ed00 2024-05-09T05:00:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.630+0000 7f229448fb80 10 client.17397 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.630+0000 7f229448fb80 1 -- 172.21.15.46:0/2594850950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5615780b8930 con 0x561577f9ed00 2024-05-09T05:00:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.631+0000 7f225effd700 1 -- 172.21.15.46:0/2594850950 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f226800fdd0 con 0x561577f9ed00 2024-05-09T05:00:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.631+0000 7f225effd700 1 -- 172.21.15.46:0/2594850950 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f22680104d0 con 0x561577f9ed00 2024-05-09T05:00:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.631+0000 7f229448fb80 10 client.17397 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.631+0000 7f229448fb80 10 client.17397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:41.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.631+0000 7f229448fb80 20 client.17397 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.631+0000 7f229448fb80 1 --2- 172.21.15.46:0/2594850950 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5615780baf10 0x5615780bd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.631+0000 7f229448fb80 4 client.17397 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:00:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.631+0000 7f229448fb80 1 -- 172.21.15.46:0/2594850950 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561577fd1a80 con 0x5615780baf10 2024-05-09T05:00:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.631+0000 7f227f7fe700 1 --2- 172.21.15.46:0/2594850950 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5615780baf10 0x5615780bd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.631+0000 7f225effd700 10 client.17397 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:41.635 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.631+0000 7f227f7fe700 1 --2- 172.21.15.46:0/2594850950 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5615780baf10 0x5615780bd330 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ab990f8eed5191b server_cookie=903c513e0f04956a in_seq=0 out_seq=0 2024-05-09T05:00:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.632+0000 7f225effd700 1 -- 172.21.15.46:0/2594850950 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561577fd1a80 con 0x5615780baf10 2024-05-09T05:00:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.632+0000 7f225effd700 10 client.17397 handle_command_reply: tid=0 2024-05-09T05:00:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.632+0000 7f229448fb80 1 client.17397 shutdown 2024-05-09T05:00:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.633+0000 7f229448fb80 1 -- 172.21.15.46:0/2594850950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f9ed00 msgr2=0x5615780ab4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.633+0000 7f229448fb80 1 --2- 172.21.15.46:0/2594850950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f9ed00 0x5615780ab4a0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f2268005950 tx=0x7f2268005730 comp rx=0 tx=0).stop 2024-05-09T05:00:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.633+0000 7f229448fb80 1 -- 172.21.15.46:0/2594850950 shutdown_connections 2024-05-09T05:00:41.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.633+0000 7f229448fb80 1 --2- 172.21.15.46:0/2594850950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615780a1400 0x5615780ab9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.633+0000 7f229448fb80 1 --2- 172.21.15.46:0/2594850950 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5615780baf10 0x5615780bd330 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.633+0000 7f229448fb80 1 --2- 172.21.15.46:0/2594850950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 0x5615780aaf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.633+0000 7f229448fb80 1 --2- 172.21.15.46:0/2594850950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f9ed00 0x5615780ab4a0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.633+0000 7f229448fb80 1 -- 172.21.15.46:0/2594850950 >> 172.21.15.46:0/2594850950 conn(0x561577f27f70 msgr2=0x561578095ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.633+0000 7f225effd700 0 client.17397 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.633+0000 7f229448fb80 1 -- 172.21.15.46:0/2594850950 shutdown_connections 2024-05-09T05:00:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.633+0000 7f229448fb80 1 -- 172.21.15.46:0/2594850950 wait complete. 2024-05-09T05:00:41.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.633+0000 7f229448fb80 20 client.17397 trim_cache size 0 max 16384 2024-05-09T05:00:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.654+0000 7f229448fb80 1 Processor -- start 2024-05-09T05:00:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.654+0000 7f229448fb80 1 -- start start 2024-05-09T05:00:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.654+0000 7f229448fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 0x561578096a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.654+0000 7f229448fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561577f9ed00 0x56157809c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.654+0000 7f229448fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615780a1400 0x5615780b7160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.654+0000 7f229448fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f22801b18a0 con 0x5615780a1400 2024-05-09T05:00:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.654+0000 7f229448fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f22801af130 con 0x561577f244a0 2024-05-09T05:00:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.654+0000 7f229448fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f22801acfa0 con 0x561577f9ed00 2024-05-09T05:00:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.654+0000 7f227ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615780a1400 0x5615780b7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.654+0000 7f227ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615780a1400 0x5615780b7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51514/0 (socket says 172.21.15.46:51514) 2024-05-09T05:00:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.654+0000 7f227ffff700 1 -- 172.21.15.46:0/1137180930 learned_addr learned my addr 172.21.15.46:0/1137180930 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.655+0000 7f227f7fe700 1 --2- 172.21.15.46:0/1137180930 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 0x561578096a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.655+0000 7f227effd700 1 --2- 172.21.15.46:0/1137180930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561577f9ed00 0x56157809c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.655+0000 7f227ffff700 1 -- 172.21.15.46:0/1137180930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561577f9ed00 msgr2=0x56157809c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.655+0000 7f227ffff700 1 --2- 172.21.15.46:0/1137180930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561577f9ed00 0x56157809c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.655+0000 7f227ffff700 1 -- 172.21.15.46:0/1137180930 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 msgr2=0x561578096a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.655+0000 7f227ffff700 1 --2- 172.21.15.46:0/1137180930 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 0x561578096a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.655+0000 7f227ffff700 1 -- 172.21.15.46:0/1137180930 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5615780a6c40 con 0x5615780a1400 2024-05-09T05:00:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.655+0000 7f227effd700 1 --2- 172.21.15.46:0/1137180930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561577f9ed00 0x56157809c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:41.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.655+0000 7f227ffff700 1 --2- 172.21.15.46:0/1137180930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615780a1400 0x5615780b7160 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f22780a96f0 tx=0x7f22780a9270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=619509c3f9158959 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.655+0000 7f22757fa700 1 -- 172.21.15.46:0/1137180930 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22780a9760 con 0x5615780a1400 2024-05-09T05:00:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.655+0000 7f22757fa700 1 -- 172.21.15.46:0/1137180930 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22780adb10 con 0x5615780a1400 2024-05-09T05:00:41.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.655+0000 7f22757fa700 1 -- 172.21.15.46:0/1137180930 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2278091160 con 0x5615780a1400 2024-05-09T05:00:41.660 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.656+0000 7f229448fb80 1 -- 172.21.15.46:0/1137180930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615780a1400 msgr2=0x5615780b7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.656+0000 7f229448fb80 1 --2- 172.21.15.46:0/1137180930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615780a1400 0x5615780b7160 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f22780a96f0 tx=0x7f22780a9270 comp rx=0 tx=0).stop 2024-05-09T05:00:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.656+0000 7f229448fb80 1 -- 172.21.15.46:0/1137180930 shutdown_connections 2024-05-09T05:00:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.656+0000 7f229448fb80 1 --2- 172.21.15.46:0/1137180930 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5615780a1400 0x5615780b7160 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.656+0000 7f229448fb80 1 --2- 172.21.15.46:0/1137180930 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 0x561578096a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.656+0000 7f229448fb80 1 --2- 172.21.15.46:0/1137180930 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x561577f9ed00 0x56157809c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.656+0000 7f229448fb80 1 -- 172.21.15.46:0/1137180930 >> 172.21.15.46:0/1137180930 conn(0x561577f27f70 msgr2=0x5615780a5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.656+0000 7f229448fb80 1 -- 172.21.15.46:0/1137180930 shutdown_connections 2024-05-09T05:00:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.656+0000 7f229448fb80 1 -- 172.21.15.46:0/1137180930 wait complete. 2024-05-09T05:00:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.657+0000 7f229448fb80 1 -- start start 2024-05-09T05:00:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.657+0000 7f229448fb80 1 Processor -- start 2024-05-09T05:00:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.657+0000 7f229448fb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 0x5615780c0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.657+0000 7f229448fb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f9ed00 0x5615780bd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.657+0000 7f229448fb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615780a1400 0x5615780bdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.661 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.657+0000 7f229448fb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f22801ad5f0 con 0x561577f9ed00 2024-05-09T05:00:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.657+0000 7f229448fb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f22801aea60 con 0x561577f244a0 2024-05-09T05:00:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.657+0000 7f229448fb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f22801adcc0 con 0x5615780a1400 2024-05-09T05:00:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.657+0000 7f227effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f9ed00 0x5615780bd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.657+0000 7f229448fb80 10 client.0 _ll_register_callbacks cb 0x561577e356c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.657+0000 7f227effd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f9ed00 0x5615780bd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:51522/0 (socket says 172.21.15.46:51522) 2024-05-09T05:00:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.657+0000 7f227effd700 1 -- 172.21.15.46:0/967751276 learned_addr learned my addr 172.21.15.46:0/967751276 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.657+0000 7f227f7fe700 1 --2- 172.21.15.46:0/967751276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 0x5615780c0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f227ffff700 1 --2- 172.21.15.46:0/967751276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615780a1400 0x5615780bdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f227effd700 1 -- 172.21.15.46:0/967751276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615780a1400 msgr2=0x5615780bdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f227effd700 1 --2- 172.21.15.46:0/967751276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615780a1400 0x5615780bdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f227effd700 1 -- 172.21.15.46:0/967751276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 msgr2=0x5615780c0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f227effd700 1 --2- 172.21.15.46:0/967751276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 0x5615780c0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f227effd700 1 -- 172.21.15.46:0/967751276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5615780be1c0 con 0x561577f9ed00 2024-05-09T05:00:41.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f227f7fe700 1 --2- 172.21.15.46:0/967751276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 0x5615780c0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:00:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f227ffff700 1 --2- 172.21.15.46:0/967751276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615780a1400 0x5615780bdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f227effd700 1 --2- 172.21.15.46:0/967751276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f9ed00 0x5615780bd6c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x561578098a10 tx=0x7f226800a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b35debdaec01cd server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f225effd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f225effd700 1 -- 172.21.15.46:0/967751276 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2268038020 con 0x561577f9ed00 2024-05-09T05:00:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f225effd700 1 -- 172.21.15.46:0/967751276 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f226800fbd0 con 0x561577f9ed00 2024-05-09T05:00:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f229448fb80 1 -- 172.21.15.46:0/967751276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f22780adb10 con 0x561577f9ed00 2024-05-09T05:00:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f225effd700 1 -- 172.21.15.46:0/967751276 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f226801d050 con 0x561577f9ed00 2024-05-09T05:00:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f229448fb80 10 client.17403 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.658+0000 7f229448fb80 1 -- 172.21.15.46:0/967751276 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5615781295d0 con 0x561577f9ed00 2024-05-09T05:00:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.659+0000 7f225effd700 1 -- 172.21.15.46:0/967751276 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f22780adb10 con 0x561577f9ed00 2024-05-09T05:00:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.659+0000 7f225effd700 1 -- 172.21.15.46:0/967751276 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f2268018820 con 0x561577f9ed00 2024-05-09T05:00:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.659+0000 7f229448fb80 10 client.17403 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.659+0000 7f229448fb80 10 client.17403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:41.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.659+0000 7f229448fb80 20 client.17403 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.659+0000 7f229448fb80 1 --2- 172.21.15.46:0/967751276 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x561578129c70 0x56157812c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.659+0000 7f229448fb80 4 client.17403 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:00:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.659+0000 7f229448fb80 1 -- 172.21.15.46:0/967751276 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561577fcbb20 con 0x561578129c70 2024-05-09T05:00:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.659+0000 7f227f7fe700 1 --2- 172.21.15.46:0/967751276 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x561578129c70 0x56157812c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.659+0000 7f225effd700 10 client.17403 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.659+0000 7f227f7fe700 1 --2- 172.21.15.46:0/967751276 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x561578129c70 0x56157812c090 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4775549bb2cb4f server_cookie=1c87987822381b40 in_seq=0 out_seq=0 2024-05-09T05:00:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.660+0000 7f225effd700 1 -- 172.21.15.46:0/967751276 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561577fcbb20 con 0x561578129c70 2024-05-09T05:00:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.660+0000 7f225effd700 10 client.17403 handle_command_reply: tid=0 2024-05-09T05:00:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.660+0000 7f229448fb80 1 client.17403 shutdown 2024-05-09T05:00:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.660+0000 7f229448fb80 1 -- 172.21.15.46:0/967751276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f9ed00 msgr2=0x5615780bd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.660+0000 7f229448fb80 1 --2- 172.21.15.46:0/967751276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f9ed00 0x5615780bd6c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x561578098a10 tx=0x7f226800a300 comp rx=0 tx=0).stop 2024-05-09T05:00:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.660+0000 7f229448fb80 1 -- 172.21.15.46:0/967751276 shutdown_connections 2024-05-09T05:00:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.660+0000 7f229448fb80 1 --2- 172.21.15.46:0/967751276 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5615780a1400 0x5615780bdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.660+0000 7f229448fb80 1 --2- 172.21.15.46:0/967751276 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x561578129c70 0x56157812c090 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.664 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.660+0000 7f229448fb80 1 --2- 172.21.15.46:0/967751276 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x561577f244a0 0x5615780c0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.660+0000 7f229448fb80 1 --2- 172.21.15.46:0/967751276 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x561577f9ed00 0x5615780bd6c0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.660+0000 7f229448fb80 1 -- 172.21.15.46:0/967751276 >> 172.21.15.46:0/967751276 conn(0x561577f27f70 msgr2=0x5615780aaf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.660+0000 7f225effd700 0 client.17403 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.660+0000 7f229448fb80 1 -- 172.21.15.46:0/967751276 shutdown_connections 2024-05-09T05:00:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.660+0000 7f229448fb80 1 -- 172.21.15.46:0/967751276 wait complete. 2024-05-09T05:00:41.665 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.660+0000 7f229448fb80 20 client.17403 trim_cache size 0 max 16384 2024-05-09T05:00:41.665 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T05:00:41.665 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T05:00:41.665 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T05:00:41.665 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T05:00:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.661+0000 7f2285a79700 1 -- 172.21.15.46:0/2543549619 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f226c0b29f0 msgr2=0x7f226c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.661+0000 7f2285a79700 1 --2- 172.21.15.46:0/2543549619 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f226c0b29f0 0x7f226c0b4f00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f227000d010 tx=0x7f227000b040 comp rx=0 tx=0).stop 2024-05-09T05:00:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.661+0000 7f2285a79700 1 -- 172.21.15.46:0/2543549619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2280108760 msgr2=0x7f228019c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.661+0000 7f2285a79700 1 --2- 172.21.15.46:0/2543549619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2280108760 0x7f228019c750 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f227800d980 tx=0x7f227800dc90 comp rx=0 tx=0).stop 2024-05-09T05:00:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.661+0000 7f2285a79700 1 -- 172.21.15.46:0/2543549619 shutdown_connections 2024-05-09T05:00:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.661+0000 7f2285a79700 1 --2- 172.21.15.46:0/2543549619 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f226c0b29f0 0x7f226c0b4f00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.661+0000 7f2285a79700 1 --2- 172.21.15.46:0/2543549619 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f2280108760 0x7f228019c750 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.666 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.661+0000 7f2285a79700 1 --2- 172.21.15.46:0/2543549619 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f22801034e0 0x7f228019bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.661+0000 7f2285a79700 1 --2- 172.21.15.46:0/2543549619 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f2280107e10 0x7f228019c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.661+0000 7f2285a79700 1 -- 172.21.15.46:0/2543549619 >> 172.21.15.46:0/2543549619 conn(0x7f22800fef30 msgr2=0x7f22801063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.661+0000 7f2285a79700 1 -- 172.21.15.46:0/2543549619 shutdown_connections 2024-05-09T05:00:41.667 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:41.661+0000 7f2285a79700 1 -- 172.21.15.46:0/2543549619 wait complete. 2024-05-09T05:00:42.026 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:41 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T05:00:42.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:41 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T05:00:42.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:41 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T05:00:42.441 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T05:00:42.441 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:00:42.697 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:43.072 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:42 smithi046 ceph-mon[30136]: pgmap v297: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-05-09T05:00:43.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:42 smithi072 ceph-mon[34088]: pgmap v297: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-05-09T05:00:43.208 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:42 smithi110 ceph-mon[34904]: pgmap v297: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 7 op/s 2024-05-09T05:00:44.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.405+0000 7ff37469f700 1 -- 172.21.15.46:0/2604813137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff36c103360 msgr2=0x7ff36c112fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:44.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.405+0000 7ff37469f700 1 --2- 172.21.15.46:0/2604813137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff36c103360 0x7ff36c112fe0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ff36400b0a0 tx=0x7ff36400b3b0 comp rx=0 tx=0).stop 2024-05-09T05:00:44.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.406+0000 7ff37469f700 1 -- 172.21.15.46:0/2604813137 shutdown_connections 2024-05-09T05:00:44.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.406+0000 7ff37469f700 1 --2- 172.21.15.46:0/2604813137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff36c103360 0x7ff36c112fe0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.406+0000 7ff37469f700 1 --2- 172.21.15.46:0/2604813137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff36c105210 0x7ff36c102790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.406+0000 7ff37469f700 1 --2- 172.21.15.46:0/2604813137 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff36c105620 0x7ff36c102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.406+0000 7ff37469f700 1 -- 172.21.15.46:0/2604813137 >> 172.21.15.46:0/2604813137 conn(0x7ff36c0fe540 msgr2=0x7ff36c100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.406+0000 7ff37469f700 1 -- 172.21.15.46:0/2604813137 shutdown_connections 2024-05-09T05:00:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.406+0000 7ff37469f700 1 -- 172.21.15.46:0/2604813137 wait complete. 2024-05-09T05:00:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.406+0000 7ff37469f700 1 Processor -- start 2024-05-09T05:00:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.406+0000 7ff37469f700 1 -- start start 2024-05-09T05:00:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff37469f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff36c103360 0x7ff36c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff37469f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff36c105210 0x7ff36c19d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff37469f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff36c105620 0x7ff36c1a1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff37469f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff36c116660 con 0x7ff36c105620 2024-05-09T05:00:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff37469f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff36c1164e0 con 0x7ff36c103360 2024-05-09T05:00:44.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff37469f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff36c1167e0 con 0x7ff36c105210 2024-05-09T05:00:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff372c3c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff36c105620 0x7ff36c1a1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff372c3c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff36c105620 0x7ff36c1a1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39810/0 (socket says 172.21.15.46:39810) 2024-05-09T05:00:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff372c3c700 1 -- 172.21.15.46:0/230013309 learned_addr learned my addr 172.21.15.46:0/230013309 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff37243b700 1 --2- 172.21.15.46:0/230013309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff36c103360 0x7ff36c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff371c3a700 1 --2- 172.21.15.46:0/230013309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff36c105210 0x7ff36c19d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff372c3c700 1 -- 172.21.15.46:0/230013309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff36c105210 msgr2=0x7ff36c19d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff372c3c700 1 --2- 172.21.15.46:0/230013309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff36c105210 0x7ff36c19d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff372c3c700 1 -- 172.21.15.46:0/230013309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff36c103360 msgr2=0x7ff36c19cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff372c3c700 1 --2- 172.21.15.46:0/230013309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff36c103360 0x7ff36c19cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff372c3c700 1 -- 172.21.15.46:0/230013309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff36c1a1e20 con 0x7ff36c105620 2024-05-09T05:00:44.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.407+0000 7ff372c3c700 1 --2- 172.21.15.46:0/230013309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff36c105620 0x7ff36c1a1780 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ff36400b830 tx=0x7ff364004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.408+0000 7ff35f7fe700 1 -- 172.21.15.46:0/230013309 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364021070 con 0x7ff36c105620 2024-05-09T05:00:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.408+0000 7ff37469f700 1 -- 172.21.15.46:0/230013309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff36c1a20b0 con 0x7ff36c105620 2024-05-09T05:00:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.408+0000 7ff35f7fe700 1 -- 172.21.15.46:0/230013309 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff364007e30 con 0x7ff36c105620 2024-05-09T05:00:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.408+0000 7ff35f7fe700 1 -- 172.21.15.46:0/230013309 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3640047d0 con 0x7ff36c105620 2024-05-09T05:00:44.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.408+0000 7ff37469f700 1 -- 172.21.15.46:0/230013309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff36c1a9a30 con 0x7ff36c105620 2024-05-09T05:00:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.409+0000 7ff37469f700 1 -- 172.21.15.46:0/230013309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff36c0512c0 con 0x7ff36c105620 2024-05-09T05:00:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.410+0000 7ff35f7fe700 1 -- 172.21.15.46:0/230013309 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff364007980 con 0x7ff36c105620 2024-05-09T05:00:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.411+0000 7ff35f7fe700 1 --2- 172.21.15.46:0/230013309 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff3580b29f0 0x7ff3580b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.411+0000 7ff37243b700 1 --2- 172.21.15.46:0/230013309 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff3580b29f0 0x7ff3580b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.411+0000 7ff35f7fe700 1 -- 172.21.15.46:0/230013309 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7ff3640e3730 con 0x7ff36c105620 2024-05-09T05:00:44.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.412+0000 7ff37243b700 1 --2- 172.21.15.46:0/230013309 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff3580b29f0 0x7ff3580b4f00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff36c06c040 tx=0x7ff36000a380 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:44.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.415+0000 7ff35f7fe700 1 -- 172.21.15.46:0/230013309 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff3640ab150 con 0x7ff36c105620 2024-05-09T05:00:44.681 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.681+0000 7ff37469f700 1 -- 172.21.15.46:0/230013309 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff36c02e5f0 con 0x7ff36c105620 2024-05-09T05:00:44.682 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.682+0000 7ff35f7fe700 1 -- 172.21.15.46:0/230013309 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7ff3640aaa50 con 0x7ff36c105620 2024-05-09T05:00:44.683 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T05:00:44.683 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T05:00:44.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.684+0000 7ff37469f700 1 -- 172.21.15.46:0/230013309 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff3580b29f0 msgr2=0x7ff3580b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:44.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.684+0000 7ff37469f700 1 --2- 172.21.15.46:0/230013309 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff3580b29f0 0x7ff3580b4f00 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7ff36c06c040 tx=0x7ff36000a380 comp rx=0 tx=0).stop 2024-05-09T05:00:44.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.684+0000 7ff37469f700 1 -- 172.21.15.46:0/230013309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff36c105620 msgr2=0x7ff36c1a1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:44.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.684+0000 7ff37469f700 1 --2- 172.21.15.46:0/230013309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff36c105620 0x7ff36c1a1780 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ff36400b830 tx=0x7ff364004ce0 comp rx=0 tx=0).stop 2024-05-09T05:00:44.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.685+0000 7ff37469f700 1 -- 172.21.15.46:0/230013309 shutdown_connections 2024-05-09T05:00:44.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.685+0000 7ff37469f700 1 --2- 172.21.15.46:0/230013309 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff3580b29f0 0x7ff3580b4f00 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:44.685 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.685+0000 7ff37469f700 1 --2- 172.21.15.46:0/230013309 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff36c105620 0x7ff36c1a1780 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:44.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.685+0000 7ff37469f700 1 --2- 172.21.15.46:0/230013309 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff36c103360 0x7ff36c19cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:44.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.685+0000 7ff37469f700 1 --2- 172.21.15.46:0/230013309 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff36c105210 0x7ff36c19d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:44.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.685+0000 7ff37469f700 1 -- 172.21.15.46:0/230013309 >> 172.21.15.46:0/230013309 conn(0x7ff36c0fe540 msgr2=0x7ff36c0ffac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:44.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.685+0000 7ff37469f700 1 -- 172.21.15.46:0/230013309 shutdown_connections 2024-05-09T05:00:44.686 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:44.685+0000 7ff37469f700 1 -- 172.21.15.46:0/230013309 wait complete. 2024-05-09T05:00:44.686 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T05:00:45.010 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:44 smithi046 ceph-mon[30136]: pgmap v298: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T05:00:45.010 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:44 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/230013309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:00:45.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:44 smithi072 ceph-mon[34088]: pgmap v298: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T05:00:45.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:44 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/230013309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:00:45.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:44 smithi110 ceph-mon[34904]: pgmap v298: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 6 op/s 2024-05-09T05:00:45.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:44 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/230013309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:00:45.459 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T05:00:45.712 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:46 smithi072 ceph-mon[34088]: pgmap v299: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T05:00:47.165 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:46 smithi046 ceph-mon[30136]: pgmap v299: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T05:00:47.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:46 smithi110 ceph-mon[34904]: pgmap v299: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T05:00:47.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.577+0000 7f9cf1a28700 1 -- 172.21.15.46:0/1410014674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cec109300 msgr2=0x7f9cec109710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:47.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.577+0000 7f9cf1a28700 1 --2- 172.21.15.46:0/1410014674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cec109300 0x7f9cec109710 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc009990 tx=0x7f9cdc009ca0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.577+0000 7f9cf1a28700 1 -- 172.21.15.46:0/1410014674 shutdown_connections 2024-05-09T05:00:47.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.577+0000 7f9cf1a28700 1 --2- 172.21.15.46:0/1410014674 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cec1030e0 0x7f9cec111170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.577+0000 7f9cf1a28700 1 --2- 172.21.15.46:0/1410014674 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cec109300 0x7f9cec109710 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.577 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.577+0000 7f9cf1a28700 1 --2- 172.21.15.46:0/1410014674 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cec102790 0x7f9cec102ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.577+0000 7f9cf1a28700 1 -- 172.21.15.46:0/1410014674 >> 172.21.15.46:0/1410014674 conn(0x7f9cec0fe4e0 msgr2=0x7f9cec100930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.577+0000 7f9cf1a28700 1 -- 172.21.15.46:0/1410014674 shutdown_connections 2024-05-09T05:00:47.578 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.577+0000 7f9cf1a28700 1 -- 172.21.15.46:0/1410014674 wait complete. 2024-05-09T05:00:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.577+0000 7f9cf1a28700 1 Processor -- start 2024-05-09T05:00:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.578+0000 7f9cf1a28700 1 -- start start 2024-05-09T05:00:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.578+0000 7f9cf1a28700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cec102790 0x7f9cec19afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.578+0000 7f9cf1a28700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cec1030e0 0x7f9cec19b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.578+0000 7f9cf1a28700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cec109300 0x7f9cec19f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.578+0000 7f9cf1a28700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec114590 con 0x7f9cec1030e0 2024-05-09T05:00:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.578+0000 7f9cf1a28700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec114410 con 0x7f9cec102790 2024-05-09T05:00:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.578+0000 7f9cf1a28700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec114710 con 0x7f9cec109300 2024-05-09T05:00:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.578+0000 7f9cea7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cec1030e0 0x7f9cec19b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.578+0000 7f9cea7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cec1030e0 0x7f9cec19b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39816/0 (socket says 172.21.15.46:39816) 2024-05-09T05:00:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.578+0000 7f9cea7fc700 1 -- 172.21.15.46:0/2726162051 learned_addr learned my addr 172.21.15.46:0/2726162051 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:47.583 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.578+0000 7f9ceb7fe700 1 --2- 172.21.15.46:0/2726162051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cec109300 0x7f9cec19f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.578+0000 7f9ceaffd700 1 --2- 172.21.15.46:0/2726162051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cec102790 0x7f9cec19afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.579+0000 7f9cea7fc700 1 -- 172.21.15.46:0/2726162051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cec109300 msgr2=0x7f9cec19f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.579+0000 7f9cea7fc700 1 --2- 172.21.15.46:0/2726162051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cec109300 0x7f9cec19f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.579+0000 7f9cea7fc700 1 -- 172.21.15.46:0/2726162051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cec102790 msgr2=0x7f9cec19afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.579+0000 7f9cea7fc700 1 --2- 172.21.15.46:0/2726162051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cec102790 0x7f9cec19afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.579+0000 7f9cea7fc700 1 -- 172.21.15.46:0/2726162051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cec19fe40 con 0x7f9cec1030e0 2024-05-09T05:00:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.579+0000 7f9cea7fc700 1 --2- 172.21.15.46:0/2726162051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cec1030e0 0x7f9cec19b560 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000d980 tx=0x7f9ce000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.579+0000 7f9cf0a26700 1 -- 172.21.15.46:0/2726162051 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0017070 con 0x7f9cec1030e0 2024-05-09T05:00:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.579+0000 7f9cf0a26700 1 -- 172.21.15.46:0/2726162051 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ce000fd40 con 0x7f9cec1030e0 2024-05-09T05:00:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.579+0000 7f9cf1a28700 1 -- 172.21.15.46:0/2726162051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cec1a0130 con 0x7f9cec1030e0 2024-05-09T05:00:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.579+0000 7f9cf0a26700 1 -- 172.21.15.46:0/2726162051 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce000e3b0 con 0x7f9cec1030e0 2024-05-09T05:00:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.579+0000 7f9cf1a28700 1 -- 172.21.15.46:0/2726162051 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cec1038a0 con 0x7f9cec1030e0 2024-05-09T05:00:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.582+0000 7f9cf0a26700 1 -- 172.21.15.46:0/2726162051 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9ce000e550 con 0x7f9cec1030e0 2024-05-09T05:00:47.584 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.582+0000 7f9cf0a26700 1 --2- 172.21.15.46:0/2726162051 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9cd40b2a70 0x7f9cd40b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.583+0000 7f9cf0a26700 1 -- 172.21.15.46:0/2726162051 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f9ce00e2090 con 0x7f9cec1030e0 2024-05-09T05:00:47.585 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.583+0000 7f9ceaffd700 1 --2- 172.21.15.46:0/2726162051 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9cd40b2a70 0x7f9cd40b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.583+0000 7f9ceaffd700 1 --2- 172.21.15.46:0/2726162051 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9cd40b2a70 0x7f9cd40b4f80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0059d0 tx=0x7f9cdc005960 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:47.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.583+0000 7f9d0043eb80 1 Processor -- start 2024-05-09T05:00:47.589 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.583+0000 7f9d0043eb80 1 -- start start 2024-05-09T05:00:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9d0043eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bada96160 0x564badb2f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9d0043eb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 0x564badb2fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9d0043eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564badb296a0 0x564badb29ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9d0043eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564bada4bff0 con 0x564badb296a0 2024-05-09T05:00:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9d0043eb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x564bad9ea7f0 con 0x564bad9ac4a0 2024-05-09T05:00:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9d0043eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564bada47fb0 con 0x564bada96160 2024-05-09T05:00:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9ceb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564badb296a0 0x564badb29ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9ceb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564badb296a0 0x564badb29ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39828/0 (socket says 172.21.15.46:39828) 2024-05-09T05:00:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9ceb7fe700 1 -- 172.21.15.46:0/4038815137 learned_addr learned my addr 172.21.15.46:0/4038815137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9cea7fc700 1 --2- 172.21.15.46:0/4038815137 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 0x564badb2fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9ceaffd700 1 --2- 172.21.15.46:0/4038815137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bada96160 0x564badb2f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9ceb7fe700 1 -- 172.21.15.46:0/4038815137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bada96160 msgr2=0x564badb2f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9ceb7fe700 1 --2- 172.21.15.46:0/4038815137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bada96160 0x564badb2f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.590 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9ceb7fe700 1 -- 172.21.15.46:0/4038815137 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 msgr2=0x564badb2fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9ceb7fe700 1 --2- 172.21.15.46:0/4038815137 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 0x564badb2fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9ceb7fe700 1 -- 172.21.15.46:0/4038815137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564bad9e2370 con 0x564badb296a0 2024-05-09T05:00:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9cea7fc700 1 --2- 172.21.15.46:0/4038815137 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 0x564badb2fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9ceb7fe700 1 --2- 172.21.15.46:0/4038815137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564badb296a0 0x564badb29ad0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4003f30 tx=0x7f9ce4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2615693ff7106e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.584+0000 7f9cd1ffb700 1 -- 172.21.15.46:0/4038815137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4004740 con 0x564badb296a0 2024-05-09T05:00:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.585+0000 7f9cd1ffb700 1 -- 172.21.15.46:0/4038815137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ce401a070 con 0x564badb296a0 2024-05-09T05:00:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.585+0000 7f9cd1ffb700 1 -- 172.21.15.46:0/4038815137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce40156e0 con 0x564badb296a0 2024-05-09T05:00:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.585+0000 7f9d0043eb80 1 -- 172.21.15.46:0/4038815137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564badb296a0 msgr2=0x564badb29ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.585+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/4038815137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564badb296a0 0x564badb29ad0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4003f30 tx=0x7f9ce4009c20 comp rx=0 tx=0).stop 2024-05-09T05:00:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.585+0000 7f9d0043eb80 1 -- 172.21.15.46:0/4038815137 shutdown_connections 2024-05-09T05:00:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.585+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/4038815137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564badb296a0 0x564badb29ad0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.585+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/4038815137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bada96160 0x564badb2f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.585+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/4038815137 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 0x564badb2fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.585+0000 7f9d0043eb80 1 -- 172.21.15.46:0/4038815137 >> 172.21.15.46:0/4038815137 conn(0x564bada0c6a0 msgr2=0x564badb24400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:47.591 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.586+0000 7f9d0043eb80 1 -- 172.21.15.46:0/4038815137 shutdown_connections 2024-05-09T05:00:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.586+0000 7f9d0043eb80 1 -- 172.21.15.46:0/4038815137 wait complete. 2024-05-09T05:00:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.586+0000 7f9d0043eb80 1 -- start start 2024-05-09T05:00:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.586+0000 7f9d0043eb80 1 Processor -- start 2024-05-09T05:00:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.586+0000 7f9d0043eb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 0x564badb332f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.586+0000 7f9d0043eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bada96160 0x564badb33830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.586+0000 7f9d0043eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564badb296a0 0x564badb33d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.586+0000 7f9d0043eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564bad8c7ba0 con 0x564badb296a0 2024-05-09T05:00:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.586+0000 7f9d0043eb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x564bada06150 con 0x564bad9ac4a0 2024-05-09T05:00:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9d0043eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564badb361d0 con 0x564bada96160 2024-05-09T05:00:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9d0043eb80 10 client.0 _ll_register_callbacks cb 0x564bad8bd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9ceb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564badb296a0 0x564badb33d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9ceb7fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564badb296a0 0x564badb33d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39844/0 (socket says 172.21.15.46:39844) 2024-05-09T05:00:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9ceb7fe700 1 -- 172.21.15.46:0/1218490212 learned_addr learned my addr 172.21.15.46:0/1218490212 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:47.592 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9ceaffd700 1 --2- 172.21.15.46:0/1218490212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 0x564badb332f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9cea7fc700 1 --2- 172.21.15.46:0/1218490212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bada96160 0x564badb33830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9ceb7fe700 1 -- 172.21.15.46:0/1218490212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bada96160 msgr2=0x564badb33830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9ceb7fe700 1 --2- 172.21.15.46:0/1218490212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bada96160 0x564badb33830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9ceb7fe700 1 -- 172.21.15.46:0/1218490212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 msgr2=0x564badb332f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9ceb7fe700 1 --2- 172.21.15.46:0/1218490212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 0x564badb332f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9ceb7fe700 1 -- 172.21.15.46:0/1218490212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564bad8c7490 con 0x564badb296a0 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9ceb7fe700 1 --2- 172.21.15.46:0/1218490212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564badb296a0 0x564badb33d70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4007e20 tx=0x7f9ce40077b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e013485023978f1b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9ccaffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9ccaffd700 1 -- 172.21.15.46:0/1218490212 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4015680 con 0x564badb296a0 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9ccaffd700 1 -- 172.21.15.46:0/1218490212 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f9ce4015820 con 0x564badb296a0 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.587+0000 7f9d0043eb80 1 -- 172.21.15.46:0/1218490212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ce401a070 con 0x564badb296a0 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.588+0000 7f9ccaffd700 1 -- 172.21.15.46:0/1218490212 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9ce4015a20 con 0x564badb296a0 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.588+0000 7f9d0043eb80 10 client.17421 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.588+0000 7f9d0043eb80 1 -- 172.21.15.46:0/1218490212 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564badb40c30 con 0x564badb296a0 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.588+0000 7f9ccaffd700 1 -- 172.21.15.46:0/1218490212 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ce401a070 con 0x564badb296a0 2024-05-09T05:00:47.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.588+0000 7f9ccaffd700 1 -- 172.21.15.46:0/1218490212 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9ce4026b70 con 0x564badb296a0 2024-05-09T05:00:47.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.588+0000 7f9d0043eb80 10 client.17421 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:47.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.588+0000 7f9d0043eb80 10 client.17421 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T05:00:47.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.588+0000 7f9d0043eb80 20 client.17421 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:47.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.588+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/1218490212 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564badb43210 0x564badb45630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.588+0000 7f9d0043eb80 4 client.17421 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:00:47.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.588+0000 7f9d0043eb80 1 -- 172.21.15.46:0/1218490212 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564bada599d0 con 0x564badb43210 2024-05-09T05:00:47.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.588+0000 7f9ceaffd700 1 --2- 172.21.15.46:0/1218490212 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564badb43210 0x564badb45630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.589+0000 7f9ccaffd700 10 client.17421 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T05:00:47.594 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.589+0000 7f9ceaffd700 1 --2- 172.21.15.46:0/1218490212 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564badb43210 0x564badb45630 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bc52add9bedf9d61 server_cookie=acdca858e1f0f33b in_seq=0 out_seq=0 2024-05-09T05:00:47.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.590+0000 7f9ccaffd700 1 -- 172.21.15.46:0/1218490212 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564bada599d0 con 0x564badb43210 2024-05-09T05:00:47.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.590+0000 7f9ccaffd700 10 client.17421 handle_command_reply: tid=0 2024-05-09T05:00:47.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.590+0000 7f9d0043eb80 1 client.17421 shutdown 2024-05-09T05:00:47.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.590+0000 7f9d0043eb80 1 -- 172.21.15.46:0/1218490212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564badb296a0 msgr2=0x564badb33d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:47.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.590+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/1218490212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564badb296a0 0x564badb33d70 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f9ce4007e20 tx=0x7f9ce40077b0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.595 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.590+0000 7f9d0043eb80 1 -- 172.21.15.46:0/1218490212 shutdown_connections 2024-05-09T05:00:47.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.590+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/1218490212 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564badb296a0 0x564badb33d70 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.590+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/1218490212 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 0x564badb332f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.590+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/1218490212 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564badb43210 0x564badb45630 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.590+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/1218490212 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bada96160 0x564badb33830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.590+0000 7f9d0043eb80 1 -- 172.21.15.46:0/1218490212 >> 172.21.15.46:0/1218490212 conn(0x564bada0c6a0 msgr2=0x564badb241c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:47.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.590+0000 7f9ccaffd700 0 client.17421 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T05:00:47.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.590+0000 7f9d0043eb80 1 -- 172.21.15.46:0/1218490212 shutdown_connections 2024-05-09T05:00:47.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.590+0000 7f9d0043eb80 1 -- 172.21.15.46:0/1218490212 wait complete. 2024-05-09T05:00:47.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.590+0000 7f9d0043eb80 20 client.17421 trim_cache size 0 max 16384 2024-05-09T05:00:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.605+0000 7f9d0043eb80 1 Processor -- start 2024-05-09T05:00:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.605+0000 7f9d0043eb80 1 -- start start 2024-05-09T05:00:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9d0043eb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 0x564badb2e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9d0043eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bada96160 0x564badb2d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9d0043eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564badb296a0 0x564badb32bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9d0043eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec1af7b0 con 0x564bada96160 2024-05-09T05:00:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9d0043eb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec1ad040 con 0x564bad9ac4a0 2024-05-09T05:00:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9d0043eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec1aaed0 con 0x564badb296a0 2024-05-09T05:00:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9cea7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bada96160 0x564badb2d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.607 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9cea7fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bada96160 0x564badb2d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39850/0 (socket says 172.21.15.46:39850) 2024-05-09T05:00:47.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9cea7fc700 1 -- 172.21.15.46:0/1543060848 learned_addr learned my addr 172.21.15.46:0/1543060848 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:47.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9ceaffd700 1 --2- 172.21.15.46:0/1543060848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 0x564badb2e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9cea7fc700 1 -- 172.21.15.46:0/1543060848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564badb296a0 msgr2=0x564badb32bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:47.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9ceb7fe700 1 --2- 172.21.15.46:0/1543060848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564badb296a0 0x564badb32bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9cea7fc700 1 --2- 172.21.15.46:0/1543060848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564badb296a0 0x564badb32bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9cea7fc700 1 -- 172.21.15.46:0/1543060848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 msgr2=0x564badb2e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:47.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9cea7fc700 1 --2- 172.21.15.46:0/1543060848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 0x564badb2e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9cea7fc700 1 -- 172.21.15.46:0/1543060848 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564badb31860 con 0x564bada96160 2024-05-09T05:00:47.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9ceb7fe700 1 --2- 172.21.15.46:0/1543060848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564badb296a0 0x564badb32bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:00:47.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9ceaffd700 1 --2- 172.21.15.46:0/1543060848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 0x564badb2e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:47.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.606+0000 7f9cea7fc700 1 --2- 172.21.15.46:0/1543060848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bada96160 0x564badb2d860 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000be80 tx=0x7f9ce00a8bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f10322cc45df6b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:47.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.607+0000 7f9cd17fa700 1 -- 172.21.15.46:0/1543060848 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce0020040 con 0x564bada96160 2024-05-09T05:00:47.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.607+0000 7f9cd17fa700 1 -- 172.21.15.46:0/1543060848 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ce00ad6d0 con 0x564bada96160 2024-05-09T05:00:47.608 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.607+0000 7f9cd17fa700 1 -- 172.21.15.46:0/1543060848 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ce00a8890 con 0x564bada96160 2024-05-09T05:00:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.607+0000 7f9d0043eb80 1 -- 172.21.15.46:0/1543060848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bada96160 msgr2=0x564badb2d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.607+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/1543060848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bada96160 0x564badb2d860 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000be80 tx=0x7f9ce00a8bc0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.608+0000 7f9d0043eb80 1 -- 172.21.15.46:0/1543060848 shutdown_connections 2024-05-09T05:00:47.612 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.608+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/1543060848 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564badb296a0 0x564badb32bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.608+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/1543060848 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564bad9ac4a0 0x564badb2e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.608+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/1543060848 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bada96160 0x564badb2d860 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.608+0000 7f9d0043eb80 1 -- 172.21.15.46:0/1543060848 >> 172.21.15.46:0/1543060848 conn(0x564bada0c6a0 msgr2=0x564badb2e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.608+0000 7f9d0043eb80 1 -- 172.21.15.46:0/1543060848 shutdown_connections 2024-05-09T05:00:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.608+0000 7f9d0043eb80 1 -- 172.21.15.46:0/1543060848 wait complete. 2024-05-09T05:00:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.608+0000 7f9d0043eb80 1 -- start start 2024-05-09T05:00:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.608+0000 7f9d0043eb80 1 Processor -- start 2024-05-09T05:00:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.608+0000 7f9d0043eb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bad9ac4a0 0x564badb43210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.608+0000 7f9d0043eb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bada96160 0x564badb43750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.608+0000 7f9d0043eb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564badb296a0 0x564badb43c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.608+0000 7f9d0043eb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec1ab520 con 0x564bad9ac4a0 2024-05-09T05:00:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9d0043eb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec1ac970 con 0x564badb296a0 2024-05-09T05:00:47.613 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9d0043eb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f9cec1abbd0 con 0x564bada96160 2024-05-09T05:00:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9d0043eb80 10 client.0 _ll_register_callbacks cb 0x564bad8bd6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9ceaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bad9ac4a0 0x564badb43210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9ceaffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bad9ac4a0 0x564badb43210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39856/0 (socket says 172.21.15.46:39856) 2024-05-09T05:00:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9ceaffd700 1 -- 172.21.15.46:0/4145382042 learned_addr learned my addr 172.21.15.46:0/4145382042 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9cea7fc700 1 --2- 172.21.15.46:0/4145382042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bada96160 0x564badb43750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9ceb7fe700 1 --2- 172.21.15.46:0/4145382042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564badb296a0 0x564badb43c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9ceaffd700 1 -- 172.21.15.46:0/4145382042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bada96160 msgr2=0x564badb43750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9ceaffd700 1 --2- 172.21.15.46:0/4145382042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bada96160 0x564badb43750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9ceaffd700 1 -- 172.21.15.46:0/4145382042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564badb296a0 msgr2=0x564badb43c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9ceaffd700 1 --2- 172.21.15.46:0/4145382042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564badb296a0 0x564badb43c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9ceaffd700 1 -- 172.21.15.46:0/4145382042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564badbab810 con 0x564bad9ac4a0 2024-05-09T05:00:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9ceb7fe700 1 --2- 172.21.15.46:0/4145382042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564badb296a0 0x564badb43c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:47.614 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9ceaffd700 1 --2- 172.21.15.46:0/4145382042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bad9ac4a0 0x564badb43210 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9cec06be70 tx=0x7f9cdc01e050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d170d37946f94cf server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9ccaffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9ccaffd700 1 -- 172.21.15.46:0/4145382042 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc01cd80 con 0x564bad9ac4a0 2024-05-09T05:00:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9ccaffd700 1 -- 172.21.15.46:0/4145382042 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f9cdc005050 con 0x564bad9ac4a0 2024-05-09T05:00:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.609+0000 7f9d0043eb80 1 -- 172.21.15.46:0/4145382042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9ce00ad6d0 con 0x564bad9ac4a0 2024-05-09T05:00:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.610+0000 7f9ccaffd700 1 -- 172.21.15.46:0/4145382042 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9cdc045050 con 0x564bad9ac4a0 2024-05-09T05:00:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.610+0000 7f9d0043eb80 10 client.17427 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.610+0000 7f9d0043eb80 1 -- 172.21.15.46:0/4145382042 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564badbaca40 con 0x564bad9ac4a0 2024-05-09T05:00:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.610+0000 7f9ccaffd700 1 -- 172.21.15.46:0/4145382042 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ce00ad6d0 con 0x564bad9ac4a0 2024-05-09T05:00:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.610+0000 7f9ccaffd700 1 -- 172.21.15.46:0/4145382042 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f9cdc026070 con 0x564bad9ac4a0 2024-05-09T05:00:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.610+0000 7f9d0043eb80 10 client.17427 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.610+0000 7f9d0043eb80 10 client.17427 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T05:00:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.610+0000 7f9d0043eb80 20 client.17427 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.610+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/4145382042 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564badbad180 0x564badbaf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.610+0000 7f9d0043eb80 4 client.17427 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:00:47.615 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.610+0000 7f9d0043eb80 1 -- 172.21.15.46:0/4145382042 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564bada53a70 con 0x564badbad180 2024-05-09T05:00:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.610+0000 7f9cea7fc700 1 --2- 172.21.15.46:0/4145382042 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564badbad180 0x564badbaf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9ccaffd700 10 client.17427 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T05:00:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9cea7fc700 1 --2- 172.21.15.46:0/4145382042 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564badbad180 0x564badbaf5a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=403381baea4737dc server_cookie=8541bdfda93bf802 in_seq=0 out_seq=0 2024-05-09T05:00:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9ccaffd700 1 -- 172.21.15.46:0/4145382042 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564bada53a70 con 0x564badbad180 2024-05-09T05:00:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9ccaffd700 10 client.17427 handle_command_reply: tid=0 2024-05-09T05:00:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9d0043eb80 1 client.17427 shutdown 2024-05-09T05:00:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9d0043eb80 1 -- 172.21.15.46:0/4145382042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bad9ac4a0 msgr2=0x564badb43210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/4145382042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bad9ac4a0 0x564badb43210 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f9cec06be70 tx=0x7f9cdc01e050 comp rx=0 tx=0).stop 2024-05-09T05:00:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9d0043eb80 1 -- 172.21.15.46:0/4145382042 shutdown_connections 2024-05-09T05:00:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/4145382042 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564badb296a0 0x564badb43c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/4145382042 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564bad9ac4a0 0x564badb43210 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/4145382042 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564badbad180 0x564badbaf5a0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9d0043eb80 1 --2- 172.21.15.46:0/4145382042 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564bada96160 0x564badb43750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.616 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9d0043eb80 1 -- 172.21.15.46:0/4145382042 >> 172.21.15.46:0/4145382042 conn(0x564bada0c6a0 msgr2=0x564badb28690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:47.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9ccaffd700 0 client.17427 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T05:00:47.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9d0043eb80 1 -- 172.21.15.46:0/4145382042 shutdown_connections 2024-05-09T05:00:47.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.611+0000 7f9d0043eb80 1 -- 172.21.15.46:0/4145382042 wait complete. 2024-05-09T05:00:47.617 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.612+0000 7f9d0043eb80 20 client.17427 trim_cache size 0 max 16384 2024-05-09T05:00:47.617 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T05:00:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.612+0000 7f9cf1a28700 1 -- 172.21.15.46:0/2726162051 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9cd40b2a70 msgr2=0x7f9cd40b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.612+0000 7f9cf1a28700 1 --2- 172.21.15.46:0/2726162051 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9cd40b2a70 0x7f9cd40b4f80 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0059d0 tx=0x7f9cdc005960 comp rx=0 tx=0).stop 2024-05-09T05:00:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.612+0000 7f9cf1a28700 1 -- 172.21.15.46:0/2726162051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cec1030e0 msgr2=0x7f9cec19b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.612+0000 7f9cf1a28700 1 --2- 172.21.15.46:0/2726162051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cec1030e0 0x7f9cec19b560 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f9ce000d980 tx=0x7f9ce000dc90 comp rx=0 tx=0).stop 2024-05-09T05:00:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.612+0000 7f9cf1a28700 1 -- 172.21.15.46:0/2726162051 shutdown_connections 2024-05-09T05:00:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.612+0000 7f9cf1a28700 1 --2- 172.21.15.46:0/2726162051 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f9cd40b2a70 0x7f9cd40b4f80 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.612+0000 7f9cf1a28700 1 --2- 172.21.15.46:0/2726162051 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f9cec109300 0x7f9cec19f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.612+0000 7f9cf1a28700 1 --2- 172.21.15.46:0/2726162051 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f9cec102790 0x7f9cec19afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.612+0000 7f9cf1a28700 1 --2- 172.21.15.46:0/2726162051 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f9cec1030e0 0x7f9cec19b560 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.612+0000 7f9cf1a28700 1 -- 172.21.15.46:0/2726162051 >> 172.21.15.46:0/2726162051 conn(0x7f9cec0fe4e0 msgr2=0x7f9cec1071d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:47.618 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.612+0000 7f9cf1a28700 1 -- 172.21.15.46:0/2726162051 shutdown_connections 2024-05-09T05:00:47.619 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:47.612+0000 7f9cf1a28700 1 -- 172.21.15.46:0/2726162051 wait complete. 2024-05-09T05:00:48.424 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T05:00:48.677 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:49.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:48 smithi046 ceph-mon[30136]: pgmap v300: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T05:00:49.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:48 smithi072 ceph-mon[34088]: pgmap v300: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T05:00:49.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:48 smithi110 ceph-mon[34904]: pgmap v300: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 8 op/s 2024-05-09T05:00:50.506 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.506+0000 7f04767ee700 1 -- 172.21.15.46:0/125591119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0470109870 msgr2=0x7f04701111d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:50.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.506+0000 7f04767ee700 1 --2- 172.21.15.46:0/125591119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0470109870 0x7f04701111d0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f046800b0d0 tx=0x7f046800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.506+0000 7f04767ee700 1 -- 172.21.15.46:0/125591119 shutdown_connections 2024-05-09T05:00:50.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.506+0000 7f04767ee700 1 --2- 172.21.15.46:0/125591119 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f0470109870 0x7f04701111d0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.506+0000 7f04767ee700 1 --2- 172.21.15.46:0/125591119 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f04701045f0 0x7f0470104a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.506+0000 7f04767ee700 1 --2- 172.21.15.46:0/125591119 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0470108f20 0x7f0470109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.506+0000 7f04767ee700 1 -- 172.21.15.46:0/125591119 >> 172.21.15.46:0/125591119 conn(0x7f0470100040 msgr2=0x7f0470102460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:50.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.506+0000 7f04767ee700 1 -- 172.21.15.46:0/125591119 shutdown_connections 2024-05-09T05:00:50.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.506+0000 7f04767ee700 1 -- 172.21.15.46:0/125591119 wait complete. 2024-05-09T05:00:50.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.506+0000 7f04767ee700 1 Processor -- start 2024-05-09T05:00:50.507 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f04767ee700 1 -- start start 2024-05-09T05:00:50.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f04767ee700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04701045f0 0x7f047019ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f04767ee700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0470108f20 0x7f047019b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f04767ee700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0470109870 0x7f047019f690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f04767ee700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f0470114510 con 0x7f04701045f0 2024-05-09T05:00:50.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f04767ee700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f0470114390 con 0x7f0470109870 2024-05-09T05:00:50.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f04767ee700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f0470114690 con 0x7f0470108f20 2024-05-09T05:00:50.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f046ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04701045f0 0x7f047019ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f046ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04701045f0 0x7f047019ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39880/0 (socket says 172.21.15.46:39880) 2024-05-09T05:00:50.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f046ffff700 1 -- 172.21.15.46:0/2545757312 learned_addr learned my addr 172.21.15.46:0/2545757312 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:50.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f0474d8b700 1 --2- 172.21.15.46:0/2545757312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0470109870 0x7f047019f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f0467fff700 1 --2- 172.21.15.46:0/2545757312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0470108f20 0x7f047019b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f046ffff700 1 -- 172.21.15.46:0/2545757312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0470108f20 msgr2=0x7f047019b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:50.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f046ffff700 1 --2- 172.21.15.46:0/2545757312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0470108f20 0x7f047019b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f046ffff700 1 -- 172.21.15.46:0/2545757312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0470109870 msgr2=0x7f047019f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:50.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f046ffff700 1 --2- 172.21.15.46:0/2545757312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0470109870 0x7f047019f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.507+0000 7f046ffff700 1 -- 172.21.15.46:0/2545757312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f047019fd30 con 0x7f04701045f0 2024-05-09T05:00:50.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.508+0000 7f046ffff700 1 --2- 172.21.15.46:0/2545757312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04701045f0 0x7f047019ae60 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f046000d950 tx=0x7f046000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:50.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.508+0000 7f046dffb700 1 -- 172.21.15.46:0/2545757312 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0460016070 con 0x7f04701045f0 2024-05-09T05:00:50.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.508+0000 7f046dffb700 1 -- 172.21.15.46:0/2545757312 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0460009df0 con 0x7f04701045f0 2024-05-09T05:00:50.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.508+0000 7f04767ee700 1 -- 172.21.15.46:0/2545757312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04701a0020 con 0x7f04701045f0 2024-05-09T05:00:50.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.508+0000 7f046dffb700 1 -- 172.21.15.46:0/2545757312 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f046000f3c0 con 0x7f04701045f0 2024-05-09T05:00:50.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.508+0000 7f04767ee700 1 -- 172.21.15.46:0/2545757312 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04701a7940 con 0x7f04701045f0 2024-05-09T05:00:50.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.510+0000 7f046dffb700 1 -- 172.21.15.46:0/2545757312 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0460019070 con 0x7f04701045f0 2024-05-09T05:00:50.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.512+0000 7f046dffb700 1 --2- 172.21.15.46:0/2545757312 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f045c0b2a70 0x7f045c0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.512+0000 7f0467fff700 1 --2- 172.21.15.46:0/2545757312 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f045c0b2a70 0x7f045c0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.513 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.512+0000 7f046dffb700 1 -- 172.21.15.46:0/2545757312 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f04600e2210 con 0x7f04701045f0 2024-05-09T05:00:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.512+0000 7f0485204b80 1 Processor -- start 2024-05-09T05:00:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.512+0000 7f0485204b80 1 -- start start 2024-05-09T05:00:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.512+0000 7f0467fff700 1 --2- 172.21.15.46:0/2545757312 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f045c0b2a70 0x7f045c0b4f80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0458006fd0 tx=0x7f0458008040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.512+0000 7f0485204b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49f01d00 0x55ad4a00a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.512+0000 7f0485204b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad49e874a0 0x55ad4a00a780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.512+0000 7f0485204b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad4a004400 0x55ad4a004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.512+0000 7f0485204b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ad49f270a0 con 0x55ad49e874a0 2024-05-09T05:00:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.512+0000 7f0485204b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ad49ecc9a0 con 0x55ad49f01d00 2024-05-09T05:00:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.512+0000 7f0485204b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ad49f23060 con 0x55ad4a004400 2024-05-09T05:00:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.513+0000 7f0467fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad49e874a0 0x55ad4a00a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.514 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.513+0000 7f0467fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad49e874a0 0x55ad4a00a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39896/0 (socket says 172.21.15.46:39896) 2024-05-09T05:00:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.513+0000 7f0467fff700 1 -- 172.21.15.46:0/746457927 learned_addr learned my addr 172.21.15.46:0/746457927 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.513+0000 7f046ffff700 1 --2- 172.21.15.46:0/746457927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49f01d00 0x55ad4a00a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.513+0000 7f0474d8b700 1 --2- 172.21.15.46:0/746457927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad4a004400 0x55ad4a004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.513+0000 7f0467fff700 1 -- 172.21.15.46:0/746457927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad4a004400 msgr2=0x55ad4a004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.513+0000 7f0467fff700 1 --2- 172.21.15.46:0/746457927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad4a004400 0x55ad4a004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.513+0000 7f0467fff700 1 -- 172.21.15.46:0/746457927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49f01d00 msgr2=0x55ad4a00a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.513+0000 7f0467fff700 1 --2- 172.21.15.46:0/746457927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49f01d00 0x55ad4a00a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.513+0000 7f0467fff700 1 -- 172.21.15.46:0/746457927 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad49ebd500 con 0x55ad49e874a0 2024-05-09T05:00:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.513+0000 7f0467fff700 1 --2- 172.21.15.46:0/746457927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad49e874a0 0x55ad4a00a780 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f045800d6a0 tx=0x7f0458010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdde63be2f6fc56 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.513+0000 7f04667fc700 1 -- 172.21.15.46:0/746457927 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0458024070 con 0x55ad49e874a0 2024-05-09T05:00:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.513+0000 7f04667fc700 1 -- 172.21.15.46:0/746457927 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f045800de40 con 0x55ad49e874a0 2024-05-09T05:00:50.515 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.513+0000 7f04667fc700 1 -- 172.21.15.46:0/746457927 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045801fa80 con 0x55ad49e874a0 2024-05-09T05:00:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.514+0000 7f0485204b80 1 -- 172.21.15.46:0/746457927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad49e874a0 msgr2=0x55ad4a00a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.514+0000 7f0485204b80 1 --2- 172.21.15.46:0/746457927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad49e874a0 0x55ad4a00a780 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f045800d6a0 tx=0x7f0458010000 comp rx=0 tx=0).stop 2024-05-09T05:00:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.514+0000 7f0485204b80 1 -- 172.21.15.46:0/746457927 shutdown_connections 2024-05-09T05:00:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.514+0000 7f0485204b80 1 --2- 172.21.15.46:0/746457927 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad4a004400 0x55ad4a004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.514+0000 7f0485204b80 1 --2- 172.21.15.46:0/746457927 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49f01d00 0x55ad4a00a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.514+0000 7f0485204b80 1 --2- 172.21.15.46:0/746457927 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad49e874a0 0x55ad4a00a780 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.514+0000 7f0485204b80 1 -- 172.21.15.46:0/746457927 >> 172.21.15.46:0/746457927 conn(0x55ad49e8af70 msgr2=0x55ad49fff220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.514+0000 7f0485204b80 1 -- 172.21.15.46:0/746457927 shutdown_connections 2024-05-09T05:00:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.514+0000 7f0485204b80 1 -- 172.21.15.46:0/746457927 wait complete. 2024-05-09T05:00:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.515+0000 7f0485204b80 1 -- start start 2024-05-09T05:00:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.515+0000 7f0485204b80 1 Processor -- start 2024-05-09T05:00:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.515+0000 7f0485204b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49e874a0 0x55ad4a00dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.515+0000 7f0485204b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad49f01d00 0x55ad4a00e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.520 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.515+0000 7f0485204b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad4a004400 0x55ad4a00ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.515+0000 7f0485204b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55ad4a005160 con 0x55ad4a004400 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.515+0000 7f0485204b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55ad49ec1b20 con 0x55ad49e874a0 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.515+0000 7f0485204b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55ad4a011020 con 0x55ad49f01d00 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.515+0000 7f0485204b80 10 client.0 _ll_register_callbacks cb 0x55ad49d986c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.515+0000 7f0474d8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad4a004400 0x55ad4a00ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.515+0000 7f0474d8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad4a004400 0x55ad4a00ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39912/0 (socket says 172.21.15.46:39912) 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.516+0000 7f0474d8b700 1 -- 172.21.15.46:0/2510527854 learned_addr learned my addr 172.21.15.46:0/2510527854 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.516+0000 7f0467fff700 1 --2- 172.21.15.46:0/2510527854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad49f01d00 0x55ad4a00e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.516+0000 7f046ffff700 1 --2- 172.21.15.46:0/2510527854 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49e874a0 0x55ad4a00dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.516+0000 7f0474d8b700 1 -- 172.21.15.46:0/2510527854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad49f01d00 msgr2=0x55ad4a00e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.516+0000 7f0474d8b700 1 --2- 172.21.15.46:0/2510527854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad49f01d00 0x55ad4a00e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.516+0000 7f0474d8b700 1 -- 172.21.15.46:0/2510527854 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49e874a0 msgr2=0x55ad4a00dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.516+0000 7f0474d8b700 1 --2- 172.21.15.46:0/2510527854 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49e874a0 0x55ad4a00dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.516+0000 7f0474d8b700 1 -- 172.21.15.46:0/2510527854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad49da2490 con 0x55ad4a004400 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.516+0000 7f046ffff700 1 --2- 172.21.15.46:0/2510527854 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49e874a0 0x55ad4a00dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:00:50.521 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.516+0000 7f0467fff700 1 --2- 172.21.15.46:0/2510527854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad49f01d00 0x55ad4a00e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:00:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.516+0000 7f0474d8b700 1 --2- 172.21.15.46:0/2510527854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad4a004400 0x55ad4a00ea70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0468015000 tx=0x7f04680075c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db61b33168f39254 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.516+0000 7f044f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.516+0000 7f044f7fe700 1 -- 172.21.15.46:0/2510527854 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468003e10 con 0x55ad4a004400 2024-05-09T05:00:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.516+0000 7f0485204b80 1 -- 172.21.15.46:0/2510527854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f045800de40 con 0x55ad4a004400 2024-05-09T05:00:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.516+0000 7f044f7fe700 1 -- 172.21.15.46:0/2510527854 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f0468007860 con 0x55ad4a004400 2024-05-09T05:00:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.517+0000 7f044f7fe700 1 -- 172.21.15.46:0/2510527854 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f0468007a60 con 0x55ad4a004400 2024-05-09T05:00:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.517+0000 7f0485204b80 10 client.17439 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.517+0000 7f0485204b80 1 -- 172.21.15.46:0/2510527854 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ad4a01ba50 con 0x55ad4a004400 2024-05-09T05:00:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.517+0000 7f044f7fe700 1 -- 172.21.15.46:0/2510527854 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f045800de40 con 0x55ad4a004400 2024-05-09T05:00:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.517+0000 7f044f7fe700 1 -- 172.21.15.46:0/2510527854 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f0468024020 con 0x55ad4a004400 2024-05-09T05:00:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.517+0000 7f0485204b80 10 client.17439 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.517+0000 7f0485204b80 10 client.17439 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T05:00:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.517+0000 7f0485204b80 20 client.17439 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.517+0000 7f0485204b80 1 --2- 172.21.15.46:0/2510527854 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55ad4a01e090 0x55ad4a0204b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.522 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.517+0000 7f0485204b80 4 client.17439 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.517+0000 7f0485204b80 1 -- 172.21.15.46:0/2510527854 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad49f34a80 con 0x55ad4a01e090 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.517+0000 7f046ffff700 1 --2- 172.21.15.46:0/2510527854 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55ad4a01e090 0x55ad4a0204b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.518+0000 7f044f7fe700 10 client.17439 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.518+0000 7f046ffff700 1 --2- 172.21.15.46:0/2510527854 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55ad4a01e090 0x55ad4a0204b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c67eb700b84983bd server_cookie=ee6a4a81ceddd645 in_seq=0 out_seq=0 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.519+0000 7f044f7fe700 1 -- 172.21.15.46:0/2510527854 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ad49f34a80 con 0x55ad4a01e090 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.519+0000 7f044f7fe700 10 client.17439 handle_command_reply: tid=0 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.519+0000 7f0485204b80 1 client.17439 shutdown 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.519+0000 7f0485204b80 1 -- 172.21.15.46:0/2510527854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad4a004400 msgr2=0x55ad4a00ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.519+0000 7f0485204b80 1 --2- 172.21.15.46:0/2510527854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad4a004400 0x55ad4a00ea70 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f0468015000 tx=0x7f04680075c0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.519+0000 7f0485204b80 1 -- 172.21.15.46:0/2510527854 shutdown_connections 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.519+0000 7f0485204b80 1 --2- 172.21.15.46:0/2510527854 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55ad4a01e090 0x55ad4a0204b0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.519+0000 7f0485204b80 1 --2- 172.21.15.46:0/2510527854 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad4a004400 0x55ad4a00ea70 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.519+0000 7f0485204b80 1 --2- 172.21.15.46:0/2510527854 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49e874a0 0x55ad4a00dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.519+0000 7f0485204b80 1 --2- 172.21.15.46:0/2510527854 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad49f01d00 0x55ad4a00e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.519+0000 7f044f7fe700 0 client.17439 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.519+0000 7f0485204b80 1 -- 172.21.15.46:0/2510527854 >> 172.21.15.46:0/2510527854 conn(0x55ad49e8af70 msgr2=0x55ad49ff8ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:50.523 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.519+0000 7f0485204b80 1 -- 172.21.15.46:0/2510527854 shutdown_connections 2024-05-09T05:00:50.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.519+0000 7f0485204b80 1 -- 172.21.15.46:0/2510527854 wait complete. 2024-05-09T05:00:50.524 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.519+0000 7f0485204b80 20 client.17439 trim_cache size 0 max 16384 2024-05-09T05:00:50.545 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.534+0000 7f0485204b80 1 Processor -- start 2024-05-09T05:00:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.534+0000 7f0485204b80 1 -- start start 2024-05-09T05:00:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0485204b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49e874a0 0x55ad4a003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0485204b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad49f01d00 0x55ad4a00e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0485204b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad4a004400 0x55ad4a019e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.546 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0485204b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f04701b17c0 con 0x55ad4a004400 2024-05-09T05:00:50.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0485204b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f04701af050 con 0x55ad49e874a0 2024-05-09T05:00:50.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0485204b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04701acec0 con 0x55ad49f01d00 2024-05-09T05:00:50.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0474d8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad4a004400 0x55ad4a019e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0474d8b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad4a004400 0x55ad4a019e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39922/0 (socket says 172.21.15.46:39922) 2024-05-09T05:00:50.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0474d8b700 1 -- 172.21.15.46:0/3580275094 learned_addr learned my addr 172.21.15.46:0/3580275094 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:50.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f046ffff700 1 --2- 172.21.15.46:0/3580275094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49e874a0 0x55ad4a003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.547 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0467fff700 1 --2- 172.21.15.46:0/3580275094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad49f01d00 0x55ad4a00e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0474d8b700 1 -- 172.21.15.46:0/3580275094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad49f01d00 msgr2=0x55ad4a00e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0474d8b700 1 --2- 172.21.15.46:0/3580275094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad49f01d00 0x55ad4a00e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0474d8b700 1 -- 172.21.15.46:0/3580275094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49e874a0 msgr2=0x55ad4a003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0474d8b700 1 --2- 172.21.15.46:0/3580275094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49e874a0 0x55ad4a003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0474d8b700 1 -- 172.21.15.46:0/3580275094 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad49ffb9d0 con 0x55ad4a004400 2024-05-09T05:00:50.548 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f046ffff700 1 --2- 172.21.15.46:0/3580275094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49e874a0 0x55ad4a003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:00:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0467fff700 1 --2- 172.21.15.46:0/3580275094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad49f01d00 0x55ad4a00e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:00:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.535+0000 7f0474d8b700 1 --2- 172.21.15.46:0/3580275094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad4a004400 0x55ad4a019e60 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0468009c00 tx=0x7f0468007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8deb6ce2fec7ffbc server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.536+0000 7f0465ffb700 1 -- 172.21.15.46:0/3580275094 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0468013e20 con 0x55ad4a004400 2024-05-09T05:00:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.536+0000 7f0465ffb700 1 -- 172.21.15.46:0/3580275094 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0468007c00 con 0x55ad4a004400 2024-05-09T05:00:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.536+0000 7f0465ffb700 1 -- 172.21.15.46:0/3580275094 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04680050c0 con 0x55ad4a004400 2024-05-09T05:00:50.549 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.536+0000 7f0485204b80 1 -- 172.21.15.46:0/3580275094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad4a004400 msgr2=0x55ad4a019e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.536+0000 7f0485204b80 1 --2- 172.21.15.46:0/3580275094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad4a004400 0x55ad4a019e60 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f0468009c00 tx=0x7f0468007990 comp rx=0 tx=0).stop 2024-05-09T05:00:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.536+0000 7f0485204b80 1 -- 172.21.15.46:0/3580275094 shutdown_connections 2024-05-09T05:00:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.536+0000 7f0485204b80 1 --2- 172.21.15.46:0/3580275094 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad4a004400 0x55ad4a019e60 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.536+0000 7f0485204b80 1 --2- 172.21.15.46:0/3580275094 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49e874a0 0x55ad4a003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.550 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.536+0000 7f0485204b80 1 --2- 172.21.15.46:0/3580275094 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad49f01d00 0x55ad4a00e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.536+0000 7f0485204b80 1 -- 172.21.15.46:0/3580275094 >> 172.21.15.46:0/3580275094 conn(0x55ad49e8af70 msgr2=0x55ad49ff9660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:50.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.537+0000 7f0485204b80 1 -- 172.21.15.46:0/3580275094 shutdown_connections 2024-05-09T05:00:50.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.537+0000 7f0485204b80 1 -- 172.21.15.46:0/3580275094 wait complete. 2024-05-09T05:00:50.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.537+0000 7f0485204b80 1 -- start start 2024-05-09T05:00:50.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.537+0000 7f0485204b80 1 Processor -- start 2024-05-09T05:00:50.551 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f0485204b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad49e874a0 0x55ad4a016610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f0485204b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49f01d00 0x55ad4a013170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f0485204b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad4a004400 0x55ad4a0136b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f0485204b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f04701ad510 con 0x55ad49e874a0 2024-05-09T05:00:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f0485204b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f04701ae980 con 0x55ad49f01d00 2024-05-09T05:00:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f0485204b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f04701adbe0 con 0x55ad4a004400 2024-05-09T05:00:50.552 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f0485204b80 10 client.0 _ll_register_callbacks cb 0x55ad49d986c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:50.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f046ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad49e874a0 0x55ad4a016610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f046ffff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad49e874a0 0x55ad4a016610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39936/0 (socket says 172.21.15.46:39936) 2024-05-09T05:00:50.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f046ffff700 1 -- 172.21.15.46:0/1892918811 learned_addr learned my addr 172.21.15.46:0/1892918811 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:50.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f0474d8b700 1 --2- 172.21.15.46:0/1892918811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad4a004400 0x55ad4a0136b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f0467fff700 1 --2- 172.21.15.46:0/1892918811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49f01d00 0x55ad4a013170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f046ffff700 1 -- 172.21.15.46:0/1892918811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad4a004400 msgr2=0x55ad4a0136b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:50.553 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f046ffff700 1 --2- 172.21.15.46:0/1892918811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad4a004400 0x55ad4a0136b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f046ffff700 1 -- 172.21.15.46:0/1892918811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49f01d00 msgr2=0x55ad4a013170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f046ffff700 1 --2- 172.21.15.46:0/1892918811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49f01d00 0x55ad4a013170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f046ffff700 1 -- 172.21.15.46:0/1892918811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad49ff8cc0 con 0x55ad49e874a0 2024-05-09T05:00:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f046ffff700 1 --2- 172.21.15.46:0/1892918811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad49e874a0 0x55ad4a016610 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f04600a5520 tx=0x7f0460005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71048b59feff30 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f044f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:50.554 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.538+0000 7f044f7fe700 1 -- 172.21.15.46:0/1892918811 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04600c3060 con 0x55ad49e874a0 2024-05-09T05:00:50.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.539+0000 7f044f7fe700 1 -- 172.21.15.46:0/1892918811 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7f04600a9700 con 0x55ad49e874a0 2024-05-09T05:00:50.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.539+0000 7f0485204b80 1 -- 172.21.15.46:0/1892918811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0468007c00 con 0x55ad49e874a0 2024-05-09T05:00:50.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.539+0000 7f044f7fe700 1 -- 172.21.15.46:0/1892918811 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f04600ca050 con 0x55ad49e874a0 2024-05-09T05:00:50.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.539+0000 7f0485204b80 10 client.17445 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:50.555 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.539+0000 7f0485204b80 1 -- 172.21.15.46:0/1892918811 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55ad4a00aaf0 con 0x55ad49e874a0 2024-05-09T05:00:50.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.539+0000 7f044f7fe700 1 -- 172.21.15.46:0/1892918811 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0468007c00 con 0x55ad49e874a0 2024-05-09T05:00:50.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.539+0000 7f044f7fe700 1 -- 172.21.15.46:0/1892918811 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f04600b1e70 con 0x55ad49e874a0 2024-05-09T05:00:50.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.539+0000 7f0485204b80 10 client.17445 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:50.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.539+0000 7f0485204b80 10 client.17445 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T05:00:50.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.539+0000 7f0485204b80 20 client.17445 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:50.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.539+0000 7f0485204b80 1 --2- 172.21.15.46:0/1892918811 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55ad4a089800 0x55ad4a08bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:50.556 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.539+0000 7f0485204b80 4 client.17445 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:00:50.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.539+0000 7f0485204b80 1 -- 172.21.15.46:0/1892918811 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ad49f2eb20 con 0x55ad4a089800 2024-05-09T05:00:50.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.539+0000 7f0467fff700 1 --2- 172.21.15.46:0/1892918811 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55ad4a089800 0x55ad4a08bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:50.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.540+0000 7f044f7fe700 10 client.17445 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T05:00:50.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.540+0000 7f0467fff700 1 --2- 172.21.15.46:0/1892918811 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55ad4a089800 0x55ad4a08bc20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f62a1ca2b4edce3 server_cookie=cd2e8ca5a170bb4b in_seq=0 out_seq=0 2024-05-09T05:00:50.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.540+0000 7f044f7fe700 1 -- 172.21.15.46:0/1892918811 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ad49f2eb20 con 0x55ad4a089800 2024-05-09T05:00:50.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.540+0000 7f044f7fe700 10 client.17445 handle_command_reply: tid=0 2024-05-09T05:00:50.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.540+0000 7f0485204b80 1 client.17445 shutdown 2024-05-09T05:00:50.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.540+0000 7f0485204b80 1 -- 172.21.15.46:0/1892918811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad49e874a0 msgr2=0x55ad4a016610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:50.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.540+0000 7f0485204b80 1 --2- 172.21.15.46:0/1892918811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad49e874a0 0x55ad4a016610 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f04600a5520 tx=0x7f0460005c80 comp rx=0 tx=0).stop 2024-05-09T05:00:50.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.540+0000 7f0485204b80 1 -- 172.21.15.46:0/1892918811 shutdown_connections 2024-05-09T05:00:50.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.540+0000 7f0485204b80 1 --2- 172.21.15.46:0/1892918811 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55ad4a089800 0x55ad4a08bc20 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.557 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.540+0000 7f0485204b80 1 --2- 172.21.15.46:0/1892918811 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55ad4a004400 0x55ad4a0136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.540+0000 7f0485204b80 1 --2- 172.21.15.46:0/1892918811 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55ad49e874a0 0x55ad4a016610 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.540+0000 7f0485204b80 1 --2- 172.21.15.46:0/1892918811 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55ad49f01d00 0x55ad4a013170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.540+0000 7f0485204b80 1 -- 172.21.15.46:0/1892918811 >> 172.21.15.46:0/1892918811 conn(0x55ad49e8af70 msgr2=0x55ad4a00f630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:50.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.540+0000 7f044f7fe700 0 client.17445 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T05:00:50.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.541+0000 7f0485204b80 1 -- 172.21.15.46:0/1892918811 shutdown_connections 2024-05-09T05:00:50.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.541+0000 7f0485204b80 1 -- 172.21.15.46:0/1892918811 wait complete. 2024-05-09T05:00:50.558 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.541+0000 7f0485204b80 20 client.17445 trim_cache size 0 max 16384 2024-05-09T05:00:50.558 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T05:00:50.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.541+0000 7f04767ee700 1 -- 172.21.15.46:0/2545757312 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f045c0b2a70 msgr2=0x7f045c0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:50.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.541+0000 7f04767ee700 1 --2- 172.21.15.46:0/2545757312 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f045c0b2a70 0x7f045c0b4f80 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0458006fd0 tx=0x7f0458008040 comp rx=0 tx=0).stop 2024-05-09T05:00:50.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.541+0000 7f04767ee700 1 -- 172.21.15.46:0/2545757312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04701045f0 msgr2=0x7f047019ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:50.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.541+0000 7f04767ee700 1 --2- 172.21.15.46:0/2545757312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04701045f0 0x7f047019ae60 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f046000d950 tx=0x7f046000dc60 comp rx=0 tx=0).stop 2024-05-09T05:00:50.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.541+0000 7f04767ee700 1 -- 172.21.15.46:0/2545757312 shutdown_connections 2024-05-09T05:00:50.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.541+0000 7f04767ee700 1 --2- 172.21.15.46:0/2545757312 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f045c0b2a70 0x7f045c0b4f80 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.541+0000 7f04767ee700 1 --2- 172.21.15.46:0/2545757312 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f0470109870 0x7f047019f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.541+0000 7f04767ee700 1 --2- 172.21.15.46:0/2545757312 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f04701045f0 0x7f047019ae60 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.559 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.541+0000 7f04767ee700 1 --2- 172.21.15.46:0/2545757312 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f0470108f20 0x7f047019b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:50.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.541+0000 7f04767ee700 1 -- 172.21.15.46:0/2545757312 >> 172.21.15.46:0/2545757312 conn(0x7f0470100040 msgr2=0x7f0470108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:50.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.541+0000 7f04767ee700 1 -- 172.21.15.46:0/2545757312 shutdown_connections 2024-05-09T05:00:50.560 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:50.541+0000 7f04767ee700 1 -- 172.21.15.46:0/2545757312 wait complete. 2024-05-09T05:00:51.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:50 smithi072 ceph-mon[34088]: pgmap v301: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T05:00:51.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:50 smithi110 ceph-mon[34904]: pgmap v301: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T05:00:51.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:50 smithi046 ceph-mon[30136]: pgmap v301: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 5 op/s 2024-05-09T05:00:51.360 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T05:00:51.612 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:52.001 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:51 smithi046 ceph-mon[30136]: pgmap v302: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T05:00:52.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:51 smithi072 ceph-mon[34088]: pgmap v302: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T05:00:52.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:51 smithi110 ceph-mon[34904]: pgmap v302: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-05-09T05:00:53.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.373+0000 7fd884b6a700 1 -- 172.21.15.46:0/3347058239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8801045e0 msgr2=0x7fd8801049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:53.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.373+0000 7fd884b6a700 1 --2- 172.21.15.46:0/3347058239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8801045e0 0x7fd8801049f0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fd8700099c0 tx=0x7fd870009cd0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.373+0000 7fd884b6a700 1 -- 172.21.15.46:0/3347058239 shutdown_connections 2024-05-09T05:00:53.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.373+0000 7fd884b6a700 1 --2- 172.21.15.46:0/3347058239 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd880109860 0x7fd8801111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.373+0000 7fd884b6a700 1 --2- 172.21.15.46:0/3347058239 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8801045e0 0x7fd8801049f0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.373+0000 7fd884b6a700 1 --2- 172.21.15.46:0/3347058239 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd880108f10 0x7fd880109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.373+0000 7fd884b6a700 1 -- 172.21.15.46:0/3347058239 >> 172.21.15.46:0/3347058239 conn(0x7fd8800ffed0 msgr2=0x7fd880102320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:53.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.373+0000 7fd884b6a700 1 -- 172.21.15.46:0/3347058239 shutdown_connections 2024-05-09T05:00:53.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.373+0000 7fd884b6a700 1 -- 172.21.15.46:0/3347058239 wait complete. 2024-05-09T05:00:53.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.374+0000 7fd884b6a700 1 Processor -- start 2024-05-09T05:00:53.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.374+0000 7fd884b6a700 1 -- start start 2024-05-09T05:00:53.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.374+0000 7fd884b6a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8801045e0 0x7fd88019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.374+0000 7fd884b6a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd880108f10 0x7fd88019b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.374+0000 7fd884b6a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd880109860 0x7fd88019f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd884b6a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd880114490 con 0x7fd8801045e0 2024-05-09T05:00:53.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd884b6a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd880114310 con 0x7fd880108f10 2024-05-09T05:00:53.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd884b6a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd880114610 con 0x7fd880109860 2024-05-09T05:00:53.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd87e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8801045e0 0x7fd88019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd87e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8801045e0 0x7fd88019ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39956/0 (socket says 172.21.15.46:39956) 2024-05-09T05:00:53.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd87e59c700 1 -- 172.21.15.46:0/1088046251 learned_addr learned my addr 172.21.15.46:0/1088046251 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:53.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd87ed9d700 1 --2- 172.21.15.46:0/1088046251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd880109860 0x7fd88019f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/1088046251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd880108f10 0x7fd88019b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd87e59c700 1 -- 172.21.15.46:0/1088046251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd880109860 msgr2=0x7fd88019f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:53.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd87e59c700 1 --2- 172.21.15.46:0/1088046251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd880109860 0x7fd88019f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd87e59c700 1 -- 172.21.15.46:0/1088046251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd880108f10 msgr2=0x7fd88019b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:53.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd87e59c700 1 --2- 172.21.15.46:0/1088046251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd880108f10 0x7fd88019b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd87e59c700 1 -- 172.21.15.46:0/1088046251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd88019fcb0 con 0x7fd8801045e0 2024-05-09T05:00:53.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/1088046251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd880108f10 0x7fd88019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:00:53.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd87e59c700 1 --2- 172.21.15.46:0/1088046251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8801045e0 0x7fd88019ade0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd87000b580 tx=0x7fd8700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:53.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd8777fe700 1 -- 172.21.15.46:0/1088046251 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870027070 con 0x7fd8801045e0 2024-05-09T05:00:53.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd8777fe700 1 -- 172.21.15.46:0/1088046251 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd870003f20 con 0x7fd8801045e0 2024-05-09T05:00:53.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.375+0000 7fd884b6a700 1 -- 172.21.15.46:0/1088046251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd88019ff40 con 0x7fd8801045e0 2024-05-09T05:00:53.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.376+0000 7fd8777fe700 1 -- 172.21.15.46:0/1088046251 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd870022d70 con 0x7fd8801045e0 2024-05-09T05:00:53.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.376+0000 7fd884b6a700 1 -- 172.21.15.46:0/1088046251 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8801a0420 con 0x7fd8801045e0 2024-05-09T05:00:53.381 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.378+0000 7fd8777fe700 1 -- 172.21.15.46:0/1088046251 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd87000f460 con 0x7fd8801045e0 2024-05-09T05:00:53.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.379+0000 7fd8777fe700 1 --2- 172.21.15.46:0/1088046251 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd86c0b29f0 0x7fd86c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.379+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/1088046251 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd86c0b29f0 0x7fd86c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.379+0000 7fd8777fe700 1 -- 172.21.15.46:0/1088046251 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7fd8700e3340 con 0x7fd8801045e0 2024-05-09T05:00:53.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.379+0000 7fd893580b80 1 Processor -- start 2024-05-09T05:00:53.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.380+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/1088046251 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd86c0b29f0 0x7fd86c0b4f00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd868005950 tx=0x7fd8680058e0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:53.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.380+0000 7fd893580b80 1 -- start start 2024-05-09T05:00:53.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.380+0000 7fd893580b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d247a5dd00 0x55d247b66240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.380+0000 7fd893580b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d2479e34a0 0x55d247b66780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.380+0000 7fd893580b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d247b60400 0x55d247b60810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.380+0000 7fd893580b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d247a830a0 con 0x55d247a5dd00 2024-05-09T05:00:53.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.380+0000 7fd893580b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d247a289a0 con 0x55d247b60400 2024-05-09T05:00:53.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.380+0000 7fd893580b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d247a7f060 con 0x55d2479e34a0 2024-05-09T05:00:53.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.380+0000 7fd87dd9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d2479e34a0 0x55d247b66780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.380+0000 7fd87e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d247a5dd00 0x55d247b66240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.380+0000 7fd87dd9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d2479e34a0 0x55d247b66780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.110:3300/0 says I am v2:172.21.15.46:41998/0 (socket says 172.21.15.46:41998) 2024-05-09T05:00:53.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.380+0000 7fd87dd9b700 1 -- 172.21.15.46:0/1503637103 learned_addr learned my addr 172.21.15.46:0/1503637103 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:53.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.380+0000 7fd87ed9d700 1 --2- 172.21.15.46:0/1503637103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d247b60400 0x55d247b60810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.381+0000 7fd87dd9b700 1 -- 172.21.15.46:0/1503637103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d247b60400 msgr2=0x55d247b60810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:53.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.381+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/1503637103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d247b60400 0x55d247b60810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.381+0000 7fd87dd9b700 1 -- 172.21.15.46:0/1503637103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d247a5dd00 msgr2=0x55d247b66240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:53.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.381+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/1503637103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d247a5dd00 0x55d247b66240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.381+0000 7fd87dd9b700 1 -- 172.21.15.46:0/1503637103 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d247a19500 con 0x55d2479e34a0 2024-05-09T05:00:53.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.381+0000 7fd87e59c700 1 --2- 172.21.15.46:0/1503637103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d247a5dd00 0x55d247b66240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:00:53.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.381+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/1503637103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d2479e34a0 0x55d247b66780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd86800b5d0 tx=0x7fd868012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5143c6c92c7b362 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:53.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.381+0000 7fd874ff9700 1 -- 172.21.15.46:0/1503637103 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd868003880 con 0x55d2479e34a0 2024-05-09T05:00:53.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.381+0000 7fd874ff9700 1 -- 172.21.15.46:0/1503637103 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd868024070 con 0x55d2479e34a0 2024-05-09T05:00:53.384 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.381+0000 7fd874ff9700 1 -- 172.21.15.46:0/1503637103 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86801fcc0 con 0x55d2479e34a0 2024-05-09T05:00:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.382+0000 7fd893580b80 1 -- 172.21.15.46:0/1503637103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d2479e34a0 msgr2=0x55d247b66780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.382+0000 7fd893580b80 1 --2- 172.21.15.46:0/1503637103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d2479e34a0 0x55d247b66780 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd86800b5d0 tx=0x7fd868012000 comp rx=0 tx=0).stop 2024-05-09T05:00:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.382+0000 7fd893580b80 1 -- 172.21.15.46:0/1503637103 shutdown_connections 2024-05-09T05:00:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.382+0000 7fd893580b80 1 --2- 172.21.15.46:0/1503637103 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d247b60400 0x55d247b60810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.382+0000 7fd893580b80 1 --2- 172.21.15.46:0/1503637103 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d247a5dd00 0x55d247b66240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.382+0000 7fd893580b80 1 --2- 172.21.15.46:0/1503637103 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d2479e34a0 0x55d247b66780 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.382+0000 7fd893580b80 1 -- 172.21.15.46:0/1503637103 >> 172.21.15.46:0/1503637103 conn(0x55d2479e6f70 msgr2=0x55d247b5b220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.382+0000 7fd893580b80 1 -- 172.21.15.46:0/1503637103 shutdown_connections 2024-05-09T05:00:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.382+0000 7fd893580b80 1 -- 172.21.15.46:0/1503637103 wait complete. 2024-05-09T05:00:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.383+0000 7fd893580b80 1 -- start start 2024-05-09T05:00:53.388 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.383+0000 7fd893580b80 1 Processor -- start 2024-05-09T05:00:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.383+0000 7fd893580b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2479e34a0 0x55d247b6a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.383+0000 7fd893580b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d247a5dd00 0x55d247b6a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.383+0000 7fd893580b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d247b60400 0x55d247b6ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.383+0000 7fd893580b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55d247b61160 con 0x55d2479e34a0 2024-05-09T05:00:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.383+0000 7fd893580b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55d247a1db20 con 0x55d247b60400 2024-05-09T05:00:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.383+0000 7fd893580b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55d247b6d020 con 0x55d247a5dd00 2024-05-09T05:00:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.383+0000 7fd893580b80 10 client.0 _ll_register_callbacks cb 0x55d2478f46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.383+0000 7fd87e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2479e34a0 0x55d247b6a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.383+0000 7fd87e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2479e34a0 0x55d247b6a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39966/0 (socket says 172.21.15.46:39966) 2024-05-09T05:00:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.383+0000 7fd87e59c700 1 -- 172.21.15.46:0/259132865 learned_addr learned my addr 172.21.15.46:0/259132865 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.383+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/259132865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d247a5dd00 0x55d247b6a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.383+0000 7fd87ed9d700 1 --2- 172.21.15.46:0/259132865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d247b60400 0x55d247b6ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.389 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.384+0000 7fd87e59c700 1 -- 172.21.15.46:0/259132865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d247a5dd00 msgr2=0x55d247b6a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.384+0000 7fd87e59c700 1 --2- 172.21.15.46:0/259132865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d247a5dd00 0x55d247b6a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.384+0000 7fd87e59c700 1 -- 172.21.15.46:0/259132865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d247b60400 msgr2=0x55d247b6ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.384+0000 7fd87e59c700 1 --2- 172.21.15.46:0/259132865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d247b60400 0x55d247b6ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.384+0000 7fd87e59c700 1 -- 172.21.15.46:0/259132865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2478fe490 con 0x55d2479e34a0 2024-05-09T05:00:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.384+0000 7fd87e59c700 1 --2- 172.21.15.46:0/259132865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2479e34a0 0x55d247b6a110 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd8700aa360 tx=0x7fd8700227e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23373eb6730fea07 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.384+0000 7fd865ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.384+0000 7fd865ffb700 1 -- 172.21.15.46:0/259132865 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8700aa660 con 0x55d2479e34a0 2024-05-09T05:00:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.384+0000 7fd865ffb700 1 -- 172.21.15.46:0/259132865 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7fd8700b2fa0 con 0x55d2479e34a0 2024-05-09T05:00:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.384+0000 7fd893580b80 1 -- 172.21.15.46:0/259132865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd868024070 con 0x55d2479e34a0 2024-05-09T05:00:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.384+0000 7fd865ffb700 1 -- 172.21.15.46:0/259132865 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd8700aea10 con 0x55d2479e34a0 2024-05-09T05:00:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.384+0000 7fd893580b80 10 client.17457 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:53.390 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.384+0000 7fd893580b80 1 -- 172.21.15.46:0/259132865 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d247b77ae0 con 0x55d2479e34a0 2024-05-09T05:00:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.384+0000 7fd865ffb700 1 -- 172.21.15.46:0/259132865 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd868024070 con 0x55d2479e34a0 2024-05-09T05:00:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.385+0000 7fd865ffb700 1 -- 172.21.15.46:0/259132865 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd8700a98c0 con 0x55d2479e34a0 2024-05-09T05:00:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.385+0000 7fd893580b80 10 client.17457 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.385+0000 7fd893580b80 10 client.17457 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.385+0000 7fd893580b80 20 client.17457 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.385+0000 7fd893580b80 1 --2- 172.21.15.46:0/259132865 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d247b7a0a0 0x55d247b7c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.385+0000 7fd893580b80 4 client.17457 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:00:53.391 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.385+0000 7fd893580b80 1 -- 172.21.15.46:0/259132865 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d247a90a80 con 0x55d247b7a0a0 2024-05-09T05:00:53.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.385+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/259132865 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d247b7a0a0 0x55d247b7c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.385+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/259132865 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d247b7a0a0 0x55d247b7c4e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=500f493576e2e2af server_cookie=bd6f2d48a652419b in_seq=0 out_seq=0 2024-05-09T05:00:53.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.385+0000 7fd865ffb700 10 client.17457 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:53.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.386+0000 7fd865ffb700 1 -- 172.21.15.46:0/259132865 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d247a90a80 con 0x55d247b7a0a0 2024-05-09T05:00:53.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.386+0000 7fd865ffb700 10 client.17457 handle_command_reply: tid=0 2024-05-09T05:00:53.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.386+0000 7fd893580b80 1 client.17457 shutdown 2024-05-09T05:00:53.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.386+0000 7fd893580b80 1 -- 172.21.15.46:0/259132865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2479e34a0 msgr2=0x55d247b6a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:53.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.386+0000 7fd893580b80 1 --2- 172.21.15.46:0/259132865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2479e34a0 0x55d247b6a110 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fd8700aa360 tx=0x7fd8700227e0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.386+0000 7fd893580b80 1 -- 172.21.15.46:0/259132865 shutdown_connections 2024-05-09T05:00:53.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.386+0000 7fd893580b80 1 --2- 172.21.15.46:0/259132865 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d247b7a0a0 0x55d247b7c4e0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.386+0000 7fd893580b80 1 --2- 172.21.15.46:0/259132865 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d247b60400 0x55d247b6ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.386+0000 7fd893580b80 1 --2- 172.21.15.46:0/259132865 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2479e34a0 0x55d247b6a110 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.392 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.386+0000 7fd893580b80 1 --2- 172.21.15.46:0/259132865 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d247a5dd00 0x55d247b6a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.386+0000 7fd893580b80 1 -- 172.21.15.46:0/259132865 >> 172.21.15.46:0/259132865 conn(0x55d2479e6f70 msgr2=0x55d247b54ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:53.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.387+0000 7fd865ffb700 0 client.17457 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:53.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.387+0000 7fd893580b80 1 -- 172.21.15.46:0/259132865 shutdown_connections 2024-05-09T05:00:53.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.387+0000 7fd893580b80 1 -- 172.21.15.46:0/259132865 wait complete. 2024-05-09T05:00:53.393 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.387+0000 7fd893580b80 20 client.17457 trim_cache size 0 max 16384 2024-05-09T05:00:53.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.402+0000 7fd893580b80 1 Processor -- start 2024-05-09T05:00:53.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.402+0000 7fd893580b80 1 -- start start 2024-05-09T05:00:53.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.402+0000 7fd893580b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d2479e34a0 0x55d247b65570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.402+0000 7fd893580b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d247a5dd00 0x55d247b55700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.402+0000 7fd893580b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d247b60400 0x55d247b81c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.402+0000 7fd893580b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801b16a0 con 0x55d247a5dd00 2024-05-09T05:00:53.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.402+0000 7fd893580b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801aef30 con 0x55d2479e34a0 2024-05-09T05:00:53.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.402+0000 7fd893580b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801acda0 con 0x55d247b60400 2024-05-09T05:00:53.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.402+0000 7fd87dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d247a5dd00 0x55d247b55700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.402+0000 7fd87dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d247a5dd00 0x55d247b55700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39974/0 (socket says 172.21.15.46:39974) 2024-05-09T05:00:53.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.402+0000 7fd87dd9b700 1 -- 172.21.15.46:0/893385667 learned_addr learned my addr 172.21.15.46:0/893385667 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.402+0000 7fd87e59c700 1 --2- 172.21.15.46:0/893385667 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d2479e34a0 0x55d247b65570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.402+0000 7fd87ed9d700 1 --2- 172.21.15.46:0/893385667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d247b60400 0x55d247b81c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.403+0000 7fd87dd9b700 1 -- 172.21.15.46:0/893385667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d247b60400 msgr2=0x55d247b81c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.403+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/893385667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d247b60400 0x55d247b81c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.403+0000 7fd87dd9b700 1 -- 172.21.15.46:0/893385667 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d2479e34a0 msgr2=0x55d247b65570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.403+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/893385667 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d2479e34a0 0x55d247b65570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.403+0000 7fd87dd9b700 1 -- 172.21.15.46:0/893385667 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d247b61040 con 0x55d247a5dd00 2024-05-09T05:00:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.403+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/893385667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d247a5dd00 0x55d247b55700 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd8680045f0 tx=0x7fd868005ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b1f17ed603c22af server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.403+0000 7fd867fff700 1 -- 172.21.15.46:0/893385667 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd86803b070 con 0x55d247a5dd00 2024-05-09T05:00:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.403+0000 7fd867fff700 1 -- 172.21.15.46:0/893385667 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd868024040 con 0x55d247a5dd00 2024-05-09T05:00:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.403+0000 7fd867fff700 1 -- 172.21.15.46:0/893385667 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8680363d0 con 0x55d247a5dd00 2024-05-09T05:00:53.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.404+0000 7fd893580b80 1 -- 172.21.15.46:0/893385667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d247a5dd00 msgr2=0x55d247b55700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:53.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.404+0000 7fd893580b80 1 --2- 172.21.15.46:0/893385667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d247a5dd00 0x55d247b55700 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fd8680045f0 tx=0x7fd868005ca0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.404+0000 7fd893580b80 1 -- 172.21.15.46:0/893385667 shutdown_connections 2024-05-09T05:00:53.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.404+0000 7fd893580b80 1 --2- 172.21.15.46:0/893385667 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d247b60400 0x55d247b81c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.404+0000 7fd893580b80 1 --2- 172.21.15.46:0/893385667 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d2479e34a0 0x55d247b65570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.404+0000 7fd893580b80 1 --2- 172.21.15.46:0/893385667 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d247a5dd00 0x55d247b55700 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.404+0000 7fd893580b80 1 -- 172.21.15.46:0/893385667 >> 172.21.15.46:0/893385667 conn(0x55d2479e6f70 msgr2=0x55d247b54ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:53.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.404+0000 7fd893580b80 1 -- 172.21.15.46:0/893385667 shutdown_connections 2024-05-09T05:00:53.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.404+0000 7fd893580b80 1 -- 172.21.15.46:0/893385667 wait complete. 2024-05-09T05:00:53.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.404+0000 7fd893580b80 1 -- start start 2024-05-09T05:00:53.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.404+0000 7fd893580b80 1 Processor -- start 2024-05-09T05:00:53.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd893580b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2479e34a0 0x55d247b6f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd893580b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d247a5dd00 0x55d247b6f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd893580b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d247b60400 0x55d247b6faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd893580b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801ad3f0 con 0x55d2479e34a0 2024-05-09T05:00:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd893580b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801ae860 con 0x55d247a5dd00 2024-05-09T05:00:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd893580b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fd8801adac0 con 0x55d247b60400 2024-05-09T05:00:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd893580b80 10 client.0 _ll_register_callbacks cb 0x55d2478f46c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd87e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2479e34a0 0x55d247b6f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd87e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2479e34a0 0x55d247b6f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:39984/0 (socket says 172.21.15.46:39984) 2024-05-09T05:00:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd87e59c700 1 -- 172.21.15.46:0/4203724514 learned_addr learned my addr 172.21.15.46:0/4203724514 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/4203724514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d247a5dd00 0x55d247b6f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd87ed9d700 1 --2- 172.21.15.46:0/4203724514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d247b60400 0x55d247b6faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd87e59c700 1 -- 172.21.15.46:0/4203724514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d247b60400 msgr2=0x55d247b6faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd87e59c700 1 --2- 172.21.15.46:0/4203724514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d247b60400 0x55d247b6faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd87e59c700 1 -- 172.21.15.46:0/4203724514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d247a5dd00 msgr2=0x55d247b6f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:53.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd87e59c700 1 --2- 172.21.15.46:0/4203724514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d247a5dd00 0x55d247b6f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd87e59c700 1 -- 172.21.15.46:0/4203724514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d247be6c30 con 0x55d2479e34a0 2024-05-09T05:00:53.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd87e59c700 1 --2- 172.21.15.46:0/4203724514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2479e34a0 0x55d247b6f070 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd8700aa4e0 tx=0x7fd870009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759a08e651164701 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:53.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd865ffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:53.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd865ffb700 1 -- 172.21.15.46:0/4203724514 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8700aa660 con 0x55d2479e34a0 2024-05-09T05:00:53.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd865ffb700 1 -- 172.21.15.46:0/4203724514 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7fd8700afdb0 con 0x55d2479e34a0 2024-05-09T05:00:53.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.405+0000 7fd893580b80 1 -- 172.21.15.46:0/4203724514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd868024040 con 0x55d2479e34a0 2024-05-09T05:00:53.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.406+0000 7fd865ffb700 1 -- 172.21.15.46:0/4203724514 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fd8700b3430 con 0x55d2479e34a0 2024-05-09T05:00:53.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.406+0000 7fd893580b80 10 client.17463 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:53.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.406+0000 7fd893580b80 1 -- 172.21.15.46:0/4203724514 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55d247be7e40 con 0x55d2479e34a0 2024-05-09T05:00:53.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.406+0000 7fd865ffb700 1 -- 172.21.15.46:0/4203724514 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd868024040 con 0x55d2479e34a0 2024-05-09T05:00:53.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.406+0000 7fd865ffb700 1 -- 172.21.15.46:0/4203724514 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fd8700a5fd0 con 0x55d2479e34a0 2024-05-09T05:00:53.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.406+0000 7fd893580b80 10 client.17463 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:53.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.406+0000 7fd893580b80 10 client.17463 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:53.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.406+0000 7fd893580b80 20 client.17463 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:53.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.406+0000 7fd893580b80 1 --2- 172.21.15.46:0/4203724514 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d247be8340 0x55d247bea760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:53.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.406+0000 7fd893580b80 4 client.17463 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:00:53.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.406+0000 7fd893580b80 1 -- 172.21.15.46:0/4203724514 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d247a8ab20 con 0x55d247be8340 2024-05-09T05:00:53.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.406+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/4203724514 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d247be8340 0x55d247bea760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:53.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd87dd9b700 1 --2- 172.21.15.46:0/4203724514 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d247be8340 0x55d247bea760 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99b3e7b4fd950286 server_cookie=56db58fe59f354c2 in_seq=0 out_seq=0 2024-05-09T05:00:53.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd865ffb700 10 client.17463 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:53.413 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd865ffb700 1 -- 172.21.15.46:0/4203724514 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d247a8ab20 con 0x55d247be8340 2024-05-09T05:00:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd865ffb700 10 client.17463 handle_command_reply: tid=0 2024-05-09T05:00:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd893580b80 1 client.17463 shutdown 2024-05-09T05:00:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd893580b80 1 -- 172.21.15.46:0/4203724514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2479e34a0 msgr2=0x55d247b6f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd893580b80 1 --2- 172.21.15.46:0/4203724514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2479e34a0 0x55d247b6f070 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fd8700aa4e0 tx=0x7fd870009cd0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd893580b80 1 -- 172.21.15.46:0/4203724514 shutdown_connections 2024-05-09T05:00:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd893580b80 1 --2- 172.21.15.46:0/4203724514 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55d247b60400 0x55d247b6faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd893580b80 1 --2- 172.21.15.46:0/4203724514 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55d2479e34a0 0x55d247b6f070 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd893580b80 1 --2- 172.21.15.46:0/4203724514 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55d247be8340 0x55d247bea760 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd865ffb700 0 client.17463 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd893580b80 1 --2- 172.21.15.46:0/4203724514 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55d247a5dd00 0x55d247b6f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd893580b80 1 -- 172.21.15.46:0/4203724514 >> 172.21.15.46:0/4203724514 conn(0x55d2479e6f70 msgr2=0x55d247b55780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:53.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd893580b80 1 -- 172.21.15.46:0/4203724514 shutdown_connections 2024-05-09T05:00:53.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd893580b80 1 -- 172.21.15.46:0/4203724514 wait complete. 2024-05-09T05:00:53.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.407+0000 7fd893580b80 20 client.17463 trim_cache size 0 max 16384 2024-05-09T05:00:53.415 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T05:00:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.408+0000 7fd884b6a700 1 -- 172.21.15.46:0/1088046251 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd86c0b29f0 msgr2=0x7fd86c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.408+0000 7fd884b6a700 1 --2- 172.21.15.46:0/1088046251 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd86c0b29f0 0x7fd86c0b4f00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd868005950 tx=0x7fd8680058e0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.408+0000 7fd884b6a700 1 -- 172.21.15.46:0/1088046251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8801045e0 msgr2=0x7fd88019ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.408+0000 7fd884b6a700 1 --2- 172.21.15.46:0/1088046251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8801045e0 0x7fd88019ade0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fd87000b580 tx=0x7fd8700047b0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.408+0000 7fd884b6a700 1 -- 172.21.15.46:0/1088046251 shutdown_connections 2024-05-09T05:00:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.408+0000 7fd884b6a700 1 --2- 172.21.15.46:0/1088046251 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fd86c0b29f0 0x7fd86c0b4f00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.408+0000 7fd884b6a700 1 --2- 172.21.15.46:0/1088046251 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fd880109860 0x7fd88019f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.408+0000 7fd884b6a700 1 --2- 172.21.15.46:0/1088046251 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fd8801045e0 0x7fd88019ade0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.408+0000 7fd884b6a700 1 --2- 172.21.15.46:0/1088046251 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fd880108f10 0x7fd88019b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.408+0000 7fd884b6a700 1 -- 172.21.15.46:0/1088046251 >> 172.21.15.46:0/1088046251 conn(0x7fd8800ffed0 msgr2=0x7fd8801086f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.408+0000 7fd884b6a700 1 -- 172.21.15.46:0/1088046251 shutdown_connections 2024-05-09T05:00:53.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:53.408+0000 7fd884b6a700 1 -- 172.21.15.46:0/1088046251 wait complete. 2024-05-09T05:00:55.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:54 smithi046 ceph-mon[30136]: pgmap v303: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T05:00:55.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:54 smithi072 ceph-mon[34088]: pgmap v303: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T05:00:55.180 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-05-09T05:00:55.181 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-05-09T05:00:55.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:54 smithi110 ceph-mon[34904]: pgmap v303: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-05-09T05:00:55.447 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:56.920 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:56 smithi046 ceph-mon[30136]: pgmap v304: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:00:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:56 smithi072 ceph-mon[34088]: pgmap v304: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:00:57.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:56 smithi110 ceph-mon[34904]: pgmap v304: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:00:57.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.345+0000 7fe2b9e9d700 1 -- 172.21.15.46:0/454795743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b4107df0 msgr2=0x7fe2b4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.345+0000 7fe2b9e9d700 1 --2- 172.21.15.46:0/454795743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b4107df0 0x7fe2b4108200 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80099c0 tx=0x7fe2a8009cd0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.345+0000 7fe2b9e9d700 1 -- 172.21.15.46:0/454795743 shutdown_connections 2024-05-09T05:00:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.345+0000 7fe2b9e9d700 1 --2- 172.21.15.46:0/454795743 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2b4108740 0x7fe2b41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.345+0000 7fe2b9e9d700 1 --2- 172.21.15.46:0/454795743 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b41034c0 0x7fe2b41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.345+0000 7fe2b9e9d700 1 --2- 172.21.15.46:0/454795743 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b4107df0 0x7fe2b4108200 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.345+0000 7fe2b9e9d700 1 -- 172.21.15.46:0/454795743 >> 172.21.15.46:0/454795743 conn(0x7fe2b40fedf0 msgr2=0x7fe2b4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.345+0000 7fe2b9e9d700 1 -- 172.21.15.46:0/454795743 shutdown_connections 2024-05-09T05:00:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.345+0000 7fe2b9e9d700 1 -- 172.21.15.46:0/454795743 wait complete. 2024-05-09T05:00:57.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.345+0000 7fe2b9e9d700 1 Processor -- start 2024-05-09T05:00:57.347 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.346+0000 7fe2b9e9d700 1 -- start start 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b9e9d700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b41034c0 0x7fe2b419ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b9e9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b4107df0 0x7fe2b419b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b9e9d700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2b4108740 0x7fe2b419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b9e9d700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b4113370 con 0x7fe2b4107df0 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b9e9d700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41131f0 con 0x7fe2b4108740 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b9e9d700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41134f0 con 0x7fe2b41034c0 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b4107df0 0x7fe2b419b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b4107df0 0x7fe2b419b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54140/0 (socket says 172.21.15.46:54140) 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b2ffd700 1 -- 172.21.15.46:0/2418460823 learned_addr learned my addr 172.21.15.46:0/2418460823 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b37fe700 1 --2- 172.21.15.46:0/2418460823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b41034c0 0x7fe2b419ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b3fff700 1 --2- 172.21.15.46:0/2418460823 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2b4108740 0x7fe2b419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b2ffd700 1 -- 172.21.15.46:0/2418460823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b41034c0 msgr2=0x7fe2b419ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b2ffd700 1 --2- 172.21.15.46:0/2418460823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b41034c0 0x7fe2b419ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b2ffd700 1 -- 172.21.15.46:0/2418460823 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2b4108740 msgr2=0x7fe2b419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b2ffd700 1 --2- 172.21.15.46:0/2418460823 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2b4108740 0x7fe2b419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.348 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b2ffd700 1 -- 172.21.15.46:0/2418460823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe2b419fcb0 con 0x7fe2b4107df0 2024-05-09T05:00:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.347+0000 7fe2b2ffd700 1 --2- 172.21.15.46:0/2418460823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b4107df0 0x7fe2b419b430 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80098e0 tx=0x7fe2a8003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.348+0000 7fe2b0ff9700 1 -- 172.21.15.46:0/2418460823 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a8027070 con 0x7fe2b4107df0 2024-05-09T05:00:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.348+0000 7fe2b9e9d700 1 -- 172.21.15.46:0/2418460823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe2b419ff40 con 0x7fe2b4107df0 2024-05-09T05:00:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.348+0000 7fe2b9e9d700 1 -- 172.21.15.46:0/2418460823 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe2b41a0470 con 0x7fe2b4107df0 2024-05-09T05:00:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.348+0000 7fe2b0ff9700 1 -- 172.21.15.46:0/2418460823 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a80040a0 con 0x7fe2b4107df0 2024-05-09T05:00:57.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.348+0000 7fe2b0ff9700 1 -- 172.21.15.46:0/2418460823 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a800f460 con 0x7fe2b4107df0 2024-05-09T05:00:57.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.352+0000 7fe2b0ff9700 1 -- 172.21.15.46:0/2418460823 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe2a802a070 con 0x7fe2b4107df0 2024-05-09T05:00:57.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.352+0000 7fe2c88b3b80 1 Processor -- start 2024-05-09T05:00:57.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.352+0000 7fe2c88b3b80 1 -- start start 2024-05-09T05:00:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.352+0000 7fe2c88b3b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e2fb1e0 0x563d6e394570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.352+0000 7fe2c88b3b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e211520 0x563d6e394ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2c88b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e38e720 0x563d6e38eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2c88b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563d6e2b1070 con 0x563d6e38e720 2024-05-09T05:00:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2c88b3b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x563d6e24f870 con 0x563d6e211520 2024-05-09T05:00:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2c88b3b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563d6e2ad030 con 0x563d6e2fb1e0 2024-05-09T05:00:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2b2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e38e720 0x563d6e38eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.354 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2b37fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e211520 0x563d6e394ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2b37fe700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e211520 0x563d6e394ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:50660/0 (socket says 172.21.15.46:50660) 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2b37fe700 1 -- 172.21.15.46:0/329247254 learned_addr learned my addr 172.21.15.46:0/329247254 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2b0ff9700 1 --2- 172.21.15.46:0/2418460823 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe29c0b29f0 0x7fe29c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2b3fff700 1 --2- 172.21.15.46:0/2418460823 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe29c0b29f0 0x7fe29c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2b0ff9700 1 -- 172.21.15.46:0/2418460823 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7fe2a80e3700 con 0x7fe2b4107df0 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2b37fe700 1 -- 172.21.15.46:0/329247254 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e2fb1e0 msgr2=0x563d6e394570 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2b37fe700 1 --2- 172.21.15.46:0/329247254 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e2fb1e0 0x563d6e394570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2b37fe700 1 -- 172.21.15.46:0/329247254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e38e720 msgr2=0x563d6e38eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2b37fe700 1 --2- 172.21.15.46:0/329247254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e38e720 0x563d6e38eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2b37fe700 1 -- 172.21.15.46:0/329247254 --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6e2473f0 con 0x563d6e211520 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.353+0000 7fe2b2ffd700 1 --2- 172.21.15.46:0/329247254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e38e720 0x563d6e38eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.354+0000 7fe2b37fe700 1 --2- 172.21.15.46:0/329247254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e211520 0x563d6e394ab0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4008c60 tx=0x7fe2a4008f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a52c60c97479c2 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.354+0000 7fe299ffb700 1 -- 172.21.15.46:0/329247254 <== mon.2 v2:172.21.15.72:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4014070 con 0x563d6e211520 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.354+0000 7fe2b3fff700 1 --2- 172.21.15.46:0/2418460823 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe29c0b29f0 0x7fe29c0b4f00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac006130 tx=0x7fe2ac006060 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.354+0000 7fe299ffb700 1 -- 172.21.15.46:0/329247254 <== mon.2 v2:172.21.15.72:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a400ed70 con 0x563d6e211520 2024-05-09T05:00:57.355 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.354+0000 7fe299ffb700 1 -- 172.21.15.46:0/329247254 <== mon.2 v2:172.21.15.72:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4010430 con 0x563d6e211520 2024-05-09T05:00:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.355+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/329247254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e211520 msgr2=0x563d6e394ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.355+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/329247254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e211520 0x563d6e394ab0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4008c60 tx=0x7fe2a4008f70 comp rx=0 tx=0).stop 2024-05-09T05:00:57.356 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.355+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/329247254 shutdown_connections 2024-05-09T05:00:57.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.355+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/329247254 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e38e720 0x563d6e38eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.355+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/329247254 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e2fb1e0 0x563d6e394570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.355+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/329247254 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e211520 0x563d6e394ab0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.357 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.355+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/329247254 >> 172.21.15.46:0/329247254 conn(0x563d6e271720 msgr2=0x563d6e389480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:57.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.368+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/329247254 shutdown_connections 2024-05-09T05:00:57.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.368+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/329247254 wait complete. 2024-05-09T05:00:57.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.368+0000 7fe2c88b3b80 1 -- start start 2024-05-09T05:00:57.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.369+0000 7fe2c88b3b80 1 Processor -- start 2024-05-09T05:00:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.369+0000 7fe2c88b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e211520 0x563d6e398310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.369+0000 7fe2c88b3b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e2fb1e0 0x563d6e398850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.369+0000 7fe2c88b3b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e38e720 0x563d6e398d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.369+0000 7fe2c88b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x563d6e12cc20 con 0x563d6e211520 2024-05-09T05:00:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.369+0000 7fe2c88b3b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x563d6e26b1d0 con 0x563d6e38e720 2024-05-09T05:00:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.369+0000 7fe2c88b3b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x563d6e39b250 con 0x563d6e2fb1e0 2024-05-09T05:00:57.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.370+0000 7fe2b37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e211520 0x563d6e398310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.370+0000 7fe2b37fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e211520 0x563d6e398310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54162/0 (socket says 172.21.15.46:54162) 2024-05-09T05:00:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.370+0000 7fe2b37fe700 1 -- 172.21.15.46:0/4046818527 learned_addr learned my addr 172.21.15.46:0/4046818527 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.370+0000 7fe2b3fff700 1 --2- 172.21.15.46:0/4046818527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e38e720 0x563d6e398d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.370+0000 7fe2b2ffd700 1 --2- 172.21.15.46:0/4046818527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e2fb1e0 0x563d6e398850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.370+0000 7fe2b37fe700 1 -- 172.21.15.46:0/4046818527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e2fb1e0 msgr2=0x563d6e398850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.370+0000 7fe2b37fe700 1 --2- 172.21.15.46:0/4046818527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e2fb1e0 0x563d6e398850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.370+0000 7fe2b37fe700 1 -- 172.21.15.46:0/4046818527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e38e720 msgr2=0x563d6e398d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.370+0000 7fe2b37fe700 1 --2- 172.21.15.46:0/4046818527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e38e720 0x563d6e398d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.370+0000 7fe2b37fe700 1 -- 172.21.15.46:0/4046818527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d6e12c510 con 0x563d6e211520 2024-05-09T05:00:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.370+0000 7fe2b3fff700 1 --2- 172.21.15.46:0/4046818527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e38e720 0x563d6e398d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:00:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.370+0000 7fe2c88b3b80 10 client.0 _ll_register_callbacks cb 0x563d6e122740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.370+0000 7fe2b37fe700 1 --2- 172.21.15.46:0/4046818527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e211520 0x563d6e398310 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4006010 tx=0x7fe2a4010660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfaf8bb9fd4207b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.370+0000 7fe292ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.371+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/4046818527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe2a400ed70 con 0x563d6e211520 2024-05-09T05:00:57.372 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.371+0000 7fe292ffd700 1 -- 172.21.15.46:0/4046818527 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a4027060 con 0x563d6e211520 2024-05-09T05:00:57.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.371+0000 7fe292ffd700 1 -- 172.21.15.46:0/4046818527 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7fe2a4010c90 con 0x563d6e211520 2024-05-09T05:00:57.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.371+0000 7fe292ffd700 1 -- 172.21.15.46:0/4046818527 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe2a4014070 con 0x563d6e211520 2024-05-09T05:00:57.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.371+0000 7fe2c88b3b80 10 client.17475 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:57.373 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.371+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/4046818527 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({fsmap=25}) -- 0x563d6e3a5ae0 con 0x563d6e211520 2024-05-09T05:00:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.372+0000 7fe292ffd700 1 -- 172.21.15.46:0/4046818527 <== mon.0 v2:172.21.15.46:3300/0 4 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fe2a4022460 con 0x563d6e211520 2024-05-09T05:00:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.372+0000 7fe2c88b3b80 10 client.17475 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.372+0000 7fe2c88b3b80 10 client.17475 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.372+0000 7fe2c88b3b80 20 client.17475 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.372+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/4046818527 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563d6e3a7fb0 0x563d6e3aa3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.372+0000 7fe2c88b3b80 4 client.17475 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:00:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.372+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/4046818527 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d6e2bea50 con 0x563d6e3a7fb0 2024-05-09T05:00:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.372+0000 7fe2b2ffd700 1 --2- 172.21.15.46:0/4046818527 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563d6e3a7fb0 0x563d6e3aa3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.373+0000 7fe292ffd700 10 client.17475 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.373+0000 7fe2b2ffd700 1 --2- 172.21.15.46:0/4046818527 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563d6e3a7fb0 0x563d6e3aa3d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaeb8a98f00517bd server_cookie=d0b76d0a31d50978 in_seq=0 out_seq=0 2024-05-09T05:00:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.373+0000 7fe292ffd700 1 -- 172.21.15.46:0/4046818527 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563d6e2bea50 con 0x563d6e3a7fb0 2024-05-09T05:00:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.373+0000 7fe292ffd700 10 client.17475 handle_command_reply: tid=0 2024-05-09T05:00:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.373+0000 7fe2c88b3b80 1 client.17475 shutdown 2024-05-09T05:00:57.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.374+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/4046818527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e211520 msgr2=0x563d6e398310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.374+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/4046818527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e211520 0x563d6e398310 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fe2a4006010 tx=0x7fe2a4010660 comp rx=0 tx=0).stop 2024-05-09T05:00:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.374+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/4046818527 shutdown_connections 2024-05-09T05:00:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.374+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/4046818527 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563d6e3a7fb0 0x563d6e3aa3d0 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.374+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/4046818527 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e38e720 0x563d6e398d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.374+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/4046818527 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e211520 0x563d6e398310 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.374+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/4046818527 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e2fb1e0 0x563d6e398850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.374+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/4046818527 >> 172.21.15.46:0/4046818527 conn(0x563d6e271720 msgr2=0x563d6e389240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.374+0000 7fe292ffd700 0 client.17475 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.374+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/4046818527 shutdown_connections 2024-05-09T05:00:57.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.374+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/4046818527 wait complete. 2024-05-09T05:00:57.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.383+0000 7fe2c88b3b80 20 client.17475 trim_cache size 0 max 16384 2024-05-09T05:00:57.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.403+0000 7fe2c88b3b80 1 Processor -- start 2024-05-09T05:00:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.403+0000 7fe2c88b3b80 1 -- start start 2024-05-09T05:00:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.403+0000 7fe2c88b3b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e211520 0x563d6e3928b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.403+0000 7fe2c88b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e2fb1e0 0x563d6e397270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.403+0000 7fe2c88b3b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e38e720 0x563d6e398bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.403+0000 7fe2c88b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41b3a00 con 0x563d6e2fb1e0 2024-05-09T05:00:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.403+0000 7fe2c88b3b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41b1290 con 0x563d6e211520 2024-05-09T05:00:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.403+0000 7fe2c88b3b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41af120 con 0x563d6e38e720 2024-05-09T05:00:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.404+0000 7fe2b2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e2fb1e0 0x563d6e397270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.404+0000 7fe2b2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e2fb1e0 0x563d6e397270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54174/0 (socket says 172.21.15.46:54174) 2024-05-09T05:00:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.404+0000 7fe2b2ffd700 1 -- 172.21.15.46:0/3536918950 learned_addr learned my addr 172.21.15.46:0/3536918950 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.404+0000 7fe2b37fe700 1 --2- 172.21.15.46:0/3536918950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e211520 0x563d6e3928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.404+0000 7fe2b3fff700 1 --2- 172.21.15.46:0/3536918950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e38e720 0x563d6e398bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.404+0000 7fe2b2ffd700 1 -- 172.21.15.46:0/3536918950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e38e720 msgr2=0x563d6e398bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.404+0000 7fe2b2ffd700 1 --2- 172.21.15.46:0/3536918950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e38e720 0x563d6e398bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.404+0000 7fe2b2ffd700 1 -- 172.21.15.46:0/3536918950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e211520 msgr2=0x563d6e3928b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.404+0000 7fe2b2ffd700 1 --2- 172.21.15.46:0/3536918950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e211520 0x563d6e3928b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.404+0000 7fe2b2ffd700 1 -- 172.21.15.46:0/3536918950 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d6e38afb0 con 0x563d6e2fb1e0 2024-05-09T05:00:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.404+0000 7fe2b3fff700 1 --2- 172.21.15.46:0/3536918950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e38e720 0x563d6e398bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:00:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.404+0000 7fe2b2ffd700 1 --2- 172.21.15.46:0/3536918950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e2fb1e0 0x563d6e397270 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8005560 tx=0x7fe2a8009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d28aa27dcd4592f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.404+0000 7fe2997fa700 1 -- 172.21.15.46:0/3536918950 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80a6150 con 0x563d6e2fb1e0 2024-05-09T05:00:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.404+0000 7fe2997fa700 1 -- 172.21.15.46:0/3536918950 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe2a80b30d0 con 0x563d6e2fb1e0 2024-05-09T05:00:57.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.405+0000 7fe2997fa700 1 -- 172.21.15.46:0/3536918950 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2a80aed70 con 0x563d6e2fb1e0 2024-05-09T05:00:57.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.406+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/3536918950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e2fb1e0 msgr2=0x563d6e397270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:57.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.406+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/3536918950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e2fb1e0 0x563d6e397270 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8005560 tx=0x7fe2a8009390 comp rx=0 tx=0).stop 2024-05-09T05:00:57.409 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.406+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/3536918950 shutdown_connections 2024-05-09T05:00:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.406+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/3536918950 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e38e720 0x563d6e398bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.406+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/3536918950 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e211520 0x563d6e3928b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.406+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/3536918950 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e2fb1e0 0x563d6e397270 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.406+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/3536918950 >> 172.21.15.46:0/3536918950 conn(0x563d6e271720 msgr2=0x563d6e38d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.406+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/3536918950 shutdown_connections 2024-05-09T05:00:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.406+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/3536918950 wait complete. 2024-05-09T05:00:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.407+0000 7fe2c88b3b80 1 -- start start 2024-05-09T05:00:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.407+0000 7fe2c88b3b80 1 Processor -- start 2024-05-09T05:00:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.407+0000 7fe2c88b3b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e211520 0x563d6e3ad530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.407+0000 7fe2c88b3b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e2fb1e0 0x563d6e3ab910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.407+0000 7fe2c88b3b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e38e720 0x563d6e3abe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.407+0000 7fe2c88b3b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41af770 con 0x563d6e38e720 2024-05-09T05:00:57.410 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.407+0000 7fe2c88b3b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41b0bc0 con 0x563d6e2fb1e0 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.407+0000 7fe2c88b3b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fe2b41afe20 con 0x563d6e211520 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.407+0000 7fe2c88b3b80 10 client.0 _ll_register_callbacks cb 0x563d6e393f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.407+0000 7fe2b2ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e2fb1e0 0x563d6e3ab910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.407+0000 7fe2b3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e38e720 0x563d6e3abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.407+0000 7fe2b2ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e2fb1e0 0x563d6e3ab910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:50684/0 (socket says 172.21.15.46:50684) 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.407+0000 7fe2b2ffd700 1 -- 172.21.15.46:0/3180015426 learned_addr learned my addr 172.21.15.46:0/3180015426 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.407+0000 7fe2b37fe700 1 --2- 172.21.15.46:0/3180015426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e211520 0x563d6e3ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.408+0000 7fe2b3fff700 1 -- 172.21.15.46:0/3180015426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e211520 msgr2=0x563d6e3ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.408+0000 7fe2b3fff700 1 --2- 172.21.15.46:0/3180015426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e211520 0x563d6e3ad530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.408+0000 7fe2b3fff700 1 -- 172.21.15.46:0/3180015426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e2fb1e0 msgr2=0x563d6e3ab910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.408+0000 7fe2b3fff700 1 --2- 172.21.15.46:0/3180015426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e2fb1e0 0x563d6e3ab910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.408+0000 7fe2b3fff700 1 -- 172.21.15.46:0/3180015426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d6e382fe0 con 0x563d6e38e720 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.408+0000 7fe2b3fff700 1 --2- 172.21.15.46:0/3180015426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e38e720 0x563d6e3abe50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac008bd0 tx=0x7fe2ac00c050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37be79e2959828e server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.408+0000 7fe292ffd700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.408+0000 7fe292ffd700 1 -- 172.21.15.46:0/3180015426 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac004bb0 con 0x563d6e38e720 2024-05-09T05:00:57.411 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.408+0000 7fe292ffd700 1 -- 172.21.15.46:0/3180015426 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7fe2ac004d50 con 0x563d6e38e720 2024-05-09T05:00:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.408+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/3180015426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563d6e2b8af0 con 0x563d6e38e720 2024-05-09T05:00:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.408+0000 7fe292ffd700 1 -- 172.21.15.46:0/3180015426 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fe2ac02e050 con 0x563d6e38e720 2024-05-09T05:00:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.408+0000 7fe2c88b3b80 10 client.17481 fetch_fsmap learned FSMap version 25 2024-05-09T05:00:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.409+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/3180015426 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x563d6e413210 con 0x563d6e38e720 2024-05-09T05:00:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.409+0000 7fe292ffd700 1 -- 172.21.15.46:0/3180015426 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x563d6e2b8af0 con 0x563d6e38e720 2024-05-09T05:00:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.409+0000 7fe292ffd700 1 -- 172.21.15.46:0/3180015426 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fe2ac021a20 con 0x563d6e38e720 2024-05-09T05:00:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.409+0000 7fe2c88b3b80 10 client.17481 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:00:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.409+0000 7fe2c88b3b80 10 client.17481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:00:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.409+0000 7fe2c88b3b80 20 client.17481 populate_metadata read hostname 'smithi046' 2024-05-09T05:00:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.409+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/3180015426 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563d6e413910 0x563d6e415d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:00:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.409+0000 7fe2c88b3b80 4 client.17481 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-05-09T05:00:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.409+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/3180015426 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563d6e12c9f0 con 0x563d6e413910 2024-05-09T05:00:57.412 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.409+0000 7fe2b37fe700 1 --2- 172.21.15.46:0/3180015426 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563d6e413910 0x563d6e415d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:00:57.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.410+0000 7fe2b37fe700 1 --2- 172.21.15.46:0/3180015426 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563d6e413910 0x563d6e415d30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fcb91e8dc84d7ce server_cookie=1c2f979900557bca in_seq=0 out_seq=0 2024-05-09T05:00:57.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.410+0000 7fe292ffd700 10 client.17481 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:57.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.411+0000 7fe292ffd700 1 -- 172.21.15.46:0/3180015426 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563d6e12c9f0 con 0x563d6e413910 2024-05-09T05:00:57.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.411+0000 7fe292ffd700 10 client.17481 handle_command_reply: tid=0 2024-05-09T05:00:57.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.411+0000 7fe2c88b3b80 1 client.17481 shutdown 2024-05-09T05:00:57.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.411+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/3180015426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e38e720 msgr2=0x563d6e3abe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:57.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.411+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/3180015426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e38e720 0x563d6e3abe50 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac008bd0 tx=0x7fe2ac00c050 comp rx=0 tx=0).stop 2024-05-09T05:00:57.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.411+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/3180015426 shutdown_connections 2024-05-09T05:00:57.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.411+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/3180015426 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x563d6e38e720 0x563d6e3abe50 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.411+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/3180015426 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x563d6e211520 0x563d6e3ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.411+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/3180015426 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x563d6e413910 0x563d6e415d30 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.411+0000 7fe2c88b3b80 1 --2- 172.21.15.46:0/3180015426 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x563d6e2fb1e0 0x563d6e3ab910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.414 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.411+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/3180015426 >> 172.21.15.46:0/3180015426 conn(0x563d6e271720 msgr2=0x563d6e392610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:57.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.411+0000 7fe292ffd700 0 client.17481 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:00:57.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.411+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/3180015426 shutdown_connections 2024-05-09T05:00:57.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.411+0000 7fe2c88b3b80 1 -- 172.21.15.46:0/3180015426 wait complete. 2024-05-09T05:00:57.415 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.412+0000 7fe2c88b3b80 20 client.17481 trim_cache size 0 max 16384 2024-05-09T05:00:57.415 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T05:00:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "return_code": 0, 2024-05-09T05:00:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "scrub_tag": "7ca5e20e-fcf9-4d0e-8968-31e1700d5925", 2024-05-09T05:00:57.415 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": "asynchronous" 2024-05-09T05:00:57.415 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T05:00:57.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.412+0000 7fe2b9e9d700 1 -- 172.21.15.46:0/2418460823 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe29c0b29f0 msgr2=0x7fe29c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:57.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.412+0000 7fe2b9e9d700 1 --2- 172.21.15.46:0/2418460823 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe29c0b29f0 0x7fe29c0b4f00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac006130 tx=0x7fe2ac006060 comp rx=0 tx=0).stop 2024-05-09T05:00:57.416 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.412+0000 7fe2b9e9d700 1 -- 172.21.15.46:0/2418460823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b4107df0 msgr2=0x7fe2b419b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:00:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.412+0000 7fe2b9e9d700 1 --2- 172.21.15.46:0/2418460823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b4107df0 0x7fe2b419b430 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80098e0 tx=0x7fe2a8003680 comp rx=0 tx=0).stop 2024-05-09T05:00:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.412+0000 7fe2b9e9d700 1 -- 172.21.15.46:0/2418460823 shutdown_connections 2024-05-09T05:00:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.412+0000 7fe2b9e9d700 1 --2- 172.21.15.46:0/2418460823 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fe29c0b29f0 0x7fe29c0b4f00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.412+0000 7fe2b9e9d700 1 --2- 172.21.15.46:0/2418460823 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fe2b4108740 0x7fe2b419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.412+0000 7fe2b9e9d700 1 --2- 172.21.15.46:0/2418460823 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fe2b41034c0 0x7fe2b419ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:00:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.412+0000 7fe2b9e9d700 1 --2- 172.21.15.46:0/2418460823 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fe2b4107df0 0x7fe2b419b430 secure :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fe2a80098e0 tx=0x7fe2a8003680 comp rx=0 tx=0).stop 2024-05-09T05:00:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.412+0000 7fe2b9e9d700 1 -- 172.21.15.46:0/2418460823 >> 172.21.15.46:0/2418460823 conn(0x7fe2b40fedf0 msgr2=0x7fe2b4075d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:00:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.413+0000 7fe2b9e9d700 1 -- 172.21.15.46:0/2418460823 shutdown_connections 2024-05-09T05:00:57.417 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:00:57.413+0000 7fe2b9e9d700 1 -- 172.21.15.46:0/2418460823 wait complete. 2024-05-09T05:00:58.218 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:00:58.475 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:00:58.849 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:58 smithi046 ceph-mon[30136]: scrub queued for path: / 2024-05-09T05:00:58.849 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:58 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:58.849 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:58 smithi046 ceph-mon[30136]: scrub summary: active paths [/] 2024-05-09T05:00:58.849 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:58 smithi046 ceph-mon[30136]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:58.849 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:00:58 smithi046 ceph-mon[30136]: pgmap v305: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:00:59.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:58 smithi072 ceph-mon[34088]: scrub queued for path: / 2024-05-09T05:00:59.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:58 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:59.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:58 smithi072 ceph-mon[34088]: scrub summary: active paths [/] 2024-05-09T05:00:59.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:58 smithi072 ceph-mon[34088]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:59.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:00:58 smithi072 ceph-mon[34088]: pgmap v305: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:00:59.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:58 smithi110 ceph-mon[34904]: scrub queued for path: / 2024-05-09T05:00:59.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:58 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:59.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:58 smithi110 ceph-mon[34904]: scrub summary: active paths [/] 2024-05-09T05:00:59.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:58 smithi110 ceph-mon[34904]: scrub summary: idle+waiting paths [/] 2024-05-09T05:00:59.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:00:58 smithi110 ceph-mon[34904]: pgmap v305: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:01:00.334 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.334+0000 7efdfc7bb700 1 -- 172.21.15.46:0/1907209240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdf4108740 msgr2=0x7efdf41100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.334+0000 7efdfc7bb700 1 --2- 172.21.15.46:0/1907209240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdf4108740 0x7efdf41100a0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7efdec00b0d0 tx=0x7efdec00b3e0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.334+0000 7efdfc7bb700 1 -- 172.21.15.46:0/1907209240 shutdown_connections 2024-05-09T05:01:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.334+0000 7efdfc7bb700 1 --2- 172.21.15.46:0/1907209240 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdf4108740 0x7efdf41100a0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.334+0000 7efdfc7bb700 1 --2- 172.21.15.46:0/1907209240 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdf41034c0 0x7efdf41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.334+0000 7efdfc7bb700 1 --2- 172.21.15.46:0/1907209240 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efdf4107df0 0x7efdf4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.334+0000 7efdfc7bb700 1 -- 172.21.15.46:0/1907209240 >> 172.21.15.46:0/1907209240 conn(0x7efdf40fedf0 msgr2=0x7efdf4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.334+0000 7efdfc7bb700 1 -- 172.21.15.46:0/1907209240 shutdown_connections 2024-05-09T05:01:00.335 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.334+0000 7efdfc7bb700 1 -- 172.21.15.46:0/1907209240 wait complete. 2024-05-09T05:01:00.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.335+0000 7efdfc7bb700 1 Processor -- start 2024-05-09T05:01:00.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.335+0000 7efdfc7bb700 1 -- start start 2024-05-09T05:01:00.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.335+0000 7efdfc7bb700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efdf41034c0 0x7efdf419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.335+0000 7efdfc7bb700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdf4107df0 0x7efdf419b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.335+0000 7efdfc7bb700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdf4108740 0x7efdf419f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.337 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.335+0000 7efdfc7bb700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efdf41144c0 con 0x7efdf4108740 2024-05-09T05:01:00.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.335+0000 7efdfc7bb700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7efdf4114340 con 0x7efdf41034c0 2024-05-09T05:01:00.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.335+0000 7efdfc7bb700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efdf4114640 con 0x7efdf4107df0 2024-05-09T05:01:00.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.335+0000 7efdfad58700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdf4108740 0x7efdf419f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.335+0000 7efdfad58700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdf4108740 0x7efdf419f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54198/0 (socket says 172.21.15.46:54198) 2024-05-09T05:01:00.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.335+0000 7efdfad58700 1 -- 172.21.15.46:0/1610424875 learned_addr learned my addr 172.21.15.46:0/1610424875 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:00.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.336+0000 7efdf9d56700 1 --2- 172.21.15.46:0/1610424875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdf4107df0 0x7efdf419b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.336+0000 7efdfa557700 1 --2- 172.21.15.46:0/1610424875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efdf41034c0 0x7efdf419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.336+0000 7efdfad58700 1 -- 172.21.15.46:0/1610424875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdf4107df0 msgr2=0x7efdf419b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:00.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.336+0000 7efdfad58700 1 --2- 172.21.15.46:0/1610424875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdf4107df0 0x7efdf419b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.336+0000 7efdfad58700 1 -- 172.21.15.46:0/1610424875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efdf41034c0 msgr2=0x7efdf419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:00.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.336+0000 7efdfad58700 1 --2- 172.21.15.46:0/1610424875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efdf41034c0 0x7efdf419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.338 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.336+0000 7efdfad58700 1 -- 172.21.15.46:0/1610424875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdf419fcc0 con 0x7efdf4108740 2024-05-09T05:01:00.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.336+0000 7efdfad58700 1 --2- 172.21.15.46:0/1610424875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdf4108740 0x7efdf419f620 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7efdec0095c0 tx=0x7efdec007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:00.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.336+0000 7efdeb7fe700 1 -- 172.21.15.46:0/1610424875 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec01f070 con 0x7efdf4108740 2024-05-09T05:01:00.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.336+0000 7efdeb7fe700 1 -- 172.21.15.46:0/1610424875 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdec007e20 con 0x7efdf4108740 2024-05-09T05:01:00.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.336+0000 7efdfc7bb700 1 -- 172.21.15.46:0/1610424875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdf419ff50 con 0x7efdf4108740 2024-05-09T05:01:00.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.336+0000 7efdeb7fe700 1 -- 172.21.15.46:0/1610424875 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec01b4a0 con 0x7efdf4108740 2024-05-09T05:01:00.339 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.336+0000 7efdfc7bb700 1 -- 172.21.15.46:0/1610424875 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdf41a0430 con 0x7efdf4108740 2024-05-09T05:01:00.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.339+0000 7efdeb7fe700 1 -- 172.21.15.46:0/1610424875 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efdec022070 con 0x7efdf4108740 2024-05-09T05:01:00.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.339+0000 7efdeb7fe700 1 --2- 172.21.15.46:0/1610424875 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7efde00b29f0 0x7efde00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.340+0000 7efdeb7fe700 1 -- 172.21.15.46:0/1610424875 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7efdec00f020 con 0x7efdf4108740 2024-05-09T05:01:00.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.340+0000 7efdfa557700 1 --2- 172.21.15.46:0/1610424875 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7efde00b29f0 0x7efde00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.340 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.340+0000 7efe0b1d1b80 1 Processor -- start 2024-05-09T05:01:00.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.340+0000 7efdfa557700 1 --2- 172.21.15.46:0/1610424875 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7efde00b29f0 0x7efde00b4f00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efdf0004630 tx=0x7efdf000b3c0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.340+0000 7efe0b1d1b80 1 -- start start 2024-05-09T05:01:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.340+0000 7efe0b1d1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d1c0d00 0x55b75d2cd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.340+0000 7efe0b1d1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.340+0000 7efe0b1d1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d2c7810 0x55b75d2c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efe0b1d1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b75d1e60a0 con 0x55b75d1464a0 2024-05-09T05:01:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efe0b1d1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55b75d18b9a0 con 0x55b75d2c7810 2024-05-09T05:01:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efe0b1d1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b75d1e2060 con 0x55b75d1c0d00 2024-05-09T05:01:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efdf9d56700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efdf9d56700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2cdb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54200/0 (socket says 172.21.15.46:54200) 2024-05-09T05:01:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efdf9d56700 1 -- 172.21.15.46:0/521624971 learned_addr learned my addr 172.21.15.46:0/521624971 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efdfa557700 1 --2- 172.21.15.46:0/521624971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d1c0d00 0x55b75d2cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efdfad58700 1 --2- 172.21.15.46:0/521624971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d2c7810 0x55b75d2c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.343 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efdf9d56700 1 -- 172.21.15.46:0/521624971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d1c0d00 msgr2=0x55b75d2cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efdf9d56700 1 --2- 172.21.15.46:0/521624971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d1c0d00 0x55b75d2cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efdf9d56700 1 -- 172.21.15.46:0/521624971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d2c7810 msgr2=0x55b75d2c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efdf9d56700 1 --2- 172.21.15.46:0/521624971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d2c7810 0x55b75d2c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efdf9d56700 1 -- 172.21.15.46:0/521624971 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b75d17c500 con 0x55b75d1464a0 2024-05-09T05:01:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efdf9d56700 1 --2- 172.21.15.46:0/521624971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2cdb90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7efde4007be0 tx=0x7efde4007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65eef5fabef3fab6 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efde8ff9700 1 -- 172.21.15.46:0/521624971 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4004310 con 0x55b75d1464a0 2024-05-09T05:01:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efde8ff9700 1 -- 172.21.15.46:0/521624971 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde4015070 con 0x55b75d1464a0 2024-05-09T05:01:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.341+0000 7efde8ff9700 1 -- 172.21.15.46:0/521624971 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efde4003d20 con 0x55b75d1464a0 2024-05-09T05:01:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.342+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/521624971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 msgr2=0x55b75d2cdb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.342+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/521624971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2cdb90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7efde4007be0 tx=0x7efde4007ef0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.342+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/521624971 shutdown_connections 2024-05-09T05:01:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.342+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/521624971 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d2c7810 0x55b75d2c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.342+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/521624971 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d1c0d00 0x55b75d2cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.344 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.342+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/521624971 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2cdb90 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.342+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/521624971 >> 172.21.15.46:0/521624971 conn(0x55b75d149f70 msgr2=0x55b75d2be220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:00.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.343+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/521624971 shutdown_connections 2024-05-09T05:01:00.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.343+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/521624971 wait complete. 2024-05-09T05:01:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.343+0000 7efe0b1d1b80 1 -- start start 2024-05-09T05:01:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.343+0000 7efe0b1d1b80 1 Processor -- start 2024-05-09T05:01:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.343+0000 7efe0b1d1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2d1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.349 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.343+0000 7efe0b1d1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d1c0d00 0x55b75d2d18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.343+0000 7efe0b1d1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d2c7810 0x55b75d2d1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.343+0000 7efe0b1d1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55b75d2c8570 con 0x55b75d1464a0 2024-05-09T05:01:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.343+0000 7efe0b1d1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55b75d180b20 con 0x55b75d2c7810 2024-05-09T05:01:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.343+0000 7efe0b1d1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55b75d2d4430 con 0x55b75d1c0d00 2024-05-09T05:01:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.343+0000 7efe0b1d1b80 10 client.0 _ll_register_callbacks cb 0x55b75d0576c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:01:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.343+0000 7efdfa557700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2d1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efdfa557700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2d1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54210/0 (socket says 172.21.15.46:54210) 2024-05-09T05:01:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efdfa557700 1 -- 172.21.15.46:0/1120609670 learned_addr learned my addr 172.21.15.46:0/1120609670 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efdfad58700 1 --2- 172.21.15.46:0/1120609670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d2c7810 0x55b75d2d1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efdf9d56700 1 --2- 172.21.15.46:0/1120609670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d1c0d00 0x55b75d2d18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efdfa557700 1 -- 172.21.15.46:0/1120609670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d1c0d00 msgr2=0x55b75d2d18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efdfa557700 1 --2- 172.21.15.46:0/1120609670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d1c0d00 0x55b75d2d18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.350 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efdfa557700 1 -- 172.21.15.46:0/1120609670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d2c7810 msgr2=0x55b75d2d1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efdfa557700 1 --2- 172.21.15.46:0/1120609670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d2c7810 0x55b75d2d1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efdfa557700 1 -- 172.21.15.46:0/1120609670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b75d061490 con 0x55b75d1464a0 2024-05-09T05:01:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efdfa557700 1 --2- 172.21.15.46:0/1120609670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2d1370 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7efdf0003f40 tx=0x7efdf00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50f3873c049b2ba server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efdddffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:01:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efdddffb700 1 -- 172.21.15.46:0/1120609670 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0011ce0 con 0x55b75d1464a0 2024-05-09T05:01:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efdddffb700 1 -- 172.21.15.46:0/1120609670 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7efdf0028470 con 0x55b75d1464a0 2024-05-09T05:01:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/1120609670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efde4015070 con 0x55b75d1464a0 2024-05-09T05:01:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efdddffb700 1 -- 172.21.15.46:0/1120609670 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efdf00286b0 con 0x55b75d1464a0 2024-05-09T05:01:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efe0b1d1b80 10 client.17493 fetch_fsmap learned FSMap version 25 2024-05-09T05:01:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.344+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/1120609670 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b75d2deda0 con 0x55b75d1464a0 2024-05-09T05:01:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.345+0000 7efdddffb700 1 -- 172.21.15.46:0/1120609670 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efde4015070 con 0x55b75d1464a0 2024-05-09T05:01:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.345+0000 7efdddffb700 1 -- 172.21.15.46:0/1120609670 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7efdf001f820 con 0x55b75d1464a0 2024-05-09T05:01:00.351 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.345+0000 7efe0b1d1b80 10 client.17493 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.345+0000 7efe0b1d1b80 10 client.17493 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.345+0000 7efe0b1d1b80 20 client.17493 populate_metadata read hostname 'smithi046' 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.345+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/1120609670 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b75d2e1430 0x55b75d2e3850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.345+0000 7efe0b1d1b80 4 client.17493 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.345+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/1120609670 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b75d1f3a80 con 0x55b75d2e1430 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.345+0000 7efdf9d56700 1 --2- 172.21.15.46:0/1120609670 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b75d2e1430 0x55b75d2e3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.346+0000 7efdf9d56700 1 --2- 172.21.15.46:0/1120609670 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b75d2e1430 0x55b75d2e3850 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79b4fd394be76435 server_cookie=e23946e433d8e417 in_seq=0 out_seq=0 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.346+0000 7efdddffb700 10 client.17493 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.346+0000 7efdddffb700 1 -- 172.21.15.46:0/1120609670 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b75d1f3a80 con 0x55b75d2e1430 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.346+0000 7efdddffb700 10 client.17493 handle_command_reply: tid=0 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.347+0000 7efe0b1d1b80 1 client.17493 shutdown 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.347+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/1120609670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 msgr2=0x55b75d2d1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.347+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/1120609670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2d1370 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7efdf0003f40 tx=0x7efdf00047c0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.347+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/1120609670 shutdown_connections 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.347+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/1120609670 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b75d2e1430 0x55b75d2e3850 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.352 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.347+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/1120609670 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d2c7810 0x55b75d2d1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.347+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/1120609670 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2d1370 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.347+0000 7efdddffb700 0 client.17493 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:01:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.347+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/1120609670 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d1c0d00 0x55b75d2d18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.347+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/1120609670 >> 172.21.15.46:0/1120609670 conn(0x55b75d149f70 msgr2=0x55b75d2b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.347+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/1120609670 shutdown_connections 2024-05-09T05:01:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.347+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/1120609670 wait complete. 2024-05-09T05:01:00.353 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.347+0000 7efe0b1d1b80 20 client.17493 trim_cache size 0 max 16384 2024-05-09T05:01:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efe0b1d1b80 1 Processor -- start 2024-05-09T05:01:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efe0b1d1b80 1 -- start start 2024-05-09T05:01:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efe0b1d1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2d0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efe0b1d1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d1c0d00 0x55b75d2be350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efe0b1d1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d2c7810 0x55b75d2d1c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efe0b1d1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efdf41af620 con 0x55b75d1464a0 2024-05-09T05:01:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efe0b1d1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7efdf41aceb0 con 0x55b75d1c0d00 2024-05-09T05:01:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efe0b1d1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efdf41aad20 con 0x55b75d2c7810 2024-05-09T05:01:00.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efdfa557700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2d0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efdfa557700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2d0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54222/0 (socket says 172.21.15.46:54222) 2024-05-09T05:01:00.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efdfa557700 1 -- 172.21.15.46:0/4294139532 learned_addr learned my addr 172.21.15.46:0/4294139532 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:00.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efdfad58700 1 --2- 172.21.15.46:0/4294139532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d2c7810 0x55b75d2d1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efdf9d56700 1 --2- 172.21.15.46:0/4294139532 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d1c0d00 0x55b75d2be350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efdfa557700 1 -- 172.21.15.46:0/4294139532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d2c7810 msgr2=0x55b75d2d1c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:00.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efdfa557700 1 --2- 172.21.15.46:0/4294139532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d2c7810 0x55b75d2d1c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efdfa557700 1 -- 172.21.15.46:0/4294139532 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d1c0d00 msgr2=0x55b75d2be350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:00.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efdfa557700 1 --2- 172.21.15.46:0/4294139532 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d1c0d00 0x55b75d2be350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efdfa557700 1 -- 172.21.15.46:0/4294139532 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b75d2cd050 con 0x55b75d1464a0 2024-05-09T05:01:00.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.368+0000 7efdf9d56700 1 --2- 172.21.15.46:0/4294139532 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d1c0d00 0x55b75d2be350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:01:00.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.369+0000 7efdfa557700 1 --2- 172.21.15.46:0/4294139532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2d0690 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7efdf0005cf0 tx=0x7efdf001e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dca7c525a27e036 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:00.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.369+0000 7efddffff700 1 -- 172.21.15.46:0/4294139532 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf003d070 con 0x55b75d1464a0 2024-05-09T05:01:00.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.369+0000 7efddffff700 1 -- 172.21.15.46:0/4294139532 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdf0023070 con 0x55b75d1464a0 2024-05-09T05:01:00.375 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.369+0000 7efddffff700 1 -- 172.21.15.46:0/4294139532 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdf0034040 con 0x55b75d1464a0 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.370+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/4294139532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 msgr2=0x55b75d2d0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.370+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/4294139532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2d0690 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7efdf0005cf0 tx=0x7efdf001e4b0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.370+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/4294139532 shutdown_connections 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.370+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/4294139532 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d2c7810 0x55b75d2d1c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.370+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/4294139532 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d1464a0 0x55b75d2d0690 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.370+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/4294139532 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d1c0d00 0x55b75d2be350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.370+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/4294139532 >> 172.21.15.46:0/4294139532 conn(0x55b75d149f70 msgr2=0x55b75d2cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.370+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/4294139532 shutdown_connections 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.370+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/4294139532 wait complete. 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.370+0000 7efe0b1d1b80 1 -- start start 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.370+0000 7efe0b1d1b80 1 Processor -- start 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efe0b1d1b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d1464a0 0x55b75d2e8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efe0b1d1b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d1c0d00 0x55b75d2e8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efe0b1d1b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d2c7810 0x55b75d2e9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efe0b1d1b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7efdf41ab370 con 0x55b75d2c7810 2024-05-09T05:01:00.376 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efe0b1d1b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7efdf41ac7e0 con 0x55b75d1c0d00 2024-05-09T05:01:00.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efe0b1d1b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7efdf41aba40 con 0x55b75d1464a0 2024-05-09T05:01:00.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efe0b1d1b80 10 client.0 _ll_register_callbacks cb 0x55b75d0576c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:01:00.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efdfad58700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d2c7810 0x55b75d2e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efdfad58700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d2c7810 0x55b75d2e9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54230/0 (socket says 172.21.15.46:54230) 2024-05-09T05:01:00.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efdfad58700 1 -- 172.21.15.46:0/2825295311 learned_addr learned my addr 172.21.15.46:0/2825295311 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:00.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efdf9d56700 1 --2- 172.21.15.46:0/2825295311 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d1c0d00 0x55b75d2e8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efdfa557700 1 --2- 172.21.15.46:0/2825295311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d1464a0 0x55b75d2e8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efdfad58700 1 -- 172.21.15.46:0/2825295311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d1464a0 msgr2=0x55b75d2e8800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:00.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efdfad58700 1 --2- 172.21.15.46:0/2825295311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d1464a0 0x55b75d2e8800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efdfad58700 1 -- 172.21.15.46:0/2825295311 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d1c0d00 msgr2=0x55b75d2e8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:00.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efdfad58700 1 --2- 172.21.15.46:0/2825295311 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d1c0d00 0x55b75d2e8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efdfad58700 1 -- 172.21.15.46:0/2825295311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b75d34ac00 con 0x55b75d2c7810 2024-05-09T05:01:00.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efdfad58700 1 --2- 172.21.15.46:0/2825295311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d2c7810 0x55b75d2e9280 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7efdec0e1550 tx=0x7efdec0a6b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4448c892fbdc259 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:00.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efdddffb700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:01:00.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.371+0000 7efdddffb700 1 -- 172.21.15.46:0/2825295311 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdec0b3280 con 0x55b75d2c7810 2024-05-09T05:01:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.372+0000 7efdddffb700 1 -- 172.21.15.46:0/2825295311 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 8600+0+0 (secure 0 0 0) 0x7efdec0a7160 con 0x55b75d2c7810 2024-05-09T05:01:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.372+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/2825295311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdf0023070 con 0x55b75d2c7810 2024-05-09T05:01:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.372+0000 7efdddffb700 1 -- 172.21.15.46:0/2825295311 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7efdec0aa680 con 0x55b75d2c7810 2024-05-09T05:01:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.372+0000 7efe0b1d1b80 10 client.17499 fetch_fsmap learned FSMap version 25 2024-05-09T05:01:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.372+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/2825295311 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55b75d34bca0 con 0x55b75d2c7810 2024-05-09T05:01:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.372+0000 7efdddffb700 1 -- 172.21.15.46:0/2825295311 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efdf0023070 con 0x55b75d2c7810 2024-05-09T05:01:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.372+0000 7efdddffb700 1 -- 172.21.15.46:0/2825295311 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7efdec011070 con 0x55b75d2c7810 2024-05-09T05:01:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.372+0000 7efe0b1d1b80 10 client.17499 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:01:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.372+0000 7efe0b1d1b80 10 client.17499 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:01:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.372+0000 7efe0b1d1b80 20 client.17499 populate_metadata read hostname 'smithi046' 2024-05-09T05:01:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.372+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/2825295311 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b75d2e1430 0x55b75d34e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.372+0000 7efe0b1d1b80 4 client.17499 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:01:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.372+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/2825295311 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b75d1edb20 con 0x55b75d2e1430 2024-05-09T05:01:00.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.372+0000 7efdfa557700 1 --2- 172.21.15.46:0/2825295311 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b75d2e1430 0x55b75d34e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efdfa557700 1 --2- 172.21.15.46:0/2825295311 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b75d2e1430 0x55b75d34e340 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fe4e90bfb6d35e5 server_cookie=ea0c1276d4f3bbf6 in_seq=0 out_seq=0 2024-05-09T05:01:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efdddffb700 10 client.17499 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:01:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efdddffb700 1 -- 172.21.15.46:0/2825295311 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b75d1edb20 con 0x55b75d2e1430 2024-05-09T05:01:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efdddffb700 10 client.17499 handle_command_reply: tid=0 2024-05-09T05:01:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efe0b1d1b80 1 client.17499 shutdown 2024-05-09T05:01:00.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/2825295311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d2c7810 msgr2=0x55b75d2e9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/2825295311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d2c7810 0x55b75d2e9280 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7efdec0e1550 tx=0x7efdec0a6b30 comp rx=0 tx=0).stop 2024-05-09T05:01:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/2825295311 shutdown_connections 2024-05-09T05:01:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/2825295311 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55b75d2c7810 0x55b75d2e9280 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/2825295311 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55b75d1464a0 0x55b75d2e8800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/2825295311 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55b75d2e1430 0x55b75d34e340 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efe0b1d1b80 1 --2- 172.21.15.46:0/2825295311 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55b75d1c0d00 0x55b75d2e8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efdddffb700 0 client.17499 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:01:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/2825295311 >> 172.21.15.46:0/2825295311 conn(0x55b75d149f70 msgr2=0x55b75d2b7ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/2825295311 shutdown_connections 2024-05-09T05:01:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efe0b1d1b80 1 -- 172.21.15.46:0/2825295311 wait complete. 2024-05-09T05:01:00.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.373+0000 7efe0b1d1b80 20 client.17499 trim_cache size 0 max 16384 2024-05-09T05:01:00.381 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T05:01:00.381 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "IDLE (waiting for more scrubs)", 2024-05-09T05:01:00.381 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": { 2024-05-09T05:01:00.381 INFO:teuthology.orchestra.run.smithi046.stdout: "7ca5e20e-fcf9-4d0e-8968-31e1700d5925": { 2024-05-09T05:01:00.381 INFO:teuthology.orchestra.run.smithi046.stdout: "path": "/", 2024-05-09T05:01:00.381 INFO:teuthology.orchestra.run.smithi046.stdout: "tag": "7ca5e20e-fcf9-4d0e-8968-31e1700d5925", 2024-05-09T05:01:00.381 INFO:teuthology.orchestra.run.smithi046.stdout: "options": "recursive,force" 2024-05-09T05:01:00.381 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:00.381 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:00.381 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T05:01:00.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.374+0000 7efdfc7bb700 1 -- 172.21.15.46:0/1610424875 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7efde00b29f0 msgr2=0x7efde00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:00.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.374+0000 7efdfc7bb700 1 --2- 172.21.15.46:0/1610424875 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7efde00b29f0 0x7efde00b4f00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7efdf0004630 tx=0x7efdf000b3c0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.374+0000 7efdfc7bb700 1 -- 172.21.15.46:0/1610424875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdf4108740 msgr2=0x7efdf419f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:00.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.374+0000 7efdfc7bb700 1 --2- 172.21.15.46:0/1610424875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdf4108740 0x7efdf419f620 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7efdec0095c0 tx=0x7efdec007a20 comp rx=0 tx=0).stop 2024-05-09T05:01:00.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.374+0000 7efdfc7bb700 1 -- 172.21.15.46:0/1610424875 shutdown_connections 2024-05-09T05:01:00.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.374+0000 7efdfc7bb700 1 --2- 172.21.15.46:0/1610424875 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7efde00b29f0 0x7efde00b4f00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.374+0000 7efdfc7bb700 1 --2- 172.21.15.46:0/1610424875 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7efdf4108740 0x7efdf419f620 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.374+0000 7efdfc7bb700 1 --2- 172.21.15.46:0/1610424875 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7efdf41034c0 0x7efdf419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.374+0000 7efdfc7bb700 1 --2- 172.21.15.46:0/1610424875 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7efdf4107df0 0x7efdf419b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:00.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.374+0000 7efdfc7bb700 1 -- 172.21.15.46:0/1610424875 >> 172.21.15.46:0/1610424875 conn(0x7efdf40fedf0 msgr2=0x7efdf4107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:00.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.374+0000 7efdfc7bb700 1 -- 172.21.15.46:0/1610424875 shutdown_connections 2024-05-09T05:01:00.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:00.374+0000 7efdfc7bb700 1 -- 172.21.15.46:0/1610424875 wait complete. 2024-05-09T05:01:01.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:00 smithi046 ceph-mon[30136]: pgmap v306: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:01.130 INFO:tasks.cephfs.filesystem:scrub status for tag:7ca5e20e-fcf9-4d0e-8968-31e1700d5925 - {'path': '/', 'tag': '7ca5e20e-fcf9-4d0e-8968-31e1700d5925', 'options': 'recursive,force'} 2024-05-09T05:01:01.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:00 smithi072 ceph-mon[34088]: pgmap v306: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:01.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:00 smithi110 ceph-mon[34904]: pgmap v306: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:02.081 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:01 smithi046 ceph-mon[30136]: scrub summary: idle 2024-05-09T05:01:02.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:01 smithi072 ceph-mon[34088]: scrub summary: idle 2024-05-09T05:01:02.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:01 smithi110 ceph-mon[34904]: scrub summary: idle 2024-05-09T05:01:03.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:02 smithi046 ceph-mon[30136]: pgmap v307: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-09T05:01:03.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:02 smithi072 ceph-mon[34088]: pgmap v307: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-09T05:01:03.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:02 smithi110 ceph-mon[34904]: pgmap v307: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-09T05:01:05.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:04 smithi072 ceph-mon[34088]: pgmap v308: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-09T05:01:05.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:04 smithi110 ceph-mon[34904]: pgmap v308: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-09T05:01:05.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:04 smithi046 ceph-mon[30136]: pgmap v308: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-09T05:01:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:06 smithi072 ceph-mon[34088]: pgmap v309: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-09T05:01:07.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:06 smithi110 ceph-mon[34904]: pgmap v309: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-09T05:01:07.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:06 smithi046 ceph-mon[30136]: pgmap v309: 97 pgs: 97 active+clean; 3.9 MiB data, 539 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-09T05:01:09.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:08 smithi072 ceph-mon[34088]: pgmap v310: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-09T05:01:09.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:08 smithi110 ceph-mon[34904]: pgmap v310: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-09T05:01:09.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:08 smithi046 ceph-mon[30136]: pgmap v310: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-09T05:01:11.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:10 smithi072 ceph-mon[34088]: pgmap v311: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-09T05:01:11.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:10 smithi110 ceph-mon[34904]: pgmap v311: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-09T05:01:11.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:10 smithi046 ceph-mon[30136]: pgmap v311: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 44 KiB/s wr, 23 op/s 2024-05-09T05:01:13.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:12 smithi110 ceph-mon[34904]: pgmap v312: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-09T05:01:13.110 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:12 smithi110 ceph-mon[34904]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T05:01:13.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:12 smithi072 ceph-mon[34088]: pgmap v312: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-09T05:01:13.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:12 smithi072 ceph-mon[34088]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T05:01:13.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:12 smithi046 ceph-mon[30136]: pgmap v312: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 44 KiB/s wr, 25 op/s 2024-05-09T05:01:13.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:12 smithi046 ceph-mon[30136]: osdmap e92: 12 total, 12 up, 12 in 2024-05-09T05:01:15.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:14 smithi072 ceph-mon[34088]: pgmap v314: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:01:15.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:14 smithi110 ceph-mon[34904]: pgmap v314: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:01:15.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:14 smithi046 ceph-mon[30136]: pgmap v314: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:01:17.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:16 smithi072 ceph-mon[34088]: pgmap v315: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:01:17.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:16 smithi110 ceph-mon[34904]: pgmap v315: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:01:17.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:16 smithi046 ceph-mon[30136]: pgmap v315: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:01:18.859 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-09T05:01:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.992+0000 7f22942d1640 1 -- 172.21.15.46:0/1191669282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f228c10e240 msgr2=0x7f228c110710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.992+0000 7f22942d1640 1 --2- 172.21.15.46:0/1191669282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f228c10e240 0x7f228c110710 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f2284009ca0 tx=0x7f228402a950 comp rx=0 tx=0).stop 2024-05-09T05:01:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.992+0000 7f22942d1640 1 -- 172.21.15.46:0/1191669282 shutdown_connections 2024-05-09T05:01:18.994 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.992+0000 7f22942d1640 1 --2- 172.21.15.46:0/1191669282 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f228c10b8e0 0x7f228c10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.992+0000 7f22942d1640 1 --2- 172.21.15.46:0/1191669282 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f228c10e240 0x7f228c110710 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.992+0000 7f22942d1640 1 --2- 172.21.15.46:0/1191669282 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f228c104e80 0x7f228c105290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.992+0000 7f22942d1640 1 -- 172.21.15.46:0/1191669282 >> 172.21.15.46:0/1191669282 conn(0x7f228c0fe390 msgr2=0x7f228c1007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.992+0000 7f22942d1640 1 -- 172.21.15.46:0/1191669282 shutdown_connections 2024-05-09T05:01:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.993+0000 7f22942d1640 1 -- 172.21.15.46:0/1191669282 wait complete. 2024-05-09T05:01:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.993+0000 7f22942d1640 1 Processor -- start 2024-05-09T05:01:18.995 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.993+0000 7f22942d1640 1 -- start start 2024-05-09T05:01:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f22942d1640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f228c104e80 0x7f228c102740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f22942d1640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f228c10b8e0 0x7f228c102c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f22942d1640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f228c10e240 0x7f228c1031c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f22942d1640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f228c113e80 con 0x7f228c10b8e0 2024-05-09T05:01:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f22942d1640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f228c113d00 con 0x7f228c104e80 2024-05-09T05:01:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f22942d1640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f228c114000 con 0x7f228c10e240 2024-05-09T05:01:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f229186c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f228c10b8e0 0x7f228c102c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f229186c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f228c10b8e0 0x7f228c102c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50830/0 (socket says 172.21.15.46:50830) 2024-05-09T05:01:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f229186c640 1 -- 172.21.15.46:0/42796318 learned_addr learned my addr 172.21.15.46:0/42796318 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:18.997 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f229206d640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f228c104e80 0x7f228c102740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f229286e640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f228c10e240 0x7f228c1031c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f229186c640 1 -- 172.21.15.46:0/42796318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f228c10e240 msgr2=0x7f228c1031c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f229186c640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f228c10e240 0x7f228c1031c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f229186c640 1 -- 172.21.15.46:0/42796318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f228c104e80 msgr2=0x7f228c102740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f229186c640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f228c104e80 0x7f228c102740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f229186c640 1 -- 172.21.15.46:0/42796318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228c1a8b60 con 0x7f228c10b8e0 2024-05-09T05:01:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f229286e640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f228c10e240 0x7f228c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:01:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f229206d640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f228c104e80 0x7f228c102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:01:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f229186c640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f228c10b8e0 0x7f228c102c80 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f227c00e9f0 tx=0x7f227c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f22837fe640 1 -- 172.21.15.46:0/42796318 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c00cd80 con 0x7f228c10b8e0 2024-05-09T05:01:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f22837fe640 1 -- 172.21.15.46:0/42796318 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f227c0054d0 con 0x7f228c10b8e0 2024-05-09T05:01:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.994+0000 7f22942d1640 1 -- 172.21.15.46:0/42796318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f228c1a8e50 con 0x7f228c10b8e0 2024-05-09T05:01:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.995+0000 7f22837fe640 1 -- 172.21.15.46:0/42796318 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f227c010690 con 0x7f228c10b8e0 2024-05-09T05:01:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.995+0000 7f22942d1640 1 -- 172.21.15.46:0/42796318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f228c1a9260 con 0x7f228c10b8e0 2024-05-09T05:01:18.998 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.996+0000 7f22942d1640 1 -- 172.21.15.46:0/42796318 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f228c050870 con 0x7f228c10b8e0 2024-05-09T05:01:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.997+0000 7f22837fe640 1 -- 172.21.15.46:0/42796318 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f227c004e30 con 0x7f228c10b8e0 2024-05-09T05:01:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.998+0000 7f22837fe640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22780b2950 0x7f22780b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.998+0000 7f229206d640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22780b2950 0x7f22780b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:18.999 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.998+0000 7f22837fe640 1 -- 172.21.15.46:0/42796318 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8644+0+0 (secure 0 0 0) 0x7f227c0e2dc0 con 0x7f228c10b8e0 2024-05-09T05:01:19.000 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:18.999+0000 7f229206d640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22780b2950 0x7f22780b4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22880058c0 tx=0x7f2288005850 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:19.002 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.002+0000 7f22837fe640 1 -- 172.21.15.46:0/42796318 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f227c054df0 con 0x7f228c10b8e0 2024-05-09T05:01:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:18 smithi072 ceph-mon[34088]: pgmap v316: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:01:19.222 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.221+0000 7f22942d1640 1 -- 172.21.15.46:0/42796318 --> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f228c10ed60 con 0x7f22780b2950 2024-05-09T05:01:19.230 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.229+0000 7f22837fe640 1 -- 172.21.15.46:0/42796318 <== mgr.15150 v2:172.21.15.46:6800/2595796783 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f228c10ed60 con 0x7f22780b2950 2024-05-09T05:01:19.230 INFO:teuthology.orchestra.run.smithi046.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-09T00:00:00", "created": "2024-05-09T04:51:16", "first": "2024-05-09T04:52:00", "last": "2024-05-09T05:01:01", "last_pruned": "2024-05-09T05:01:01", "created_count": 10, "pruned_count": 4, "active": true} 2024-05-09T05:01:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.231+0000 7f22942d1640 1 -- 172.21.15.46:0/42796318 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22780b2950 msgr2=0x7f22780b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.231+0000 7f22942d1640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22780b2950 0x7f22780b4e60 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22880058c0 tx=0x7f2288005850 comp rx=0 tx=0).stop 2024-05-09T05:01:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.231+0000 7f22942d1640 1 -- 172.21.15.46:0/42796318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f228c10b8e0 msgr2=0x7f228c102c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.231+0000 7f22942d1640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f228c10b8e0 0x7f228c102c80 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f227c00e9f0 tx=0x7f227c00edf0 comp rx=0 tx=0).stop 2024-05-09T05:01:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.231+0000 7f22942d1640 1 -- 172.21.15.46:0/42796318 shutdown_connections 2024-05-09T05:01:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.231+0000 7f22942d1640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f22780b2950 0x7f22780b4e60 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.232+0000 7f22942d1640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f228c10e240 0x7f228c1031c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.232+0000 7f22942d1640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f228c10b8e0 0x7f228c102c80 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.232+0000 7f22942d1640 1 --2- 172.21.15.46:0/42796318 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f228c104e80 0x7f228c102740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:19.233 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.232+0000 7f22942d1640 1 -- 172.21.15.46:0/42796318 >> 172.21.15.46:0/42796318 conn(0x7f228c0fe390 msgr2=0x7f228c10e650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:19.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.232+0000 7f22942d1640 1 -- 172.21.15.46:0/42796318 shutdown_connections 2024-05-09T05:01:19.234 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.232+0000 7f22942d1640 1 -- 172.21.15.46:0/42796318 wait complete. 2024-05-09T05:01:19.243 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:18 smithi046 ceph-mon[30136]: pgmap v316: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:01:19.244 DEBUG:teuthology.orchestra.run.smithi046:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-09T05:01:19.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:18 smithi110 ceph-mon[34904]: pgmap v316: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:01:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.359+0000 7fa9c4c13640 1 -- 172.21.15.46:0/3457947130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c0108f60 msgr2=0x7fa9c0110700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.359+0000 7fa9c4c13640 1 --2- 172.21.15.46:0/3457947130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c0108f60 0x7fa9c0110700 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8009ca0 tx=0x7fa9b802a8c0 comp rx=0 tx=0).stop 2024-05-09T05:01:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.359+0000 7fa9c4c13640 1 -- 172.21.15.46:0/3457947130 shutdown_connections 2024-05-09T05:01:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.359+0000 7fa9c4c13640 1 --2- 172.21.15.46:0/3457947130 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c0108f60 0x7fa9c0110700 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.359+0000 7fa9c4c13640 1 --2- 172.21.15.46:0/3457947130 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9c01084a0 0x7fa9c01088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.359+0000 7fa9c4c13640 1 --2- 172.21.15.46:0/3457947130 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9c0102420 0x7fa9c0102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.359+0000 7fa9c4c13640 1 -- 172.21.15.46:0/3457947130 >> 172.21.15.46:0/3457947130 conn(0x7fa9c00fd9d0 msgr2=0x7fa9c00ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:19.361 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.359+0000 7fa9c4c13640 1 -- 172.21.15.46:0/3457947130 shutdown_connections 2024-05-09T05:01:19.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.359+0000 7fa9c4c13640 1 -- 172.21.15.46:0/3457947130 wait complete. 2024-05-09T05:01:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.360+0000 7fa9c4c13640 1 Processor -- start 2024-05-09T05:01:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.360+0000 7fa9c4c13640 1 -- start start 2024-05-09T05:01:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9c4c13640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9c0102420 0x7fa9c01a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9c4c13640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9c01084a0 0x7fa9c01a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9c4c13640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c0108f60 0x7fa9c019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9c4c13640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0113e70 con 0x7fa9c0108f60 2024-05-09T05:01:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9c4c13640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0113cf0 con 0x7fa9c0102420 2024-05-09T05:01:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9c4c13640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa9c0113ff0 con 0x7fa9c01084a0 2024-05-09T05:01:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9bed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c0108f60 0x7fa9c019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9bed9d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c0108f60 0x7fa9c019a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:50860/0 (socket says 172.21.15.46:50860) 2024-05-09T05:01:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9bed9d640 1 -- 172.21.15.46:0/2116091536 learned_addr learned my addr 172.21.15.46:0/2116091536 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9be59c640 1 --2- 172.21.15.46:0/2116091536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9c0102420 0x7fa9c01a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:19.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9b5d9b640 1 --2- 172.21.15.46:0/2116091536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9c01084a0 0x7fa9c01a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9bed9d640 1 -- 172.21.15.46:0/2116091536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9c01084a0 msgr2=0x7fa9c01a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9bed9d640 1 --2- 172.21.15.46:0/2116091536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9c01084a0 0x7fa9c01a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9bed9d640 1 -- 172.21.15.46:0/2116091536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9c0102420 msgr2=0x7fa9c01a04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9bed9d640 1 --2- 172.21.15.46:0/2116091536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9c0102420 0x7fa9c01a04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.361+0000 7fa9bed9d640 1 -- 172.21.15.46:0/2116091536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9c019ada0 con 0x7fa9c0108f60 2024-05-09T05:01:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.362+0000 7fa9bed9d640 1 --2- 172.21.15.46:0/2116091536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c0108f60 0x7fa9c019a670 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8005cc0 tx=0x7fa9b802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.362+0000 7fa9b7fff640 1 -- 172.21.15.46:0/2116091536 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8003bf0 con 0x7fa9c0108f60 2024-05-09T05:01:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.362+0000 7fa9c4c13640 1 -- 172.21.15.46:0/2116091536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9c019afd0 con 0x7fa9c0108f60 2024-05-09T05:01:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.362+0000 7fa9b7fff640 1 -- 172.21.15.46:0/2116091536 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9b8003d90 con 0x7fa9c0108f60 2024-05-09T05:01:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.362+0000 7fa9b7fff640 1 -- 172.21.15.46:0/2116091536 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b802fbc0 con 0x7fa9c0108f60 2024-05-09T05:01:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.362+0000 7fa9c4c13640 1 -- 172.21.15.46:0/2116091536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9c01a7290 con 0x7fa9c0108f60 2024-05-09T05:01:19.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.363+0000 7fa9c4c13640 1 -- 172.21.15.46:0/2116091536 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9c0105080 con 0x7fa9c0108f60 2024-05-09T05:01:19.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.365+0000 7fa9b7fff640 1 -- 172.21.15.46:0/2116091536 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa9b802fd60 con 0x7fa9c0108f60 2024-05-09T05:01:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.366+0000 7fa9b7fff640 1 --2- 172.21.15.46:0/2116091536 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa9ac0b2950 0x7fa9ac0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.366+0000 7fa9b7fff640 1 -- 172.21.15.46:0/2116091536 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 8644+0+0 (secure 0 0 0) 0x7fa9b8103e20 con 0x7fa9c0108f60 2024-05-09T05:01:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.366+0000 7fa9be59c640 1 --2- 172.21.15.46:0/2116091536 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa9ac0b2950 0x7fa9ac0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:19.367 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.367+0000 7fa9be59c640 1 --2- 172.21.15.46:0/2116091536 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa9ac0b2950 0x7fa9ac0b4e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0005fd0 tx=0x7fa9b000b390 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:19.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.370+0000 7fa9b7fff640 1 -- 172.21.15.46:0/2116091536 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9b80cb830 con 0x7fa9c0108f60 2024-05-09T05:01:19.586 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.585+0000 7fa9c4c13640 1 -- 172.21.15.46:0/2116091536 --> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fa9c0063110 con 0x7fa9ac0b2950 2024-05-09T05:01:19.593 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.592+0000 7fa9b7fff640 1 -- 172.21.15.46:0/2116091536 <== mgr.15150 v2:172.21.15.46:6800/2595796783 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fa9c0063110 con 0x7fa9ac0b2950 2024-05-09T05:01:19.593 INFO:teuthology.orchestra.run.smithi046.stdout:/ 1m 6m3h 2024-05-09T05:01:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.594+0000 7fa9c4c13640 1 -- 172.21.15.46:0/2116091536 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa9ac0b2950 msgr2=0x7fa9ac0b4e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.594+0000 7fa9c4c13640 1 --2- 172.21.15.46:0/2116091536 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa9ac0b2950 0x7fa9ac0b4e60 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fa9b0005fd0 tx=0x7fa9b000b390 comp rx=0 tx=0).stop 2024-05-09T05:01:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.594+0000 7fa9c4c13640 1 -- 172.21.15.46:0/2116091536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c0108f60 msgr2=0x7fa9c019a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.594+0000 7fa9c4c13640 1 --2- 172.21.15.46:0/2116091536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c0108f60 0x7fa9c019a670 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8005cc0 tx=0x7fa9b802f380 comp rx=0 tx=0).stop 2024-05-09T05:01:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.594+0000 7fa9c4c13640 1 -- 172.21.15.46:0/2116091536 shutdown_connections 2024-05-09T05:01:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.594+0000 7fa9c4c13640 1 --2- 172.21.15.46:0/2116091536 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa9ac0b2950 0x7fa9ac0b4e60 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.594+0000 7fa9c4c13640 1 --2- 172.21.15.46:0/2116091536 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa9c01084a0 0x7fa9c01a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.594+0000 7fa9c4c13640 1 --2- 172.21.15.46:0/2116091536 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa9c0108f60 0x7fa9c019a670 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.594+0000 7fa9c4c13640 1 --2- 172.21.15.46:0/2116091536 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa9c0102420 0x7fa9c01a04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.594+0000 7fa9c4c13640 1 -- 172.21.15.46:0/2116091536 >> 172.21.15.46:0/2116091536 conn(0x7fa9c00fd9d0 msgr2=0x7fa9c0106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:19.596 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.595+0000 7fa9c4c13640 1 -- 172.21.15.46:0/2116091536 shutdown_connections 2024-05-09T05:01:19.597 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:19.595+0000 7fa9c4c13640 1 -- 172.21.15.46:0/2116091536 wait complete. 2024-05-09T05:01:19.607 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-05-09T05:01:19.619 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-05-09T05:01:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:20 smithi072 ceph-mon[34088]: from='client.17505 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:01:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:20 smithi072 ceph-mon[34088]: from='client.17511 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:01:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:20 smithi072 ceph-mon[34088]: pgmap v317: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:01:21.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:20 smithi110 ceph-mon[34904]: from='client.17505 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:01:21.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:20 smithi110 ceph-mon[34904]: from='client.17511 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:01:21.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:20 smithi110 ceph-mon[34904]: pgmap v317: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:01:21.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:20 smithi046 ceph-mon[30136]: from='client.17505 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:01:21.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:20 smithi046 ceph-mon[30136]: from='client.17511 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-09T05:01:21.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:20 smithi046 ceph-mon[30136]: pgmap v317: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:01:22.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:22 smithi110 ceph-mon[34904]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T05:01:22.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:22 smithi110 ceph-mon[34904]: pgmap v319: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 103 B/s wr, 5 op/s 2024-05-09T05:01:22.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:22 smithi046 ceph-mon[30136]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T05:01:22.580 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:22 smithi046 ceph-mon[30136]: pgmap v319: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 103 B/s wr, 5 op/s 2024-05-09T05:01:22.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:22 smithi072 ceph-mon[34088]: osdmap e93: 12 total, 12 up, 12 in 2024-05-09T05:01:22.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:22 smithi072 ceph-mon[34088]: pgmap v319: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 103 B/s wr, 5 op/s 2024-05-09T05:01:25.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:24 smithi046 ceph-mon[30136]: pgmap v320: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-05-09T05:01:25.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:24 smithi072 ceph-mon[34088]: pgmap v320: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-05-09T05:01:25.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:24 smithi110 ceph-mon[34904]: pgmap v320: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-05-09T05:01:27.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:26 smithi046 ceph-mon[30136]: pgmap v321: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-05-09T05:01:27.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:26 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:01:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:26 smithi072 ceph-mon[34088]: pgmap v321: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-05-09T05:01:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:26 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:01:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:26 smithi110 ceph-mon[34904]: pgmap v321: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 9 op/s 2024-05-09T05:01:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:26 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:01:28.228 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:28 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.228 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:28 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.228 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:28 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.228 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:28 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.228 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:28 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.229 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:28 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.229 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:28 smithi110 ceph-mon[34904]: pgmap v322: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-09T05:01:28.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:28 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:28 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:28 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:28 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:28 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:28 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:28 smithi046 ceph-mon[30136]: pgmap v322: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-09T05:01:28.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:28 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:28 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:28 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:28 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:28 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:28 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:28.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:28 smithi072 ceph-mon[34088]: pgmap v322: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-09T05:01:30.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:01:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:01:30.132 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:01:30.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:01:30.309 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:30.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-09T05:01:30.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:01:30.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:01:30.331 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:01:31.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:30 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi072 to 176.1M 2024-05-09T05:01:31.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:30 smithi072 ceph-mon[34088]: Unable to set osd_memory_target on smithi072 to 184725811: error parsing value: Value '184725811' is below minimum 939524096 2024-05-09T05:01:31.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:30 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi110 to 176.2M 2024-05-09T05:01:31.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:30 smithi072 ceph-mon[34088]: Unable to set osd_memory_target on smithi110 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-05-09T05:01:31.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:30 smithi072 ceph-mon[34088]: Adjusting osd_memory_target on smithi046 to 177.2M 2024-05-09T05:01:31.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:30 smithi072 ceph-mon[34088]: Unable to set osd_memory_target on smithi046 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-09T05:01:31.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:30 smithi072 ceph-mon[34088]: pgmap v323: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-09T05:01:31.131 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-05-09T05:01:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:30 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi072 to 176.1M 2024-05-09T05:01:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:30 smithi046 ceph-mon[30136]: Unable to set osd_memory_target on smithi072 to 184725811: error parsing value: Value '184725811' is below minimum 939524096 2024-05-09T05:01:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:30 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi110 to 176.2M 2024-05-09T05:01:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:30 smithi046 ceph-mon[30136]: Unable to set osd_memory_target on smithi110 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-05-09T05:01:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:30 smithi046 ceph-mon[30136]: Adjusting osd_memory_target on smithi046 to 177.2M 2024-05-09T05:01:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:30 smithi046 ceph-mon[30136]: Unable to set osd_memory_target on smithi046 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-09T05:01:31.160 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:30 smithi046 ceph-mon[30136]: pgmap v323: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-09T05:01:31.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:30 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi072 to 176.1M 2024-05-09T05:01:31.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:30 smithi110 ceph-mon[34904]: Unable to set osd_memory_target on smithi072 to 184725811: error parsing value: Value '184725811' is below minimum 939524096 2024-05-09T05:01:31.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:30 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi110 to 176.2M 2024-05-09T05:01:31.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:30 smithi110 ceph-mon[34904]: Unable to set osd_memory_target on smithi110 to 184763801: error parsing value: Value '184763801' is below minimum 939524096 2024-05-09T05:01:31.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:30 smithi110 ceph-mon[34904]: Adjusting osd_memory_target on smithi046 to 177.2M 2024-05-09T05:01:31.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:30 smithi110 ceph-mon[34904]: Unable to set osd_memory_target on smithi046 to 185879142: error parsing value: Value '185879142' is below minimum 939524096 2024-05-09T05:01:31.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:30 smithi110 ceph-mon[34904]: pgmap v323: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2024-05-09T05:01:31.407 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:01:33.072 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:32 smithi046 ceph-mon[30136]: pgmap v324: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-09T05:01:33.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:32 smithi072 ceph-mon[34088]: pgmap v324: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-09T05:01:33.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:32 smithi110 ceph-mon[34904]: pgmap v324: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 0 B/s wr, 10 op/s 2024-05-09T05:01:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.361+0000 7fc80d287700 1 -- 172.21.15.46:0/3148775511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc808108f00 msgr2=0x7fc808109310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.361+0000 7fc80d287700 1 --2- 172.21.15.46:0/3148775511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc808108f00 0x7fc808109310 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0099c0 tx=0x7fc7fc009cd0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.361+0000 7fc80d287700 1 -- 172.21.15.46:0/3148775511 shutdown_connections 2024-05-09T05:01:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.361+0000 7fc80d287700 1 --2- 172.21.15.46:0/3148775511 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc808109850 0x7fc8081111b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.361+0000 7fc80d287700 1 --2- 172.21.15.46:0/3148775511 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc8081045d0 0x7fc8081049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.361+0000 7fc80d287700 1 --2- 172.21.15.46:0/3148775511 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc808108f00 0x7fc808109310 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.361+0000 7fc80d287700 1 -- 172.21.15.46:0/3148775511 >> 172.21.15.46:0/3148775511 conn(0x7fc8080ffea0 msgr2=0x7fc8081022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.361+0000 7fc80d287700 1 -- 172.21.15.46:0/3148775511 shutdown_connections 2024-05-09T05:01:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.361+0000 7fc80d287700 1 -- 172.21.15.46:0/3148775511 wait complete. 2024-05-09T05:01:33.362 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.362+0000 7fc80d287700 1 Processor -- start 2024-05-09T05:01:33.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.362+0000 7fc80d287700 1 -- start start 2024-05-09T05:01:33.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.362+0000 7fc80d287700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc8081045d0 0x7fc80819ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.362+0000 7fc80d287700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc808108f00 0x7fc80819b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.362+0000 7fc80d287700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc808109850 0x7fc80819f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.362+0000 7fc80d287700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc808114460 con 0x7fc808108f00 2024-05-09T05:01:33.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.362+0000 7fc80d287700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc8081142e0 con 0x7fc8081045d0 2024-05-09T05:01:33.364 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.362+0000 7fc80d287700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc8081145e0 con 0x7fc808109850 2024-05-09T05:01:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.362+0000 7fc8067fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc808108f00 0x7fc80819b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.362+0000 7fc806ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc8081045d0 0x7fc80819ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.362+0000 7fc8067fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc808108f00 0x7fc80819b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54650/0 (socket says 172.21.15.46:54650) 2024-05-09T05:01:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.362+0000 7fc806ffd700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc8081045d0 0x7fc80819ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.72:3300/0 says I am v2:172.21.15.46:33264/0 (socket says 172.21.15.46:33264) 2024-05-09T05:01:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.362+0000 7fc8067fc700 1 -- 172.21.15.46:0/4145740550 learned_addr learned my addr 172.21.15.46:0/4145740550 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.363+0000 7fc8077fe700 1 --2- 172.21.15.46:0/4145740550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc808109850 0x7fc80819f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.363+0000 7fc8067fc700 1 -- 172.21.15.46:0/4145740550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc808109850 msgr2=0x7fc80819f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.363+0000 7fc8067fc700 1 --2- 172.21.15.46:0/4145740550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc808109850 0x7fc80819f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.363+0000 7fc8067fc700 1 -- 172.21.15.46:0/4145740550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc8081045d0 msgr2=0x7fc80819ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.363+0000 7fc8067fc700 1 --2- 172.21.15.46:0/4145740550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc8081045d0 0x7fc80819ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.363+0000 7fc8067fc700 1 -- 172.21.15.46:0/4145740550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80819fd00 con 0x7fc808108f00 2024-05-09T05:01:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.363+0000 7fc8067fc700 1 --2- 172.21.15.46:0/4145740550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc808108f00 0x7fc80819b3d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0098e0 tx=0x7fc7fc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.363+0000 7fc7effff700 1 -- 172.21.15.46:0/4145740550 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc027070 con 0x7fc808108f00 2024-05-09T05:01:33.365 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.363+0000 7fc7effff700 1 -- 172.21.15.46:0/4145740550 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7fc003f20 con 0x7fc808108f00 2024-05-09T05:01:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.363+0000 7fc80d287700 1 -- 172.21.15.46:0/4145740550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80819ff90 con 0x7fc808108f00 2024-05-09T05:01:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.363+0000 7fc7effff700 1 -- 172.21.15.46:0/4145740550 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc022d70 con 0x7fc808108f00 2024-05-09T05:01:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.363+0000 7fc80d287700 1 -- 172.21.15.46:0/4145740550 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8081a0470 con 0x7fc808108f00 2024-05-09T05:01:33.366 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.366+0000 7fc7effff700 1 -- 172.21.15.46:0/4145740550 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc7fc02a070 con 0x7fc808108f00 2024-05-09T05:01:33.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.367+0000 7fc7effff700 1 --2- 172.21.15.46:0/4145740550 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc7f00b29f0 0x7fc7f00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc806ffd700 1 --2- 172.21.15.46:0/4145740550 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc7f00b29f0 0x7fc7f00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc81bc9db80 1 Processor -- start 2024-05-09T05:01:33.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc7effff700 1 -- 172.21.15.46:0/4145740550 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7fc7fc0e3430 con 0x7fc808108f00 2024-05-09T05:01:33.368 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc81bc9db80 1 -- start start 2024-05-09T05:01:33.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc81bc9db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c85d9160 0x5596c86724f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc806ffd700 1 --2- 172.21.15.46:0/4145740550 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc7f00b29f0 0x7fc7f00b4f00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8007be0 tx=0x7fc7f80073d0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:33.369 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc81bc9db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c84ef4a0 0x5596c8672a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc81bc9db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c866c6a0 0x5596c866cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc81bc9db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5596c858eff0 con 0x5596c866c6a0 2024-05-09T05:01:33.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc81bc9db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5596c852d7f0 con 0x5596c85d9160 2024-05-09T05:01:33.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc81bc9db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5596c858afb0 con 0x5596c84ef4a0 2024-05-09T05:01:33.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c866c6a0 0x5596c866cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c866c6a0 0x5596c866cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54662/0 (socket says 172.21.15.46:54662) 2024-05-09T05:01:33.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc8077fe700 1 -- 172.21.15.46:0/4130651843 learned_addr learned my addr 172.21.15.46:0/4130651843 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:33.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc8067fc700 1 --2- 172.21.15.46:0/4130651843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c84ef4a0 0x5596c8672a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.368+0000 7fc806ffd700 1 --2- 172.21.15.46:0/4130651843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c85d9160 0x5596c86724f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.369+0000 7fc8077fe700 1 -- 172.21.15.46:0/4130651843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c84ef4a0 msgr2=0x5596c8672a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:33.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.369+0000 7fc8077fe700 1 --2- 172.21.15.46:0/4130651843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c84ef4a0 0x5596c8672a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.369+0000 7fc8077fe700 1 -- 172.21.15.46:0/4130651843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c85d9160 msgr2=0x5596c86724f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:33.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.369+0000 7fc8077fe700 1 --2- 172.21.15.46:0/4130651843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c85d9160 0x5596c86724f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.370 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.369+0000 7fc8077fe700 1 -- 172.21.15.46:0/4130651843 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596c8525370 con 0x5596c866c6a0 2024-05-09T05:01:33.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.369+0000 7fc806ffd700 1 --2- 172.21.15.46:0/4130651843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c85d9160 0x5596c86724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:01:33.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.369+0000 7fc8077fe700 1 --2- 172.21.15.46:0/4130651843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c866c6a0 0x5596c866cad0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc800009d50 tx=0x7fc8000077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b7dcace7162037 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:33.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.369+0000 7fc7ed7fa700 1 -- 172.21.15.46:0/4130651843 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800003a00 con 0x5596c866c6a0 2024-05-09T05:01:33.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.369+0000 7fc7ed7fa700 1 -- 172.21.15.46:0/4130651843 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc800003ba0 con 0x5596c866c6a0 2024-05-09T05:01:33.371 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.369+0000 7fc7ed7fa700 1 -- 172.21.15.46:0/4130651843 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8000048b0 con 0x5596c866c6a0 2024-05-09T05:01:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.370+0000 7fc81bc9db80 1 -- 172.21.15.46:0/4130651843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c866c6a0 msgr2=0x5596c866cad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.370+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/4130651843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c866c6a0 0x5596c866cad0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fc800009d50 tx=0x7fc8000077e0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.370+0000 7fc81bc9db80 1 -- 172.21.15.46:0/4130651843 shutdown_connections 2024-05-09T05:01:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.370+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/4130651843 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c866c6a0 0x5596c866cad0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.370+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/4130651843 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c85d9160 0x5596c86724f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.374 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.370+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/4130651843 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c84ef4a0 0x5596c8672a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.370+0000 7fc81bc9db80 1 -- 172.21.15.46:0/4130651843 >> 172.21.15.46:0/4130651843 conn(0x5596c854f6a0 msgr2=0x5596c8667400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.370+0000 7fc81bc9db80 1 -- 172.21.15.46:0/4130651843 shutdown_connections 2024-05-09T05:01:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.370+0000 7fc81bc9db80 1 -- 172.21.15.46:0/4130651843 wait complete. 2024-05-09T05:01:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.370+0000 7fc81bc9db80 1 -- start start 2024-05-09T05:01:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc81bc9db80 1 Processor -- start 2024-05-09T05:01:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc81bc9db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c84ef4a0 0x5596c8676140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc81bc9db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c85d9160 0x5596c8676680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc81bc9db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c866c6a0 0x5596c8676bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.377 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc81bc9db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x5596c840aba0 con 0x5596c84ef4a0 2024-05-09T05:01:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc81bc9db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x5596c8549150 con 0x5596c866c6a0 2024-05-09T05:01:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc81bc9db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x5596c867d5e0 con 0x5596c85d9160 2024-05-09T05:01:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc806ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c84ef4a0 0x5596c8676140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc81bc9db80 10 client.0 _ll_register_callbacks cb 0x5596c84006c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:01:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc806ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c84ef4a0 0x5596c8676140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54668/0 (socket says 172.21.15.46:54668) 2024-05-09T05:01:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc806ffd700 1 -- 172.21.15.46:0/2945151363 learned_addr learned my addr 172.21.15.46:0/2945151363 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc8067fc700 1 --2- 172.21.15.46:0/2945151363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c85d9160 0x5596c8676680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc8077fe700 1 --2- 172.21.15.46:0/2945151363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c866c6a0 0x5596c8676bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc806ffd700 1 -- 172.21.15.46:0/2945151363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c85d9160 msgr2=0x5596c8676680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc806ffd700 1 --2- 172.21.15.46:0/2945151363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c85d9160 0x5596c8676680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc806ffd700 1 -- 172.21.15.46:0/2945151363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c866c6a0 msgr2=0x5596c8676bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc806ffd700 1 --2- 172.21.15.46:0/2945151363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c866c6a0 0x5596c8676bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc806ffd700 1 -- 172.21.15.46:0/2945151363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596c840a490 con 0x5596c84ef4a0 2024-05-09T05:01:33.378 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.371+0000 7fc8077fe700 1 --2- 172.21.15.46:0/2945151363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c866c6a0 0x5596c8676bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:01:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.372+0000 7fc806ffd700 1 --2- 172.21.15.46:0/2945151363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c84ef4a0 0x5596c8676140 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80040b0 tx=0x7fc7f80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b1a7962c22431 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.372+0000 7fc7eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:01:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.372+0000 7fc7eaff5700 1 -- 172.21.15.46:0/2945151363 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8024070 con 0x5596c84ef4a0 2024-05-09T05:01:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.372+0000 7fc7eaff5700 1 -- 172.21.15.46:0/2945151363 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7fc7f8003a90 con 0x5596c84ef4a0 2024-05-09T05:01:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.372+0000 7fc81bc9db80 1 -- 172.21.15.46:0/2945151363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc800003ba0 con 0x5596c84ef4a0 2024-05-09T05:01:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.372+0000 7fc7eaff5700 1 -- 172.21.15.46:0/2945151363 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7f8030050 con 0x5596c84ef4a0 2024-05-09T05:01:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.372+0000 7fc81bc9db80 10 client.17523 fetch_fsmap learned FSMap version 25 2024-05-09T05:01:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.372+0000 7fc81bc9db80 1 -- 172.21.15.46:0/2945151363 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5596c8687f80 con 0x5596c84ef4a0 2024-05-09T05:01:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.372+0000 7fc7eaff5700 1 -- 172.21.15.46:0/2945151363 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc800003ba0 con 0x5596c84ef4a0 2024-05-09T05:01:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.373+0000 7fc7eaff5700 1 -- 172.21.15.46:0/2945151363 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc7f8020410 con 0x5596c84ef4a0 2024-05-09T05:01:33.379 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.373+0000 7fc81bc9db80 10 client.17523 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:01:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.373+0000 7fc81bc9db80 10 client.17523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:01:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.373+0000 7fc81bc9db80 20 client.17523 populate_metadata read hostname 'smithi046' 2024-05-09T05:01:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.373+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/2945151363 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5596c868a560 0x5596c868c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.373+0000 7fc81bc9db80 4 client.17523 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:01:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.373+0000 7fc81bc9db80 1 -- 172.21.15.46:0/2945151363 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596c859c9d0 con 0x5596c868a560 2024-05-09T05:01:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.373+0000 7fc8067fc700 1 --2- 172.21.15.46:0/2945151363 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5596c868a560 0x5596c868c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.373+0000 7fc7eaff5700 10 client.17523 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:01:33.380 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.373+0000 7fc8067fc700 1 --2- 172.21.15.46:0/2945151363 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5596c868a560 0x5596c868c9a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bbbd9c964bfd151 server_cookie=a823e244772b2ef6 in_seq=0 out_seq=0 2024-05-09T05:01:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.374+0000 7fc7eaff5700 1 -- 172.21.15.46:0/2945151363 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5596c859c9d0 con 0x5596c868a560 2024-05-09T05:01:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.374+0000 7fc7eaff5700 10 client.17523 handle_command_reply: tid=0 2024-05-09T05:01:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.374+0000 7fc81bc9db80 1 client.17523 shutdown 2024-05-09T05:01:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.374+0000 7fc81bc9db80 1 -- 172.21.15.46:0/2945151363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c84ef4a0 msgr2=0x5596c8676140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.374+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/2945151363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c84ef4a0 0x5596c8676140 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fc7f80040b0 tx=0x7fc7f80047c0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.374+0000 7fc81bc9db80 1 -- 172.21.15.46:0/2945151363 shutdown_connections 2024-05-09T05:01:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.374+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/2945151363 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5596c868a560 0x5596c868c9a0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.374+0000 7fc7eaff5700 0 client.17523 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:01:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.374+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/2945151363 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c866c6a0 0x5596c8676bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.374+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/2945151363 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c84ef4a0 0x5596c8676140 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.382 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.374+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/2945151363 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c85d9160 0x5596c8676680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.374+0000 7fc81bc9db80 1 -- 172.21.15.46:0/2945151363 >> 172.21.15.46:0/2945151363 conn(0x5596c854f6a0 msgr2=0x5596c86671c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.374+0000 7fc81bc9db80 1 -- 172.21.15.46:0/2945151363 shutdown_connections 2024-05-09T05:01:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.374+0000 7fc81bc9db80 1 -- 172.21.15.46:0/2945151363 wait complete. 2024-05-09T05:01:33.383 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.375+0000 7fc81bc9db80 20 client.17523 trim_cache size 0 max 16384 2024-05-09T05:01:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.395+0000 7fc81bc9db80 1 Processor -- start 2024-05-09T05:01:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.395+0000 7fc81bc9db80 1 -- start start 2024-05-09T05:01:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.395+0000 7fc81bc9db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c84ef4a0 0x5596c8670720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.396 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.395+0000 7fc81bc9db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c85d9160 0x5596c86750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.395+0000 7fc81bc9db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c866c6a0 0x5596c8676a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.395+0000 7fc81bc9db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8081af5c0 con 0x5596c85d9160 2024-05-09T05:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.395+0000 7fc81bc9db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc8081ace50 con 0x5596c84ef4a0 2024-05-09T05:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.395+0000 7fc81bc9db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc8081aace0 con 0x5596c866c6a0 2024-05-09T05:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.395+0000 7fc8067fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c85d9160 0x5596c86750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.396+0000 7fc8067fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c85d9160 0x5596c86750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54680/0 (socket says 172.21.15.46:54680) 2024-05-09T05:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.396+0000 7fc8067fc700 1 -- 172.21.15.46:0/565902769 learned_addr learned my addr 172.21.15.46:0/565902769 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.396+0000 7fc806ffd700 1 --2- 172.21.15.46:0/565902769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c84ef4a0 0x5596c8670720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.396+0000 7fc8077fe700 1 --2- 172.21.15.46:0/565902769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c866c6a0 0x5596c8676a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.396+0000 7fc8067fc700 1 -- 172.21.15.46:0/565902769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c866c6a0 msgr2=0x5596c8676a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.396+0000 7fc8067fc700 1 --2- 172.21.15.46:0/565902769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c866c6a0 0x5596c8676a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.396+0000 7fc8067fc700 1 -- 172.21.15.46:0/565902769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c84ef4a0 msgr2=0x5596c8670720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.396+0000 7fc8067fc700 1 --2- 172.21.15.46:0/565902769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c84ef4a0 0x5596c8670720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.397 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.396+0000 7fc8067fc700 1 -- 172.21.15.46:0/565902769 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596c866d030 con 0x5596c85d9160 2024-05-09T05:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.396+0000 7fc806ffd700 1 --2- 172.21.15.46:0/565902769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c84ef4a0 0x5596c8670720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.396+0000 7fc8067fc700 1 --2- 172.21.15.46:0/565902769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c85d9160 0x5596c86750a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0b11d0 tx=0x7fc7fc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bbce7790f3871b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.396+0000 7fc7ecff9700 1 -- 172.21.15.46:0/565902769 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc0aae50 con 0x5596c85d9160 2024-05-09T05:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.396+0000 7fc7ecff9700 1 -- 172.21.15.46:0/565902769 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7fc0aa750 con 0x5596c85d9160 2024-05-09T05:01:33.398 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.396+0000 7fc7ecff9700 1 -- 172.21.15.46:0/565902769 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc0aa9b0 con 0x5596c85d9160 2024-05-09T05:01:33.401 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.397+0000 7fc81bc9db80 1 -- 172.21.15.46:0/565902769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c85d9160 msgr2=0x5596c86750a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.397+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/565902769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c85d9160 0x5596c86750a0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0b11d0 tx=0x7fc7fc009310 comp rx=0 tx=0).stop 2024-05-09T05:01:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.397+0000 7fc81bc9db80 1 -- 172.21.15.46:0/565902769 shutdown_connections 2024-05-09T05:01:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.397+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/565902769 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c866c6a0 0x5596c8676a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.397+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/565902769 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c84ef4a0 0x5596c8670720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.397+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/565902769 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c85d9160 0x5596c86750a0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.397+0000 7fc81bc9db80 1 -- 172.21.15.46:0/565902769 >> 172.21.15.46:0/565902769 conn(0x5596c854f6a0 msgr2=0x5596c866b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.397+0000 7fc81bc9db80 1 -- 172.21.15.46:0/565902769 shutdown_connections 2024-05-09T05:01:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.397+0000 7fc81bc9db80 1 -- 172.21.15.46:0/565902769 wait complete. 2024-05-09T05:01:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.398+0000 7fc81bc9db80 1 -- start start 2024-05-09T05:01:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.398+0000 7fc81bc9db80 1 Processor -- start 2024-05-09T05:01:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.398+0000 7fc81bc9db80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c84ef4a0 0x5596c86828f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.402 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.398+0000 7fc81bc9db80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c85d9160 0x5596c8682e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.398+0000 7fc81bc9db80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c866c6a0 0x5596c867d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.398+0000 7fc81bc9db80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc8081ab330 con 0x5596c85d9160 2024-05-09T05:01:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.398+0000 7fc81bc9db80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc8081ac780 con 0x5596c84ef4a0 2024-05-09T05:01:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.398+0000 7fc81bc9db80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc8081ab9e0 con 0x5596c866c6a0 2024-05-09T05:01:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.398+0000 7fc81bc9db80 10 client.0 _ll_register_callbacks cb 0x5596c84006c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:01:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.398+0000 7fc8067fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c85d9160 0x5596c8682e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.398+0000 7fc8067fc700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c85d9160 0x5596c8682e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:54686/0 (socket says 172.21.15.46:54686) 2024-05-09T05:01:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.398+0000 7fc8067fc700 1 -- 172.21.15.46:0/3270256714 learned_addr learned my addr 172.21.15.46:0/3270256714 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.398+0000 7fc8077fe700 1 --2- 172.21.15.46:0/3270256714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c866c6a0 0x5596c867d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.398+0000 7fc806ffd700 1 --2- 172.21.15.46:0/3270256714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c84ef4a0 0x5596c86828f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.398+0000 7fc8067fc700 1 -- 172.21.15.46:0/3270256714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c866c6a0 msgr2=0x5596c867d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.399+0000 7fc8067fc700 1 --2- 172.21.15.46:0/3270256714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c866c6a0 0x5596c867d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.399+0000 7fc8067fc700 1 -- 172.21.15.46:0/3270256714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c84ef4a0 msgr2=0x5596c86828f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:33.403 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.399+0000 7fc8067fc700 1 --2- 172.21.15.46:0/3270256714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c84ef4a0 0x5596c86828f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.399+0000 7fc8067fc700 1 -- 172.21.15.46:0/3270256714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596c867df50 con 0x5596c85d9160 2024-05-09T05:01:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.399+0000 7fc8067fc700 1 --2- 172.21.15.46:0/3270256714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c85d9160 0x5596c8682e30 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0e1130 tx=0x7fc7fc0a9f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca52c2e8d5b5272 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.399+0000 7fc7eaff5700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:01:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.399+0000 7fc7eaff5700 1 -- 172.21.15.46:0/3270256714 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc06b030 con 0x5596c85d9160 2024-05-09T05:01:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.399+0000 7fc7eaff5700 1 -- 172.21.15.46:0/3270256714 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7fc7fc0a6d40 con 0x5596c85d9160 2024-05-09T05:01:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.399+0000 7fc81bc9db80 1 -- 172.21.15.46:0/3270256714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7fc0aa750 con 0x5596c85d9160 2024-05-09T05:01:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.399+0000 7fc7eaff5700 1 -- 172.21.15.46:0/3270256714 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fc7fc0aae50 con 0x5596c85d9160 2024-05-09T05:01:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.399+0000 7fc81bc9db80 10 client.17529 fetch_fsmap learned FSMap version 25 2024-05-09T05:01:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.399+0000 7fc81bc9db80 1 -- 172.21.15.46:0/3270256714 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5596c86f5850 con 0x5596c85d9160 2024-05-09T05:01:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.399+0000 7fc7eaff5700 1 -- 172.21.15.46:0/3270256714 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7fc0aa750 con 0x5596c85d9160 2024-05-09T05:01:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.399+0000 7fc7eaff5700 1 -- 172.21.15.46:0/3270256714 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7fc7fc0e3070 con 0x5596c85d9160 2024-05-09T05:01:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.400+0000 7fc81bc9db80 10 client.17529 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:01:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.400+0000 7fc81bc9db80 10 client.17529 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:01:33.404 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.400+0000 7fc81bc9db80 20 client.17529 populate_metadata read hostname 'smithi046' 2024-05-09T05:01:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.400+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/3270256714 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5596c868a560 0x5596c86f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.400+0000 7fc81bc9db80 4 client.17529 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-05-09T05:01:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.400+0000 7fc81bc9db80 1 -- 172.21.15.46:0/3270256714 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5596c8596a70 con 0x5596c868a560 2024-05-09T05:01:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.400+0000 7fc806ffd700 1 --2- 172.21.15.46:0/3270256714 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5596c868a560 0x5596c86f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.400+0000 7fc7eaff5700 10 client.17529 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:01:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.400+0000 7fc806ffd700 1 --2- 172.21.15.46:0/3270256714 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5596c868a560 0x5596c86f7e70 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13aa341dc7dec07f server_cookie=d35335802a716041 in_seq=0 out_seq=0 2024-05-09T05:01:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.400+0000 7fc7eaff5700 1 -- 172.21.15.46:0/3270256714 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5596c8596a70 con 0x5596c868a560 2024-05-09T05:01:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.400+0000 7fc7eaff5700 10 client.17529 handle_command_reply: tid=0 2024-05-09T05:01:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.400+0000 7fc81bc9db80 1 client.17529 shutdown 2024-05-09T05:01:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.401+0000 7fc81bc9db80 1 -- 172.21.15.46:0/3270256714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c85d9160 msgr2=0x5596c8682e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.401+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/3270256714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c85d9160 0x5596c8682e30 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0e1130 tx=0x7fc7fc0a9f70 comp rx=0 tx=0).stop 2024-05-09T05:01:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.401+0000 7fc81bc9db80 1 -- 172.21.15.46:0/3270256714 shutdown_connections 2024-05-09T05:01:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.401+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/3270256714 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x5596c866c6a0 0x5596c867d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.405 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.401+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/3270256714 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x5596c868a560 0x5596c86f7e70 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.401+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/3270256714 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x5596c84ef4a0 0x5596c86828f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.401+0000 7fc7eaff5700 0 client.17529 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:01:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.401+0000 7fc81bc9db80 1 --2- 172.21.15.46:0/3270256714 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x5596c85d9160 0x5596c8682e30 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.401+0000 7fc81bc9db80 1 -- 172.21.15.46:0/3270256714 >> 172.21.15.46:0/3270256714 conn(0x5596c854f6a0 msgr2=0x5596c866fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.401+0000 7fc81bc9db80 1 -- 172.21.15.46:0/3270256714 shutdown_connections 2024-05-09T05:01:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.401+0000 7fc81bc9db80 1 -- 172.21.15.46:0/3270256714 wait complete. 2024-05-09T05:01:33.406 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.401+0000 7fc81bc9db80 20 client.17529 trim_cache size 0 max 16384 2024-05-09T05:01:33.406 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T05:01:33.406 INFO:teuthology.orchestra.run.smithi046.stdout: "status": "no active scrubs running", 2024-05-09T05:01:33.406 INFO:teuthology.orchestra.run.smithi046.stdout: "scrubs": {} 2024-05-09T05:01:33.407 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T05:01:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.402+0000 7fc80d287700 1 -- 172.21.15.46:0/4145740550 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc7f00b29f0 msgr2=0x7fc7f00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.402+0000 7fc80d287700 1 --2- 172.21.15.46:0/4145740550 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc7f00b29f0 0x7fc7f00b4f00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8007be0 tx=0x7fc7f80073d0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.407 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.402+0000 7fc80d287700 1 -- 172.21.15.46:0/4145740550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc808108f00 msgr2=0x7fc80819b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.402+0000 7fc80d287700 1 --2- 172.21.15.46:0/4145740550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc808108f00 0x7fc80819b3d0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0098e0 tx=0x7fc7fc0057c0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.402+0000 7fc80d287700 1 -- 172.21.15.46:0/4145740550 shutdown_connections 2024-05-09T05:01:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.402+0000 7fc80d287700 1 --2- 172.21.15.46:0/4145740550 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc7f00b29f0 0x7fc7f00b4f00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.402+0000 7fc80d287700 1 --2- 172.21.15.46:0/4145740550 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc808109850 0x7fc80819f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.402+0000 7fc80d287700 1 --2- 172.21.15.46:0/4145740550 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc8081045d0 0x7fc80819ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.402+0000 7fc80d287700 1 --2- 172.21.15.46:0/4145740550 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc808108f00 0x7fc80819b3d0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.402+0000 7fc80d287700 1 -- 172.21.15.46:0/4145740550 >> 172.21.15.46:0/4145740550 conn(0x7fc8080ffea0 msgr2=0x7fc80810a7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.402+0000 7fc80d287700 1 -- 172.21.15.46:0/4145740550 shutdown_connections 2024-05-09T05:01:33.408 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:33.402+0000 7fc80d287700 1 -- 172.21.15.46:0/4145740550 wait complete. 2024-05-09T05:01:34.111 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-05-09T05:01:34.112 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:01:34.409 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:01:35.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:34 smithi046 ceph-mon[30136]: pgmap v325: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-09T05:01:35.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:34 smithi072 ceph-mon[34088]: pgmap v325: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-09T05:01:35.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:34 smithi110 ceph-mon[34904]: pgmap v325: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 0 B/s wr, 7 op/s 2024-05-09T05:01:36.155 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.153+0000 7f5ffb52a700 1 -- 172.21.15.46:0/1656097114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff4108760 msgr2=0x7f5ff41100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.153+0000 7f5ffb52a700 1 --2- 172.21.15.46:0/1656097114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff4108760 0x7f5ff41100c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f5fec00b0d0 tx=0x7f5fec00b3e0 comp rx=0 tx=0).stop 2024-05-09T05:01:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.153+0000 7f5ffb52a700 1 -- 172.21.15.46:0/1656097114 shutdown_connections 2024-05-09T05:01:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.153+0000 7f5ffb52a700 1 --2- 172.21.15.46:0/1656097114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff4108760 0x7f5ff41100c0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.153+0000 7f5ffb52a700 1 --2- 172.21.15.46:0/1656097114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ff41034e0 0x7f5ff41038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.153+0000 7f5ffb52a700 1 --2- 172.21.15.46:0/1656097114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ff4107e10 0x7f5ff4108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.153+0000 7f5ffb52a700 1 -- 172.21.15.46:0/1656097114 >> 172.21.15.46:0/1656097114 conn(0x7f5ff40fef30 msgr2=0x7f5ff4101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.153+0000 7f5ffb52a700 1 -- 172.21.15.46:0/1656097114 shutdown_connections 2024-05-09T05:01:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.153+0000 7f5ffb52a700 1 -- 172.21.15.46:0/1656097114 wait complete. 2024-05-09T05:01:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.153+0000 7f5ffb52a700 1 Processor -- start 2024-05-09T05:01:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ffb52a700 1 -- start start 2024-05-09T05:01:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ffb52a700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ff41034e0 0x7f5ff40ff410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ffb52a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff4107e10 0x7f5ff40ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ffb52a700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ff4108760 0x7f5ff40ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:36.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ffb52a700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4113390 con 0x7f5ff4107e10 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ffb52a700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4113210 con 0x7f5ff41034e0 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ffb52a700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5ff4113510 con 0x7f5ff4108760 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ff8ac5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff4107e10 0x7f5ff40ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ff92c6700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ff41034e0 0x7f5ff40ff410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ff8ac5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff4107e10 0x7f5ff40ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34592/0 (socket says 172.21.15.46:34592) 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ff8ac5700 1 -- 172.21.15.46:0/1882490053 learned_addr learned my addr 172.21.15.46:0/1882490053 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ff9ac7700 1 --2- 172.21.15.46:0/1882490053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ff4108760 0x7f5ff40ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ff8ac5700 1 -- 172.21.15.46:0/1882490053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ff4108760 msgr2=0x7f5ff40ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ff8ac5700 1 --2- 172.21.15.46:0/1882490053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ff4108760 0x7f5ff40ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ff8ac5700 1 -- 172.21.15.46:0/1882490053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ff41034e0 msgr2=0x7f5ff40ff410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ff8ac5700 1 --2- 172.21.15.46:0/1882490053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ff41034e0 0x7f5ff40ff410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ff8ac5700 1 -- 172.21.15.46:0/1882490053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ff41a9190 con 0x7f5ff4107e10 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.154+0000 7f5ff92c6700 1 --2- 172.21.15.46:0/1882490053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ff41034e0 0x7f5ff40ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.155+0000 7f5ff8ac5700 1 --2- 172.21.15.46:0/1882490053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff4107e10 0x7f5ff40ff950 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c8f0 tx=0x7f5fe400cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.155+0000 7f5fea7fc700 1 -- 172.21.15.46:0/1882490053 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe40078d0 con 0x7f5ff4107e10 2024-05-09T05:01:36.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.155+0000 7f5ffb52a700 1 -- 172.21.15.46:0/1882490053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ff41a9480 con 0x7f5ff4107e10 2024-05-09T05:01:36.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.155+0000 7f5fea7fc700 1 -- 172.21.15.46:0/1882490053 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fe400f450 con 0x7f5ff4107e10 2024-05-09T05:01:36.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.155+0000 7f5fea7fc700 1 -- 172.21.15.46:0/1882490053 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fe4018660 con 0x7f5ff4107e10 2024-05-09T05:01:36.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.155+0000 7f5ffb52a700 1 -- 172.21.15.46:0/1882490053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ff41a99c0 con 0x7f5ff4107e10 2024-05-09T05:01:36.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.156+0000 7f5ffb52a700 1 -- 172.21.15.46:0/1882490053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ff40512c0 con 0x7f5ff4107e10 2024-05-09T05:01:36.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.158+0000 7f5fea7fc700 1 -- 172.21.15.46:0/1882490053 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5fe400f900 con 0x7f5ff4107e10 2024-05-09T05:01:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.159+0000 7f5fea7fc700 1 --2- 172.21.15.46:0/1882490053 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5fe00b29f0 0x7f5fe00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.159+0000 7f5ff92c6700 1 --2- 172.21.15.46:0/1882490053 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5fe00b29f0 0x7f5fe00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.159+0000 7f5fea7fc700 1 -- 172.21.15.46:0/1882490053 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7f5fe40e24f0 con 0x7f5ff4107e10 2024-05-09T05:01:36.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.159+0000 7f5ff92c6700 1 --2- 172.21.15.46:0/1882490053 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5fe00b29f0 0x7f5fe00b4f00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0007fd0 tx=0x7f5ff00073d0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:36.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.162+0000 7f5fea7fc700 1 -- 172.21.15.46:0/1882490053 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5fe4014030 con 0x7f5ff4107e10 2024-05-09T05:01:36.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.423+0000 7f5ffb52a700 1 -- 172.21.15.46:0/1882490053 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ff402e5f0 con 0x7f5ff4107e10 2024-05-09T05:01:36.424 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.424+0000 7f5fea7fc700 1 -- 172.21.15.46:0/1882490053 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7f5fe40a9eb0 con 0x7f5ff4107e10 2024-05-09T05:01:36.424 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T05:01:36.425 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T05:01:36.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.427+0000 7f5ffb52a700 1 -- 172.21.15.46:0/1882490053 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5fe00b29f0 msgr2=0x7f5fe00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:36.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.427+0000 7f5ffb52a700 1 --2- 172.21.15.46:0/1882490053 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5fe00b29f0 0x7f5fe00b4f00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5ff0007fd0 tx=0x7f5ff00073d0 comp rx=0 tx=0).stop 2024-05-09T05:01:36.427 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.427+0000 7f5ffb52a700 1 -- 172.21.15.46:0/1882490053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff4107e10 msgr2=0x7f5ff40ff950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.427+0000 7f5ffb52a700 1 --2- 172.21.15.46:0/1882490053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff4107e10 0x7f5ff40ff950 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f5fe400c8f0 tx=0x7f5fe400cc00 comp rx=0 tx=0).stop 2024-05-09T05:01:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.427+0000 7f5ffb52a700 1 -- 172.21.15.46:0/1882490053 shutdown_connections 2024-05-09T05:01:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.427+0000 7f5ffb52a700 1 --2- 172.21.15.46:0/1882490053 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5fe00b29f0 0x7f5fe00b4f00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.427+0000 7f5ffb52a700 1 --2- 172.21.15.46:0/1882490053 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5ff4108760 0x7f5ff40ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.427+0000 7f5ffb52a700 1 --2- 172.21.15.46:0/1882490053 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5ff41034e0 0x7f5ff40ff410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.427+0000 7f5ffb52a700 1 --2- 172.21.15.46:0/1882490053 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5ff4107e10 0x7f5ff40ff950 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.427+0000 7f5ffb52a700 1 -- 172.21.15.46:0/1882490053 >> 172.21.15.46:0/1882490053 conn(0x7f5ff40fef30 msgr2=0x7f5ff406c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.427+0000 7f5ffb52a700 1 -- 172.21.15.46:0/1882490053 shutdown_connections 2024-05-09T05:01:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:36.427+0000 7f5ffb52a700 1 -- 172.21.15.46:0/1882490053 wait complete. 2024-05-09T05:01:36.428 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T05:01:37.075 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:36 smithi046 ceph-mon[30136]: pgmap v326: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:37.076 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:36 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1882490053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:01:37.105 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-05-09T05:01:37.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:36 smithi072 ceph-mon[34088]: pgmap v326: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:37.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:36 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1882490053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:01:37.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:36 smithi110 ceph-mon[34904]: pgmap v326: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:37.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:36 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1882490053' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:01:37.359 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:01:39.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:38 smithi046 ceph-mon[30136]: pgmap v327: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:01:39.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.082+0000 7f5dba8e8700 1 -- 172.21.15.46:0/3463751947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db41034c0 msgr2=0x7f5db41038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:39.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.082+0000 7f5dba8e8700 1 --2- 172.21.15.46:0/3463751947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db41034c0 0x7f5db41038d0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f5da40099c0 tx=0x7f5da4009cd0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.083+0000 7f5dba8e8700 1 -- 172.21.15.46:0/3463751947 shutdown_connections 2024-05-09T05:01:39.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.083+0000 7f5dba8e8700 1 --2- 172.21.15.46:0/3463751947 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4108740 0x7f5db41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.083+0000 7f5dba8e8700 1 --2- 172.21.15.46:0/3463751947 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db41034c0 0x7f5db41038d0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.083 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.083+0000 7f5dba8e8700 1 --2- 172.21.15.46:0/3463751947 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5db4107df0 0x7f5db4108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.083+0000 7f5dba8e8700 1 -- 172.21.15.46:0/3463751947 >> 172.21.15.46:0/3463751947 conn(0x7f5db40fedc0 msgr2=0x7f5db41011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.083+0000 7f5dba8e8700 1 -- 172.21.15.46:0/3463751947 shutdown_connections 2024-05-09T05:01:39.084 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.083+0000 7f5dba8e8700 1 -- 172.21.15.46:0/3463751947 wait complete. 2024-05-09T05:01:39.085 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.083+0000 7f5dba8e8700 1 Processor -- start 2024-05-09T05:01:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.083+0000 7f5dba8e8700 1 -- start start 2024-05-09T05:01:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5dba8e8700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5db41034c0 0x7f5db419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5dba8e8700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4107df0 0x7f5db419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5dba8e8700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4108740 0x7f5db419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5dba8e8700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41144c0 con 0x7f5db4108740 2024-05-09T05:01:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5dba8e8700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4114340 con 0x7f5db41034c0 2024-05-09T05:01:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5dba8e8700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5db4114640 con 0x7f5db4107df0 2024-05-09T05:01:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5db8e85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4108740 0x7f5db419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5db8e85700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4108740 0x7f5db419f600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34606/0 (socket says 172.21.15.46:34606) 2024-05-09T05:01:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5db8e85700 1 -- 172.21.15.46:0/4067736396 learned_addr learned my addr 172.21.15.46:0/4067736396 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5db3fff700 1 --2- 172.21.15.46:0/4067736396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5db41034c0 0x7f5db419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5dabfff700 1 --2- 172.21.15.46:0/4067736396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4107df0 0x7f5db419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.086 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5db8e85700 1 -- 172.21.15.46:0/4067736396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4107df0 msgr2=0x7f5db419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5db8e85700 1 --2- 172.21.15.46:0/4067736396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4107df0 0x7f5db419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5db8e85700 1 -- 172.21.15.46:0/4067736396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5db41034c0 msgr2=0x7f5db419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5db8e85700 1 --2- 172.21.15.46:0/4067736396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5db41034c0 0x7f5db419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5db8e85700 1 -- 172.21.15.46:0/4067736396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db419fca0 con 0x7f5db4108740 2024-05-09T05:01:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.084+0000 7f5db8e85700 1 --2- 172.21.15.46:0/4067736396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4108740 0x7f5db419f600 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f5dac007ee0 tx=0x7f5dac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.085+0000 7f5db1ffb700 1 -- 172.21.15.46:0/4067736396 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac004bb0 con 0x7f5db4108740 2024-05-09T05:01:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.085+0000 7f5db1ffb700 1 -- 172.21.15.46:0/4067736396 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dac004d50 con 0x7f5db4108740 2024-05-09T05:01:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.085+0000 7f5dba8e8700 1 -- 172.21.15.46:0/4067736396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db419ff90 con 0x7f5db4108740 2024-05-09T05:01:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.085+0000 7f5db1ffb700 1 -- 172.21.15.46:0/4067736396 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dac005920 con 0x7f5db4108740 2024-05-09T05:01:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.085+0000 7f5dba8e8700 1 -- 172.21.15.46:0/4067736396 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db41a78b0 con 0x7f5db4108740 2024-05-09T05:01:39.087 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.087+0000 7f5db1ffb700 1 -- 172.21.15.46:0/4067736396 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5dac01a020 con 0x7f5db4108740 2024-05-09T05:01:39.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.088+0000 7f5db1ffb700 1 --2- 172.21.15.46:0/4067736396 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5da00b29f0 0x7f5da00b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.088+0000 7f5db3fff700 1 --2- 172.21.15.46:0/4067736396 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5da00b29f0 0x7f5da00b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.088+0000 7f5db1ffb700 1 -- 172.21.15.46:0/4067736396 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7f5dac0e1cf0 con 0x7f5db4108740 2024-05-09T05:01:39.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.089+0000 7f5dc92feb80 1 Processor -- start 2024-05-09T05:01:39.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.089+0000 7f5dc92feb80 1 -- start start 2024-05-09T05:01:39.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.089+0000 7f5db3fff700 1 --2- 172.21.15.46:0/4067736396 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5da00b29f0 0x7f5da00b4f00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009990 tx=0x7f5da401a040 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:39.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.089+0000 7f5dc92feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450b5fd00 0x564450c68240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.090 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.089+0000 7f5dc92feb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450ae54a0 0x564450c68780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.089+0000 7f5dc92feb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450c62400 0x564450c62810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.089+0000 7f5dc92feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564450b850a0 con 0x564450b5fd00 2024-05-09T05:01:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.089+0000 7f5dc92feb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x564450b2a9a0 con 0x564450c62400 2024-05-09T05:01:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.089+0000 7f5dc92feb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564450b81060 con 0x564450ae54a0 2024-05-09T05:01:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.089+0000 7f5db3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450b5fd00 0x564450c68240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.089+0000 7f5db3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450b5fd00 0x564450c68240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34612/0 (socket says 172.21.15.46:34612) 2024-05-09T05:01:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.089+0000 7f5db3fff700 1 -- 172.21.15.46:0/902203708 learned_addr learned my addr 172.21.15.46:0/902203708 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.089+0000 7f5dabfff700 1 --2- 172.21.15.46:0/902203708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450ae54a0 0x564450c68780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.089+0000 7f5db8e85700 1 --2- 172.21.15.46:0/902203708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450c62400 0x564450c62810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.090+0000 7f5db3fff700 1 -- 172.21.15.46:0/902203708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450ae54a0 msgr2=0x564450c68780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.090+0000 7f5db3fff700 1 --2- 172.21.15.46:0/902203708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450ae54a0 0x564450c68780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.090+0000 7f5db3fff700 1 -- 172.21.15.46:0/902203708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450c62400 msgr2=0x564450c62810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.090+0000 7f5db3fff700 1 --2- 172.21.15.46:0/902203708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450c62400 0x564450c62810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.091 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.090+0000 7f5db3fff700 1 -- 172.21.15.46:0/902203708 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564450b1b500 con 0x564450b5fd00 2024-05-09T05:01:39.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.090+0000 7f5db3fff700 1 --2- 172.21.15.46:0/902203708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450b5fd00 0x564450c68240 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5da4015000 tx=0x7f5da401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b79d8d06b2b8228f server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:39.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.090+0000 7f5daa7fc700 1 -- 172.21.15.46:0/902203708 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4032070 con 0x564450b5fd00 2024-05-09T05:01:39.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.090+0000 7f5daa7fc700 1 -- 172.21.15.46:0/902203708 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5da400b790 con 0x564450b5fd00 2024-05-09T05:01:39.092 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.090+0000 7f5daa7fc700 1 -- 172.21.15.46:0/902203708 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da400f6a0 con 0x564450b5fd00 2024-05-09T05:01:39.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.091+0000 7f5dc92feb80 1 -- 172.21.15.46:0/902203708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450b5fd00 msgr2=0x564450c68240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:39.094 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.091+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/902203708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450b5fd00 0x564450c68240 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5da4015000 tx=0x7f5da401f000 comp rx=0 tx=0).stop 2024-05-09T05:01:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.091+0000 7f5dc92feb80 1 -- 172.21.15.46:0/902203708 shutdown_connections 2024-05-09T05:01:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.091+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/902203708 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450c62400 0x564450c62810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.091+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/902203708 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450b5fd00 0x564450c68240 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.091+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/902203708 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450ae54a0 0x564450c68780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.091+0000 7f5dc92feb80 1 -- 172.21.15.46:0/902203708 >> 172.21.15.46:0/902203708 conn(0x564450ae8f70 msgr2=0x564450c5d220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.091+0000 7f5dc92feb80 1 -- 172.21.15.46:0/902203708 shutdown_connections 2024-05-09T05:01:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.091+0000 7f5dc92feb80 1 -- 172.21.15.46:0/902203708 wait complete. 2024-05-09T05:01:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.091+0000 7f5dc92feb80 1 -- start start 2024-05-09T05:01:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.092+0000 7f5dc92feb80 1 Processor -- start 2024-05-09T05:01:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.092+0000 7f5dc92feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae54a0 0x564450c6bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.092+0000 7f5dc92feb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450b5fd00 0x564450c6c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.092+0000 7f5dc92feb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450c62400 0x564450c6ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.095 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.092+0000 7f5dc92feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x564450c63100 con 0x564450ae54a0 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.092+0000 7f5dc92feb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x564450b1fb20 con 0x564450c62400 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.092+0000 7f5dc92feb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x564450c78200 con 0x564450b5fd00 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.092+0000 7f5dc92feb80 10 client.0 _ll_register_callbacks cb 0x5644509f66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.092+0000 7f5db3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae54a0 0x564450c6bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.092+0000 7f5db3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae54a0 0x564450c6bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34626/0 (socket says 172.21.15.46:34626) 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.092+0000 7f5db3fff700 1 -- 172.21.15.46:0/2008459310 learned_addr learned my addr 172.21.15.46:0/2008459310 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.092+0000 7f5dabfff700 1 --2- 172.21.15.46:0/2008459310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450b5fd00 0x564450c6c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.093+0000 7f5db8e85700 1 --2- 172.21.15.46:0/2008459310 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450c62400 0x564450c6ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.093+0000 7f5db3fff700 1 -- 172.21.15.46:0/2008459310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450b5fd00 msgr2=0x564450c6c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.093+0000 7f5db3fff700 1 --2- 172.21.15.46:0/2008459310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450b5fd00 0x564450c6c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.093+0000 7f5db3fff700 1 -- 172.21.15.46:0/2008459310 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450c62400 msgr2=0x564450c6ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.093+0000 7f5db3fff700 1 --2- 172.21.15.46:0/2008459310 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450c62400 0x564450c6ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.093+0000 7f5db3fff700 1 -- 172.21.15.46:0/2008459310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564450a00490 con 0x564450ae54a0 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.093+0000 7f5db3fff700 1 --2- 172.21.15.46:0/2008459310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae54a0 0x564450c6bfc0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5da4005b40 tx=0x7f5da4009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91c4bcdd4f865198 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.093+0000 7f5d937fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:01:39.096 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.093+0000 7f5d937fe700 1 -- 172.21.15.46:0/2008459310 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4032050 con 0x564450ae54a0 2024-05-09T05:01:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.093+0000 7f5d937fe700 1 -- 172.21.15.46:0/2008459310 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7f5da4005580 con 0x564450ae54a0 2024-05-09T05:01:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.093+0000 7f5dc92feb80 1 -- 172.21.15.46:0/2008459310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5da400b790 con 0x564450ae54a0 2024-05-09T05:01:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.093+0000 7f5d937fe700 1 -- 172.21.15.46:0/2008459310 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5da4044050 con 0x564450ae54a0 2024-05-09T05:01:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.093+0000 7f5dc92feb80 10 client.17547 fetch_fsmap learned FSMap version 25 2024-05-09T05:01:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.093+0000 7f5dc92feb80 1 -- 172.21.15.46:0/2008459310 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564450c79ad0 con 0x564450ae54a0 2024-05-09T05:01:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.093+0000 7f5d937fe700 1 -- 172.21.15.46:0/2008459310 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5da400b790 con 0x564450ae54a0 2024-05-09T05:01:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.094+0000 7f5d937fe700 1 -- 172.21.15.46:0/2008459310 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5da403f030 con 0x564450ae54a0 2024-05-09T05:01:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.094+0000 7f5dc92feb80 10 client.17547 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:01:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.094+0000 7f5dc92feb80 10 client.17547 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T05:01:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.094+0000 7f5dc92feb80 20 client.17547 populate_metadata read hostname 'smithi046' 2024-05-09T05:01:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.094+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/2008459310 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564450c7bfa0 0x564450c7e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.094+0000 7f5dc92feb80 4 client.17547 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:01:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.094+0000 7f5dc92feb80 1 -- 172.21.15.46:0/2008459310 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564450b92a80 con 0x564450c7bfa0 2024-05-09T05:01:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.094+0000 7f5dabfff700 1 --2- 172.21.15.46:0/2008459310 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564450c7bfa0 0x564450c7e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.097 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.095+0000 7f5d937fe700 10 client.17547 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T05:01:39.098 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.095+0000 7f5dabfff700 1 --2- 172.21.15.46:0/2008459310 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564450c7bfa0 0x564450c7e3c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f2b2f7eb3c9cf7a9 server_cookie=77ca41471fdff6a4 in_seq=0 out_seq=0 2024-05-09T05:01:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.095+0000 7f5d937fe700 1 -- 172.21.15.46:0/2008459310 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564450b92a80 con 0x564450c7bfa0 2024-05-09T05:01:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.095+0000 7f5d937fe700 10 client.17547 handle_command_reply: tid=0 2024-05-09T05:01:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.095+0000 7f5dc92feb80 1 client.17547 shutdown 2024-05-09T05:01:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.095+0000 7f5dc92feb80 1 -- 172.21.15.46:0/2008459310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae54a0 msgr2=0x564450c6bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.095+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/2008459310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae54a0 0x564450c6bfc0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5da4005b40 tx=0x7f5da4009290 comp rx=0 tx=0).stop 2024-05-09T05:01:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.096+0000 7f5dc92feb80 1 -- 172.21.15.46:0/2008459310 shutdown_connections 2024-05-09T05:01:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.096+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/2008459310 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450c62400 0x564450c6ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.096+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/2008459310 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae54a0 0x564450c6bfc0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.096+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/2008459310 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564450c7bfa0 0x564450c7e3c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.096+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/2008459310 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450b5fd00 0x564450c6c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.096+0000 7f5dc92feb80 1 -- 172.21.15.46:0/2008459310 >> 172.21.15.46:0/2008459310 conn(0x564450ae8f70 msgr2=0x564450c56ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.096+0000 7f5d937fe700 0 client.17547 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T05:01:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.096+0000 7f5dc92feb80 1 -- 172.21.15.46:0/2008459310 shutdown_connections 2024-05-09T05:01:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.096+0000 7f5dc92feb80 1 -- 172.21.15.46:0/2008459310 wait complete. 2024-05-09T05:01:39.099 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.096+0000 7f5dc92feb80 20 client.17547 trim_cache size 0 max 16384 2024-05-09T05:01:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.111+0000 7f5dc92feb80 1 Processor -- start 2024-05-09T05:01:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.111+0000 7f5dc92feb80 1 -- start start 2024-05-09T05:01:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5dc92feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae8f70 0x564450c57a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5dc92feb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450ae54a0 0x564450c6af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5dc92feb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450b5fd00 0x564450c6c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5dc92feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41b1850 con 0x564450ae8f70 2024-05-09T05:01:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5dc92feb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41af0e0 con 0x564450b5fd00 2024-05-09T05:01:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5dc92feb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41acf50 con 0x564450ae54a0 2024-05-09T05:01:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5db3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae8f70 0x564450c57a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.113 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5db3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae8f70 0x564450c57a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34636/0 (socket says 172.21.15.46:34636) 2024-05-09T05:01:39.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5db3fff700 1 -- 172.21.15.46:0/1056800114 learned_addr learned my addr 172.21.15.46:0/1056800114 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:39.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5db8e85700 1 --2- 172.21.15.46:0/1056800114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450b5fd00 0x564450c6c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5dabfff700 1 --2- 172.21.15.46:0/1056800114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450ae54a0 0x564450c6af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5db3fff700 1 -- 172.21.15.46:0/1056800114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450ae54a0 msgr2=0x564450c6af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:39.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5db3fff700 1 --2- 172.21.15.46:0/1056800114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450ae54a0 0x564450c6af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5db3fff700 1 -- 172.21.15.46:0/1056800114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450b5fd00 msgr2=0x564450c6c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:39.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5db3fff700 1 --2- 172.21.15.46:0/1056800114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450b5fd00 0x564450c6c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5db3fff700 1 -- 172.21.15.46:0/1056800114 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564450c599d0 con 0x564450ae8f70 2024-05-09T05:01:39.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5dabfff700 1 --2- 172.21.15.46:0/1056800114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450ae54a0 0x564450c6af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:01:39.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.112+0000 7f5db3fff700 1 --2- 172.21.15.46:0/1056800114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae8f70 0x564450c57a20 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009d10 tx=0x7f5da4005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93bc017469196c42 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:39.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.113+0000 7f5da9ffb700 1 -- 172.21.15.46:0/1056800114 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4032040 con 0x564450ae8f70 2024-05-09T05:01:39.114 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.113+0000 7f5da9ffb700 1 -- 172.21.15.46:0/1056800114 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5da4033040 con 0x564450ae8f70 2024-05-09T05:01:39.115 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.113+0000 7f5da9ffb700 1 -- 172.21.15.46:0/1056800114 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da402d9a0 con 0x564450ae8f70 2024-05-09T05:01:39.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.113+0000 7f5dc92feb80 1 -- 172.21.15.46:0/1056800114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae8f70 msgr2=0x564450c57a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:39.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.114+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/1056800114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae8f70 0x564450c57a20 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009d10 tx=0x7f5da4005a90 comp rx=0 tx=0).stop 2024-05-09T05:01:39.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.114+0000 7f5dc92feb80 1 -- 172.21.15.46:0/1056800114 shutdown_connections 2024-05-09T05:01:39.118 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.114+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/1056800114 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450b5fd00 0x564450c6c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.114+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/1056800114 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae8f70 0x564450c57a20 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.114+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/1056800114 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450ae54a0 0x564450c6af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.114+0000 7f5dc92feb80 1 -- 172.21.15.46:0/1056800114 >> 172.21.15.46:0/1056800114 conn(0x564450c62ae0 msgr2=0x564450c65c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.114+0000 7f5dc92feb80 1 -- 172.21.15.46:0/1056800114 shutdown_connections 2024-05-09T05:01:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.114+0000 7f5dc92feb80 1 -- 172.21.15.46:0/1056800114 wait complete. 2024-05-09T05:01:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.114+0000 7f5dc92feb80 1 -- start start 2024-05-09T05:01:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.114+0000 7f5dc92feb80 1 Processor -- start 2024-05-09T05:01:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5dc92feb80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae54a0 0x564450cdd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5dc92feb80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450ae8f70 0x564450cdde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5dc92feb80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450b5fd00 0x564450c7e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5dc92feb80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41ad5a0 con 0x564450ae54a0 2024-05-09T05:01:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5dc92feb80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41aea10 con 0x564450ae8f70 2024-05-09T05:01:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5dc92feb80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f5db41adc70 con 0x564450b5fd00 2024-05-09T05:01:39.119 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5dc92feb80 10 client.0 _ll_register_callbacks cb 0x5644509f66c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:01:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5db3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae54a0 0x564450cdd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5db3fff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae54a0 0x564450cdd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34652/0 (socket says 172.21.15.46:34652) 2024-05-09T05:01:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5db3fff700 1 -- 172.21.15.46:0/77399773 learned_addr learned my addr 172.21.15.46:0/77399773 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5dabfff700 1 --2- 172.21.15.46:0/77399773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450ae8f70 0x564450cdde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5db8e85700 1 --2- 172.21.15.46:0/77399773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450b5fd00 0x564450c7e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5db3fff700 1 -- 172.21.15.46:0/77399773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450b5fd00 msgr2=0x564450c7e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5db3fff700 1 --2- 172.21.15.46:0/77399773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450b5fd00 0x564450c7e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5db3fff700 1 -- 172.21.15.46:0/77399773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450ae8f70 msgr2=0x564450cdde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5db3fff700 1 --2- 172.21.15.46:0/77399773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450ae8f70 0x564450cdde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5db3fff700 1 -- 172.21.15.46:0/77399773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564450c7efa0 con 0x564450ae54a0 2024-05-09T05:01:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5db3fff700 1 --2- 172.21.15.46:0/77399773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae54a0 0x564450cdd8e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5da40098e0 tx=0x7f5da4037bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=753957ecba9371ff server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.115+0000 7f5d937fe700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:01:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.116+0000 7f5d937fe700 1 -- 172.21.15.46:0/77399773 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5da4032040 con 0x564450ae54a0 2024-05-09T05:01:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.116+0000 7f5d937fe700 1 -- 172.21.15.46:0/77399773 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7f5da400f460 con 0x564450ae54a0 2024-05-09T05:01:39.120 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.116+0000 7f5dc92feb80 1 -- 172.21.15.46:0/77399773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5da4033040 con 0x564450ae54a0 2024-05-09T05:01:39.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.116+0000 7f5d937fe700 1 -- 172.21.15.46:0/77399773 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f5da400f660 con 0x564450ae54a0 2024-05-09T05:01:39.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.116+0000 7f5dc92feb80 10 client.17553 fetch_fsmap learned FSMap version 25 2024-05-09T05:01:39.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.116+0000 7f5dc92feb80 1 -- 172.21.15.46:0/77399773 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x564450ce5f20 con 0x564450ae54a0 2024-05-09T05:01:39.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.116+0000 7f5d937fe700 1 -- 172.21.15.46:0/77399773 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5da4033040 con 0x564450ae54a0 2024-05-09T05:01:39.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.116+0000 7f5d937fe700 1 -- 172.21.15.46:0/77399773 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f5da403f070 con 0x564450ae54a0 2024-05-09T05:01:39.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.116+0000 7f5dc92feb80 10 client.17553 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:01:39.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.116+0000 7f5dc92feb80 10 client.17553 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2024-05-09T05:01:39.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.116+0000 7f5dc92feb80 20 client.17553 populate_metadata read hostname 'smithi046' 2024-05-09T05:01:39.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.116+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/77399773 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564450c62400 0x564450ce8480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:39.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.116+0000 7f5dc92feb80 4 client.17553 mds_command: new command op to 24448 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:01:39.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.116+0000 7f5dc92feb80 1 -- 172.21.15.46:0/77399773 --> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564450b8cb20 con 0x564450c62400 2024-05-09T05:01:39.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.117+0000 7f5dabfff700 1 --2- 172.21.15.46:0/77399773 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564450c62400 0x564450ce8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:39.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.117+0000 7f5d937fe700 10 client.17553 ms_handle_connect on v2:172.21.15.110:6838/556391603 2024-05-09T05:01:39.121 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.117+0000 7f5dabfff700 1 --2- 172.21.15.46:0/77399773 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564450c62400 0x564450ce8480 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7da0ffd33374f6bf server_cookie=445ccf2f40da3d4e in_seq=0 out_seq=0 2024-05-09T05:01:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.117+0000 7f5d937fe700 1 -- 172.21.15.46:0/77399773 <== mds.2 v2:172.21.15.110:6838/556391603 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564450b8cb20 con 0x564450c62400 2024-05-09T05:01:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.117+0000 7f5d937fe700 10 client.17553 handle_command_reply: tid=0 2024-05-09T05:01:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.117+0000 7f5dc92feb80 1 client.17553 shutdown 2024-05-09T05:01:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.117+0000 7f5dc92feb80 1 -- 172.21.15.46:0/77399773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae54a0 msgr2=0x564450cdd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.117+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/77399773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae54a0 0x564450cdd8e0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5da40098e0 tx=0x7f5da4037bd0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.117+0000 7f5dc92feb80 1 -- 172.21.15.46:0/77399773 shutdown_connections 2024-05-09T05:01:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.117+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/77399773 >> [v2:172.21.15.110:6838/556391603,v1:172.21.15.110:6839/556391603] conn(0x564450c62400 0x564450ce8480 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.117+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/77399773 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x564450b5fd00 0x564450c7e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.117+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/77399773 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x564450ae54a0 0x564450cdd8e0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.117+0000 7f5dc92feb80 1 --2- 172.21.15.46:0/77399773 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x564450ae8f70 0x564450cdde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.117+0000 7f5d937fe700 0 client.17553 ms_handle_reset on v2:172.21.15.110:6838/556391603 2024-05-09T05:01:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.117+0000 7f5dc92feb80 1 -- 172.21.15.46:0/77399773 >> 172.21.15.46:0/77399773 conn(0x564450c62ae0 msgr2=0x564450c57780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.118+0000 7f5dc92feb80 1 -- 172.21.15.46:0/77399773 shutdown_connections 2024-05-09T05:01:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.118+0000 7f5dc92feb80 1 -- 172.21.15.46:0/77399773 wait complete. 2024-05-09T05:01:39.122 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.118+0000 7f5dc92feb80 20 client.17553 trim_cache size 0 max 16384 2024-05-09T05:01:39.123 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T05:01:39.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.118+0000 7f5dba8e8700 1 -- 172.21.15.46:0/4067736396 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5da00b29f0 msgr2=0x7f5da00b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:39.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.118+0000 7f5dba8e8700 1 --2- 172.21.15.46:0/4067736396 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5da00b29f0 0x7f5da00b4f00 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5da4009990 tx=0x7f5da401a040 comp rx=0 tx=0).stop 2024-05-09T05:01:39.123 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.118+0000 7f5dba8e8700 1 -- 172.21.15.46:0/4067736396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4108740 msgr2=0x7f5db419f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.118+0000 7f5dba8e8700 1 --2- 172.21.15.46:0/4067736396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4108740 0x7f5db419f600 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f5dac007ee0 tx=0x7f5dac00d3b0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.118+0000 7f5dba8e8700 1 -- 172.21.15.46:0/4067736396 shutdown_connections 2024-05-09T05:01:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.118+0000 7f5dba8e8700 1 --2- 172.21.15.46:0/4067736396 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f5da00b29f0 0x7f5da00b4f00 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.118+0000 7f5dba8e8700 1 --2- 172.21.15.46:0/4067736396 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f5db4108740 0x7f5db419f600 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.118+0000 7f5dba8e8700 1 --2- 172.21.15.46:0/4067736396 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f5db41034c0 0x7f5db419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.118+0000 7f5dba8e8700 1 --2- 172.21.15.46:0/4067736396 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f5db4107df0 0x7f5db419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.118+0000 7f5dba8e8700 1 -- 172.21.15.46:0/4067736396 >> 172.21.15.46:0/4067736396 conn(0x7f5db40fedc0 msgr2=0x7f5db41075e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.118+0000 7f5dba8e8700 1 -- 172.21.15.46:0/4067736396 shutdown_connections 2024-05-09T05:01:39.124 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:39.118+0000 7f5dba8e8700 1 -- 172.21.15.46:0/4067736396 wait complete. 2024-05-09T05:01:39.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:38 smithi072 ceph-mon[34088]: pgmap v327: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:01:39.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:38 smithi110 ceph-mon[34904]: pgmap v327: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:01:39.773 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-05-09T05:01:40.028 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:01:41.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:40 smithi072 ceph-mon[34088]: pgmap v328: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:41.234 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:40 smithi046 ceph-mon[30136]: pgmap v328: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:41.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:40 smithi110 ceph-mon[34904]: pgmap v328: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.807+0000 7ff28759f700 1 -- 172.21.15.46:0/4208042035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff280108f40 msgr2=0x7ff280109350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.807+0000 7ff28759f700 1 --2- 172.21.15.46:0/4208042035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff280108f40 0x7ff280109350 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7ff2740099c0 tx=0x7ff274009cd0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.807+0000 7ff28759f700 1 -- 172.21.15.46:0/4208042035 shutdown_connections 2024-05-09T05:01:41.808 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.807+0000 7ff28759f700 1 --2- 172.21.15.46:0/4208042035 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff280109890 0x7ff2801111f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.807+0000 7ff28759f700 1 --2- 172.21.15.46:0/4208042035 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff280102b20 0x7ff280102f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.807+0000 7ff28759f700 1 --2- 172.21.15.46:0/4208042035 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff280108f40 0x7ff280109350 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.807+0000 7ff28759f700 1 -- 172.21.15.46:0/4208042035 >> 172.21.15.46:0/4208042035 conn(0x7ff2800fe540 msgr2=0x7ff280100990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.808+0000 7ff28759f700 1 -- 172.21.15.46:0/4208042035 shutdown_connections 2024-05-09T05:01:41.809 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.808+0000 7ff28759f700 1 -- 172.21.15.46:0/4208042035 wait complete. 2024-05-09T05:01:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.808+0000 7ff28759f700 1 Processor -- start 2024-05-09T05:01:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.808+0000 7ff28759f700 1 -- start start 2024-05-09T05:01:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff28759f700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff280102b20 0x7ff28019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff28759f700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff280108f40 0x7ff28019b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff28759f700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff280109890 0x7ff28019f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.811 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff28759f700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff280114510 con 0x7ff280108f40 2024-05-09T05:01:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff28759f700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff280114390 con 0x7ff280102b20 2024-05-09T05:01:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff28759f700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff280114690 con 0x7ff280109890 2024-05-09T05:01:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff284b3a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff280108f40 0x7ff28019b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff284b3a700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff280108f40 0x7ff28019b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34672/0 (socket says 172.21.15.46:34672) 2024-05-09T05:01:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff284b3a700 1 -- 172.21.15.46:0/1334823677 learned_addr learned my addr 172.21.15.46:0/1334823677 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff28533b700 1 --2- 172.21.15.46:0/1334823677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff280102b20 0x7ff28019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff285b3c700 1 --2- 172.21.15.46:0/1334823677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff280109890 0x7ff28019f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff284b3a700 1 -- 172.21.15.46:0/1334823677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff280109890 msgr2=0x7ff28019f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff284b3a700 1 --2- 172.21.15.46:0/1334823677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff280109890 0x7ff28019f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff284b3a700 1 -- 172.21.15.46:0/1334823677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff280102b20 msgr2=0x7ff28019ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff284b3a700 1 --2- 172.21.15.46:0/1334823677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff280102b20 0x7ff28019ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.812 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff284b3a700 1 -- 172.21.15.46:0/1334823677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff28019fd50 con 0x7ff280108f40 2024-05-09T05:01:41.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.809+0000 7ff28533b700 1 --2- 172.21.15.46:0/1334823677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff280102b20 0x7ff28019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:01:41.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.810+0000 7ff284b3a700 1 --2- 172.21.15.46:0/1334823677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff280108f40 0x7ff28019b420 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ff274006010 tx=0x7ff2740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:41.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.810+0000 7ff27e7fc700 1 -- 172.21.15.46:0/1334823677 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274027070 con 0x7ff280108f40 2024-05-09T05:01:41.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.810+0000 7ff27e7fc700 1 -- 172.21.15.46:0/1334823677 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff274003fa0 con 0x7ff280108f40 2024-05-09T05:01:41.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.810+0000 7ff28759f700 1 -- 172.21.15.46:0/1334823677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff28019ffe0 con 0x7ff280108f40 2024-05-09T05:01:41.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.810+0000 7ff27e7fc700 1 -- 172.21.15.46:0/1334823677 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff274022dd0 con 0x7ff280108f40 2024-05-09T05:01:41.813 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.810+0000 7ff28759f700 1 -- 172.21.15.46:0/1334823677 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2801a04c0 con 0x7ff280108f40 2024-05-09T05:01:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.813+0000 7ff27e7fc700 1 -- 172.21.15.46:0/1334823677 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff27402a070 con 0x7ff280108f40 2024-05-09T05:01:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.813+0000 7ff27e7fc700 1 --2- 172.21.15.46:0/1334823677 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff2680b29f0 0x7ff2680b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.813+0000 7ff28533b700 1 --2- 172.21.15.46:0/1334823677 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff2680b29f0 0x7ff2680b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.813+0000 7ff27e7fc700 1 -- 172.21.15.46:0/1334823677 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7ff2740e33e0 con 0x7ff280108f40 2024-05-09T05:01:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.814+0000 7ff28533b700 1 --2- 172.21.15.46:0/1334823677 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff2680b29f0 0x7ff2680b4f00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff28006bc90 tx=0x7ff27800b5f0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.814+0000 7ff295fb5b80 1 Processor -- start 2024-05-09T05:01:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.814+0000 7ff295fb5b80 1 -- start start 2024-05-09T05:01:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.814+0000 7ff295fb5b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdb19d00 0x55adcdc22240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.814+0000 7ff295fb5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcda9f4a0 0x55adcdc22780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.814+0000 7ff295fb5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcdc1c400 0x55adcdc1c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.816 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.814+0000 7ff295fb5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55adcdb3f0a0 con 0x55adcdc1c400 2024-05-09T05:01:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.814+0000 7ff295fb5b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55adcdae49a0 con 0x55adcdb19d00 2024-05-09T05:01:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.814+0000 7ff295fb5b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55adcdb3b060 con 0x55adcda9f4a0 2024-05-09T05:01:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.815+0000 7ff285b3c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcdc1c400 0x55adcdc1c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.815+0000 7ff285b3c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcdc1c400 0x55adcdc1c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34686/0 (socket says 172.21.15.46:34686) 2024-05-09T05:01:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.815+0000 7ff285b3c700 1 -- 172.21.15.46:0/2583766151 learned_addr learned my addr 172.21.15.46:0/2583766151 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.815+0000 7ff284b3a700 1 --2- 172.21.15.46:0/2583766151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcda9f4a0 0x55adcdc22780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.815+0000 7ff28533b700 1 --2- 172.21.15.46:0/2583766151 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdb19d00 0x55adcdc22240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.815+0000 7ff285b3c700 1 -- 172.21.15.46:0/2583766151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcda9f4a0 msgr2=0x55adcdc22780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.815+0000 7ff285b3c700 1 --2- 172.21.15.46:0/2583766151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcda9f4a0 0x55adcdc22780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.815+0000 7ff285b3c700 1 -- 172.21.15.46:0/2583766151 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdb19d00 msgr2=0x55adcdc22240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.815+0000 7ff285b3c700 1 --2- 172.21.15.46:0/2583766151 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdb19d00 0x55adcdc22240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.817 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.815+0000 7ff285b3c700 1 -- 172.21.15.46:0/2583766151 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adcdad5500 con 0x55adcdc1c400 2024-05-09T05:01:41.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.815+0000 7ff28533b700 1 --2- 172.21.15.46:0/2583766151 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdb19d00 0x55adcdc22240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:01:41.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.815+0000 7ff285b3c700 1 --2- 172.21.15.46:0/2583766151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcdc1c400 0x55adcdc1c810 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff270009f20 tx=0x7ff270009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e044baa1ff7e84b server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:41.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.815+0000 7ff2677fe700 1 -- 172.21.15.46:0/2583766151 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27001a070 con 0x55adcdc1c400 2024-05-09T05:01:41.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.815+0000 7ff2677fe700 1 -- 172.21.15.46:0/2583766151 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff270015410 con 0x55adcdc1c400 2024-05-09T05:01:41.818 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.815+0000 7ff2677fe700 1 -- 172.21.15.46:0/2583766151 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27000a660 con 0x55adcdc1c400 2024-05-09T05:01:41.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.816+0000 7ff295fb5b80 1 -- 172.21.15.46:0/2583766151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcdc1c400 msgr2=0x55adcdc1c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:41.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.816+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/2583766151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcdc1c400 0x55adcdc1c810 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff270009f20 tx=0x7ff270009f50 comp rx=0 tx=0).stop 2024-05-09T05:01:41.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.816+0000 7ff295fb5b80 1 -- 172.21.15.46:0/2583766151 shutdown_connections 2024-05-09T05:01:41.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.816+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/2583766151 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcdc1c400 0x55adcdc1c810 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.816+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/2583766151 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdb19d00 0x55adcdc22240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.816+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/2583766151 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcda9f4a0 0x55adcdc22780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.816+0000 7ff295fb5b80 1 -- 172.21.15.46:0/2583766151 >> 172.21.15.46:0/2583766151 conn(0x55adcdaa2f70 msgr2=0x55adcdc17220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:41.823 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.816+0000 7ff295fb5b80 1 -- 172.21.15.46:0/2583766151 shutdown_connections 2024-05-09T05:01:41.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.816+0000 7ff295fb5b80 1 -- 172.21.15.46:0/2583766151 wait complete. 2024-05-09T05:01:41.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.817+0000 7ff295fb5b80 1 -- start start 2024-05-09T05:01:41.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.817+0000 7ff295fb5b80 1 Processor -- start 2024-05-09T05:01:41.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.817+0000 7ff295fb5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc26110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.817+0000 7ff295fb5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcdb19d00 0x55adcdc26650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.817+0000 7ff295fb5b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdc1c400 0x55adcdc26b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.817+0000 7ff295fb5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55adcdc1d160 con 0x55adcda9f4a0 2024-05-09T05:01:41.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.817+0000 7ff295fb5b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55adcdad9b20 con 0x55adcdc1c400 2024-05-09T05:01:41.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.817+0000 7ff295fb5b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55adcdc29020 con 0x55adcdb19d00 2024-05-09T05:01:41.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.817+0000 7ff28533b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc26110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.818+0000 7ff28533b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc26110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34696/0 (socket says 172.21.15.46:34696) 2024-05-09T05:01:41.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.818+0000 7ff28533b700 1 -- 172.21.15.46:0/911609746 learned_addr learned my addr 172.21.15.46:0/911609746 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:41.824 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.818+0000 7ff295fb5b80 10 client.0 _ll_register_callbacks cb 0x55adcd9b06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:01:41.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.818+0000 7ff284b3a700 1 --2- 172.21.15.46:0/911609746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcdb19d00 0x55adcdc26650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.818+0000 7ff28533b700 1 -- 172.21.15.46:0/911609746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcdb19d00 msgr2=0x55adcdc26650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:41.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.818+0000 7ff28533b700 1 --2- 172.21.15.46:0/911609746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcdb19d00 0x55adcdc26650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.818+0000 7ff28533b700 1 -- 172.21.15.46:0/911609746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdc1c400 msgr2=0x55adcdc26b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:41.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.818+0000 7ff28533b700 1 --2- 172.21.15.46:0/911609746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdc1c400 0x55adcdc26b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.818+0000 7ff28533b700 1 -- 172.21.15.46:0/911609746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adcd9ba490 con 0x55adcda9f4a0 2024-05-09T05:01:41.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.818+0000 7ff284b3a700 1 --2- 172.21.15.46:0/911609746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcdb19d00 0x55adcdc26650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:01:41.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.818+0000 7ff28533b700 1 --2- 172.21.15.46:0/911609746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc26110 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff27800ff60 tx=0x7ff278004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58ca0b250e44074 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:41.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.818+0000 7ff264ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:01:41.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.818+0000 7ff264ff9700 1 -- 172.21.15.46:0/911609746 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27801f630 con 0x55adcda9f4a0 2024-05-09T05:01:41.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.818+0000 7ff264ff9700 1 -- 172.21.15.46:0/911609746 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7ff27801f7d0 con 0x55adcda9f4a0 2024-05-09T05:01:41.825 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.818+0000 7ff295fb5b80 1 -- 172.21.15.46:0/911609746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff270015410 con 0x55adcda9f4a0 2024-05-09T05:01:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.819+0000 7ff264ff9700 1 -- 172.21.15.46:0/911609746 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff27801f9d0 con 0x55adcda9f4a0 2024-05-09T05:01:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.819+0000 7ff295fb5b80 10 client.17565 fetch_fsmap learned FSMap version 25 2024-05-09T05:01:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.819+0000 7ff295fb5b80 1 -- 172.21.15.46:0/911609746 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55adcdc33b40 con 0x55adcda9f4a0 2024-05-09T05:01:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.819+0000 7ff264ff9700 1 -- 172.21.15.46:0/911609746 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff270015410 con 0x55adcda9f4a0 2024-05-09T05:01:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.819+0000 7ff264ff9700 1 -- 172.21.15.46:0/911609746 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff278027b80 con 0x55adcda9f4a0 2024-05-09T05:01:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.819+0000 7ff295fb5b80 10 client.17565 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:01:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.819+0000 7ff295fb5b80 10 client.17565 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T05:01:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.819+0000 7ff295fb5b80 20 client.17565 populate_metadata read hostname 'smithi046' 2024-05-09T05:01:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.819+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/911609746 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55adcdc36180 0x55adcdc38620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.819+0000 7ff295fb5b80 4 client.17565 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:01:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.819+0000 7ff295fb5b80 1 -- 172.21.15.46:0/911609746 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adcdb4ca80 con 0x55adcdc36180 2024-05-09T05:01:41.826 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.819+0000 7ff284b3a700 1 --2- 172.21.15.46:0/911609746 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55adcdc36180 0x55adcdc38620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.820+0000 7ff284b3a700 1 --2- 172.21.15.46:0/911609746 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55adcdc36180 0x55adcdc38620 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f4d4240053c276ea server_cookie=4be2a1a4683b3b7d in_seq=0 out_seq=0 2024-05-09T05:01:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.820+0000 7ff264ff9700 10 client.17565 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T05:01:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.821+0000 7ff264ff9700 1 -- 172.21.15.46:0/911609746 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7ff274084e50 con 0x55adcdc36180 2024-05-09T05:01:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.821+0000 7ff264ff9700 10 client.17565 handle_command_reply: tid=0 2024-05-09T05:01:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.821+0000 7ff295fb5b80 1 client.17565 shutdown 2024-05-09T05:01:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.821+0000 7ff295fb5b80 1 -- 172.21.15.46:0/911609746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 msgr2=0x55adcdc26110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.821+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/911609746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc26110 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff27800ff60 tx=0x7ff278004750 comp rx=0 tx=0).stop 2024-05-09T05:01:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.821+0000 7ff295fb5b80 1 -- 172.21.15.46:0/911609746 shutdown_connections 2024-05-09T05:01:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.821+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/911609746 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55adcdc36180 0x55adcdc38620 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.821+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/911609746 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdc1c400 0x55adcdc26b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.821+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/911609746 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc26110 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.821+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/911609746 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcdb19d00 0x55adcdc26650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.827 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.821+0000 7ff295fb5b80 1 -- 172.21.15.46:0/911609746 >> 172.21.15.46:0/911609746 conn(0x55adcdaa2f70 msgr2=0x55adcdc10ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:41.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.821+0000 7ff264ff9700 0 client.17565 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T05:01:41.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.821+0000 7ff295fb5b80 1 -- 172.21.15.46:0/911609746 shutdown_connections 2024-05-09T05:01:41.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.821+0000 7ff295fb5b80 1 -- 172.21.15.46:0/911609746 wait complete. 2024-05-09T05:01:41.828 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.821+0000 7ff295fb5b80 20 client.17565 trim_cache size 0 max 16384 2024-05-09T05:01:41.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.836+0000 7ff295fb5b80 1 Processor -- start 2024-05-09T05:01:41.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.836+0000 7ff295fb5b80 1 -- start start 2024-05-09T05:01:41.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff295fb5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc21860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff295fb5b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdb19d00 0x55adcdc38500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff295fb5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcdc1c400 0x55adcdc11a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff295fb5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801b1880 con 0x55adcda9f4a0 2024-05-09T05:01:41.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff295fb5b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801af110 con 0x55adcdb19d00 2024-05-09T05:01:41.838 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff295fb5b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801acf80 con 0x55adcdc1c400 2024-05-09T05:01:41.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff28533b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc21860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff28533b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc21860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34708/0 (socket says 172.21.15.46:34708) 2024-05-09T05:01:41.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff28533b700 1 -- 172.21.15.46:0/42076108 learned_addr learned my addr 172.21.15.46:0/42076108 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:41.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff284b3a700 1 --2- 172.21.15.46:0/42076108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdb19d00 0x55adcdc38500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff28533b700 1 -- 172.21.15.46:0/42076108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcdc1c400 msgr2=0x55adcdc11a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:41.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff28533b700 1 --2- 172.21.15.46:0/42076108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcdc1c400 0x55adcdc11a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff28533b700 1 -- 172.21.15.46:0/42076108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdb19d00 msgr2=0x55adcdc38500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:41.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff28533b700 1 --2- 172.21.15.46:0/42076108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdb19d00 0x55adcdc38500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff28533b700 1 -- 172.21.15.46:0/42076108 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adcdc24680 con 0x55adcda9f4a0 2024-05-09T05:01:41.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.837+0000 7ff28533b700 1 --2- 172.21.15.46:0/42076108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc21860 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff278005cd0 tx=0x7ff27800f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273c2ae4a157ff18 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:41.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.838+0000 7ff266ffd700 1 -- 172.21.15.46:0/42076108 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27803c020 con 0x55adcda9f4a0 2024-05-09T05:01:41.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.838+0000 7ff266ffd700 1 -- 172.21.15.46:0/42076108 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff27801fd70 con 0x55adcda9f4a0 2024-05-09T05:01:41.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.838+0000 7ff266ffd700 1 -- 172.21.15.46:0/42076108 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff278033040 con 0x55adcda9f4a0 2024-05-09T05:01:41.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.838+0000 7ff295fb5b80 1 -- 172.21.15.46:0/42076108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 msgr2=0x55adcdc21860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:41.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.838+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/42076108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc21860 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7ff278005cd0 tx=0x7ff27800f920 comp rx=0 tx=0).stop 2024-05-09T05:01:41.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.838+0000 7ff295fb5b80 1 -- 172.21.15.46:0/42076108 shutdown_connections 2024-05-09T05:01:41.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.838+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/42076108 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcdc1c400 0x55adcdc11a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.838+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/42076108 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc21860 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.838+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/42076108 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdb19d00 0x55adcdc38500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.838+0000 7ff295fb5b80 1 -- 172.21.15.46:0/42076108 >> 172.21.15.46:0/42076108 conn(0x55adcdaa2f70 msgr2=0x55adcdc1b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:41.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.838+0000 7ff295fb5b80 1 -- 172.21.15.46:0/42076108 shutdown_connections 2024-05-09T05:01:41.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.839+0000 7ff295fb5b80 1 -- 172.21.15.46:0/42076108 wait complete. 2024-05-09T05:01:41.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.839+0000 7ff295fb5b80 1 -- start start 2024-05-09T05:01:41.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.839+0000 7ff295fb5b80 1 Processor -- start 2024-05-09T05:01:41.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.839+0000 7ff295fb5b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc98dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.839+0000 7ff295fb5b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcdb19d00 0x55adcdc99310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.839+0000 7ff295fb5b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdc1c400 0x55adcdc2b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.839+0000 7ff295fb5b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801ad5d0 con 0x55adcda9f4a0 2024-05-09T05:01:41.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.839+0000 7ff295fb5b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801aea40 con 0x55adcdc1c400 2024-05-09T05:01:41.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.839+0000 7ff295fb5b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff2801adca0 con 0x55adcdb19d00 2024-05-09T05:01:41.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff295fb5b80 10 client.0 _ll_register_callbacks cb 0x55adcd9b06c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:01:41.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff28533b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc98dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff28533b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc98dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:34714/0 (socket says 172.21.15.46:34714) 2024-05-09T05:01:41.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff28533b700 1 -- 172.21.15.46:0/2977373755 learned_addr learned my addr 172.21.15.46:0/2977373755 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:41.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff285b3c700 1 --2- 172.21.15.46:0/2977373755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdc1c400 0x55adcdc2b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff284b3a700 1 --2- 172.21.15.46:0/2977373755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcdb19d00 0x55adcdc99310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.845 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff28533b700 1 -- 172.21.15.46:0/2977373755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcdb19d00 msgr2=0x55adcdc99310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff28533b700 1 --2- 172.21.15.46:0/2977373755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcdb19d00 0x55adcdc99310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff28533b700 1 -- 172.21.15.46:0/2977373755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdc1c400 msgr2=0x55adcdc2b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff28533b700 1 --2- 172.21.15.46:0/2977373755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdc1c400 0x55adcdc2b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff28533b700 1 -- 172.21.15.46:0/2977373755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adcdc2bd00 con 0x55adcda9f4a0 2024-05-09T05:01:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff28533b700 1 --2- 172.21.15.46:0/2977373755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc98dd0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff278000c00 tx=0x7ff278012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7813589c9c23360 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff264ff9700 10 client.0 ms_handle_connect on v2:172.21.15.46:3300/0 2024-05-09T05:01:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff264ff9700 1 -- 172.21.15.46:0/2977373755 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff27803c030 con 0x55adcda9f4a0 2024-05-09T05:01:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff264ff9700 1 -- 172.21.15.46:0/2977373755 <== mon.0 v2:172.21.15.46:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7ff278005020 con 0x55adcda9f4a0 2024-05-09T05:01:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.840+0000 7ff295fb5b80 1 -- 172.21.15.46:0/2977373755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff27801fd70 con 0x55adcda9f4a0 2024-05-09T05:01:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.841+0000 7ff264ff9700 1 -- 172.21.15.46:0/2977373755 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7ff278050050 con 0x55adcda9f4a0 2024-05-09T05:01:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.841+0000 7ff295fb5b80 10 client.17571 fetch_fsmap learned FSMap version 25 2024-05-09T05:01:41.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.841+0000 7ff295fb5b80 1 -- 172.21.15.46:0/2977373755 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55adcdca03d0 con 0x55adcda9f4a0 2024-05-09T05:01:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.841+0000 7ff264ff9700 1 -- 172.21.15.46:0/2977373755 <== mon.0 v2:172.21.15.46:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff27801fd70 con 0x55adcda9f4a0 2024-05-09T05:01:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.841+0000 7ff264ff9700 1 -- 172.21.15.46:0/2977373755 <== mon.0 v2:172.21.15.46:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7ff278010790 con 0x55adcda9f4a0 2024-05-09T05:01:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.841+0000 7ff295fb5b80 10 client.17571 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:01:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.841+0000 7ff295fb5b80 10 client.17571 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-05-09T05:01:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.841+0000 7ff295fb5b80 20 client.17571 populate_metadata read hostname 'smithi046' 2024-05-09T05:01:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.841+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/2977373755 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55adcdca0910 0x55adcdca2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.841+0000 7ff295fb5b80 4 client.17571 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:01:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.841+0000 7ff295fb5b80 1 -- 172.21.15.46:0/2977373755 --> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55adcdb46b20 con 0x55adcdca0910 2024-05-09T05:01:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.841+0000 7ff284b3a700 1 --2- 172.21.15.46:0/2977373755 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55adcdca0910 0x55adcdca2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff284b3a700 1 --2- 172.21.15.46:0/2977373755 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55adcdca0910 0x55adcdca2d30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=835292ca36fe1ea3 server_cookie=8f1341d7b644ad18 in_seq=0 out_seq=0 2024-05-09T05:01:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff264ff9700 10 client.17571 ms_handle_connect on v2:172.21.15.72:6836/3365501578 2024-05-09T05:01:41.847 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff264ff9700 1 -- 172.21.15.46:0/2977373755 <== mds.1 v2:172.21.15.72:6836/3365501578 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55adcdb46b20 con 0x55adcdca0910 2024-05-09T05:01:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff264ff9700 10 client.17571 handle_command_reply: tid=0 2024-05-09T05:01:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff295fb5b80 1 client.17571 shutdown 2024-05-09T05:01:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff295fb5b80 1 -- 172.21.15.46:0/2977373755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 msgr2=0x55adcdc98dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/2977373755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc98dd0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7ff278000c00 tx=0x7ff278012000 comp rx=0 tx=0).stop 2024-05-09T05:01:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff295fb5b80 1 -- 172.21.15.46:0/2977373755 shutdown_connections 2024-05-09T05:01:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/2977373755 >> [v2:172.21.15.72:6836/3365501578,v1:172.21.15.72:6837/3365501578] conn(0x55adcdca0910 0x55adcdca2d30 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/2977373755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55adcdc1c400 0x55adcdc2b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/2977373755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55adcda9f4a0 0x55adcdc98dd0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff264ff9700 0 client.17571 ms_handle_reset on v2:172.21.15.72:6836/3365501578 2024-05-09T05:01:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff295fb5b80 1 --2- 172.21.15.46:0/2977373755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55adcdb19d00 0x55adcdc99310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.848 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff295fb5b80 1 -- 172.21.15.46:0/2977373755 >> 172.21.15.46:0/2977373755 conn(0x55adcdaa2f70 msgr2=0x55adcdc11780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:41.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff295fb5b80 1 -- 172.21.15.46:0/2977373755 shutdown_connections 2024-05-09T05:01:41.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.842+0000 7ff295fb5b80 1 -- 172.21.15.46:0/2977373755 wait complete. 2024-05-09T05:01:41.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.843+0000 7ff295fb5b80 20 client.17571 trim_cache size 0 max 16384 2024-05-09T05:01:41.849 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T05:01:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.843+0000 7ff28759f700 1 -- 172.21.15.46:0/1334823677 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff2680b29f0 msgr2=0x7ff2680b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.843+0000 7ff28759f700 1 --2- 172.21.15.46:0/1334823677 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff2680b29f0 0x7ff2680b4f00 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff28006bc90 tx=0x7ff27800b5f0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.843+0000 7ff28759f700 1 -- 172.21.15.46:0/1334823677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff280108f40 msgr2=0x7ff28019b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.843+0000 7ff28759f700 1 --2- 172.21.15.46:0/1334823677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff280108f40 0x7ff28019b420 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7ff274006010 tx=0x7ff2740047b0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.843+0000 7ff28759f700 1 -- 172.21.15.46:0/1334823677 shutdown_connections 2024-05-09T05:01:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.843+0000 7ff28759f700 1 --2- 172.21.15.46:0/1334823677 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff2680b29f0 0x7ff2680b4f00 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.843+0000 7ff28759f700 1 --2- 172.21.15.46:0/1334823677 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff280109890 0x7ff28019f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.843+0000 7ff28759f700 1 --2- 172.21.15.46:0/1334823677 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff280102b20 0x7ff28019ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.843+0000 7ff28759f700 1 --2- 172.21.15.46:0/1334823677 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff280108f40 0x7ff28019b420 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.843+0000 7ff28759f700 1 -- 172.21.15.46:0/1334823677 >> 172.21.15.46:0/1334823677 conn(0x7ff2800fe540 msgr2=0x7ff280100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:41.850 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.843+0000 7ff28759f700 1 -- 172.21.15.46:0/1334823677 shutdown_connections 2024-05-09T05:01:41.851 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:41.843+0000 7ff28759f700 1 -- 172.21.15.46:0/1334823677 wait complete. 2024-05-09T05:01:42.613 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-05-09T05:01:42.876 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:01:43.109 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:42 smithi110 ceph-mon[34904]: pgmap v329: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T05:01:43.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:42 smithi072 ceph-mon[34088]: pgmap v329: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T05:01:43.134 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:42 smithi046 ceph-mon[30136]: pgmap v329: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T05:01:44.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.632+0000 7f3e44dd0700 1 -- 172.21.15.46:0/3073676794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e40108740 msgr2=0x7f3e401100a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:44.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.632+0000 7f3e44dd0700 1 --2- 172.21.15.46:0/3073676794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e40108740 0x7f3e401100a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800b0d0 tx=0x7f3e3800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.632+0000 7f3e44dd0700 1 -- 172.21.15.46:0/3073676794 shutdown_connections 2024-05-09T05:01:44.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.632+0000 7f3e44dd0700 1 --2- 172.21.15.46:0/3073676794 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e40108740 0x7f3e401100a0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.632+0000 7f3e44dd0700 1 --2- 172.21.15.46:0/3073676794 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e401034c0 0x7f3e401038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.632+0000 7f3e44dd0700 1 --2- 172.21.15.46:0/3073676794 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e40107df0 0x7f3e40108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.632+0000 7f3e44dd0700 1 -- 172.21.15.46:0/3073676794 >> 172.21.15.46:0/3073676794 conn(0x7f3e400fedf0 msgr2=0x7f3e40101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:44.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.632+0000 7f3e44dd0700 1 -- 172.21.15.46:0/3073676794 shutdown_connections 2024-05-09T05:01:44.634 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.632+0000 7f3e44dd0700 1 -- 172.21.15.46:0/3073676794 wait complete. 2024-05-09T05:01:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.633+0000 7f3e44dd0700 1 Processor -- start 2024-05-09T05:01:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.633+0000 7f3e44dd0700 1 -- start start 2024-05-09T05:01:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.633+0000 7f3e44dd0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e401034c0 0x7f3e4019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.633+0000 7f3e44dd0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e40107df0 0x7f3e4019c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.633+0000 7f3e44dd0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e40108740 0x7f3e401a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.633+0000 7f3e44dd0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3e40113370 con 0x7f3e401034c0 2024-05-09T05:01:44.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e44dd0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401131f0 con 0x7f3e40108740 2024-05-09T05:01:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e44dd0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401134f0 con 0x7f3e40107df0 2024-05-09T05:01:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e3e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e401034c0 0x7f3e4019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e3e59c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e401034c0 0x7f3e4019bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48714/0 (socket says 172.21.15.46:48714) 2024-05-09T05:01:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e3e59c700 1 -- 172.21.15.46:0/4266944375 learned_addr learned my addr 172.21.15.46:0/4266944375 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e3dd9b700 1 --2- 172.21.15.46:0/4266944375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e40107df0 0x7f3e4019c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e3ed9d700 1 --2- 172.21.15.46:0/4266944375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e40108740 0x7f3e401a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e3e59c700 1 -- 172.21.15.46:0/4266944375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e40107df0 msgr2=0x7f3e4019c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e3e59c700 1 --2- 172.21.15.46:0/4266944375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e40107df0 0x7f3e4019c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e3e59c700 1 -- 172.21.15.46:0/4266944375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e40108740 msgr2=0x7f3e401a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e3e59c700 1 --2- 172.21.15.46:0/4266944375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e40108740 0x7f3e401a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.642 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e3e59c700 1 -- 172.21.15.46:0/4266944375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e4019c9c0 con 0x7f3e401034c0 2024-05-09T05:01:44.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e3ed9d700 1 --2- 172.21.15.46:0/4266944375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e40108740 0x7f3e401a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:01:44.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e3e59c700 1 --2- 172.21.15.46:0/4266944375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e401034c0 0x7f3e4019bbd0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000d950 tx=0x7f3e3000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:44.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e2f7fe700 1 -- 172.21.15.46:0/4266944375 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e30016070 con 0x7f3e401034c0 2024-05-09T05:01:44.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e44dd0700 1 -- 172.21.15.46:0/4266944375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e40199a20 con 0x7f3e401034c0 2024-05-09T05:01:44.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e2f7fe700 1 -- 172.21.15.46:0/4266944375 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e30009df0 con 0x7f3e401034c0 2024-05-09T05:01:44.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.634+0000 7f3e2f7fe700 1 -- 172.21.15.46:0/4266944375 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000f3c0 con 0x7f3e401034c0 2024-05-09T05:01:44.643 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.635+0000 7f3e44dd0700 1 -- 172.21.15.46:0/4266944375 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e40199f60 con 0x7f3e401034c0 2024-05-09T05:01:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.637+0000 7f3e2f7fe700 1 -- 172.21.15.46:0/4266944375 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3e30019070 con 0x7f3e401034c0 2024-05-09T05:01:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.638+0000 7f3e2f7fe700 1 --2- 172.21.15.46:0/4266944375 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f3e280b29f0 0x7f3e280b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.638+0000 7f3e3dd9b700 1 --2- 172.21.15.46:0/4266944375 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f3e280b29f0 0x7f3e280b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.638+0000 7f3e2f7fe700 1 -- 172.21.15.46:0/4266944375 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7f3e300e22f0 con 0x7f3e401034c0 2024-05-09T05:01:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.638+0000 7f3e537e6b80 1 Processor -- start 2024-05-09T05:01:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e537e6b80 1 -- start start 2024-05-09T05:01:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e537e6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe53a84160 0x55fe53b1d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e537e6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe5399a4a0 0x55fe53b1da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e537e6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 0x55fe53b17ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e3dd9b700 1 --2- 172.21.15.46:0/4266944375 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f3e280b29f0 0x7f3e280b4f00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3e40106750 tx=0x7f3e34009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e537e6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fe53a39ff0 con 0x55fe5399a4a0 2024-05-09T05:01:44.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e537e6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55fe539d87f0 con 0x55fe53a84160 2024-05-09T05:01:44.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e537e6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fe53a35fb0 con 0x55fe53b176a0 2024-05-09T05:01:44.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e3dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe5399a4a0 0x55fe53b1da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e3dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe5399a4a0 0x55fe53b1da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48720/0 (socket says 172.21.15.46:48720) 2024-05-09T05:01:44.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e3dd9b700 1 -- 172.21.15.46:0/3098018792 learned_addr learned my addr 172.21.15.46:0/3098018792 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:44.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e3ed9d700 1 --2- 172.21.15.46:0/3098018792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 0x55fe53b17ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e3e59c700 1 --2- 172.21.15.46:0/3098018792 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe53a84160 0x55fe53b1d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e3ed9d700 1 -- 172.21.15.46:0/3098018792 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe53a84160 msgr2=0x55fe53b1d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:44.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e3ed9d700 1 --2- 172.21.15.46:0/3098018792 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe53a84160 0x55fe53b1d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e3ed9d700 1 -- 172.21.15.46:0/3098018792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe5399a4a0 msgr2=0x55fe53b1da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:44.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.639+0000 7f3e3ed9d700 1 --2- 172.21.15.46:0/3098018792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe5399a4a0 0x55fe53b1da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.640+0000 7f3e3ed9d700 1 -- 172.21.15.46:0/3098018792 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe539d0370 con 0x55fe53b176a0 2024-05-09T05:01:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.640+0000 7f3e3ed9d700 1 --2- 172.21.15.46:0/3098018792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 0x55fe53b17ad0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800b0d0 tx=0x7f3e38009b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=510ad84779d074e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.640+0000 7f3e2cff9700 1 -- 172.21.15.46:0/3098018792 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3801f030 con 0x55fe53b176a0 2024-05-09T05:01:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.640+0000 7f3e2cff9700 1 -- 172.21.15.46:0/3098018792 <== mon.1 v2:172.21.15.110:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e380092e0 con 0x55fe53b176a0 2024-05-09T05:01:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.640+0000 7f3e2cff9700 1 -- 172.21.15.46:0/3098018792 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e38007940 con 0x55fe53b176a0 2024-05-09T05:01:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.641+0000 7f3e537e6b80 1 -- 172.21.15.46:0/3098018792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 msgr2=0x55fe53b17ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.641+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/3098018792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 0x55fe53b17ad0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3e3800b0d0 tx=0x7f3e38009b80 comp rx=0 tx=0).stop 2024-05-09T05:01:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.641+0000 7f3e537e6b80 1 -- 172.21.15.46:0/3098018792 shutdown_connections 2024-05-09T05:01:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.641+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/3098018792 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 0x55fe53b17ad0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.641+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/3098018792 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe53a84160 0x55fe53b1d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.641+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/3098018792 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe5399a4a0 0x55fe53b1da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.641+0000 7f3e537e6b80 1 -- 172.21.15.46:0/3098018792 >> 172.21.15.46:0/3098018792 conn(0x55fe539fa6a0 msgr2=0x55fe53b12400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:44.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.641+0000 7f3e537e6b80 1 -- 172.21.15.46:0/3098018792 shutdown_connections 2024-05-09T05:01:44.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.641+0000 7f3e537e6b80 1 -- 172.21.15.46:0/3098018792 wait complete. 2024-05-09T05:01:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.641+0000 7f3e537e6b80 1 -- start start 2024-05-09T05:01:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.641+0000 7f3e537e6b80 1 Processor -- start 2024-05-09T05:01:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e537e6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 0x55fe53b211a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e537e6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53a84160 0x55fe53b216e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e537e6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 0x55fe53b21c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e537e6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x55fe538b5ba0 con 0x55fe53a84160 2024-05-09T05:01:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e537e6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x55fe539f4150 con 0x55fe5399a4a0 2024-05-09T05:01:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e537e6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x55fe53b241d0 con 0x55fe53b176a0 2024-05-09T05:01:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e3dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53a84160 0x55fe53b216e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e3dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53a84160 0x55fe53b216e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48734/0 (socket says 172.21.15.46:48734) 2024-05-09T05:01:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e3dd9b700 1 -- 172.21.15.46:0/346491754 learned_addr learned my addr 172.21.15.46:0/346491754 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:44.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e537e6b80 10 client.0 _ll_register_callbacks cb 0x55fe538ab6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:01:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e3ed9d700 1 --2- 172.21.15.46:0/346491754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 0x55fe53b21c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e3e59c700 1 --2- 172.21.15.46:0/346491754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 0x55fe53b211a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e3ed9d700 1 -- 172.21.15.46:0/346491754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 msgr2=0x55fe53b211a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e3ed9d700 1 --2- 172.21.15.46:0/346491754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 0x55fe53b211a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e3ed9d700 1 -- 172.21.15.46:0/346491754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53a84160 msgr2=0x55fe53b216e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e3ed9d700 1 --2- 172.21.15.46:0/346491754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53a84160 0x55fe53b216e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e3ed9d700 1 -- 172.21.15.46:0/346491754 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe538b5490 con 0x55fe53b176a0 2024-05-09T05:01:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e3dd9b700 1 --2- 172.21.15.46:0/346491754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53a84160 0x55fe53b216e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:01:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.642+0000 7f3e3e59c700 1 --2- 172.21.15.46:0/346491754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 0x55fe53b211a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:01:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.643+0000 7f3e3ed9d700 1 --2- 172.21.15.46:0/346491754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 0x55fe53b21c20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55fe53b1e0c0 tx=0x7f3e3800bd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac20fb742237c948 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.643+0000 7f3e25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T05:01:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.643+0000 7f3e25ffb700 1 -- 172.21.15.46:0/346491754 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3802a070 con 0x55fe53b176a0 2024-05-09T05:01:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.643+0000 7f3e537e6b80 1 -- 172.21.15.46:0/346491754 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e380092e0 con 0x55fe53b176a0 2024-05-09T05:01:44.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.643+0000 7f3e25ffb700 1 -- 172.21.15.46:0/346491754 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7f3e380077c0 con 0x55fe53b176a0 2024-05-09T05:01:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.643+0000 7f3e25ffb700 1 -- 172.21.15.46:0/346491754 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3e3801f040 con 0x55fe53b176a0 2024-05-09T05:01:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.643+0000 7f3e537e6b80 10 client.26899 fetch_fsmap learned FSMap version 25 2024-05-09T05:01:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.643+0000 7f3e537e6b80 1 -- 172.21.15.46:0/346491754 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fe53b2eb30 con 0x55fe53b176a0 2024-05-09T05:01:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.643+0000 7f3e25ffb700 1 -- 172.21.15.46:0/346491754 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e380092e0 con 0x55fe53b176a0 2024-05-09T05:01:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.643+0000 7f3e25ffb700 1 -- 172.21.15.46:0/346491754 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3e38011070 con 0x55fe53b176a0 2024-05-09T05:01:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.643+0000 7f3e537e6b80 10 client.26899 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:01:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.643+0000 7f3e537e6b80 10 client.26899 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:01:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.643+0000 7f3e537e6b80 20 client.26899 populate_metadata read hostname 'smithi046' 2024-05-09T05:01:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.644+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/346491754 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55fe53b31000 0x55fe53b33420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.644+0000 7f3e537e6b80 4 client.26899 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-09T05:01:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.644+0000 7f3e537e6b80 1 -- 172.21.15.46:0/346491754 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe53a479d0 con 0x55fe53b31000 2024-05-09T05:01:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.644+0000 7f3e3e59c700 1 --2- 172.21.15.46:0/346491754 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55fe53b31000 0x55fe53b33420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.644+0000 7f3e3e59c700 1 --2- 172.21.15.46:0/346491754 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55fe53b31000 0x55fe53b33420 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e107483fc6290d4 server_cookie=ce8e89c525b7064f in_seq=0 out_seq=0 2024-05-09T05:01:44.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.644+0000 7f3e25ffb700 10 client.26899 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:01:44.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.645+0000 7f3e25ffb700 1 -- 172.21.15.46:0/346491754 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fe53a479d0 con 0x55fe53b31000 2024-05-09T05:01:44.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.645+0000 7f3e25ffb700 10 client.26899 handle_command_reply: tid=0 2024-05-09T05:01:44.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.645+0000 7f3e537e6b80 1 client.26899 shutdown 2024-05-09T05:01:44.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.645+0000 7f3e537e6b80 1 -- 172.21.15.46:0/346491754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 msgr2=0x55fe53b21c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:44.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.645+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/346491754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 0x55fe53b21c20 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x55fe53b1e0c0 tx=0x7f3e3800bd50 comp rx=0 tx=0).stop 2024-05-09T05:01:44.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.645+0000 7f3e537e6b80 1 -- 172.21.15.46:0/346491754 shutdown_connections 2024-05-09T05:01:44.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.645+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/346491754 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 0x55fe53b21c20 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.645+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/346491754 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55fe53b31000 0x55fe53b33420 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.645+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/346491754 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 0x55fe53b211a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.645+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/346491754 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53a84160 0x55fe53b216e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.645+0000 7f3e25ffb700 0 client.26899 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:01:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.645+0000 7f3e537e6b80 1 -- 172.21.15.46:0/346491754 >> 172.21.15.46:0/346491754 conn(0x55fe539fa6a0 msgr2=0x55fe53b121c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.645+0000 7f3e537e6b80 1 -- 172.21.15.46:0/346491754 shutdown_connections 2024-05-09T05:01:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.645+0000 7f3e537e6b80 1 -- 172.21.15.46:0/346491754 wait complete. 2024-05-09T05:01:44.655 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.645+0000 7f3e537e6b80 20 client.26899 trim_cache size 0 max 16384 2024-05-09T05:01:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.660+0000 7f3e537e6b80 1 Processor -- start 2024-05-09T05:01:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e537e6b80 1 -- start start 2024-05-09T05:01:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e537e6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 0x55fe53b20100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e537e6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53a84160 0x55fe53b2d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e537e6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 0x55fe53b21a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e537e6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401b38c0 con 0x55fe53a84160 2024-05-09T05:01:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e537e6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401b1150 con 0x55fe5399a4a0 2024-05-09T05:01:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e537e6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401aefc0 con 0x55fe53b176a0 2024-05-09T05:01:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e3dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53a84160 0x55fe53b2d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e3dd9b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53a84160 0x55fe53b2d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48746/0 (socket says 172.21.15.46:48746) 2024-05-09T05:01:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e3dd9b700 1 -- 172.21.15.46:0/3524161507 learned_addr learned my addr 172.21.15.46:0/3524161507 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:44.662 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e3ed9d700 1 --2- 172.21.15.46:0/3524161507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 0x55fe53b21a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e3e59c700 1 --2- 172.21.15.46:0/3524161507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 0x55fe53b20100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e3dd9b700 1 -- 172.21.15.46:0/3524161507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 msgr2=0x55fe53b21a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e3dd9b700 1 --2- 172.21.15.46:0/3524161507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 0x55fe53b21a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e3dd9b700 1 -- 172.21.15.46:0/3524161507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 msgr2=0x55fe53b20100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e3dd9b700 1 --2- 172.21.15.46:0/3524161507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 0x55fe53b20100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.661+0000 7f3e3dd9b700 1 -- 172.21.15.46:0/3524161507 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe53b21fa0 con 0x55fe53a84160 2024-05-09T05:01:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.662+0000 7f3e3dd9b700 1 --2- 172.21.15.46:0/3524161507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53a84160 0x55fe53b2d2c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f3e3401d640 tx=0x7f3e34014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f6aafe04cd76c55 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.662+0000 7f3e27fff700 1 -- 172.21.15.46:0/3524161507 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3401dda0 con 0x55fe53a84160 2024-05-09T05:01:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.662+0000 7f3e27fff700 1 -- 172.21.15.46:0/3524161507 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e34004500 con 0x55fe53a84160 2024-05-09T05:01:44.663 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.662+0000 7f3e27fff700 1 -- 172.21.15.46:0/3524161507 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e3401e420 con 0x55fe53a84160 2024-05-09T05:01:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.662+0000 7f3e537e6b80 1 -- 172.21.15.46:0/3524161507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53a84160 msgr2=0x55fe53b2d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.662+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/3524161507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53a84160 0x55fe53b2d2c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f3e3401d640 tx=0x7f3e34014000 comp rx=0 tx=0).stop 2024-05-09T05:01:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 1 -- 172.21.15.46:0/3524161507 shutdown_connections 2024-05-09T05:01:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/3524161507 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53b176a0 0x55fe53b21a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/3524161507 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 0x55fe53b20100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/3524161507 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53a84160 0x55fe53b2d2c0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 1 -- 172.21.15.46:0/3524161507 >> 172.21.15.46:0/3524161507 conn(0x55fe539fa6a0 msgr2=0x55fe53b1ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 1 -- 172.21.15.46:0/3524161507 shutdown_connections 2024-05-09T05:01:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 1 -- 172.21.15.46:0/3524161507 wait complete. 2024-05-09T05:01:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 1 -- start start 2024-05-09T05:01:44.668 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 1 Processor -- start 2024-05-09T05:01:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 0x55fe53b33a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53a84160 0x55fe53b33fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53b176a0 0x55fe53b93580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401af610 con 0x55fe53b176a0 2024-05-09T05:01:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401b0a80 con 0x55fe5399a4a0 2024-05-09T05:01:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f3e401afce0 con 0x55fe53a84160 2024-05-09T05:01:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.663+0000 7f3e537e6b80 10 client.0 _ll_register_callbacks cb 0x55fe538ab6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-09T05:01:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e3ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53b176a0 0x55fe53b93580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e3dd9b700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53a84160 0x55fe53b33fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e3ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53b176a0 0x55fe53b93580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48760/0 (socket says 172.21.15.46:48760) 2024-05-09T05:01:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e3ed9d700 1 -- 172.21.15.46:0/176294755 learned_addr learned my addr 172.21.15.46:0/176294755 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e3e59c700 1 --2- 172.21.15.46:0/176294755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 0x55fe53b33a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e3dd9b700 1 -- 172.21.15.46:0/176294755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 msgr2=0x55fe53b33a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:44.669 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e3dd9b700 1 --2- 172.21.15.46:0/176294755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 0x55fe53b33a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e3dd9b700 1 -- 172.21.15.46:0/176294755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53b176a0 msgr2=0x55fe53b93580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-09T05:01:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e3dd9b700 1 --2- 172.21.15.46:0/176294755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53b176a0 0x55fe53b93580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e3dd9b700 1 -- 172.21.15.46:0/176294755 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe53b93b70 con 0x55fe53a84160 2024-05-09T05:01:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e3ed9d700 1 --2- 172.21.15.46:0/176294755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53b176a0 0x55fe53b93580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-09T05:01:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e3dd9b700 1 --2- 172.21.15.46:0/176294755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53a84160 0x55fe53b33fa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55fe53b18970 tx=0x7f3e340090e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c1f175f49f2b8d9a server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e25ffb700 10 client.0 ms_handle_connect on v2:172.21.15.110:3300/0 2024-05-09T05:01:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e25ffb700 1 -- 172.21.15.46:0/176294755 <== mon.1 v2:172.21.15.110:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e34003890 con 0x55fe53a84160 2024-05-09T05:01:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e537e6b80 1 -- 172.21.15.46:0/176294755 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3e34004500 con 0x55fe53a84160 2024-05-09T05:01:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.664+0000 7f3e25ffb700 1 -- 172.21.15.46:0/176294755 <== mon.1 v2:172.21.15.110:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7f3e34003a30 con 0x55fe53a84160 2024-05-09T05:01:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.665+0000 7f3e25ffb700 1 -- 172.21.15.46:0/176294755 <== mon.1 v2:172.21.15.110:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f3e3401e420 con 0x55fe53a84160 2024-05-09T05:01:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.665+0000 7f3e537e6b80 10 client.26905 fetch_fsmap learned FSMap version 25 2024-05-09T05:01:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.665+0000 7f3e537e6b80 1 -- 172.21.15.46:0/176294755 --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55fe53b9ae50 con 0x55fe53a84160 2024-05-09T05:01:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.665+0000 7f3e25ffb700 1 -- 172.21.15.46:0/176294755 <== mon.1 v2:172.21.15.110:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e34004500 con 0x55fe53a84160 2024-05-09T05:01:44.670 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.665+0000 7f3e25ffb700 1 -- 172.21.15.46:0/176294755 <== mon.1 v2:172.21.15.110:3300/0 5 ==== fsmap(e 25) ==== 8338+0+0 (secure 0 0 0) 0x7f3e34029510 con 0x55fe53a84160 2024-05-09T05:01:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.665+0000 7f3e537e6b80 10 client.26905 fetch_fsmap finished waiting for FSMap version 25 2024-05-09T05:01:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.665+0000 7f3e537e6b80 10 client.26905 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-05-09T05:01:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.665+0000 7f3e537e6b80 20 client.26905 populate_metadata read hostname 'smithi046' 2024-05-09T05:01:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.665+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/176294755 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55fe53b9b4e0 0x55fe53b9d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.665+0000 7f3e537e6b80 4 client.26905 mds_command: new command op to 24503 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-05-09T05:01:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.665+0000 7f3e537e6b80 1 -- 172.21.15.46:0/176294755 --> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fe53a41a70 con 0x55fe53b9b4e0 2024-05-09T05:01:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.665+0000 7f3e3e59c700 1 --2- 172.21.15.46:0/176294755 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55fe53b9b4e0 0x55fe53b9d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e3e59c700 1 --2- 172.21.15.46:0/176294755 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55fe53b9b4e0 0x55fe53b9d900 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eabd890457eaabb9 server_cookie=f6dfd241e54e5c72 in_seq=0 out_seq=0 2024-05-09T05:01:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e25ffb700 10 client.26905 ms_handle_connect on v2:172.21.15.72:6838/2905856798 2024-05-09T05:01:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e25ffb700 1 -- 172.21.15.46:0/176294755 <== mds.0 v2:172.21.15.72:6838/2905856798 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fe53a41a70 con 0x55fe53b9b4e0 2024-05-09T05:01:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e25ffb700 10 client.26905 handle_command_reply: tid=0 2024-05-09T05:01:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e537e6b80 1 client.26905 shutdown 2024-05-09T05:01:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e537e6b80 1 -- 172.21.15.46:0/176294755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53a84160 msgr2=0x55fe53b33fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/176294755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53a84160 0x55fe53b33fa0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x55fe53b18970 tx=0x7f3e340090e0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.671 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e537e6b80 1 -- 172.21.15.46:0/176294755 shutdown_connections 2024-05-09T05:01:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/176294755 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x55fe53b176a0 0x55fe53b93580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/176294755 >> [v2:172.21.15.72:6838/2905856798,v1:172.21.15.72:6839/2905856798] conn(0x55fe53b9b4e0 0x55fe53b9d900 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/176294755 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x55fe5399a4a0 0x55fe53b33a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e25ffb700 0 client.26905 ms_handle_reset on v2:172.21.15.72:6838/2905856798 2024-05-09T05:01:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e537e6b80 1 --2- 172.21.15.46:0/176294755 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x55fe53a84160 0x55fe53b33fa0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e537e6b80 1 -- 172.21.15.46:0/176294755 >> 172.21.15.46:0/176294755 conn(0x55fe539fa6a0 msgr2=0x55fe53b1ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e537e6b80 1 -- 172.21.15.46:0/176294755 shutdown_connections 2024-05-09T05:01:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e537e6b80 1 -- 172.21.15.46:0/176294755 wait complete. 2024-05-09T05:01:44.672 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.666+0000 7f3e537e6b80 20 client.26905 trim_cache size 0 max 16384 2024-05-09T05:01:44.672 INFO:teuthology.orchestra.run.smithi046.stdout:[] 2024-05-09T05:01:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.667+0000 7f3e44dd0700 1 -- 172.21.15.46:0/4266944375 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f3e280b29f0 msgr2=0x7f3e280b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.667+0000 7f3e44dd0700 1 --2- 172.21.15.46:0/4266944375 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f3e280b29f0 0x7f3e280b4f00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3e40106750 tx=0x7f3e34009260 comp rx=0 tx=0).stop 2024-05-09T05:01:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.667+0000 7f3e44dd0700 1 -- 172.21.15.46:0/4266944375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e401034c0 msgr2=0x7f3e4019bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.667+0000 7f3e44dd0700 1 --2- 172.21.15.46:0/4266944375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e401034c0 0x7f3e4019bbd0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f3e3000d950 tx=0x7f3e3000dc60 comp rx=0 tx=0).stop 2024-05-09T05:01:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.667+0000 7f3e44dd0700 1 -- 172.21.15.46:0/4266944375 shutdown_connections 2024-05-09T05:01:44.673 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.667+0000 7f3e44dd0700 1 --2- 172.21.15.46:0/4266944375 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f3e280b29f0 0x7f3e280b4f00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.667+0000 7f3e44dd0700 1 --2- 172.21.15.46:0/4266944375 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f3e40108740 0x7f3e401a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.667+0000 7f3e44dd0700 1 --2- 172.21.15.46:0/4266944375 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f3e401034c0 0x7f3e4019bbd0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.667+0000 7f3e44dd0700 1 --2- 172.21.15.46:0/4266944375 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f3e40107df0 0x7f3e4019c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.667+0000 7f3e44dd0700 1 -- 172.21.15.46:0/4266944375 >> 172.21.15.46:0/4266944375 conn(0x7f3e400fedf0 msgr2=0x7f3e40107560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.667+0000 7f3e44dd0700 1 -- 172.21.15.46:0/4266944375 shutdown_connections 2024-05-09T05:01:44.674 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:44.667+0000 7f3e44dd0700 1 -- 172.21.15.46:0/4266944375 wait complete. 2024-05-09T05:01:45.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:44 smithi046 ceph-mon[30136]: pgmap v330: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T05:01:45.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:44 smithi072 ceph-mon[34088]: pgmap v330: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T05:01:45.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:44 smithi110 ceph-mon[34904]: pgmap v330: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T05:01:45.494 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_c9da4049f638c0f84bdd9b4e33c42e86046ac554/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-05-09T05:01:45.495 INFO:tasks.fwd_scrub:done joining 2024-05-09T05:01:45.495 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-09T05:01:45.507 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-09T05:01:45.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.632+0000 7f804ff7d640 1 -- 172.21.15.46:0/744934200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8048108b20 msgr2=0x7f80481026c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:45.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.632+0000 7f804ff7d640 1 --2- 172.21.15.46:0/744934200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8048108b20 0x7f80481026c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f8044009ca0 tx=0x7f804402a8c0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.632+0000 7f804ff7d640 1 -- 172.21.15.46:0/744934200 shutdown_connections 2024-05-09T05:01:45.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.632+0000 7f804ff7d640 1 --2- 172.21.15.46:0/744934200 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8048103270 0x7f804810f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.632+0000 7f804ff7d640 1 --2- 172.21.15.46:0/744934200 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8048108fc0 0x7f8048102c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.632+0000 7f804ff7d640 1 --2- 172.21.15.46:0/744934200 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8048108b20 0x7f80481026c0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.632+0000 7f804ff7d640 1 -- 172.21.15.46:0/744934200 >> 172.21.15.46:0/744934200 conn(0x7f8048078240 msgr2=0x7f8048078680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:45.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.632+0000 7f804ff7d640 1 -- 172.21.15.46:0/744934200 shutdown_connections 2024-05-09T05:01:45.633 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.632+0000 7f804ff7d640 1 -- 172.21.15.46:0/744934200 wait complete. 2024-05-09T05:01:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.633+0000 7f804ff7d640 1 Processor -- start 2024-05-09T05:01:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.633+0000 7f804ff7d640 1 -- start start 2024-05-09T05:01:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.633+0000 7f804ff7d640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8048103270 0x7f8048199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.633+0000 7f804ff7d640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8048108b20 0x7f8048199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.633+0000 7f804ff7d640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8048108fc0 0x7f804819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.633+0000 7f804ff7d640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f8048112d60 con 0x7f8048108fc0 2024-05-09T05:01:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.633+0000 7f804ff7d640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f8048112be0 con 0x7f8048108b20 2024-05-09T05:01:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.633+0000 7f804ff7d640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f8048112ee0 con 0x7f8048103270 2024-05-09T05:01:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f804e51a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8048108fc0 0x7f804819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f804e51a640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8048108fc0 0x7f804819df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48770/0 (socket says 172.21.15.46:48770) 2024-05-09T05:01:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f804e51a640 1 -- 172.21.15.46:0/3637598237 learned_addr learned my addr 172.21.15.46:0/3637598237 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:45.636 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f804dd19640 1 --2- 172.21.15.46:0/3637598237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8048103270 0x7f8048199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f804d518640 1 --2- 172.21.15.46:0/3637598237 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8048108b20 0x7f8048199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f804e51a640 1 -- 172.21.15.46:0/3637598237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8048103270 msgr2=0x7f8048199780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f804e51a640 1 --2- 172.21.15.46:0/3637598237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8048103270 0x7f8048199780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f804e51a640 1 -- 172.21.15.46:0/3637598237 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8048108b20 msgr2=0x7f8048199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f804e51a640 1 --2- 172.21.15.46:0/3637598237 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8048108b20 0x7f8048199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f804e51a640 1 -- 172.21.15.46:0/3637598237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804819e590 con 0x7f8048108fc0 2024-05-09T05:01:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f804e51a640 1 --2- 172.21.15.46:0/3637598237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8048108fc0 0x7f804819df50 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f804000e970 tx=0x7f804000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f803effd640 1 -- 172.21.15.46:0/3637598237 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f804000ccc0 con 0x7f8048108fc0 2024-05-09T05:01:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f803effd640 1 -- 172.21.15.46:0/3637598237 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f804000ce60 con 0x7f8048108fc0 2024-05-09T05:01:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f804ff7d640 1 -- 172.21.15.46:0/3637598237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804819e880 con 0x7f8048108fc0 2024-05-09T05:01:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f803effd640 1 -- 172.21.15.46:0/3637598237 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8040010430 con 0x7f8048108fc0 2024-05-09T05:01:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.634+0000 7f804ff7d640 1 -- 172.21.15.46:0/3637598237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80481a6200 con 0x7f8048108fc0 2024-05-09T05:01:45.637 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.635+0000 7f804ff7d640 1 -- 172.21.15.46:0/3637598237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f804806b0d0 con 0x7f8048108fc0 2024-05-09T05:01:45.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.637+0000 7f803effd640 1 -- 172.21.15.46:0/3637598237 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8040004f70 con 0x7f8048108fc0 2024-05-09T05:01:45.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.638+0000 7f803effd640 1 --2- 172.21.15.46:0/3637598237 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f80340b29e0 0x7f80340b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:45.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.638+0000 7f804dd19640 1 --2- 172.21.15.46:0/3637598237 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f80340b29e0 0x7f80340b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:45.638 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.638+0000 7f803effd640 1 -- 172.21.15.46:0/3637598237 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7f80400e2bc0 con 0x7f8048108fc0 2024-05-09T05:01:45.639 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.638+0000 7f804dd19640 1 --2- 172.21.15.46:0/3637598237 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f80340b29e0 0x7f80340b4ef0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8044004af0 tx=0x7f8044005fd0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:45.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.641+0000 7f803effd640 1 -- 172.21.15.46:0/3637598237 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8040054df0 con 0x7f8048108fc0 2024-05-09T05:01:45.839 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.839+0000 7f804ff7d640 1 -- 172.21.15.46:0/3637598237 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8048107020 con 0x7f8048108fc0 2024-05-09T05:01:45.841 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.841+0000 7f803effd640 1 -- 172.21.15.46:0/3637598237 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+23152 (secure 0 0 0) 0x7f80400aa550 con 0x7f8048108fc0 2024-05-09T05:01:45.841 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T05:01:45.842 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":93,"fsid":"ecf9ae2c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:25.633876+0000","modified":"2024-05-09T05:01:21.129300+0000","last_up_change":"2024-05-09T04:46:46.418597+0000","last_in_change":"2024-05-09T04:46:25.903582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:42:53.301400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:47:43.413569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:47:47.264915+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":92,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d79cd3e7-6f82-4d60-96d9-26d8146c962f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6803","nonce":1576436508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6805","nonce":1576436508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6809","nonce":1576436508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6807","nonce":1576436508}]},"public_addr":"172.21.15.46:6803/1576436508","cluster_addr":"172.21.15.46:6805/1576436508","heartbeat_back_addr":"172.21.15.46:6809/1576436508","heartbeat_front_addr":"172.21.15.46:6807/1576436508","state":["exists","up"]},{"osd":1,"uuid":"698eba80-1ebd-4076-b6f9-953d2e7daf76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6801","nonce":379039775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6803","nonce":379039775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6807","nonce":379039775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6805","nonce":379039775}]},"public_addr":"172.21.15.72:6801/379039775","cluster_addr":"172.21.15.72:6803/379039775","heartbeat_back_addr":"172.21.15.72:6807/379039775","heartbeat_front_addr":"172.21.15.72:6805/379039775","state":["exists","up"]},{"osd":2,"uuid":"c549b131-cfa7-4724-926f-83e91a752d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6801","nonce":293447227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6803","nonce":293447227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6807","nonce":293447227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6805","nonce":293447227}]},"public_addr":"172.21.15.110:6801/293447227","cluster_addr":"172.21.15.110:6803/293447227","heartbeat_back_addr":"172.21.15.110:6807/293447227","heartbeat_front_addr":"172.21.15.110:6805/293447227","state":["exists","up"]},{"osd":3,"uuid":"549bb4f7-54d9-4cbc-ad40-7f3e717964f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6811","nonce":1115835653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6813","nonce":1115835653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6817","nonce":1115835653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6815","nonce":1115835653}]},"public_addr":"172.21.15.46:6811/1115835653","cluster_addr":"172.21.15.46:6813/1115835653","heartbeat_back_addr":"172.21.15.46:6817/1115835653","heartbeat_front_addr":"172.21.15.46:6815/1115835653","state":["exists","up"]},{"osd":4,"uuid":"444cd36b-6034-46cd-9a6b-29dce4ba6405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6809","nonce":59186693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6811","nonce":59186693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6815","nonce":59186693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6813","nonce":59186693}]},"public_addr":"172.21.15.72:6809/59186693","cluster_addr":"172.21.15.72:6811/59186693","heartbeat_back_addr":"172.21.15.72:6815/59186693","heartbeat_front_addr":"172.21.15.72:6813/59186693","state":["exists","up"]},{"osd":5,"uuid":"2c2bdd2f-43cf-4208-9ae1-f912807c84e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6809","nonce":1712162502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6811","nonce":1712162502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6815","nonce":1712162502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6813","nonce":1712162502}]},"public_addr":"172.21.15.110:6809/1712162502","cluster_addr":"172.21.15.110:6811/1712162502","heartbeat_back_addr":"172.21.15.110:6815/1712162502","heartbeat_front_addr":"172.21.15.110:6813/1712162502","state":["exists","up"]},{"osd":6,"uuid":"beb0b87f-c3a4-40dd-b224-1843e9301ca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6819","nonce":2429650065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6821","nonce":2429650065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6825","nonce":2429650065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6823","nonce":2429650065}]},"public_addr":"172.21.15.46:6819/2429650065","cluster_addr":"172.21.15.46:6821/2429650065","heartbeat_back_addr":"172.21.15.46:6825/2429650065","heartbeat_front_addr":"172.21.15.46:6823/2429650065","state":["exists","up"]},{"osd":7,"uuid":"4a634675-fad7-4a43-840c-8dec2631dec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6817","nonce":337796672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6819","nonce":337796672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6823","nonce":337796672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6821","nonce":337796672}]},"public_addr":"172.21.15.72:6817/337796672","cluster_addr":"172.21.15.72:6819/337796672","heartbeat_back_addr":"172.21.15.72:6823/337796672","heartbeat_front_addr":"172.21.15.72:6821/337796672","state":["exists","up"]},{"osd":8,"uuid":"01b5f638-90f0-4cf4-8042-20d20a725669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6817","nonce":189513203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6819","nonce":189513203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6823","nonce":189513203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6821","nonce":189513203}]},"public_addr":"172.21.15.110:6817/189513203","cluster_addr":"172.21.15.110:6819/189513203","heartbeat_back_addr":"172.21.15.110:6823/189513203","heartbeat_front_addr":"172.21.15.110:6821/189513203","state":["exists","up"]},{"osd":9,"uuid":"e423de41-50ec-48f4-a4ed-216de33c4fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6827","nonce":1762262822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6829","nonce":1762262822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6833","nonce":1762262822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6831","nonce":1762262822}]},"public_addr":"172.21.15.46:6827/1762262822","cluster_addr":"172.21.15.46:6829/1762262822","heartbeat_back_addr":"172.21.15.46:6833/1762262822","heartbeat_front_addr":"172.21.15.46:6831/1762262822","state":["exists","up"]},{"osd":10,"uuid":"f594d004-9f80-4508-b127-fb693d355f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6825","nonce":38453898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6827","nonce":38453898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6831","nonce":38453898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6829","nonce":38453898}]},"public_addr":"172.21.15.72:6825/38453898","cluster_addr":"172.21.15.72:6827/38453898","heartbeat_back_addr":"172.21.15.72:6831/38453898","heartbeat_front_addr":"172.21.15.72:6829/38453898","state":["exists","up"]},{"osd":11,"uuid":"f771d29f-89e3-4f96-9c29-00e424230c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6825","nonce":4153529632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6827","nonce":4153529632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6831","nonce":4153529632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6829","nonce":4153529632}]},"public_addr":"172.21.15.110:6825/4153529632","cluster_addr":"172.21.15.110:6827/4153529632","heartbeat_back_addr":"172.21.15.110:6831/4153529632","heartbeat_front_addr":"172.21.15.110:6829/4153529632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:02.615801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:26.453477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:50.334627+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:15.357320+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:40.403549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:06.030093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:31.904091+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:57.616372+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:24.045975+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:51.160898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:17.597845+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:44.636629+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/181034377":"2024-05-10T04:51:05.539029+0000","172.21.15.46:0/1963687293":"2024-05-10T04:51:05.539029+0000","172.21.15.46:0/2215389074":"2024-05-10T04:51:05.539029+0000","172.21.15.46:6800/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1596672353":"2024-05-10T04:40:27.201119+0000","172.21.15.46:6801/3691531627":"2024-05-10T04:51:05.539029+0000","172.21.15.46:0/1833069872":"2024-05-10T04:51:05.539029+0000","172.21.15.46:0/1846482551":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/1069571878":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6800/2738158981":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/3625745146":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6800/3691531627":"2024-05-10T04:51:05.539029+0000","172.21.15.46:6801/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/564539954":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1797199863":"2024-05-10T04:40:27.201119+0000","172.21.15.46:6801/2738158981":"2024-05-10T04:39:59.515628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:01:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.843+0000 7f804ff7d640 1 -- 172.21.15.46:0/3637598237 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f80340b29e0 msgr2=0x7f80340b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:45.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.843+0000 7f804ff7d640 1 --2- 172.21.15.46:0/3637598237 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f80340b29e0 0x7f80340b4ef0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f8044004af0 tx=0x7f8044005fd0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.843+0000 7f804ff7d640 1 -- 172.21.15.46:0/3637598237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8048108fc0 msgr2=0x7f804819df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.843+0000 7f804ff7d640 1 --2- 172.21.15.46:0/3637598237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8048108fc0 0x7f804819df50 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f804000e970 tx=0x7f804000ed70 comp rx=0 tx=0).stop 2024-05-09T05:01:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.843+0000 7f804ff7d640 1 -- 172.21.15.46:0/3637598237 shutdown_connections 2024-05-09T05:01:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.843+0000 7f804ff7d640 1 --2- 172.21.15.46:0/3637598237 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f80340b29e0 0x7f80340b4ef0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.843+0000 7f804ff7d640 1 --2- 172.21.15.46:0/3637598237 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f8048108fc0 0x7f804819df50 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.843+0000 7f804ff7d640 1 --2- 172.21.15.46:0/3637598237 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f8048108b20 0x7f8048199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.843+0000 7f804ff7d640 1 --2- 172.21.15.46:0/3637598237 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f8048103270 0x7f8048199780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.843+0000 7f804ff7d640 1 -- 172.21.15.46:0/3637598237 >> 172.21.15.46:0/3637598237 conn(0x7f8048078240 msgr2=0x7f80481041c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.843+0000 7f804ff7d640 1 -- 172.21.15.46:0/3637598237 shutdown_connections 2024-05-09T05:01:45.844 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.843+0000 7f804ff7d640 1 -- 172.21.15.46:0/3637598237 wait complete. 2024-05-09T05:01:45.857 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-09T04:42:53.301400+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-09T04:47:43.413569+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-09T04:47:47.264915+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 92, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-09T05:01:45.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-09T05:01:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.975+0000 7f15f8f86640 1 -- 172.21.15.46:0/2158211123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f4102420 msgr2=0x7f15f4102830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.975+0000 7f15f8f86640 1 --2- 172.21.15.46:0/2158211123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f4102420 0x7f15f4102830 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f15e4009900 tx=0x7f15e402a410 comp rx=0 tx=0).stop 2024-05-09T05:01:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.976+0000 7f15f8f86640 1 -- 172.21.15.46:0/2158211123 shutdown_connections 2024-05-09T05:01:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.976+0000 7f15f8f86640 1 --2- 172.21.15.46:0/2158211123 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f4108f60 0x7f15f4110690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.976+0000 7f15f8f86640 1 --2- 172.21.15.46:0/2158211123 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15f41084a0 0x7f15f41088d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.976+0000 7f15f8f86640 1 --2- 172.21.15.46:0/2158211123 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f4102420 0x7f15f4102830 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.976+0000 7f15f8f86640 1 -- 172.21.15.46:0/2158211123 >> 172.21.15.46:0/2158211123 conn(0x7f15f40779b0 msgr2=0x7f15f4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:45.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.976+0000 7f15f8f86640 1 -- 172.21.15.46:0/2158211123 shutdown_connections 2024-05-09T05:01:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.976+0000 7f15f8f86640 1 -- 172.21.15.46:0/2158211123 wait complete. 2024-05-09T05:01:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.976+0000 7f15f8f86640 1 Processor -- start 2024-05-09T05:01:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f8f86640 1 -- start start 2024-05-09T05:01:45.979 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f8f86640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f4102420 0x7f15f41a0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f8f86640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f41084a0 0x7f15f41a0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f8f86640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15f4108f60 0x7f15f419a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f8f86640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f15f4113e00 con 0x7f15f4102420 2024-05-09T05:01:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f8f86640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f15f4113c80 con 0x7f15f4108f60 2024-05-09T05:01:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f8f86640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f15f4113f80 con 0x7f15f41084a0 2024-05-09T05:01:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f4102420 0x7f15f41a0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f259c640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f4102420 0x7f15f41a0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48798/0 (socket says 172.21.15.46:48798) 2024-05-09T05:01:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f259c640 1 -- 172.21.15.46:0/3538324944 learned_addr learned my addr 172.21.15.46:0/3538324944 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15e9d9b640 1 --2- 172.21.15.46:0/3538324944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f41084a0 0x7f15f41a0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f2d9d640 1 --2- 172.21.15.46:0/3538324944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15f4108f60 0x7f15f419a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f259c640 1 -- 172.21.15.46:0/3538324944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f41084a0 msgr2=0x7f15f41a0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:45.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f259c640 1 --2- 172.21.15.46:0/3538324944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f41084a0 0x7f15f41a0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f259c640 1 -- 172.21.15.46:0/3538324944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15f4108f60 msgr2=0x7f15f419a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f259c640 1 --2- 172.21.15.46:0/3538324944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15f4108f60 0x7f15f419a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.977+0000 7f15f259c640 1 -- 172.21.15.46:0/3538324944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f419ae10 con 0x7f15f4102420 2024-05-09T05:01:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.978+0000 7f15f259c640 1 --2- 172.21.15.46:0/3538324944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f4102420 0x7f15f41a0560 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f15e4005940 tx=0x7f15e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.978+0000 7f15ebfff640 1 -- 172.21.15.46:0/3538324944 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e403f030 con 0x7f15f4102420 2024-05-09T05:01:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.978+0000 7f15ebfff640 1 -- 172.21.15.46:0/3538324944 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e4003cb0 con 0x7f15f4102420 2024-05-09T05:01:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.978+0000 7f15f8f86640 1 -- 172.21.15.46:0/3538324944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f419b040 con 0x7f15f4102420 2024-05-09T05:01:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.978+0000 7f15ebfff640 1 -- 172.21.15.46:0/3538324944 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e4035070 con 0x7f15f4102420 2024-05-09T05:01:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.978+0000 7f15f8f86640 1 -- 172.21.15.46:0/3538324944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f419b530 con 0x7f15f4102420 2024-05-09T05:01:45.981 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.979+0000 7f15f8f86640 1 -- 172.21.15.46:0/3538324944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f4105080 con 0x7f15f4102420 2024-05-09T05:01:45.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.980+0000 7f15ebfff640 1 -- 172.21.15.46:0/3538324944 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f15e4042030 con 0x7f15f4102420 2024-05-09T05:01:45.982 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.981+0000 7f15ebfff640 1 --2- 172.21.15.46:0/3538324944 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f15e00b2960 0x7f15e00b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:45.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.981+0000 7f15e9d9b640 1 --2- 172.21.15.46:0/3538324944 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f15e00b2960 0x7f15e00b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:45.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.981+0000 7f15ebfff640 1 -- 172.21.15.46:0/3538324944 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7f15e4103170 con 0x7f15f4102420 2024-05-09T05:01:45.983 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.982+0000 7f15e9d9b640 1 --2- 172.21.15.46:0/3538324944 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f15e00b2960 0x7f15e00b4e70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f15f419be80 tx=0x7f15dc00b390 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:45.984 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:45.985+0000 7f15ebfff640 1 -- 172.21.15.46:0/3538324944 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f15e40ca450 con 0x7f15f4102420 2024-05-09T05:01:46.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:45 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3637598237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:01:46.175 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.175+0000 7f15f8f86640 1 -- 172.21.15.46:0/3538324944 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f15f406a290 con 0x7f15f4102420 2024-05-09T05:01:46.176 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.176+0000 7f15ebfff640 1 -- 172.21.15.46:0/3538324944 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f15f406a290 con 0x7f15f4102420 2024-05-09T05:01:46.176 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 1 2024-05-09T05:01:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.178+0000 7f15f8f86640 1 -- 172.21.15.46:0/3538324944 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f15e00b2960 msgr2=0x7f15e00b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.178+0000 7f15f8f86640 1 --2- 172.21.15.46:0/3538324944 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f15e00b2960 0x7f15e00b4e70 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f15f419be80 tx=0x7f15dc00b390 comp rx=0 tx=0).stop 2024-05-09T05:01:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.178+0000 7f15f8f86640 1 -- 172.21.15.46:0/3538324944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f4102420 msgr2=0x7f15f41a0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.178+0000 7f15f8f86640 1 --2- 172.21.15.46:0/3538324944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f4102420 0x7f15f41a0560 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f15e4005940 tx=0x7f15e40042f0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.178+0000 7f15f8f86640 1 -- 172.21.15.46:0/3538324944 shutdown_connections 2024-05-09T05:01:46.178 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.178+0000 7f15f8f86640 1 --2- 172.21.15.46:0/3538324944 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f15e00b2960 0x7f15e00b4e70 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.178+0000 7f15f8f86640 1 --2- 172.21.15.46:0/3538324944 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f15f41084a0 0x7f15f41a0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.178+0000 7f15f8f86640 1 --2- 172.21.15.46:0/3538324944 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f15f4108f60 0x7f15f419a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.178+0000 7f15f8f86640 1 --2- 172.21.15.46:0/3538324944 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f15f4102420 0x7f15f41a0560 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.178+0000 7f15f8f86640 1 -- 172.21.15.46:0/3538324944 >> 172.21.15.46:0/3538324944 conn(0x7f15f40779b0 msgr2=0x7f15f4100ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.178+0000 7f15f8f86640 1 -- 172.21.15.46:0/3538324944 shutdown_connections 2024-05-09T05:01:46.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.178+0000 7f15f8f86640 1 -- 172.21.15.46:0/3538324944 wait complete. 2024-05-09T05:01:46.189 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:45 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3637598237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:01:46.190 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-09T05:01:46.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:45 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3637598237' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:01:46.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.307+0000 7f49c5b1f640 1 -- 172.21.15.46:0/1706688943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49c0109c20 msgr2=0x7f49c0101ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:46.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.307+0000 7f49c5b1f640 1 --2- 172.21.15.46:0/1706688943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49c0109c20 0x7f49c0101ca0 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009900 tx=0x7f49b002a410 comp rx=0 tx=0).stop 2024-05-09T05:01:46.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.308+0000 7f49c5b1f640 1 -- 172.21.15.46:0/1706688943 shutdown_connections 2024-05-09T05:01:46.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.308+0000 7f49c5b1f640 1 --2- 172.21.15.46:0/1706688943 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49c0102850 0x7f49c01106a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.308+0000 7f49c5b1f640 1 --2- 172.21.15.46:0/1706688943 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49c010a0c0 0x7f49c01021e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.308+0000 7f49c5b1f640 1 --2- 172.21.15.46:0/1706688943 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49c0109c20 0x7f49c0101ca0 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.308+0000 7f49c5b1f640 1 -- 172.21.15.46:0/1706688943 >> 172.21.15.46:0/1706688943 conn(0x7f49c00779b0 msgr2=0x7f49c0077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:46.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.308+0000 7f49c5b1f640 1 -- 172.21.15.46:0/1706688943 shutdown_connections 2024-05-09T05:01:46.308 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.308+0000 7f49c5b1f640 1 -- 172.21.15.46:0/1706688943 wait complete. 2024-05-09T05:01:46.310 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.308+0000 7f49c5b1f640 1 Processor -- start 2024-05-09T05:01:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.308+0000 7f49c5b1f640 1 -- start start 2024-05-09T05:01:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49c5b1f640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49c0102850 0x7f49c01a04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49c5b1f640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49c0109c20 0x7f49c01a0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49c5b1f640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49c010a0c0 0x7f49c019a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49c5b1f640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49c0113e30 con 0x7f49c0102850 2024-05-09T05:01:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49c5b1f640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f49c0113cb0 con 0x7f49c010a0c0 2024-05-09T05:01:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49c5b1f640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f49c0113fb0 con 0x7f49c0109c20 2024-05-09T05:01:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49bf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49c0102850 0x7f49c01a04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49bf7fe640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49c0102850 0x7f49c01a04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48814/0 (socket says 172.21.15.46:48814) 2024-05-09T05:01:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49bf7fe640 1 -- 172.21.15.46:0/179832834 learned_addr learned my addr 172.21.15.46:0/179832834 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49bffff640 1 --2- 172.21.15.46:0/179832834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49c010a0c0 0x7f49c019a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49beffd640 1 --2- 172.21.15.46:0/179832834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49c0109c20 0x7f49c01a0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49bf7fe640 1 -- 172.21.15.46:0/179832834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49c0109c20 msgr2=0x7f49c01a0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:46.311 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49bf7fe640 1 --2- 172.21.15.46:0/179832834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49c0109c20 0x7f49c01a0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49bf7fe640 1 -- 172.21.15.46:0/179832834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49c010a0c0 msgr2=0x7f49c019a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49bf7fe640 1 --2- 172.21.15.46:0/179832834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49c010a0c0 0x7f49c019a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49bf7fe640 1 -- 172.21.15.46:0/179832834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c019ada0 con 0x7f49c0102850 2024-05-09T05:01:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.309+0000 7f49bf7fe640 1 --2- 172.21.15.46:0/179832834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49c0102850 0x7f49c01a04f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f49b0005940 tx=0x7f49b00040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.310+0000 7f49bcff9640 1 -- 172.21.15.46:0/179832834 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b003f030 con 0x7f49c0102850 2024-05-09T05:01:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.310+0000 7f49bcff9640 1 -- 172.21.15.46:0/179832834 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49b00044c0 con 0x7f49c0102850 2024-05-09T05:01:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.310+0000 7f49bcff9640 1 -- 172.21.15.46:0/179832834 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b0035070 con 0x7f49c0102850 2024-05-09T05:01:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.310+0000 7f49c5b1f640 1 -- 172.21.15.46:0/179832834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c019afd0 con 0x7f49c0102850 2024-05-09T05:01:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.310+0000 7f49c5b1f640 1 -- 172.21.15.46:0/179832834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c01a7290 con 0x7f49c0102850 2024-05-09T05:01:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.311+0000 7f49c5b1f640 1 -- 172.21.15.46:0/179832834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c0104fc0 con 0x7f49c0102850 2024-05-09T05:01:46.312 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.312+0000 7f49bcff9640 1 -- 172.21.15.46:0/179832834 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f49b0042030 con 0x7f49c0102850 2024-05-09T05:01:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.313+0000 7f49bcff9640 1 --2- 172.21.15.46:0/179832834 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49a80b29e0 0x7f49a80b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.313+0000 7f49beffd640 1 --2- 172.21.15.46:0/179832834 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49a80b29e0 0x7f49a80b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:46.313 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.313+0000 7f49bcff9640 1 -- 172.21.15.46:0/179832834 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7f49b0102c00 con 0x7f49c0102850 2024-05-09T05:01:46.314 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.314+0000 7f49beffd640 1 --2- 172.21.15.46:0/179832834 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49a80b29e0 0x7f49a80b4ef0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f49c019be10 tx=0x7f49b4005f90 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:46.317 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.317+0000 7f49bcff9640 1 -- 172.21.15.46:0/179832834 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49b00ca590 con 0x7f49c0102850 2024-05-09T05:01:46.508 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.507+0000 7f49c5b1f640 1 -- 172.21.15.46:0/179832834 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f49c0108150 con 0x7f49c0102850 2024-05-09T05:01:46.509 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.508+0000 7f49bcff9640 1 -- 172.21.15.46:0/179832834 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f49b00c9e90 con 0x7f49c0102850 2024-05-09T05:01:46.509 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 32 2024-05-09T05:01:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.510+0000 7f49c5b1f640 1 -- 172.21.15.46:0/179832834 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49a80b29e0 msgr2=0x7f49a80b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.510+0000 7f49c5b1f640 1 --2- 172.21.15.46:0/179832834 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49a80b29e0 0x7f49a80b4ef0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f49c019be10 tx=0x7f49b4005f90 comp rx=0 tx=0).stop 2024-05-09T05:01:46.510 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.510+0000 7f49c5b1f640 1 -- 172.21.15.46:0/179832834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49c0102850 msgr2=0x7f49c01a04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.510+0000 7f49c5b1f640 1 --2- 172.21.15.46:0/179832834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49c0102850 0x7f49c01a04f0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f49b0005940 tx=0x7f49b00040c0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.510+0000 7f49c5b1f640 1 -- 172.21.15.46:0/179832834 shutdown_connections 2024-05-09T05:01:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.510+0000 7f49c5b1f640 1 --2- 172.21.15.46:0/179832834 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49a80b29e0 0x7f49a80b4ef0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.510+0000 7f49c5b1f640 1 --2- 172.21.15.46:0/179832834 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49c0109c20 0x7f49c01a0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.510+0000 7f49c5b1f640 1 --2- 172.21.15.46:0/179832834 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49c010a0c0 0x7f49c019a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.510+0000 7f49c5b1f640 1 --2- 172.21.15.46:0/179832834 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49c0102850 0x7f49c01a04f0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.510+0000 7f49c5b1f640 1 -- 172.21.15.46:0/179832834 >> 172.21.15.46:0/179832834 conn(0x7f49c00779b0 msgr2=0x7f49c0105320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.510+0000 7f49c5b1f640 1 -- 172.21.15.46:0/179832834 shutdown_connections 2024-05-09T05:01:46.511 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.510+0000 7f49c5b1f640 1 -- 172.21.15.46:0/179832834 wait complete. 2024-05-09T05:01:46.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-09T05:01:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.640+0000 7f1212d15640 1 -- 172.21.15.46:0/1960351723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f120c109c40 msgr2=0x7f120c1037e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.640+0000 7f1212d15640 1 --2- 172.21.15.46:0/1960351723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f120c109c40 0x7f120c1037e0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f1200009900 tx=0x7f120002a410 comp rx=0 tx=0).stop 2024-05-09T05:01:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.640+0000 7f1212d15640 1 -- 172.21.15.46:0/1960351723 shutdown_connections 2024-05-09T05:01:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.640+0000 7f1212d15640 1 --2- 172.21.15.46:0/1960351723 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f120c104390 0x7f120c110760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.640+0000 7f1212d15640 1 --2- 172.21.15.46:0/1960351723 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f120c10a0e0 0x7f120c103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.640+0000 7f1212d15640 1 --2- 172.21.15.46:0/1960351723 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f120c109c40 0x7f120c1037e0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.640+0000 7f1212d15640 1 -- 172.21.15.46:0/1960351723 >> 172.21.15.46:0/1960351723 conn(0x7f120c0ff360 msgr2=0x7f120c1017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.640+0000 7f1212d15640 1 -- 172.21.15.46:0/1960351723 shutdown_connections 2024-05-09T05:01:46.641 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.640+0000 7f1212d15640 1 -- 172.21.15.46:0/1960351723 wait complete. 2024-05-09T05:01:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.641+0000 7f1212d15640 1 Processor -- start 2024-05-09T05:01:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.641+0000 7f1212d15640 1 -- start start 2024-05-09T05:01:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.641+0000 7f1212d15640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f120c104390 0x7f120c1a0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.641+0000 7f1212d15640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f120c109c40 0x7f120c1a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.641+0000 7f1212d15640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f120c10a0e0 0x7f120c19a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f1212d15640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f120c113e30 con 0x7f120c109c40 2024-05-09T05:01:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f1212d15640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f120c113cb0 con 0x7f120c10a0e0 2024-05-09T05:01:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f1212d15640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f120c113fb0 con 0x7f120c104390 2024-05-09T05:01:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f120bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f120c109c40 0x7f120c1a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f120bfff640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f120c109c40 0x7f120c1a0a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48824/0 (socket says 172.21.15.46:48824) 2024-05-09T05:01:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f120bfff640 1 -- 172.21.15.46:0/2339737838 learned_addr learned my addr 172.21.15.46:0/2339737838 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:46.644 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f1210ab1640 1 --2- 172.21.15.46:0/2339737838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f120c104390 0x7f120c1a0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f12112b2640 1 --2- 172.21.15.46:0/2339737838 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f120c10a0e0 0x7f120c19a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f120bfff640 1 -- 172.21.15.46:0/2339737838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f120c104390 msgr2=0x7f120c1a0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f120bfff640 1 --2- 172.21.15.46:0/2339737838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f120c104390 0x7f120c1a0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f120bfff640 1 -- 172.21.15.46:0/2339737838 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f120c10a0e0 msgr2=0x7f120c19a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f120bfff640 1 --2- 172.21.15.46:0/2339737838 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f120c10a0e0 0x7f120c19a6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f120bfff640 1 -- 172.21.15.46:0/2339737838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f120c19ad10 con 0x7f120c109c40 2024-05-09T05:01:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f120bfff640 1 --2- 172.21.15.46:0/2339737838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f120c109c40 0x7f120c1a0a90 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f11f800c970 tx=0x7f11f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f1209ffb640 1 -- 172.21.15.46:0/2339737838 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f8007c70 con 0x7f120c109c40 2024-05-09T05:01:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f1209ffb640 1 -- 172.21.15.46:0/2339737838 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f8007e10 con 0x7f120c109c40 2024-05-09T05:01:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f1212d15640 1 -- 172.21.15.46:0/2339737838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f120c19b000 con 0x7f120c109c40 2024-05-09T05:01:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.642+0000 7f1209ffb640 1 -- 172.21.15.46:0/2339737838 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f80187b0 con 0x7f120c109c40 2024-05-09T05:01:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.643+0000 7f1212d15640 1 -- 172.21.15.46:0/2339737838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f120c1a72f0 con 0x7f120c109c40 2024-05-09T05:01:46.645 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.643+0000 7f1212d15640 1 -- 172.21.15.46:0/2339737838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f120c104f20 con 0x7f120c109c40 2024-05-09T05:01:46.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.645+0000 7f1209ffb640 1 -- 172.21.15.46:0/2339737838 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f11f800fab0 con 0x7f120c109c40 2024-05-09T05:01:46.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.646+0000 7f1209ffb640 1 --2- 172.21.15.46:0/2339737838 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f11fc0b29e0 0x7f11fc0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:46.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.646+0000 7f1210ab1640 1 --2- 172.21.15.46:0/2339737838 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f11fc0b29e0 0x7f11fc0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:46.646 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.646+0000 7f1209ffb640 1 -- 172.21.15.46:0/2339737838 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7f11f80e2660 con 0x7f120c109c40 2024-05-09T05:01:46.647 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.647+0000 7f1210ab1640 1 --2- 172.21.15.46:0/2339737838 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f11fc0b29e0 0x7f11fc0b4ef0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1200000c00 tx=0x7f1200031400 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:46.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.651+0000 7f1209ffb640 1 -- 172.21.15.46:0/2339737838 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f11f8014030 con 0x7f120c109c40 2024-05-09T05:01:46.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.843+0000 7f1212d15640 1 -- 172.21.15.46:0/2339737838 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f120c1a74c0 con 0x7f120c109c40 2024-05-09T05:01:46.843 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.843+0000 7f1209ffb640 1 -- 172.21.15.46:0/2339737838 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f11f80a9f90 con 0x7f120c109c40 2024-05-09T05:01:46.843 INFO:teuthology.orchestra.run.smithi046.stdout:pg_num: 64 2024-05-09T05:01:46.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.845+0000 7f1212d15640 1 -- 172.21.15.46:0/2339737838 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f11fc0b29e0 msgr2=0x7f11fc0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:46.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.845+0000 7f1212d15640 1 --2- 172.21.15.46:0/2339737838 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f11fc0b29e0 0x7f11fc0b4ef0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1200000c00 tx=0x7f1200031400 comp rx=0 tx=0).stop 2024-05-09T05:01:46.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.845+0000 7f1212d15640 1 -- 172.21.15.46:0/2339737838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f120c109c40 msgr2=0x7f120c1a0a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:46.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.845+0000 7f1212d15640 1 --2- 172.21.15.46:0/2339737838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f120c109c40 0x7f120c1a0a90 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f11f800c970 tx=0x7f11f800ce30 comp rx=0 tx=0).stop 2024-05-09T05:01:46.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.845+0000 7f1212d15640 1 -- 172.21.15.46:0/2339737838 shutdown_connections 2024-05-09T05:01:46.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.845+0000 7f1212d15640 1 --2- 172.21.15.46:0/2339737838 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f11fc0b29e0 0x7f11fc0b4ef0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.845+0000 7f1212d15640 1 --2- 172.21.15.46:0/2339737838 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f120c10a0e0 0x7f120c19a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.845+0000 7f1212d15640 1 --2- 172.21.15.46:0/2339737838 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f120c109c40 0x7f120c1a0a90 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.845+0000 7f1212d15640 1 --2- 172.21.15.46:0/2339737838 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f120c104390 0x7f120c1a0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.845+0000 7f1212d15640 1 -- 172.21.15.46:0/2339737838 >> 172.21.15.46:0/2339737838 conn(0x7f120c0ff360 msgr2=0x7f120c0ff770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:46.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.845+0000 7f1212d15640 1 -- 172.21.15.46:0/2339737838 shutdown_connections 2024-05-09T05:01:46.846 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.845+0000 7f1212d15640 1 -- 172.21.15.46:0/2339737838 wait complete. 2024-05-09T05:01:46.857 DEBUG:teuthology.orchestra.run.smithi046:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-09T05:01:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.972+0000 7fdf8f7be640 1 -- 172.21.15.46:0/4056604549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf88109c60 msgr2=0x7fdf88103800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:46.974 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.972+0000 7fdf8f7be640 1 --2- 172.21.15.46:0/4056604549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf88109c60 0x7fdf88103800 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fdf84009900 tx=0x7fdf8402a410 comp rx=0 tx=0).stop 2024-05-09T05:01:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.972+0000 7fdf8f7be640 1 -- 172.21.15.46:0/4056604549 shutdown_connections 2024-05-09T05:01:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.972+0000 7fdf8f7be640 1 --2- 172.21.15.46:0/4056604549 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf8810a100 0x7fdf88103d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.972+0000 7fdf8f7be640 1 --2- 172.21.15.46:0/4056604549 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf881043b0 0x7fdf88110780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.972+0000 7fdf8f7be640 1 --2- 172.21.15.46:0/4056604549 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf88109c60 0x7fdf88103800 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.972+0000 7fdf8f7be640 1 -- 172.21.15.46:0/4056604549 >> 172.21.15.46:0/4056604549 conn(0x7fdf880ff340 msgr2=0x7fdf88101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.972+0000 7fdf8f7be640 1 -- 172.21.15.46:0/4056604549 shutdown_connections 2024-05-09T05:01:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.972+0000 7fdf8f7be640 1 -- 172.21.15.46:0/4056604549 wait complete. 2024-05-09T05:01:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8f7be640 1 Processor -- start 2024-05-09T05:01:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8f7be640 1 -- start start 2024-05-09T05:01:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8f7be640 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf881043b0 0x7fdf881a05f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8f7be640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf88109c60 0x7fdf881a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8f7be640 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf8810a100 0x7fdf8819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:46.975 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8f7be640 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fdf88113eb0 con 0x7fdf88109c60 2024-05-09T05:01:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8f7be640 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fdf88113d30 con 0x7fdf881043b0 2024-05-09T05:01:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8f7be640 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fdf88114030 con 0x7fdf8810a100 2024-05-09T05:01:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8cd59640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf88109c60 0x7fdf881a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8cd59640 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf88109c60 0x7fdf881a0b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:48842/0 (socket says 172.21.15.46:48842) 2024-05-09T05:01:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8cd59640 1 -- 172.21.15.46:0/1250331018 learned_addr learned my addr 172.21.15.46:0/1250331018 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:01:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8d55a640 1 --2- 172.21.15.46:0/1250331018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf881043b0 0x7fdf881a05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8dd5b640 1 --2- 172.21.15.46:0/1250331018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf8810a100 0x7fdf8819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8cd59640 1 -- 172.21.15.46:0/1250331018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf8810a100 msgr2=0x7fdf8819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8cd59640 1 --2- 172.21.15.46:0/1250331018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf8810a100 0x7fdf8819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8cd59640 1 -- 172.21.15.46:0/1250331018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf881043b0 msgr2=0x7fdf881a05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8cd59640 1 --2- 172.21.15.46:0/1250331018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf881043b0 0x7fdf881a05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8cd59640 1 -- 172.21.15.46:0/1250331018 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8819adb0 con 0x7fdf88109c60 2024-05-09T05:01:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.973+0000 7fdf8dd5b640 1 --2- 172.21.15.46:0/1250331018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf8810a100 0x7fdf8819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:01:46.976 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.974+0000 7fdf8cd59640 1 --2- 172.21.15.46:0/1250331018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf88109c60 0x7fdf881a0b30 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fdf78006fd0 tx=0x7fdf7800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.974+0000 7fdf7e7fc640 1 -- 172.21.15.46:0/1250331018 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78011830 con 0x7fdf88109c60 2024-05-09T05:01:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.974+0000 7fdf8f7be640 1 -- 172.21.15.46:0/1250331018 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8819b0a0 con 0x7fdf88109c60 2024-05-09T05:01:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.974+0000 7fdf7e7fc640 1 -- 172.21.15.46:0/1250331018 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf78016070 con 0x7fdf88109c60 2024-05-09T05:01:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.974+0000 7fdf7e7fc640 1 -- 172.21.15.46:0/1250331018 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf78010e00 con 0x7fdf88109c60 2024-05-09T05:01:46.977 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.974+0000 7fdf8f7be640 1 -- 172.21.15.46:0/1250331018 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf8819b5a0 con 0x7fdf88109c60 2024-05-09T05:01:46.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.975+0000 7fdf8f7be640 1 -- 172.21.15.46:0/1250331018 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf88104f40 con 0x7fdf88109c60 2024-05-09T05:01:46.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.977+0000 7fdf7e7fc640 1 -- 172.21.15.46:0/1250331018 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdf780119d0 con 0x7fdf88109c60 2024-05-09T05:01:46.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.977+0000 7fdf7e7fc640 1 --2- 172.21.15.46:0/1250331018 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdf740b2960 0x7fdf740b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:01:46.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.977+0000 7fdf7e7fc640 1 -- 172.21.15.46:0/1250331018 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 8600+0+0 (secure 0 0 0) 0x7fdf780e2250 con 0x7fdf88109c60 2024-05-09T05:01:46.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.977+0000 7fdf8d55a640 1 --2- 172.21.15.46:0/1250331018 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdf740b2960 0x7fdf740b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:01:46.978 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.978+0000 7fdf8d55a640 1 --2- 172.21.15.46:0/1250331018 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdf740b2960 0x7fdf740b4e70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf84000c00 tx=0x7fdf84031400 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:01:46.980 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:46.981+0000 7fdf7e7fc640 1 -- 172.21.15.46:0/1250331018 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf780a9cf0 con 0x7fdf88109c60 2024-05-09T05:01:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:46 smithi072 ceph-mon[34088]: pgmap v331: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:46 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3538324944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T05:01:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:46 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/179832834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T05:01:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:46 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2339737838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T05:01:47.227 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:47.227+0000 7fdf8f7be640 1 -- 172.21.15.46:0/1250331018 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fdf881a7450 con 0x7fdf88109c60 2024-05-09T05:01:47.236 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:47.237+0000 7fdf7e7fc640 1 -- 172.21.15.46:0/1250331018 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+816175 (secure 0 0 0) 0x7fdf780a95f0 con 0x7fdf88109c60 2024-05-09T05:01:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T05:01:47.247 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T05:01:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "epoch": 26, 2024-05-09T05:01:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "active_gid": 15150, 2024-05-09T05:01:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "active_name": "x", 2024-05-09T05:01:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "active_addrs": { 2024-05-09T05:01:47.247 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-09T05:01:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-09T05:01:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6800", 2024-05-09T05:01:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2595796783 2024-05-09T05:01:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v1", 2024-05-09T05:01:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:6801", 2024-05-09T05:01:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2595796783 2024-05-09T05:01:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T05:01:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "active_addr": "172.21.15.46:6801/2595796783", 2024-05-09T05:01:47.248 INFO:teuthology.orchestra.run.smithi046.stdout: "active_change": "2024-05-09T04:51:05.539261+0000", 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "active_mgr_features": 4540701547738038271, 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "available": true, 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "standbys": [ 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 24880, 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "z", 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_features": 4540701547738038271, 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T05:01:47.249 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T05:01:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T05:01:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.250 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.251 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.252 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.253 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.254 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.255 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-05-09T05:01:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-05-09T05:01:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.256 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-05-09T05:01:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-05-09T05:01:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-05-09T05:01:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-05-09T05:01:47.257 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-05-09T05:01:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-05-09T05:01:47.258 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.259 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.260 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-05-09T05:01:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-05-09T05:01:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-05-09T05:01:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-05-09T05:01:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.261 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-05-09T05:01:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-05-09T05:01:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-05-09T05:01:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.262 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T05:01:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-05-09T05:01:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-05-09T05:01:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.263 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-05-09T05:01:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.264 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-05-09T05:01:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.265 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-05-09T05:01:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-05-09T05:01:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T05:01:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-05-09T05:01:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-05-09T05:01:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-05-09T05:01:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T05:01:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T05:01:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.267 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-05-09T05:01:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-05-09T05:01:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T05:01:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-09T05:01:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-09T05:01:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T05:01:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T05:01:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-05-09T05:01:47.268 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-05-09T05:01:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T05:01:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-05-09T05:01:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T05:01:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-05-09T05:01:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T05:01:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-05-09T05:01:47.269 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-05-09T05:01:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T05:01:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T05:01:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T05:01:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-05-09T05:01:47.270 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-05-09T05:01:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-05-09T05:01:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-05-09T05:01:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-05-09T05:01:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-05-09T05:01:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T05:01:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-05-09T05:01:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-05-09T05:01:47.271 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-05-09T05:01:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T05:01:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T05:01:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-05-09T05:01:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-05-09T05:01:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T05:01:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-09T05:01:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-09T05:01:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T05:01:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T05:01:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.273 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.274 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.275 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.276 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.277 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-05-09T05:01:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-05-09T05:01:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.278 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T05:01:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-09T05:01:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-09T05:01:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.279 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "read", 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap", 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap-read" 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-05-09T05:01:47.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T05:01:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T05:01:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T05:01:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.281 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T05:01:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T05:01:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.282 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-05-09T05:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-05-09T05:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.283 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T05:01:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T05:01:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T05:01:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T05:01:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-05-09T05:01:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-05-09T05:01:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T05:01:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:47.284 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T05:01:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-05-09T05:01:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.285 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-05-09T05:01:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-05-09T05:01:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-09T05:01:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T05:01:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.286 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-05-09T05:01:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-05-09T05:01:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-05-09T05:01:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T05:01:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-05-09T05:01:47.287 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.288 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T05:01:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T05:01:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-05-09T05:01:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-05-09T05:01:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.289 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T05:01:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T05:01:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-05-09T05:01:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T05:01:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.290 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_log_destination": { 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm_log_destination", 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.291 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.293 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "file", 2024-05-09T05:01:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "file,syslog", 2024-05-09T05:01:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "syslog" 2024-05-09T05:01:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T05:01:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-05-09T05:01:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-05-09T05:01:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.294 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.295 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T05:01:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-05-09T05:01:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-05-09T05:01:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.296 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T05:01:47.297 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-05-09T05:01:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-05-09T05:01:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T05:01:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-05-09T05:01:47.298 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-05-09T05:01:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-05-09T05:01:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T05:01:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-05-09T05:01:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.299 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-05-09T05:01:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-05-09T05:01:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T05:01:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T05:01:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.300 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-05-09T05:01:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-05-09T05:01:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T05:01:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-05-09T05:01:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.301 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-05-09T05:01:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-05-09T05:01:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T05:01:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-05-09T05:01:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.302 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-05-09T05:01:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-05-09T05:01:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T05:01:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-05-09T05:01:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.303 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-05-09T05:01:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-05-09T05:01:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T05:01:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-05-09T05:01:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-05-09T05:01:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-05-09T05:01:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T05:01:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-05-09T05:01:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.305 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-05-09T05:01:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-05-09T05:01:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T05:01:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-05-09T05:01:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.306 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-05-09T05:01:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-05-09T05:01:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T05:01:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T05:01:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-05-09T05:01:47.307 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-05-09T05:01:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T05:01:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-05-09T05:01:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-05-09T05:01:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-05-09T05:01:47.308 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_samba": { 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_samba", 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T05:01:47.310 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Samba/SMB container image", 2024-05-09T05:01:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-05-09T05:01:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-05-09T05:01:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T05:01:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.311 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.312 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.313 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-05-09T05:01:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T05:01:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-05-09T05:01:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T05:01:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.314 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.315 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-05-09T05:01:47.316 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T05:01:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-05-09T05:01:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-05-09T05:01:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.317 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T05:01:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-05-09T05:01:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "grafana_dashboards_path": { 2024-05-09T05:01:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "grafana_dashboards_path", 2024-05-09T05:01:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T05:01:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T05:01:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-05-09T05:01:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-05-09T05:01:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.319 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T05:01:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-05-09T05:01:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-05-09T05:01:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-05-09T05:01:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.320 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T05:01:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-05-09T05:01:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-05-09T05:01:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.321 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T05:01:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.322 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-05-09T05:01:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-05-09T05:01:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T05:01:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T05:01:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.325 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.326 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.327 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T05:01:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T05:01:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.328 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T05:01:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T05:01:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T05:01:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-05-09T05:01:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.329 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T05:01:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-05-09T05:01:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-05-09T05:01:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.330 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-05-09T05:01:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-05-09T05:01:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-05-09T05:01:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T05:01:47.331 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-05-09T05:01:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-05-09T05:01:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-05-09T05:01:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.332 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T05:01:47.333 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-05-09T05:01:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-05-09T05:01:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-05-09T05:01:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T05:01:47.334 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-05-09T05:01:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-05-09T05:01:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T05:01:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T05:01:47.335 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-05-09T05:01:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-05-09T05:01:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T05:01:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.336 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-05-09T05:01:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-05-09T05:01:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T05:01:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.337 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-05-09T05:01:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-05-09T05:01:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T05:01:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.338 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-05-09T05:01:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-05-09T05:01:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T05:01:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-05-09T05:01:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-05-09T05:01:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T05:01:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-05-09T05:01:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-05-09T05:01:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-05-09T05:01:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-05-09T05:01:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.341 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.342 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-05-09T05:01:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-05-09T05:01:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T05:01:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-05-09T05:01:47.343 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-05-09T05:01:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T05:01:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-05-09T05:01:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-05-09T05:01:47.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T05:01:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-05-09T05:01:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-05-09T05:01:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.345 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T05:01:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-05-09T05:01:47.346 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-05-09T05:01:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T05:01:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-05-09T05:01:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-05-09T05:01:47.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T05:01:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.348 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-05-09T05:01:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.349 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.350 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.351 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.353 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-05-09T05:01:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-05-09T05:01:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-05-09T05:01:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T05:01:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.354 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-05-09T05:01:47.355 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-05-09T05:01:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-05-09T05:01:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T05:01:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.356 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.357 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T05:01:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T05:01:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.358 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T05:01:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T05:01:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.359 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-05-09T05:01:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T05:01:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.360 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T05:01:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T05:01:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.361 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T05:01:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T05:01:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.362 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T05:01:47.364 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T05:01:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T05:01:47.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T05:01:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T05:01:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.366 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.367 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.368 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T05:01:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T05:01:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.369 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T05:01:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T05:01:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T05:01:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.370 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T05:01:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T05:01:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.371 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-05-09T05:01:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-05-09T05:01:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.372 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T05:01:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T05:01:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T05:01:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.373 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T05:01:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T05:01:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.374 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T05:01:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T05:01:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.375 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T05:01:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T05:01:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.376 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T05:01:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T05:01:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T05:01:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T05:01:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "[]", 2024-05-09T05:01:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.378 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T05:01:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T05:01:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "{}", 2024-05-09T05:01:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T05:01:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T05:01:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.380 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T05:01:47.381 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T05:01:47.382 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T05:01:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T05:01:47.383 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T05:01:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.384 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T05:01:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T05:01:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.385 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T05:01:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T05:01:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.386 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T05:01:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T05:01:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T05:01:47.387 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T05:01:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T05:01:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T05:01:47.388 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.389 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T05:01:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T05:01:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T05:01:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.390 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T05:01:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T05:01:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.391 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-05-09T05:01:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T05:01:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T05:01:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-05-09T05:01:47.392 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.393 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T05:01:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T05:01:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T05:01:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.394 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T05:01:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T05:01:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.395 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T05:01:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T05:01:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.396 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T05:01:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T05:01:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T05:01:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T05:01:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.398 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T05:01:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T05:01:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.399 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T05:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T05:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T05:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-05-09T05:01:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-05-09T05:01:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.401 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-05-09T05:01:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-05-09T05:01:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-05-09T05:01:47.402 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-05-09T05:01:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-05-09T05:01:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-05-09T05:01:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-05-09T05:01:47.403 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-05-09T05:01:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-09T05:01:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-09T05:01:47.404 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.405 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.406 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.407 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.408 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.409 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-05-09T05:01:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-05-09T05:01:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.410 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-05-09T05:01:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-05-09T05:01:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T05:01:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.411 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T05:01:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T05:01:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.412 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-09T05:01:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T05:01:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T05:01:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.413 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-05-09T05:01:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.414 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-09T05:01:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-09T05:01:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.415 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.416 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-05-09T05:01:47.417 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-09T05:01:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-09T05:01:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-09T05:01:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-09T05:01:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-09T05:01:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.418 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.419 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-05-09T05:01:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-05-09T05:01:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-05-09T05:01:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-05-09T05:01:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.421 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.422 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.423 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.424 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.425 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.426 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-05-09T05:01:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-05-09T05:01:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-05-09T05:01:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T05:01:47.427 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-05-09T05:01:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-05-09T05:01:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-05-09T05:01:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T05:01:47.428 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-05-09T05:01:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-05-09T05:01:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-05-09T05:01:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-05-09T05:01:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.429 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-05-09T05:01:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-05-09T05:01:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-09T05:01:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T05:01:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-05-09T05:01:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-05-09T05:01:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T05:01:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.431 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T05:01:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T05:01:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T05:01:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T05:01:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-05-09T05:01:47.433 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-05-09T05:01:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-05-09T05:01:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T05:01:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-05-09T05:01:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.435 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.437 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.438 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.439 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-05-09T05:01:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-05-09T05:01:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-09T05:01:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.440 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-05-09T05:01:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-05-09T05:01:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-05-09T05:01:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.441 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T05:01:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T05:01:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T05:01:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.443 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-05-09T05:01:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-05-09T05:01:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-05-09T05:01:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-05-09T05:01:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-05-09T05:01:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-05-09T05:01:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.444 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T05:01:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-05-09T05:01:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-05-09T05:01:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-05-09T05:01:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.445 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T05:01:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-05-09T05:01:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-05-09T05:01:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-05-09T05:01:47.446 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.447 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.448 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.449 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.450 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.451 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-05-09T05:01:47.452 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-05-09T05:01:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-05-09T05:01:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-05-09T05:01:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.453 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-05-09T05:01:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-05-09T05:01:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.454 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-09T05:01:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-09T05:01:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.455 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T05:01:47.456 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T05:01:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-05-09T05:01:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T05:01:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-05-09T05:01:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-05-09T05:01:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.457 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-05-09T05:01:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-05-09T05:01:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-05-09T05:01:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-05-09T05:01:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.458 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T05:01:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-05-09T05:01:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.459 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.460 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.461 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.462 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.463 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.464 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-05-09T05:01:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.465 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.466 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.467 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.468 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.469 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.470 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.471 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-05-09T05:01:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-05-09T05:01:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-05-09T05:01:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T05:01:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.472 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-05-09T05:01:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-05-09T05:01:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-05-09T05:01:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T05:01:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.473 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.474 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.475 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.476 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.477 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.478 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-05-09T05:01:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.479 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-05-09T05:01:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-05-09T05:01:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-05-09T05:01:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-05-09T05:01:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T05:01:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.480 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.481 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.482 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.483 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.484 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-05-09T05:01:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-05-09T05:01:47.485 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-05-09T05:01:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T05:01:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-05-09T05:01:47.486 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.487 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-05-09T05:01:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T05:01:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-05-09T05:01:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-05-09T05:01:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.488 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.489 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-05-09T05:01:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-05-09T05:01:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-05-09T05:01:47.490 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.491 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.492 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.493 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.494 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.495 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.496 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.497 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.498 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.500 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.501 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.502 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-05-09T05:01:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.503 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.504 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.505 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.506 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.507 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.508 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-09T05:01:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.509 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-05-09T05:01:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-05-09T05:01:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T05:01:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.511 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.512 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.513 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.514 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-05-09T05:01:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-09T05:01:47.515 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-05-09T05:01:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-05-09T05:01:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-05-09T05:01:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-05-09T05:01:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.517 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-05-09T05:01:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.518 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.519 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.521 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.522 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.523 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-05-09T05:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.524 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.525 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.527 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.528 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.529 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-05-09T05:01:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.530 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.531 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.532 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.533 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.534 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T05:01:47.535 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.536 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-05-09T05:01:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-05-09T05:01:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-09T05:01:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-05-09T05:01:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.537 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-05-09T05:01:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T05:01:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-05-09T05:01:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-05-09T05:01:47.538 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-05-09T05:01:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T05:01:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.539 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-09T05:01:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-09T05:01:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.540 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.541 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.542 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.543 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.544 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-05-09T05:01:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-05-09T05:01:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-05-09T05:01:47.545 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-05-09T05:01:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T05:01:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T05:01:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T05:01:47.546 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-05-09T05:01:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.547 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-05-09T05:01:47.548 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-05-09T05:01:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-05-09T05:01:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.549 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T05:01:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T05:01:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.550 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.551 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.552 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.553 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.554 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.555 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-05-09T05:01:47.556 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-05-09T05:01:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.557 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T05:01:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T05:01:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-05-09T05:01:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.558 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-05-09T05:01:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-05-09T05:01:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-05-09T05:01:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.559 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T05:01:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T05:01:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-09T05:01:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.560 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T05:01:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T05:01:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T05:01:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-05-09T05:01:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.561 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T05:01:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.562 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-05-09T05:01:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-05-09T05:01:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-05-09T05:01:47.563 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-09T05:01:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-09T05:01:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.564 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-05-09T05:01:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-05-09T05:01:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-05-09T05:01:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-09T05:01:47.565 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-09T05:01:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-09T05:01:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-09T05:01:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-09T05:01:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.566 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-05-09T05:01:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.567 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.568 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.569 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.570 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.571 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.572 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-05-09T05:01:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-05-09T05:01:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.573 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-05-09T05:01:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T05:01:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.574 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.575 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-05-09T05:01:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-05-09T05:01:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.576 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-05-09T05:01:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.577 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-05-09T05:01:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-05-09T05:01:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.578 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-09T05:01:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-09T05:01:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.579 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.580 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.581 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.582 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.583 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.584 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.585 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T05:01:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T05:01:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.586 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T05:01:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T05:01:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.587 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.588 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-05-09T05:01:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.589 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.590 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.591 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.592 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.593 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.594 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.595 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.596 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-05-09T05:01:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.597 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.598 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.599 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.600 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.602 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.603 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-05-09T05:01:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-05-09T05:01:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-05-09T05:01:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T05:01:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.604 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-05-09T05:01:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.605 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.606 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.607 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.608 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.610 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-05-09T05:01:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-05-09T05:01:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-05-09T05:01:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-05-09T05:01:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-05-09T05:01:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-05-09T05:01:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-05-09T05:01:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.613 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-05-09T05:01:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-05-09T05:01:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.614 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-05-09T05:01:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-05-09T05:01:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-05-09T05:01:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-05-09T05:01:47.616 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.617 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-05-09T05:01:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-05-09T05:01:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.618 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-05-09T05:01:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-05-09T05:01:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.619 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-05-09T05:01:47.620 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-05-09T05:01:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T05:01:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-05-09T05:01:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.622 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-05-09T05:01:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-05-09T05:01:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.623 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-05-09T05:01:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-05-09T05:01:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.624 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-05-09T05:01:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-05-09T05:01:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.625 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smb", 2024-05-09T05:01:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.627 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.628 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.629 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.630 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.631 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.633 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "update_orchestration": { 2024-05-09T05:01:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "update_orchestration", 2024-05-09T05:01:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.634 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T05:01:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-05-09T05:01:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.635 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-05-09T05:01:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-05-09T05:01:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T05:01:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.636 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-05-09T05:01:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-05-09T05:01:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-05-09T05:01:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.637 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.638 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.639 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.640 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.641 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.642 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.643 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.644 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-05-09T05:01:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.645 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.646 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.647 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.648 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.649 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.650 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-05-09T05:01:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.652 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.653 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.654 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.655 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.656 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.657 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.658 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-05-09T05:01:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-05-09T05:01:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-05-09T05:01:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.659 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T05:01:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T05:01:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T05:01:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.660 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-05-09T05:01:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.661 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.662 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.663 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.664 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.665 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.666 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.667 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.668 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-05-09T05:01:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-05-09T05:01:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-05-09T05:01:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.669 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T05:01:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-05-09T05:01:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-05-09T05:01:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.670 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T05:01:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-05-09T05:01:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-05-09T05:01:47.671 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T05:01:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-05-09T05:01:47.672 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-05-09T05:01:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T05:01:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.673 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-05-09T05:01:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-05-09T05:01:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T05:01:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-05-09T05:01:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-05-09T05:01:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-05-09T05:01:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-05-09T05:01:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.676 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-05-09T05:01:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-05-09T05:01:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T05:01:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.677 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-09T05:01:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-09T05:01:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.678 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T05:01:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T05:01:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-05-09T05:01:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-05-09T05:01:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.679 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-05-09T05:01:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-05-09T05:01:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-05-09T05:01:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.680 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-05-09T05:01:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-05-09T05:01:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.681 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-05-09T05:01:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-05-09T05:01:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.682 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.683 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.684 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.685 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.686 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.687 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.688 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-05-09T05:01:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-05-09T05:01:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.689 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-05-09T05:01:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-05-09T05:01:47.690 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.691 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.692 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-05-09T05:01:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-05-09T05:01:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T05:01:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.693 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-05-09T05:01:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.694 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.695 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.696 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.697 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.698 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.699 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.700 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.701 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-05-09T05:01:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.702 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.703 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.704 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.705 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.706 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-05-09T05:01:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-05-09T05:01:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-05-09T05:01:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T05:01:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.708 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "periodic_async_work": { 2024-05-09T05:01:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "periodic_async_work", 2024-05-09T05:01:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.709 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Periodically check for async work", 2024-05-09T05:01:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-05-09T05:01:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-05-09T05:01:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.710 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T05:01:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-05-09T05:01:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T05:01:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.711 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T05:01:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.712 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.713 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-05-09T05:01:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-05-09T05:01:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-05-09T05:01:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T05:01:47.714 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-05-09T05:01:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-05-09T05:01:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-05-09T05:01:47.715 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T05:01:47.716 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T05:01:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T05:01:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.717 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.718 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.719 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.720 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.721 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.722 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.723 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-05-09T05:01:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-05-09T05:01:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.724 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-05-09T05:01:47.725 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-05-09T05:01:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-05-09T05:01:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.726 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-05-09T05:01:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-05-09T05:01:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T05:01:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.727 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T05:01:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "gid": 24920, 2024-05-09T05:01:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "y", 2024-05-09T05:01:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "mgr_features": 4540701547738038271, 2024-05-09T05:01:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-05-09T05:01:47.728 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-05-09T05:01:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T05:01:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T05:01:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T05:01:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.729 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T05:01:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.730 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.731 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.732 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.733 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.734 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.735 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-05-09T05:01:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-05-09T05:01:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-05-09T05:01:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.736 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-05-09T05:01:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-05-09T05:01:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-05-09T05:01:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-05-09T05:01:47.737 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-05-09T05:01:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-05-09T05:01:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.738 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-05-09T05:01:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-05-09T05:01:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-05-09T05:01:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.739 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-05-09T05:01:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-05-09T05:01:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-05-09T05:01:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.740 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-05-09T05:01:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-05-09T05:01:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.741 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-05-09T05:01:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-05-09T05:01:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-05-09T05:01:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.742 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-05-09T05:01:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-05-09T05:01:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T05:01:47.743 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-05-09T05:01:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-05-09T05:01:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.744 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-05-09T05:01:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.745 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-05-09T05:01:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.746 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-05-09T05:01:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-05-09T05:01:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T05:01:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.747 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-05-09T05:01:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-05-09T05:01:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-05-09T05:01:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T05:01:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T05:01:47.748 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-05-09T05:01:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-05-09T05:01:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T05:01:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-09T05:01:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-09T05:01:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T05:01:47.749 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T05:01:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-05-09T05:01:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-05-09T05:01:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T05:01:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-05-09T05:01:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T05:01:47.750 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-05-09T05:01:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T05:01:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-05-09T05:01:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-05-09T05:01:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.751 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T05:01:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T05:01:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T05:01:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-05-09T05:01:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-05-09T05:01:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:47.752 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-05-09T05:01:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-05-09T05:01:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-05-09T05:01:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-05-09T05:01:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T05:01:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-05-09T05:01:47.753 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-05-09T05:01:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-05-09T05:01:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T05:01:47.754 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T05:01:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-05-09T05:01:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-05-09T05:01:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T05:01:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-09T05:01:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-09T05:01:47.755 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T05:01:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T05:01:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.756 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.757 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.758 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.759 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.760 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.761 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-05-09T05:01:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-05-09T05:01:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.762 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T05:01:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-09T05:01:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-09T05:01:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.763 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-05-09T05:01:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-05-09T05:01:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-05-09T05:01:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "read", 2024-05-09T05:01:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap", 2024-05-09T05:01:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap-read" 2024-05-09T05:01:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-05-09T05:01:47.764 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-05-09T05:01:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-05-09T05:01:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.765 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T05:01:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T05:01:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T05:01:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T05:01:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.766 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T05:01:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.767 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-05-09T05:01:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-05-09T05:01:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T05:01:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T05:01:47.768 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T05:01:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T05:01:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-05-09T05:01:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-05-09T05:01:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T05:01:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.769 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T05:01:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.770 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-05-09T05:01:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-05-09T05:01:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-05-09T05:01:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-09T05:01:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.771 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T05:01:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-05-09T05:01:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-05-09T05:01:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-05-09T05:01:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.772 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T05:01:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-05-09T05:01:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-05-09T05:01:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-05-09T05:01:47.773 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T05:01:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-05-09T05:01:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-05-09T05:01:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.774 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T05:01:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T05:01:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-05-09T05:01:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-05-09T05:01:47.775 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T05:01:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T05:01:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-05-09T05:01:47.776 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T05:01:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-05-09T05:01:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T05:01:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_log_destination": { 2024-05-09T05:01:47.777 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm_log_destination", 2024-05-09T05:01:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "file", 2024-05-09T05:01:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "file,syslog", 2024-05-09T05:01:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "syslog" 2024-05-09T05:01:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.778 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T05:01:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-05-09T05:01:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-05-09T05:01:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.779 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T05:01:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-05-09T05:01:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-05-09T05:01:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.780 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T05:01:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-05-09T05:01:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-05-09T05:01:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.781 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T05:01:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.782 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-05-09T05:01:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-05-09T05:01:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T05:01:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T05:01:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.783 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-05-09T05:01:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-05-09T05:01:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T05:01:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-05-09T05:01:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.784 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-05-09T05:01:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-05-09T05:01:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T05:01:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-05-09T05:01:47.785 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-05-09T05:01:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-05-09T05:01:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T05:01:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.786 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T05:01:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-05-09T05:01:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-05-09T05:01:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T05:01:47.787 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-05-09T05:01:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-05-09T05:01:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-05-09T05:01:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.788 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T05:01:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-05-09T05:01:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-05-09T05:01:47.789 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-05-09T05:01:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T05:01:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-05-09T05:01:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.790 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-05-09T05:01:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-05-09T05:01:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T05:01:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-05-09T05:01:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.791 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-05-09T05:01:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-05-09T05:01:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T05:01:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-05-09T05:01:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.792 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-05-09T05:01:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-05-09T05:01:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T05:01:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-05-09T05:01:47.793 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-05-09T05:01:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-05-09T05:01:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T05:01:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.794 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T05:01:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-05-09T05:01:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-05-09T05:01:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.795 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T05:01:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-05-09T05:01:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-05-09T05:01:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-05-09T05:01:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.796 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T05:01:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T05:01:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-05-09T05:01:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-05-09T05:01:47.797 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T05:01:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-05-09T05:01:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.798 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_samba": { 2024-05-09T05:01:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_samba", 2024-05-09T05:01:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T05:01:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Samba/SMB container image", 2024-05-09T05:01:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.799 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-05-09T05:01:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-05-09T05:01:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T05:01:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-05-09T05:01:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.800 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-05-09T05:01:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-05-09T05:01:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T05:01:47.801 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-05-09T05:01:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-05-09T05:01:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T05:01:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.802 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T05:01:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-05-09T05:01:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T05:01:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-05-09T05:01:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.803 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T05:01:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-05-09T05:01:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-05-09T05:01:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-05-09T05:01:47.804 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T05:01:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-05-09T05:01:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-05-09T05:01:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-05-09T05:01:47.805 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-05-09T05:01:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-05-09T05:01:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-05-09T05:01:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.806 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T05:01:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-05-09T05:01:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-05-09T05:01:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.807 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T05:01:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-05-09T05:01:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "grafana_dashboards_path": { 2024-05-09T05:01:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "grafana_dashboards_path", 2024-05-09T05:01:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.808 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T05:01:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T05:01:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-05-09T05:01:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-05-09T05:01:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.809 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T05:01:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-05-09T05:01:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-05-09T05:01:47.810 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-05-09T05:01:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T05:01:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.811 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-05-09T05:01:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-05-09T05:01:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T05:01:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.812 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.813 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-05-09T05:01:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-05-09T05:01:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.814 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T05:01:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.815 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T05:01:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.816 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.817 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.818 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T05:01:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T05:01:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T05:01:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.819 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T05:01:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T05:01:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-05-09T05:01:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T05:01:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.820 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-05-09T05:01:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-05-09T05:01:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.821 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-05-09T05:01:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-05-09T05:01:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-05-09T05:01:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.822 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T05:01:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-05-09T05:01:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-05-09T05:01:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.823 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-05-09T05:01:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-09T05:01:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-09T05:01:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.824 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-05-09T05:01:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-05-09T05:01:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-05-09T05:01:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T05:01:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-05-09T05:01:47.825 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-05-09T05:01:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-05-09T05:01:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T05:01:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-05-09T05:01:47.826 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-05-09T05:01:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T05:01:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T05:01:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-05-09T05:01:47.827 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-05-09T05:01:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T05:01:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-05-09T05:01:47.828 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-05-09T05:01:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T05:01:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-05-09T05:01:47.829 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-05-09T05:01:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T05:01:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.830 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-05-09T05:01:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-05-09T05:01:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.831 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T05:01:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-05-09T05:01:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-05-09T05:01:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.832 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T05:01:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-05-09T05:01:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-05-09T05:01:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.833 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-05-09T05:01:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-05-09T05:01:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.834 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-05-09T05:01:47.835 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-05-09T05:01:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T05:01:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.836 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-05-09T05:01:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-05-09T05:01:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T05:01:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.837 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-05-09T05:01:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-05-09T05:01:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T05:01:47.838 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-05-09T05:01:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-05-09T05:01:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.839 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T05:01:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-05-09T05:01:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-05-09T05:01:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.840 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T05:01:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-05-09T05:01:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-05-09T05:01:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.841 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T05:01:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.842 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-05-09T05:01:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.843 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.844 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.845 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.846 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.847 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-05-09T05:01:47.848 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-05-09T05:01:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-05-09T05:01:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T05:01:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.849 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.850 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-05-09T05:01:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-05-09T05:01:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-05-09T05:01:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.851 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T05:01:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-05-09T05:01:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T05:01:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T05:01:47.852 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T05:01:47.853 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T05:01:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.854 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T05:01:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T05:01:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.855 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-05-09T05:01:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T05:01:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.856 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T05:01:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T05:01:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.857 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T05:01:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T05:01:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.858 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T05:01:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T05:01:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.859 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T05:01:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T05:01:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.860 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T05:01:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T05:01:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.861 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T05:01:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T05:01:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.862 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T05:01:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T05:01:47.863 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T05:01:47.864 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T05:01:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.865 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T05:01:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T05:01:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.866 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T05:01:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T05:01:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.867 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T05:01:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T05:01:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T05:01:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.868 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T05:01:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T05:01:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.869 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-05-09T05:01:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-05-09T05:01:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.870 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T05:01:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T05:01:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.871 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T05:01:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T05:01:47.872 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T05:01:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.873 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T05:01:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T05:01:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.874 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T05:01:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T05:01:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.875 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T05:01:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T05:01:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.876 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T05:01:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T05:01:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "[]", 2024-05-09T05:01:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.877 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T05:01:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T05:01:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.878 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "{}", 2024-05-09T05:01:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T05:01:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T05:01:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.879 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T05:01:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T05:01:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.880 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T05:01:47.881 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T05:01:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.882 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T05:01:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T05:01:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.883 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T05:01:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T05:01:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.884 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T05:01:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T05:01:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.885 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T05:01:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T05:01:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.886 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T05:01:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T05:01:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.887 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T05:01:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T05:01:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.888 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T05:01:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T05:01:47.889 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.890 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T05:01:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T05:01:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T05:01:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.891 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T05:01:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T05:01:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.892 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T05:01:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T05:01:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-05-09T05:01:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.893 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T05:01:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T05:01:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.894 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-05-09T05:01:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T05:01:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T05:01:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.895 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T05:01:47.896 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T05:01:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T05:01:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.897 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T05:01:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T05:01:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.898 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T05:01:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T05:01:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.899 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T05:01:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T05:01:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.900 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T05:01:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T05:01:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.901 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T05:01:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T05:01:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.902 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T05:01:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T05:01:47.903 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T05:01:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-05-09T05:01:47.904 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-05-09T05:01:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-05-09T05:01:47.905 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-05-09T05:01:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.906 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-05-09T05:01:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-05-09T05:01:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-05-09T05:01:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.907 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-05-09T05:01:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-05-09T05:01:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-05-09T05:01:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.908 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-09T05:01:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-09T05:01:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.909 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.910 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.911 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.912 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.913 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.914 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-05-09T05:01:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-05-09T05:01:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.915 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-05-09T05:01:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-05-09T05:01:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T05:01:47.916 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T05:01:47.917 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T05:01:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-09T05:01:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.918 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T05:01:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T05:01:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-05-09T05:01:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.919 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.920 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-09T05:01:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-09T05:01:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.921 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-05-09T05:01:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-05-09T05:01:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.922 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-05-09T05:01:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-09T05:01:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-09T05:01:47.923 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-05-09T05:01:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-05-09T05:01:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.924 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-09T05:01:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-09T05:01:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-09T05:01:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-09T05:01:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-09T05:01:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.925 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-05-09T05:01:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-05-09T05:01:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.926 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-05-09T05:01:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.927 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-05-09T05:01:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-05-09T05:01:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-05-09T05:01:47.928 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.929 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.930 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.931 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.932 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.933 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.934 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-05-09T05:01:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-05-09T05:01:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-05-09T05:01:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.935 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T05:01:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-05-09T05:01:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-05-09T05:01:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-05-09T05:01:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.936 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T05:01:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-05-09T05:01:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-05-09T05:01:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.937 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-05-09T05:01:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-05-09T05:01:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-05-09T05:01:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-05-09T05:01:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.938 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-09T05:01:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T05:01:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-05-09T05:01:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-05-09T05:01:47.939 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T05:01:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.940 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T05:01:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T05:01:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T05:01:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T05:01:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.941 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.942 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-05-09T05:01:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-05-09T05:01:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.943 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-05-09T05:01:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T05:01:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.944 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-05-09T05:01:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.945 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.946 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.947 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.948 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.948 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.948 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.948 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.948 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.948 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.948 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.948 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.948 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.948 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.948 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.949 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.949 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.949 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.949 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.949 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.949 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.949 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.949 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.949 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.949 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.949 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.949 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.949 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.950 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.950 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.950 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.950 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.950 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.950 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.950 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.950 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.950 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.950 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.950 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-05-09T05:01:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-05-09T05:01:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-09T05:01:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.951 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-05-09T05:01:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-05-09T05:01:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-05-09T05:01:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.952 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T05:01:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T05:01:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T05:01:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.953 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.954 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.955 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-05-09T05:01:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-05-09T05:01:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-05-09T05:01:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-05-09T05:01:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-05-09T05:01:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-05-09T05:01:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.956 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T05:01:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-05-09T05:01:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-05-09T05:01:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.957 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-05-09T05:01:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T05:01:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-05-09T05:01:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-05-09T05:01:47.958 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-05-09T05:01:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T05:01:47.959 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T05:01:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-05-09T05:01:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-05-09T05:01:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T05:01:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.960 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.961 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.962 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.963 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.964 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.965 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-05-09T05:01:47.966 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-05-09T05:01:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-05-09T05:01:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.967 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-05-09T05:01:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-05-09T05:01:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-05-09T05:01:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-05-09T05:01:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.968 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.969 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-09T05:01:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-09T05:01:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-05-09T05:01:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T05:01:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.970 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-05-09T05:01:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-05-09T05:01:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T05:01:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T05:01:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-05-09T05:01:47.971 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T05:01:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-05-09T05:01:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-05-09T05:01:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.972 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-05-09T05:01:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-05-09T05:01:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-05-09T05:01:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.973 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-05-09T05:01:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T05:01:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.974 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-05-09T05:01:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.975 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.976 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.977 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.978 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.979 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.980 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.981 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-05-09T05:01:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.982 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.983 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.984 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.985 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.986 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.987 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.988 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-05-09T05:01:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.989 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-05-09T05:01:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-05-09T05:01:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-05-09T05:01:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.990 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T05:01:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-05-09T05:01:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-05-09T05:01:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-05-09T05:01:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.991 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T05:01:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.992 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.993 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.994 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:47.995 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:47.996 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.997 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:47.998 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-05-09T05:01:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-05-09T05:01:47.999 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-05-09T05:01:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-05-09T05:01:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-05-09T05:01:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T05:01:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.000 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.001 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.002 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.003 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.004 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.005 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.006 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-05-09T05:01:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-05-09T05:01:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-05-09T05:01:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T05:01:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.007 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-05-09T05:01:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-05-09T05:01:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-05-09T05:01:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-05-09T05:01:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.008 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-05-09T05:01:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-05-09T05:01:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-05-09T05:01:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T05:01:48.009 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-05-09T05:01:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-05-09T05:01:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.010 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-05-09T05:01:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.011 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-05-09T05:01:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-05-09T05:01:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.012 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-05-09T05:01:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T05:01:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T05:01:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.013 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-05-09T05:01:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.014 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.015 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.016 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.017 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.018 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.019 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.020 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-05-09T05:01:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.021 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.022 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.023 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.024 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.025 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.026 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.027 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-05-09T05:01:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.028 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.029 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.030 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.031 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.032 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.033 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.034 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.035 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-09T05:01:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-05-09T05:01:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-05-09T05:01:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.036 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T05:01:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.037 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.038 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.039 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.040 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.041 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.042 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-05-09T05:01:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-09T05:01:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.043 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-05-09T05:01:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-05-09T05:01:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-05-09T05:01:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-05-09T05:01:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.044 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.045 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-05-09T05:01:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.046 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.047 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.048 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.049 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.050 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.051 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.052 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-05-09T05:01:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.053 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.054 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.055 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.056 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.057 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.058 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.059 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-05-09T05:01:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.060 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.061 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.062 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.063 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.064 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.065 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T05:01:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T05:01:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T05:01:48.066 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.067 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-05-09T05:01:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-05-09T05:01:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:48.068 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-09T05:01:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-05-09T05:01:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-05-09T05:01:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T05:01:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.069 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-05-09T05:01:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-05-09T05:01:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-05-09T05:01:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.070 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T05:01:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-09T05:01:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-09T05:01:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.071 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.072 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.073 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.074 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.075 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.076 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.077 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-05-09T05:01:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-05-09T05:01:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-05-09T05:01:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-05-09T05:01:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.078 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T05:01:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T05:01:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T05:01:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-05-09T05:01:48.079 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.080 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-05-09T05:01:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.081 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-05-09T05:01:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-05-09T05:01:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.082 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-05-09T05:01:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-05-09T05:01:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.083 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T05:01:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T05:01:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.084 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.085 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.086 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.087 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.088 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-05-09T05:01:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-05-09T05:01:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.089 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T05:01:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T05:01:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.090 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-05-09T05:01:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-05-09T05:01:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-05-09T05:01:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:48.091 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-05-09T05:01:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T05:01:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T05:01:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.092 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-09T05:01:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T05:01:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T05:01:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T05:01:48.093 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-05-09T05:01:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T05:01:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.094 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.095 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-05-09T05:01:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-05-09T05:01:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-05-09T05:01:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.096 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-09T05:01:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-09T05:01:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-05-09T05:01:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.097 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-05-09T05:01:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-05-09T05:01:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-09T05:01:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-09T05:01:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.098 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-09T05:01:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-09T05:01:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-09T05:01:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.099 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-05-09T05:01:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.100 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.101 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.102 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.103 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.104 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.105 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-05-09T05:01:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-05-09T05:01:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.106 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:48.109 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-05-09T05:01:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T05:01:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.110 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.111 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-05-09T05:01:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-05-09T05:01:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.112 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-05-09T05:01:48.113 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-09T05:01:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-09T05:01:48.114 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.115 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.116 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.117 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.118 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.119 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T05:01:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T05:01:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.120 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T05:01:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T05:01:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.121 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.122 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.123 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.124 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.124 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.124 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.124 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.124 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.124 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.124 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.124 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.124 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.124 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.124 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.124 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.124 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.124 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.124 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.125 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.125 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.125 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.125 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.125 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.125 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.125 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.125 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.125 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.125 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.125 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.125 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.125 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.125 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.125 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.126 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.126 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.126 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.126 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.126 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.126 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.126 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.126 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.126 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.126 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.126 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.126 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.126 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.126 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.126 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.127 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.127 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.127 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.127 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.127 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.127 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.127 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.127 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.127 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.127 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.127 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.127 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.127 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.127 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.128 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.129 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-05-09T05:01:48.130 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.130 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.130 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.130 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.130 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.130 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.130 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.130 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.130 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.130 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.130 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.130 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.130 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.130 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.130 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.131 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.131 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.131 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.131 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.131 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.131 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.131 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.131 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.131 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.131 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.131 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.131 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.131 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.131 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.131 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.132 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.132 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.132 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.132 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.132 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.132 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.132 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.132 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.132 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.132 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.132 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.132 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.132 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.132 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.133 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.133 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.133 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.133 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.133 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.133 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.133 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.133 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.133 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.133 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.133 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.133 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.133 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.133 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.134 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.134 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.134 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.134 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.134 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.134 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.134 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.134 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.134 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.134 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.134 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.134 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.134 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.134 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.135 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.135 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.135 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.135 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.135 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.135 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.135 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.135 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.135 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.135 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.135 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.135 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.135 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.135 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-05-09T05:01:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-05-09T05:01:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-05-09T05:01:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T05:01:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.136 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-05-09T05:01:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.137 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.138 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.138 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.138 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.138 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.138 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.138 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.138 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.138 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.138 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.138 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.138 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.138 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.138 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.138 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.139 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.139 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.139 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.139 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.139 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.139 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.139 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.139 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.139 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.139 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.139 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.139 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.139 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.139 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.140 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.141 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.141 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.141 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.141 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.141 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.141 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.141 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.141 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.141 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.141 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.141 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.141 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.141 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.141 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.142 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.142 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.142 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.142 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.142 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.142 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-05-09T05:01:48.142 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-05-09T05:01:48.142 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.142 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.142 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.142 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.142 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.142 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.142 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.143 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.143 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.143 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.143 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.143 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.143 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-05-09T05:01:48.143 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-05-09T05:01:48.143 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.143 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.143 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.143 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-05-09T05:01:48.143 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.143 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.143 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-05-09T05:01:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-05-09T05:01:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.144 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.145 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.145 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.145 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.145 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.145 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-05-09T05:01:48.145 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-05-09T05:01:48.145 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.145 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.145 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.145 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.145 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.145 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.145 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.145 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.145 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.146 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.146 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.146 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.146 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-05-09T05:01:48.146 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-05-09T05:01:48.146 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:48.146 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.146 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.146 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.146 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.146 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.146 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.146 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.147 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.147 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.147 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.147 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.147 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-05-09T05:01:48.147 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-05-09T05:01:48.147 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.147 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.147 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.147 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.147 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.147 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.147 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.147 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.147 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.148 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.148 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.148 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.148 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-05-09T05:01:48.148 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-05-09T05:01:48.148 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.148 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.148 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.148 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.148 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.148 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.148 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.148 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.148 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.149 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.149 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.149 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.149 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-05-09T05:01:48.149 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-05-09T05:01:48.149 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.149 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.149 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.149 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.149 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.149 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.149 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.149 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.149 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.149 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.150 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.150 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.150 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-05-09T05:01:48.150 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-05-09T05:01:48.150 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.150 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.150 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.150 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.150 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T05:01:48.150 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-05-09T05:01:48.150 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.150 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.150 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.150 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.151 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.151 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.151 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.151 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.151 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.151 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.151 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.151 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.151 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.151 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.151 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.151 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.151 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.151 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.151 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.152 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-05-09T05:01:48.152 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-05-09T05:01:48.152 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.152 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.152 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.152 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.152 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.152 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.152 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.152 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.152 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.152 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.152 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.152 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.153 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-05-09T05:01:48.153 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-05-09T05:01:48.153 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.153 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.153 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.153 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.153 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.153 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.153 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.153 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.153 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.153 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.153 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.153 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.154 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-05-09T05:01:48.154 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-05-09T05:01:48.154 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.154 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.154 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.154 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.154 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.154 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.154 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.154 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.154 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.154 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.154 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.154 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.154 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.155 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.155 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.155 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smb", 2024-05-09T05:01:48.155 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.155 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.155 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.155 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.155 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.155 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.155 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.155 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.155 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.155 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.155 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.156 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.156 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.156 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.156 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.156 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.156 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.156 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.156 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.156 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.156 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.156 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.156 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.156 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.156 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.156 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.157 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.157 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.157 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.157 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.157 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.157 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.157 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.157 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.157 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.157 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.157 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.157 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.157 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.157 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.157 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.158 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.158 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.158 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.158 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.158 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.158 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.158 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.158 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.158 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.158 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.158 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.158 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.158 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.158 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.159 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.159 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.159 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.159 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.159 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.159 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.159 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.159 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.159 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.159 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.159 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.159 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.159 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.159 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.160 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.160 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.160 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.160 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.160 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.160 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.160 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.160 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.160 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.160 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.160 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.160 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.160 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.160 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.161 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.161 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.161 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.161 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.161 INFO:teuthology.orchestra.run.smithi046.stdout: "update_orchestration": { 2024-05-09T05:01:48.161 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "update_orchestration", 2024-05-09T05:01:48.161 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.161 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.161 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.161 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.161 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.161 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.161 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.161 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T05:01:48.161 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.162 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.162 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.162 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.162 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.162 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.162 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.162 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-05-09T05:01:48.162 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.162 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.162 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.162 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-05-09T05:01:48.162 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-05-09T05:01:48.162 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.162 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.162 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.163 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.163 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.163 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.163 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.163 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T05:01:48.163 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.163 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.163 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.163 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.163 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-05-09T05:01:48.163 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-05-09T05:01:48.163 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.163 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.163 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.163 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.164 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.164 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.164 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.164 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-05-09T05:01:48.164 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.164 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.164 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.164 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.164 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.164 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.164 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.164 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.164 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.164 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.165 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.165 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.165 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.165 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.165 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.165 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.165 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.165 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.165 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.165 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.165 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.165 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.165 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.165 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.165 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.166 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.166 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.166 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.166 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.166 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.166 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.166 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.166 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.166 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.166 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.166 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.166 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.166 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.166 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.167 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.167 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.167 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.167 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.167 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.167 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.167 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.167 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.167 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.167 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.167 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.167 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.167 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.167 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.167 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.168 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.168 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.168 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.168 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.168 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.168 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.168 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.168 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.168 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.168 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.168 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.168 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.168 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.169 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.169 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.169 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.169 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.169 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.169 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.169 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.169 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.169 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.169 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.169 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.169 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.169 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.169 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.170 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.170 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.170 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.170 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.170 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.170 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.170 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.170 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.170 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.170 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.170 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-05-09T05:01:48.170 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.170 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.170 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.171 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.171 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.171 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.171 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.171 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.171 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.171 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.171 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.171 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.171 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.171 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.171 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.171 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.171 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.172 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.172 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.172 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.172 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.172 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.172 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.172 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.172 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.172 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.172 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.172 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.172 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.172 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.172 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.172 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.173 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.173 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.173 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.173 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.173 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.173 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.173 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.173 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.173 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.173 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.173 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.173 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.173 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.173 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.174 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.174 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.174 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.174 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.174 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.174 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.174 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.174 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.174 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.174 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.174 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.174 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.174 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.174 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.175 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.175 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.175 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.175 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.175 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.175 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.175 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.175 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.175 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.175 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.175 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.175 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.175 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.175 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.175 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.176 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.176 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.176 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.176 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.176 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.176 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.176 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.176 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.176 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.176 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.176 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.176 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.176 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.176 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.177 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-05-09T05:01:48.177 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.177 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.177 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.177 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.177 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.177 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.177 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.177 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.177 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.177 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.177 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.177 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.177 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.177 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.178 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.178 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.178 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.178 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.178 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.178 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.178 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.178 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.178 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.178 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.178 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.178 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.178 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.178 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.178 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.179 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.179 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.179 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.179 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.179 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.179 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.179 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.179 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.179 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.179 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.179 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.179 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.179 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.179 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.179 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.180 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.180 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.180 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.180 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.180 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.180 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.180 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.180 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.180 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.180 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.180 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.180 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.180 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.180 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.180 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.181 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.181 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.181 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.181 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.181 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.181 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.181 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.181 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.181 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.181 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.181 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.181 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.181 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.181 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.182 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.182 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.182 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.182 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.182 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.182 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.182 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.182 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.182 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.182 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.182 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.182 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.182 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.183 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.183 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.183 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.183 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.183 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-05-09T05:01:48.183 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.183 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.183 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.183 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-05-09T05:01:48.183 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-05-09T05:01:48.183 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.183 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.183 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.183 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T05:01:48.184 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.184 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.184 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.184 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.184 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.184 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.184 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.184 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.184 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T05:01:48.184 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T05:01:48.184 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.184 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.184 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.184 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-05-09T05:01:48.185 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.185 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.185 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.185 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.185 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.185 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.185 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.185 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.185 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.185 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.185 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.185 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.185 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.185 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.185 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.186 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.186 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.186 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.186 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.186 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.186 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.186 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.186 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.186 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.186 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.186 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.186 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.186 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.187 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.187 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.187 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.187 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.187 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.187 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.187 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.187 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.187 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.187 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.187 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.187 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.187 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.187 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.188 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.188 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.188 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.188 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.188 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.188 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.188 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.188 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.188 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.188 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.188 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.188 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.188 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.189 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.190 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.190 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.190 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.190 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.190 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.190 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.190 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.190 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.190 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.190 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.190 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.190 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.190 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.190 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.191 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.191 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.191 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.191 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.191 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.191 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.191 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.191 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.191 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.191 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.191 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-05-09T05:01:48.191 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.191 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.191 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.191 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-05-09T05:01:48.192 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-05-09T05:01:48.192 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.192 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.192 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.192 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.192 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.192 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.192 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.192 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T05:01:48.192 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.192 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.192 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.192 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.192 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-05-09T05:01:48.192 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-05-09T05:01:48.193 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.193 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.193 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.193 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.193 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.193 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.193 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.193 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T05:01:48.193 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.193 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.193 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.193 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.193 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-05-09T05:01:48.193 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-05-09T05:01:48.194 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.194 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.194 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.194 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.194 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.194 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.194 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.194 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T05:01:48.194 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.194 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.194 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.194 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.194 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-05-09T05:01:48.194 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-05-09T05:01:48.195 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.195 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.195 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.195 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.195 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.195 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.195 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.195 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T05:01:48.195 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.195 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.195 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.195 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.195 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-05-09T05:01:48.196 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-05-09T05:01:48.196 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.196 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.196 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.196 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.196 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.196 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.196 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.196 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T05:01:48.196 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.196 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.196 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.196 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.196 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-05-09T05:01:48.196 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-05-09T05:01:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-05-09T05:01:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-05-09T05:01:48.197 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.198 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.198 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.198 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.198 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.198 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.198 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.198 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.198 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.198 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.198 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.198 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.198 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-05-09T05:01:48.198 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-05-09T05:01:48.198 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.199 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.199 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.199 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T05:01:48.199 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.199 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.199 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.199 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.199 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.199 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.199 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.199 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.199 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-09T05:01:48.199 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-09T05:01:48.199 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.200 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.200 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.200 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.200 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.200 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.200 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.200 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.200 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.200 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.200 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.200 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.200 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T05:01:48.200 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.201 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.202 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.202 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-05-09T05:01:48.202 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.202 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.202 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.202 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.202 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.202 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.202 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.202 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.202 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-05-09T05:01:48.202 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-05-09T05:01:48.202 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.202 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.202 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.203 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.204 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.204 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.204 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.204 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.204 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.204 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.204 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.204 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.204 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.204 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.204 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.204 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.204 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.204 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.205 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.206 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.206 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.206 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.206 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.206 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.206 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.206 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.206 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.206 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.206 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.206 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.206 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.206 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.206 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.206 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.207 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-05-09T05:01:48.208 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-05-09T05:01:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-05-09T05:01:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-05-09T05:01:48.209 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.210 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.210 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.210 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.210 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.210 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.210 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.210 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.210 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.210 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.210 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.210 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.210 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.210 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.210 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.211 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.211 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.211 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.211 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.211 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.211 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.211 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.211 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.211 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.211 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.211 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.211 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-05-09T05:01:48.211 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.212 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.213 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.214 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.214 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.214 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.214 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.214 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.214 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.214 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.214 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.214 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.214 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.214 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.214 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.214 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.214 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.214 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.215 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.216 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.217 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.218 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.218 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.218 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.218 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.218 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.218 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.218 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.218 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.218 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.218 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.218 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-05-09T05:01:48.218 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.218 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.218 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.218 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.219 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.220 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.220 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.220 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.220 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.220 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.220 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.220 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.220 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.220 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.220 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.220 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.220 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.220 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.220 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.221 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.221 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.221 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.221 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.221 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.221 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.221 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.221 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.221 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.221 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.221 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.221 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.221 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.221 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.222 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.222 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.222 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.222 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.222 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.222 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.222 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.222 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.222 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.222 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.222 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.222 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.222 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.222 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-05-09T05:01:48.223 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.224 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.224 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.224 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T05:01:48.224 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.224 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.224 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.224 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.224 INFO:teuthology.orchestra.run.smithi046.stdout: "periodic_async_work": { 2024-05-09T05:01:48.224 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "periodic_async_work", 2024-05-09T05:01:48.224 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.224 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.224 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.224 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.224 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.224 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Periodically check for async work", 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.225 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T05:01:48.226 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.226 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.226 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.226 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.226 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-05-09T05:01:48.226 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T05:01:48.226 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.226 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.226 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.226 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.226 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.226 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.226 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.226 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T05:01:48.227 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.227 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.227 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.227 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.227 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.227 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.227 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.227 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.227 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.227 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.227 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.227 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.227 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.227 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.227 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.228 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.228 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.228 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.228 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.228 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.228 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.228 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-05-09T05:01:48.228 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.228 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.228 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.228 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-05-09T05:01:48.228 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-05-09T05:01:48.228 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T05:01:48.229 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.229 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.229 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-05-09T05:01:48.229 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.229 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.229 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.229 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.229 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.229 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.229 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.229 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.229 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-05-09T05:01:48.229 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-05-09T05:01:48.229 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.229 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.230 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.230 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.230 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.230 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.230 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.230 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.230 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.230 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.230 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.230 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.230 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T05:01:48.230 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T05:01:48.230 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.231 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.232 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.232 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.232 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.232 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.232 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.232 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.232 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.232 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.232 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.232 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.232 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.232 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.232 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.232 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.232 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.233 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.234 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.234 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.234 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.234 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.234 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.234 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.234 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.234 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.234 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.234 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.234 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.234 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.234 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.234 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.235 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.235 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.235 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.235 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.235 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.235 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.235 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.235 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.235 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.235 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.235 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.235 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.235 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.235 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.236 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.237 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.237 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.237 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.237 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.237 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-05-09T05:01:48.237 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-05-09T05:01:48.237 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.237 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.237 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.237 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.237 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.237 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.237 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.237 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.238 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.238 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.238 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.238 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.238 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-05-09T05:01:48.238 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-05-09T05:01:48.238 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.238 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.238 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.238 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-05-09T05:01:48.238 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.238 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.238 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.239 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.239 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.239 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.239 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.239 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.239 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-05-09T05:01:48.239 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-05-09T05:01:48.239 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.239 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.239 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.239 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T05:01:48.239 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.239 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.239 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.239 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.240 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.240 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.240 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.240 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.240 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.240 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.240 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T05:01:48.240 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.240 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.240 INFO:teuthology.orchestra.run.smithi046.stdout: "modules": [ 2024-05-09T05:01:48.240 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-05-09T05:01:48.240 INFO:teuthology.orchestra.run.smithi046.stdout: "dashboard", 2024-05-09T05:01:48.240 INFO:teuthology.orchestra.run.smithi046.stdout: "iostat", 2024-05-09T05:01:48.240 INFO:teuthology.orchestra.run.smithi046.stdout: "nfs", 2024-05-09T05:01:48.240 INFO:teuthology.orchestra.run.smithi046.stdout: "restful", 2024-05-09T05:01:48.241 INFO:teuthology.orchestra.run.smithi046.stdout: "snap_schedule" 2024-05-09T05:01:48.241 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.241 INFO:teuthology.orchestra.run.smithi046.stdout: "available_modules": [ 2024-05-09T05:01:48.241 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.241 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "alerts", 2024-05-09T05:01:48.241 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.241 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.241 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.241 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T05:01:48.241 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T05:01:48.241 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.241 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.241 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.241 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T05:01:48.242 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.242 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.242 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.242 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How frequently to reexamine health status", 2024-05-09T05:01:48.242 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.242 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.242 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.242 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.242 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.242 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.242 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.242 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.242 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.242 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.243 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.244 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.245 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.246 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_destination": { 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_destination", 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email address to send alerts to", 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.247 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_from_name": { 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_from_name", 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "Ceph", 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Email From: name", 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_host": { 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_host", 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.248 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP server", 2024-05-09T05:01:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_password": { 2024-05-09T05:01:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_password", 2024-05-09T05:01:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.249 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Password to authenticate with", 2024-05-09T05:01:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_port": { 2024-05-09T05:01:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_port", 2024-05-09T05:01:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.250 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "465", 2024-05-09T05:01:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP port", 2024-05-09T05:01:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_sender": { 2024-05-09T05:01:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_sender", 2024-05-09T05:01:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.251 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SMTP envelope sender", 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_ssl": { 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_ssl", 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.252 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-09T05:01:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "smtp_user": { 2024-05-09T05:01:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smtp_user", 2024-05-09T05:01:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.253 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "User to authenticate as", 2024-05-09T05:01:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.254 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "balancer", 2024-05-09T05:01:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "active": { 2024-05-09T05:01:48.255 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "active", 2024-05-09T05:01:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically balance PGs across cluster", 2024-05-09T05:01:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_time": { 2024-05-09T05:01:48.256 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_time", 2024-05-09T05:01:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0000", 2024-05-09T05:01:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "beginning time of day to automatically balance", 2024-05-09T05:01:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T05:01:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: "begin_weekday": { 2024-05-09T05:01:48.257 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "begin_weekday", 2024-05-09T05:01:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T05:01:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-09T05:01:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-09T05:01:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-09T05:01:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T05:01:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_max_iterations": { 2024-05-09T05:01:48.258 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_max_iterations", 2024-05-09T05:01:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T05:01:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "25", 2024-05-09T05:01:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T05:01:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "250", 2024-05-09T05:01:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-09T05:01:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_metrics": { 2024-05-09T05:01:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_metrics", 2024-05-09T05:01:48.259 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "pgs,objects,bytes", 2024-05-09T05:01:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-09T05:01:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-09T05:01:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "crush_compat_step": { 2024-05-09T05:01:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crush_compat_step", 2024-05-09T05:01:48.260 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.5", 2024-05-09T05:01:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0.001", 2024-05-09T05:01:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "0.999", 2024-05-09T05:01:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "aggressiveness of optimization", 2024-05-09T05:01:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-09T05:01:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "end_time": { 2024-05-09T05:01:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_time", 2024-05-09T05:01:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.261 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.262 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.262 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2359", 2024-05-09T05:01:48.262 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.262 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.262 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.262 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ending time of day to automatically balance", 2024-05-09T05:01:48.262 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-09T05:01:48.262 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.262 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.262 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.262 INFO:teuthology.orchestra.run.smithi046.stdout: "end_weekday": { 2024-05-09T05:01:48.262 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "end_weekday", 2024-05-09T05:01:48.262 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T05:01:48.262 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.263 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.263 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.263 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "0", 2024-05-09T05:01:48.263 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "6", 2024-05-09T05:01:48.263 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.263 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-09T05:01:48.263 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-09T05:01:48.263 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.263 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.263 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.263 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.263 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.264 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.265 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.265 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.265 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.265 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.265 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.265 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.265 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.265 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.265 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.265 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.265 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.265 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.265 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.265 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.266 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.266 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.266 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.266 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.266 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.266 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.266 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.266 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.266 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.266 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.266 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.266 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.266 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.266 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.267 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.267 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.267 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.267 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.267 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.267 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.267 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.267 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.267 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.267 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.267 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.267 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.267 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.267 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.267 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.268 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.268 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.268 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.268 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.268 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.268 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.268 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.268 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.268 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.268 INFO:teuthology.orchestra.run.smithi046.stdout: "min_score": { 2024-05-09T05:01:48.268 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_score", 2024-05-09T05:01:48.268 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:48.268 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.268 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.269 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.269 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.269 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.269 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.269 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-09T05:01:48.269 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.269 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.269 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.269 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.269 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-09T05:01:48.269 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-09T05:01:48.269 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.269 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.269 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.269 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "upmap", 2024-05-09T05:01:48.270 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.270 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.270 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.270 INFO:teuthology.orchestra.run.smithi046.stdout: "crush-compat", 2024-05-09T05:01:48.270 INFO:teuthology.orchestra.run.smithi046.stdout: "none", 2024-05-09T05:01:48.270 INFO:teuthology.orchestra.run.smithi046.stdout: "read", 2024-05-09T05:01:48.270 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap", 2024-05-09T05:01:48.270 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap-read" 2024-05-09T05:01:48.270 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.270 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Balancer mode", 2024-05-09T05:01:48.270 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.270 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.270 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.270 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.270 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_ids": { 2024-05-09T05:01:48.271 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_ids", 2024-05-09T05:01:48.271 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.271 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.271 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.271 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.271 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.271 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.271 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.271 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-09T05:01:48.271 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.271 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.271 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.271 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.271 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T05:01:48.271 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T05:01:48.272 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.272 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.272 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.272 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T05:01:48.272 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.272 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.272 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.272 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-09T05:01:48.272 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.272 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.272 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.272 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.272 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.272 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.273 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.273 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.273 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.273 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.273 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.273 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.273 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.273 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.273 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.273 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.273 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.273 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.273 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_deviation": { 2024-05-09T05:01:48.273 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_deviation", 2024-05-09T05:01:48.273 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.274 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.274 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.274 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T05:01:48.274 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T05:01:48.274 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.274 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.274 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-09T05:01:48.274 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-09T05:01:48.274 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.274 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.274 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.274 INFO:teuthology.orchestra.run.smithi046.stdout: "upmap_max_optimizations": { 2024-05-09T05:01:48.274 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "upmap_max_optimizations", 2024-05-09T05:01:48.274 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm", 2024-05-09T05:01:48.275 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.276 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.276 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.276 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_down_multiplier": { 2024-05-09T05:01:48.276 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_down_multiplier", 2024-05-09T05:01:48.276 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:48.276 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.276 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.276 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-09T05:01:48.276 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.276 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.276 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.276 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-09T05:01:48.276 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.276 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.276 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.277 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_refresh_rate": { 2024-05-09T05:01:48.277 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_refresh_rate", 2024-05-09T05:01:48.277 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.277 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.277 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.277 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "20", 2024-05-09T05:01:48.277 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.277 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.277 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.277 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-09T05:01:48.277 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.277 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.277 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.277 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.278 INFO:teuthology.orchestra.run.smithi046.stdout: "agent_starting_port": { 2024-05-09T05:01:48.278 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "agent_starting_port", 2024-05-09T05:01:48.278 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.278 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.278 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.278 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4721", 2024-05-09T05:01:48.278 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.278 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.278 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.278 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-09T05:01:48.278 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.278 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.278 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.279 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_ptrace": { 2024-05-09T05:01:48.279 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_ptrace", 2024-05-09T05:01:48.279 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.279 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.279 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.279 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.279 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.279 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.279 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.279 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-09T05:01:48.279 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-09T05:01:48.279 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.279 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.279 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_interval": { 2024-05-09T05:01:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_interval", 2024-05-09T05:01:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T05:01:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-09T05:01:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.280 INFO:teuthology.orchestra.run.smithi046.stdout: "autotune_memory_target_ratio": { 2024-05-09T05:01:48.281 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "autotune_memory_target_ratio", 2024-05-09T05:01:48.281 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:48.281 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.281 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.281 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0.7", 2024-05-09T05:01:48.281 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.281 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.281 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.281 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-09T05:01:48.281 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.281 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.281 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.281 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.281 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm_log_destination": { 2024-05-09T05:01:48.281 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cephadm_log_destination", 2024-05-09T05:01:48.282 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.282 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.282 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.282 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.282 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.282 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.282 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.282 INFO:teuthology.orchestra.run.smithi046.stdout: "file", 2024-05-09T05:01:48.282 INFO:teuthology.orchestra.run.smithi046.stdout: "file,syslog", 2024-05-09T05:01:48.282 INFO:teuthology.orchestra.run.smithi046.stdout: "syslog" 2024-05-09T05:01:48.282 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.282 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-09T05:01:48.282 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.282 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: "cgroups_split": { 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cgroups_split", 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.283 INFO:teuthology.orchestra.run.smithi046.stdout: "config_checks_enabled": { 2024-05-09T05:01:48.284 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_checks_enabled", 2024-05-09T05:01:48.284 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.284 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.284 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.284 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.284 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.284 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.284 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.284 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-09T05:01:48.284 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.284 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.284 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.284 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.284 INFO:teuthology.orchestra.run.smithi046.stdout: "config_dashboard": { 2024-05-09T05:01:48.284 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_dashboard", 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_alertmanager": { 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_alertmanager", 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.285 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.286 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.286 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-09T05:01:48.286 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.286 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.286 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.286 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T05:01:48.286 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.286 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.286 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.286 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.286 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_base": { 2024-05-09T05:01:48.286 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_base", 2024-05-09T05:01:48.286 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Container image name, without the tag", 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_elasticsearch": { 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_elasticsearch", 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.287 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "elasticsearch container image", 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_grafana": { 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_grafana", 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-09T05:01:48.288 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T05:01:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_haproxy": { 2024-05-09T05:01:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_haproxy", 2024-05-09T05:01:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-09T05:01:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.289 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.290 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.290 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "HAproxy container image", 2024-05-09T05:01:48.290 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.290 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.290 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.290 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.290 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_agent": { 2024-05-09T05:01:48.290 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_agent", 2024-05-09T05:01:48.290 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.290 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.290 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.290 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-09T05:01:48.290 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.290 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.291 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.291 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger agent container image", 2024-05-09T05:01:48.291 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.291 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.291 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.291 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.291 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_collector": { 2024-05-09T05:01:48.291 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_collector", 2024-05-09T05:01:48.291 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.291 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.291 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.291 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-09T05:01:48.291 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger collector container image", 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_jaeger_query": { 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_jaeger_query", 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.292 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Jaeger query container image", 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_keepalived": { 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_keepalived", 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Keepalived container image", 2024-05-09T05:01:48.293 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.294 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.294 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.294 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.294 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_loki": { 2024-05-09T05:01:48.294 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_loki", 2024-05-09T05:01:48.294 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.294 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.294 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.294 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-09T05:01:48.294 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.294 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.294 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.294 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Loki container image", 2024-05-09T05:01:48.294 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.294 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.295 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.295 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.295 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_node_exporter": { 2024-05-09T05:01:48.295 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_node_exporter", 2024-05-09T05:01:48.295 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.295 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.295 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.295 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-09T05:01:48.295 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.295 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.295 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.295 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T05:01:48.295 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.295 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.295 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.296 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_nvmeof": { 2024-05-09T05:01:48.296 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_nvmeof", 2024-05-09T05:01:48.296 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.296 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.296 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.296 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-09T05:01:48.296 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.296 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.296 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.296 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Nvme-of container image", 2024-05-09T05:01:48.296 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.296 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.296 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.296 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.297 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_prometheus": { 2024-05-09T05:01:48.297 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_prometheus", 2024-05-09T05:01:48.297 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.297 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.297 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.297 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-09T05:01:48.297 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.297 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.297 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.297 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Prometheus container image", 2024-05-09T05:01:48.297 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.297 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.297 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.297 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.297 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_promtail": { 2024-05-09T05:01:48.298 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_promtail", 2024-05-09T05:01:48.298 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.298 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.298 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.298 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-09T05:01:48.298 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.298 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.298 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.298 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Promtail container image", 2024-05-09T05:01:48.298 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.298 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.298 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.298 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.298 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_samba": { 2024-05-09T05:01:48.299 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_samba", 2024-05-09T05:01:48.299 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.299 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.299 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.299 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-09T05:01:48.299 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.299 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.299 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.299 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Samba/SMB container image", 2024-05-09T05:01:48.299 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.299 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.299 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.299 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.299 INFO:teuthology.orchestra.run.smithi046.stdout: "container_image_snmp_gateway": { 2024-05-09T05:01:48.299 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_image_snmp_gateway", 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "SNMP Gateway container image", 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: "container_init": { 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "container_init", 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.300 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Run podman/docker with `--init`", 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: "daemon_cache_timeout": { 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "daemon_cache_timeout", 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.301 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T05:01:48.302 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.302 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.302 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.302 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-09T05:01:48.302 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.302 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.302 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.302 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.302 INFO:teuthology.orchestra.run.smithi046.stdout: "default_cephadm_command_timeout": { 2024-05-09T05:01:48.302 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_cephadm_command_timeout", 2024-05-09T05:01:48.302 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.302 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.302 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.302 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "900", 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: "default_registry": { 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "default_registry", 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "docker.io", 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.303 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.304 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.304 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-09T05:01:48.304 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.304 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.304 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.304 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.304 INFO:teuthology.orchestra.run.smithi046.stdout: "device_cache_timeout": { 2024-05-09T05:01:48.304 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_cache_timeout", 2024-05-09T05:01:48.304 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.304 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.304 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.304 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1800", 2024-05-09T05:01:48.304 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.304 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.305 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.305 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache device inventory", 2024-05-09T05:01:48.305 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.305 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.305 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.305 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.305 INFO:teuthology.orchestra.run.smithi046.stdout: "device_enhanced_scan": { 2024-05-09T05:01:48.305 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_enhanced_scan", 2024-05-09T05:01:48.305 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.305 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.305 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.305 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.305 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.305 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.305 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.306 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-09T05:01:48.306 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.306 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.306 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.306 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.306 INFO:teuthology.orchestra.run.smithi046.stdout: "facts_cache_timeout": { 2024-05-09T05:01:48.306 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "facts_cache_timeout", 2024-05-09T05:01:48.306 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.306 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.306 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.306 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T05:01:48.306 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.306 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.307 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.307 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "seconds to cache host facts data", 2024-05-09T05:01:48.307 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.307 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.307 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.307 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.307 INFO:teuthology.orchestra.run.smithi046.stdout: "grafana_dashboards_path": { 2024-05-09T05:01:48.307 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "grafana_dashboards_path", 2024-05-09T05:01:48.307 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.307 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.307 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.307 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-09T05:01:48.307 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.307 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.308 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.308 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-09T05:01:48.308 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.308 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.308 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.308 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.308 INFO:teuthology.orchestra.run.smithi046.stdout: "host_check_interval": { 2024-05-09T05:01:48.308 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "host_check_interval", 2024-05-09T05:01:48.308 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.308 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.308 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.308 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T05:01:48.308 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.308 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.308 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.309 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to perform a host check", 2024-05-09T05:01:48.309 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.309 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.309 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.309 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.309 INFO:teuthology.orchestra.run.smithi046.stdout: "hw_monitoring": { 2024-05-09T05:01:48.309 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hw_monitoring", 2024-05-09T05:01:48.309 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.309 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.309 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.309 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.309 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.309 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.309 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.309 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-09T05:01:48.310 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.310 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.310 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.310 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.310 INFO:teuthology.orchestra.run.smithi046.stdout: "inventory_list_all": { 2024-05-09T05:01:48.310 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "inventory_list_all", 2024-05-09T05:01:48.310 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.310 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.310 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.310 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.310 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.310 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.310 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.310 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-09T05:01:48.311 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.311 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.311 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.311 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.311 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.311 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.311 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.311 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.311 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.311 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.311 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.311 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.311 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.311 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.312 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.312 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.312 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.312 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.312 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.312 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.312 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.312 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.312 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.312 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.312 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.312 INFO:teuthology.orchestra.run.smithi046.stdout: "log_refresh_metadata": { 2024-05-09T05:01:48.312 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_refresh_metadata", 2024-05-09T05:01:48.312 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-09T05:01:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.313 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-09T05:01:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.314 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.315 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.316 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.317 INFO:teuthology.orchestra.run.smithi046.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-09T05:01:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-09T05:01:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "*", 2024-05-09T05:01:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-09T05:01:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.318 INFO:teuthology.orchestra.run.smithi046.stdout: "max_count_per_host": { 2024-05-09T05:01:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_count_per_host", 2024-05-09T05:01:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of daemons per service per host", 2024-05-09T05:01:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.319 INFO:teuthology.orchestra.run.smithi046.stdout: "max_osd_draining_count": { 2024-05-09T05:01:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_osd_draining_count", 2024-05-09T05:01:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-09T05:01:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.320 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "migration_current": { 2024-05-09T05:01:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "migration_current", 2024-05-09T05:01:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "internal - do not modify", 2024-05-09T05:01:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.321 INFO:teuthology.orchestra.run.smithi046.stdout: "mode": { 2024-05-09T05:01:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mode", 2024-05-09T05:01:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "root", 2024-05-09T05:01:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm-package", 2024-05-09T05:01:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "root" 2024-05-09T05:01:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "mode for remote execution of cephadm", 2024-05-09T05:01:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.322 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "oob_default_addr": { 2024-05-09T05:01:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "oob_default_addr", 2024-05-09T05:01:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "169.254.1.1", 2024-05-09T05:01:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-09T05:01:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.323 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "prometheus_alerts_path": { 2024-05-09T05:01:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus_alerts_path", 2024-05-09T05:01:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-09T05:01:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-09T05:01:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.324 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_insecure": { 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_insecure", 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.325 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_password": { 2024-05-09T05:01:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_password", 2024-05-09T05:01:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-09T05:01:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_url": { 2024-05-09T05:01:48.326 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_url", 2024-05-09T05:01:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-09T05:01:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "registry_username": { 2024-05-09T05:01:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "registry_username", 2024-05-09T05:01:48.327 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-09T05:01:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "secure_monitoring_stack": { 2024-05-09T05:01:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "secure_monitoring_stack", 2024-05-09T05:01:48.328 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "service_discovery_port": { 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "service_discovery_port", 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.329 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8765", 2024-05-09T05:01:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "cephadm service discovery port", 2024-05-09T05:01:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.330 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "ssh_config_file": { 2024-05-09T05:01:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssh_config_file", 2024-05-09T05:01:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.331 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-09T05:01:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "use_agent": { 2024-05-09T05:01:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_agent", 2024-05-09T05:01:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.332 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-09T05:01:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "use_repo_digest": { 2024-05-09T05:01:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "use_repo_digest", 2024-05-09T05:01:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.333 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-09T05:01:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_failed_host_check": { 2024-05-09T05:01:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_failed_host_check", 2024-05-09T05:01:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.334 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if the host check fails", 2024-05-09T05:01:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_daemons": { 2024-05-09T05:01:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_daemons", 2024-05-09T05:01:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.335 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-09T05:01:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_on_stray_hosts": { 2024-05-09T05:01:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_on_stray_hosts", 2024-05-09T05:01:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.336 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-09T05:01:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crash", 2024-05-09T05:01:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.337 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.338 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.339 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.340 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.341 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "retain_interval": { 2024-05-09T05:01:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retain_interval", 2024-05-09T05:01:48.342 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "31536000", 2024-05-09T05:01:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-09T05:01:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.343 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_recent_interval": { 2024-05-09T05:01:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_recent_interval", 2024-05-09T05:01:48.344 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1209600", 2024-05-09T05:01:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-09T05:01:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.345 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dashboard", 2024-05-09T05:01:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-09T05:01:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-09T05:01:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.346 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_HOST": { 2024-05-09T05:01:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-09T05:01:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.347 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-09T05:01:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-09T05:01:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.348 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_ENABLED": { 2024-05-09T05:01:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_ENABLED", 2024-05-09T05:01:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.349 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-09T05:01:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-09T05:01:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.350 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "ENABLE_BROWSABLE_API": { 2024-05-09T05:01:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-09T05:01:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.351 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.352 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-09T05:01:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-09T05:01:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.353 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-09T05:01:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-09T05:01:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.354 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-09T05:01:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-09T05:01:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.355 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.356 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-09T05:01:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-09T05:01:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.357 INFO:teuthology.orchestra.run.smithi046.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-09T05:01:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-09T05:01:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.358 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-09T05:01:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-09T05:01:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.359 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_PASSWORD": { 2024-05-09T05:01:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-09T05:01:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T05:01:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-09T05:01:48.360 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-09T05:01:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_URL": { 2024-05-09T05:01:48.361 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_URL", 2024-05-09T05:01:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_API_USERNAME": { 2024-05-09T05:01:48.362 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-09T05:01:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T05:01:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-09T05:01:48.363 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-09T05:01:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-09T05:01:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-09T05:01:48.364 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-09T05:01:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-09T05:01:48.365 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-09T05:01:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-09T05:01:48.366 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-09T05:01:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-09T05:01:48.367 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "[]", 2024-05-09T05:01:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "MULTICLUSTER_CONFIG": { 2024-05-09T05:01:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-09T05:01:48.368 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "{}", 2024-05-09T05:01:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_HOST": { 2024-05-09T05:01:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-09T05:01:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.369 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-09T05:01:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-09T05:01:48.370 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-09T05:01:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-09T05:01:48.371 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-09T05:01:48.372 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-09T05:01:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-09T05:01:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-09T05:01:48.373 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-09T05:01:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-09T05:01:48.374 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-09T05:01:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-09T05:01:48.375 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-09T05:01:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-09T05:01:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.376 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-09T05:01:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-09T05:01:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.377 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_ENABLED": { 2024-05-09T05:01:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-09T05:01:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.378 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-09T05:01:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-09T05:01:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.379 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-09T05:01:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-09T05:01:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-09T05:01:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.380 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.381 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-09T05:01:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-09T05:01:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8", 2024-05-09T05:01:48.382 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.383 INFO:teuthology.orchestra.run.smithi046.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-09T05:01:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-09T05:01:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "45", 2024-05-09T05:01:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.384 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ACCESS_KEY": { 2024-05-09T05:01:48.385 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-09T05:01:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.386 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-09T05:01:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-09T05:01:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "admin", 2024-05-09T05:01:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.387 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SECRET_KEY": { 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "RGW_API_SSL_VERIFY": { 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-09T05:01:48.388 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "UNSAFE_TLS_v1_2": { 2024-05-09T05:01:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-09T05:01:48.389 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-09T05:01:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-09T05:01:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.390 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-09T05:01:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-09T05:01:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.391 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-09T05:01:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-09T05:01:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.392 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T05:01:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "cross_origin_url": { 2024-05-09T05:01:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cross_origin_url", 2024-05-09T05:01:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.393 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "crt_file": { 2024-05-09T05:01:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "crt_file", 2024-05-09T05:01:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.394 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "debug": { 2024-05-09T05:01:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "debug", 2024-05-09T05:01:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.395 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Enable/disable debug options", 2024-05-09T05:01:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "jwt_token_ttl": { 2024-05-09T05:01:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "jwt_token_ttl", 2024-05-09T05:01:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.396 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "28800", 2024-05-09T05:01:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-09T05:01:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-09T05:01:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.397 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.398 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.399 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.400 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.401 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.402 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "motd": { 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "motd", 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.403 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "The message of the day", 2024-05-09T05:01:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect_resolve_ip_addr": { 2024-05-09T05:01:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "redirect_resolve_ip_addr", 2024-05-09T05:01:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.404 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.405 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T05:01:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T05:01:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8080", 2024-05-09T05:01:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.406 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.407 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-09T05:01:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-09T05:01:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.408 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl_server_port": { 2024-05-09T05:01:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl_server_port", 2024-05-09T05:01:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8443", 2024-05-09T05:01:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.409 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-09T05:01:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-09T05:01:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "redirect", 2024-05-09T05:01:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.410 INFO:teuthology.orchestra.run.smithi046.stdout: "redirect" 2024-05-09T05:01:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-09T05:01:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-09T05:01:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-09T05:01:48.411 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-09T05:01:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-09T05:01:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "url_prefix": { 2024-05-09T05:01:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url_prefix", 2024-05-09T05:01:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.412 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-05-09T05:01:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.413 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_monitoring": { 2024-05-09T05:01:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_monitoring", 2024-05-09T05:01:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "monitor device health metrics", 2024-05-09T05:01:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.414 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.415 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.416 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.417 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.418 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.419 INFO:teuthology.orchestra.run.smithi046.stdout: "mark_out_threshold": { 2024-05-09T05:01:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mark_out_threshold", 2024-05-09T05:01:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "2419200", 2024-05-09T05:01:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-09T05:01:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "pool_name": { 2024-05-09T05:01:48.420 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pool_name", 2024-05-09T05:01:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "device_health_metrics", 2024-05-09T05:01:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-09T05:01:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "retention_period": { 2024-05-09T05:01:48.421 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "retention_period", 2024-05-09T05:01:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15552000", 2024-05-09T05:01:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long to retain device health metrics", 2024-05-09T05:01:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_frequency": { 2024-05-09T05:01:48.422 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_frequency", 2024-05-09T05:01:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-09T05:01:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-09T05:01:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "self_heal": { 2024-05-09T05:01:48.423 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "self_heal", 2024-05-09T05:01:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-09T05:01:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T05:01:48.424 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T05:01:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T05:01:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how frequently to wake up and check device health", 2024-05-09T05:01:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.425 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "warn_threshold": { 2024-05-09T05:01:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "warn_threshold", 2024-05-09T05:01:48.426 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7257600", 2024-05-09T05:01:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-09T05:01:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.427 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "diskprediction_local", 2024-05-09T05:01:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.428 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.429 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.430 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.431 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.431 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.431 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.431 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.431 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.431 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.431 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.431 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.431 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.431 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.431 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.431 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.431 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.431 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.432 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.432 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.432 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.432 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.432 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.432 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.432 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.432 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.432 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.432 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.432 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.432 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.432 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.432 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.433 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.433 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.433 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.433 INFO:teuthology.orchestra.run.smithi046.stdout: "predict_interval": { 2024-05-09T05:01:48.433 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predict_interval", 2024-05-09T05:01:48.433 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.433 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.433 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.433 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "86400", 2024-05-09T05:01:48.433 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.433 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.433 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.433 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.433 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.434 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.434 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.434 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.434 INFO:teuthology.orchestra.run.smithi046.stdout: "predictor_model": { 2024-05-09T05:01:48.434 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "predictor_model", 2024-05-09T05:01:48.434 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.434 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.434 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.434 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "prophetstor", 2024-05-09T05:01:48.434 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.434 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.434 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.434 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.434 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.435 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.435 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.435 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.435 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T05:01:48.435 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T05:01:48.435 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.435 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.435 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.435 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "600", 2024-05-09T05:01:48.435 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.435 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.435 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.435 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.435 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.436 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.436 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.436 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.436 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.436 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.436 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.436 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.436 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.436 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.436 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.436 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.436 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.436 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.436 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.437 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.437 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.437 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.437 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.437 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.437 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.437 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "influx", 2024-05-09T05:01:48.437 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": false, 2024-05-09T05:01:48.437 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "influxdb python module not found", 2024-05-09T05:01:48.437 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.437 INFO:teuthology.orchestra.run.smithi046.stdout: "batch_size": { 2024-05-09T05:01:48.437 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "batch_size", 2024-05-09T05:01:48.437 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.437 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.438 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.438 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5000", 2024-05-09T05:01:48.438 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.438 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.438 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.438 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-09T05:01:48.438 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.438 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.438 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.438 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.438 INFO:teuthology.orchestra.run.smithi046.stdout: "database": { 2024-05-09T05:01:48.438 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "database", 2024-05-09T05:01:48.439 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.439 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.439 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.439 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "ceph", 2024-05-09T05:01:48.439 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.439 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.439 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.439 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-09T05:01:48.439 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.439 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.439 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.439 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.439 INFO:teuthology.orchestra.run.smithi046.stdout: "hostname": { 2024-05-09T05:01:48.439 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "hostname", 2024-05-09T05:01:48.439 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.440 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.440 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.440 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.440 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.440 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.440 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.440 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server hostname", 2024-05-09T05:01:48.440 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.440 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.440 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.440 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T05:01:48.440 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T05:01:48.440 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.440 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.441 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.441 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "30", 2024-05-09T05:01:48.441 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "5", 2024-05-09T05:01:48.441 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.441 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.441 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-09T05:01:48.441 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.441 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.441 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.441 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.441 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.441 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.441 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.442 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.442 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.442 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.442 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.442 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.442 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.442 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.442 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.442 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.442 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.442 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.442 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.442 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.442 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.443 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.443 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.443 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.443 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.443 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.443 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.443 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.443 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.443 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.443 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.443 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.443 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.443 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.444 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.445 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.446 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.446 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.446 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.446 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.446 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.446 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.446 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.446 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.446 INFO:teuthology.orchestra.run.smithi046.stdout: "password": { 2024-05-09T05:01:48.446 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "password", 2024-05-09T05:01:48.446 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.446 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.446 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.446 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.447 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.447 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.447 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.447 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "password of InfluxDB server user", 2024-05-09T05:01:48.447 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.447 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.447 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.447 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.447 INFO:teuthology.orchestra.run.smithi046.stdout: "port": { 2024-05-09T05:01:48.447 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "port", 2024-05-09T05:01:48.447 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.447 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.447 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.447 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "8086", 2024-05-09T05:01:48.447 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "InfluxDB server port", 2024-05-09T05:01:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.448 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.449 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.449 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.449 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.449 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.449 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.449 INFO:teuthology.orchestra.run.smithi046.stdout: "ssl": { 2024-05-09T05:01:48.449 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ssl", 2024-05-09T05:01:48.449 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.449 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.449 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.449 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "false", 2024-05-09T05:01:48.449 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.450 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.450 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.450 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T05:01:48.450 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.450 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.450 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.450 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.450 INFO:teuthology.orchestra.run.smithi046.stdout: "threads": { 2024-05-09T05:01:48.450 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threads", 2024-05-09T05:01:48.450 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.450 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.450 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.450 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T05:01:48.450 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T05:01:48.451 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "32", 2024-05-09T05:01:48.451 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.451 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-09T05:01:48.451 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.451 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.451 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.451 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.451 INFO:teuthology.orchestra.run.smithi046.stdout: "username": { 2024-05-09T05:01:48.451 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "username", 2024-05-09T05:01:48.451 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.451 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.451 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.451 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.451 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.452 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.452 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.452 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "username of InfluxDB server user", 2024-05-09T05:01:48.452 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.452 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.452 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.452 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.452 INFO:teuthology.orchestra.run.smithi046.stdout: "verify_ssl": { 2024-05-09T05:01:48.452 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "verify_ssl", 2024-05-09T05:01:48.452 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.452 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.452 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.452 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "true", 2024-05-09T05:01:48.452 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.453 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.453 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.453 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-09T05:01:48.453 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.453 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.453 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.453 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.453 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.453 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.453 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "insights", 2024-05-09T05:01:48.453 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.453 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.453 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.454 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.454 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.454 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.454 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.454 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.454 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.454 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.454 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.454 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.454 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.454 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.454 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.454 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.455 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.455 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.455 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.455 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.455 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.455 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.455 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.455 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.455 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.455 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.455 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.455 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.455 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.455 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.455 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.456 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.457 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.457 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.457 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.457 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.457 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.457 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.457 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.457 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.457 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.457 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.457 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.457 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.457 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.457 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.458 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.458 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.458 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.458 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.458 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.458 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.458 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.458 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.458 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.458 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.458 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.458 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.458 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.459 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.459 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.459 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.459 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.459 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.459 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.459 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.459 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.459 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.459 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.459 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.459 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.459 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.459 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.460 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.460 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.460 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "iostat", 2024-05-09T05:01:48.460 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.460 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.460 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.460 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.460 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.460 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.460 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.460 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.460 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.460 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.460 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.461 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.462 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.462 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.462 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.462 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.462 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.462 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.462 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.462 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.462 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.462 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.462 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.462 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.462 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.462 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.462 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.463 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.463 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.463 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.463 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.463 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.463 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.463 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.463 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.463 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.463 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.463 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.463 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.463 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.463 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.464 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.464 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.464 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.464 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.464 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.464 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.464 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.464 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.464 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.464 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.464 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.464 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.464 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.464 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.465 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.465 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.465 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.465 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.465 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.465 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.465 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.465 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.465 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.465 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.465 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.465 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.465 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.465 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.465 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.466 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.466 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.466 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.466 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.466 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.466 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.466 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "k8sevents", 2024-05-09T05:01:48.466 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.466 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.466 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.466 INFO:teuthology.orchestra.run.smithi046.stdout: "ceph_event_retention_days": { 2024-05-09T05:01:48.466 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "ceph_event_retention_days", 2024-05-09T05:01:48.466 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.466 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.467 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.467 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "7", 2024-05-09T05:01:48.467 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.467 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.467 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.467 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-09T05:01:48.467 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.467 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.467 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.467 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.467 INFO:teuthology.orchestra.run.smithi046.stdout: "config_check_secs": { 2024-05-09T05:01:48.467 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "config_check_secs", 2024-05-09T05:01:48.467 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.467 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.467 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.468 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:48.468 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "10", 2024-05-09T05:01:48.468 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.468 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.468 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-09T05:01:48.468 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.468 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.468 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.468 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.468 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.468 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.468 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.468 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.468 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.468 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.469 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.469 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.469 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.469 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.469 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.469 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.469 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.469 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.469 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.469 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.469 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.469 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.469 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.469 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.469 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.470 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.470 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.470 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.470 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.470 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.470 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.470 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.470 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.470 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.470 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.470 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.470 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.470 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.471 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.471 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.471 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.471 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.471 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.471 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.471 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.471 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.471 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.471 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.471 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.471 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.471 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.471 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.472 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.472 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.472 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.472 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.472 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.472 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.472 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.472 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.472 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.472 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.472 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.472 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.472 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.472 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.472 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.473 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.473 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.473 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.473 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.473 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.473 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.473 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.473 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.473 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.473 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.473 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.473 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.473 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.473 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.474 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.474 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.474 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.474 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.474 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.474 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.474 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.474 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.474 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.474 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.474 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "localpool", 2024-05-09T05:01:48.474 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.474 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.474 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.475 INFO:teuthology.orchestra.run.smithi046.stdout: "failure_domain": { 2024-05-09T05:01:48.475 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "failure_domain", 2024-05-09T05:01:48.475 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.475 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.475 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.475 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "host", 2024-05-09T05:01:48.475 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.475 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.475 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.475 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "failure domain for any created local pool", 2024-05-09T05:01:48.475 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-09T05:01:48.475 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.475 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.475 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.475 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.476 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.476 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.476 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.476 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.476 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.476 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.476 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.476 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.476 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.476 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.476 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.476 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.476 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.476 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.477 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.477 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.477 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.477 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.477 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.477 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.477 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.477 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.477 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.477 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.477 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.477 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.477 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.477 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.477 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.478 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.478 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.478 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.478 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.478 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.478 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.478 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.478 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.478 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.478 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.478 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.478 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.478 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.478 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.479 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.479 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.479 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.479 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.479 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.479 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.479 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.479 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.479 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.479 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.479 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.479 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.479 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.479 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.479 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.480 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.480 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.480 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.480 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.480 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.480 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.480 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.480 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.480 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.480 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.480 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.480 INFO:teuthology.orchestra.run.smithi046.stdout: "min_size": { 2024-05-09T05:01:48.480 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "min_size", 2024-05-09T05:01:48.480 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.481 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.481 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.481 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.481 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.481 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.481 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.481 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default min_size for any created local pool", 2024-05-09T05:01:48.481 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-09T05:01:48.481 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.481 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.481 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.481 INFO:teuthology.orchestra.run.smithi046.stdout: "num_rep": { 2024-05-09T05:01:48.481 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "num_rep", 2024-05-09T05:01:48.482 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.482 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.482 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.482 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3", 2024-05-09T05:01:48.482 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.482 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.482 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.482 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default replica count for any created local pool", 2024-05-09T05:01:48.482 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.482 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.482 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.482 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.482 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_num": { 2024-05-09T05:01:48.482 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_num", 2024-05-09T05:01:48.483 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.483 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.483 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.483 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "128", 2024-05-09T05:01:48.483 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.483 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.483 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.483 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "default pg_num for any created local pool", 2024-05-09T05:01:48.483 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.483 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.483 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.483 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.483 INFO:teuthology.orchestra.run.smithi046.stdout: "prefix": { 2024-05-09T05:01:48.483 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prefix", 2024-05-09T05:01:48.483 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "name prefix for any created local pool", 2024-05-09T05:01:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.484 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.485 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.485 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.485 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.485 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.485 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.485 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.485 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.485 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.485 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.485 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.485 INFO:teuthology.orchestra.run.smithi046.stdout: "subtree": { 2024-05-09T05:01:48.485 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "subtree", 2024-05-09T05:01:48.485 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.485 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.486 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.486 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "rack", 2024-05-09T05:01:48.486 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.486 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.486 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.486 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-09T05:01:48.486 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-09T05:01:48.486 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.486 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.486 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.486 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.486 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.486 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.486 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mds_autoscaler", 2024-05-09T05:01:48.487 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.487 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.487 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.487 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.487 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.487 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.487 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.487 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.487 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.487 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.487 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.487 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.487 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.487 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.487 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.488 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.488 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.488 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.488 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.488 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.488 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.488 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.488 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.488 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.488 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.488 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.488 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.488 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.488 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.489 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.489 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.489 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.489 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.489 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.489 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.489 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.489 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.489 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.489 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.489 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.489 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.489 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.489 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.490 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.490 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.490 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.490 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.490 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.490 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.490 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.490 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.490 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.490 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.490 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.490 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.490 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.490 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.490 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.491 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.491 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.491 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.491 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.491 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.491 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.491 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.491 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.491 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.491 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.491 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.491 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.491 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.491 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.492 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.492 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.492 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.492 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.492 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.492 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.492 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.492 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.492 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.492 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.492 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.492 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.492 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.492 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirroring", 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.493 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.494 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.494 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.494 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.494 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.494 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.494 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.494 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.494 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.494 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.494 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.494 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.494 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.494 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.494 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.494 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.495 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.495 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.495 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.495 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.495 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.495 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.495 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.495 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.495 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.495 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.495 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.495 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.495 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.495 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.496 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.496 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.496 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.496 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.496 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.496 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.496 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.496 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.496 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.496 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.496 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.496 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.496 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.496 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.497 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.497 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.497 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.497 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.497 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.497 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.497 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.497 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.497 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.497 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.497 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.497 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.497 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.497 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.498 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.498 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.498 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.498 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.498 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.498 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.498 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.498 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.498 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.498 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.498 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.498 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.498 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.499 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.499 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.499 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.499 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.499 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.499 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.499 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "nfs", 2024-05-09T05:01:48.499 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.499 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.499 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.499 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.499 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.499 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.500 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.500 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.500 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.500 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.500 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.500 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.500 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.500 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.500 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.500 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.500 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.500 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.500 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.500 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.501 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.501 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.501 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.501 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.501 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.501 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.501 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.501 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.501 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.501 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.501 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.501 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.501 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.502 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.502 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.502 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.502 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.502 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.502 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.502 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.502 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.502 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.502 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.502 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.502 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.502 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.503 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.503 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.503 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.503 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.503 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.503 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.503 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.503 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.503 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.503 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.503 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.503 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.503 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.504 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.505 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: "fail_fs": { 2024-05-09T05:01:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "fail_fs", 2024-05-09T05:01:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-09T05:01:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.506 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.507 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.507 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.507 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.507 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.507 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.507 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.507 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.507 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.507 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.507 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.507 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.507 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.507 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.507 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.508 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.509 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.509 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.509 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.509 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.509 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.509 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.509 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.509 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.509 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.509 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.509 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.509 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.509 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.509 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.510 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.510 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.510 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.510 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.510 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.510 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.510 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.510 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.510 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.510 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.510 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.510 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.510 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.510 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.510 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator": { 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "orchestrator", 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.511 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.512 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.512 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.512 INFO:teuthology.orchestra.run.smithi046.stdout: "cephadm", 2024-05-09T05:01:48.512 INFO:teuthology.orchestra.run.smithi046.stdout: "rook", 2024-05-09T05:01:48.512 INFO:teuthology.orchestra.run.smithi046.stdout: "test_orchestrator" 2024-05-09T05:01:48.512 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.512 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Orchestrator backend", 2024-05-09T05:01:48.512 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.512 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.512 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.512 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.512 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.512 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.512 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.512 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_perf_query", 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.513 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.514 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.514 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.514 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.514 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.514 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.514 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.514 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.514 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.514 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.514 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.514 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.514 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.514 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.515 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.516 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.516 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.516 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.516 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.516 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.516 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.516 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.516 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.516 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.516 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.516 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.516 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.516 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.516 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.517 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.518 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.518 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.518 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.518 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.518 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.518 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.518 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.518 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.518 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.518 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.518 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.518 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.518 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.519 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.519 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.519 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.519 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.519 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.519 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.519 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.519 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.519 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.519 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.519 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.519 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.519 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.520 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.520 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.520 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.520 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "osd_support", 2024-05-09T05:01:48.520 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.520 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.520 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.520 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.520 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.520 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.520 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.520 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.520 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.520 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.520 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.521 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.521 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.521 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.521 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.521 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.521 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.521 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.521 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.521 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.521 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.521 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.521 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.521 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.521 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.522 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.522 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.522 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.522 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.522 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.522 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.522 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.522 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.522 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.522 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.522 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.522 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.522 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.522 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.523 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.523 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.523 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.523 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.523 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.523 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.523 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.523 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.523 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.523 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.523 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.523 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.523 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.523 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.524 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.524 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.524 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.524 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.524 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.524 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.524 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.524 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.524 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.524 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.524 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.524 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.524 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.524 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.525 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.525 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.525 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.525 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.525 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.525 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.525 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.525 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.525 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.525 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.525 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.525 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.525 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.525 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.525 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.526 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.526 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.526 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.526 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.526 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.526 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.526 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.526 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.526 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "pg_autoscaler", 2024-05-09T05:01:48.526 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.526 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.526 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.526 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.526 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.526 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.527 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.527 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.527 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.527 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.527 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.527 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.527 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.527 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.527 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.527 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.527 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.527 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.527 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.527 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.528 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.528 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.528 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.528 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.528 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.528 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.528 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.528 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.528 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.528 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.528 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.528 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.528 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.528 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.528 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.529 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.529 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.529 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.529 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.529 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.529 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.529 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.529 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.529 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.529 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.529 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.529 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.529 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.529 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.530 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.530 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.530 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.530 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.530 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.530 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.530 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.530 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.530 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.530 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.530 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.530 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.530 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.530 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T05:01:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T05:01:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.531 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.532 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T05:01:48.532 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.532 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.532 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.532 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.532 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.532 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.532 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.532 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.532 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.532 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.532 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.532 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.532 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.533 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.533 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.533 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.533 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.533 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.533 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.533 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.533 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.533 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.533 INFO:teuthology.orchestra.run.smithi046.stdout: "threshold": { 2024-05-09T05:01:48.533 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "threshold", 2024-05-09T05:01:48.533 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:48.533 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.533 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.534 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "3.0", 2024-05-09T05:01:48.534 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1.0", 2024-05-09T05:01:48.534 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.534 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.534 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "scaling threshold", 2024-05-09T05:01:48.534 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-09T05:01:48.534 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.534 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.534 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.534 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.534 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.534 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "progress", 2024-05-09T05:01:48.534 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.535 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.535 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.535 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_pg_recovery_event": { 2024-05-09T05:01:48.535 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_pg_recovery_event", 2024-05-09T05:01:48.535 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.535 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.535 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.535 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.535 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.535 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.535 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.535 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-09T05:01:48.535 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.536 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.536 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.536 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.536 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-09T05:01:48.536 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-09T05:01:48.536 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.536 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.536 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.536 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.536 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.536 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.536 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.536 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.536 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.537 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.537 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.537 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.537 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.537 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.537 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.537 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.537 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.537 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.537 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.537 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.537 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.537 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.537 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.537 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.538 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.538 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.538 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.538 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.538 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.538 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.538 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.538 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.538 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.538 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.538 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.538 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.538 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.538 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.539 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.539 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.539 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.539 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.539 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.539 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.539 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.539 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.539 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.539 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.539 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.539 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.539 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.539 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.540 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.540 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.540 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.540 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.540 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.540 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.540 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.540 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.540 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.540 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.540 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.540 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.540 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.540 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.541 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.541 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.541 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.541 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.541 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.541 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.541 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.541 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.541 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.541 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.541 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.541 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.541 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.541 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.541 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.542 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.542 INFO:teuthology.orchestra.run.smithi046.stdout: "max_completed_events": { 2024-05-09T05:01:48.542 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_completed_events", 2024-05-09T05:01:48.542 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.542 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.542 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.542 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "50", 2024-05-09T05:01:48.542 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.542 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.542 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.542 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "number of past completed events to remember", 2024-05-09T05:01:48.542 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.542 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: "sleep_interval": { 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sleep_interval", 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "5", 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "how long the module is going to sleep", 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.543 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.544 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "prometheus", 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: "cache": { 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "cache", 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.545 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.546 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.546 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.546 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.546 INFO:teuthology.orchestra.run.smithi046.stdout: "exclude_perf_counters": { 2024-05-09T05:01:48.546 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "exclude_perf_counters", 2024-05-09T05:01:48.546 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.546 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.546 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.546 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.546 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.546 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.546 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.546 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-09T05:01:48.546 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.547 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.548 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.548 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.548 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.548 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.548 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.548 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.548 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.548 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.548 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.548 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.548 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.548 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.548 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.548 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.549 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.549 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.549 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.549 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.549 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.549 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.549 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.549 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.549 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.549 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.549 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.549 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.549 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.549 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.550 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.550 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.550 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.550 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.550 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.550 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.550 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.550 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.550 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.550 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.550 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.550 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.550 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.550 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.551 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.551 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.551 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.551 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.551 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.551 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.551 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.551 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.551 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.551 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.551 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.551 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.551 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.551 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.552 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.552 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools": { 2024-05-09T05:01:48.552 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools", 2024-05-09T05:01:48.552 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.552 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.552 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.552 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.552 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.552 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.552 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.552 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.552 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.552 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.553 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.553 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.553 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-09T05:01:48.553 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-09T05:01:48.553 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.553 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.553 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.553 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "300", 2024-05-09T05:01:48.553 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.553 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.553 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.553 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.553 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.553 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.553 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.554 INFO:teuthology.orchestra.run.smithi046.stdout: "scrape_interval": { 2024-05-09T05:01:48.554 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "scrape_interval", 2024-05-09T05:01:48.554 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:48.554 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.554 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.554 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15.0", 2024-05-09T05:01:48.554 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.554 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.554 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.554 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.554 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.554 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.554 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.554 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.555 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T05:01:48.555 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T05:01:48.555 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.555 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.555 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.555 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "::", 2024-05-09T05:01:48.555 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.555 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.555 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.555 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-09T05:01:48.555 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.555 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.555 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.555 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "9283", 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.556 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.557 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.557 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.557 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.557 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.557 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.557 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.557 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.557 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.557 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.557 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.557 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.557 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.557 INFO:teuthology.orchestra.run.smithi046.stdout: "stale_cache_strategy": { 2024-05-09T05:01:48.557 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stale_cache_strategy", 2024-05-09T05:01:48.558 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.558 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.558 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.558 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "log", 2024-05-09T05:01:48.558 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.558 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.558 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.558 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.558 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.558 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.558 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.558 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.558 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_behaviour": { 2024-05-09T05:01:48.558 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_behaviour", 2024-05-09T05:01:48.559 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.559 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.559 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.559 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "default", 2024-05-09T05:01:48.559 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.559 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.559 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.559 INFO:teuthology.orchestra.run.smithi046.stdout: "default", 2024-05-09T05:01:48.559 INFO:teuthology.orchestra.run.smithi046.stdout: "error" 2024-05-09T05:01:48.559 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.559 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.559 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.559 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.559 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.560 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.560 INFO:teuthology.orchestra.run.smithi046.stdout: "standby_error_status_code": { 2024-05-09T05:01:48.560 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "standby_error_status_code", 2024-05-09T05:01:48.560 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.560 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.560 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.560 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "500", 2024-05-09T05:01:48.560 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "400", 2024-05-09T05:01:48.560 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "599", 2024-05-09T05:01:48.560 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.560 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.560 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.560 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.560 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.560 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-05-09T05:01:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.561 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.562 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.562 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.562 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.562 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.562 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.562 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.562 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.562 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.562 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.562 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.562 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.562 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.562 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.563 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.563 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.563 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.563 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.563 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.563 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.563 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.563 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.563 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.563 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.563 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.563 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.563 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.563 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.564 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.564 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.564 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.564 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.564 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.564 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.564 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.564 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.564 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.564 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.564 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.564 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.564 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.564 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.564 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.565 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.565 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.565 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.565 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.565 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.565 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.565 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.565 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.565 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.565 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.565 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.565 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.565 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.565 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_snap_create": { 2024-05-09T05:01:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_snap_create", 2024-05-09T05:01:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10", 2024-05-09T05:01:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.566 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.567 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.567 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.567 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.567 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.567 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.567 INFO:teuthology.orchestra.run.smithi046.stdout: "mirror_snapshot_schedule": { 2024-05-09T05:01:48.567 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "mirror_snapshot_schedule", 2024-05-09T05:01:48.567 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.567 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.567 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.567 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.567 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.567 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.567 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.568 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.568 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.568 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.568 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.568 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.568 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.568 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.568 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.568 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.568 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.568 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.568 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.568 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.568 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.568 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.569 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.569 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.569 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.569 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.569 INFO:teuthology.orchestra.run.smithi046.stdout: "trash_purge_schedule": { 2024-05-09T05:01:48.569 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "trash_purge_schedule", 2024-05-09T05:01:48.569 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.569 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.569 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.569 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.569 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.569 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.569 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.569 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.570 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.570 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.570 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.570 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.570 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.570 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.570 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "restful", 2024-05-09T05:01:48.570 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.570 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.570 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.570 INFO:teuthology.orchestra.run.smithi046.stdout: "enable_auth": { 2024-05-09T05:01:48.570 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enable_auth", 2024-05-09T05:01:48.570 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.570 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.571 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.571 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.571 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.571 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.571 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.571 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.571 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.571 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.571 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.571 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.571 INFO:teuthology.orchestra.run.smithi046.stdout: "key_file": { 2024-05-09T05:01:48.571 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "key_file", 2024-05-09T05:01:48.571 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.571 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.571 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.572 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.572 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.572 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.572 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.572 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.572 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.572 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.572 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.572 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.572 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.572 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.572 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.572 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.572 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.573 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.573 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.573 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.573 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.573 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.573 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.573 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.573 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.573 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.573 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.573 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.573 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.573 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.573 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.574 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.574 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.574 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.574 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.574 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.574 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.574 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.574 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.574 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.574 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.574 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.574 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.574 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.574 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.575 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.576 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.576 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.576 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.576 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.576 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.576 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.576 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.576 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.576 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.576 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.576 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.576 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.576 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: "server_addr": { 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_addr", 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.577 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.578 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.578 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.578 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.578 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.578 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.578 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.578 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.578 INFO:teuthology.orchestra.run.smithi046.stdout: "server_port": { 2024-05-09T05:01:48.578 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "server_port", 2024-05-09T05:01:48.578 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.578 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.578 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.578 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.578 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.579 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.579 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.579 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.579 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.579 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.579 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.579 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.579 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.579 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.579 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.579 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.579 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.579 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.579 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.580 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.580 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.580 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.580 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.580 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.580 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.580 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.580 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.580 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.580 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.580 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rgw", 2024-05-09T05:01:48.580 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.580 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.580 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.581 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.581 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.581 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.581 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.581 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.581 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.581 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.581 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.581 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.581 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.581 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.581 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.581 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.582 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.582 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.582 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.582 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.582 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.582 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.582 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.582 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.582 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.582 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.582 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.582 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.582 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.582 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.583 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.584 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.584 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.584 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.584 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.584 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.584 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.584 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.584 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.584 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.584 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.584 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.584 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.584 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.584 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.584 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.585 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.585 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.585 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.585 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.585 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.585 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.585 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.585 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.585 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.585 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.585 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.585 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.585 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.585 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.586 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.586 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.586 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.586 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.586 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.586 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.586 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.586 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.586 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.586 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.586 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.586 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.586 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rook", 2024-05-09T05:01:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.587 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.588 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.589 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.589 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.589 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.589 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.589 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.589 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.589 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.589 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.589 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.589 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.589 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.589 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.589 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.589 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.590 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.590 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.590 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.590 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.590 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.590 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.590 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.590 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.590 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.590 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.590 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.590 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.590 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.590 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.591 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.591 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.591 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.591 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.591 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.591 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.591 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.591 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.591 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.591 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.591 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.591 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.591 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.591 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.592 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.592 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.592 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.592 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.592 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.592 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.592 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.592 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.592 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.592 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.592 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.592 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.592 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.592 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.592 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.593 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.593 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.593 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.593 INFO:teuthology.orchestra.run.smithi046.stdout: "storage_class": { 2024-05-09T05:01:48.593 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "storage_class", 2024-05-09T05:01:48.593 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.593 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.593 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.593 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "local", 2024-05-09T05:01:48.593 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.593 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.593 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.593 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-09T05:01:48.593 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.594 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.594 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.594 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.594 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.594 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.594 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "selftest", 2024-05-09T05:01:48.594 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.594 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.594 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.594 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.594 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.594 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.594 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.594 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.595 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.595 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.595 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.595 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.595 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.595 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.595 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.595 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.595 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.595 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.595 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.595 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.595 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.595 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.596 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.596 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.596 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.596 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.596 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.596 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.596 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.596 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.596 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.596 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.596 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.596 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.596 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.596 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.597 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.597 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.597 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.597 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.597 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.597 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.597 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.597 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.597 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.597 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.597 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.597 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.597 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.597 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.598 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.598 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.598 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.598 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.598 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.598 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.598 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.598 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.598 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.598 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.598 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.598 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.598 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.598 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.599 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.599 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.599 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.599 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.599 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.599 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.599 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.599 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.599 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.599 INFO:teuthology.orchestra.run.smithi046.stdout: "roption1": { 2024-05-09T05:01:48.599 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption1", 2024-05-09T05:01:48.599 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.599 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.599 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.600 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.600 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.600 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.600 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.600 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.600 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.600 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.600 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.600 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.600 INFO:teuthology.orchestra.run.smithi046.stdout: "roption2": { 2024-05-09T05:01:48.600 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "roption2", 2024-05-09T05:01:48.600 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.600 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.600 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.600 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "xyz", 2024-05-09T05:01:48.601 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.601 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.601 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.601 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.601 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.601 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.601 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.601 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.601 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption1": { 2024-05-09T05:01:48.601 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption1", 2024-05-09T05:01:48.601 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.601 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.601 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.601 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption2": { 2024-05-09T05:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption2", 2024-05-09T05:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.602 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.603 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.603 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.603 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.603 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.603 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.603 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.603 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.603 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.603 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption3": { 2024-05-09T05:01:48.603 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption3", 2024-05-09T05:01:48.603 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "float", 2024-05-09T05:01:48.603 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.603 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.603 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.604 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.604 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.604 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.604 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.604 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.604 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.604 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.604 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.604 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption4": { 2024-05-09T05:01:48.604 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption4", 2024-05-09T05:01:48.604 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.604 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.604 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.604 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.605 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.605 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.605 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.605 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.605 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.605 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.605 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.605 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.605 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption5": { 2024-05-09T05:01:48.605 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption5", 2024-05-09T05:01:48.605 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.605 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.605 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.605 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.606 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.606 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.606 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.606 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.606 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.606 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.606 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.606 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.606 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption6": { 2024-05-09T05:01:48.606 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption6", 2024-05-09T05:01:48.606 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.606 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.606 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.606 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.607 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.607 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.607 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.607 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.607 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.607 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.607 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.607 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.607 INFO:teuthology.orchestra.run.smithi046.stdout: "rwoption7": { 2024-05-09T05:01:48.607 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rwoption7", 2024-05-09T05:01:48.607 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.607 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.607 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.607 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.607 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "1", 2024-05-09T05:01:48.608 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "42", 2024-05-09T05:01:48.608 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.608 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.608 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.608 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.608 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.608 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.608 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.608 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.608 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.608 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.608 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.608 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.608 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.609 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.609 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.609 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.609 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.609 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.609 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.609 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.609 INFO:teuthology.orchestra.run.smithi046.stdout: "testkey": { 2024-05-09T05:01:48.609 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testkey", 2024-05-09T05:01:48.609 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.609 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.609 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.609 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.609 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.610 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.610 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.610 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.610 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.610 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.610 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.610 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.610 INFO:teuthology.orchestra.run.smithi046.stdout: "testlkey": { 2024-05-09T05:01:48.610 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testlkey", 2024-05-09T05:01:48.610 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.610 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.610 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.610 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.610 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.611 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.611 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.611 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.611 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.611 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.611 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.611 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.611 INFO:teuthology.orchestra.run.smithi046.stdout: "testnewline": { 2024-05-09T05:01:48.611 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "testnewline", 2024-05-09T05:01:48.611 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.611 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.611 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.611 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.611 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.611 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.612 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.612 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.612 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.612 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.612 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.612 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.612 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.612 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.612 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.612 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "smb", 2024-05-09T05:01:48.612 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.612 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.612 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.612 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.612 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.613 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.613 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.613 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.613 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.613 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.613 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.613 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.613 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.613 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.613 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.613 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.613 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.613 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.613 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.614 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.614 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.614 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.614 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.614 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.614 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.614 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.614 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.614 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.614 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.614 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.614 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.614 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.614 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.615 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.615 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.615 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.615 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.615 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.615 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.615 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.615 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.615 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.615 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.615 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.615 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.615 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.615 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.616 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.616 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.616 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.616 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.616 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.616 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.616 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.616 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.616 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.616 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.616 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.616 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.616 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.616 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.617 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.617 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.617 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.617 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.617 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.617 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.617 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.617 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.617 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.617 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.617 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.617 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.617 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.617 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.618 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.618 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.618 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.618 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.618 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.618 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.618 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.618 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.618 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.618 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.618 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.618 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.619 INFO:teuthology.orchestra.run.smithi046.stdout: "update_orchestration": { 2024-05-09T05:01:48.619 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "update_orchestration", 2024-05-09T05:01:48.619 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.619 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.619 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.619 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.619 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.619 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.619 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.619 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-09T05:01:48.619 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.619 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.619 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.619 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.620 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.620 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.620 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.620 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-05-09T05:01:48.620 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.620 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.620 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.620 INFO:teuthology.orchestra.run.smithi046.stdout: "allow_m_granularity": { 2024-05-09T05:01:48.620 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "allow_m_granularity", 2024-05-09T05:01:48.620 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.620 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.620 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.620 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.620 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "allow minute scheduled snapshots", 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: "dump_on_update": { 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "dump_on_update", 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.621 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.622 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "dump database to debug log on update", 2024-05-09T05:01:48.622 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.622 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.622 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.622 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.622 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.622 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.622 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.622 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.622 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.622 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.622 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.622 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.622 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.622 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.623 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.623 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.623 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.623 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.623 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.623 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.623 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.623 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.623 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.623 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.623 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.623 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.623 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.623 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.624 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.624 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.624 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.624 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.624 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.624 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.624 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.624 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.624 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.624 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.624 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.624 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.624 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.624 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.624 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.625 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.626 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.627 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.627 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.627 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.627 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.627 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.627 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.627 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.627 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.627 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.627 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.627 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.627 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.627 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.627 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "stats", 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.628 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.629 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.629 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.629 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.629 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.629 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.629 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.629 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.629 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.629 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.629 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.629 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.629 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.629 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.629 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.630 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.630 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.630 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.630 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.630 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.630 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.630 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.630 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.630 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.630 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.630 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.630 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.630 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.630 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.630 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.631 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.632 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "status", 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.633 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.634 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.634 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.634 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.634 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.634 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.634 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.634 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.634 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.634 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.634 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.634 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.634 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.634 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.635 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.635 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.635 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.635 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.635 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.635 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.635 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.635 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.635 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.635 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.635 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.635 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.635 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.635 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.636 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.636 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.636 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.636 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.636 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.636 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.636 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.636 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.636 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.636 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.636 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.636 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.636 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.636 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.636 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.637 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.637 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.637 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.637 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.637 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.637 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.637 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.637 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.637 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.637 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.637 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.637 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.637 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.637 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.637 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.638 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.638 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.638 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.638 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.638 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.638 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.638 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.638 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.638 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.638 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.638 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.638 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.638 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.638 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.639 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.639 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.639 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.639 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.639 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.639 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.639 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.639 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.639 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.639 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.639 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.639 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.639 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.639 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.639 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telegraf", 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: "address": { 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "address", 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.640 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "15", 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.641 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.642 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.642 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.642 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.642 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.642 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.642 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.642 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.642 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.642 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.642 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.642 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.642 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.642 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.642 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.643 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.644 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.645 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.645 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.645 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.645 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.645 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.645 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.645 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.645 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.645 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.645 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.645 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.645 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.645 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.645 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.646 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.646 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.646 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.646 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.646 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.646 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.646 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.646 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.646 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.646 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.646 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.646 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.646 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.646 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.647 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.647 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.647 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.647 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.647 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.647 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.647 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.647 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.647 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.647 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.647 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.647 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.647 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.648 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "telemetry", 2024-05-09T05:01:48.648 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.648 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.648 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.648 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_basic": { 2024-05-09T05:01:48.648 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_basic", 2024-05-09T05:01:48.648 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.648 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.648 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.648 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.648 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.648 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.648 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.648 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-09T05:01:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_crash": { 2024-05-09T05:01:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_crash", 2024-05-09T05:01:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-09T05:01:48.649 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.650 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.650 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.650 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.650 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_device": { 2024-05-09T05:01:48.650 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_device", 2024-05-09T05:01:48.650 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.650 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.650 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.650 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.650 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.650 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.650 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.650 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-09T05:01:48.650 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.651 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.651 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.651 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.651 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_ident": { 2024-05-09T05:01:48.651 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_ident", 2024-05-09T05:01:48.651 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.651 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.651 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.651 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.651 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.651 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.651 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.651 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-09T05:01:48.651 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.652 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.652 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.652 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.652 INFO:teuthology.orchestra.run.smithi046.stdout: "channel_perf": { 2024-05-09T05:01:48.652 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "channel_perf", 2024-05-09T05:01:48.652 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.652 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.652 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.652 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.652 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.652 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.652 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.652 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-09T05:01:48.652 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.653 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.653 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.653 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.653 INFO:teuthology.orchestra.run.smithi046.stdout: "contact": { 2024-05-09T05:01:48.653 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "contact", 2024-05-09T05:01:48.653 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.653 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.653 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.653 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.653 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.653 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.653 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.653 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.653 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.654 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.654 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.654 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.654 INFO:teuthology.orchestra.run.smithi046.stdout: "description": { 2024-05-09T05:01:48.654 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "description", 2024-05-09T05:01:48.654 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.654 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.654 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.654 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.654 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.654 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.654 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.654 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: "device_url": { 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "device_url", 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.655 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.656 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.656 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.656 INFO:teuthology.orchestra.run.smithi046.stdout: "enabled": { 2024-05-09T05:01:48.656 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "enabled", 2024-05-09T05:01:48.656 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.656 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.656 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.656 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.656 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.656 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.656 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.656 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.656 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.656 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.656 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "24", 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "8", 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.657 INFO:teuthology.orchestra.run.smithi046.stdout: "last_opt_revision": { 2024-05-09T05:01:48.658 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "last_opt_revision", 2024-05-09T05:01:48.658 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.658 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.658 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.658 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "1", 2024-05-09T05:01:48.658 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.658 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.658 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.658 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.658 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.658 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.658 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.658 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.658 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard": { 2024-05-09T05:01:48.659 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard", 2024-05-09T05:01:48.659 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.659 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.659 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.659 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.659 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.659 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.659 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.659 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.659 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.659 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.659 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.659 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.659 INFO:teuthology.orchestra.run.smithi046.stdout: "leaderboard_description": { 2024-05-09T05:01:48.659 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "leaderboard_description", 2024-05-09T05:01:48.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.660 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.660 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.660 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.660 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.660 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.660 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.660 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.660 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.660 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.660 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.660 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.660 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.660 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.660 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.661 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.661 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.661 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.661 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.661 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.661 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.661 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.661 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.661 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.661 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.661 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.661 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.661 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.661 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.661 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.662 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.662 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.662 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.662 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.662 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.662 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.662 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.662 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.662 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.662 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.662 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.662 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.662 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.662 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.663 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.663 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.663 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.663 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.663 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.663 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.663 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.663 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.663 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.663 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.663 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.663 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.663 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.664 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.664 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.664 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.664 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.664 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.664 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.664 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.664 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.664 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.664 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.664 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.664 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.664 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.664 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.664 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.665 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.665 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.665 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.665 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.665 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.665 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.665 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.665 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.665 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.665 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.665 INFO:teuthology.orchestra.run.smithi046.stdout: "organization": { 2024-05-09T05:01:48.665 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "organization", 2024-05-09T05:01:48.665 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.665 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.666 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.666 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.666 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.666 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.666 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.666 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.666 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.666 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.666 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.666 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.666 INFO:teuthology.orchestra.run.smithi046.stdout: "proxy": { 2024-05-09T05:01:48.666 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "proxy", 2024-05-09T05:01:48.666 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.666 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.667 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.667 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.667 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.667 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.667 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.667 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.667 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.667 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.667 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.667 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.667 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.667 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.667 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.667 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.667 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: "url": { 2024-05-09T05:01:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "url", 2024-05-09T05:01:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.668 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-09T05:01:48.669 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.669 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.669 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.669 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.669 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.669 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.669 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.669 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.669 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.669 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.669 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.669 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "test_orchestrator", 2024-05-09T05:01:48.669 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.669 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.670 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.670 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.670 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.670 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.670 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.670 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.670 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.670 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.670 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.670 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.670 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.670 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.670 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.670 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.671 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.672 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.673 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.673 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.673 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.673 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.673 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.673 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.673 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.673 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.673 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.673 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.673 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.673 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.673 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.673 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.673 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.674 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.674 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.674 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.674 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.674 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.674 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.674 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.674 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.674 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.674 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.674 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.674 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.674 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.674 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.675 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.676 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.676 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.676 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.676 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.676 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.676 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.676 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.676 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.676 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.676 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.676 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.676 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.676 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.676 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.676 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.677 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.677 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.677 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.677 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.677 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.677 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.677 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.677 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.677 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.677 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.677 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.677 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.677 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.677 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.677 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.678 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.678 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.678 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.678 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.678 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.678 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.678 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.678 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.678 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.678 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.678 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.678 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.678 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.678 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.678 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.679 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.679 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.679 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.679 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.679 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.679 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.679 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.679 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.679 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.679 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.679 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.679 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.679 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.679 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.679 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.680 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.680 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.680 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.680 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.680 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.680 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.680 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.680 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.680 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.680 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.680 INFO:teuthology.orchestra.run.smithi046.stdout: "max_concurrent_clones": { 2024-05-09T05:01:48.680 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "max_concurrent_clones", 2024-05-09T05:01:48.680 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.680 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.680 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.681 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "4", 2024-05-09T05:01:48.681 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.681 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.681 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.681 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-09T05:01:48.681 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.681 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.681 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.681 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.681 INFO:teuthology.orchestra.run.smithi046.stdout: "periodic_async_work": { 2024-05-09T05:01:48.681 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "periodic_async_work", 2024-05-09T05:01:48.681 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.681 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.681 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.682 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.682 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.682 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.682 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.682 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Periodically check for async work", 2024-05-09T05:01:48.682 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.682 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.682 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.682 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.682 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_delay": { 2024-05-09T05:01:48.682 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_delay", 2024-05-09T05:01:48.682 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.682 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.682 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.683 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.683 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.683 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.683 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.683 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-09T05:01:48.683 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.683 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.683 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.683 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.683 INFO:teuthology.orchestra.run.smithi046.stdout: "snapshot_clone_no_wait": { 2024-05-09T05:01:48.683 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snapshot_clone_no_wait", 2024-05-09T05:01:48.683 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.683 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.683 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.684 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "True", 2024-05-09T05:01:48.684 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.684 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.684 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.684 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-09T05:01:48.684 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.684 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.684 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.684 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.684 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.684 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.684 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.684 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.684 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.685 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.685 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.685 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.685 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.685 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.685 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.685 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.685 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.685 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.685 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.685 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.685 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix", 2024-05-09T05:01:48.685 INFO:teuthology.orchestra.run.smithi046.stdout: "can_run": true, 2024-05-09T05:01:48.685 INFO:teuthology.orchestra.run.smithi046.stdout: "error_string": "", 2024-05-09T05:01:48.686 INFO:teuthology.orchestra.run.smithi046.stdout: "module_options": { 2024-05-09T05:01:48.686 INFO:teuthology.orchestra.run.smithi046.stdout: "discovery_interval": { 2024-05-09T05:01:48.686 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "discovery_interval", 2024-05-09T05:01:48.686 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "uint", 2024-05-09T05:01:48.686 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.686 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.686 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "100", 2024-05-09T05:01:48.686 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.686 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.686 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.686 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.686 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.686 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.686 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.687 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.687 INFO:teuthology.orchestra.run.smithi046.stdout: "identifier": { 2024-05-09T05:01:48.687 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "identifier", 2024-05-09T05:01:48.687 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.687 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.687 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.687 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.687 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.687 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.687 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.687 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.687 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.687 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.687 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.688 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.688 INFO:teuthology.orchestra.run.smithi046.stdout: "interval": { 2024-05-09T05:01:48.688 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "interval", 2024-05-09T05:01:48.688 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "secs", 2024-05-09T05:01:48.688 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.688 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.688 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "60", 2024-05-09T05:01:48.688 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.688 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.688 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.688 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.688 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.688 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.688 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.689 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.689 INFO:teuthology.orchestra.run.smithi046.stdout: "log_level": { 2024-05-09T05:01:48.689 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_level", 2024-05-09T05:01:48.689 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.689 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.689 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.689 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.689 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.689 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.689 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.689 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.689 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.689 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.689 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.689 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.690 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.690 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.690 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.690 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.690 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.690 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.690 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.690 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster": { 2024-05-09T05:01:48.690 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster", 2024-05-09T05:01:48.690 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.690 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.690 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.690 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.691 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.691 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.691 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.691 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.691 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.691 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.691 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.691 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.691 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_cluster_level": { 2024-05-09T05:01:48.691 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_cluster_level", 2024-05-09T05:01:48.691 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.691 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.691 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.691 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "info", 2024-05-09T05:01:48.691 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.692 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.692 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [ 2024-05-09T05:01:48.692 INFO:teuthology.orchestra.run.smithi046.stdout: "", 2024-05-09T05:01:48.692 INFO:teuthology.orchestra.run.smithi046.stdout: "critical", 2024-05-09T05:01:48.692 INFO:teuthology.orchestra.run.smithi046.stdout: "debug", 2024-05-09T05:01:48.692 INFO:teuthology.orchestra.run.smithi046.stdout: "error", 2024-05-09T05:01:48.692 INFO:teuthology.orchestra.run.smithi046.stdout: "info", 2024-05-09T05:01:48.692 INFO:teuthology.orchestra.run.smithi046.stdout: "warning" 2024-05-09T05:01:48.692 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.692 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.692 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.692 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.692 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.692 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.693 INFO:teuthology.orchestra.run.smithi046.stdout: "log_to_file": { 2024-05-09T05:01:48.693 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "log_to_file", 2024-05-09T05:01:48.693 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "bool", 2024-05-09T05:01:48.693 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.693 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.693 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "False", 2024-05-09T05:01:48.693 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.693 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.693 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.693 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.693 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.693 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.693 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.693 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.694 INFO:teuthology.orchestra.run.smithi046.stdout: "sqlite3_killpoint": { 2024-05-09T05:01:48.694 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "sqlite3_killpoint", 2024-05-09T05:01:48.694 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.694 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "dev", 2024-05-09T05:01:48.694 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 1, 2024-05-09T05:01:48.694 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "0", 2024-05-09T05:01:48.694 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.694 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.694 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.694 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.694 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.694 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.694 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.694 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.695 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_host": { 2024-05-09T05:01:48.695 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_host", 2024-05-09T05:01:48.695 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.695 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.695 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.695 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "", 2024-05-09T05:01:48.695 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.695 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.695 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.695 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.695 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.695 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.695 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.695 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.696 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_port": { 2024-05-09T05:01:48.696 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_port", 2024-05-09T05:01:48.696 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "int", 2024-05-09T05:01:48.696 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.696 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.696 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "10051", 2024-05-09T05:01:48.696 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.696 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.696 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.696 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.696 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.696 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.696 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.696 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.697 INFO:teuthology.orchestra.run.smithi046.stdout: "zabbix_sender": { 2024-05-09T05:01:48.697 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "zabbix_sender", 2024-05-09T05:01:48.697 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "str", 2024-05-09T05:01:48.697 INFO:teuthology.orchestra.run.smithi046.stdout: "level": "advanced", 2024-05-09T05:01:48.697 INFO:teuthology.orchestra.run.smithi046.stdout: "flags": 0, 2024-05-09T05:01:48.697 INFO:teuthology.orchestra.run.smithi046.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-09T05:01:48.697 INFO:teuthology.orchestra.run.smithi046.stdout: "min": "", 2024-05-09T05:01:48.697 INFO:teuthology.orchestra.run.smithi046.stdout: "max": "", 2024-05-09T05:01:48.697 INFO:teuthology.orchestra.run.smithi046.stdout: "enum_allowed": [], 2024-05-09T05:01:48.697 INFO:teuthology.orchestra.run.smithi046.stdout: "desc": "", 2024-05-09T05:01:48.697 INFO:teuthology.orchestra.run.smithi046.stdout: "long_desc": "", 2024-05-09T05:01:48.697 INFO:teuthology.orchestra.run.smithi046.stdout: "tags": [], 2024-05-09T05:01:48.697 INFO:teuthology.orchestra.run.smithi046.stdout: "see_also": [] 2024-05-09T05:01:48.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.698 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.698 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.698 INFO:teuthology.orchestra.run.smithi046.stdout: "services": { 2024-05-09T05:01:48.698 INFO:teuthology.orchestra.run.smithi046.stdout: "dashboard": "https://172.21.15.46:8443/" 2024-05-09T05:01:48.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.698 INFO:teuthology.orchestra.run.smithi046.stdout: "always_on_modules": { 2024-05-09T05:01:48.698 INFO:teuthology.orchestra.run.smithi046.stdout: "octopus": [ 2024-05-09T05:01:48.698 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-05-09T05:01:48.698 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-05-09T05:01:48.698 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-05-09T05:01:48.698 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-05-09T05:01:48.698 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-05-09T05:01:48.699 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-05-09T05:01:48.699 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-05-09T05:01:48.699 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-05-09T05:01:48.699 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-05-09T05:01:48.699 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-05-09T05:01:48.699 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.699 INFO:teuthology.orchestra.run.smithi046.stdout: "pacific": [ 2024-05-09T05:01:48.699 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-05-09T05:01:48.699 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-05-09T05:01:48.699 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-05-09T05:01:48.699 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-05-09T05:01:48.699 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-05-09T05:01:48.699 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-05-09T05:01:48.699 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-05-09T05:01:48.699 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-05-09T05:01:48.700 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-05-09T05:01:48.700 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-05-09T05:01:48.700 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.700 INFO:teuthology.orchestra.run.smithi046.stdout: "quincy": [ 2024-05-09T05:01:48.700 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-05-09T05:01:48.700 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-05-09T05:01:48.700 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-05-09T05:01:48.700 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-05-09T05:01:48.700 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-05-09T05:01:48.700 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-05-09T05:01:48.700 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-05-09T05:01:48.700 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-05-09T05:01:48.700 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-05-09T05:01:48.700 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-05-09T05:01:48.700 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.701 INFO:teuthology.orchestra.run.smithi046.stdout: "reef": [ 2024-05-09T05:01:48.701 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-05-09T05:01:48.701 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-05-09T05:01:48.701 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-05-09T05:01:48.701 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-05-09T05:01:48.701 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-05-09T05:01:48.701 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-05-09T05:01:48.701 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-05-09T05:01:48.701 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-05-09T05:01:48.701 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-05-09T05:01:48.701 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-05-09T05:01:48.701 INFO:teuthology.orchestra.run.smithi046.stdout: ], 2024-05-09T05:01:48.701 INFO:teuthology.orchestra.run.smithi046.stdout: "squid": [ 2024-05-09T05:01:48.701 INFO:teuthology.orchestra.run.smithi046.stdout: "balancer", 2024-05-09T05:01:48.702 INFO:teuthology.orchestra.run.smithi046.stdout: "crash", 2024-05-09T05:01:48.702 INFO:teuthology.orchestra.run.smithi046.stdout: "devicehealth", 2024-05-09T05:01:48.702 INFO:teuthology.orchestra.run.smithi046.stdout: "orchestrator", 2024-05-09T05:01:48.702 INFO:teuthology.orchestra.run.smithi046.stdout: "pg_autoscaler", 2024-05-09T05:01:48.702 INFO:teuthology.orchestra.run.smithi046.stdout: "progress", 2024-05-09T05:01:48.702 INFO:teuthology.orchestra.run.smithi046.stdout: "rbd_support", 2024-05-09T05:01:48.702 INFO:teuthology.orchestra.run.smithi046.stdout: "status", 2024-05-09T05:01:48.702 INFO:teuthology.orchestra.run.smithi046.stdout: "telemetry", 2024-05-09T05:01:48.702 INFO:teuthology.orchestra.run.smithi046.stdout: "volumes" 2024-05-09T05:01:48.702 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T05:01:48.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.705 INFO:teuthology.orchestra.run.smithi046.stdout: "last_failure_osd_epoch": 85, 2024-05-09T05:01:48.705 INFO:teuthology.orchestra.run.smithi046.stdout: "active_clients": [ 2024-05-09T05:01:48.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.705 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "devicehealth", 2024-05-09T05:01:48.705 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-09T05:01:48.705 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.705 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-09T05:01:48.706 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-09T05:01:48.706 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2361387445 2024-05-09T05:01:48.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.706 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T05:01:48.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.706 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "libcephsqlite", 2024-05-09T05:01:48.706 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-09T05:01:48.706 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.706 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-09T05:01:48.706 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-09T05:01:48.706 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1764967317 2024-05-09T05:01:48.706 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.706 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T05:01:48.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.707 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "rbd_support", 2024-05-09T05:01:48.707 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-09T05:01:48.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.707 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-09T05:01:48.707 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-09T05:01:48.707 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2544743482 2024-05-09T05:01:48.707 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.707 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T05:01:48.707 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.707 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.707 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-05-09T05:01:48.707 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-09T05:01:48.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-09T05:01:48.708 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-09T05:01:48.708 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 4120326011 2024-05-09T05:01:48.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.708 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T05:01:48.708 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.708 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "snap_schedule", 2024-05-09T05:01:48.708 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-09T05:01:48.708 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.708 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-09T05:01:48.708 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-09T05:01:48.708 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 1172308860 2024-05-09T05:01:48.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.709 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T05:01:48.709 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:48.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.709 INFO:teuthology.orchestra.run.smithi046.stdout: "name": "volumes", 2024-05-09T05:01:48.709 INFO:teuthology.orchestra.run.smithi046.stdout: "addrvec": [ 2024-05-09T05:01:48.709 INFO:teuthology.orchestra.run.smithi046.stdout: { 2024-05-09T05:01:48.709 INFO:teuthology.orchestra.run.smithi046.stdout: "type": "v2", 2024-05-09T05:01:48.709 INFO:teuthology.orchestra.run.smithi046.stdout: "addr": "172.21.15.46:0", 2024-05-09T05:01:48.709 INFO:teuthology.orchestra.run.smithi046.stdout: "nonce": 2464271865 2024-05-09T05:01:48.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.709 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T05:01:48.709 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:48.709 INFO:teuthology.orchestra.run.smithi046.stdout: ] 2024-05-09T05:01:48.709 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T05:01:48.711 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:47 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/1250331018' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T05:01:48.711 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:46 smithi110 ceph-mon[34904]: pgmap v331: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:48.711 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:46 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3538324944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T05:01:48.711 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:46 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/179832834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T05:01:48.711 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:46 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2339737838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T05:01:48.711 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:47 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/1250331018' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T05:01:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:47.242+0000 7fdf8f7be640 1 -- 172.21.15.46:0/1250331018 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdf740b2960 msgr2=0x7fdf740b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:47.242+0000 7fdf8f7be640 1 --2- 172.21.15.46:0/1250331018 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdf740b2960 0x7fdf740b4e70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fdf84000c00 tx=0x7fdf84031400 comp rx=0 tx=0).stop 2024-05-09T05:01:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:47.242+0000 7fdf8f7be640 1 -- 172.21.15.46:0/1250331018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf88109c60 msgr2=0x7fdf881a0b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:01:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:47.242+0000 7fdf8f7be640 1 --2- 172.21.15.46:0/1250331018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf88109c60 0x7fdf881a0b30 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fdf78006fd0 tx=0x7fdf7800b8e0 comp rx=0 tx=0).stop 2024-05-09T05:01:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:47.242+0000 7fdf8f7be640 1 -- 172.21.15.46:0/1250331018 shutdown_connections 2024-05-09T05:01:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:47.242+0000 7fdf8f7be640 1 --2- 172.21.15.46:0/1250331018 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fdf740b2960 0x7fdf740b4e70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:47.242+0000 7fdf8f7be640 1 --2- 172.21.15.46:0/1250331018 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fdf8810a100 0x7fdf8819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:47.242+0000 7fdf8f7be640 1 --2- 172.21.15.46:0/1250331018 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fdf88109c60 0x7fdf881a0b30 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:47.242+0000 7fdf8f7be640 1 --2- 172.21.15.46:0/1250331018 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fdf881043b0 0x7fdf881a05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:01:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:47.242+0000 7fdf8f7be640 1 -- 172.21.15.46:0/1250331018 >> 172.21.15.46:0/1250331018 conn(0x7fdf880ff340 msgr2=0x7fdf880ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:01:48.712 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:47.242+0000 7fdf8f7be640 1 -- 172.21.15.46:0/1250331018 shutdown_connections 2024-05-09T05:01:48.713 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:01:47.242+0000 7fdf8f7be640 1 -- 172.21.15.46:0/1250331018 wait complete. 2024-05-09T05:01:48.713 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:46 smithi046 ceph-mon[30136]: pgmap v331: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:48.713 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:46 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3538324944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-09T05:01:48.713 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:46 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/179832834' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-09T05:01:48.713 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:46 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2339737838' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-09T05:01:48.762 DEBUG:tasks.check_counter:Getting stats from a 2024-05-09T05:01:48.762 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-09T05:01:48.763 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:47 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/1250331018' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-09T05:01:49.027 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:01:49.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:48 smithi046 ceph-mon[30136]: pgmap v332: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:01:49.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:48 smithi072 ceph-mon[34088]: pgmap v332: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:01:49.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:48 smithi110 ceph-mon[34904]: pgmap v332: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:01:50.836 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T05:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T05:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 4, 2024-05-09T05:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 4, 2024-05-09T05:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 2992, 2024-05-09T05:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 448, 2024-05-09T05:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-05-09T05:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-05-09T05:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.001853849, 2024-05-09T05:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000038808, 2024-05-09T05:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000767945, 2024-05-09T05:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T05:01:50.837 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:01:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 3, 2024-05-09T05:01:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000988411, 2024-05-09T05:01:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000329470 2024-05-09T05:01:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:01:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-09T05:01:50.838 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T05:01:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T05:01:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2, 2024-05-09T05:01:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 175, 2024-05-09T05:01:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 320, 2024-05-09T05:01:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 188160, 2024-05-09T05:01:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 5, 2024-05-09T05:01:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-05-09T05:01:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.014208539, 2024-05-09T05:01:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.009737768, 2024-05-09T05:01:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.002125365, 2024-05-09T05:01:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T05:01:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:01:50.839 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 172, 2024-05-09T05:01:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.009213712, 2024-05-09T05:01:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000053568 2024-05-09T05:01:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:01:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T05:01:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 188160 2024-05-09T05:01:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:50.840 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T05:01:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 240, 2024-05-09T05:01:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 229, 2024-05-09T05:01:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1715680, 2024-05-09T05:01:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 141344, 2024-05-09T05:01:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-05-09T05:01:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-05-09T05:01:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.051736799, 2024-05-09T05:01:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.015297929, 2024-05-09T05:01:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.043669326, 2024-05-09T05:01:50.841 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.004661277, 2024-05-09T05:01:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:01:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 228, 2024-05-09T05:01:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.012127721, 2024-05-09T05:01:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000053191 2024-05-09T05:01:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:01:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1715680, 2024-05-09T05:01:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 141344 2024-05-09T05:01:50.842 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-05-09T05:01:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-05-09T05:01:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-05-09T05:01:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-05-09T05:01:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-05-09T05:01:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-05-09T05:01:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T05:01:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-05-09T05:01:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T05:01:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-05-09T05:01:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T05:01:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T05:01:50.843 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T05:01:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T05:01:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T05:01:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-05-09T05:01:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T05:01:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T05:01:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-05-09T05:01:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-05-09T05:01:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-05-09T05:01:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-05-09T05:01:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T05:01:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T05:01:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T05:01:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T05:01:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T05:01:50.844 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-05-09T05:01:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-05-09T05:01:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-05-09T05:01:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T05:01:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T05:01:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-05-09T05:01:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-05-09T05:01:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-05-09T05:01:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-05-09T05:01:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T05:01:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-05-09T05:01:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T05:01:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-05-09T05:01:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 148714, 2024-05-09T05:01:50.845 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 30, 2024-05-09T05:01:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-05-09T05:01:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-05-09T05:01:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-05-09T05:01:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-05-09T05:01:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-05-09T05:01:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-05-09T05:01:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-05-09T05:01:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-05-09T05:01:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-05-09T05:01:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-05-09T05:01:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T05:01:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-05-09T05:01:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-05-09T05:01:50.846 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-05-09T05:01:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-05-09T05:01:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-05-09T05:01:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-05-09T05:01:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-05-09T05:01:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-05-09T05:01:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-05-09T05:01:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T05:01:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-09T05:01:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-05-09T05:01:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-09T05:01:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 242, 2024-05-09T05:01:50.847 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1681809, 2024-05-09T05:01:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-09T05:01:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 242, 2024-05-09T05:01:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-09T05:01:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-09T05:01:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 242, 2024-05-09T05:01:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1681809, 2024-05-09T05:01:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-09T05:01:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:50.848 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:50.849 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:50.849 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T05:01:51.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:50 smithi046 ceph-mon[30136]: pgmap v333: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:51.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:50 smithi072 ceph-mon[34088]: pgmap v333: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:51.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:50 smithi110 ceph-mon[34904]: pgmap v333: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:51.631 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-05-09T05:01:51.632 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-05-09T05:01:51.632 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-05-09T05:01:51.632 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-05-09T05:01:51.632 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-05-09T05:01:51.632 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-05-09T05:01:51.632 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-05-09T05:01:51.632 DEBUG:tasks.check_counter:Getting stats from d 2024-05-09T05:01:51.632 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-09T05:01:51.892 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:01:52.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:51 smithi072 ceph-mon[34088]: pgmap v334: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T05:01:52.166 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:51 smithi046 ceph-mon[30136]: pgmap v334: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T05:01:52.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:51 smithi110 ceph-mon[34904]: pgmap v334: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-05-09T05:01:53.696 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T05:01:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T05:01:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-05-09T05:01:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-05-09T05:01:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-05-09T05:01:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-05-09T05:01:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-05-09T05:01:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-05-09T05:01:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.001417667, 2024-05-09T05:01:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T05:01:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000350712, 2024-05-09T05:01:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T05:01:53.697 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:01:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:01:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T05:01:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T05:01:53.698 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T05:01:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 251, 2024-05-09T05:01:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 238, 2024-05-09T05:01:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1720640, 2024-05-09T05:01:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 145792, 2024-05-09T05:01:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-05-09T05:01:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-05-09T05:01:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.055133349, 2024-05-09T05:01:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.015614319, 2024-05-09T05:01:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.048145898, 2024-05-09T05:01:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.004906665, 2024-05-09T05:01:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:01:53.699 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 238, 2024-05-09T05:01:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.012102010, 2024-05-09T05:01:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000050848 2024-05-09T05:01:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:01:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1720640, 2024-05-09T05:01:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 145792 2024-05-09T05:01:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T05:01:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2, 2024-05-09T05:01:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 181, 2024-05-09T05:01:53.700 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 320, 2024-05-09T05:01:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 189216, 2024-05-09T05:01:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 5, 2024-05-09T05:01:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-05-09T05:01:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.015167465, 2024-05-09T05:01:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.010484561, 2024-05-09T05:01:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.002035059, 2024-05-09T05:01:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T05:01:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:01:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 178, 2024-05-09T05:01:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.012942175, 2024-05-09T05:01:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000072708 2024-05-09T05:01:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:01:53.701 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T05:01:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 189216 2024-05-09T05:01:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-05-09T05:01:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-05-09T05:01:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-05-09T05:01:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-05-09T05:01:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-05-09T05:01:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-05-09T05:01:53.702 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T05:01:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-05-09T05:01:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T05:01:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-05-09T05:01:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T05:01:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T05:01:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T05:01:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T05:01:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T05:01:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-05-09T05:01:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T05:01:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T05:01:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-05-09T05:01:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-05-09T05:01:53.703 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-05-09T05:01:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-05-09T05:01:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T05:01:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T05:01:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T05:01:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T05:01:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T05:01:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-05-09T05:01:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-05-09T05:01:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-05-09T05:01:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T05:01:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T05:01:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-05-09T05:01:53.704 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-05-09T05:01:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-05-09T05:01:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-05-09T05:01:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T05:01:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-05-09T05:01:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T05:01:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-05-09T05:01:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 148754, 2024-05-09T05:01:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 31, 2024-05-09T05:01:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-05-09T05:01:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-05-09T05:01:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-05-09T05:01:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-05-09T05:01:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-05-09T05:01:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-05-09T05:01:53.705 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-05-09T05:01:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-05-09T05:01:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-05-09T05:01:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-05-09T05:01:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T05:01:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-05-09T05:01:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-05-09T05:01:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-05-09T05:01:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-05-09T05:01:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-05-09T05:01:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-05-09T05:01:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-05-09T05:01:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-05-09T05:01:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-05-09T05:01:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:53.706 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T05:01:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-09T05:01:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-05-09T05:01:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-09T05:01:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 249, 2024-05-09T05:01:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1682783, 2024-05-09T05:01:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-09T05:01:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 249, 2024-05-09T05:01:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-09T05:01:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-09T05:01:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 249, 2024-05-09T05:01:53.707 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1682783, 2024-05-09T05:01:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-09T05:01:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:53.708 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:53.708 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T05:01:54.443 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-09T05:01:54.443 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-09T05:01:54.443 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-05-09T05:01:54.443 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-05-09T05:01:54.443 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-09T05:01:54.443 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-09T05:01:54.443 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-09T05:01:54.443 DEBUG:tasks.check_counter:Getting stats from g 2024-05-09T05:01:54.444 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-09T05:01:54.697 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:01:55.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:54 smithi046 ceph-mon[30136]: pgmap v335: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T05:01:55.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:54 smithi072 ceph-mon[34088]: pgmap v335: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T05:01:55.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:54 smithi110 ceph-mon[34904]: pgmap v335: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-05-09T05:01:56.497 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T05:01:56.497 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T05:01:56.497 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 242, 2024-05-09T05:01:56.497 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 231, 2024-05-09T05:01:56.497 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1716224, 2024-05-09T05:01:56.498 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 142624, 2024-05-09T05:01:56.498 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-05-09T05:01:56.498 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-05-09T05:01:56.498 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.053025810, 2024-05-09T05:01:56.498 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.014516645, 2024-05-09T05:01:56.498 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.046030841, 2024-05-09T05:01:56.498 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.004766548, 2024-05-09T05:01:56.498 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:01:56.498 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 230, 2024-05-09T05:01:56.498 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.010605141, 2024-05-09T05:01:56.498 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000046109 2024-05-09T05:01:56.498 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:56.498 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:01:56.499 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:56.499 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:56.499 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:56.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:56.499 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1716224, 2024-05-09T05:01:56.499 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 142624 2024-05-09T05:01:56.499 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:56.499 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T05:01:56.499 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 6, 2024-05-09T05:01:56.499 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 179, 2024-05-09T05:01:56.499 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 3312, 2024-05-09T05:01:56.499 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 188960, 2024-05-09T05:01:56.500 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 5, 2024-05-09T05:01:56.500 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-05-09T05:01:56.500 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.015064521, 2024-05-09T05:01:56.500 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.010300186, 2024-05-09T05:01:56.500 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.002409804, 2024-05-09T05:01:56.500 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T05:01:56.500 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:01:56.500 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 177, 2024-05-09T05:01:56.500 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.010389365, 2024-05-09T05:01:56.500 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000058696 2024-05-09T05:01:56.500 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:56.500 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:01:56.501 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:56.501 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:56.501 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:56.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:56.501 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-09T05:01:56.501 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 188960 2024-05-09T05:01:56.501 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:56.501 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T05:01:56.501 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 0, 2024-05-09T05:01:56.501 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 2, 2024-05-09T05:01:56.501 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 0, 2024-05-09T05:01:56.501 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 0, 2024-05-09T05:01:56.501 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-05-09T05:01:56.501 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-05-09T05:01:56.502 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.000411547, 2024-05-09T05:01:56.502 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T05:01:56.502 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000333971, 2024-05-09T05:01:56.502 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T05:01:56.502 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:01:56.502 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:56.502 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:56.502 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:56.502 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:56.502 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:01:56.502 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:56.502 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:56.503 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:56.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:56.503 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T05:01:56.503 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T05:01:56.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:56.503 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-05-09T05:01:56.503 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-05-09T05:01:56.503 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-05-09T05:01:56.503 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:56.503 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-05-09T05:01:56.503 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-05-09T05:01:56.503 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-05-09T05:01:56.503 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T05:01:56.504 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-05-09T05:01:56.504 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T05:01:56.504 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-05-09T05:01:56.504 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T05:01:56.504 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T05:01:56.504 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T05:01:56.504 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T05:01:56.504 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T05:01:56.504 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-05-09T05:01:56.504 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T05:01:56.504 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T05:01:56.504 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-05-09T05:01:56.504 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-05-09T05:01:56.504 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-05-09T05:01:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-05-09T05:01:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T05:01:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T05:01:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T05:01:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T05:01:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T05:01:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-05-09T05:01:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-05-09T05:01:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-05-09T05:01:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T05:01:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T05:01:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-05-09T05:01:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-05-09T05:01:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-05-09T05:01:56.505 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-05-09T05:01:56.506 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T05:01:56.506 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-05-09T05:01:56.506 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T05:01:56.506 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-05-09T05:01:56.506 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 148714, 2024-05-09T05:01:56.506 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 30, 2024-05-09T05:01:56.506 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-05-09T05:01:56.506 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-05-09T05:01:56.506 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-05-09T05:01:56.506 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-05-09T05:01:56.506 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-05-09T05:01:56.506 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-05-09T05:01:56.506 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-05-09T05:01:56.506 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-05-09T05:01:56.507 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-05-09T05:01:56.507 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-05-09T05:01:56.507 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T05:01:56.507 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-05-09T05:01:56.507 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-05-09T05:01:56.507 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-05-09T05:01:56.507 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-05-09T05:01:56.507 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-05-09T05:01:56.507 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-05-09T05:01:56.507 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-05-09T05:01:56.507 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-05-09T05:01:56.507 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-05-09T05:01:56.507 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:56.507 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T05:01:56.508 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-09T05:01:56.508 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-05-09T05:01:56.508 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-09T05:01:56.508 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 244, 2024-05-09T05:01:56.508 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1682069, 2024-05-09T05:01:56.508 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-09T05:01:56.508 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 244, 2024-05-09T05:01:56.508 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-09T05:01:56.508 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-09T05:01:56.508 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 244, 2024-05-09T05:01:56.508 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1682069, 2024-05-09T05:01:56.508 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-09T05:01:56.508 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:56.509 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:56.509 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:56.509 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:56.509 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:56.509 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T05:01:57.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:56 smithi072 ceph-mon[34088]: pgmap v336: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:57.229 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:56 smithi046 ceph-mon[30136]: pgmap v336: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:57.260 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-05-09T05:01:57.260 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-05-09T05:01:57.260 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-05-09T05:01:57.260 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-05-09T05:01:57.260 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-05-09T05:01:57.260 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-05-09T05:01:57.260 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-05-09T05:01:57.261 DEBUG:tasks.check_counter:Getting stats from j 2024-05-09T05:01:57.261 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-09T05:01:57.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:56 smithi110 ceph-mon[34904]: pgmap v336: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:01:57.520 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:01:58.995 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:01:58 smithi046 ceph-mon[30136]: pgmap v337: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:01:59.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:01:58 smithi072 ceph-mon[34088]: pgmap v337: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:01:59.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:01:58 smithi110 ceph-mon[34904]: pgmap v337: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2024-05-09T05:01:59.439 INFO:teuthology.orchestra.run.smithi046.stdout:{ 2024-05-09T05:01:59.439 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T05:01:59.439 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 250, 2024-05-09T05:01:59.439 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 239, 2024-05-09T05:01:59.439 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 1718464, 2024-05-09T05:01:59.439 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 147264, 2024-05-09T05:01:59.440 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 4, 2024-05-09T05:01:59.440 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 0, 2024-05-09T05:01:59.440 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.052022836, 2024-05-09T05:01:59.440 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.015469450, 2024-05-09T05:01:59.440 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.044779225, 2024-05-09T05:01:59.440 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.004692431, 2024-05-09T05:01:59.440 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:01:59.440 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 238, 2024-05-09T05:01:59.440 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.011401240, 2024-05-09T05:01:59.440 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000047904 2024-05-09T05:01:59.440 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:59.440 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:01:59.440 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:59.440 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:59.440 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:59.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:59.441 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 1718464, 2024-05-09T05:01:59.441 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 147264 2024-05-09T05:01:59.441 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:59.441 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T05:01:59.441 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 2, 2024-05-09T05:01:59.441 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 182, 2024-05-09T05:01:59.441 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 320, 2024-05-09T05:01:59.441 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 189392, 2024-05-09T05:01:59.441 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 5, 2024-05-09T05:01:59.441 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 3, 2024-05-09T05:01:59.441 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.016631761, 2024-05-09T05:01:59.441 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.011633161, 2024-05-09T05:01:59.441 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.002136165, 2024-05-09T05:01:59.442 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T05:01:59.442 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:01:59.442 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 179, 2024-05-09T05:01:59.442 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.011016251, 2024-05-09T05:01:59.442 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000061543 2024-05-09T05:01:59.442 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:59.442 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:01:59.442 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:59.442 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:59.442 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:59.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:59.443 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T05:01:59.443 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 189392 2024-05-09T05:01:59.443 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:59.443 INFO:teuthology.orchestra.run.smithi046.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T05:01:59.443 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_messages": 4, 2024-05-09T05:01:59.443 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages": 4, 2024-05-09T05:01:59.443 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_bytes": 2992, 2024-05-09T05:01:59.443 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_bytes": 448, 2024-05-09T05:01:59.443 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_created_connections": 2, 2024-05-09T05:01:59.443 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_active_connections": 1, 2024-05-09T05:01:59.443 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_total_time": 0.000749662, 2024-05-09T05:01:59.443 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_send_time": 0.000043915, 2024-05-09T05:01:59.443 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_recv_time": 0.000684590, 2024-05-09T05:01:59.443 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T05:01:59.444 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:01:59.444 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 3, 2024-05-09T05:01:59.444 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000782481, 2024-05-09T05:01:59.444 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000260827 2024-05-09T05:01:59.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:59.444 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:01:59.444 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:59.444 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:59.444 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:59.444 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:59.444 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-09T05:01:59.445 INFO:teuthology.orchestra.run.smithi046.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T05:01:59.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:59.445 INFO:teuthology.orchestra.run.smithi046.stdout: "cct": { 2024-05-09T05:01:59.445 INFO:teuthology.orchestra.run.smithi046.stdout: "total_workers": 0, 2024-05-09T05:01:59.445 INFO:teuthology.orchestra.run.smithi046.stdout: "unhealthy_workers": 0 2024-05-09T05:01:59.445 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:59.445 INFO:teuthology.orchestra.run.smithi046.stdout: "mempool": { 2024-05-09T05:01:59.445 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_bytes": 0, 2024-05-09T05:01:59.445 INFO:teuthology.orchestra.run.smithi046.stdout: "bloom_filter_items": 0, 2024-05-09T05:01:59.445 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T05:01:59.445 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_alloc_items": 0, 2024-05-09T05:01:59.445 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T05:01:59.445 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_data_items": 0, 2024-05-09T05:01:59.445 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T05:01:59.446 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T05:01:59.446 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T05:01:59.446 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T05:01:59.446 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T05:01:59.446 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_other_items": 0, 2024-05-09T05:01:59.446 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T05:01:59.446 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T05:01:59.446 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_bytes": 0, 2024-05-09T05:01:59.446 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_extent_items": 0, 2024-05-09T05:01:59.446 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_bytes": 0, 2024-05-09T05:01:59.446 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_blob_items": 0, 2024-05-09T05:01:59.446 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T05:01:59.446 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T05:01:59.446 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T05:01:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T05:01:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T05:01:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_fsck_items": 0, 2024-05-09T05:01:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_bytes": 0, 2024-05-09T05:01:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_txc_items": 0, 2024-05-09T05:01:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T05:01:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T05:01:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_bytes": 0, 2024-05-09T05:01:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: "bluestore_writing_items": 0, 2024-05-09T05:01:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_bytes": 0, 2024-05-09T05:01:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_items": 0, 2024-05-09T05:01:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T05:01:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_reader_items": 0, 2024-05-09T05:01:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T05:01:59.447 INFO:teuthology.orchestra.run.smithi046.stdout: "bluefs_file_writer_items": 0, 2024-05-09T05:01:59.448 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_bytes": 148731, 2024-05-09T05:01:59.448 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_anon_items": 31, 2024-05-09T05:01:59.448 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_bytes": 0, 2024-05-09T05:01:59.448 INFO:teuthology.orchestra.run.smithi046.stdout: "buffer_meta_items": 0, 2024-05-09T05:01:59.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_bytes": 0, 2024-05-09T05:01:59.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_items": 0, 2024-05-09T05:01:59.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_bytes": 0, 2024-05-09T05:01:59.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_mapbl_items": 0, 2024-05-09T05:01:59.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_bytes": 0, 2024-05-09T05:01:59.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osd_pglog_items": 0, 2024-05-09T05:01:59.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_bytes": 0, 2024-05-09T05:01:59.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_items": 0, 2024-05-09T05:01:59.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T05:01:59.448 INFO:teuthology.orchestra.run.smithi046.stdout: "osdmap_mapping_items": 0, 2024-05-09T05:01:59.448 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_bytes": 0, 2024-05-09T05:01:59.449 INFO:teuthology.orchestra.run.smithi046.stdout: "pgmap_items": 0, 2024-05-09T05:01:59.449 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_bytes": 928, 2024-05-09T05:01:59.449 INFO:teuthology.orchestra.run.smithi046.stdout: "mds_co_items": 2, 2024-05-09T05:01:59.449 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_bytes": 0, 2024-05-09T05:01:59.449 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_1_items": 0, 2024-05-09T05:01:59.449 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_bytes": 0, 2024-05-09T05:01:59.449 INFO:teuthology.orchestra.run.smithi046.stdout: "unittest_2_items": 0 2024-05-09T05:01:59.449 INFO:teuthology.orchestra.run.smithi046.stdout: }, 2024-05-09T05:01:59.449 INFO:teuthology.orchestra.run.smithi046.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T05:01:59.449 INFO:teuthology.orchestra.run.smithi046.stdout: "val": 0, 2024-05-09T05:01:59.449 INFO:teuthology.orchestra.run.smithi046.stdout: "max": 104857600, 2024-05-09T05:01:59.450 INFO:teuthology.orchestra.run.smithi046.stdout: "get_started": 0, 2024-05-09T05:01:59.450 INFO:teuthology.orchestra.run.smithi046.stdout: "get": 252, 2024-05-09T05:01:59.450 INFO:teuthology.orchestra.run.smithi046.stdout: "get_sum": 1683173, 2024-05-09T05:01:59.450 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_fail": 0, 2024-05-09T05:01:59.450 INFO:teuthology.orchestra.run.smithi046.stdout: "get_or_fail_success": 252, 2024-05-09T05:01:59.450 INFO:teuthology.orchestra.run.smithi046.stdout: "take": 0, 2024-05-09T05:01:59.450 INFO:teuthology.orchestra.run.smithi046.stdout: "take_sum": 0, 2024-05-09T05:01:59.450 INFO:teuthology.orchestra.run.smithi046.stdout: "put": 252, 2024-05-09T05:01:59.450 INFO:teuthology.orchestra.run.smithi046.stdout: "put_sum": 1683173, 2024-05-09T05:01:59.450 INFO:teuthology.orchestra.run.smithi046.stdout: "wait": { 2024-05-09T05:01:59.450 INFO:teuthology.orchestra.run.smithi046.stdout: "avgcount": 0, 2024-05-09T05:01:59.450 INFO:teuthology.orchestra.run.smithi046.stdout: "sum": 0.000000000, 2024-05-09T05:01:59.450 INFO:teuthology.orchestra.run.smithi046.stdout: "avgtime": 0.000000000 2024-05-09T05:01:59.450 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:59.451 INFO:teuthology.orchestra.run.smithi046.stdout: } 2024-05-09T05:01:59.451 INFO:teuthology.orchestra.run.smithi046.stdout:} 2024-05-09T05:02:00.097 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-09T05:02:00.097 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-09T05:02:00.097 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-05-09T05:02:00.097 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-05-09T05:02:00.097 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-09T05:02:00.097 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-09T05:02:00.097 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-09T05:02:00.098 DEBUG:tasks.check_counter:Getting stats from b 2024-05-09T05:02:00.098 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-09T05:02:00.350 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T05:02:01.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:00 smithi072 ceph-mon[34088]: pgmap v338: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:02:01.231 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:00 smithi046 ceph-mon[30136]: pgmap v338: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:02:01.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:00 smithi110 ceph-mon[34904]: pgmap v338: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-05-09T05:02:02.363 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-05-09T05:02:02.363 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T05:02:02.363 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 5729, 2024-05-09T05:02:02.363 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 5489, 2024-05-09T05:02:02.363 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 1670090, 2024-05-09T05:02:02.363 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 11901157, 2024-05-09T05:02:02.363 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 159, 2024-05-09T05:02:02.363 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 4, 2024-05-09T05:02:02.364 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.562252627, 2024-05-09T05:02:02.364 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.170038023, 2024-05-09T05:02:02.364 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.199038982, 2024-05-09T05:02:02.364 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.150543643, 2024-05-09T05:02:02.364 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:02.364 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 5487, 2024-05-09T05:02:02.364 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.531388937, 2024-05-09T05:02:02.364 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000096845 2024-05-09T05:02:02.364 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.364 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:02.364 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1283, 2024-05-09T05:02:02.364 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000338640, 2024-05-09T05:02:02.364 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000263 2024-05-09T05:02:02.365 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.365 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 42528, 2024-05-09T05:02:02.365 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 1136 2024-05-09T05:02:02.365 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.365 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T05:02:02.365 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 8179, 2024-05-09T05:02:02.365 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 9194, 2024-05-09T05:02:02.365 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 2616797, 2024-05-09T05:02:02.365 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 6404167, 2024-05-09T05:02:02.365 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 16, 2024-05-09T05:02:02.365 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 6, 2024-05-09T05:02:02.365 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.647255334, 2024-05-09T05:02:02.365 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.208150321, 2024-05-09T05:02:02.365 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.417272032, 2024-05-09T05:02:02.365 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.145731378, 2024-05-09T05:02:02.366 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:02.366 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 9192, 2024-05-09T05:02:02.366 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 1.646576457, 2024-05-09T05:02:02.366 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000179131 2024-05-09T05:02:02.366 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.366 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:02.366 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 9189, 2024-05-09T05:02:02.366 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.003595802, 2024-05-09T05:02:02.366 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000391 2024-05-09T05:02:02.366 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.366 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 109632, 2024-05-09T05:02:02.366 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 517664 2024-05-09T05:02:02.366 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.366 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T05:02:02.366 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 3444, 2024-05-09T05:02:02.367 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 2696, 2024-05-09T05:02:02.367 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 2882890, 2024-05-09T05:02:02.367 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 1166797, 2024-05-09T05:02:02.367 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 20, 2024-05-09T05:02:02.367 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 7, 2024-05-09T05:02:02.367 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.353099080, 2024-05-09T05:02:02.367 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.104649250, 2024-05-09T05:02:02.367 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.173487645, 2024-05-09T05:02:02.367 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.092479726, 2024-05-09T05:02:02.367 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:02.367 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2695, 2024-05-09T05:02:02.367 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.455166688, 2024-05-09T05:02:02.367 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000168893 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1940, 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000513731, 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000264 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 2094176, 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 272400 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 1, 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-MDSRank": { 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-05-09T05:02:02.368 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-05-09T05:02:02.369 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 5663, 2024-05-09T05:02:02.369 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 2.797309019, 2024-05-09T05:02:02.369 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000493962 2024-05-09T05:02:02.369 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:02.369 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.369 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-PurgeQueue": { 2024-05-09T05:02:02.369 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-05-09T05:02:02.369 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-05-09T05:02:02.369 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 4405, 2024-05-09T05:02:02.369 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.140671883, 2024-05-09T05:02:02.369 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000031934 2024-05-09T05:02:02.369 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:02.369 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.369 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": { 2024-05-09T05:02:02.369 INFO:teuthology.orchestra.run.smithi072.stdout: "request": 172, 2024-05-09T05:02:02.370 INFO:teuthology.orchestra.run.smithi072.stdout: "reply": 172, 2024-05-09T05:02:02.370 INFO:teuthology.orchestra.run.smithi072.stdout: "reply_latency": { 2024-05-09T05:02:02.370 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 172, 2024-05-09T05:02:02.370 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.185211837, 2024-05-09T05:02:02.370 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.001076813 2024-05-09T05:02:02.370 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.370 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_reply": 0, 2024-05-09T05:02:02.370 INFO:teuthology.orchestra.run.smithi072.stdout: "forward": 0, 2024-05-09T05:02:02.370 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_complete": 0, 2024-05-09T05:02:02.370 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_keys": 0, 2024-05-09T05:02:02.370 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_commit": 76, 2024-05-09T05:02:02.370 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_split": 0, 2024-05-09T05:02:02.370 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_merge": 0, 2024-05-09T05:02:02.370 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes": 33, 2024-05-09T05:02:02.371 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_top": 8, 2024-05-09T05:02:02.371 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_bottom": 25, 2024-05-09T05:02:02.371 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pin_tail": 0, 2024-05-09T05:02:02.371 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pinned": 21, 2024-05-09T05:02:02.371 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_expired": 0, 2024-05-09T05:02:02.371 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_with_caps": 2, 2024-05-09T05:02:02.371 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": 3, 2024-05-09T05:02:02.371 INFO:teuthology.orchestra.run.smithi072.stdout: "subtrees": 2, 2024-05-09T05:02:02.371 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse": 229, 2024-05-09T05:02:02.371 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_hit": 219, 2024-05-09T05:02:02.371 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_forward": 0, 2024-05-09T05:02:02.371 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_discover": 0, 2024-05-09T05:02:02.371 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_dir_fetch": 0, 2024-05-09T05:02:02.371 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_remote_ino": 0, 2024-05-09T05:02:02.372 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_lock": 0, 2024-05-09T05:02:02.372 INFO:teuthology.orchestra.run.smithi072.stdout: "load_cent": 0, 2024-05-09T05:02:02.372 INFO:teuthology.orchestra.run.smithi072.stdout: "q": 0, 2024-05-09T05:02:02.372 INFO:teuthology.orchestra.run.smithi072.stdout: "exported": 0, 2024-05-09T05:02:02.372 INFO:teuthology.orchestra.run.smithi072.stdout: "exported_inodes": 0, 2024-05-09T05:02:02.372 INFO:teuthology.orchestra.run.smithi072.stdout: "imported": 0, 2024-05-09T05:02:02.372 INFO:teuthology.orchestra.run.smithi072.stdout: "imported_inodes": 0, 2024-05-09T05:02:02.372 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_dir_fetch": 0, 2024-05-09T05:02:02.372 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_backtrace_fetch": 234, 2024-05-09T05:02:02.372 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_peer_discover": 0, 2024-05-09T05:02:02.372 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rfiles": 4, 2024-05-09T05:02:02.372 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rbytes": 234, 2024-05-09T05:02:02.372 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rsnaps": 6, 2024-05-09T05:02:02.372 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_fetch": 234, 2024-05-09T05:02:02.372 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_set_tag": 0, 2024-05-09T05:02:02.373 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T05:02:02.373 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_inotable_repaired": 0, 2024-05-09T05:02:02.373 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_inodes": 150, 2024-05-09T05:02:02.373 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_base_inodes": 20, 2024-05-09T05:02:02.373 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dirfrag_rstats": 130, 2024-05-09T05:02:02.373 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_file_inodes": 84, 2024-05-09T05:02:02.373 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_inode_file_caps": 0, 2024-05-09T05:02:02.373 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_revoke": 12, 2024-05-09T05:02:02.373 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_grant": 1648, 2024-05-09T05:02:02.373 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T05:02:02.373 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T05:02:02.373 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T05:02:02.373 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps": 4064, 2024-05-09T05:02:02.373 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps_dirty": 4048, 2024-05-09T05:02:02.373 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_cap_release": 5, 2024-05-09T05:02:02.374 INFO:teuthology.orchestra.run.smithi072.stdout: "process_request_cap_release": 25 2024-05-09T05:02:02.374 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.374 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_cache": { 2024-05-09T05:02:02.374 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update": 6, 2024-05-09T05:02:02.374 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update_receipt": 0, 2024-05-09T05:02:02.374 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_try_discover": 0, 2024-05-09T05:02:02.374 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_send_discover": 0, 2024-05-09T05:02:02.374 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_handle_discover": 12, 2024-05-09T05:02:02.374 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays": 2, 2024-05-09T05:02:02.374 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_delayed": 0, 2024-05-09T05:02:02.374 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_enqueuing": 0, 2024-05-09T05:02:02.374 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_created": 5, 2024-05-09T05:02:02.374 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_enqueued": 3, 2024-05-09T05:02:02.374 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_reintegrated": 0, 2024-05-09T05:02:02.375 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_migrated": 0, 2024-05-09T05:02:02.375 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_processing": 0, 2024-05-09T05:02:02.375 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_enqueued": 0, 2024-05-09T05:02:02.375 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_prioritized": 0, 2024-05-09T05:02:02.375 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_started": 0, 2024-05-09T05:02:02.375 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_completed": 0, 2024-05-09T05:02:02.375 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_path": 0, 2024-05-09T05:02:02.375 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_inode": 0, 2024-05-09T05:02:02.375 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_enqueue_scrub": 20, 2024-05-09T05:02:02.375 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_exportdir": 0, 2024-05-09T05:02:02.375 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_flush": 0, 2024-05-09T05:02:02.375 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragmentdir": 0, 2024-05-09T05:02:02.375 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragstats": 0, 2024-05-09T05:02:02.375 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_inodestats": 0 2024-05-09T05:02:02.375 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_log": { 2024-05-09T05:02:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: "evlrg": 0, 2024-05-09T05:02:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: "evadd": 4197, 2024-05-09T05:02:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: "evex": 4136, 2024-05-09T05:02:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: "evtrm": 4136, 2024-05-09T05:02:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: "ev": 61, 2024-05-09T05:02:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: "evexg": 0, 2024-05-09T05:02:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: "evexd": 8, 2024-05-09T05:02:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: "segadd": 18, 2024-05-09T05:02:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: "segex": 17, 2024-05-09T05:02:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: "segtrm": 17, 2024-05-09T05:02:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: "seg": 1, 2024-05-09T05:02:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: "segmjr": 1, 2024-05-09T05:02:02.376 INFO:teuthology.orchestra.run.smithi072.stdout: "segexg": 0, 2024-05-09T05:02:02.377 INFO:teuthology.orchestra.run.smithi072.stdout: "segexd": 1, 2024-05-09T05:02:02.377 INFO:teuthology.orchestra.run.smithi072.stdout: "expos": 15255912, 2024-05-09T05:02:02.377 INFO:teuthology.orchestra.run.smithi072.stdout: "wrpos": 15324584, 2024-05-09T05:02:02.377 INFO:teuthology.orchestra.run.smithi072.stdout: "rdpos": 0, 2024-05-09T05:02:02.377 INFO:teuthology.orchestra.run.smithi072.stdout: "jlat": { 2024-05-09T05:02:02.377 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 4167, 2024-05-09T05:02:02.377 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 71.753143546, 2024-05-09T05:02:02.377 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.017219376 2024-05-09T05:02:02.377 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.377 INFO:teuthology.orchestra.run.smithi072.stdout: "replayed": 0 2024-05-09T05:02:02.377 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.377 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_mem": { 2024-05-09T05:02:02.377 INFO:teuthology.orchestra.run.smithi072.stdout: "ino": 27, 2024-05-09T05:02:02.377 INFO:teuthology.orchestra.run.smithi072.stdout: "ino+": 50, 2024-05-09T05:02:02.377 INFO:teuthology.orchestra.run.smithi072.stdout: "ino-": 23, 2024-05-09T05:02:02.378 INFO:teuthology.orchestra.run.smithi072.stdout: "dir": 20, 2024-05-09T05:02:02.378 INFO:teuthology.orchestra.run.smithi072.stdout: "dir+": 20, 2024-05-09T05:02:02.378 INFO:teuthology.orchestra.run.smithi072.stdout: "dir-": 0, 2024-05-09T05:02:02.378 INFO:teuthology.orchestra.run.smithi072.stdout: "dn": 33, 2024-05-09T05:02:02.378 INFO:teuthology.orchestra.run.smithi072.stdout: "dn+": 39, 2024-05-09T05:02:02.378 INFO:teuthology.orchestra.run.smithi072.stdout: "dn-": 6, 2024-05-09T05:02:02.378 INFO:teuthology.orchestra.run.smithi072.stdout: "cap": 3, 2024-05-09T05:02:02.378 INFO:teuthology.orchestra.run.smithi072.stdout: "cap+": 19, 2024-05-09T05:02:02.378 INFO:teuthology.orchestra.run.smithi072.stdout: "cap-": 16, 2024-05-09T05:02:02.378 INFO:teuthology.orchestra.run.smithi072.stdout: "rss": 128352, 2024-05-09T05:02:02.378 INFO:teuthology.orchestra.run.smithi072.stdout: "heap": 231824 2024-05-09T05:02:02.378 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.378 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_server": { 2024-05-09T05:02:02.378 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_client_request": 219, 2024-05-09T05:02:02.378 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_server_request": 1, 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_request": 172, 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_session": 36, 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_peer_request": 2, 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: "req_create_latency": { 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 7, 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.008106169, 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.001158024 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getattr_latency": { 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 109, 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.021969677, 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000201556 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getfilelock_latency": { 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.379 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: "req_link_latency": { 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookup_latency": { 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2, 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000347340, 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000173670 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookuphash_latency": { 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.380 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupino_latency": { 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1, 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000204629, 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000204629 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupname_latency": { 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupparent_latency": { 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupsnap_latency": { 2024-05-09T05:02:02.381 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.382 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.382 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.382 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.382 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lssnap_latency": { 2024-05-09T05:02:02.382 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 11, 2024-05-09T05:02:02.382 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.001852485, 2024-05-09T05:02:02.382 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000168407 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mkdir_latency": { 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 8, 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.014108422, 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.001763552 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mknod_latency": { 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mksnap_latency": { 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 11, 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.066996088, 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.006090553 2024-05-09T05:02:02.383 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: "req_open_latency": { 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1, 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000347486, 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000347486 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: "req_readdir_latency": { 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2, 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.005417076, 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.002708538 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rename_latency": { 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2, 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.002481047, 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.001240523 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.384 INFO:teuthology.orchestra.run.smithi072.stdout: "req_renamesnap_latency": { 2024-05-09T05:02:02.385 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.385 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.385 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.385 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.385 INFO:teuthology.orchestra.run.smithi072.stdout: "req_snapdiff_latency": { 2024-05-09T05:02:02.385 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.385 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.385 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.385 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.385 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmdir_latency": { 2024-05-09T05:02:02.385 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2, 2024-05-09T05:02:02.385 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.005604706, 2024-05-09T05:02:02.385 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.002802353 2024-05-09T05:02:02.385 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.386 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmsnap_latency": { 2024-05-09T05:02:02.386 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 5, 2024-05-09T05:02:02.386 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.026667917, 2024-05-09T05:02:02.386 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.005333583 2024-05-09T05:02:02.386 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.386 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmxattr_latency": { 2024-05-09T05:02:02.386 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.386 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.386 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.386 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.386 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setattr_latency": { 2024-05-09T05:02:02.386 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1, 2024-05-09T05:02:02.386 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.001346583, 2024-05-09T05:02:02.386 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.001346583 2024-05-09T05:02:02.386 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setdirlayout_latency": { 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setfilelock_latency": { 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setlayout_latency": { 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.387 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setxattr_latency": { 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 7, 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.021450330, 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.003064332 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: "req_symlink_latency": { 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: "req_unlink_latency": { 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 3, 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.008311882, 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.002770627 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_revoke_eviction": 0, 2024-05-09T05:02:02.388 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_acquisition_throttle": 0, 2024-05-09T05:02:02.389 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getvxattr_latency": { 2024-05-09T05:02:02.389 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.389 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.389 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.389 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:02.389 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.389 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_sessions": { 2024-05-09T05:02:02.389 INFO:teuthology.orchestra.run.smithi072.stdout: "session_count": 2, 2024-05-09T05:02:02.389 INFO:teuthology.orchestra.run.smithi072.stdout: "session_add": 5, 2024-05-09T05:02:02.389 INFO:teuthology.orchestra.run.smithi072.stdout: "session_remove": 3, 2024-05-09T05:02:02.389 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_open": 2, 2024-05-09T05:02:02.389 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_stale": 0, 2024-05-09T05:02:02.389 INFO:teuthology.orchestra.run.smithi072.stdout: "total_load": 27, 2024-05-09T05:02:02.389 INFO:teuthology.orchestra.run.smithi072.stdout: "average_load": 13, 2024-05-09T05:02:02.389 INFO:teuthology.orchestra.run.smithi072.stdout: "avg_session_uptime": 629, 2024-05-09T05:02:02.390 INFO:teuthology.orchestra.run.smithi072.stdout: "mdthresh_evicted": 0 2024-05-09T05:02:02.390 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.390 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-05-09T05:02:02.390 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-05-09T05:02:02.390 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-05-09T05:02:02.390 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T05:02:02.390 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-05-09T05:02:02.390 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T05:02:02.390 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-05-09T05:02:02.390 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T05:02:02.390 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T05:02:02.390 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T05:02:02.390 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T05:02:02.390 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T05:02:02.391 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T05:02:02.392 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-05-09T05:02:02.392 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-05-09T05:02:02.392 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-05-09T05:02:02.392 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-05-09T05:02:02.392 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T05:02:02.392 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-05-09T05:02:02.392 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T05:02:02.392 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-05-09T05:02:02.392 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 376333, 2024-05-09T05:02:02.392 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 157, 2024-05-09T05:02:02.392 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 968, 2024-05-09T05:02:02.392 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 11, 2024-05-09T05:02:02.392 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-05-09T05:02:02.392 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-05-09T05:02:02.392 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-05-09T05:02:02.393 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-05-09T05:02:02.393 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-05-09T05:02:02.393 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-05-09T05:02:02.393 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 8320, 2024-05-09T05:02:02.393 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 173, 2024-05-09T05:02:02.393 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T05:02:02.393 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-05-09T05:02:02.393 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-05-09T05:02:02.393 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-05-09T05:02:02.393 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 129397, 2024-05-09T05:02:02.393 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 1069, 2024-05-09T05:02:02.393 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-05-09T05:02:02.393 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-05-09T05:02:02.393 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-05-09T05:02:02.394 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-05-09T05:02:02.394 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.394 INFO:teuthology.orchestra.run.smithi072.stdout: "objecter": { 2024-05-09T05:02:02.394 INFO:teuthology.orchestra.run.smithi072.stdout: "op_active": 0, 2024-05-09T05:02:02.394 INFO:teuthology.orchestra.run.smithi072.stdout: "op_laggy": 0, 2024-05-09T05:02:02.394 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send": 9803, 2024-05-09T05:02:02.394 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send_bytes": 11261474, 2024-05-09T05:02:02.394 INFO:teuthology.orchestra.run.smithi072.stdout: "op_resend": 37, 2024-05-09T05:02:02.394 INFO:teuthology.orchestra.run.smithi072.stdout: "op_reply": 9803, 2024-05-09T05:02:02.394 INFO:teuthology.orchestra.run.smithi072.stdout: "op_latency": { 2024-05-09T05:02:02.394 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 9803, 2024-05-09T05:02:02.394 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 133.171151158, 2024-05-09T05:02:02.394 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.013584734 2024-05-09T05:02:02.394 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.394 INFO:teuthology.orchestra.run.smithi072.stdout: "op_inflight": 0, 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: "oplen_avg": { 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 9803, 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 10025 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: "op": 9803, 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: "op_r": 277, 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: "op_w": 9526, 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: "op_rmw": 0, 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: "op_pg": 0, 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_stat": 56, 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_create": 32, 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_read": 23, 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_write": 4170, 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writefull": 71, 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writesame": 0, 2024-05-09T05:02:02.395 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_append": 0, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_zero": 0, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_truncate": 0, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_delete": 5126, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_mapext": 0, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_sparse_read": 0, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_clonerange": 0, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_getxattr": 254, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_setxattr": 64, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_cmpxattr": 0, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_rmxattr": 0, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_resetxattrs": 0, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_call": 0, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_watch": 0, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_notify": 0, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T05:02:02.396 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls": 0, 2024-05-09T05:02:02.397 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls_filter": 0, 2024-05-09T05:02:02.397 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_other": 0, 2024-05-09T05:02:02.397 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_active": 0, 2024-05-09T05:02:02.397 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_send": 0, 2024-05-09T05:02:02.397 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_resend": 0, 2024-05-09T05:02:02.397 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_ping": 0, 2024-05-09T05:02:02.397 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_active": 0, 2024-05-09T05:02:02.397 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_send": 0, 2024-05-09T05:02:02.397 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_resend": 0, 2024-05-09T05:02:02.397 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_active": 0, 2024-05-09T05:02:02.397 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_send": 0, 2024-05-09T05:02:02.397 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_resend": 0, 2024-05-09T05:02:02.397 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_active": 0, 2024-05-09T05:02:02.397 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_send": 0, 2024-05-09T05:02:02.397 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_resend": 0, 2024-05-09T05:02:02.398 INFO:teuthology.orchestra.run.smithi072.stdout: "command_active": 0, 2024-05-09T05:02:02.398 INFO:teuthology.orchestra.run.smithi072.stdout: "command_send": 0, 2024-05-09T05:02:02.398 INFO:teuthology.orchestra.run.smithi072.stdout: "command_resend": 0, 2024-05-09T05:02:02.398 INFO:teuthology.orchestra.run.smithi072.stdout: "map_epoch": 93, 2024-05-09T05:02:02.398 INFO:teuthology.orchestra.run.smithi072.stdout: "map_full": 0, 2024-05-09T05:02:02.398 INFO:teuthology.orchestra.run.smithi072.stdout: "map_inc": 13, 2024-05-09T05:02:02.398 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_sessions": 12, 2024-05-09T05:02:02.398 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_open": 12, 2024-05-09T05:02:02.398 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_close": 0, 2024-05-09T05:02:02.398 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_laggy": 0, 2024-05-09T05:02:02.398 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_wr": 210, 2024-05-09T05:02:02.398 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_rd": 0, 2024-05-09T05:02:02.398 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_del": 19 2024-05-09T05:02:02.398 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: "oft": { 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_objs": 1, 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_kv_pairs": 0, 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_updates": 0, 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_removes": 0 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: "osd-slow-ops": { 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_ops_count": 0 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: "purge_queue": { 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops": 0, 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops_high_water": 3584, 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing": 0, 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_high_water": 1, 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed_ops": 4610, 2024-05-09T05:02:02.399 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed": 3, 2024-05-09T05:02:02.400 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_item_in_journal": 0 2024-05-09T05:02:02.400 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.400 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T05:02:02.400 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-09T05:02:02.400 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-05-09T05:02:02.400 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:02.400 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 17348, 2024-05-09T05:02:02.400 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 5539206, 2024-05-09T05:02:02.400 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:02.400 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 17348, 2024-05-09T05:02:02.400 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-09T05:02:02.400 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-09T05:02:02.400 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 17348, 2024-05-09T05:02:02.400 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 5539206, 2024-05-09T05:02:02.400 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:02.401 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.401 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.401 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.401 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:02.401 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.401 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_bytes": { 2024-05-09T05:02:02.401 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-09T05:02:02.401 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-05-09T05:02:02.401 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:02.401 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-09T05:02:02.401 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-09T05:02:02.401 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:02.401 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:02.401 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 9803, 2024-05-09T05:02:02.401 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 11261793, 2024-05-09T05:02:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 9803, 2024-05-09T05:02:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 11261793, 2024-05-09T05:02:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_ops": { 2024-05-09T05:02:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-09T05:02:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 1024, 2024-05-09T05:02:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-09T05:02:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-09T05:02:02.402 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:02.403 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:02.403 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 9803, 2024-05-09T05:02:02.403 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 9803, 2024-05-09T05:02:02.403 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 9803, 2024-05-09T05:02:02.403 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 9803, 2024-05-09T05:02:02.403 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:02.403 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.403 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.403 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.403 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:02.403 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.403 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle": { 2024-05-09T05:02:02.403 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-09T05:02:02.403 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-05-09T05:02:02.403 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:02.404 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 3, 2024-05-09T05:02:02.404 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 319, 2024-05-09T05:02:02.404 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:02.404 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 3, 2024-05-09T05:02:02.404 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-09T05:02:02.404 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-09T05:02:02.404 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 3, 2024-05-09T05:02:02.404 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 319, 2024-05-09T05:02:02.404 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:02.404 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.404 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.404 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.404 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:02.404 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle-0x55bd8ba864e0": { 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 60, 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 4197, 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 11130280, 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 4197, 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 4167, 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 11130220, 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:02.405 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:02.406 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:02.406 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:02.406 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-05-09T05:02:02.880 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:02 smithi072 ceph-mon[34088]: pgmap v339: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-09T05:02:03.229 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-05-09T05:02:03.230 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-05-09T05:02:03.230 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=6 2024-05-09T05:02:03.230 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-05-09T05:02:03.230 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-09T05:02:03.230 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-05-09T05:02:03.230 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-05-09T05:02:03.230 DEBUG:tasks.check_counter:Getting stats from e 2024-05-09T05:02:03.230 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-09T05:02:03.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:02 smithi110 ceph-mon[34904]: pgmap v339: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-09T05:02:03.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:02 smithi046 ceph-mon[30136]: pgmap v339: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 23 KiB/s wr, 10 op/s 2024-05-09T05:02:03.481 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T05:02:04.970 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:04 smithi072 ceph-mon[34088]: pgmap v340: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T05:02:05.249 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-05-09T05:02:05.249 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T05:02:05.249 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 1217, 2024-05-09T05:02:05.249 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 1218, 2024-05-09T05:02:05.249 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 4574989, 2024-05-09T05:02:05.249 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 386698, 2024-05-09T05:02:05.249 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 6, 2024-05-09T05:02:05.249 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 2, 2024-05-09T05:02:05.249 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.130551042, 2024-05-09T05:02:05.249 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.051057812, 2024-05-09T05:02:05.249 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.030587777, 2024-05-09T05:02:05.250 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.036093653, 2024-05-09T05:02:05.250 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:05.250 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1216, 2024-05-09T05:02:05.250 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.059907384, 2024-05-09T05:02:05.250 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000049265 2024-05-09T05:02:05.250 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.250 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:05.250 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.250 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.250 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.250 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.250 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T05:02:05.250 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T05:02:05.250 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.251 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T05:02:05.251 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 1035, 2024-05-09T05:02:05.251 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 1215, 2024-05-09T05:02:05.251 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 3004706, 2024-05-09T05:02:05.251 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 804768, 2024-05-09T05:02:05.251 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 7, 2024-05-09T05:02:05.251 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 5, 2024-05-09T05:02:05.251 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.087298856, 2024-05-09T05:02:05.251 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.025615606, 2024-05-09T05:02:05.251 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.024754170, 2024-05-09T05:02:05.251 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.029607162, 2024-05-09T05:02:05.251 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:05.251 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1212, 2024-05-09T05:02:05.251 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.052070286, 2024-05-09T05:02:05.251 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000042962 2024-05-09T05:02:05.252 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.252 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:05.252 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.252 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.252 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.252 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.252 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T05:02:05.252 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 476208 2024-05-09T05:02:05.252 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.252 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T05:02:05.252 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 624, 2024-05-09T05:02:05.252 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 609, 2024-05-09T05:02:05.252 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 6031931, 2024-05-09T05:02:05.252 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 261653, 2024-05-09T05:02:05.252 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 3, 2024-05-09T05:02:05.253 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 2, 2024-05-09T05:02:05.253 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.089393719, 2024-05-09T05:02:05.253 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.025133632, 2024-05-09T05:02:05.253 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.055265147, 2024-05-09T05:02:05.253 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.018397639, 2024-05-09T05:02:05.253 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:05.253 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 609, 2024-05-09T05:02:05.253 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.024532177, 2024-05-09T05:02:05.253 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000040282 2024-05-09T05:02:05.253 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.253 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:05.253 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.253 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.253 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 1751072, 2024-05-09T05:02:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 150352 2024-05-09T05:02:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-05-09T05:02:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 1, 2024-05-09T05:02:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-05-09T05:02:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-MDSRank": { 2024-05-09T05:02:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-05-09T05:02:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-05-09T05:02:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 4343, 2024-05-09T05:02:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.119176361, 2024-05-09T05:02:05.254 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000027441 2024-05-09T05:02:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-PurgeQueue": { 2024-05-09T05:02:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-05-09T05:02:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-05-09T05:02:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.255 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": { 2024-05-09T05:02:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: "request": 0, 2024-05-09T05:02:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: "reply": 0, 2024-05-09T05:02:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: "reply_latency": { 2024-05-09T05:02:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_reply": 0, 2024-05-09T05:02:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: "forward": 0, 2024-05-09T05:02:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_complete": 0, 2024-05-09T05:02:05.256 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_keys": 0, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_commit": 0, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_split": 0, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_merge": 0, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes": 30, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_top": 11, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_bottom": 19, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pin_tail": 0, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pinned": 15, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_expired": 0, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_with_caps": 0, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": 0, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "subtrees": 2, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse": 0, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_hit": 0, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_forward": 0, 2024-05-09T05:02:05.257 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_discover": 0, 2024-05-09T05:02:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_dir_fetch": 0, 2024-05-09T05:02:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_remote_ino": 0, 2024-05-09T05:02:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_lock": 0, 2024-05-09T05:02:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: "load_cent": 0, 2024-05-09T05:02:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: "q": 0, 2024-05-09T05:02:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: "exported": 0, 2024-05-09T05:02:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: "exported_inodes": 0, 2024-05-09T05:02:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: "imported": 0, 2024-05-09T05:02:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: "imported_inodes": 0, 2024-05-09T05:02:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_dir_fetch": 0, 2024-05-09T05:02:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_backtrace_fetch": 0, 2024-05-09T05:02:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_peer_discover": 0, 2024-05-09T05:02:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rfiles": 4, 2024-05-09T05:02:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rbytes": 234, 2024-05-09T05:02:05.258 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rsnaps": 6, 2024-05-09T05:02:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T05:02:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_set_tag": 0, 2024-05-09T05:02:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T05:02:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_inotable_repaired": 0, 2024-05-09T05:02:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_inodes": 0, 2024-05-09T05:02:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T05:02:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T05:02:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_file_inodes": 0, 2024-05-09T05:02:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_inode_file_caps": 0, 2024-05-09T05:02:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T05:02:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_grant": 0, 2024-05-09T05:02:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T05:02:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T05:02:05.259 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T05:02:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps": 0, 2024-05-09T05:02:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps_dirty": 0, 2024-05-09T05:02:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_cap_release": 0, 2024-05-09T05:02:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: "process_request_cap_release": 0 2024-05-09T05:02:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_cache": { 2024-05-09T05:02:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update": 0, 2024-05-09T05:02:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update_receipt": 0, 2024-05-09T05:02:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_try_discover": 0, 2024-05-09T05:02:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_send_discover": 0, 2024-05-09T05:02:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_handle_discover": 0, 2024-05-09T05:02:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays": 0, 2024-05-09T05:02:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_delayed": 0, 2024-05-09T05:02:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_enqueuing": 0, 2024-05-09T05:02:05.260 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_created": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_enqueued": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_reintegrated": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_migrated": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_processing": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_enqueued": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_prioritized": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_started": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_completed": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_path": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_inode": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_exportdir": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_flush": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragmentdir": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragstats": 0, 2024-05-09T05:02:05.261 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_inodestats": 0 2024-05-09T05:02:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_log": { 2024-05-09T05:02:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: "evlrg": 0, 2024-05-09T05:02:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: "evadd": 0, 2024-05-09T05:02:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: "evex": 0, 2024-05-09T05:02:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: "evtrm": 0, 2024-05-09T05:02:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: "ev": 4196, 2024-05-09T05:02:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: "evexg": 0, 2024-05-09T05:02:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: "evexd": 0, 2024-05-09T05:02:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: "segadd": 0, 2024-05-09T05:02:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: "segex": 0, 2024-05-09T05:02:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: "segtrm": 0, 2024-05-09T05:02:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: "seg": 2, 2024-05-09T05:02:05.262 INFO:teuthology.orchestra.run.smithi072.stdout: "segmjr": 15, 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: "segexg": 0, 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: "segexd": 0, 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: "expos": 15255912, 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: "wrpos": 15324524, 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: "rdpos": 15324440, 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: "jlat": { 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: "replayed": 4196 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_mem": { 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: "ino": 21, 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: "ino+": 24, 2024-05-09T05:02:05.263 INFO:teuthology.orchestra.run.smithi072.stdout: "ino-": 3, 2024-05-09T05:02:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: "dir": 14, 2024-05-09T05:02:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: "dir+": 14, 2024-05-09T05:02:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: "dir-": 0, 2024-05-09T05:02:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: "dn": 30, 2024-05-09T05:02:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: "dn+": 30, 2024-05-09T05:02:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: "dn-": 0, 2024-05-09T05:02:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: "cap": 0, 2024-05-09T05:02:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: "cap+": 0, 2024-05-09T05:02:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: "cap-": 0, 2024-05-09T05:02:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: "rss": 124036, 2024-05-09T05:02:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: "heap": 215440 2024-05-09T05:02:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_server": { 2024-05-09T05:02:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_client_request": 0, 2024-05-09T05:02:05.264 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_server_request": 0, 2024-05-09T05:02:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_request": 0, 2024-05-09T05:02:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_session": 0, 2024-05-09T05:02:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_peer_request": 0, 2024-05-09T05:02:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: "req_create_latency": { 2024-05-09T05:02:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.265 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getattr_latency": { 2024-05-09T05:02:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getfilelock_latency": { 2024-05-09T05:02:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: "req_link_latency": { 2024-05-09T05:02:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.266 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookup_latency": { 2024-05-09T05:02:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookuphash_latency": { 2024-05-09T05:02:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupino_latency": { 2024-05-09T05:02:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.267 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupname_latency": { 2024-05-09T05:02:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupparent_latency": { 2024-05-09T05:02:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupsnap_latency": { 2024-05-09T05:02:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.268 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.269 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.269 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.269 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lssnap_latency": { 2024-05-09T05:02:05.269 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.269 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.269 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.269 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.269 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mkdir_latency": { 2024-05-09T05:02:05.269 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.269 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.269 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.269 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.269 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mknod_latency": { 2024-05-09T05:02:05.269 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.269 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.270 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.270 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.270 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mksnap_latency": { 2024-05-09T05:02:05.270 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.270 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.270 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.270 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.270 INFO:teuthology.orchestra.run.smithi072.stdout: "req_open_latency": { 2024-05-09T05:02:05.270 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.270 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.270 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.270 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.270 INFO:teuthology.orchestra.run.smithi072.stdout: "req_readdir_latency": { 2024-05-09T05:02:05.270 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.270 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.271 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.271 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.271 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rename_latency": { 2024-05-09T05:02:05.271 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.271 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.271 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.271 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.271 INFO:teuthology.orchestra.run.smithi072.stdout: "req_renamesnap_latency": { 2024-05-09T05:02:05.271 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.271 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.271 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.271 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.271 INFO:teuthology.orchestra.run.smithi072.stdout: "req_snapdiff_latency": { 2024-05-09T05:02:05.271 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.271 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.272 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.272 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.272 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmdir_latency": { 2024-05-09T05:02:05.272 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.272 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.272 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.272 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.272 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmsnap_latency": { 2024-05-09T05:02:05.272 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.272 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.272 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.272 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.272 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmxattr_latency": { 2024-05-09T05:02:05.272 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.273 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.273 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.273 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.273 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setattr_latency": { 2024-05-09T05:02:05.273 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.273 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.273 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.273 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.273 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setdirlayout_latency": { 2024-05-09T05:02:05.273 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.273 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.274 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.274 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.274 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setfilelock_latency": { 2024-05-09T05:02:05.274 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.274 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.274 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.274 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.274 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setlayout_latency": { 2024-05-09T05:02:05.274 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.274 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.274 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.274 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.274 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setxattr_latency": { 2024-05-09T05:02:05.274 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.275 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.275 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.275 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.275 INFO:teuthology.orchestra.run.smithi072.stdout: "req_symlink_latency": { 2024-05-09T05:02:05.275 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.275 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.275 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.275 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.275 INFO:teuthology.orchestra.run.smithi072.stdout: "req_unlink_latency": { 2024-05-09T05:02:05.275 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.275 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.276 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.276 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.276 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_revoke_eviction": 0, 2024-05-09T05:02:05.276 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_acquisition_throttle": 0, 2024-05-09T05:02:05.276 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getvxattr_latency": { 2024-05-09T05:02:05.276 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.276 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.276 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.276 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:05.276 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.276 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_sessions": { 2024-05-09T05:02:05.277 INFO:teuthology.orchestra.run.smithi072.stdout: "session_count": 2, 2024-05-09T05:02:05.277 INFO:teuthology.orchestra.run.smithi072.stdout: "session_add": 5, 2024-05-09T05:02:05.277 INFO:teuthology.orchestra.run.smithi072.stdout: "session_remove": 3, 2024-05-09T05:02:05.277 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_open": 4, 2024-05-09T05:02:05.277 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_stale": 0, 2024-05-09T05:02:05.277 INFO:teuthology.orchestra.run.smithi072.stdout: "total_load": 0, 2024-05-09T05:02:05.277 INFO:teuthology.orchestra.run.smithi072.stdout: "average_load": 0, 2024-05-09T05:02:05.277 INFO:teuthology.orchestra.run.smithi072.stdout: "avg_session_uptime": 3513, 2024-05-09T05:02:05.277 INFO:teuthology.orchestra.run.smithi072.stdout: "mdthresh_evicted": 0 2024-05-09T05:02:05.277 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.277 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-05-09T05:02:05.277 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-05-09T05:02:05.277 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-05-09T05:02:05.277 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T05:02:05.277 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-05-09T05:02:05.278 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T05:02:05.278 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-05-09T05:02:05.278 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T05:02:05.278 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T05:02:05.278 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T05:02:05.278 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T05:02:05.278 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T05:02:05.278 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-05-09T05:02:05.278 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T05:02:05.278 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T05:02:05.278 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-05-09T05:02:05.278 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-05-09T05:02:05.278 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-05-09T05:02:05.278 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-05-09T05:02:05.279 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T05:02:05.279 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T05:02:05.279 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T05:02:05.279 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T05:02:05.279 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T05:02:05.279 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-05-09T05:02:05.279 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-05-09T05:02:05.279 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-05-09T05:02:05.279 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T05:02:05.279 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T05:02:05.279 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-05-09T05:02:05.279 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-05-09T05:02:05.279 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-05-09T05:02:05.279 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-05-09T05:02:05.279 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T05:02:05.280 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-05-09T05:02:05.280 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T05:02:05.280 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-05-09T05:02:05.280 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 190524, 2024-05-09T05:02:05.280 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 61, 2024-05-09T05:02:05.280 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-05-09T05:02:05.280 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-05-09T05:02:05.280 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-05-09T05:02:05.280 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-05-09T05:02:05.280 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-05-09T05:02:05.280 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-05-09T05:02:05.280 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-05-09T05:02:05.280 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-05-09T05:02:05.280 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 8320, 2024-05-09T05:02:05.280 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 173, 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 102989, 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 693, 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: "objecter": { 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: "op_active": 0, 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: "op_laggy": 0, 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send": 2599, 2024-05-09T05:02:05.281 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send_bytes": 26, 2024-05-09T05:02:05.282 INFO:teuthology.orchestra.run.smithi072.stdout: "op_resend": 4, 2024-05-09T05:02:05.282 INFO:teuthology.orchestra.run.smithi072.stdout: "op_reply": 2599, 2024-05-09T05:02:05.282 INFO:teuthology.orchestra.run.smithi072.stdout: "op_latency": { 2024-05-09T05:02:05.282 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2599, 2024-05-09T05:02:05.282 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 1.440969217, 2024-05-09T05:02:05.282 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000554432 2024-05-09T05:02:05.282 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.282 INFO:teuthology.orchestra.run.smithi072.stdout: "op_inflight": 0, 2024-05-09T05:02:05.282 INFO:teuthology.orchestra.run.smithi072.stdout: "oplen_avg": { 2024-05-09T05:02:05.282 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2599, 2024-05-09T05:02:05.282 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 2600 2024-05-09T05:02:05.282 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.282 INFO:teuthology.orchestra.run.smithi072.stdout: "op": 2599, 2024-05-09T05:02:05.282 INFO:teuthology.orchestra.run.smithi072.stdout: "op_r": 2599, 2024-05-09T05:02:05.283 INFO:teuthology.orchestra.run.smithi072.stdout: "op_w": 0, 2024-05-09T05:02:05.283 INFO:teuthology.orchestra.run.smithi072.stdout: "op_rmw": 0, 2024-05-09T05:02:05.283 INFO:teuthology.orchestra.run.smithi072.stdout: "op_pg": 0, 2024-05-09T05:02:05.283 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_stat": 1663, 2024-05-09T05:02:05.283 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_create": 0, 2024-05-09T05:02:05.283 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_read": 933, 2024-05-09T05:02:05.283 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_write": 0, 2024-05-09T05:02:05.283 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writefull": 0, 2024-05-09T05:02:05.283 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writesame": 0, 2024-05-09T05:02:05.283 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_append": 0, 2024-05-09T05:02:05.283 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_zero": 0, 2024-05-09T05:02:05.283 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_truncate": 0, 2024-05-09T05:02:05.283 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_delete": 0, 2024-05-09T05:02:05.283 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_mapext": 0, 2024-05-09T05:02:05.283 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_sparse_read": 0, 2024-05-09T05:02:05.284 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_clonerange": 0, 2024-05-09T05:02:05.284 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_getxattr": 2, 2024-05-09T05:02:05.284 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_setxattr": 0, 2024-05-09T05:02:05.284 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_cmpxattr": 0, 2024-05-09T05:02:05.284 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_rmxattr": 0, 2024-05-09T05:02:05.284 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_resetxattrs": 0, 2024-05-09T05:02:05.284 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_call": 0, 2024-05-09T05:02:05.284 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_watch": 0, 2024-05-09T05:02:05.284 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_notify": 0, 2024-05-09T05:02:05.284 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T05:02:05.284 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls": 0, 2024-05-09T05:02:05.284 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls_filter": 0, 2024-05-09T05:02:05.284 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_other": 0, 2024-05-09T05:02:05.284 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_active": 0, 2024-05-09T05:02:05.284 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_send": 0, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_resend": 0, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_ping": 0, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_active": 0, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_send": 0, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_resend": 0, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_active": 0, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_send": 0, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_resend": 0, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_active": 0, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_send": 0, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_resend": 0, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "command_active": 0, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "command_send": 0, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "command_resend": 0, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "map_epoch": 93, 2024-05-09T05:02:05.285 INFO:teuthology.orchestra.run.smithi072.stdout: "map_full": 0, 2024-05-09T05:02:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: "map_inc": 13, 2024-05-09T05:02:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_sessions": 5, 2024-05-09T05:02:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_open": 5, 2024-05-09T05:02:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_close": 0, 2024-05-09T05:02:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_laggy": 0, 2024-05-09T05:02:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_wr": 0, 2024-05-09T05:02:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_rd": 2, 2024-05-09T05:02:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_del": 0 2024-05-09T05:02:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: "oft": { 2024-05-09T05:02:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_objs": 0, 2024-05-09T05:02:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_kv_pairs": 0, 2024-05-09T05:02:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_updates": 0, 2024-05-09T05:02:05.286 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_removes": 0 2024-05-09T05:02:05.287 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.287 INFO:teuthology.orchestra.run.smithi072.stdout: "osd-slow-ops": { 2024-05-09T05:02:05.287 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_ops_count": 0 2024-05-09T05:02:05.287 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.287 INFO:teuthology.orchestra.run.smithi072.stdout: "purge_queue": { 2024-05-09T05:02:05.287 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops": 0, 2024-05-09T05:02:05.287 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T05:02:05.287 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing": 0, 2024-05-09T05:02:05.287 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_high_water": 0, 2024-05-09T05:02:05.287 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed_ops": 0, 2024-05-09T05:02:05.287 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed": 0, 2024-05-09T05:02:05.287 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_item_in_journal": 0 2024-05-09T05:02:05.287 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.287 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T05:02:05.287 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 2872, 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 13336280, 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 2872, 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 2872, 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 13336280, 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:05.288 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.289 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_bytes": { 2024-05-09T05:02:05.289 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-09T05:02:05.289 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-05-09T05:02:05.289 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:05.289 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-09T05:02:05.289 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-09T05:02:05.289 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:05.289 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:05.289 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 2599, 2024-05-09T05:02:05.289 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 11130230, 2024-05-09T05:02:05.289 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 2599, 2024-05-09T05:02:05.289 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 11130230, 2024-05-09T05:02:05.289 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:05.289 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.289 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.290 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.290 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:05.290 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.290 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_ops": { 2024-05-09T05:02:05.290 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-09T05:02:05.290 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 1024, 2024-05-09T05:02:05.290 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:05.290 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-09T05:02:05.290 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-09T05:02:05.290 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:05.290 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:05.290 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 2599, 2024-05-09T05:02:05.290 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 2599, 2024-05-09T05:02:05.290 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 2599, 2024-05-09T05:02:05.290 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 2599, 2024-05-09T05:02:05.291 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:05.291 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.291 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.291 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.291 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:05.291 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.291 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle": { 2024-05-09T05:02:05.291 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-09T05:02:05.291 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-05-09T05:02:05.291 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:05.292 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-09T05:02:05.292 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-09T05:02:05.292 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:05.292 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:05.292 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-09T05:02:05.292 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-09T05:02:05.292 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 0, 2024-05-09T05:02:05.292 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 0, 2024-05-09T05:02:05.292 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:05.292 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.292 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.292 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.292 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:05.292 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle-0x5642d81b44e0": { 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 0, 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 0, 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:05.293 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:05.294 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:05.294 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:05.294 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-05-09T05:02:05.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:04 smithi110 ceph-mon[34904]: pgmap v340: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T05:02:05.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:04 smithi046 ceph-mon[30136]: pgmap v340: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T05:02:06.179 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-05-09T05:02:06.179 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-05-09T05:02:06.180 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-05-09T05:02:06.180 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-05-09T05:02:06.180 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-05-09T05:02:06.180 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-09T05:02:06.180 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T05:02:06.180 DEBUG:tasks.check_counter:Getting stats from h 2024-05-09T05:02:06.180 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-09T05:02:06.420 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T05:02:07.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:06 smithi072 ceph-mon[34088]: pgmap v341: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T05:02:07.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:06 smithi110 ceph-mon[34904]: pgmap v341: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T05:02:07.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:06 smithi046 ceph-mon[30136]: pgmap v341: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 44 KiB/s wr, 22 op/s 2024-05-09T05:02:07.959 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:07 smithi072 ceph-mon[34088]: osdmap e94: 12 total, 12 up, 12 in 2024-05-09T05:02:08.217 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-05-09T05:02:08.217 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T05:02:08.217 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 97, 2024-05-09T05:02:08.217 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 119, 2024-05-09T05:02:08.217 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 28709, 2024-05-09T05:02:08.217 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 133345, 2024-05-09T05:02:08.217 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 18, 2024-05-09T05:02:08.217 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 5, 2024-05-09T05:02:08.217 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.023405002, 2024-05-09T05:02:08.217 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.005456076, 2024-05-09T05:02:08.217 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.012550294, 2024-05-09T05:02:08.218 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000575719, 2024-05-09T05:02:08.218 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:08.218 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 113, 2024-05-09T05:02:08.218 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.015902430, 2024-05-09T05:02:08.218 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000140729 2024-05-09T05:02:08.218 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.218 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:08.218 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 135, 2024-05-09T05:02:08.218 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000031733, 2024-05-09T05:02:08.218 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000235 2024-05-09T05:02:08.218 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.218 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T05:02:08.218 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T05:02:08.219 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.219 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T05:02:08.219 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 775, 2024-05-09T05:02:08.219 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 1046, 2024-05-09T05:02:08.219 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 1892651, 2024-05-09T05:02:08.219 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 825307, 2024-05-09T05:02:08.219 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 8, 2024-05-09T05:02:08.219 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 4, 2024-05-09T05:02:08.219 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.135787432, 2024-05-09T05:02:08.219 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.056851892, 2024-05-09T05:02:08.219 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.083961008, 2024-05-09T05:02:08.219 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.010435763, 2024-05-09T05:02:08.219 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:08.219 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1050, 2024-05-09T05:02:08.220 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.073207385, 2024-05-09T05:02:08.220 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000069721 2024-05-09T05:02:08.220 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.220 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:08.220 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 1061, 2024-05-09T05:02:08.220 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000303343, 2024-05-09T05:02:08.220 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000285 2024-05-09T05:02:08.220 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.220 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 1768240, 2024-05-09T05:02:08.220 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 481264 2024-05-09T05:02:08.220 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.220 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T05:02:08.220 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 75, 2024-05-09T05:02:08.220 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 157, 2024-05-09T05:02:08.220 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 15165, 2024-05-09T05:02:08.221 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 608002, 2024-05-09T05:02:08.221 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 46, 2024-05-09T05:02:08.221 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 5, 2024-05-09T05:02:08.221 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.024699302, 2024-05-09T05:02:08.221 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.004843968, 2024-05-09T05:02:08.221 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.018430819, 2024-05-09T05:02:08.221 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000249958, 2024-05-09T05:02:08.221 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:08.221 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 154, 2024-05-09T05:02:08.221 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.010960594, 2024-05-09T05:02:08.221 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000071172 2024-05-09T05:02:08.221 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.221 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:08.221 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 122, 2024-05-09T05:02:08.222 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000022766, 2024-05-09T05:02:08.222 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000186 2024-05-09T05:02:08.222 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.222 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 7152, 2024-05-09T05:02:08.222 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 168832 2024-05-09T05:02:08.222 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.222 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-05-09T05:02:08.222 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 1, 2024-05-09T05:02:08.222 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-05-09T05:02:08.222 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.222 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-MDSRank": { 2024-05-09T05:02:08.223 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-05-09T05:02:08.223 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-05-09T05:02:08.223 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 55, 2024-05-09T05:02:08.223 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.001647298, 2024-05-09T05:02:08.223 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000029950 2024-05-09T05:02:08.223 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:08.223 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.223 INFO:teuthology.orchestra.run.smithi072.stdout: "finisher-PurgeQueue": { 2024-05-09T05:02:08.223 INFO:teuthology.orchestra.run.smithi072.stdout: "queue_len": 0, 2024-05-09T05:02:08.224 INFO:teuthology.orchestra.run.smithi072.stdout: "complete_latency": { 2024-05-09T05:02:08.224 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2, 2024-05-09T05:02:08.224 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000024642, 2024-05-09T05:02:08.224 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000012321 2024-05-09T05:02:08.224 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:08.224 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.224 INFO:teuthology.orchestra.run.smithi072.stdout: "mds": { 2024-05-09T05:02:08.224 INFO:teuthology.orchestra.run.smithi072.stdout: "request": 12, 2024-05-09T05:02:08.224 INFO:teuthology.orchestra.run.smithi072.stdout: "reply": 7, 2024-05-09T05:02:08.224 INFO:teuthology.orchestra.run.smithi072.stdout: "reply_latency": { 2024-05-09T05:02:08.224 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 7, 2024-05-09T05:02:08.224 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.002826439, 2024-05-09T05:02:08.224 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000403777 2024-05-09T05:02:08.225 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.225 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_reply": 0, 2024-05-09T05:02:08.225 INFO:teuthology.orchestra.run.smithi072.stdout: "forward": 5, 2024-05-09T05:02:08.225 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_complete": 0, 2024-05-09T05:02:08.225 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_fetch_keys": 0, 2024-05-09T05:02:08.225 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_commit": 11, 2024-05-09T05:02:08.225 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_split": 0, 2024-05-09T05:02:08.225 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_merge": 0, 2024-05-09T05:02:08.225 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes": 14, 2024-05-09T05:02:08.225 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_top": 1, 2024-05-09T05:02:08.225 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_bottom": 13, 2024-05-09T05:02:08.225 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pin_tail": 0, 2024-05-09T05:02:08.225 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_pinned": 12, 2024-05-09T05:02:08.225 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_expired": 0, 2024-05-09T05:02:08.225 INFO:teuthology.orchestra.run.smithi072.stdout: "inodes_with_caps": 1, 2024-05-09T05:02:08.226 INFO:teuthology.orchestra.run.smithi072.stdout: "caps": 1, 2024-05-09T05:02:08.226 INFO:teuthology.orchestra.run.smithi072.stdout: "subtrees": 2, 2024-05-09T05:02:08.226 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse": 23, 2024-05-09T05:02:08.226 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_hit": 17, 2024-05-09T05:02:08.226 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_forward": 3, 2024-05-09T05:02:08.226 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_discover": 2, 2024-05-09T05:02:08.226 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_dir_fetch": 0, 2024-05-09T05:02:08.226 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_remote_ino": 0, 2024-05-09T05:02:08.226 INFO:teuthology.orchestra.run.smithi072.stdout: "traverse_lock": 0, 2024-05-09T05:02:08.226 INFO:teuthology.orchestra.run.smithi072.stdout: "load_cent": 0, 2024-05-09T05:02:08.226 INFO:teuthology.orchestra.run.smithi072.stdout: "q": 0, 2024-05-09T05:02:08.226 INFO:teuthology.orchestra.run.smithi072.stdout: "exported": 0, 2024-05-09T05:02:08.226 INFO:teuthology.orchestra.run.smithi072.stdout: "exported_inodes": 0, 2024-05-09T05:02:08.226 INFO:teuthology.orchestra.run.smithi072.stdout: "imported": 0, 2024-05-09T05:02:08.227 INFO:teuthology.orchestra.run.smithi072.stdout: "imported_inodes": 0, 2024-05-09T05:02:08.227 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_dir_fetch": 0, 2024-05-09T05:02:08.227 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_backtrace_fetch": 0, 2024-05-09T05:02:08.227 INFO:teuthology.orchestra.run.smithi072.stdout: "openino_peer_discover": 0, 2024-05-09T05:02:08.227 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rfiles": 4, 2024-05-09T05:02:08.227 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rbytes": 234, 2024-05-09T05:02:08.227 INFO:teuthology.orchestra.run.smithi072.stdout: "root_rsnaps": 6, 2024-05-09T05:02:08.227 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T05:02:08.227 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_set_tag": 0, 2024-05-09T05:02:08.227 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T05:02:08.227 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_inotable_repaired": 0, 2024-05-09T05:02:08.227 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_inodes": 0, 2024-05-09T05:02:08.228 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T05:02:08.228 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T05:02:08.228 INFO:teuthology.orchestra.run.smithi072.stdout: "scrub_file_inodes": 0, 2024-05-09T05:02:08.228 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_inode_file_caps": 0, 2024-05-09T05:02:08.228 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T05:02:08.228 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_grant": 0, 2024-05-09T05:02:08.228 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T05:02:08.228 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T05:02:08.228 INFO:teuthology.orchestra.run.smithi072.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T05:02:08.228 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps": 0, 2024-05-09T05:02:08.228 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_caps_dirty": 0, 2024-05-09T05:02:08.228 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_cap_release": 0, 2024-05-09T05:02:08.229 INFO:teuthology.orchestra.run.smithi072.stdout: "process_request_cap_release": 0 2024-05-09T05:02:08.229 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.229 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_cache": { 2024-05-09T05:02:08.229 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update": 0, 2024-05-09T05:02:08.229 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_update_receipt": 8, 2024-05-09T05:02:08.229 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_try_discover": 5, 2024-05-09T05:02:08.229 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_send_discover": 6, 2024-05-09T05:02:08.229 INFO:teuthology.orchestra.run.smithi072.stdout: "dir_handle_discover": 0, 2024-05-09T05:02:08.229 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays": 0, 2024-05-09T05:02:08.229 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_delayed": 0, 2024-05-09T05:02:08.229 INFO:teuthology.orchestra.run.smithi072.stdout: "num_strays_enqueuing": 0, 2024-05-09T05:02:08.229 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_created": 0, 2024-05-09T05:02:08.229 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_enqueued": 0, 2024-05-09T05:02:08.229 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_reintegrated": 0, 2024-05-09T05:02:08.230 INFO:teuthology.orchestra.run.smithi072.stdout: "strays_migrated": 0, 2024-05-09T05:02:08.230 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_processing": 0, 2024-05-09T05:02:08.230 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_enqueued": 0, 2024-05-09T05:02:08.230 INFO:teuthology.orchestra.run.smithi072.stdout: "num_recovering_prioritized": 0, 2024-05-09T05:02:08.230 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_started": 0, 2024-05-09T05:02:08.230 INFO:teuthology.orchestra.run.smithi072.stdout: "recovery_completed": 0, 2024-05-09T05:02:08.230 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_path": 0, 2024-05-09T05:02:08.230 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_quiesce_inode": 0, 2024-05-09T05:02:08.230 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T05:02:08.230 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_exportdir": 0, 2024-05-09T05:02:08.230 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_flush": 0, 2024-05-09T05:02:08.230 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragmentdir": 0, 2024-05-09T05:02:08.230 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_fragstats": 0, 2024-05-09T05:02:08.230 INFO:teuthology.orchestra.run.smithi072.stdout: "ireq_inodestats": 0 2024-05-09T05:02:08.231 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.231 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_log": { 2024-05-09T05:02:08.231 INFO:teuthology.orchestra.run.smithi072.stdout: "evlrg": 0, 2024-05-09T05:02:08.231 INFO:teuthology.orchestra.run.smithi072.stdout: "evadd": 8, 2024-05-09T05:02:08.231 INFO:teuthology.orchestra.run.smithi072.stdout: "evex": 0, 2024-05-09T05:02:08.231 INFO:teuthology.orchestra.run.smithi072.stdout: "evtrm": 0, 2024-05-09T05:02:08.231 INFO:teuthology.orchestra.run.smithi072.stdout: "ev": 8, 2024-05-09T05:02:08.231 INFO:teuthology.orchestra.run.smithi072.stdout: "evexg": 0, 2024-05-09T05:02:08.231 INFO:teuthology.orchestra.run.smithi072.stdout: "evexd": 0, 2024-05-09T05:02:08.231 INFO:teuthology.orchestra.run.smithi072.stdout: "segadd": 2, 2024-05-09T05:02:08.231 INFO:teuthology.orchestra.run.smithi072.stdout: "segex": 0, 2024-05-09T05:02:08.231 INFO:teuthology.orchestra.run.smithi072.stdout: "segtrm": 0, 2024-05-09T05:02:08.231 INFO:teuthology.orchestra.run.smithi072.stdout: "seg": 2, 2024-05-09T05:02:08.231 INFO:teuthology.orchestra.run.smithi072.stdout: "segmjr": 2, 2024-05-09T05:02:08.232 INFO:teuthology.orchestra.run.smithi072.stdout: "segexg": 0, 2024-05-09T05:02:08.232 INFO:teuthology.orchestra.run.smithi072.stdout: "segexd": 0, 2024-05-09T05:02:08.232 INFO:teuthology.orchestra.run.smithi072.stdout: "expos": 4194304, 2024-05-09T05:02:08.232 INFO:teuthology.orchestra.run.smithi072.stdout: "wrpos": 4196655, 2024-05-09T05:02:08.232 INFO:teuthology.orchestra.run.smithi072.stdout: "rdpos": 0, 2024-05-09T05:02:08.232 INFO:teuthology.orchestra.run.smithi072.stdout: "jlat": { 2024-05-09T05:02:08.232 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 7, 2024-05-09T05:02:08.232 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.013080124, 2024-05-09T05:02:08.232 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.001868589 2024-05-09T05:02:08.232 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.232 INFO:teuthology.orchestra.run.smithi072.stdout: "replayed": 0 2024-05-09T05:02:08.232 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.232 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_mem": { 2024-05-09T05:02:08.232 INFO:teuthology.orchestra.run.smithi072.stdout: "ino": 17, 2024-05-09T05:02:08.233 INFO:teuthology.orchestra.run.smithi072.stdout: "ino+": 17, 2024-05-09T05:02:08.233 INFO:teuthology.orchestra.run.smithi072.stdout: "ino-": 0, 2024-05-09T05:02:08.233 INFO:teuthology.orchestra.run.smithi072.stdout: "dir": 16, 2024-05-09T05:02:08.233 INFO:teuthology.orchestra.run.smithi072.stdout: "dir+": 18, 2024-05-09T05:02:08.233 INFO:teuthology.orchestra.run.smithi072.stdout: "dir-": 2, 2024-05-09T05:02:08.233 INFO:teuthology.orchestra.run.smithi072.stdout: "dn": 14, 2024-05-09T05:02:08.233 INFO:teuthology.orchestra.run.smithi072.stdout: "dn+": 14, 2024-05-09T05:02:08.233 INFO:teuthology.orchestra.run.smithi072.stdout: "dn-": 0, 2024-05-09T05:02:08.233 INFO:teuthology.orchestra.run.smithi072.stdout: "cap": 1, 2024-05-09T05:02:08.233 INFO:teuthology.orchestra.run.smithi072.stdout: "cap+": 5, 2024-05-09T05:02:08.233 INFO:teuthology.orchestra.run.smithi072.stdout: "cap-": 4, 2024-05-09T05:02:08.233 INFO:teuthology.orchestra.run.smithi072.stdout: "rss": 121688, 2024-05-09T05:02:08.233 INFO:teuthology.orchestra.run.smithi072.stdout: "heap": 223632 2024-05-09T05:02:08.233 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.233 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_server": { 2024-05-09T05:02:08.234 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_client_request": 15, 2024-05-09T05:02:08.234 INFO:teuthology.orchestra.run.smithi072.stdout: "dispatch_server_request": 0, 2024-05-09T05:02:08.234 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_request": 12, 2024-05-09T05:02:08.234 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_client_session": 26, 2024-05-09T05:02:08.234 INFO:teuthology.orchestra.run.smithi072.stdout: "handle_peer_request": 1, 2024-05-09T05:02:08.234 INFO:teuthology.orchestra.run.smithi072.stdout: "req_create_latency": { 2024-05-09T05:02:08.234 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.234 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.234 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.235 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.235 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getattr_latency": { 2024-05-09T05:02:08.235 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 5, 2024-05-09T05:02:08.235 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.002306802, 2024-05-09T05:02:08.235 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000461360 2024-05-09T05:02:08.235 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.235 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getfilelock_latency": { 2024-05-09T05:02:08.235 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.235 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.235 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.235 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.235 INFO:teuthology.orchestra.run.smithi072.stdout: "req_link_latency": { 2024-05-09T05:02:08.235 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.235 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.235 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.236 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.236 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookup_latency": { 2024-05-09T05:02:08.236 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 2, 2024-05-09T05:02:08.236 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000519637, 2024-05-09T05:02:08.236 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000259818 2024-05-09T05:02:08.236 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.236 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookuphash_latency": { 2024-05-09T05:02:08.236 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.236 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.236 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.236 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.236 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupino_latency": { 2024-05-09T05:02:08.236 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.236 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.237 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.237 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupname_latency": { 2024-05-09T05:02:08.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.237 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.237 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.237 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupparent_latency": { 2024-05-09T05:02:08.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.237 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.237 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.237 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lookupsnap_latency": { 2024-05-09T05:02:08.237 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.238 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.238 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.238 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.238 INFO:teuthology.orchestra.run.smithi072.stdout: "req_lssnap_latency": { 2024-05-09T05:02:08.238 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.238 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.238 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.238 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.238 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mkdir_latency": { 2024-05-09T05:02:08.238 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.239 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.239 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.239 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.239 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mknod_latency": { 2024-05-09T05:02:08.239 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.239 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.239 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.239 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.240 INFO:teuthology.orchestra.run.smithi072.stdout: "req_mksnap_latency": { 2024-05-09T05:02:08.240 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.240 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.240 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.240 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.240 INFO:teuthology.orchestra.run.smithi072.stdout: "req_open_latency": { 2024-05-09T05:02:08.240 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.240 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.240 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.240 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.240 INFO:teuthology.orchestra.run.smithi072.stdout: "req_readdir_latency": { 2024-05-09T05:02:08.241 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.241 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.241 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.241 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.241 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rename_latency": { 2024-05-09T05:02:08.241 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.241 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.241 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.241 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.241 INFO:teuthology.orchestra.run.smithi072.stdout: "req_renamesnap_latency": { 2024-05-09T05:02:08.241 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.241 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.241 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.241 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.241 INFO:teuthology.orchestra.run.smithi072.stdout: "req_snapdiff_latency": { 2024-05-09T05:02:08.242 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.242 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.242 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.242 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.242 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmdir_latency": { 2024-05-09T05:02:08.242 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.242 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.242 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.242 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.242 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmsnap_latency": { 2024-05-09T05:02:08.242 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.242 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.242 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: "req_rmxattr_latency": { 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setattr_latency": { 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setdirlayout_latency": { 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.243 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.244 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setfilelock_latency": { 2024-05-09T05:02:08.244 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.244 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.244 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.244 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.244 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setlayout_latency": { 2024-05-09T05:02:08.244 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.244 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.244 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.244 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.244 INFO:teuthology.orchestra.run.smithi072.stdout: "req_setxattr_latency": { 2024-05-09T05:02:08.244 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.244 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.244 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.245 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.245 INFO:teuthology.orchestra.run.smithi072.stdout: "req_symlink_latency": { 2024-05-09T05:02:08.245 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.245 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.245 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.245 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.245 INFO:teuthology.orchestra.run.smithi072.stdout: "req_unlink_latency": { 2024-05-09T05:02:08.245 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.245 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.245 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.245 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.245 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_revoke_eviction": 0, 2024-05-09T05:02:08.245 INFO:teuthology.orchestra.run.smithi072.stdout: "cap_acquisition_throttle": 0, 2024-05-09T05:02:08.245 INFO:teuthology.orchestra.run.smithi072.stdout: "req_getvxattr_latency": { 2024-05-09T05:02:08.246 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.246 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.246 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.246 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:08.246 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.246 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_sessions": { 2024-05-09T05:02:08.246 INFO:teuthology.orchestra.run.smithi072.stdout: "session_count": 2, 2024-05-09T05:02:08.246 INFO:teuthology.orchestra.run.smithi072.stdout: "session_add": 4, 2024-05-09T05:02:08.246 INFO:teuthology.orchestra.run.smithi072.stdout: "session_remove": 2, 2024-05-09T05:02:08.246 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_open": 2, 2024-05-09T05:02:08.246 INFO:teuthology.orchestra.run.smithi072.stdout: "sessions_stale": 0, 2024-05-09T05:02:08.246 INFO:teuthology.orchestra.run.smithi072.stdout: "total_load": 5, 2024-05-09T05:02:08.246 INFO:teuthology.orchestra.run.smithi072.stdout: "average_load": 1, 2024-05-09T05:02:08.246 INFO:teuthology.orchestra.run.smithi072.stdout: "avg_session_uptime": 636, 2024-05-09T05:02:08.246 INFO:teuthology.orchestra.run.smithi072.stdout: "mdthresh_evicted": 0 2024-05-09T05:02:08.247 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.247 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-05-09T05:02:08.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-05-09T05:02:08.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-05-09T05:02:08.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T05:02:08.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-05-09T05:02:08.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T05:02:08.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-05-09T05:02:08.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T05:02:08.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T05:02:08.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T05:02:08.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T05:02:08.247 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T05:02:08.248 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-05-09T05:02:08.248 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T05:02:08.248 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T05:02:08.248 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-05-09T05:02:08.248 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-05-09T05:02:08.248 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-05-09T05:02:08.248 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-05-09T05:02:08.248 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T05:02:08.248 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T05:02:08.248 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T05:02:08.248 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T05:02:08.248 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T05:02:08.248 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-05-09T05:02:08.248 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 257735, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 102, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 88, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 1, 2024-05-09T05:02:08.249 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-05-09T05:02:08.250 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-05-09T05:02:08.250 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-05-09T05:02:08.250 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-05-09T05:02:08.250 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-05-09T05:02:08.250 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-05-09T05:02:08.250 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 8440, 2024-05-09T05:02:08.250 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 176, 2024-05-09T05:02:08.250 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T05:02:08.250 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-05-09T05:02:08.250 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-05-09T05:02:08.250 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-05-09T05:02:08.250 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 69472, 2024-05-09T05:02:08.250 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 263, 2024-05-09T05:02:08.250 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-05-09T05:02:08.251 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-05-09T05:02:08.251 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-05-09T05:02:08.251 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-05-09T05:02:08.251 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.251 INFO:teuthology.orchestra.run.smithi072.stdout: "objecter": { 2024-05-09T05:02:08.251 INFO:teuthology.orchestra.run.smithi072.stdout: "op_active": 0, 2024-05-09T05:02:08.251 INFO:teuthology.orchestra.run.smithi072.stdout: "op_laggy": 0, 2024-05-09T05:02:08.251 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send": 46, 2024-05-09T05:02:08.251 INFO:teuthology.orchestra.run.smithi072.stdout: "op_send_bytes": 12888, 2024-05-09T05:02:08.251 INFO:teuthology.orchestra.run.smithi072.stdout: "op_resend": 33, 2024-05-09T05:02:08.251 INFO:teuthology.orchestra.run.smithi072.stdout: "op_reply": 46, 2024-05-09T05:02:08.251 INFO:teuthology.orchestra.run.smithi072.stdout: "op_latency": { 2024-05-09T05:02:08.251 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 46, 2024-05-09T05:02:08.251 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.255994501, 2024-05-09T05:02:08.251 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.005565097 2024-05-09T05:02:08.252 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.252 INFO:teuthology.orchestra.run.smithi072.stdout: "op_inflight": 0, 2024-05-09T05:02:08.252 INFO:teuthology.orchestra.run.smithi072.stdout: "oplen_avg": { 2024-05-09T05:02:08.252 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 46, 2024-05-09T05:02:08.252 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 70 2024-05-09T05:02:08.252 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.252 INFO:teuthology.orchestra.run.smithi072.stdout: "op": 46, 2024-05-09T05:02:08.252 INFO:teuthology.orchestra.run.smithi072.stdout: "op_r": 0, 2024-05-09T05:02:08.252 INFO:teuthology.orchestra.run.smithi072.stdout: "op_w": 46, 2024-05-09T05:02:08.252 INFO:teuthology.orchestra.run.smithi072.stdout: "op_rmw": 0, 2024-05-09T05:02:08.252 INFO:teuthology.orchestra.run.smithi072.stdout: "op_pg": 0, 2024-05-09T05:02:08.252 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_stat": 0, 2024-05-09T05:02:08.252 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_create": 10, 2024-05-09T05:02:08.252 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_read": 0, 2024-05-09T05:02:08.253 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_write": 7, 2024-05-09T05:02:08.253 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writefull": 8, 2024-05-09T05:02:08.253 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_writesame": 0, 2024-05-09T05:02:08.253 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_append": 0, 2024-05-09T05:02:08.253 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_zero": 0, 2024-05-09T05:02:08.253 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_truncate": 0, 2024-05-09T05:02:08.253 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_delete": 6, 2024-05-09T05:02:08.253 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_mapext": 0, 2024-05-09T05:02:08.253 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_sparse_read": 0, 2024-05-09T05:02:08.253 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_clonerange": 0, 2024-05-09T05:02:08.253 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_getxattr": 0, 2024-05-09T05:02:08.253 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_setxattr": 20, 2024-05-09T05:02:08.253 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_cmpxattr": 0, 2024-05-09T05:02:08.253 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_rmxattr": 0, 2024-05-09T05:02:08.253 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_resetxattrs": 0, 2024-05-09T05:02:08.254 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_call": 0, 2024-05-09T05:02:08.254 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_watch": 0, 2024-05-09T05:02:08.254 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_notify": 0, 2024-05-09T05:02:08.254 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T05:02:08.254 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls": 0, 2024-05-09T05:02:08.254 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_pgls_filter": 0, 2024-05-09T05:02:08.254 INFO:teuthology.orchestra.run.smithi072.stdout: "osdop_other": 0, 2024-05-09T05:02:08.254 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_active": 0, 2024-05-09T05:02:08.254 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_send": 0, 2024-05-09T05:02:08.254 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_resend": 0, 2024-05-09T05:02:08.254 INFO:teuthology.orchestra.run.smithi072.stdout: "linger_ping": 0, 2024-05-09T05:02:08.254 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_active": 0, 2024-05-09T05:02:08.254 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_send": 0, 2024-05-09T05:02:08.254 INFO:teuthology.orchestra.run.smithi072.stdout: "poolop_resend": 0, 2024-05-09T05:02:08.255 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_active": 0, 2024-05-09T05:02:08.255 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_send": 0, 2024-05-09T05:02:08.255 INFO:teuthology.orchestra.run.smithi072.stdout: "poolstat_resend": 0, 2024-05-09T05:02:08.255 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_active": 0, 2024-05-09T05:02:08.255 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_send": 0, 2024-05-09T05:02:08.255 INFO:teuthology.orchestra.run.smithi072.stdout: "statfs_resend": 0, 2024-05-09T05:02:08.255 INFO:teuthology.orchestra.run.smithi072.stdout: "command_active": 0, 2024-05-09T05:02:08.255 INFO:teuthology.orchestra.run.smithi072.stdout: "command_send": 0, 2024-05-09T05:02:08.255 INFO:teuthology.orchestra.run.smithi072.stdout: "command_resend": 0, 2024-05-09T05:02:08.255 INFO:teuthology.orchestra.run.smithi072.stdout: "map_epoch": 94, 2024-05-09T05:02:08.255 INFO:teuthology.orchestra.run.smithi072.stdout: "map_full": 0, 2024-05-09T05:02:08.255 INFO:teuthology.orchestra.run.smithi072.stdout: "map_inc": 14, 2024-05-09T05:02:08.255 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_sessions": 9, 2024-05-09T05:02:08.255 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_open": 9, 2024-05-09T05:02:08.256 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_session_close": 0, 2024-05-09T05:02:08.256 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_laggy": 0, 2024-05-09T05:02:08.256 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_wr": 17, 2024-05-09T05:02:08.256 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_rd": 0, 2024-05-09T05:02:08.256 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_del": 2 2024-05-09T05:02:08.256 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.256 INFO:teuthology.orchestra.run.smithi072.stdout: "oft": { 2024-05-09T05:02:08.256 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_objs": 1, 2024-05-09T05:02:08.256 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_kv_pairs": 0, 2024-05-09T05:02:08.256 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_updates": 0, 2024-05-09T05:02:08.256 INFO:teuthology.orchestra.run.smithi072.stdout: "omap_total_removes": 0 2024-05-09T05:02:08.256 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.256 INFO:teuthology.orchestra.run.smithi072.stdout: "osd-slow-ops": { 2024-05-09T05:02:08.256 INFO:teuthology.orchestra.run.smithi072.stdout: "slow_ops_count": 0 2024-05-09T05:02:08.256 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.257 INFO:teuthology.orchestra.run.smithi072.stdout: "purge_queue": { 2024-05-09T05:02:08.257 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops": 0, 2024-05-09T05:02:08.257 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T05:02:08.257 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing": 0, 2024-05-09T05:02:08.257 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executing_high_water": 0, 2024-05-09T05:02:08.257 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed_ops": 0, 2024-05-09T05:02:08.257 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_executed": 0, 2024-05-09T05:02:08.257 INFO:teuthology.orchestra.run.smithi072.stdout: "pq_item_in_journal": 0 2024-05-09T05:02:08.257 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.257 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T05:02:08.257 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-09T05:02:08.257 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-05-09T05:02:08.257 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:08.257 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 943, 2024-05-09T05:02:08.257 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 1831995, 2024-05-09T05:02:08.258 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:08.258 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 943, 2024-05-09T05:02:08.258 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-09T05:02:08.258 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-09T05:02:08.258 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 943, 2024-05-09T05:02:08.258 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 1831995, 2024-05-09T05:02:08.258 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:08.258 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.258 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.258 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.258 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:08.258 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.258 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_bytes": { 2024-05-09T05:02:08.258 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-09T05:02:08.258 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-05-09T05:02:08.259 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:08.259 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-09T05:02:08.259 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-09T05:02:08.259 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:08.259 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:08.259 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 46, 2024-05-09T05:02:08.259 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 12888, 2024-05-09T05:02:08.259 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 46, 2024-05-09T05:02:08.259 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 12888, 2024-05-09T05:02:08.259 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:08.259 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.259 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.259 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.259 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:08.259 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.260 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-objecter_ops": { 2024-05-09T05:02:08.260 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-09T05:02:08.260 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 1024, 2024-05-09T05:02:08.260 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:08.260 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-09T05:02:08.260 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-09T05:02:08.260 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:08.260 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:08.260 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 46, 2024-05-09T05:02:08.260 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 46, 2024-05-09T05:02:08.260 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 46, 2024-05-09T05:02:08.260 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 46, 2024-05-09T05:02:08.260 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:08.260 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.260 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.261 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.261 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:08.261 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.261 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle": { 2024-05-09T05:02:08.261 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-09T05:02:08.261 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-05-09T05:02:08.261 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:08.261 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 0, 2024-05-09T05:02:08.261 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 0, 2024-05-09T05:02:08.261 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:08.261 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:08.261 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-09T05:02:08.261 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-09T05:02:08.261 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 0, 2024-05-09T05:02:08.262 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 0, 2024-05-09T05:02:08.262 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:08.262 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.262 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.262 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.262 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:08.262 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:08.262 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-write_buf_throttle-0x5586d232c680": { 2024-05-09T05:02:08.262 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-09T05:02:08.262 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 3758096384, 2024-05-09T05:02:08.262 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:08.262 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 8, 2024-05-09T05:02:08.262 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 2351, 2024-05-09T05:02:08.262 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:08.262 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 8, 2024-05-09T05:02:08.263 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-09T05:02:08.263 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-09T05:02:08.263 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 7, 2024-05-09T05:02:08.263 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 2351, 2024-05-09T05:02:08.263 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:08.263 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:08.263 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:08.263 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:08.263 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:08.263 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:08.263 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-05-09T05:02:08.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:07 smithi110 ceph-mon[34904]: osdmap e94: 12 total, 12 up, 12 in 2024-05-09T05:02:08.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:07 smithi046 ceph-mon[30136]: osdmap e94: 12 total, 12 up, 12 in 2024-05-09T05:02:09.030 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:08 smithi072 ceph-mon[34088]: pgmap v343: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T05:02:09.056 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-05-09T05:02:09.056 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-05-09T05:02:09.056 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-05-09T05:02:09.056 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=8 2024-05-09T05:02:09.056 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-05-09T05:02:09.056 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-09T05:02:09.056 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T05:02:09.057 DEBUG:tasks.check_counter:Getting stats from k 2024-05-09T05:02:09.057 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-09T05:02:09.293 INFO:teuthology.orchestra.run.smithi072.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.b/config 2024-05-09T05:02:09.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:08 smithi110 ceph-mon[34904]: pgmap v343: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T05:02:09.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:08 smithi046 ceph-mon[30136]: pgmap v343: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T05:02:11.085 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:10 smithi072 ceph-mon[34088]: pgmap v344: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T05:02:11.121 INFO:teuthology.orchestra.run.smithi072.stdout:{ 2024-05-09T05:02:11.121 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T05:02:11.121 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 0, 2024-05-09T05:02:11.121 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 2, 2024-05-09T05:02:11.121 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 0, 2024-05-09T05:02:11.121 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 0, 2024-05-09T05:02:11.121 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 4, 2024-05-09T05:02:11.121 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 0, 2024-05-09T05:02:11.121 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.001038696, 2024-05-09T05:02:11.122 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T05:02:11.122 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.000252913, 2024-05-09T05:02:11.122 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T05:02:11.122 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:11.122 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:11.122 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:11.122 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:11.122 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:11.122 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:11.122 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:11.122 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:11.122 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:11.122 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:11.122 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T05:02:11.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T05:02:11.123 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:11.123 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T05:02:11.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 255, 2024-05-09T05:02:11.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 244, 2024-05-09T05:02:11.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 1720448, 2024-05-09T05:02:11.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 150160, 2024-05-09T05:02:11.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 2, 2024-05-09T05:02:11.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 1, 2024-05-09T05:02:11.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.055960861, 2024-05-09T05:02:11.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.016626788, 2024-05-09T05:02:11.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.043915652, 2024-05-09T05:02:11.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.006632716, 2024-05-09T05:02:11.123 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:11.123 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 243, 2024-05-09T05:02:11.124 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.014782793, 2024-05-09T05:02:11.124 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000060834 2024-05-09T05:02:11.124 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:11.124 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:11.124 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:11.124 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:11.124 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:11.124 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:11.124 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 1720448, 2024-05-09T05:02:11.124 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 150160 2024-05-09T05:02:11.124 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:11.124 INFO:teuthology.orchestra.run.smithi072.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T05:02:11.124 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_messages": 6, 2024-05-09T05:02:11.124 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages": 187, 2024-05-09T05:02:11.125 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_bytes": 3312, 2024-05-09T05:02:11.125 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_bytes": 190368, 2024-05-09T05:02:11.125 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_created_connections": 5, 2024-05-09T05:02:11.125 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_active_connections": 3, 2024-05-09T05:02:11.125 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_total_time": 0.013659822, 2024-05-09T05:02:11.125 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_send_time": 0.008943701, 2024-05-09T05:02:11.125 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_recv_time": 0.002157250, 2024-05-09T05:02:11.125 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T05:02:11.125 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:11.125 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 185, 2024-05-09T05:02:11.125 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.015423583, 2024-05-09T05:02:11.125 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000083370 2024-05-09T05:02:11.125 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:11.125 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:11.125 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:11.126 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:11.126 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:11.126 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:11.126 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-09T05:02:11.126 INFO:teuthology.orchestra.run.smithi072.stdout: "msgr_send_encrypted_bytes": 190368 2024-05-09T05:02:11.126 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:11.126 INFO:teuthology.orchestra.run.smithi072.stdout: "cct": { 2024-05-09T05:02:11.126 INFO:teuthology.orchestra.run.smithi072.stdout: "total_workers": 0, 2024-05-09T05:02:11.126 INFO:teuthology.orchestra.run.smithi072.stdout: "unhealthy_workers": 0 2024-05-09T05:02:11.126 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:11.126 INFO:teuthology.orchestra.run.smithi072.stdout: "mempool": { 2024-05-09T05:02:11.126 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_bytes": 0, 2024-05-09T05:02:11.126 INFO:teuthology.orchestra.run.smithi072.stdout: "bloom_filter_items": 0, 2024-05-09T05:02:11.126 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T05:02:11.126 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_alloc_items": 0, 2024-05-09T05:02:11.127 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T05:02:11.127 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_data_items": 0, 2024-05-09T05:02:11.127 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T05:02:11.127 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T05:02:11.127 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T05:02:11.127 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T05:02:11.127 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T05:02:11.127 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_other_items": 0, 2024-05-09T05:02:11.127 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T05:02:11.127 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T05:02:11.127 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_bytes": 0, 2024-05-09T05:02:11.127 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_extent_items": 0, 2024-05-09T05:02:11.127 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_bytes": 0, 2024-05-09T05:02:11.127 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_blob_items": 0, 2024-05-09T05:02:11.127 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_fsck_items": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_bytes": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_txc_items": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_bytes": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluestore_writing_items": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_bytes": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_items": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_reader_items": 0, 2024-05-09T05:02:11.128 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T05:02:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: "bluefs_file_writer_items": 0, 2024-05-09T05:02:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_bytes": 148754, 2024-05-09T05:02:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_anon_items": 31, 2024-05-09T05:02:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_bytes": 0, 2024-05-09T05:02:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: "buffer_meta_items": 0, 2024-05-09T05:02:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_bytes": 0, 2024-05-09T05:02:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_items": 0, 2024-05-09T05:02:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_bytes": 0, 2024-05-09T05:02:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_mapbl_items": 0, 2024-05-09T05:02:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_bytes": 0, 2024-05-09T05:02:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: "osd_pglog_items": 0, 2024-05-09T05:02:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_bytes": 0, 2024-05-09T05:02:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_items": 0, 2024-05-09T05:02:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T05:02:11.129 INFO:teuthology.orchestra.run.smithi072.stdout: "osdmap_mapping_items": 0, 2024-05-09T05:02:11.130 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_bytes": 0, 2024-05-09T05:02:11.130 INFO:teuthology.orchestra.run.smithi072.stdout: "pgmap_items": 0, 2024-05-09T05:02:11.130 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_bytes": 928, 2024-05-09T05:02:11.130 INFO:teuthology.orchestra.run.smithi072.stdout: "mds_co_items": 2, 2024-05-09T05:02:11.130 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_bytes": 0, 2024-05-09T05:02:11.130 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_1_items": 0, 2024-05-09T05:02:11.130 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_bytes": 0, 2024-05-09T05:02:11.130 INFO:teuthology.orchestra.run.smithi072.stdout: "unittest_2_items": 0 2024-05-09T05:02:11.130 INFO:teuthology.orchestra.run.smithi072.stdout: }, 2024-05-09T05:02:11.130 INFO:teuthology.orchestra.run.smithi072.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T05:02:11.131 INFO:teuthology.orchestra.run.smithi072.stdout: "val": 0, 2024-05-09T05:02:11.131 INFO:teuthology.orchestra.run.smithi072.stdout: "max": 104857600, 2024-05-09T05:02:11.131 INFO:teuthology.orchestra.run.smithi072.stdout: "get_started": 0, 2024-05-09T05:02:11.131 INFO:teuthology.orchestra.run.smithi072.stdout: "get": 257, 2024-05-09T05:02:11.131 INFO:teuthology.orchestra.run.smithi072.stdout: "get_sum": 1684460, 2024-05-09T05:02:11.131 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:11.131 INFO:teuthology.orchestra.run.smithi072.stdout: "get_or_fail_success": 257, 2024-05-09T05:02:11.131 INFO:teuthology.orchestra.run.smithi072.stdout: "take": 0, 2024-05-09T05:02:11.131 INFO:teuthology.orchestra.run.smithi072.stdout: "take_sum": 0, 2024-05-09T05:02:11.131 INFO:teuthology.orchestra.run.smithi072.stdout: "put": 257, 2024-05-09T05:02:11.131 INFO:teuthology.orchestra.run.smithi072.stdout: "put_sum": 1684460, 2024-05-09T05:02:11.131 INFO:teuthology.orchestra.run.smithi072.stdout: "wait": { 2024-05-09T05:02:11.131 INFO:teuthology.orchestra.run.smithi072.stdout: "avgcount": 0, 2024-05-09T05:02:11.131 INFO:teuthology.orchestra.run.smithi072.stdout: "sum": 0.000000000, 2024-05-09T05:02:11.131 INFO:teuthology.orchestra.run.smithi072.stdout: "avgtime": 0.000000000 2024-05-09T05:02:11.132 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:11.132 INFO:teuthology.orchestra.run.smithi072.stdout: } 2024-05-09T05:02:11.132 INFO:teuthology.orchestra.run.smithi072.stdout:} 2024-05-09T05:02:11.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:10 smithi110 ceph-mon[34904]: pgmap v344: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T05:02:11.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:10 smithi046 ceph-mon[30136]: pgmap v344: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 53 KiB/s wr, 26 op/s 2024-05-09T05:02:11.864 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2024-05-09T05:02:11.864 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2024-05-09T05:02:11.864 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2024-05-09T05:02:11.864 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2024-05-09T05:02:11.864 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-05-09T05:02:11.864 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-05-09T05:02:11.864 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-05-09T05:02:11.865 DEBUG:tasks.check_counter:Getting stats from c 2024-05-09T05:02:11.865 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-09T05:02:12.118 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:02:12.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:11 smithi072 ceph-mon[34088]: pgmap v345: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-05-09T05:02:12.240 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:11 smithi110 ceph-mon[34904]: pgmap v345: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-05-09T05:02:12.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:11 smithi046 ceph-mon[30136]: pgmap v345: 97 pgs: 97 active+clean; 3.9 MiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 26 KiB/s wr, 23 op/s 2024-05-09T05:02:14.033 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-05-09T05:02:14.033 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T05:02:14.033 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 2368, 2024-05-09T05:02:14.033 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 2370, 2024-05-09T05:02:14.034 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 675072, 2024-05-09T05:02:14.034 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 753031, 2024-05-09T05:02:14.034 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 6, 2024-05-09T05:02:14.034 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 2, 2024-05-09T05:02:14.034 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.227464138, 2024-05-09T05:02:14.034 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.073099414, 2024-05-09T05:02:14.034 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.058217702, 2024-05-09T05:02:14.034 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.078009627, 2024-05-09T05:02:14.034 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:14.034 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2368, 2024-05-09T05:02:14.034 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.097503857, 2024-05-09T05:02:14.034 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000041175 2024-05-09T05:02:14.034 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.034 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:14.035 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.035 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.035 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.035 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T05:02:14.035 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T05:02:14.035 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.035 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T05:02:14.035 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 4, 2024-05-09T05:02:14.035 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 185, 2024-05-09T05:02:14.035 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 876, 2024-05-09T05:02:14.035 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 480314, 2024-05-09T05:02:14.035 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 6, 2024-05-09T05:02:14.035 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2024-05-09T05:02:14.035 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.011394040, 2024-05-09T05:02:14.036 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.007093485, 2024-05-09T05:02:14.036 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.002198624, 2024-05-09T05:02:14.036 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000077927, 2024-05-09T05:02:14.036 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:14.036 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 182, 2024-05-09T05:02:14.036 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.014805516, 2024-05-09T05:02:14.036 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000081348 2024-05-09T05:02:14.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.036 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:14.036 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.036 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.036 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.036 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.036 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-09T05:02:14.037 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 479616 2024-05-09T05:02:14.037 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.037 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T05:02:14.037 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 276, 2024-05-09T05:02:14.037 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 261, 2024-05-09T05:02:14.037 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 1752316, 2024-05-09T05:02:14.037 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 151326, 2024-05-09T05:02:14.037 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 3, 2024-05-09T05:02:14.037 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 2, 2024-05-09T05:02:14.037 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.058127664, 2024-05-09T05:02:14.037 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.016867574, 2024-05-09T05:02:14.037 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.047235532, 2024-05-09T05:02:14.037 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.006323446, 2024-05-09T05:02:14.037 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:14.037 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 261, 2024-05-09T05:02:14.038 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.015856575, 2024-05-09T05:02:14.038 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000060753 2024-05-09T05:02:14.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.038 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:14.038 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.038 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.038 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.038 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 1752048, 2024-05-09T05:02:14.038 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 151008 2024-05-09T05:02:14.038 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.038 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-05-09T05:02:14.039 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-05-09T05:02:14.039 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-05-09T05:02:14.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.039 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-05-09T05:02:14.039 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-05-09T05:02:14.039 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-05-09T05:02:14.039 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 4064, 2024-05-09T05:02:14.039 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.125088688, 2024-05-09T05:02:14.039 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000030779 2024-05-09T05:02:14.039 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:14.039 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.039 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-05-09T05:02:14.039 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-05-09T05:02:14.039 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-05-09T05:02:14.039 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-05-09T05:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 0, 2024-05-09T05:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 0, 2024-05-09T05:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-05-09T05:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-05-09T05:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 0, 2024-05-09T05:02:14.040 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-05-09T05:02:14.041 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-05-09T05:02:14.041 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 0, 2024-05-09T05:02:14.041 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2024-05-09T05:02:14.041 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-05-09T05:02:14.041 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 0, 2024-05-09T05:02:14.041 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 0, 2024-05-09T05:02:14.041 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 0, 2024-05-09T05:02:14.041 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 0, 2024-05-09T05:02:14.041 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 0, 2024-05-09T05:02:14.041 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 0, 2024-05-09T05:02:14.041 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 0, 2024-05-09T05:02:14.041 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 0, 2024-05-09T05:02:14.041 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 1, 2024-05-09T05:02:14.041 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 0, 2024-05-09T05:02:14.041 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 0, 2024-05-09T05:02:14.042 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 0, 2024-05-09T05:02:14.042 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 0, 2024-05-09T05:02:14.042 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-05-09T05:02:14.042 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-05-09T05:02:14.042 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 0, 2024-05-09T05:02:14.042 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 0, 2024-05-09T05:02:14.042 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-05-09T05:02:14.042 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 0, 2024-05-09T05:02:14.042 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 0, 2024-05-09T05:02:14.042 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 0, 2024-05-09T05:02:14.042 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 0, 2024-05-09T05:02:14.042 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-05-09T05:02:14.042 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-05-09T05:02:14.042 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T05:02:14.043 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T05:02:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T05:02:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 0, 2024-05-09T05:02:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 0, 2024-05-09T05:02:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 0, 2024-05-09T05:02:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 0 2024-05-09T05:02:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-05-09T05:02:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 0, 2024-05-09T05:02:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 0, 2024-05-09T05:02:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 0, 2024-05-09T05:02:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 0, 2024-05-09T05:02:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 0, 2024-05-09T05:02:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2024-05-09T05:02:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-05-09T05:02:14.044 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-05-09T05:02:14.045 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 0, 2024-05-09T05:02:14.045 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 0, 2024-05-09T05:02:14.045 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-05-09T05:02:14.045 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-05-09T05:02:14.045 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-05-09T05:02:14.045 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-05-09T05:02:14.045 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-05-09T05:02:14.045 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-05-09T05:02:14.045 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-05-09T05:02:14.045 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-05-09T05:02:14.045 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-05-09T05:02:14.045 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T05:02:14.045 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 0, 2024-05-09T05:02:14.045 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-05-09T05:02:14.045 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2024-05-09T05:02:14.046 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-05-09T05:02:14.046 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0 2024-05-09T05:02:14.046 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.046 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-05-09T05:02:14.046 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-05-09T05:02:14.046 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 0, 2024-05-09T05:02:14.046 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2024-05-09T05:02:14.046 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2024-05-09T05:02:14.046 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 8, 2024-05-09T05:02:14.046 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-05-09T05:02:14.046 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2024-05-09T05:02:14.046 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 0, 2024-05-09T05:02:14.046 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2024-05-09T05:02:14.046 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2024-05-09T05:02:14.046 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 2, 2024-05-09T05:02:14.047 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 2, 2024-05-09T05:02:14.047 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-05-09T05:02:14.047 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2024-05-09T05:02:14.047 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2024-05-09T05:02:14.047 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 4196655, 2024-05-09T05:02:14.047 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 4196514, 2024-05-09T05:02:14.047 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-05-09T05:02:14.047 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.047 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.047 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.047 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 8 2024-05-09T05:02:14.047 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.047 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-05-09T05:02:14.047 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 3, 2024-05-09T05:02:14.048 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 3, 2024-05-09T05:02:14.048 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 0, 2024-05-09T05:02:14.048 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 1, 2024-05-09T05:02:14.048 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 1, 2024-05-09T05:02:14.048 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 0, 2024-05-09T05:02:14.048 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 0, 2024-05-09T05:02:14.048 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 0, 2024-05-09T05:02:14.048 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 0, 2024-05-09T05:02:14.048 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 0, 2024-05-09T05:02:14.048 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 0, 2024-05-09T05:02:14.048 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 0, 2024-05-09T05:02:14.048 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 118212, 2024-05-09T05:02:14.048 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 215440 2024-05-09T05:02:14.048 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.048 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 0, 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 0, 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 0, 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 0, 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 0, 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.049 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-05-09T05:02:14.050 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.050 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.050 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.050 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.050 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-05-09T05:02:14.050 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.050 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.050 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.051 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-05-09T05:02:14.051 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.051 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.051 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.051 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-05-09T05:02:14.051 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.051 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.051 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.051 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.051 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-05-09T05:02:14.051 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.051 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.052 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.053 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-05-09T05:02:14.053 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.053 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.053 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.053 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.053 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-05-09T05:02:14.053 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.054 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.054 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.054 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-05-09T05:02:14.054 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.054 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.054 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.054 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.054 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-05-09T05:02:14.054 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.055 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.055 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.055 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-05-09T05:02:14.055 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.055 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.055 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.055 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.055 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-05-09T05:02:14.055 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.055 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.055 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.056 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-05-09T05:02:14.056 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.056 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.056 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.056 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-05-09T05:02:14.056 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.056 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.056 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.056 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.056 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-05-09T05:02:14.056 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.056 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.056 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.057 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-05-09T05:02:14.057 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.057 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.057 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.057 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-05-09T05:02:14.057 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.057 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.057 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.057 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.057 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-05-09T05:02:14.057 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.057 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.057 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.058 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-05-09T05:02:14.058 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.058 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.058 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.058 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-05-09T05:02:14.058 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.058 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.058 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.058 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.058 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-05-09T05:02:14.058 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.058 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.059 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.060 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-05-09T05:02:14.060 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.060 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.060 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.060 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-05-09T05:02:14.060 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-05-09T05:02:14.060 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-05-09T05:02:14.060 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.060 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.060 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.060 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:14.060 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.060 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-05-09T05:02:14.061 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 2, 2024-05-09T05:02:14.061 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 4, 2024-05-09T05:02:14.061 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 2, 2024-05-09T05:02:14.061 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 4, 2024-05-09T05:02:14.061 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-05-09T05:02:14.061 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 0, 2024-05-09T05:02:14.061 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 0, 2024-05-09T05:02:14.061 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 3585, 2024-05-09T05:02:14.061 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-05-09T05:02:14.061 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.061 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-05-09T05:02:14.061 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-05-09T05:02:14.061 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-05-09T05:02:14.061 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T05:02:14.061 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-05-09T05:02:14.062 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T05:02:14.062 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-05-09T05:02:14.062 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T05:02:14.062 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T05:02:14.062 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T05:02:14.062 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T05:02:14.062 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T05:02:14.062 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-05-09T05:02:14.062 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T05:02:14.062 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T05:02:14.062 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-05-09T05:02:14.062 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-05-09T05:02:14.062 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-05-09T05:02:14.062 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T05:02:14.063 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 182088, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 48, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8440, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 176, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T05:02:14.064 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-05-09T05:02:14.065 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-05-09T05:02:14.065 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-05-09T05:02:14.065 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 8512, 2024-05-09T05:02:14.065 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 13, 2024-05-09T05:02:14.065 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-05-09T05:02:14.065 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-05-09T05:02:14.065 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-05-09T05:02:14.065 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-05-09T05:02:14.065 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.065 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-05-09T05:02:14.065 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-05-09T05:02:14.065 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-05-09T05:02:14.065 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 2371, 2024-05-09T05:02:14.065 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 21, 2024-05-09T05:02:14.065 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 3, 2024-05-09T05:02:14.066 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 2371, 2024-05-09T05:02:14.066 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-05-09T05:02:14.066 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2371, 2024-05-09T05:02:14.066 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.332976549, 2024-05-09T05:02:14.066 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000562200 2024-05-09T05:02:14.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.066 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-05-09T05:02:14.066 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-05-09T05:02:14.066 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2371, 2024-05-09T05:02:14.066 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 2372 2024-05-09T05:02:14.066 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.066 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 2371, 2024-05-09T05:02:14.066 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 2371, 2024-05-09T05:02:14.066 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 0, 2024-05-09T05:02:14.066 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-05-09T05:02:14.067 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-05-09T05:02:14.067 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 1516, 2024-05-09T05:02:14.067 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 0, 2024-05-09T05:02:14.067 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 853, 2024-05-09T05:02:14.067 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 0, 2024-05-09T05:02:14.067 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 0, 2024-05-09T05:02:14.067 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-05-09T05:02:14.067 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-05-09T05:02:14.067 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-05-09T05:02:14.067 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2024-05-09T05:02:14.067 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 0, 2024-05-09T05:02:14.067 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-05-09T05:02:14.067 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-05-09T05:02:14.067 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-05-09T05:02:14.067 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 1, 2024-05-09T05:02:14.068 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 0, 2024-05-09T05:02:14.068 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-05-09T05:02:14.068 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-05-09T05:02:14.068 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-05-09T05:02:14.068 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-05-09T05:02:14.068 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-05-09T05:02:14.068 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-05-09T05:02:14.068 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T05:02:14.068 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-05-09T05:02:14.068 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-05-09T05:02:14.068 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2024-05-09T05:02:14.068 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-05-09T05:02:14.068 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-05-09T05:02:14.068 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-05-09T05:02:14.068 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 94, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 14, 2024-05-09T05:02:14.069 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 4, 2024-05-09T05:02:14.070 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 4, 2024-05-09T05:02:14.070 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-05-09T05:02:14.070 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-05-09T05:02:14.070 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 0, 2024-05-09T05:02:14.070 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 2, 2024-05-09T05:02:14.070 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 0 2024-05-09T05:02:14.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.070 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-05-09T05:02:14.070 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 0, 2024-05-09T05:02:14.070 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-05-09T05:02:14.070 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-05-09T05:02:14.070 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-05-09T05:02:14.070 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.070 INFO:teuthology.orchestra.run.smithi110.stdout: "osd-slow-ops": { 2024-05-09T05:02:14.070 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 0, 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 0, 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 0, 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 2644, 2024-05-09T05:02:14.071 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 2173381, 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 2644, 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 2644, 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 2173381, 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-05-09T05:02:14.072 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-09T05:02:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-09T05:02:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 2371, 2024-05-09T05:02:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 2356, 2024-05-09T05:02:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 2371, 2024-05-09T05:02:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 2356, 2024-05-09T05:02:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.073 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-05-09T05:02:14.074 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:14.074 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-05-09T05:02:14.074 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:14.074 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-09T05:02:14.074 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-09T05:02:14.074 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:14.074 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:14.074 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 2371, 2024-05-09T05:02:14.074 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 2371, 2024-05-09T05:02:14.074 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 2371, 2024-05-09T05:02:14.074 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 2371, 2024-05-09T05:02:14.074 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:14.074 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.074 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.074 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:14.075 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x56499aec2410": { 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-05-09T05:02:14.076 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-05-09T05:02:14.077 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:14.077 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:14.077 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:14.077 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:14.077 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:14.077 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:14.077 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-05-09T05:02:14.998 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-09T05:02:14.998 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-09T05:02:14.998 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-05-09T05:02:14.998 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-05-09T05:02:14.998 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2024-05-09T05:02:14.998 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-09T05:02:14.998 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T05:02:14.999 DEBUG:tasks.check_counter:Getting stats from f 2024-05-09T05:02:14.999 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-09T05:02:15.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:14 smithi072 ceph-mon[34088]: pgmap v346: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:02:15.242 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:02:15.269 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:14 smithi110 ceph-mon[34904]: pgmap v346: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:02:15.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:14 smithi046 ceph-mon[30136]: pgmap v346: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:02:16.946 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:16 smithi110 ceph-mon[34904]: pgmap v347: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:02:16.946 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:16 smithi110 ceph-mon[34904]: osdmap e95: 12 total, 12 up, 12 in 2024-05-09T05:02:17.093 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-05-09T05:02:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T05:02:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 857, 2024-05-09T05:02:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 859, 2024-05-09T05:02:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 227412, 2024-05-09T05:02:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 272590, 2024-05-09T05:02:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 6, 2024-05-09T05:02:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 2, 2024-05-09T05:02:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.090063118, 2024-05-09T05:02:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.027875199, 2024-05-09T05:02:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.024722772, 2024-05-09T05:02:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.028196938, 2024-05-09T05:02:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:17.094 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 857, 2024-05-09T05:02:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.035701404, 2024-05-09T05:02:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000041658 2024-05-09T05:02:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T05:02:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T05:02:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T05:02:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 746, 2024-05-09T05:02:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 927, 2024-05-09T05:02:17.095 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 185374, 2024-05-09T05:02:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 717357, 2024-05-09T05:02:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 7, 2024-05-09T05:02:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 5, 2024-05-09T05:02:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.096611181, 2024-05-09T05:02:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.029690776, 2024-05-09T05:02:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.022937568, 2024-05-09T05:02:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.034648678, 2024-05-09T05:02:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 925, 2024-05-09T05:02:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.047725267, 2024-05-09T05:02:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000051594 2024-05-09T05:02:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:17.096 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-09T05:02:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 482032 2024-05-09T05:02:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T05:02:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 1122, 2024-05-09T05:02:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 1109, 2024-05-09T05:02:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 2035552, 2024-05-09T05:02:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 421698, 2024-05-09T05:02:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 3, 2024-05-09T05:02:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 2, 2024-05-09T05:02:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.170103830, 2024-05-09T05:02:17.097 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.049457874, 2024-05-09T05:02:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.078576815, 2024-05-09T05:02:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.045497075, 2024-05-09T05:02:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1108, 2024-05-09T05:02:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.065501510, 2024-05-09T05:02:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000059116 2024-05-09T05:02:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.098 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 1750960, 2024-05-09T05:02:17.099 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 152352 2024-05-09T05:02:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-05-09T05:02:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-05-09T05:02:17.100 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-05-09T05:02:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-05-09T05:02:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-05-09T05:02:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-05-09T05:02:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 4099, 2024-05-09T05:02:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.135889001, 2024-05-09T05:02:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000033151 2024-05-09T05:02:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-05-09T05:02:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-05-09T05:02:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-05-09T05:02:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.101 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-05-09T05:02:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 0, 2024-05-09T05:02:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 0, 2024-05-09T05:02:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-05-09T05:02:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-05-09T05:02:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 0, 2024-05-09T05:02:17.102 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-05-09T05:02:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-05-09T05:02:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 0, 2024-05-09T05:02:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2024-05-09T05:02:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-05-09T05:02:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 0, 2024-05-09T05:02:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 0, 2024-05-09T05:02:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 0, 2024-05-09T05:02:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 0, 2024-05-09T05:02:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 0, 2024-05-09T05:02:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 0, 2024-05-09T05:02:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 0, 2024-05-09T05:02:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 0, 2024-05-09T05:02:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 1, 2024-05-09T05:02:17.103 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 0, 2024-05-09T05:02:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 0, 2024-05-09T05:02:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 0, 2024-05-09T05:02:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 0, 2024-05-09T05:02:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-05-09T05:02:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-05-09T05:02:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 0, 2024-05-09T05:02:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 0, 2024-05-09T05:02:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 0, 2024-05-09T05:02:17.104 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 0, 2024-05-09T05:02:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 0, 2024-05-09T05:02:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 0, 2024-05-09T05:02:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 0, 2024-05-09T05:02:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-05-09T05:02:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-05-09T05:02:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-05-09T05:02:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 0, 2024-05-09T05:02:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 0, 2024-05-09T05:02:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 0, 2024-05-09T05:02:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T05:02:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-05-09T05:02:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T05:02:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-05-09T05:02:17.105 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-05-09T05:02:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T05:02:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T05:02:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-05-09T05:02:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 0, 2024-05-09T05:02:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T05:02:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 0, 2024-05-09T05:02:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T05:02:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T05:02:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T05:02:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 0, 2024-05-09T05:02:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 0, 2024-05-09T05:02:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 0, 2024-05-09T05:02:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 0 2024-05-09T05:02:17.106 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-05-09T05:02:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 0, 2024-05-09T05:02:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 0, 2024-05-09T05:02:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 0, 2024-05-09T05:02:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 0, 2024-05-09T05:02:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 0, 2024-05-09T05:02:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2024-05-09T05:02:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-05-09T05:02:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-05-09T05:02:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 0, 2024-05-09T05:02:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 0, 2024-05-09T05:02:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-05-09T05:02:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-05-09T05:02:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-05-09T05:02:17.107 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-05-09T05:02:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-05-09T05:02:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-05-09T05:02:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-05-09T05:02:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-05-09T05:02:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-05-09T05:02:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T05:02:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 0, 2024-05-09T05:02:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-05-09T05:02:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2024-05-09T05:02:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-05-09T05:02:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0 2024-05-09T05:02:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-05-09T05:02:17.108 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-05-09T05:02:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 0, 2024-05-09T05:02:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2024-05-09T05:02:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2024-05-09T05:02:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 8, 2024-05-09T05:02:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-05-09T05:02:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2024-05-09T05:02:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 0, 2024-05-09T05:02:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2024-05-09T05:02:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2024-05-09T05:02:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 2, 2024-05-09T05:02:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 2, 2024-05-09T05:02:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-05-09T05:02:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2024-05-09T05:02:17.109 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2024-05-09T05:02:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 4196655, 2024-05-09T05:02:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 4196514, 2024-05-09T05:02:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-05-09T05:02:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 8 2024-05-09T05:02:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-05-09T05:02:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 3, 2024-05-09T05:02:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 3, 2024-05-09T05:02:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 0, 2024-05-09T05:02:17.110 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 1, 2024-05-09T05:02:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 1, 2024-05-09T05:02:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 0, 2024-05-09T05:02:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 0, 2024-05-09T05:02:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 0, 2024-05-09T05:02:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 0, 2024-05-09T05:02:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 0, 2024-05-09T05:02:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 0, 2024-05-09T05:02:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 0, 2024-05-09T05:02:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 122000, 2024-05-09T05:02:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 215440 2024-05-09T05:02:17.111 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-05-09T05:02:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 0, 2024-05-09T05:02:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 0, 2024-05-09T05:02:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 0, 2024-05-09T05:02:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 0, 2024-05-09T05:02:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 0, 2024-05-09T05:02:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-05-09T05:02:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-05-09T05:02:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.112 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-05-09T05:02:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-05-09T05:02:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-05-09T05:02:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.113 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-05-09T05:02:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-05-09T05:02:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-05-09T05:02:17.114 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-05-09T05:02:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-05-09T05:02:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.115 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-05-09T05:02:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-05-09T05:02:17.116 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-05-09T05:02:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.117 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-05-09T05:02:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-05-09T05:02:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.118 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-05-09T05:02:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-05-09T05:02:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-05-09T05:02:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.119 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-05-09T05:02:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-05-09T05:02:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-05-09T05:02:17.120 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-05-09T05:02:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-05-09T05:02:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.121 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-05-09T05:02:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-05-09T05:02:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-05-09T05:02:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.122 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-05-09T05:02:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-05-09T05:02:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-05-09T05:02:17.123 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-05-09T05:02:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-05-09T05:02:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-05-09T05:02:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-05-09T05:02:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 2, 2024-05-09T05:02:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 4, 2024-05-09T05:02:17.124 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 2, 2024-05-09T05:02:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 3, 2024-05-09T05:02:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-05-09T05:02:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 0, 2024-05-09T05:02:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 0, 2024-05-09T05:02:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 2861, 2024-05-09T05:02:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-05-09T05:02:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-05-09T05:02:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-05-09T05:02:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-05-09T05:02:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T05:02:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-05-09T05:02:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T05:02:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-05-09T05:02:17.125 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T05:02:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T05:02:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T05:02:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T05:02:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T05:02:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-05-09T05:02:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T05:02:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T05:02:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-05-09T05:02:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-05-09T05:02:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-05-09T05:02:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-05-09T05:02:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T05:02:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T05:02:17.126 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T05:02:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T05:02:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T05:02:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-05-09T05:02:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-05-09T05:02:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-05-09T05:02:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T05:02:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T05:02:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-05-09T05:02:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-05-09T05:02:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-05-09T05:02:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-05-09T05:02:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T05:02:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-05-09T05:02:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T05:02:17.127 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-05-09T05:02:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 190185, 2024-05-09T05:02:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 53, 2024-05-09T05:02:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-05-09T05:02:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-05-09T05:02:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-05-09T05:02:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-05-09T05:02:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-05-09T05:02:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-05-09T05:02:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-05-09T05:02:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-05-09T05:02:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8320, 2024-05-09T05:02:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 173, 2024-05-09T05:02:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T05:02:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-05-09T05:02:17.128 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-05-09T05:02:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-05-09T05:02:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 8512, 2024-05-09T05:02:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 13, 2024-05-09T05:02:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-05-09T05:02:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-05-09T05:02:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-05-09T05:02:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-05-09T05:02:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-05-09T05:02:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-05-09T05:02:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-05-09T05:02:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 2444, 2024-05-09T05:02:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 21, 2024-05-09T05:02:17.129 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 3, 2024-05-09T05:02:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 2444, 2024-05-09T05:02:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-05-09T05:02:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2444, 2024-05-09T05:02:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 1.814967990, 2024-05-09T05:02:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000742621 2024-05-09T05:02:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-05-09T05:02:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-05-09T05:02:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2444, 2024-05-09T05:02:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 2445 2024-05-09T05:02:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 2444, 2024-05-09T05:02:17.130 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 2444, 2024-05-09T05:02:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 0, 2024-05-09T05:02:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-05-09T05:02:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-05-09T05:02:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 1585, 2024-05-09T05:02:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 0, 2024-05-09T05:02:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 857, 2024-05-09T05:02:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 0, 2024-05-09T05:02:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 0, 2024-05-09T05:02:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-05-09T05:02:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-05-09T05:02:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-05-09T05:02:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2024-05-09T05:02:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 0, 2024-05-09T05:02:17.131 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-05-09T05:02:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-05-09T05:02:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-05-09T05:02:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 1, 2024-05-09T05:02:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 0, 2024-05-09T05:02:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-05-09T05:02:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-05-09T05:02:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-05-09T05:02:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-05-09T05:02:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-05-09T05:02:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-05-09T05:02:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T05:02:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-05-09T05:02:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-05-09T05:02:17.132 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2024-05-09T05:02:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-05-09T05:02:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-05-09T05:02:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-05-09T05:02:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-05-09T05:02:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-05-09T05:02:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-05-09T05:02:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-05-09T05:02:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-05-09T05:02:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-05-09T05:02:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-05-09T05:02:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-05-09T05:02:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-05-09T05:02:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-05-09T05:02:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-05-09T05:02:17.133 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-05-09T05:02:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-05-09T05:02:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 95, 2024-05-09T05:02:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-05-09T05:02:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 15, 2024-05-09T05:02:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 5, 2024-05-09T05:02:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 5, 2024-05-09T05:02:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-05-09T05:02:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-05-09T05:02:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 0, 2024-05-09T05:02:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 2, 2024-05-09T05:02:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 0 2024-05-09T05:02:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-05-09T05:02:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 0, 2024-05-09T05:02:17.134 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-05-09T05:02:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-05-09T05:02:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-05-09T05:02:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "osd-slow-ops": { 2024-05-09T05:02:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2024-05-09T05:02:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-05-09T05:02:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-05-09T05:02:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T05:02:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-05-09T05:02:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 0, 2024-05-09T05:02:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 0, 2024-05-09T05:02:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 0, 2024-05-09T05:02:17.135 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-05-09T05:02:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T05:02:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-05-09T05:02:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 2721, 2024-05-09T05:02:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 2186340, 2024-05-09T05:02:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 2721, 2024-05-09T05:02:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-09T05:02:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-09T05:02:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 2721, 2024-05-09T05:02:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 2186340, 2024-05-09T05:02:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:17.136 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-05-09T05:02:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-05-09T05:02:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-09T05:02:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-09T05:02:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 2444, 2024-05-09T05:02:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 2356, 2024-05-09T05:02:17.137 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 2444, 2024-05-09T05:02:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 2356, 2024-05-09T05:02:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-05-09T05:02:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-05-09T05:02:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:17.138 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-09T05:02:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-09T05:02:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 2444, 2024-05-09T05:02:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 2444, 2024-05-09T05:02:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 2444, 2024-05-09T05:02:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 2444, 2024-05-09T05:02:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:17.139 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-05-09T05:02:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-05-09T05:02:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-09T05:02:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-09T05:02:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-09T05:02:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-09T05:02:17.140 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-05-09T05:02:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-05-09T05:02:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x563730fd68f0": { 2024-05-09T05:02:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-05-09T05:02:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-09T05:02:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-09T05:02:17.141 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-09T05:02:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-09T05:02:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-05-09T05:02:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-05-09T05:02:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:17.142 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:17.142 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-05-09T05:02:17.143 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:16 smithi072 ceph-mon[34088]: pgmap v347: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:02:17.143 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:16 smithi072 ceph-mon[34088]: osdmap e95: 12 total, 12 up, 12 in 2024-05-09T05:02:17.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:16 smithi046 ceph-mon[30136]: pgmap v347: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2024-05-09T05:02:17.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:16 smithi046 ceph-mon[30136]: osdmap e95: 12 total, 12 up, 12 in 2024-05-09T05:02:17.838 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-09T05:02:17.838 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-09T05:02:17.838 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-05-09T05:02:17.839 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-05-09T05:02:17.839 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-05-09T05:02:17.839 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-09T05:02:17.839 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T05:02:17.839 DEBUG:tasks.check_counter:Getting stats from i 2024-05-09T05:02:17.839 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-09T05:02:18.078 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:02:19.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:18 smithi072 ceph-mon[34088]: pgmap v349: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T05:02:19.227 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:18 smithi110 ceph-mon[34904]: pgmap v349: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T05:02:19.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:18 smithi046 ceph-mon[30136]: pgmap v349: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T05:02:19.859 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-05-09T05:02:19.859 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T05:02:19.859 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 392, 2024-05-09T05:02:19.859 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 390, 2024-05-09T05:02:19.859 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 1789166, 2024-05-09T05:02:19.859 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 262587, 2024-05-09T05:02:19.859 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 15, 2024-05-09T05:02:19.859 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 3, 2024-05-09T05:02:19.859 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.072722923, 2024-05-09T05:02:19.860 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.021097968, 2024-05-09T05:02:19.860 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.053044722, 2024-05-09T05:02:19.860 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.006858443, 2024-05-09T05:02:19.860 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:19.860 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 391, 2024-05-09T05:02:19.860 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.024043068, 2024-05-09T05:02:19.860 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000061491 2024-05-09T05:02:19.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.860 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:19.860 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 156, 2024-05-09T05:02:19.860 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000045612, 2024-05-09T05:02:19.860 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000292 2024-05-09T05:02:19.860 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.860 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 1764304, 2024-05-09T05:02:19.860 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 161664 2024-05-09T05:02:19.861 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.861 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T05:02:19.861 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 59, 2024-05-09T05:02:19.861 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 241, 2024-05-09T05:02:19.861 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 10821, 2024-05-09T05:02:19.861 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 949592, 2024-05-09T05:02:19.861 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 47, 2024-05-09T05:02:19.861 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 4, 2024-05-09T05:02:19.861 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.028196224, 2024-05-09T05:02:19.862 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.008465827, 2024-05-09T05:02:19.862 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.015971426, 2024-05-09T05:02:19.862 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000421408, 2024-05-09T05:02:19.862 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:19.862 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 238, 2024-05-09T05:02:19.862 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.030592415, 2024-05-09T05:02:19.862 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000128539 2024-05-09T05:02:19.862 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.862 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:19.862 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 46, 2024-05-09T05:02:19.862 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000006273, 2024-05-09T05:02:19.863 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000136 2024-05-09T05:02:19.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.863 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 1248, 2024-05-09T05:02:19.863 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 483824 2024-05-09T05:02:19.863 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.863 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T05:02:19.863 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 482, 2024-05-09T05:02:19.863 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 676, 2024-05-09T05:02:19.863 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 127776, 2024-05-09T05:02:19.863 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 355205, 2024-05-09T05:02:19.863 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 8, 2024-05-09T05:02:19.863 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 6, 2024-05-09T05:02:19.863 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.077175928, 2024-05-09T05:02:19.864 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.030414497, 2024-05-09T05:02:19.864 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.040632823, 2024-05-09T05:02:19.864 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.005711225, 2024-05-09T05:02:19.864 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:19.864 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 673, 2024-05-09T05:02:19.864 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.039747801, 2024-05-09T05:02:19.864 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000059060 2024-05-09T05:02:19.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.864 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:19.864 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1102, 2024-05-09T05:02:19.864 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000425983, 2024-05-09T05:02:19.864 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000386 2024-05-09T05:02:19.864 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.865 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-09T05:02:19.865 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 448 2024-05-09T05:02:19.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.865 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-05-09T05:02:19.865 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 1, 2024-05-09T05:02:19.865 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-05-09T05:02:19.865 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.865 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-MDSRank": { 2024-05-09T05:02:19.865 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-05-09T05:02:19.865 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-05-09T05:02:19.865 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 59, 2024-05-09T05:02:19.865 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.002124884, 2024-05-09T05:02:19.865 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000036014 2024-05-09T05:02:19.865 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:19.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.866 INFO:teuthology.orchestra.run.smithi110.stdout: "finisher-PurgeQueue": { 2024-05-09T05:02:19.866 INFO:teuthology.orchestra.run.smithi110.stdout: "queue_len": 0, 2024-05-09T05:02:19.866 INFO:teuthology.orchestra.run.smithi110.stdout: "complete_latency": { 2024-05-09T05:02:19.866 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 2, 2024-05-09T05:02:19.866 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000008623, 2024-05-09T05:02:19.866 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000004311 2024-05-09T05:02:19.866 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:19.866 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.866 INFO:teuthology.orchestra.run.smithi110.stdout: "mds": { 2024-05-09T05:02:19.866 INFO:teuthology.orchestra.run.smithi110.stdout: "request": 1, 2024-05-09T05:02:19.866 INFO:teuthology.orchestra.run.smithi110.stdout: "reply": 1, 2024-05-09T05:02:19.866 INFO:teuthology.orchestra.run.smithi110.stdout: "reply_latency": { 2024-05-09T05:02:19.866 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1, 2024-05-09T05:02:19.867 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000148374, 2024-05-09T05:02:19.867 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000148374 2024-05-09T05:02:19.867 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.867 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_reply": 0, 2024-05-09T05:02:19.867 INFO:teuthology.orchestra.run.smithi110.stdout: "forward": 0, 2024-05-09T05:02:19.867 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_complete": 0, 2024-05-09T05:02:19.867 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_fetch_keys": 0, 2024-05-09T05:02:19.867 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_commit": 11, 2024-05-09T05:02:19.867 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_split": 0, 2024-05-09T05:02:19.867 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_merge": 0, 2024-05-09T05:02:19.867 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes": 14, 2024-05-09T05:02:19.867 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_top": 1, 2024-05-09T05:02:19.867 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_bottom": 13, 2024-05-09T05:02:19.868 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pin_tail": 0, 2024-05-09T05:02:19.868 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_pinned": 13, 2024-05-09T05:02:19.869 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_expired": 0, 2024-05-09T05:02:19.869 INFO:teuthology.orchestra.run.smithi110.stdout: "inodes_with_caps": 1, 2024-05-09T05:02:19.869 INFO:teuthology.orchestra.run.smithi110.stdout: "caps": 1, 2024-05-09T05:02:19.869 INFO:teuthology.orchestra.run.smithi110.stdout: "subtrees": 2, 2024-05-09T05:02:19.869 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse": 5, 2024-05-09T05:02:19.869 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_hit": 3, 2024-05-09T05:02:19.869 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_forward": 0, 2024-05-09T05:02:19.869 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_discover": 2, 2024-05-09T05:02:19.869 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_dir_fetch": 0, 2024-05-09T05:02:19.869 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_remote_ino": 0, 2024-05-09T05:02:19.869 INFO:teuthology.orchestra.run.smithi110.stdout: "traverse_lock": 0, 2024-05-09T05:02:19.869 INFO:teuthology.orchestra.run.smithi110.stdout: "load_cent": 1000, 2024-05-09T05:02:19.869 INFO:teuthology.orchestra.run.smithi110.stdout: "q": 1, 2024-05-09T05:02:19.869 INFO:teuthology.orchestra.run.smithi110.stdout: "exported": 0, 2024-05-09T05:02:19.870 INFO:teuthology.orchestra.run.smithi110.stdout: "exported_inodes": 0, 2024-05-09T05:02:19.870 INFO:teuthology.orchestra.run.smithi110.stdout: "imported": 0, 2024-05-09T05:02:19.870 INFO:teuthology.orchestra.run.smithi110.stdout: "imported_inodes": 0, 2024-05-09T05:02:19.870 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_dir_fetch": 0, 2024-05-09T05:02:19.870 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_backtrace_fetch": 0, 2024-05-09T05:02:19.870 INFO:teuthology.orchestra.run.smithi110.stdout: "openino_peer_discover": 0, 2024-05-09T05:02:19.870 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rfiles": 4, 2024-05-09T05:02:19.870 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rbytes": 234, 2024-05-09T05:02:19.871 INFO:teuthology.orchestra.run.smithi110.stdout: "root_rsnaps": 6, 2024-05-09T05:02:19.871 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_fetch": 0, 2024-05-09T05:02:19.871 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_set_tag": 0, 2024-05-09T05:02:19.871 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_backtrace_repaired": 0, 2024-05-09T05:02:19.871 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_inotable_repaired": 0, 2024-05-09T05:02:19.871 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_inodes": 0, 2024-05-09T05:02:19.871 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dir_base_inodes": 0, 2024-05-09T05:02:19.871 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_dirfrag_rstats": 0, 2024-05-09T05:02:19.871 INFO:teuthology.orchestra.run.smithi110.stdout: "scrub_file_inodes": 0, 2024-05-09T05:02:19.871 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_inode_file_caps": 0, 2024-05-09T05:02:19.871 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_revoke": 0, 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_grant": 0, 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_trunc": 0, 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps": 0, 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_caps_dirty": 0, 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_cap_release": 0, 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: "process_request_cap_release": 0 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_cache": { 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update": 0, 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_update_receipt": 8, 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_try_discover": 5, 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_send_discover": 6, 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: "dir_handle_discover": 0, 2024-05-09T05:02:19.872 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays": 0, 2024-05-09T05:02:19.873 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_delayed": 0, 2024-05-09T05:02:19.873 INFO:teuthology.orchestra.run.smithi110.stdout: "num_strays_enqueuing": 0, 2024-05-09T05:02:19.873 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_created": 0, 2024-05-09T05:02:19.873 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_enqueued": 0, 2024-05-09T05:02:19.873 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_reintegrated": 0, 2024-05-09T05:02:19.873 INFO:teuthology.orchestra.run.smithi110.stdout: "strays_migrated": 0, 2024-05-09T05:02:19.873 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_processing": 0, 2024-05-09T05:02:19.873 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_enqueued": 0, 2024-05-09T05:02:19.873 INFO:teuthology.orchestra.run.smithi110.stdout: "num_recovering_prioritized": 0, 2024-05-09T05:02:19.873 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_started": 0, 2024-05-09T05:02:19.873 INFO:teuthology.orchestra.run.smithi110.stdout: "recovery_completed": 0, 2024-05-09T05:02:19.873 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_path": 0, 2024-05-09T05:02:19.873 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_quiesce_inode": 0, 2024-05-09T05:02:19.873 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_enqueue_scrub": 0, 2024-05-09T05:02:19.874 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_exportdir": 0, 2024-05-09T05:02:19.874 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_flush": 0, 2024-05-09T05:02:19.874 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragmentdir": 0, 2024-05-09T05:02:19.874 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_fragstats": 0, 2024-05-09T05:02:19.874 INFO:teuthology.orchestra.run.smithi110.stdout: "ireq_inodestats": 0 2024-05-09T05:02:19.874 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.874 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_log": { 2024-05-09T05:02:19.874 INFO:teuthology.orchestra.run.smithi110.stdout: "evlrg": 0, 2024-05-09T05:02:19.874 INFO:teuthology.orchestra.run.smithi110.stdout: "evadd": 8, 2024-05-09T05:02:19.874 INFO:teuthology.orchestra.run.smithi110.stdout: "evex": 0, 2024-05-09T05:02:19.874 INFO:teuthology.orchestra.run.smithi110.stdout: "evtrm": 0, 2024-05-09T05:02:19.874 INFO:teuthology.orchestra.run.smithi110.stdout: "ev": 8, 2024-05-09T05:02:19.874 INFO:teuthology.orchestra.run.smithi110.stdout: "evexg": 0, 2024-05-09T05:02:19.874 INFO:teuthology.orchestra.run.smithi110.stdout: "evexd": 0, 2024-05-09T05:02:19.874 INFO:teuthology.orchestra.run.smithi110.stdout: "segadd": 2, 2024-05-09T05:02:19.875 INFO:teuthology.orchestra.run.smithi110.stdout: "segex": 0, 2024-05-09T05:02:19.875 INFO:teuthology.orchestra.run.smithi110.stdout: "segtrm": 0, 2024-05-09T05:02:19.875 INFO:teuthology.orchestra.run.smithi110.stdout: "seg": 2, 2024-05-09T05:02:19.875 INFO:teuthology.orchestra.run.smithi110.stdout: "segmjr": 2, 2024-05-09T05:02:19.875 INFO:teuthology.orchestra.run.smithi110.stdout: "segexg": 0, 2024-05-09T05:02:19.875 INFO:teuthology.orchestra.run.smithi110.stdout: "segexd": 0, 2024-05-09T05:02:19.875 INFO:teuthology.orchestra.run.smithi110.stdout: "expos": 4194304, 2024-05-09T05:02:19.875 INFO:teuthology.orchestra.run.smithi110.stdout: "wrpos": 4196655, 2024-05-09T05:02:19.875 INFO:teuthology.orchestra.run.smithi110.stdout: "rdpos": 0, 2024-05-09T05:02:19.875 INFO:teuthology.orchestra.run.smithi110.stdout: "jlat": { 2024-05-09T05:02:19.875 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 7, 2024-05-09T05:02:19.875 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.014124906, 2024-05-09T05:02:19.875 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.002017843 2024-05-09T05:02:19.875 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.875 INFO:teuthology.orchestra.run.smithi110.stdout: "replayed": 0 2024-05-09T05:02:19.876 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.876 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_mem": { 2024-05-09T05:02:19.876 INFO:teuthology.orchestra.run.smithi110.stdout: "ino": 17, 2024-05-09T05:02:19.876 INFO:teuthology.orchestra.run.smithi110.stdout: "ino+": 17, 2024-05-09T05:02:19.876 INFO:teuthology.orchestra.run.smithi110.stdout: "ino-": 0, 2024-05-09T05:02:19.876 INFO:teuthology.orchestra.run.smithi110.stdout: "dir": 16, 2024-05-09T05:02:19.876 INFO:teuthology.orchestra.run.smithi110.stdout: "dir+": 18, 2024-05-09T05:02:19.876 INFO:teuthology.orchestra.run.smithi110.stdout: "dir-": 2, 2024-05-09T05:02:19.876 INFO:teuthology.orchestra.run.smithi110.stdout: "dn": 14, 2024-05-09T05:02:19.876 INFO:teuthology.orchestra.run.smithi110.stdout: "dn+": 14, 2024-05-09T05:02:19.876 INFO:teuthology.orchestra.run.smithi110.stdout: "dn-": 0, 2024-05-09T05:02:19.876 INFO:teuthology.orchestra.run.smithi110.stdout: "cap": 1, 2024-05-09T05:02:19.876 INFO:teuthology.orchestra.run.smithi110.stdout: "cap+": 1, 2024-05-09T05:02:19.876 INFO:teuthology.orchestra.run.smithi110.stdout: "cap-": 0, 2024-05-09T05:02:19.876 INFO:teuthology.orchestra.run.smithi110.stdout: "rss": 121068, 2024-05-09T05:02:19.877 INFO:teuthology.orchestra.run.smithi110.stdout: "heap": 223632 2024-05-09T05:02:19.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.877 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_server": { 2024-05-09T05:02:19.877 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_client_request": 1, 2024-05-09T05:02:19.877 INFO:teuthology.orchestra.run.smithi110.stdout: "dispatch_server_request": 0, 2024-05-09T05:02:19.877 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_request": 1, 2024-05-09T05:02:19.877 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_client_session": 27, 2024-05-09T05:02:19.877 INFO:teuthology.orchestra.run.smithi110.stdout: "handle_peer_request": 0, 2024-05-09T05:02:19.877 INFO:teuthology.orchestra.run.smithi110.stdout: "req_create_latency": { 2024-05-09T05:02:19.877 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.877 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.877 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.877 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.877 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getattr_latency": { 2024-05-09T05:02:19.878 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.878 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.878 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.878 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getfilelock_latency": { 2024-05-09T05:02:19.878 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.878 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.878 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.878 INFO:teuthology.orchestra.run.smithi110.stdout: "req_link_latency": { 2024-05-09T05:02:19.878 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.878 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.878 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.878 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.878 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookup_latency": { 2024-05-09T05:02:19.879 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.879 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.879 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.879 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookuphash_latency": { 2024-05-09T05:02:19.879 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.879 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.879 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.879 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.879 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupino_latency": { 2024-05-09T05:02:19.879 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 1, 2024-05-09T05:02:19.879 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000148374, 2024-05-09T05:02:19.880 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000148374 2024-05-09T05:02:19.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.880 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupname_latency": { 2024-05-09T05:02:19.880 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.880 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.880 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.880 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupparent_latency": { 2024-05-09T05:02:19.880 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.880 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.880 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.880 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.880 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lookupsnap_latency": { 2024-05-09T05:02:19.880 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.881 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.881 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.881 INFO:teuthology.orchestra.run.smithi110.stdout: "req_lssnap_latency": { 2024-05-09T05:02:19.881 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.881 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.881 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.881 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.881 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mkdir_latency": { 2024-05-09T05:02:19.882 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.882 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.882 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.882 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.882 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mknod_latency": { 2024-05-09T05:02:19.883 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.883 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.883 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.883 INFO:teuthology.orchestra.run.smithi110.stdout: "req_mksnap_latency": { 2024-05-09T05:02:19.883 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.883 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.883 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.883 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.884 INFO:teuthology.orchestra.run.smithi110.stdout: "req_open_latency": { 2024-05-09T05:02:19.884 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.884 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.884 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.884 INFO:teuthology.orchestra.run.smithi110.stdout: "req_readdir_latency": { 2024-05-09T05:02:19.884 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.884 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.884 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.884 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.884 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rename_latency": { 2024-05-09T05:02:19.884 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.885 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.885 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.885 INFO:teuthology.orchestra.run.smithi110.stdout: "req_renamesnap_latency": { 2024-05-09T05:02:19.885 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.885 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.885 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.885 INFO:teuthology.orchestra.run.smithi110.stdout: "req_snapdiff_latency": { 2024-05-09T05:02:19.885 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.885 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.885 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.885 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.885 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmdir_latency": { 2024-05-09T05:02:19.885 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.886 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.886 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.886 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmsnap_latency": { 2024-05-09T05:02:19.886 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.886 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.886 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.886 INFO:teuthology.orchestra.run.smithi110.stdout: "req_rmxattr_latency": { 2024-05-09T05:02:19.886 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.886 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.886 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.886 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.887 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setattr_latency": { 2024-05-09T05:02:19.887 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.887 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.887 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.887 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setdirlayout_latency": { 2024-05-09T05:02:19.887 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.887 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.887 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.887 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.887 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setfilelock_latency": { 2024-05-09T05:02:19.887 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.887 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.887 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.888 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setlayout_latency": { 2024-05-09T05:02:19.888 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.888 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.888 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.888 INFO:teuthology.orchestra.run.smithi110.stdout: "req_setxattr_latency": { 2024-05-09T05:02:19.888 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.888 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.888 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.888 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.888 INFO:teuthology.orchestra.run.smithi110.stdout: "req_symlink_latency": { 2024-05-09T05:02:19.888 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.889 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.889 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.889 INFO:teuthology.orchestra.run.smithi110.stdout: "req_unlink_latency": { 2024-05-09T05:02:19.889 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.889 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.889 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.889 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.889 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_revoke_eviction": 0, 2024-05-09T05:02:19.889 INFO:teuthology.orchestra.run.smithi110.stdout: "cap_acquisition_throttle": 0, 2024-05-09T05:02:19.889 INFO:teuthology.orchestra.run.smithi110.stdout: "req_getvxattr_latency": { 2024-05-09T05:02:19.889 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.889 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.889 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.889 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:19.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.890 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_sessions": { 2024-05-09T05:02:19.890 INFO:teuthology.orchestra.run.smithi110.stdout: "session_count": 2, 2024-05-09T05:02:19.890 INFO:teuthology.orchestra.run.smithi110.stdout: "session_add": 4, 2024-05-09T05:02:19.890 INFO:teuthology.orchestra.run.smithi110.stdout: "session_remove": 2, 2024-05-09T05:02:19.890 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_open": 2, 2024-05-09T05:02:19.890 INFO:teuthology.orchestra.run.smithi110.stdout: "sessions_stale": 0, 2024-05-09T05:02:19.890 INFO:teuthology.orchestra.run.smithi110.stdout: "total_load": 1, 2024-05-09T05:02:19.890 INFO:teuthology.orchestra.run.smithi110.stdout: "average_load": 0, 2024-05-09T05:02:19.890 INFO:teuthology.orchestra.run.smithi110.stdout: "avg_session_uptime": 636, 2024-05-09T05:02:19.890 INFO:teuthology.orchestra.run.smithi110.stdout: "mdthresh_evicted": 0 2024-05-09T05:02:19.890 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.890 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-05-09T05:02:19.890 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-05-09T05:02:19.890 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-05-09T05:02:19.891 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T05:02:19.891 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-05-09T05:02:19.891 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T05:02:19.891 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-05-09T05:02:19.891 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T05:02:19.891 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T05:02:19.891 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T05:02:19.891 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T05:02:19.891 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T05:02:19.891 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-05-09T05:02:19.891 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T05:02:19.891 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T05:02:19.891 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-05-09T05:02:19.891 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-05-09T05:02:19.892 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-05-09T05:02:19.892 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-05-09T05:02:19.892 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T05:02:19.892 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T05:02:19.892 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T05:02:19.892 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T05:02:19.892 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T05:02:19.892 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-05-09T05:02:19.892 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-05-09T05:02:19.892 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-05-09T05:02:19.892 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T05:02:19.892 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T05:02:19.892 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-05-09T05:02:19.892 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-05-09T05:02:19.892 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-05-09T05:02:19.893 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-05-09T05:02:19.893 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T05:02:19.893 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-05-09T05:02:19.893 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T05:02:19.893 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-05-09T05:02:19.893 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 263075, 2024-05-09T05:02:19.893 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 84, 2024-05-09T05:02:19.893 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-05-09T05:02:19.893 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-05-09T05:02:19.893 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-05-09T05:02:19.893 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-05-09T05:02:19.893 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-05-09T05:02:19.894 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-05-09T05:02:19.894 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-05-09T05:02:19.894 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-05-09T05:02:19.894 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 8440, 2024-05-09T05:02:19.894 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 176, 2024-05-09T05:02:19.894 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T05:02:19.894 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-05-09T05:02:19.894 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-05-09T05:02:19.894 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-05-09T05:02:19.894 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 69392, 2024-05-09T05:02:19.894 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 253, 2024-05-09T05:02:19.894 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-05-09T05:02:19.894 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-05-09T05:02:19.894 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-05-09T05:02:19.895 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-05-09T05:02:19.895 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.895 INFO:teuthology.orchestra.run.smithi110.stdout: "objecter": { 2024-05-09T05:02:19.895 INFO:teuthology.orchestra.run.smithi110.stdout: "op_active": 0, 2024-05-09T05:02:19.895 INFO:teuthology.orchestra.run.smithi110.stdout: "op_laggy": 0, 2024-05-09T05:02:19.895 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send": 46, 2024-05-09T05:02:19.895 INFO:teuthology.orchestra.run.smithi110.stdout: "op_send_bytes": 13044, 2024-05-09T05:02:19.895 INFO:teuthology.orchestra.run.smithi110.stdout: "op_resend": 29, 2024-05-09T05:02:19.895 INFO:teuthology.orchestra.run.smithi110.stdout: "op_reply": 46, 2024-05-09T05:02:19.895 INFO:teuthology.orchestra.run.smithi110.stdout: "op_latency": { 2024-05-09T05:02:19.895 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 46, 2024-05-09T05:02:19.895 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.229995921, 2024-05-09T05:02:19.896 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.004999911 2024-05-09T05:02:19.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.896 INFO:teuthology.orchestra.run.smithi110.stdout: "op_inflight": 0, 2024-05-09T05:02:19.896 INFO:teuthology.orchestra.run.smithi110.stdout: "oplen_avg": { 2024-05-09T05:02:19.896 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 46, 2024-05-09T05:02:19.896 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 69 2024-05-09T05:02:19.896 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.896 INFO:teuthology.orchestra.run.smithi110.stdout: "op": 46, 2024-05-09T05:02:19.896 INFO:teuthology.orchestra.run.smithi110.stdout: "op_r": 0, 2024-05-09T05:02:19.896 INFO:teuthology.orchestra.run.smithi110.stdout: "op_w": 46, 2024-05-09T05:02:19.896 INFO:teuthology.orchestra.run.smithi110.stdout: "op_rmw": 0, 2024-05-09T05:02:19.896 INFO:teuthology.orchestra.run.smithi110.stdout: "op_pg": 0, 2024-05-09T05:02:19.896 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_stat": 0, 2024-05-09T05:02:19.896 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_create": 10, 2024-05-09T05:02:19.897 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_read": 0, 2024-05-09T05:02:19.897 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_write": 7, 2024-05-09T05:02:19.897 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writefull": 9, 2024-05-09T05:02:19.897 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_writesame": 0, 2024-05-09T05:02:19.897 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_append": 0, 2024-05-09T05:02:19.897 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_zero": 0, 2024-05-09T05:02:19.897 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_truncate": 0, 2024-05-09T05:02:19.897 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_delete": 6, 2024-05-09T05:02:19.897 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_mapext": 0, 2024-05-09T05:02:19.897 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_sparse_read": 0, 2024-05-09T05:02:19.897 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_clonerange": 0, 2024-05-09T05:02:19.897 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_getxattr": 0, 2024-05-09T05:02:19.897 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_setxattr": 20, 2024-05-09T05:02:19.897 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_cmpxattr": 0, 2024-05-09T05:02:19.897 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_rmxattr": 0, 2024-05-09T05:02:19.898 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_resetxattrs": 0, 2024-05-09T05:02:19.898 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_call": 0, 2024-05-09T05:02:19.898 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_watch": 0, 2024-05-09T05:02:19.898 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_notify": 0, 2024-05-09T05:02:19.898 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_src_cmpxattr": 0, 2024-05-09T05:02:19.898 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls": 0, 2024-05-09T05:02:19.898 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_pgls_filter": 0, 2024-05-09T05:02:19.898 INFO:teuthology.orchestra.run.smithi110.stdout: "osdop_other": 0, 2024-05-09T05:02:19.898 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_active": 0, 2024-05-09T05:02:19.898 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_send": 0, 2024-05-09T05:02:19.898 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_resend": 0, 2024-05-09T05:02:19.898 INFO:teuthology.orchestra.run.smithi110.stdout: "linger_ping": 0, 2024-05-09T05:02:19.898 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_active": 0, 2024-05-09T05:02:19.898 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_send": 0, 2024-05-09T05:02:19.898 INFO:teuthology.orchestra.run.smithi110.stdout: "poolop_resend": 0, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_active": 0, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_send": 0, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "poolstat_resend": 0, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_active": 0, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_send": 0, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "statfs_resend": 0, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "command_active": 0, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "command_send": 0, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "command_resend": 0, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "map_epoch": 95, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "map_full": 0, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "map_inc": 15, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_sessions": 8, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_open": 8, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_session_close": 0, 2024-05-09T05:02:19.899 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_laggy": 0, 2024-05-09T05:02:19.900 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_wr": 16, 2024-05-09T05:02:19.900 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_rd": 0, 2024-05-09T05:02:19.900 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_del": 1 2024-05-09T05:02:19.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.900 INFO:teuthology.orchestra.run.smithi110.stdout: "oft": { 2024-05-09T05:02:19.900 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_objs": 1, 2024-05-09T05:02:19.900 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_kv_pairs": 0, 2024-05-09T05:02:19.900 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_updates": 0, 2024-05-09T05:02:19.900 INFO:teuthology.orchestra.run.smithi110.stdout: "omap_total_removes": 0 2024-05-09T05:02:19.900 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.900 INFO:teuthology.orchestra.run.smithi110.stdout: "osd-slow-ops": { 2024-05-09T05:02:19.900 INFO:teuthology.orchestra.run.smithi110.stdout: "slow_ops_count": 0 2024-05-09T05:02:19.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.901 INFO:teuthology.orchestra.run.smithi110.stdout: "purge_queue": { 2024-05-09T05:02:19.901 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops": 0, 2024-05-09T05:02:19.901 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_ops_high_water": 0, 2024-05-09T05:02:19.901 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing": 0, 2024-05-09T05:02:19.901 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executing_high_water": 0, 2024-05-09T05:02:19.901 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed_ops": 0, 2024-05-09T05:02:19.901 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_executed": 0, 2024-05-09T05:02:19.901 INFO:teuthology.orchestra.run.smithi110.stdout: "pq_item_in_journal": 0 2024-05-09T05:02:19.901 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.901 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T05:02:19.901 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:19.901 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-05-09T05:02:19.901 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:19.901 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 929, 2024-05-09T05:02:19.902 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 1825076, 2024-05-09T05:02:19.902 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:19.902 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 929, 2024-05-09T05:02:19.902 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-09T05:02:19.902 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-09T05:02:19.902 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 929, 2024-05-09T05:02:19.902 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1825076, 2024-05-09T05:02:19.902 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:19.902 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.902 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.902 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_bytes": { 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 46, 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 13044, 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 46, 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 13044, 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:19.903 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.904 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.904 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.904 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:19.904 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.904 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-objecter_ops": { 2024-05-09T05:02:19.904 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:19.904 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 1024, 2024-05-09T05:02:19.904 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:19.904 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-09T05:02:19.904 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-09T05:02:19.904 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:19.904 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:19.904 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 46, 2024-05-09T05:02:19.904 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 46, 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 46, 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 46, 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle": { 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 0, 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 0, 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:19.905 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 0, 2024-05-09T05:02:19.906 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-09T05:02:19.906 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-09T05:02:19.906 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 0, 2024-05-09T05:02:19.906 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 0, 2024-05-09T05:02:19.906 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:19.906 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.906 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.906 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.906 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:19.906 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:19.906 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-write_buf_throttle-0x562439a4a680": { 2024-05-09T05:02:19.906 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:19.906 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 3758096384, 2024-05-09T05:02:19.906 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:19.907 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 8, 2024-05-09T05:02:19.907 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 2351, 2024-05-09T05:02:19.907 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:19.907 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 8, 2024-05-09T05:02:19.907 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-09T05:02:19.907 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-09T05:02:19.907 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 7, 2024-05-09T05:02:19.909 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 2351, 2024-05-09T05:02:19.909 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:19.909 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:19.909 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:19.909 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:19.909 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:19.909 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:19.909 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-05-09T05:02:20.637 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-05-09T05:02:20.637 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-05-09T05:02:20.637 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-05-09T05:02:20.637 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=8 2024-05-09T05:02:20.637 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-09T05:02:20.638 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-09T05:02:20.638 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-09T05:02:20.638 DEBUG:tasks.check_counter:Getting stats from l 2024-05-09T05:02:20.638 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-09T05:02:20.875 INFO:teuthology.orchestra.run.smithi110.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.c/config 2024-05-09T05:02:21.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:20 smithi072 ceph-mon[34088]: pgmap v350: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T05:02:21.155 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:20 smithi110 ceph-mon[34904]: pgmap v350: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T05:02:21.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:20 smithi046 ceph-mon[30136]: pgmap v350: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-05-09T05:02:22.688 INFO:teuthology.orchestra.run.smithi110.stdout:{ 2024-05-09T05:02:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-0": { 2024-05-09T05:02:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 0, 2024-05-09T05:02:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 2, 2024-05-09T05:02:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 0, 2024-05-09T05:02:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 0, 2024-05-09T05:02:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 4, 2024-05-09T05:02:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 0, 2024-05-09T05:02:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.001158879, 2024-05-09T05:02:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.000000000, 2024-05-09T05:02:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.000346198, 2024-05-09T05:02:22.689 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T05:02:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-09T05:02:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 0 2024-05-09T05:02:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:22.690 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-1": { 2024-05-09T05:02:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 6, 2024-05-09T05:02:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 191, 2024-05-09T05:02:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 3312, 2024-05-09T05:02:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 191072, 2024-05-09T05:02:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 5, 2024-05-09T05:02:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 3, 2024-05-09T05:02:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.014223676, 2024-05-09T05:02:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.009490769, 2024-05-09T05:02:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.002074300, 2024-05-09T05:02:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-09T05:02:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 189, 2024-05-09T05:02:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.020437045, 2024-05-09T05:02:22.691 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000108132 2024-05-09T05:02:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-09T05:02:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 191072 2024-05-09T05:02:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "AsyncMessenger::Worker-2": { 2024-05-09T05:02:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_messages": 258, 2024-05-09T05:02:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages": 247, 2024-05-09T05:02:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_bytes": 1721264, 2024-05-09T05:02:22.692 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_bytes": 152080, 2024-05-09T05:02:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_created_connections": 2, 2024-05-09T05:02:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_active_connections": 1, 2024-05-09T05:02:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_total_time": 0.057020743, 2024-05-09T05:02:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_send_time": 0.016925762, 2024-05-09T05:02:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_recv_time": 0.044730226, 2024-05-09T05:02:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_running_fast_dispatch_time": 0.006355712, 2024-05-09T05:02:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_messages_queue_lat": { 2024-05-09T05:02:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 246, 2024-05-09T05:02:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.015232567, 2024-05-09T05:02:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000061921 2024-05-09T05:02:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_handle_ack_lat": { 2024-05-09T05:02:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:22.693 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_recv_encrypted_bytes": 1721264, 2024-05-09T05:02:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "msgr_send_encrypted_bytes": 152080 2024-05-09T05:02:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "cct": { 2024-05-09T05:02:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "total_workers": 0, 2024-05-09T05:02:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "unhealthy_workers": 0 2024-05-09T05:02:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "mempool": { 2024-05-09T05:02:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_bytes": 0, 2024-05-09T05:02:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "bloom_filter_items": 0, 2024-05-09T05:02:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_bytes": 0, 2024-05-09T05:02:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_alloc_items": 0, 2024-05-09T05:02:22.694 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_bytes": 0, 2024-05-09T05:02:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_data_items": 0, 2024-05-09T05:02:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-09T05:02:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_onode_items": 0, 2024-05-09T05:02:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-09T05:02:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_meta_items": 0, 2024-05-09T05:02:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_bytes": 0, 2024-05-09T05:02:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_other_items": 0, 2024-05-09T05:02:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-09T05:02:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_cache_buffer_items": 0, 2024-05-09T05:02:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_bytes": 0, 2024-05-09T05:02:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_extent_items": 0, 2024-05-09T05:02:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_bytes": 0, 2024-05-09T05:02:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_blob_items": 0, 2024-05-09T05:02:22.695 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-09T05:02:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_shared_blob_items": 0, 2024-05-09T05:02:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-09T05:02:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_inline_bl_items": 0, 2024-05-09T05:02:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_bytes": 0, 2024-05-09T05:02:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_fsck_items": 0, 2024-05-09T05:02:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_bytes": 0, 2024-05-09T05:02:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_txc_items": 0, 2024-05-09T05:02:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-09T05:02:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_deferred_items": 0, 2024-05-09T05:02:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_bytes": 0, 2024-05-09T05:02:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "bluestore_writing_items": 0, 2024-05-09T05:02:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_bytes": 0, 2024-05-09T05:02:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_items": 0, 2024-05-09T05:02:22.696 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_bytes": 0, 2024-05-09T05:02:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_reader_items": 0, 2024-05-09T05:02:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_bytes": 0, 2024-05-09T05:02:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "bluefs_file_writer_items": 0, 2024-05-09T05:02:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_bytes": 148704, 2024-05-09T05:02:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_anon_items": 29, 2024-05-09T05:02:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_bytes": 0, 2024-05-09T05:02:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "buffer_meta_items": 0, 2024-05-09T05:02:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_bytes": 0, 2024-05-09T05:02:22.697 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_items": 0, 2024-05-09T05:02:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_bytes": 0, 2024-05-09T05:02:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_mapbl_items": 0, 2024-05-09T05:02:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_bytes": 0, 2024-05-09T05:02:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "osd_pglog_items": 0, 2024-05-09T05:02:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_bytes": 0, 2024-05-09T05:02:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_items": 0, 2024-05-09T05:02:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_bytes": 0, 2024-05-09T05:02:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "osdmap_mapping_items": 0, 2024-05-09T05:02:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_bytes": 0, 2024-05-09T05:02:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "pgmap_items": 0, 2024-05-09T05:02:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_bytes": 928, 2024-05-09T05:02:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "mds_co_items": 2, 2024-05-09T05:02:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_bytes": 0, 2024-05-09T05:02:22.698 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_1_items": 0, 2024-05-09T05:02:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_bytes": 0, 2024-05-09T05:02:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "unittest_2_items": 0 2024-05-09T05:02:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: }, 2024-05-09T05:02:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-09T05:02:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "val": 0, 2024-05-09T05:02:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "max": 104857600, 2024-05-09T05:02:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "get_started": 0, 2024-05-09T05:02:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "get": 260, 2024-05-09T05:02:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "get_sum": 1684850, 2024-05-09T05:02:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_fail": 0, 2024-05-09T05:02:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "get_or_fail_success": 260, 2024-05-09T05:02:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "take": 0, 2024-05-09T05:02:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "take_sum": 0, 2024-05-09T05:02:22.699 INFO:teuthology.orchestra.run.smithi110.stdout: "put": 260, 2024-05-09T05:02:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: "put_sum": 1684850, 2024-05-09T05:02:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: "wait": { 2024-05-09T05:02:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: "avgcount": 0, 2024-05-09T05:02:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: "sum": 0.000000000, 2024-05-09T05:02:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: "avgtime": 0.000000000 2024-05-09T05:02:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:22.700 INFO:teuthology.orchestra.run.smithi110.stdout: } 2024-05-09T05:02:22.700 INFO:teuthology.orchestra.run.smithi110.stdout:} 2024-05-09T05:02:23.007 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:22 smithi110 ceph-mon[34904]: pgmap v351: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T05:02:23.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:22 smithi072 ceph-mon[34088]: pgmap v351: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T05:02:23.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:22 smithi046 ceph-mon[30136]: pgmap v351: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T05:02:23.426 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-05-09T05:02:23.426 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-05-09T05:02:23.426 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-05-09T05:02:23.426 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-05-09T05:02:23.426 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-05-09T05:02:23.427 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-05-09T05:02:23.427 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-05-09T05:02:23.427 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-09T05:02:23.439 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-09T05:02:23.439 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T05:02:23.439 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T05:02:23.461 DEBUG:teuthology.orchestra.run.smithi046:> set -ex 2024-05-09T05:02:23.462 DEBUG:teuthology.orchestra.run.smithi046:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-09T05:02:23.522 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-09T05:02:23.828 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:02:25.058 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:24 smithi110 ceph-mon[34904]: pgmap v352: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T05:02:25.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:24 smithi072 ceph-mon[34088]: pgmap v352: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T05:02:25.215 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:24 smithi046 ceph-mon[30136]: pgmap v352: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T05:02:25.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.649+0000 7fb973074700 1 -- 172.21.15.46:0/1443645645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb96c1034c0 msgr2=0x7fb96c1038d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:25.651 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.649+0000 7fb973074700 1 --2- 172.21.15.46:0/1443645645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb96c1034c0 0x7fb96c1038d0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fb9600099c0 tx=0x7fb960009cd0 comp rx=0 tx=0).stop 2024-05-09T05:02:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.649+0000 7fb973074700 1 -- 172.21.15.46:0/1443645645 shutdown_connections 2024-05-09T05:02:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.649+0000 7fb973074700 1 --2- 172.21.15.46:0/1443645645 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb96c108740 0x7fb96c1100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.649+0000 7fb973074700 1 --2- 172.21.15.46:0/1443645645 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb96c1034c0 0x7fb96c1038d0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.649+0000 7fb973074700 1 --2- 172.21.15.46:0/1443645645 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb96c107df0 0x7fb96c108200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.649+0000 7fb973074700 1 -- 172.21.15.46:0/1443645645 >> 172.21.15.46:0/1443645645 conn(0x7fb96c0fedc0 msgr2=0x7fb96c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.649+0000 7fb973074700 1 -- 172.21.15.46:0/1443645645 shutdown_connections 2024-05-09T05:02:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.649+0000 7fb973074700 1 -- 172.21.15.46:0/1443645645 wait complete. 2024-05-09T05:02:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.650+0000 7fb973074700 1 Processor -- start 2024-05-09T05:02:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.650+0000 7fb973074700 1 -- start start 2024-05-09T05:02:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.650+0000 7fb973074700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb96c1034c0 0x7fb96c199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.650+0000 7fb973074700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb96c107df0 0x7fb96c199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.650+0000 7fb973074700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb96c108740 0x7fb96c19a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.650+0000 7fb973074700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c113370 con 0x7fb96c107df0 2024-05-09T05:02:25.652 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.650+0000 7fb973074700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c1131f0 con 0x7fb96c108740 2024-05-09T05:02:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.650+0000 7fb973074700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c1134f0 con 0x7fb96c1034c0 2024-05-09T05:02:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.650+0000 7fb96bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb96c107df0 0x7fb96c199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.650+0000 7fb96bfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb96c107df0 0x7fb96c199f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45360/0 (socket says 172.21.15.46:45360) 2024-05-09T05:02:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.650+0000 7fb96bfff700 1 -- 172.21.15.46:0/3036591317 learned_addr learned my addr 172.21.15.46:0/3036591317 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:02:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.650+0000 7fb970e10700 1 --2- 172.21.15.46:0/3036591317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb96c1034c0 0x7fb96c199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.650+0000 7fb971611700 1 --2- 172.21.15.46:0/3036591317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb96c108740 0x7fb96c19a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.651+0000 7fb96bfff700 1 -- 172.21.15.46:0/3036591317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb96c1034c0 msgr2=0x7fb96c199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.651+0000 7fb96bfff700 1 --2- 172.21.15.46:0/3036591317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb96c1034c0 0x7fb96c199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.651+0000 7fb96bfff700 1 -- 172.21.15.46:0/3036591317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb96c108740 msgr2=0x7fb96c19a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.651+0000 7fb96bfff700 1 --2- 172.21.15.46:0/3036591317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb96c108740 0x7fb96c19a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.651+0000 7fb96bfff700 1 -- 172.21.15.46:0/3036591317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c1a9080 con 0x7fb96c107df0 2024-05-09T05:02:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.651+0000 7fb96bfff700 1 --2- 172.21.15.46:0/3036591317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb96c107df0 0x7fb96c199f40 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fb95800ca10 tx=0x7fb95800cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.651+0000 7fb969ffb700 1 -- 172.21.15.46:0/3036591317 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958007a60 con 0x7fb96c107df0 2024-05-09T05:02:25.653 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.651+0000 7fb973074700 1 -- 172.21.15.46:0/3036591317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96c1a9370 con 0x7fb96c107df0 2024-05-09T05:02:25.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.651+0000 7fb969ffb700 1 -- 172.21.15.46:0/3036591317 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb958007c00 con 0x7fb96c107df0 2024-05-09T05:02:25.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.651+0000 7fb969ffb700 1 -- 172.21.15.46:0/3036591317 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb958018660 con 0x7fb96c107df0 2024-05-09T05:02:25.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.651+0000 7fb973074700 1 -- 172.21.15.46:0/3036591317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96c1a98b0 con 0x7fb96c107df0 2024-05-09T05:02:25.654 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.652+0000 7fb973074700 1 -- 172.21.15.46:0/3036591317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb96c104970 con 0x7fb96c107df0 2024-05-09T05:02:25.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.655+0000 7fb969ffb700 1 -- 172.21.15.46:0/3036591317 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb95800f450 con 0x7fb96c107df0 2024-05-09T05:02:25.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.656+0000 7fb969ffb700 1 --2- 172.21.15.46:0/3036591317 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb95c0b29f0 0x7fb95c0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:25.656 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.656+0000 7fb970e10700 1 --2- 172.21.15.46:0/3036591317 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb95c0b29f0 0x7fb95c0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.656+0000 7fb969ffb700 1 -- 172.21.15.46:0/3036591317 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fb9580e2920 con 0x7fb96c107df0 2024-05-09T05:02:25.657 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.656+0000 7fb970e10700 1 --2- 172.21.15.46:0/3036591317 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb95c0b29f0 0x7fb95c0b4f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb960009990 tx=0x7fb960000bc0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:25.659 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.659+0000 7fb969ffb700 1 -- 172.21.15.46:0/3036591317 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb958014030 con 0x7fb96c107df0 2024-05-09T05:02:25.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.890+0000 7fb973074700 1 -- 172.21.15.46:0/3036591317 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb96c06bc30 con 0x7fb96c107df0 2024-05-09T05:02:25.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.891+0000 7fb969ffb700 1 -- 172.21.15.46:0/3036591317 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v95) ==== 81+0+942 (secure 0 0 0) 0x7fb96c06bc30 con 0x7fb96c107df0 2024-05-09T05:02:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/181034377 2024-05-10T04:51:05.539029+0000 2024-05-09T05:02:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1963687293 2024-05-10T04:51:05.539029+0000 2024-05-09T05:02:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/2215389074 2024-05-10T04:51:05.539029+0000 2024-05-09T05:02:25.891 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/123334554 2024-05-10T04:40:27.201119+0000 2024-05-09T05:02:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1596672353 2024-05-10T04:40:27.201119+0000 2024-05-09T05:02:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/3691531627 2024-05-10T04:51:05.539029+0000 2024-05-09T05:02:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1833069872 2024-05-10T04:51:05.539029+0000 2024-05-09T05:02:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1846482551 2024-05-10T04:39:59.515628+0000 2024-05-09T05:02:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1069571878 2024-05-10T04:39:59.515628+0000 2024-05-09T05:02:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/2738158981 2024-05-10T04:39:59.515628+0000 2024-05-09T05:02:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/3625745146 2024-05-10T04:39:59.515628+0000 2024-05-09T05:02:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6800/3691531627 2024-05-10T04:51:05.539029+0000 2024-05-09T05:02:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/123334554 2024-05-10T04:40:27.201119+0000 2024-05-09T05:02:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/564539954 2024-05-10T04:40:27.201119+0000 2024-05-09T05:02:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:0/1797199863 2024-05-10T04:40:27.201119+0000 2024-05-09T05:02:25.892 INFO:teuthology.orchestra.run.smithi046.stdout:172.21.15.46:6801/2738158981 2024-05-10T04:39:59.515628+0000 2024-05-09T05:02:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.893+0000 7fb973074700 1 -- 172.21.15.46:0/3036591317 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb95c0b29f0 msgr2=0x7fb95c0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.893+0000 7fb973074700 1 --2- 172.21.15.46:0/3036591317 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb95c0b29f0 0x7fb95c0b4f00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fb960009990 tx=0x7fb960000bc0 comp rx=0 tx=0).stop 2024-05-09T05:02:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.893+0000 7fb973074700 1 -- 172.21.15.46:0/3036591317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb96c107df0 msgr2=0x7fb96c199f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.893+0000 7fb973074700 1 --2- 172.21.15.46:0/3036591317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb96c107df0 0x7fb96c199f40 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fb95800ca10 tx=0x7fb95800cd20 comp rx=0 tx=0).stop 2024-05-09T05:02:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.893+0000 7fb973074700 1 -- 172.21.15.46:0/3036591317 shutdown_connections 2024-05-09T05:02:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.893+0000 7fb973074700 1 --2- 172.21.15.46:0/3036591317 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fb95c0b29f0 0x7fb95c0b4f00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.893+0000 7fb973074700 1 --2- 172.21.15.46:0/3036591317 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fb96c108740 0x7fb96c19a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.893+0000 7fb973074700 1 --2- 172.21.15.46:0/3036591317 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fb96c1034c0 0x7fb96c199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.893+0000 7fb973074700 1 --2- 172.21.15.46:0/3036591317 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fb96c107df0 0x7fb96c199f40 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.893+0000 7fb973074700 1 -- 172.21.15.46:0/3036591317 >> 172.21.15.46:0/3036591317 conn(0x7fb96c0fedc0 msgr2=0x7fb96c1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.894+0000 7fb973074700 1 -- 172.21.15.46:0/3036591317 shutdown_connections 2024-05-09T05:02:25.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:25.894+0000 7fb973074700 1 -- 172.21.15.46:0/3036591317 wait complete. 2024-05-09T05:02:25.895 INFO:teuthology.orchestra.run.smithi046.stderr:listed 16 entries 2024-05-09T05:02:26.653 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-09T05:02:26.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:02:26.653 DEBUG:teuthology.orchestra.run.smithi046:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-09T05:02:26.731 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T05:02:26.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:02:26.731 DEBUG:teuthology.orchestra.run.smithi046:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-09T05:02:26.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:02:26.793 DEBUG:teuthology.orchestra.run.smithi046:> ip netns list 2024-05-09T05:02:26.810 INFO:teuthology.orchestra.run.smithi046.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-09T05:02:26.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:02:26.810 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-09T05:02:26.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:26 smithi046 ceph-mon[30136]: pgmap v353: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T05:02:26.873 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:26 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3036591317' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T05:02:26.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-09T05:02:26.878 DEBUG:teuthology.orchestra.run.smithi046:> sudo ip link delete ceph-brx 2024-05-09T05:02:26.993 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-09T05:02:27.004 INFO:tasks.cephadm:Teardown begin 2024-05-09T05:02:27.004 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-09T05:02:27.005 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:02:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:26 smithi072 ceph-mon[34088]: pgmap v353: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T05:02:27.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:26 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3036591317' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T05:02:27.264 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:02:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:26 smithi110 ceph-mon[34904]: pgmap v353: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2024-05-09T05:02:27.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:26 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3036591317' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-09T05:02:29.069 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.068+0000 7f49f9cd9700 1 -- 172.21.15.46:0/2125106999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f4108f10 msgr2=0x7f49f4109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.068+0000 7f49f9cd9700 1 --2- 172.21.15.46:0/2125106999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f4108f10 0x7f49f4109320 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f49dc0099c0 tx=0x7f49dc009cd0 comp rx=0 tx=0).stop 2024-05-09T05:02:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.068+0000 7f49f9cd9700 1 -- 172.21.15.46:0/2125106999 shutdown_connections 2024-05-09T05:02:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.068+0000 7f49f9cd9700 1 --2- 172.21.15.46:0/2125106999 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49f4109860 0x7f49f41111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.068+0000 7f49f9cd9700 1 --2- 172.21.15.46:0/2125106999 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49f41045e0 0x7f49f41049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.068+0000 7f49f9cd9700 1 --2- 172.21.15.46:0/2125106999 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f4108f10 0x7f49f4109320 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.068+0000 7f49f9cd9700 1 -- 172.21.15.46:0/2125106999 >> 172.21.15.46:0/2125106999 conn(0x7f49f40ffe70 msgr2=0x7f49f41022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.069+0000 7f49f9cd9700 1 -- 172.21.15.46:0/2125106999 shutdown_connections 2024-05-09T05:02:29.070 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.069+0000 7f49f9cd9700 1 -- 172.21.15.46:0/2125106999 wait complete. 2024-05-09T05:02:29.070 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:28 smithi046 ceph-mon[30136]: pgmap v354: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2024-05-09T05:02:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.069+0000 7f49f9cd9700 1 Processor -- start 2024-05-09T05:02:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.070+0000 7f49f9cd9700 1 -- start start 2024-05-09T05:02:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.070+0000 7f49f9cd9700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49f41045e0 0x7f49f419ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.070+0000 7f49f9cd9700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f4108f10 0x7f49f419b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.070+0000 7f49f9cd9700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49f4109860 0x7f49f419f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.070+0000 7f49f9cd9700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f49f4114470 con 0x7f49f4108f10 2024-05-09T05:02:29.072 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.070+0000 7f49f9cd9700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41142f0 con 0x7f49f4109860 2024-05-09T05:02:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.070+0000 7f49f9cd9700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f49f41145f0 con 0x7f49f41045e0 2024-05-09T05:02:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.070+0000 7f49f2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f4108f10 0x7f49f419b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.070+0000 7f49f2ffd700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f4108f10 0x7f49f419b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45378/0 (socket says 172.21.15.46:45378) 2024-05-09T05:02:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.070+0000 7f49f2ffd700 1 -- 172.21.15.46:0/2934872732 learned_addr learned my addr 172.21.15.46:0/2934872732 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:02:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.070+0000 7f49f37fe700 1 --2- 172.21.15.46:0/2934872732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49f41045e0 0x7f49f419ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.070+0000 7f49f3fff700 1 --2- 172.21.15.46:0/2934872732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49f4109860 0x7f49f419f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.071+0000 7f49f2ffd700 1 -- 172.21.15.46:0/2934872732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49f41045e0 msgr2=0x7f49f419ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.071+0000 7f49f2ffd700 1 --2- 172.21.15.46:0/2934872732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49f41045e0 0x7f49f419ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.071+0000 7f49f2ffd700 1 -- 172.21.15.46:0/2934872732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49f4109860 msgr2=0x7f49f419f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.071+0000 7f49f2ffd700 1 --2- 172.21.15.46:0/2934872732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49f4109860 0x7f49f419f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.071+0000 7f49f2ffd700 1 -- 172.21.15.46:0/2934872732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f419fca0 con 0x7f49f4108f10 2024-05-09T05:02:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.071+0000 7f49f2ffd700 1 --2- 172.21.15.46:0/2934872732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f4108f10 0x7f49f419b420 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f49dc006010 tx=0x7f49dc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.071+0000 7f49f0ff9700 1 -- 172.21.15.46:0/2934872732 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc027070 con 0x7f49f4108f10 2024-05-09T05:02:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.071+0000 7f49f9cd9700 1 -- 172.21.15.46:0/2934872732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f419ff30 con 0x7f49f4108f10 2024-05-09T05:02:29.073 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.071+0000 7f49f0ff9700 1 -- 172.21.15.46:0/2934872732 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f49dc004020 con 0x7f49f4108f10 2024-05-09T05:02:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.071+0000 7f49f9cd9700 1 -- 172.21.15.46:0/2934872732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f41a0460 con 0x7f49f4108f10 2024-05-09T05:02:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.071+0000 7f49f0ff9700 1 -- 172.21.15.46:0/2934872732 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49dc022be0 con 0x7f49f4108f10 2024-05-09T05:02:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.073+0000 7f49f9cd9700 1 -- 172.21.15.46:0/2934872732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f40512c0 con 0x7f49f4108f10 2024-05-09T05:02:29.074 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.074+0000 7f49f0ff9700 1 -- 172.21.15.46:0/2934872732 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f49dc02a070 con 0x7f49f4108f10 2024-05-09T05:02:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.075+0000 7f49f0ff9700 1 --2- 172.21.15.46:0/2934872732 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49e00b2a70 0x7f49e00b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.075+0000 7f49f37fe700 1 --2- 172.21.15.46:0/2934872732 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49e00b2a70 0x7f49e00b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.075+0000 7f49f0ff9700 1 -- 172.21.15.46:0/2934872732 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f49dc0e3450 con 0x7f49f4108f10 2024-05-09T05:02:29.075 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.075+0000 7f49f37fe700 1 --2- 172.21.15.46:0/2934872732 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49e00b2a70 0x7f49e00b4f80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009550 tx=0x7f49e40091b0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:29.080 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.080+0000 7f49f0ff9700 1 -- 172.21.15.46:0/2934872732 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49dc013070 con 0x7f49f4108f10 2024-05-09T05:02:29.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:28 smithi072 ceph-mon[34088]: pgmap v354: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2024-05-09T05:02:29.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:28 smithi110 ceph-mon[34904]: pgmap v354: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 87 B/s wr, 7 op/s 2024-05-09T05:02:29.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.342+0000 7f49f9cd9700 1 -- 172.21.15.46:0/2934872732 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49f419c1e0 con 0x7f49f4108f10 2024-05-09T05:02:29.342 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.342+0000 7f49f0ff9700 1 -- 172.21.15.46:0/2934872732 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7f49dc0aae10 con 0x7f49f4108f10 2024-05-09T05:02:29.343 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T05:02:29.343 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T05:02:29.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.345+0000 7f49f9cd9700 1 -- 172.21.15.46:0/2934872732 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49e00b2a70 msgr2=0x7f49e00b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:29.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.345+0000 7f49f9cd9700 1 --2- 172.21.15.46:0/2934872732 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49e00b2a70 0x7f49e00b4f80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f49e4009550 tx=0x7f49e40091b0 comp rx=0 tx=0).stop 2024-05-09T05:02:29.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.345+0000 7f49f9cd9700 1 -- 172.21.15.46:0/2934872732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f4108f10 msgr2=0x7f49f419b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:29.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.345+0000 7f49f9cd9700 1 --2- 172.21.15.46:0/2934872732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f4108f10 0x7f49f419b420 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f49dc006010 tx=0x7f49dc0047b0 comp rx=0 tx=0).stop 2024-05-09T05:02:29.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.345+0000 7f49f9cd9700 1 -- 172.21.15.46:0/2934872732 shutdown_connections 2024-05-09T05:02:29.345 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.345+0000 7f49f9cd9700 1 --2- 172.21.15.46:0/2934872732 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f49e00b2a70 0x7f49e00b4f80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:29.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.345+0000 7f49f9cd9700 1 --2- 172.21.15.46:0/2934872732 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f49f4109860 0x7f49f419f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:29.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.345+0000 7f49f9cd9700 1 --2- 172.21.15.46:0/2934872732 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f49f41045e0 0x7f49f419ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:29.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.345+0000 7f49f9cd9700 1 --2- 172.21.15.46:0/2934872732 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f49f4108f10 0x7f49f419b420 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:29.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.345+0000 7f49f9cd9700 1 -- 172.21.15.46:0/2934872732 >> 172.21.15.46:0/2934872732 conn(0x7f49f40ffe70 msgr2=0x7f49f41071d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:29.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.345+0000 7f49f9cd9700 1 -- 172.21.15.46:0/2934872732 shutdown_connections 2024-05-09T05:02:29.346 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:29.345+0000 7f49f9cd9700 1 -- 172.21.15.46:0/2934872732 wait complete. 2024-05-09T05:02:29.346 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T05:02:30.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:29 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2934872732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:02:30.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:29 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:02:30.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:29 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2934872732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:02:30.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:29 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:02:30.159 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-09T05:02:30.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:29 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2934872732' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:02:30.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:29 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-09T05:02:30.420 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:02:31.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:30 smithi072 ceph-mon[34088]: pgmap v355: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T05:02:31.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:30 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:02:31.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:30 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:02:31.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:30 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:02:31.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:30 smithi110 ceph-mon[34904]: pgmap v355: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T05:02:31.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:30 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:02:31.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:30 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:02:31.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:30 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:02:31.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:30 smithi046 ceph-mon[30136]: pgmap v355: 97 pgs: 97 active+clean; 3.9 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-05-09T05:02:31.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:30 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-09T05:02:31.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:30 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-09T05:02:31.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:30 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' 2024-05-09T05:02:32.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.153+0000 7fa417ff5700 1 -- 172.21.15.46:0/2227084683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa410108f10 msgr2=0x7fa410109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:32.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.153+0000 7fa417ff5700 1 --2- 172.21.15.46:0/2227084683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa410108f10 0x7fa410109320 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fa4000099c0 tx=0x7fa400009cd0 comp rx=0 tx=0).stop 2024-05-09T05:02:32.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.153+0000 7fa417ff5700 1 -- 172.21.15.46:0/2227084683 shutdown_connections 2024-05-09T05:02:32.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.153+0000 7fa417ff5700 1 --2- 172.21.15.46:0/2227084683 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa410109860 0x7fa4101111c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:32.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.153+0000 7fa417ff5700 1 --2- 172.21.15.46:0/2227084683 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa4101045e0 0x7fa4101049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:32.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.153+0000 7fa417ff5700 1 --2- 172.21.15.46:0/2227084683 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa410108f10 0x7fa410109320 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:32.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.153+0000 7fa417ff5700 1 -- 172.21.15.46:0/2227084683 >> 172.21.15.46:0/2227084683 conn(0x7fa4100ffef0 msgr2=0x7fa410102340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:32.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.153+0000 7fa417ff5700 1 -- 172.21.15.46:0/2227084683 shutdown_connections 2024-05-09T05:02:32.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.153+0000 7fa417ff5700 1 -- 172.21.15.46:0/2227084683 wait complete. 2024-05-09T05:02:32.156 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.154+0000 7fa417ff5700 1 Processor -- start 2024-05-09T05:02:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.154+0000 7fa417ff5700 1 -- start start 2024-05-09T05:02:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa417ff5700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4101045e0 0x7fa41019af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa417ff5700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa410108f10 0x7fa41019b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa417ff5700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa410109860 0x7fa41019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa417ff5700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fa410114530 con 0x7fa410109860 2024-05-09T05:02:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa417ff5700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fa4101143b0 con 0x7fa4101045e0 2024-05-09T05:02:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa417ff5700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fa4101146b0 con 0x7fa410108f10 2024-05-09T05:02:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa416592700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa410109860 0x7fa41019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa416592700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa410109860 0x7fa41019f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:45398/0 (socket says 172.21.15.46:45398) 2024-05-09T05:02:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa416592700 1 -- 172.21.15.46:0/4171761433 learned_addr learned my addr 172.21.15.46:0/4171761433 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:02:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa415d91700 1 --2- 172.21.15.46:0/4171761433 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4101045e0 0x7fa41019af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa415590700 1 --2- 172.21.15.46:0/4171761433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa410108f10 0x7fa41019b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa416592700 1 -- 172.21.15.46:0/4171761433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa410108f10 msgr2=0x7fa41019b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa416592700 1 --2- 172.21.15.46:0/4171761433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa410108f10 0x7fa41019b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:32.157 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa416592700 1 -- 172.21.15.46:0/4171761433 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4101045e0 msgr2=0x7fa41019af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa416592700 1 --2- 172.21.15.46:0/4171761433 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4101045e0 0x7fa41019af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa416592700 1 -- 172.21.15.46:0/4171761433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa41019fdd0 con 0x7fa410109860 2024-05-09T05:02:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.155+0000 7fa416592700 1 --2- 172.21.15.46:0/4171761433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa410109860 0x7fa41019f730 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa40800b970 tx=0x7fa40800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.156+0000 7fa406ffd700 1 -- 172.21.15.46:0/4171761433 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa40800d860 con 0x7fa410109860 2024-05-09T05:02:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.156+0000 7fa406ffd700 1 -- 172.21.15.46:0/4171761433 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa40800e430 con 0x7fa410109860 2024-05-09T05:02:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.156+0000 7fa417ff5700 1 -- 172.21.15.46:0/4171761433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4101a00c0 con 0x7fa410109860 2024-05-09T05:02:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.156+0000 7fa406ffd700 1 -- 172.21.15.46:0/4171761433 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa408017660 con 0x7fa410109860 2024-05-09T05:02:32.158 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.156+0000 7fa417ff5700 1 -- 172.21.15.46:0/4171761433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa410076b00 con 0x7fa410109860 2024-05-09T05:02:32.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.157+0000 7fa417ff5700 1 -- 172.21.15.46:0/4171761433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa410105a90 con 0x7fa410109860 2024-05-09T05:02:32.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.158+0000 7fa406ffd700 1 -- 172.21.15.46:0/4171761433 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa40800da00 con 0x7fa410109860 2024-05-09T05:02:32.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.159+0000 7fa406ffd700 1 --2- 172.21.15.46:0/4171761433 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3fc0b2a70 0x7fa3fc0b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:32.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.159+0000 7fa415d91700 1 --2- 172.21.15.46:0/4171761433 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3fc0b2a70 0x7fa3fc0b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:32.159 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.159+0000 7fa406ffd700 1 -- 172.21.15.46:0/4171761433 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fa4080e2260 con 0x7fa410109860 2024-05-09T05:02:32.160 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.160+0000 7fa415d91700 1 --2- 172.21.15.46:0/4171761433 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3fc0b2a70 0x7fa3fc0b4f80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa40c005950 tx=0x7fa40c00b410 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:32.164 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.164+0000 7fa406ffd700 1 -- 172.21.15.46:0/4171761433 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa408013030 con 0x7fa410109860 2024-05-09T05:02:32.429 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.429+0000 7fa417ff5700 1 -- 172.21.15.46:0/4171761433 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa41019c260 con 0x7fa410109860 2024-05-09T05:02:32.430 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.430+0000 7fa406ffd700 1 -- 172.21.15.46:0/4171761433 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11862 (secure 0 0 0) 0x7fa4080a9bf0 con 0x7fa410109860 2024-05-09T05:02:32.430 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T05:02:32.430 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14541,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6835/3023074192","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6834","nonce":3023074192},{"type":"v1","addr":"172.21.15.46:6835","nonce":3023074192}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14553,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6837/3965071509","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6836","nonce":3965071509},{"type":"v1","addr":"172.21.15.46:6837","nonce":3965071509}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6839/952386829","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6838","nonce":952386829},{"type":"v1","addr":"172.21.15.46:6839","nonce":952386829}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.46:6841/204132247","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6840","nonce":204132247},{"type":"v1","addr":"172.21.15.46:6841","nonce":204132247}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24400,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.110:6833/2118887975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6832","nonce":2118887975},{"type":"v1","addr":"172.21.15.110:6833","nonce":2118887975}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24419,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.72:6833/815890099","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6832","nonce":815890099},{"type":"v1","addr":"172.21.15.72:6833","nonce":815890099}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-09T04:47:50.321554+0000","modified":"2024-05-09T04:51:03.085358+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24503,"mds_1":24473,"mds_2":24448},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24427":{"gid":24427,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6835/2299300184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6834","nonce":2299300184},{"type":"v1","addr":"172.21.15.110:6835","nonce":2299300184}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.110:6837/2612088705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6836","nonce":2612088705},{"type":"v1","addr":"172.21.15.110:6837","nonce":2612088705}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.72:6835/4256517546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6834","nonce":4256517546},{"type":"v1","addr":"172.21.15.72:6835","nonce":4256517546}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"i","rank":2,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.110:6839/556391603","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6838","nonce":556391603},{"type":"v1","addr":"172.21.15.110:6839","nonce":556391603}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"h","rank":1,"incarnation":18,"state":"up:active","state_seq":12,"addr":"172.21.15.72:6837/3365501578","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6836","nonce":3365501578},{"type":"v1","addr":"172.21.15.72:6837","nonce":3365501578}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24503":{"gid":24503,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.72:6839/2905856798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6838","nonce":2905856798},{"type":"v1","addr":"172.21.15.72:6839","nonce":2905856798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24503,"qdb_cluster":[24448,24473,24503]},"id":1}]} 2024-05-09T05:02:32.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.432+0000 7fa417ff5700 1 -- 172.21.15.46:0/4171761433 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3fc0b2a70 msgr2=0x7fa3fc0b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:32.432 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.432+0000 7fa417ff5700 1 --2- 172.21.15.46:0/4171761433 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3fc0b2a70 0x7fa3fc0b4f80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fa40c005950 tx=0x7fa40c00b410 comp rx=0 tx=0).stop 2024-05-09T05:02:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.432+0000 7fa417ff5700 1 -- 172.21.15.46:0/4171761433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa410109860 msgr2=0x7fa41019f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.432+0000 7fa417ff5700 1 --2- 172.21.15.46:0/4171761433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa410109860 0x7fa41019f730 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fa40800b970 tx=0x7fa40800bc80 comp rx=0 tx=0).stop 2024-05-09T05:02:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.432+0000 7fa417ff5700 1 -- 172.21.15.46:0/4171761433 shutdown_connections 2024-05-09T05:02:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.432+0000 7fa417ff5700 1 --2- 172.21.15.46:0/4171761433 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fa3fc0b2a70 0x7fa3fc0b4f80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.432+0000 7fa417ff5700 1 --2- 172.21.15.46:0/4171761433 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fa410109860 0x7fa41019f730 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.432+0000 7fa417ff5700 1 --2- 172.21.15.46:0/4171761433 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fa4101045e0 0x7fa41019af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.432+0000 7fa417ff5700 1 --2- 172.21.15.46:0/4171761433 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fa410108f10 0x7fa41019b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.432+0000 7fa417ff5700 1 -- 172.21.15.46:0/4171761433 >> 172.21.15.46:0/4171761433 conn(0x7fa4100ffef0 msgr2=0x7fa41010a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.432+0000 7fa417ff5700 1 -- 172.21.15.46:0/4171761433 shutdown_connections 2024-05-09T05:02:32.433 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:32.432+0000 7fa417ff5700 1 -- 172.21.15.46:0/4171761433 wait complete. 2024-05-09T05:02:32.434 INFO:teuthology.orchestra.run.smithi046.stderr:dumped fsmap epoch 25 2024-05-09T05:02:33.040 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:32 smithi046 ceph-mon[30136]: pgmap v356: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-09T05:02:33.041 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:32 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/4171761433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:02:33.069 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-09T05:02:33.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:32 smithi072 ceph-mon[34088]: pgmap v356: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-09T05:02:33.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:32 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/4171761433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:02:33.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:32 smithi110 ceph-mon[34904]: pgmap v356: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-09T05:02:33.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:32 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/4171761433' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-09T05:02:33.323 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:02:35.052 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.052+0000 7fcc650ea700 1 -- 172.21.15.46:0/3590137505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc60109850 msgr2=0x7fcc601111b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:35.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.052+0000 7fcc650ea700 1 --2- 172.21.15.46:0/3590137505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc60109850 0x7fcc601111b0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800b0d0 tx=0x7fcc5800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:02:35.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.052+0000 7fcc650ea700 1 -- 172.21.15.46:0/3590137505 shutdown_connections 2024-05-09T05:02:35.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.052+0000 7fcc650ea700 1 --2- 172.21.15.46:0/3590137505 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc60109850 0x7fcc601111b0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:35.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.052+0000 7fcc650ea700 1 --2- 172.21.15.46:0/3590137505 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc601045d0 0x7fcc601049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:35.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.052+0000 7fcc650ea700 1 --2- 172.21.15.46:0/3590137505 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc60108f00 0x7fcc60109310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:35.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.052+0000 7fcc650ea700 1 -- 172.21.15.46:0/3590137505 >> 172.21.15.46:0/3590137505 conn(0x7fcc600ffea0 msgr2=0x7fcc601022f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:35.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.052+0000 7fcc650ea700 1 -- 172.21.15.46:0/3590137505 shutdown_connections 2024-05-09T05:02:35.053 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.052+0000 7fcc650ea700 1 -- 172.21.15.46:0/3590137505 wait complete. 2024-05-09T05:02:35.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.053+0000 7fcc650ea700 1 Processor -- start 2024-05-09T05:02:35.054 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:34 smithi046 ceph-mon[30136]: pgmap v357: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T05:02:35.055 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.053+0000 7fcc650ea700 1 -- start start 2024-05-09T05:02:35.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.053+0000 7fcc650ea700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc601045d0 0x7fcc6019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:35.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.053+0000 7fcc650ea700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc60108f00 0x7fcc6019b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:35.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.053+0000 7fcc650ea700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc60109850 0x7fcc6019f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:35.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.054+0000 7fcc650ea700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fcc601144a0 con 0x7fcc601045d0 2024-05-09T05:02:35.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.054+0000 7fcc650ea700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fcc60114320 con 0x7fcc60108f00 2024-05-09T05:02:35.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.054+0000 7fcc650ea700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fcc60114620 con 0x7fcc60109850 2024-05-09T05:02:35.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.054+0000 7fcc5ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc601045d0 0x7fcc6019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:35.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.054+0000 7fcc5ed9d700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc601045d0 0x7fcc6019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42154/0 (socket says 172.21.15.46:42154) 2024-05-09T05:02:35.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.054+0000 7fcc5ed9d700 1 -- 172.21.15.46:0/4130001367 learned_addr learned my addr 172.21.15.46:0/4130001367 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:02:35.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.054+0000 7fcc5f59e700 1 --2- 172.21.15.46:0/4130001367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc60109850 0x7fcc6019f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:35.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.054+0000 7fcc5e59c700 1 --2- 172.21.15.46:0/4130001367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc60108f00 0x7fcc6019b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:35.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.054+0000 7fcc5ed9d700 1 -- 172.21.15.46:0/4130001367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc60109850 msgr2=0x7fcc6019f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:35.056 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.054+0000 7fcc5ed9d700 1 --2- 172.21.15.46:0/4130001367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc60109850 0x7fcc6019f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:35.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.054+0000 7fcc5ed9d700 1 -- 172.21.15.46:0/4130001367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc60108f00 msgr2=0x7fcc6019b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:35.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.054+0000 7fcc5ed9d700 1 --2- 172.21.15.46:0/4130001367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc60108f00 0x7fcc6019b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:35.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.054+0000 7fcc5ed9d700 1 -- 172.21.15.46:0/4130001367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc6019fc30 con 0x7fcc601045d0 2024-05-09T05:02:35.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.054+0000 7fcc5ed9d700 1 --2- 172.21.15.46:0/4130001367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc601045d0 0x7fcc6019ae10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000c970 tx=0x7fcc5000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:35.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.055+0000 7fcc47fff700 1 -- 172.21.15.46:0/4130001367 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50007aa0 con 0x7fcc601045d0 2024-05-09T05:02:35.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.055+0000 7fcc650ea700 1 -- 172.21.15.46:0/4130001367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc6019ff20 con 0x7fcc601045d0 2024-05-09T05:02:35.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.055+0000 7fcc650ea700 1 -- 172.21.15.46:0/4130001367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc601a0460 con 0x7fcc601045d0 2024-05-09T05:02:35.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.055+0000 7fcc47fff700 1 -- 172.21.15.46:0/4130001367 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc50004ce0 con 0x7fcc601045d0 2024-05-09T05:02:35.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.055+0000 7fcc47fff700 1 -- 172.21.15.46:0/4130001367 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc50005510 con 0x7fcc601045d0 2024-05-09T05:02:35.057 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.056+0000 7fcc650ea700 1 -- 172.21.15.46:0/4130001367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc4c005330 con 0x7fcc601045d0 2024-05-09T05:02:35.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.058+0000 7fcc47fff700 1 -- 172.21.15.46:0/4130001367 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcc50007c40 con 0x7fcc601045d0 2024-05-09T05:02:35.058 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.059+0000 7fcc47fff700 1 --2- 172.21.15.46:0/4130001367 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fcc480b2a70 0x7fcc480b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:35.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.059+0000 7fcc5e59c700 1 --2- 172.21.15.46:0/4130001367 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fcc480b2a70 0x7fcc480b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:35.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.059+0000 7fcc47fff700 1 -- 172.21.15.46:0/4130001367 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7fcc500e24e0 con 0x7fcc601045d0 2024-05-09T05:02:35.059 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.059+0000 7fcc5e59c700 1 --2- 172.21.15.46:0/4130001367 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fcc480b2a70 0x7fcc480b4f80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcc54005fd0 tx=0x7fcc5400af20 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:35.062 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.062+0000 7fcc47fff700 1 -- 172.21.15.46:0/4130001367 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc500a9f90 con 0x7fcc601045d0 2024-05-09T05:02:35.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:34 smithi072 ceph-mon[34088]: pgmap v357: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T05:02:35.279 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.279+0000 7fcc650ea700 1 -- 172.21.15.46:0/4130001367 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fcc4c0058f0 con 0x7fcc601045d0 2024-05-09T05:02:35.281 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.280+0000 7fcc47fff700 1 -- 172.21.15.46:0/4130001367 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) ==== 74+0+23152 (secure 0 0 0) 0x7fcc500a9890 con 0x7fcc601045d0 2024-05-09T05:02:35.282 INFO:teuthology.orchestra.run.smithi046.stdout: 2024-05-09T05:02:35.282 INFO:teuthology.orchestra.run.smithi046.stdout:{"epoch":95,"fsid":"ecf9ae2c-0dbd-11ef-bc97-c7b262605968","created":"2024-05-09T04:39:25.633876+0000","modified":"2024-05-09T05:02:16.145371+0000","last_up_change":"2024-05-09T04:46:46.418597+0000","last_in_change":"2024-05-09T04:46:25.903582+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-09T04:42:53.301400+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-09T04:47:43.413569+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-09T04:47:47.264915+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d79cd3e7-6f82-4d60-96d9-26d8146c962f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6802","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6803","nonce":1576436508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6804","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6805","nonce":1576436508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6808","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6809","nonce":1576436508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6806","nonce":1576436508},{"type":"v1","addr":"172.21.15.46:6807","nonce":1576436508}]},"public_addr":"172.21.15.46:6803/1576436508","cluster_addr":"172.21.15.46:6805/1576436508","heartbeat_back_addr":"172.21.15.46:6809/1576436508","heartbeat_front_addr":"172.21.15.46:6807/1576436508","state":["exists","up"]},{"osd":1,"uuid":"698eba80-1ebd-4076-b6f9-953d2e7daf76","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6800","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6801","nonce":379039775}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6802","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6803","nonce":379039775}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6806","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6807","nonce":379039775}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6804","nonce":379039775},{"type":"v1","addr":"172.21.15.72:6805","nonce":379039775}]},"public_addr":"172.21.15.72:6801/379039775","cluster_addr":"172.21.15.72:6803/379039775","heartbeat_back_addr":"172.21.15.72:6807/379039775","heartbeat_front_addr":"172.21.15.72:6805/379039775","state":["exists","up"]},{"osd":2,"uuid":"c549b131-cfa7-4724-926f-83e91a752d44","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6800","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6801","nonce":293447227}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6802","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6803","nonce":293447227}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6806","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6807","nonce":293447227}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6804","nonce":293447227},{"type":"v1","addr":"172.21.15.110:6805","nonce":293447227}]},"public_addr":"172.21.15.110:6801/293447227","cluster_addr":"172.21.15.110:6803/293447227","heartbeat_back_addr":"172.21.15.110:6807/293447227","heartbeat_front_addr":"172.21.15.110:6805/293447227","state":["exists","up"]},{"osd":3,"uuid":"549bb4f7-54d9-4cbc-ad40-7f3e717964f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6810","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6811","nonce":1115835653}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6812","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6813","nonce":1115835653}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6816","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6817","nonce":1115835653}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6814","nonce":1115835653},{"type":"v1","addr":"172.21.15.46:6815","nonce":1115835653}]},"public_addr":"172.21.15.46:6811/1115835653","cluster_addr":"172.21.15.46:6813/1115835653","heartbeat_back_addr":"172.21.15.46:6817/1115835653","heartbeat_front_addr":"172.21.15.46:6815/1115835653","state":["exists","up"]},{"osd":4,"uuid":"444cd36b-6034-46cd-9a6b-29dce4ba6405","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6808","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6809","nonce":59186693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6810","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6811","nonce":59186693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6814","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6815","nonce":59186693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6812","nonce":59186693},{"type":"v1","addr":"172.21.15.72:6813","nonce":59186693}]},"public_addr":"172.21.15.72:6809/59186693","cluster_addr":"172.21.15.72:6811/59186693","heartbeat_back_addr":"172.21.15.72:6815/59186693","heartbeat_front_addr":"172.21.15.72:6813/59186693","state":["exists","up"]},{"osd":5,"uuid":"2c2bdd2f-43cf-4208-9ae1-f912807c84e0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6808","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6809","nonce":1712162502}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6810","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6811","nonce":1712162502}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6814","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6815","nonce":1712162502}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6812","nonce":1712162502},{"type":"v1","addr":"172.21.15.110:6813","nonce":1712162502}]},"public_addr":"172.21.15.110:6809/1712162502","cluster_addr":"172.21.15.110:6811/1712162502","heartbeat_back_addr":"172.21.15.110:6815/1712162502","heartbeat_front_addr":"172.21.15.110:6813/1712162502","state":["exists","up"]},{"osd":6,"uuid":"beb0b87f-c3a4-40dd-b224-1843e9301ca9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6818","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6819","nonce":2429650065}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6820","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6821","nonce":2429650065}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6824","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6825","nonce":2429650065}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6822","nonce":2429650065},{"type":"v1","addr":"172.21.15.46:6823","nonce":2429650065}]},"public_addr":"172.21.15.46:6819/2429650065","cluster_addr":"172.21.15.46:6821/2429650065","heartbeat_back_addr":"172.21.15.46:6825/2429650065","heartbeat_front_addr":"172.21.15.46:6823/2429650065","state":["exists","up"]},{"osd":7,"uuid":"4a634675-fad7-4a43-840c-8dec2631dec9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6816","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6817","nonce":337796672}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6818","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6819","nonce":337796672}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6822","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6823","nonce":337796672}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6820","nonce":337796672},{"type":"v1","addr":"172.21.15.72:6821","nonce":337796672}]},"public_addr":"172.21.15.72:6817/337796672","cluster_addr":"172.21.15.72:6819/337796672","heartbeat_back_addr":"172.21.15.72:6823/337796672","heartbeat_front_addr":"172.21.15.72:6821/337796672","state":["exists","up"]},{"osd":8,"uuid":"01b5f638-90f0-4cf4-8042-20d20a725669","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6816","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6817","nonce":189513203}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6818","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6819","nonce":189513203}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6822","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6823","nonce":189513203}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6820","nonce":189513203},{"type":"v1","addr":"172.21.15.110:6821","nonce":189513203}]},"public_addr":"172.21.15.110:6817/189513203","cluster_addr":"172.21.15.110:6819/189513203","heartbeat_back_addr":"172.21.15.110:6823/189513203","heartbeat_front_addr":"172.21.15.110:6821/189513203","state":["exists","up"]},{"osd":9,"uuid":"e423de41-50ec-48f4-a4ed-216de33c4fb5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":61,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6826","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6827","nonce":1762262822}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6828","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6829","nonce":1762262822}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6832","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6833","nonce":1762262822}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.46:6830","nonce":1762262822},{"type":"v1","addr":"172.21.15.46:6831","nonce":1762262822}]},"public_addr":"172.21.15.46:6827/1762262822","cluster_addr":"172.21.15.46:6829/1762262822","heartbeat_back_addr":"172.21.15.46:6833/1762262822","heartbeat_front_addr":"172.21.15.46:6831/1762262822","state":["exists","up"]},{"osd":10,"uuid":"f594d004-9f80-4508-b127-fb693d355f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":66,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6824","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6825","nonce":38453898}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6826","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6827","nonce":38453898}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6830","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6831","nonce":38453898}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.72:6828","nonce":38453898},{"type":"v1","addr":"172.21.15.72:6829","nonce":38453898}]},"public_addr":"172.21.15.72:6825/38453898","cluster_addr":"172.21.15.72:6827/38453898","heartbeat_back_addr":"172.21.15.72:6831/38453898","heartbeat_front_addr":"172.21.15.72:6829/38453898","state":["exists","up"]},{"osd":11,"uuid":"f771d29f-89e3-4f96-9c29-00e424230c1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6824","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6825","nonce":4153529632}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6826","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6827","nonce":4153529632}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6830","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6831","nonce":4153529632}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.110:6828","nonce":4153529632},{"type":"v1","addr":"172.21.15.110:6829","nonce":4153529632}]},"public_addr":"172.21.15.110:6825/4153529632","cluster_addr":"172.21.15.110:6827/4153529632","heartbeat_back_addr":"172.21.15.110:6831/4153529632","heartbeat_front_addr":"172.21.15.110:6829/4153529632","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:02.615801+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:26.453477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:42:50.334627+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:15.357320+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:43:40.403549+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:06.030093+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:31.904091+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:44:57.616372+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:24.045975+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:45:51.160898+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:17.597845+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-09T04:46:44.636629+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.14","mappings":[{"from":2,"to":4}]},{"pgid":"2.1c","mappings":[{"from":8,"to":11}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.e","mappings":[{"from":4,"to":6}]},{"pgid":"3.11","mappings":[{"from":5,"to":9}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1b","mappings":[{"from":5,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":4,"to":7}]},{"pgid":"3.38","mappings":[{"from":3,"to":9}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.46:0/181034377":"2024-05-10T04:51:05.539029+0000","172.21.15.46:0/1963687293":"2024-05-10T04:51:05.539029+0000","172.21.15.46:0/2215389074":"2024-05-10T04:51:05.539029+0000","172.21.15.46:6800/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1596672353":"2024-05-10T04:40:27.201119+0000","172.21.15.46:6801/3691531627":"2024-05-10T04:51:05.539029+0000","172.21.15.46:0/1833069872":"2024-05-10T04:51:05.539029+0000","172.21.15.46:0/1846482551":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/1069571878":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6800/2738158981":"2024-05-10T04:39:59.515628+0000","172.21.15.46:0/3625745146":"2024-05-10T04:39:59.515628+0000","172.21.15.46:6800/3691531627":"2024-05-10T04:51:05.539029+0000","172.21.15.46:6801/123334554":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/564539954":"2024-05-10T04:40:27.201119+0000","172.21.15.46:0/1797199863":"2024-05-10T04:40:27.201119+0000","172.21.15.46:6801/2738158981":"2024-05-10T04:39:59.515628+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-09T05:02:35.283 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.283+0000 7fcc650ea700 1 -- 172.21.15.46:0/4130001367 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fcc480b2a70 msgr2=0x7fcc480b4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:35.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.283+0000 7fcc650ea700 1 --2- 172.21.15.46:0/4130001367 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fcc480b2a70 0x7fcc480b4f80 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcc54005fd0 tx=0x7fcc5400af20 comp rx=0 tx=0).stop 2024-05-09T05:02:35.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.283+0000 7fcc650ea700 1 -- 172.21.15.46:0/4130001367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc601045d0 msgr2=0x7fcc6019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:35.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.283+0000 7fcc650ea700 1 --2- 172.21.15.46:0/4130001367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc601045d0 0x7fcc6019ae10 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000c970 tx=0x7fcc5000cc80 comp rx=0 tx=0).stop 2024-05-09T05:02:35.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.283+0000 7fcc650ea700 1 -- 172.21.15.46:0/4130001367 shutdown_connections 2024-05-09T05:02:35.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.283+0000 7fcc650ea700 1 --2- 172.21.15.46:0/4130001367 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fcc480b2a70 0x7fcc480b4f80 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:35.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.283+0000 7fcc650ea700 1 --2- 172.21.15.46:0/4130001367 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fcc60109850 0x7fcc6019f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:35.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.283+0000 7fcc650ea700 1 --2- 172.21.15.46:0/4130001367 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fcc601045d0 0x7fcc6019ae10 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:35.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.283+0000 7fcc650ea700 1 --2- 172.21.15.46:0/4130001367 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fcc60108f00 0x7fcc6019b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:35.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.283+0000 7fcc650ea700 1 -- 172.21.15.46:0/4130001367 >> 172.21.15.46:0/4130001367 conn(0x7fcc600ffea0 msgr2=0x7fcc60108630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:35.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.283+0000 7fcc650ea700 1 -- 172.21.15.46:0/4130001367 shutdown_connections 2024-05-09T05:02:35.284 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:35.283+0000 7fcc650ea700 1 -- 172.21.15.46:0/4130001367 wait complete. 2024-05-09T05:02:35.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:34 smithi110 ceph-mon[34904]: pgmap v357: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T05:02:35.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-09T05:02:36.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:35 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/4130001367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:02:36.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:35 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/4130001367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:02:36.186 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:02:36.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:35 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/4130001367' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-09T05:02:37.295 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:36 smithi046 ceph-mon[30136]: pgmap v358: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T05:02:37.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:36 smithi110 ceph-mon[34904]: pgmap v358: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T05:02:37.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:36 smithi072 ceph-mon[34088]: pgmap v358: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2024-05-09T05:02:37.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.889+0000 7f23f67ae700 1 -- 172.21.15.46:0/1556813457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f0108760 msgr2=0x7f23f01100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:37.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.889+0000 7f23f67ae700 1 --2- 172.21.15.46:0/1556813457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f0108760 0x7f23f01100c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f23e800b0d0 tx=0x7f23e800b3e0 comp rx=0 tx=0).stop 2024-05-09T05:02:37.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.889+0000 7f23f67ae700 1 -- 172.21.15.46:0/1556813457 shutdown_connections 2024-05-09T05:02:37.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.889+0000 7f23f67ae700 1 --2- 172.21.15.46:0/1556813457 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f0108760 0x7f23f01100c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:37.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.889+0000 7f23f67ae700 1 --2- 172.21.15.46:0/1556813457 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23f01034e0 0x7f23f01038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:37.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.889+0000 7f23f67ae700 1 --2- 172.21.15.46:0/1556813457 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23f0107e10 0x7f23f0108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:37.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.889+0000 7f23f67ae700 1 -- 172.21.15.46:0/1556813457 >> 172.21.15.46:0/1556813457 conn(0x7f23f00fef30 msgr2=0x7f23f0101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:37.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.889+0000 7f23f67ae700 1 -- 172.21.15.46:0/1556813457 shutdown_connections 2024-05-09T05:02:37.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.889+0000 7f23f67ae700 1 -- 172.21.15.46:0/1556813457 wait complete. 2024-05-09T05:02:37.893 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.890+0000 7f23f67ae700 1 Processor -- start 2024-05-09T05:02:37.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.890+0000 7f23f67ae700 1 -- start start 2024-05-09T05:02:37.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.890+0000 7f23f67ae700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f01034e0 0x7f23f019ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:37.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.890+0000 7f23f67ae700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23f0107e10 0x7f23f019b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:37.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.890+0000 7f23f67ae700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23f0108760 0x7f23f019f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:37.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.890+0000 7f23f67ae700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0113390 con 0x7f23f01034e0 2024-05-09T05:02:37.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.890+0000 7f23f67ae700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0113210 con 0x7f23f0107e10 2024-05-09T05:02:37.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.890+0000 7f23f67ae700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f23f0113510 con 0x7f23f0108760 2024-05-09T05:02:37.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.890+0000 7f23effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f01034e0 0x7f23f019ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:37.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.890+0000 7f23effff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f01034e0 0x7f23f019ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42172/0 (socket says 172.21.15.46:42172) 2024-05-09T05:02:37.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.891+0000 7f23effff700 1 -- 172.21.15.46:0/2567475607 learned_addr learned my addr 172.21.15.46:0/2567475607 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:02:37.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.891+0000 7f23ef7fe700 1 --2- 172.21.15.46:0/2567475607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23f0107e10 0x7f23f019b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:37.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.891+0000 7f23f4d4b700 1 --2- 172.21.15.46:0/2567475607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23f0108760 0x7f23f019f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:37.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.891+0000 7f23effff700 1 -- 172.21.15.46:0/2567475607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23f0108760 msgr2=0x7f23f019f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:37.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.891+0000 7f23effff700 1 --2- 172.21.15.46:0/2567475607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23f0108760 0x7f23f019f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:37.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.891+0000 7f23effff700 1 -- 172.21.15.46:0/2567475607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23f0107e10 msgr2=0x7f23f019b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:37.895 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.891+0000 7f23effff700 1 --2- 172.21.15.46:0/2567475607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23f0107e10 0x7f23f019b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:37.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.891+0000 7f23effff700 1 -- 172.21.15.46:0/2567475607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23f019fd00 con 0x7f23f01034e0 2024-05-09T05:02:37.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.891+0000 7f23f4d4b700 1 --2- 172.21.15.46:0/2567475607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23f0108760 0x7f23f019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:02:37.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.891+0000 7f23effff700 1 --2- 172.21.15.46:0/2567475607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f01034e0 0x7f23f019ae30 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f23e000e9a0 tx=0x7f23e000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:37.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.891+0000 7f23ed7fa700 1 -- 172.21.15.46:0/2567475607 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e000cbd0 con 0x7f23f01034e0 2024-05-09T05:02:37.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.891+0000 7f23ed7fa700 1 -- 172.21.15.46:0/2567475607 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23e000cd70 con 0x7f23f01034e0 2024-05-09T05:02:37.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.891+0000 7f23f67ae700 1 -- 172.21.15.46:0/2567475607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23f019fff0 con 0x7f23f01034e0 2024-05-09T05:02:37.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.891+0000 7f23ed7fa700 1 -- 172.21.15.46:0/2567475607 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23e0018680 con 0x7f23f01034e0 2024-05-09T05:02:37.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.891+0000 7f23f67ae700 1 -- 172.21.15.46:0/2567475607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23f01a7910 con 0x7f23f01034e0 2024-05-09T05:02:37.896 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.892+0000 7f23f67ae700 1 -- 172.21.15.46:0/2567475607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23f00512c0 con 0x7f23f01034e0 2024-05-09T05:02:37.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.894+0000 7f23ed7fa700 1 -- 172.21.15.46:0/2567475607 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f23e0010970 con 0x7f23f01034e0 2024-05-09T05:02:37.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.895+0000 7f23ed7fa700 1 --2- 172.21.15.46:0/2567475607 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f23d80b29f0 0x7f23d80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:37.897 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.895+0000 7f23ef7fe700 1 --2- 172.21.15.46:0/2567475607 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f23d80b29f0 0x7f23d80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:37.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.895+0000 7f23ed7fa700 1 -- 172.21.15.46:0/2567475607 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8600+0+0 (secure 0 0 0) 0x7f23e00e2e90 con 0x7f23f01034e0 2024-05-09T05:02:37.898 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.895+0000 7f23ef7fe700 1 --2- 172.21.15.46:0/2567475607 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f23d80b29f0 0x7f23d80b4f00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f23f0102810 tx=0x7f23e4009260 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:37.899 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:37.898+0000 7f23ed7fa700 1 -- 172.21.15.46:0/2567475607 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f23e00aa820 con 0x7f23f01034e0 2024-05-09T05:02:38.162 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:38.161+0000 7f23f67ae700 1 -- 172.21.15.46:0/2567475607 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f23f01a7ae0 con 0x7f23f01034e0 2024-05-09T05:02:38.163 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:37 smithi046 ceph-mon[30136]: pgmap v359: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-09T05:02:38.165 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:38.164+0000 7f23ed7fa700 1 -- 172.21.15.46:0/2567475607 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) ==== 161+0+0 (secure 0 0 0) 0x7f23e00aa120 con 0x7f23f01034e0 2024-05-09T05:02:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:38.167+0000 7f23f67ae700 1 -- 172.21.15.46:0/2567475607 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f23d80b29f0 msgr2=0x7f23d80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:38.167+0000 7f23f67ae700 1 --2- 172.21.15.46:0/2567475607 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f23d80b29f0 0x7f23d80b4f00 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f23f0102810 tx=0x7f23e4009260 comp rx=0 tx=0).stop 2024-05-09T05:02:38.168 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:38.167+0000 7f23f67ae700 1 -- 172.21.15.46:0/2567475607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f01034e0 msgr2=0x7f23f019ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:38.167+0000 7f23f67ae700 1 --2- 172.21.15.46:0/2567475607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f01034e0 0x7f23f019ae30 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f23e000e9a0 tx=0x7f23e000ed60 comp rx=0 tx=0).stop 2024-05-09T05:02:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:38.167+0000 7f23f67ae700 1 -- 172.21.15.46:0/2567475607 shutdown_connections 2024-05-09T05:02:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:38.167+0000 7f23f67ae700 1 --2- 172.21.15.46:0/2567475607 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f23d80b29f0 0x7f23d80b4f00 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:38.167+0000 7f23f67ae700 1 --2- 172.21.15.46:0/2567475607 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f23f0108760 0x7f23f019f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:38.167+0000 7f23f67ae700 1 --2- 172.21.15.46:0/2567475607 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f23f01034e0 0x7f23f019ae30 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:38.167+0000 7f23f67ae700 1 --2- 172.21.15.46:0/2567475607 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f23f0107e10 0x7f23f019b3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:38.167+0000 7f23f67ae700 1 -- 172.21.15.46:0/2567475607 >> 172.21.15.46:0/2567475607 conn(0x7f23f00fef30 msgr2=0x7f23f0107580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:38.167+0000 7f23f67ae700 1 -- 172.21.15.46:0/2567475607 shutdown_connections 2024-05-09T05:02:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:38.167+0000 7f23f67ae700 1 -- 172.21.15.46:0/2567475607 wait complete. 2024-05-09T05:02:38.169 INFO:teuthology.orchestra.run.smithi046.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-09T05:02:38.233 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:37 smithi072 ceph-mon[34088]: pgmap v359: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-09T05:02:38.237 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:37 smithi110 ceph-mon[34904]: pgmap v359: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 597 B/s wr, 8 op/s 2024-05-09T05:02:38.463 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:38 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a[30132]: 2024-05-09T05:02:38.162+0000 7f7211056700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T05:02:38.557 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 05:02:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-c[65218]: ignoring --setuser ceph since I am not root 2024-05-09T05:02:38.557 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 05:02:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-c[65218]: ignoring --setgroup ceph since I am not root 2024-05-09T05:02:38.557 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 05:02:38 smithi110 ceph-mds[65222]: -- 172.21.15.110:0/1611319300 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562b9d3b9a40 con 0x562b9d401180 2024-05-09T05:02:38.557 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 05:02:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-c[65218]: starting mds.c at 2024-05-09T05:02:38.558 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 05:02:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-f[64908]: ignoring --setuser ceph since I am not root 2024-05-09T05:02:38.558 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 05:02:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-f[64908]: ignoring --setgroup ceph since I am not root 2024-05-09T05:02:38.558 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 05:02:38 smithi110 ceph-mds[64912]: -- 172.21.15.110:0/3311760304 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a97d371a40 con 0x55a97d3b9180 2024-05-09T05:02:38.558 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 05:02:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-f[64908]: starting mds.f at 2024-05-09T05:02:38.558 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 05:02:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-i[65526]: ignoring --setuser ceph since I am not root 2024-05-09T05:02:38.558 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 05:02:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-i[65526]: ignoring --setgroup ceph since I am not root 2024-05-09T05:02:38.558 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 05:02:38 smithi110 ceph-mds[65530]: -- 172.21.15.110:0/1886817408 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ded14dba40 con 0x55ded1523a80 2024-05-09T05:02:38.559 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 05:02:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-i[65526]: starting mds.i at 2024-05-09T05:02:38.630 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 05:02:38 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-b[63750]: ignoring --setuser ceph since I am not root 2024-05-09T05:02:38.630 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 05:02:38 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-b[63750]: ignoring --setgroup ceph since I am not root 2024-05-09T05:02:38.630 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 05:02:38 smithi072 ceph-mds[63754]: -- 172.21.15.72:0/2936488314 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a264147a40 con 0x55a26418f180 2024-05-09T05:02:38.631 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 05:02:38 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-b[63750]: starting mds.b at 2024-05-09T05:02:38.631 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 05:02:38 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-h[63443]: ignoring --setuser ceph since I am not root 2024-05-09T05:02:38.631 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 05:02:38 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-h[63443]: ignoring --setgroup ceph since I am not root 2024-05-09T05:02:38.631 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 05:02:38 smithi072 ceph-mds[63447]: -- 172.21.15.72:0/3902499697 <== mon.1 v2:172.21.15.110:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55eb436a3a40 con 0x55eb436eba80 2024-05-09T05:02:38.631 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 05:02:38 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-h[63443]: starting mds.h at 2024-05-09T05:02:38.631 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 05:02:38 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-e[63126]: ignoring --setuser ceph since I am not root 2024-05-09T05:02:38.631 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 05:02:38 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-e[63126]: ignoring --setgroup ceph since I am not root 2024-05-09T05:02:38.631 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 05:02:38 smithi072 ceph-mds[63130]: -- 172.21.15.72:0/987605516 <== mon.2 v2:172.21.15.72:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e8b7cda40 con 0x563e8b815a80 2024-05-09T05:02:38.631 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 05:02:38 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-e[63126]: starting mds.e at 2024-05-09T05:02:38.632 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 05:02:38 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-e[63126]: 2024-05-09T05:02:38.237+0000 7f4c1cd75700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2024-05-09T05:02:38.784 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-09T05:02:39.038 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:02:39.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:38 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2567475607' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T05:02:39.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:38 smithi046 ceph-mon[30136]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T05:02:39.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:38 smithi046 ceph-mon[30136]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T05:02:39.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:38 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/2567475607' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T05:02:39.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:38 smithi046 ceph-mon[30136]: osdmap e96: 12 total, 12 up, 12 in 2024-05-09T05:02:39.080 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:38 smithi046 ceph-mon[30136]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-09T05:02:39.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:38 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2567475607' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T05:02:39.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:38 smithi110 ceph-mon[34904]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T05:02:39.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:38 smithi110 ceph-mon[34904]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T05:02:39.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:38 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/2567475607' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T05:02:39.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:38 smithi110 ceph-mon[34904]: osdmap e96: 12 total, 12 up, 12 in 2024-05-09T05:02:39.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:38 smithi110 ceph-mon[34904]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-09T05:02:39.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:38 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2567475607' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-09T05:02:39.380 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:38 smithi072 ceph-mon[34088]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-09T05:02:39.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:38 smithi072 ceph-mon[34088]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-09T05:02:39.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:38 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/2567475607' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-09T05:02:39.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:38 smithi072 ceph-mon[34088]: osdmap e96: 12 total, 12 up, 12 in 2024-05-09T05:02:39.381 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:38 smithi072 ceph-mon[34088]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-05-09T05:02:40.516 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:40 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.72:6835/744758963,v1:172.21.15.72:6839/744758963] up:boot 2024-05-09T05:02:40.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:40 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.72:6834/762579896,v1:172.21.15.72:6837/762579896] up:boot 2024-05-09T05:02:40.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:40 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.72:6836/414896161,v1:172.21.15.72:6838/414896161] up:boot 2024-05-09T05:02:40.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:40 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.110:6834/4115787605,v1:172.21.15.110:6835/4115787605] up:boot 2024-05-09T05:02:40.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:40 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.110:6836/2747752966,v1:172.21.15.110:6838/2747752966] up:boot 2024-05-09T05:02:40.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:40 smithi046 ceph-mon[30136]: mds.? [v2:172.21.15.110:6837/579445256,v1:172.21.15.110:6839/579445256] up:boot 2024-05-09T05:02:40.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:40 smithi046 ceph-mon[30136]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-09T05:02:40.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:40 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T05:02:40.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:40 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T05:02:40.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:40 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T05:02:40.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:40 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T05:02:40.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:40 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T05:02:40.517 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:40 smithi046 ceph-mon[30136]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T05:02:40.518 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:40 smithi046 ceph-mon[30136]: pgmap v361: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 614 B/s wr, 7 op/s 2024-05-09T05:02:40.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:40 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.72:6835/744758963,v1:172.21.15.72:6839/744758963] up:boot 2024-05-09T05:02:40.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:40 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.72:6834/762579896,v1:172.21.15.72:6837/762579896] up:boot 2024-05-09T05:02:40.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:40 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.72:6836/414896161,v1:172.21.15.72:6838/414896161] up:boot 2024-05-09T05:02:40.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:40 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.110:6834/4115787605,v1:172.21.15.110:6835/4115787605] up:boot 2024-05-09T05:02:40.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:40 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.110:6836/2747752966,v1:172.21.15.110:6838/2747752966] up:boot 2024-05-09T05:02:40.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:40 smithi110 ceph-mon[34904]: mds.? [v2:172.21.15.110:6837/579445256,v1:172.21.15.110:6839/579445256] up:boot 2024-05-09T05:02:40.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:40 smithi110 ceph-mon[34904]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-09T05:02:40.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:40 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T05:02:40.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:40 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T05:02:40.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:40 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T05:02:40.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:40 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T05:02:40.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:40 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T05:02:40.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:40 smithi110 ceph-mon[34904]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T05:02:40.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:40 smithi110 ceph-mon[34904]: pgmap v361: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 614 B/s wr, 7 op/s 2024-05-09T05:02:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:40 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.72:6835/744758963,v1:172.21.15.72:6839/744758963] up:boot 2024-05-09T05:02:40.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:40 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.72:6834/762579896,v1:172.21.15.72:6837/762579896] up:boot 2024-05-09T05:02:40.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:40 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.72:6836/414896161,v1:172.21.15.72:6838/414896161] up:boot 2024-05-09T05:02:40.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:40 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.110:6834/4115787605,v1:172.21.15.110:6835/4115787605] up:boot 2024-05-09T05:02:40.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:40 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.110:6836/2747752966,v1:172.21.15.110:6838/2747752966] up:boot 2024-05-09T05:02:40.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:40 smithi072 ceph-mon[34088]: mds.? [v2:172.21.15.110:6837/579445256,v1:172.21.15.110:6839/579445256] up:boot 2024-05-09T05:02:40.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:40 smithi072 ceph-mon[34088]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-05-09T05:02:40.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:40 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-09T05:02:40.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:40 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-09T05:02:40.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:40 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-09T05:02:40.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:40 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-09T05:02:40.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:40 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-09T05:02:40.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:40 smithi072 ceph-mon[34088]: from='mgr.15150 172.21.15.46:0/2065444660' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-09T05:02:40.631 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:40 smithi072 ceph-mon[34088]: pgmap v361: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 614 B/s wr, 7 op/s 2024-05-09T05:02:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.782+0000 7f7cc88da700 1 -- 172.21.15.46:0/2824600892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc4107df0 msgr2=0x7f7cc4108200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.782+0000 7f7cc88da700 1 --2- 172.21.15.46:0/2824600892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc4107df0 0x7f7cc4108200 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80099c0 tx=0x7f7cb8009cd0 comp rx=0 tx=0).stop 2024-05-09T05:02:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.782+0000 7f7cc88da700 1 -- 172.21.15.46:0/2824600892 shutdown_connections 2024-05-09T05:02:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.782+0000 7f7cc88da700 1 --2- 172.21.15.46:0/2824600892 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7cc4108740 0x7f7cc41100a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.782+0000 7f7cc88da700 1 --2- 172.21.15.46:0/2824600892 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cc41034c0 0x7f7cc41038d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.782+0000 7f7cc88da700 1 --2- 172.21.15.46:0/2824600892 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc4107df0 0x7f7cc4108200 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.782+0000 7f7cc88da700 1 -- 172.21.15.46:0/2824600892 >> 172.21.15.46:0/2824600892 conn(0x7f7cc40fedf0 msgr2=0x7f7cc4101210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.782+0000 7f7cc88da700 1 -- 172.21.15.46:0/2824600892 shutdown_connections 2024-05-09T05:02:40.785 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.782+0000 7f7cc88da700 1 -- 172.21.15.46:0/2824600892 wait complete. 2024-05-09T05:02:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.783+0000 7f7cc88da700 1 Processor -- start 2024-05-09T05:02:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.783+0000 7f7cc88da700 1 -- start start 2024-05-09T05:02:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc88da700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc41034c0 0x7f7cc419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc88da700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cc4107df0 0x7f7cc419b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc88da700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7cc4108740 0x7f7cc419f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc88da700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc4113370 con 0x7f7cc41034c0 2024-05-09T05:02:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc88da700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41131f0 con 0x7f7cc4108740 2024-05-09T05:02:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc88da700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41134f0 con 0x7f7cc4107df0 2024-05-09T05:02:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc41034c0 0x7f7cc419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc259c700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc41034c0 0x7f7cc419add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:42186/0 (socket says 172.21.15.46:42186) 2024-05-09T05:02:40.786 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc259c700 1 -- 172.21.15.46:0/3059063061 learned_addr learned my addr 172.21.15.46:0/3059063061 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:02:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc2d9d700 1 --2- 172.21.15.46:0/3059063061 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7cc4108740 0x7f7cc419f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc1d9b700 1 --2- 172.21.15.46:0/3059063061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cc4107df0 0x7f7cc419b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc259c700 1 -- 172.21.15.46:0/3059063061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cc4107df0 msgr2=0x7f7cc419b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc259c700 1 --2- 172.21.15.46:0/3059063061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cc4107df0 0x7f7cc419b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:40.787 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc259c700 1 -- 172.21.15.46:0/3059063061 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7cc4108740 msgr2=0x7f7cc419f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc259c700 1 --2- 172.21.15.46:0/3059063061 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7cc4108740 0x7f7cc419f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.784+0000 7f7cc259c700 1 -- 172.21.15.46:0/3059063061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc419fca0 con 0x7f7cc41034c0 2024-05-09T05:02:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.785+0000 7f7cc259c700 1 --2- 172.21.15.46:0/3059063061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc41034c0 0x7f7cc419add0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400d980 tx=0x7f7cb400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.785+0000 7f7cb37fe700 1 -- 172.21.15.46:0/3059063061 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb4017070 con 0x7f7cc41034c0 2024-05-09T05:02:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.785+0000 7f7cc88da700 1 -- 172.21.15.46:0/3059063061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc419ff90 con 0x7f7cc41034c0 2024-05-09T05:02:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.785+0000 7f7cb37fe700 1 -- 172.21.15.46:0/3059063061 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb4005020 con 0x7f7cc41034c0 2024-05-09T05:02:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.785+0000 7f7cb37fe700 1 -- 172.21.15.46:0/3059063061 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb400fa10 con 0x7f7cc41034c0 2024-05-09T05:02:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.785+0000 7f7cc88da700 1 -- 172.21.15.46:0/3059063061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc41a78b0 con 0x7f7cc41034c0 2024-05-09T05:02:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.786+0000 7f7cc88da700 1 -- 172.21.15.46:0/3059063061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7cc4104970 con 0x7f7cc41034c0 2024-05-09T05:02:40.788 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.788+0000 7f7cb37fe700 1 -- 172.21.15.46:0/3059063061 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7cb4004a30 con 0x7f7cc41034c0 2024-05-09T05:02:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.788+0000 7f7cb37fe700 1 --2- 172.21.15.46:0/3059063061 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7cac0b29f0 0x7f7cac0b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.789+0000 7f7cc1d9b700 1 --2- 172.21.15.46:0/3059063061 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7cac0b29f0 0x7f7cac0b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:40.789 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.789+0000 7f7cb37fe700 1 -- 172.21.15.46:0/3059063061 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8814+0+0 (secure 0 0 0) 0x7f7cb40e2600 con 0x7f7cc41034c0 2024-05-09T05:02:40.790 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.789+0000 7f7cc1d9b700 1 --2- 172.21.15.46:0/3059063061 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7cac0b29f0 0x7f7cac0b4f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8006010 tx=0x7f7cb8000bc0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:40.794 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:40.793+0000 7f7cb37fe700 1 -- 172.21.15.46:0/3059063061 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7cb4020020 con 0x7f7cc41034c0 2024-05-09T05:02:41.054 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:41.053+0000 7f7cc88da700 1 -- 172.21.15.46:0/3059063061 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f7cc419c130 con 0x7f7cc41034c0 2024-05-09T05:02:41.179 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:41.177+0000 7f7cb37fe700 1 -- 172.21.15.46:0/3059063061 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) ==== 104+0+0 (secure 0 0 0) 0x7f7cb40a9ec0 con 0x7f7cc41034c0 2024-05-09T05:02:41.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:41.180+0000 7f7cc88da700 1 -- 172.21.15.46:0/3059063061 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7cac0b29f0 msgr2=0x7f7cac0b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:41.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:41.180+0000 7f7cc88da700 1 --2- 172.21.15.46:0/3059063061 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7cac0b29f0 0x7f7cac0b4f00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8006010 tx=0x7f7cb8000bc0 comp rx=0 tx=0).stop 2024-05-09T05:02:41.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:41.180+0000 7f7cc88da700 1 -- 172.21.15.46:0/3059063061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc41034c0 msgr2=0x7f7cc419add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:41.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:41.180+0000 7f7cc88da700 1 --2- 172.21.15.46:0/3059063061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc41034c0 0x7f7cc419add0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f7cb400d980 tx=0x7f7cb400dc90 comp rx=0 tx=0).stop 2024-05-09T05:02:41.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:41.180+0000 7f7cc88da700 1 -- 172.21.15.46:0/3059063061 shutdown_connections 2024-05-09T05:02:41.181 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:41.180+0000 7f7cc88da700 1 --2- 172.21.15.46:0/3059063061 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7f7cac0b29f0 0x7f7cac0b4f00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:41.180+0000 7f7cc88da700 1 --2- 172.21.15.46:0/3059063061 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7f7cc4108740 0x7f7cc419f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:41.180+0000 7f7cc88da700 1 --2- 172.21.15.46:0/3059063061 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7f7cc41034c0 0x7f7cc419add0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:41.180+0000 7f7cc88da700 1 --2- 172.21.15.46:0/3059063061 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7f7cc4107df0 0x7f7cc419b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:41.180+0000 7f7cc88da700 1 -- 172.21.15.46:0/3059063061 >> 172.21.15.46:0/3059063061 conn(0x7f7cc40fedf0 msgr2=0x7f7cc4075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:41.180+0000 7f7cc88da700 1 -- 172.21.15.46:0/3059063061 shutdown_connections 2024-05-09T05:02:41.182 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:41.180+0000 7f7cc88da700 1 -- 172.21.15.46:0/3059063061 wait complete. 2024-05-09T05:02:41.330 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:41 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3059063061' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T05:02:41.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:41 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3059063061' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T05:02:41.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:41 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3059063061' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-09T05:02:41.858 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-09T05:02:42.113 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:02:42.500 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:42 smithi046 ceph-mon[30136]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T05:02:42.500 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:42 smithi046 ceph-mon[30136]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T05:02:42.500 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:42 smithi046 ceph-mon[30136]: Cluster is now healthy 2024-05-09T05:02:42.500 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:42 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/3059063061' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T05:02:42.500 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:42 smithi046 ceph-mon[30136]: fsmap 12 up:standby 2024-05-09T05:02:42.501 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:42 smithi046 ceph-mon[30136]: pgmap v362: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-09T05:02:42.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:42 smithi110 ceph-mon[34904]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T05:02:42.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:42 smithi110 ceph-mon[34904]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T05:02:42.557 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:42 smithi110 ceph-mon[34904]: Cluster is now healthy 2024-05-09T05:02:42.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:42 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/3059063061' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T05:02:42.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:42 smithi110 ceph-mon[34904]: fsmap 12 up:standby 2024-05-09T05:02:42.558 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:42 smithi110 ceph-mon[34904]: pgmap v362: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-09T05:02:42.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:42 smithi072 ceph-mon[34088]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-09T05:02:42.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:42 smithi072 ceph-mon[34088]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-09T05:02:42.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:42 smithi072 ceph-mon[34088]: Cluster is now healthy 2024-05-09T05:02:42.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:42 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/3059063061' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-09T05:02:42.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:42 smithi072 ceph-mon[34088]: fsmap 12 up:standby 2024-05-09T05:02:42.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:42 smithi072 ceph-mon[34088]: pgmap v362: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 0 B/s wr, 4 op/s 2024-05-09T05:02:43.885 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.884+0000 7fc7147b0700 1 -- 172.21.15.46:0/2498869968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc70c107e10 msgr2=0x7fc70c108220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.884+0000 7fc7147b0700 1 --2- 172.21.15.46:0/2498869968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc70c107e10 0x7fc70c108220 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0099c0 tx=0x7fc6fc009cd0 comp rx=0 tx=0).stop 2024-05-09T05:02:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.884+0000 7fc7147b0700 1 -- 172.21.15.46:0/2498869968 shutdown_connections 2024-05-09T05:02:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.884+0000 7fc7147b0700 1 --2- 172.21.15.46:0/2498869968 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc70c108760 0x7fc70c1100c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.884+0000 7fc7147b0700 1 --2- 172.21.15.46:0/2498869968 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc70c1034e0 0x7fc70c1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.884+0000 7fc7147b0700 1 --2- 172.21.15.46:0/2498869968 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc70c107e10 0x7fc70c108220 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.884+0000 7fc7147b0700 1 -- 172.21.15.46:0/2498869968 >> 172.21.15.46:0/2498869968 conn(0x7fc70c0fef30 msgr2=0x7fc70c101350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.884+0000 7fc7147b0700 1 -- 172.21.15.46:0/2498869968 shutdown_connections 2024-05-09T05:02:43.886 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.884+0000 7fc7147b0700 1 -- 172.21.15.46:0/2498869968 wait complete. 2024-05-09T05:02:43.887 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.885+0000 7fc7147b0700 1 Processor -- start 2024-05-09T05:02:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.885+0000 7fc7147b0700 1 -- start start 2024-05-09T05:02:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.885+0000 7fc7147b0700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc70c1034e0 0x7fc70c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.885+0000 7fc7147b0700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc70c107e10 0x7fc70c19b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.885+0000 7fc7147b0700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc70c108760 0x7fc70c19f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.885+0000 7fc7147b0700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c113370 con 0x7fc70c107e10 2024-05-09T05:02:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.885+0000 7fc7147b0700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1131f0 con 0x7fc70c108760 2024-05-09T05:02:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.885+0000 7fc7147b0700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7fc70c1134f0 con 0x7fc70c1034e0 2024-05-09T05:02:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.885+0000 7fc711d4b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc70c107e10 0x7fc70c19b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.885+0000 7fc711d4b700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc70c107e10 0x7fc70c19b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47002/0 (socket says 172.21.15.46:47002) 2024-05-09T05:02:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.885+0000 7fc711d4b700 1 -- 172.21.15.46:0/4257330137 learned_addr learned my addr 172.21.15.46:0/4257330137 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:02:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.885+0000 7fc71254c700 1 --2- 172.21.15.46:0/4257330137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc70c1034e0 0x7fc70c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.886+0000 7fc712d4d700 1 --2- 172.21.15.46:0/4257330137 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc70c108760 0x7fc70c19f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.886+0000 7fc711d4b700 1 -- 172.21.15.46:0/4257330137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc70c1034e0 msgr2=0x7fc70c19ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:43.888 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.886+0000 7fc711d4b700 1 --2- 172.21.15.46:0/4257330137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc70c1034e0 0x7fc70c19ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.886+0000 7fc711d4b700 1 -- 172.21.15.46:0/4257330137 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc70c108760 msgr2=0x7fc70c19f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.886+0000 7fc711d4b700 1 --2- 172.21.15.46:0/4257330137 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc70c108760 0x7fc70c19f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.886+0000 7fc711d4b700 1 -- 172.21.15.46:0/4257330137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc70c19fd00 con 0x7fc70c107e10 2024-05-09T05:02:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.886+0000 7fc712d4d700 1 --2- 172.21.15.46:0/4257330137 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc70c108760 0x7fc70c19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-09T05:02:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.886+0000 7fc711d4b700 1 --2- 172.21.15.46:0/4257330137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc70c107e10 0x7fc70c19b3d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0098e0 tx=0x7fc6fc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.886+0000 7fc7037fe700 1 -- 172.21.15.46:0/4257330137 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc027070 con 0x7fc70c107e10 2024-05-09T05:02:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.886+0000 7fc7147b0700 1 -- 172.21.15.46:0/4257330137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc70c19ff90 con 0x7fc70c107e10 2024-05-09T05:02:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.886+0000 7fc7037fe700 1 -- 172.21.15.46:0/4257330137 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6fc003f20 con 0x7fc70c107e10 2024-05-09T05:02:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.886+0000 7fc7037fe700 1 -- 172.21.15.46:0/4257330137 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6fc022d70 con 0x7fc70c107e10 2024-05-09T05:02:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.886+0000 7fc7147b0700 1 -- 172.21.15.46:0/4257330137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc70c1a0470 con 0x7fc70c107e10 2024-05-09T05:02:43.889 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.887+0000 7fc7147b0700 1 -- 172.21.15.46:0/4257330137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc70c0512c0 con 0x7fc70c107e10 2024-05-09T05:02:43.890 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.889+0000 7fc7037fe700 1 -- 172.21.15.46:0/4257330137 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc6fc02a070 con 0x7fc70c107e10 2024-05-09T05:02:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.890+0000 7fc7037fe700 1 --2- 172.21.15.46:0/4257330137 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc6f80b29f0 0x7fc6f80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.890+0000 7fc71254c700 1 --2- 172.21.15.46:0/4257330137 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc6f80b29f0 0x7fc6f80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.890+0000 7fc7037fe700 1 -- 172.21.15.46:0/4257330137 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 8814+0+0 (secure 0 0 0) 0x7fc6fc0e3e20 con 0x7fc70c107e10 2024-05-09T05:02:43.891 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.890+0000 7fc71254c700 1 --2- 172.21.15.46:0/4257330137 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc6f80b29f0 0x7fc6f80b4f00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc708007be0 tx=0x7fc7080073d0 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:43.894 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:43.893+0000 7fc7037fe700 1 -- 172.21.15.46:0/4257330137 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc6fc031080 con 0x7fc70c107e10 2024-05-09T05:02:44.109 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:44.108+0000 7fc7147b0700 1 -- 172.21.15.46:0/4257330137 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fc70c02e5f0 con 0x7fc70c107e10 2024-05-09T05:02:44.849 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:44.848+0000 7fc7037fe700 1 -- 172.21.15.46:0/4257330137 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) ==== 181+0+0 (secure 0 0 0) 0x7fc6fc013070 con 0x7fc70c107e10 2024-05-09T05:02:44.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:44.852+0000 7fc7147b0700 1 -- 172.21.15.46:0/4257330137 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc6f80b29f0 msgr2=0x7fc6f80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:44.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:44.852+0000 7fc7147b0700 1 --2- 172.21.15.46:0/4257330137 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc6f80b29f0 0x7fc6f80b4f00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc708007be0 tx=0x7fc7080073d0 comp rx=0 tx=0).stop 2024-05-09T05:02:44.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:44.852+0000 7fc7147b0700 1 -- 172.21.15.46:0/4257330137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc70c107e10 msgr2=0x7fc70c19b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:44.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:44.852+0000 7fc7147b0700 1 --2- 172.21.15.46:0/4257330137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc70c107e10 0x7fc70c19b3d0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0098e0 tx=0x7fc6fc0057c0 comp rx=0 tx=0).stop 2024-05-09T05:02:44.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:44.853+0000 7fc7147b0700 1 -- 172.21.15.46:0/4257330137 shutdown_connections 2024-05-09T05:02:44.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:44.853+0000 7fc7147b0700 1 --2- 172.21.15.46:0/4257330137 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7fc6f80b29f0 0x7fc6f80b4f00 secure :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc708007be0 tx=0x7fc7080073d0 comp rx=0 tx=0).stop 2024-05-09T05:02:44.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:44.853+0000 7fc7147b0700 1 --2- 172.21.15.46:0/4257330137 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7fc70c108760 0x7fc70c19f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:44.854 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:44.853+0000 7fc7147b0700 1 --2- 172.21.15.46:0/4257330137 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7fc70c1034e0 0x7fc70c19ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:44.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:44.853+0000 7fc7147b0700 1 --2- 172.21.15.46:0/4257330137 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7fc70c107e10 0x7fc70c19b3d0 secure :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fc6fc0098e0 tx=0x7fc6fc0057c0 comp rx=0 tx=0).stop 2024-05-09T05:02:44.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:44.853+0000 7fc7147b0700 1 -- 172.21.15.46:0/4257330137 >> 172.21.15.46:0/4257330137 conn(0x7fc70c0fef30 msgr2=0x7fc70c06c380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:44.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:44.854+0000 7fc7147b0700 1 -- 172.21.15.46:0/4257330137 shutdown_connections 2024-05-09T05:02:44.855 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:44.854+0000 7fc7147b0700 1 -- 172.21.15.46:0/4257330137 wait complete. 2024-05-09T05:02:44.856 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_metadata' removed 2024-05-09T05:02:45.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:44 smithi072 ceph-mon[34088]: pgmap v363: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-09T05:02:45.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:44 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/4257330137' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T05:02:45.173 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:44 smithi046 ceph-mon[30136]: pgmap v363: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-09T05:02:45.173 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:44 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/4257330137' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T05:02:45.260 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:44 smithi110 ceph-mon[34904]: pgmap v363: 97 pgs: 97 active+clean; 3.9 MiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2024-05-09T05:02:45.260 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:44 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/4257330137' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T05:02:45.504 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554 shell --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-09T05:02:45.762 INFO:teuthology.orchestra.run.smithi046.stderr:Inferring config /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/mon.a/config 2024-05-09T05:02:46.130 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:45 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/4257330137' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T05:02:46.130 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:45 smithi046 ceph-mon[30136]: osdmap e97: 12 total, 12 up, 12 in 2024-05-09T05:02:46.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:45 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/4257330137' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T05:02:46.131 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:45 smithi072 ceph-mon[34088]: osdmap e97: 12 total, 12 up, 12 in 2024-05-09T05:02:46.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:45 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/4257330137' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T05:02:46.308 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:45 smithi110 ceph-mon[34904]: osdmap e97: 12 total, 12 up, 12 in 2024-05-09T05:02:46.935 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:46 smithi046 ceph-mon[30136]: pgmap v365: 65 pgs: 65 active+clean; 577 KiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:02:47.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:46 smithi072 ceph-mon[34088]: pgmap v365: 65 pgs: 65 active+clean; 577 KiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:02:47.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:46 smithi110 ceph-mon[34904]: pgmap v365: 65 pgs: 65 active+clean; 577 KiB data, 548 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:02:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.520+0000 7ff8d2769700 1 -- 172.21.15.46:0/898703458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8cc108760 msgr2=0x7ff8cc1100c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.520+0000 7ff8d2769700 1 --2- 172.21.15.46:0/898703458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8cc108760 0x7ff8cc1100c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7ff8c400b0d0 tx=0x7ff8c400b3e0 comp rx=0 tx=0).stop 2024-05-09T05:02:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.520+0000 7ff8d2769700 1 -- 172.21.15.46:0/898703458 shutdown_connections 2024-05-09T05:02:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.520+0000 7ff8d2769700 1 --2- 172.21.15.46:0/898703458 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8cc108760 0x7ff8cc1100c0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.520+0000 7ff8d2769700 1 --2- 172.21.15.46:0/898703458 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff8cc1034e0 0x7ff8cc1038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.520+0000 7ff8d2769700 1 --2- 172.21.15.46:0/898703458 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8cc107e10 0x7ff8cc108220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.520+0000 7ff8d2769700 1 -- 172.21.15.46:0/898703458 >> 172.21.15.46:0/898703458 conn(0x7ff8cc0fef50 msgr2=0x7ff8cc101370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.520+0000 7ff8d2769700 1 -- 172.21.15.46:0/898703458 shutdown_connections 2024-05-09T05:02:47.526 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.521+0000 7ff8d2769700 1 -- 172.21.15.46:0/898703458 wait complete. 2024-05-09T05:02:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.522+0000 7ff8d2769700 1 Processor -- start 2024-05-09T05:02:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.522+0000 7ff8d2769700 1 -- start start 2024-05-09T05:02:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.522+0000 7ff8d2769700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8cc1034e0 0x7ff8cc19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:47.529 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.522+0000 7ff8d2769700 1 --2- >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff8cc107e10 0x7ff8cc19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.522+0000 7ff8d2769700 1 --2- >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8cc108760 0x7ff8cc19f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.522+0000 7ff8d2769700 1 -- --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc113390 con 0x7ff8cc1034e0 2024-05-09T05:02:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.522+0000 7ff8d2769700 1 -- --> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc113210 con 0x7ff8cc107e10 2024-05-09T05:02:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.522+0000 7ff8d2769700 1 -- --> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] -- mon_getmap magic: 0 -- 0x7ff8cc113510 con 0x7ff8cc108760 2024-05-09T05:02:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.522+0000 7ff8cbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8cc1034e0 0x7ff8cc19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.522+0000 7ff8cbfff700 1 --2- >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8cc1034e0 0x7ff8cc19adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.46:3300/0 says I am v2:172.21.15.46:47030/0 (socket says 172.21.15.46:47030) 2024-05-09T05:02:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.522+0000 7ff8cbfff700 1 -- 172.21.15.46:0/4237246676 learned_addr learned my addr 172.21.15.46:0/4237246676 (peer_addr_for_me v2:172.21.15.46:0/0) 2024-05-09T05:02:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.522+0000 7ff8d0d06700 1 --2- 172.21.15.46:0/4237246676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8cc108760 0x7ff8cc19f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:47.530 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.522+0000 7ff8c3fff700 1 --2- 172.21.15.46:0/4237246676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff8cc107e10 0x7ff8cc19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.522+0000 7ff8cbfff700 1 -- 172.21.15.46:0/4237246676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8cc108760 msgr2=0x7ff8cc19f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.522+0000 7ff8cbfff700 1 --2- 172.21.15.46:0/4237246676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8cc108760 0x7ff8cc19f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.523+0000 7ff8cbfff700 1 -- 172.21.15.46:0/4237246676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff8cc107e10 msgr2=0x7ff8cc19b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.523+0000 7ff8cbfff700 1 --2- 172.21.15.46:0/4237246676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff8cc107e10 0x7ff8cc19b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.523+0000 7ff8cbfff700 1 -- 172.21.15.46:0/4237246676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8cc19fc80 con 0x7ff8cc1034e0 2024-05-09T05:02:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.523+0000 7ff8c3fff700 1 --2- 172.21.15.46:0/4237246676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff8cc107e10 0x7ff8cc19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-09T05:02:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.523+0000 7ff8cbfff700 1 --2- 172.21.15.46:0/4237246676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8cc1034e0 0x7ff8cc19adb0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00d950 tx=0x7ff8bc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.523+0000 7ff8c9ffb700 1 -- 172.21.15.46:0/4237246676 <== mon.0 v2:172.21.15.46:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc016070 con 0x7ff8cc1034e0 2024-05-09T05:02:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.523+0000 7ff8d2769700 1 -- 172.21.15.46:0/4237246676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8cc19ff70 con 0x7ff8cc1034e0 2024-05-09T05:02:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.523+0000 7ff8c9ffb700 1 -- 172.21.15.46:0/4237246676 <== mon.0 v2:172.21.15.46:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8bc009df0 con 0x7ff8cc1034e0 2024-05-09T05:02:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.523+0000 7ff8c9ffb700 1 -- 172.21.15.46:0/4237246676 <== mon.0 v2:172.21.15.46:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc00f3c0 con 0x7ff8cc1034e0 2024-05-09T05:02:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.523+0000 7ff8d2769700 1 -- 172.21.15.46:0/4237246676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8cc1a7890 con 0x7ff8cc1034e0 2024-05-09T05:02:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.524+0000 7ff8d2769700 1 -- 172.21.15.46:0/4237246676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8cc104990 con 0x7ff8cc1034e0 2024-05-09T05:02:47.531 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.527+0000 7ff8c9ffb700 1 -- 172.21.15.46:0/4237246676 <== mon.0 v2:172.21.15.46:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff8bc019070 con 0x7ff8cc1034e0 2024-05-09T05:02:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.527+0000 7ff8c9ffb700 1 --2- 172.21.15.46:0/4237246676 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff8b80b29f0 0x7ff8b80b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-09T05:02:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.528+0000 7ff8c3fff700 1 --2- 172.21.15.46:0/4237246676 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff8b80b29f0 0x7ff8b80b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-09T05:02:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.528+0000 7ff8c9ffb700 1 -- 172.21.15.46:0/4237246676 <== mon.0 v2:172.21.15.46:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 8260+0+0 (secure 0 0 0) 0x7ff8bc0e2130 con 0x7ff8cc1034e0 2024-05-09T05:02:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.528+0000 7ff8c3fff700 1 --2- 172.21.15.46:0/4237246676 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff8b80b29f0 0x7ff8b80b4f00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4005e10 tx=0x7ff8b4005d60 comp rx=0 tx=0).ready entity=mgr.15150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-09T05:02:47.532 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.531+0000 7ff8c9ffb700 1 -- 172.21.15.46:0/4237246676 <== mon.0 v2:172.21.15.46:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff8bc0a9c10 con 0x7ff8cc1034e0 2024-05-09T05:02:47.745 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.743+0000 7ff8d2769700 1 -- 172.21.15.46:0/4237246676 --> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff8cc02e5f0 con 0x7ff8cc1034e0 2024-05-09T05:02:47.863 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.861+0000 7ff8c9ffb700 1 -- 172.21.15.46:0/4237246676 <== mon.0 v2:172.21.15.46:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) ==== 169+0+0 (secure 0 0 0) 0x7ff8bc0a9510 con 0x7ff8cc1034e0 2024-05-09T05:02:47.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.864+0000 7ff8d2769700 1 -- 172.21.15.46:0/4237246676 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff8b80b29f0 msgr2=0x7ff8b80b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:47.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.864+0000 7ff8d2769700 1 --2- 172.21.15.46:0/4237246676 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff8b80b29f0 0x7ff8b80b4f00 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4005e10 tx=0x7ff8b4005d60 comp rx=0 tx=0).stop 2024-05-09T05:02:47.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.865+0000 7ff8d2769700 1 -- 172.21.15.46:0/4237246676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8cc1034e0 msgr2=0x7ff8cc19adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-09T05:02:47.866 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.865+0000 7ff8d2769700 1 --2- 172.21.15.46:0/4237246676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8cc1034e0 0x7ff8cc19adb0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00d950 tx=0x7ff8bc00dc60 comp rx=0 tx=0).stop 2024-05-09T05:02:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.866+0000 7ff8d2769700 1 -- 172.21.15.46:0/4237246676 shutdown_connections 2024-05-09T05:02:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.866+0000 7ff8d2769700 1 --2- 172.21.15.46:0/4237246676 >> [v2:172.21.15.46:6800/2595796783,v1:172.21.15.46:6801/2595796783] conn(0x7ff8b80b29f0 0x7ff8b80b4f00 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.866+0000 7ff8d2769700 1 --2- 172.21.15.46:0/4237246676 >> [v2:172.21.15.110:3300/0,v1:172.21.15.110:6789/0] conn(0x7ff8cc108760 0x7ff8cc19f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.866+0000 7ff8d2769700 1 --2- 172.21.15.46:0/4237246676 >> [v2:172.21.15.46:3300/0,v1:172.21.15.46:6789/0] conn(0x7ff8cc1034e0 0x7ff8cc19adb0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.866+0000 7ff8d2769700 1 --2- 172.21.15.46:0/4237246676 >> [v2:172.21.15.72:3300/0,v1:172.21.15.72:6789/0] conn(0x7ff8cc107e10 0x7ff8cc19b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-09T05:02:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.866+0000 7ff8d2769700 1 -- 172.21.15.46:0/4237246676 >> 172.21.15.46:0/4237246676 conn(0x7ff8cc0fef50 msgr2=0x7ff8cc075cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-09T05:02:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.867+0000 7ff8d2769700 1 -- 172.21.15.46:0/4237246676 shutdown_connections 2024-05-09T05:02:47.868 INFO:teuthology.orchestra.run.smithi046.stderr:2024-05-09T05:02:47.867+0000 7ff8d2769700 1 -- 172.21.15.46:0/4237246676 wait complete. 2024-05-09T05:02:47.869 INFO:teuthology.orchestra.run.smithi046.stderr:pool 'cephfs_data' removed 2024-05-09T05:02:48.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:47 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/4237246676' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T05:02:48.229 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:47 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/4237246676' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T05:02:48.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:47 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/4237246676' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-09T05:02:48.576 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T05:02:48.607 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T05:02:48.639 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T05:02:48.674 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-09T05:02:48.675 DEBUG:teuthology.orchestra.run.smithi046:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T05:02:48.692 DEBUG:teuthology.orchestra.run.smithi072:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T05:02:48.710 DEBUG:teuthology.orchestra.run.smithi110:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-09T05:02:48.731 INFO:tasks.cephadm:Stopping all daemons... 2024-05-09T05:02:48.732 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-09T05:02:48.732 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mon.a 2024-05-09T05:02:48.830 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:48 smithi046 systemd[1]: Stopping Ceph mon.a for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:02:49.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:48 smithi110 ceph-mon[34904]: pgmap v366: 65 pgs: 65 active+clean; 577 KiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:02:49.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:48 smithi110 ceph-mon[34904]: from='client.? 172.21.15.46:0/4237246676' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T05:02:49.057 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:48 smithi110 ceph-mon[34904]: osdmap e98: 12 total, 12 up, 12 in 2024-05-09T05:02:49.117 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:48 smithi046 ceph-mon[30136]: pgmap v366: 65 pgs: 65 active+clean; 577 KiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:02:49.117 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:48 smithi046 ceph-mon[30136]: from='client.? 172.21.15.46:0/4237246676' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T05:02:49.117 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:48 smithi046 ceph-mon[30136]: osdmap e98: 12 total, 12 up, 12 in 2024-05-09T05:02:49.117 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:49 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a[30132]: 2024-05-09T05:02:49.024+0000 7f7216861700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:02:49.117 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:49 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a[30132]: 2024-05-09T05:02:49.024+0000 7f7216861700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-09T05:02:49.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:48 smithi072 ceph-mon[34088]: pgmap v366: 65 pgs: 65 active+clean; 577 KiB data, 540 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-09T05:02:49.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:48 smithi072 ceph-mon[34088]: from='client.? 172.21.15.46:0/4237246676' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-09T05:02:49.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:48 smithi072 ceph-mon[34088]: osdmap e98: 12 total, 12 up, 12 in 2024-05-09T05:02:49.418 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:49 smithi046 podman[117593]: 2024-05-09 05:02:49.117373155 +0000 UTC m=+0.190838932 container died 0809ad9b5ec45012eb937b11f1b3003ca18bd02884a956db0d2e187377516c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507) 2024-05-09T05:02:49.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:49 smithi046 podman[117593]: 2024-05-09 05:02:49.508921622 +0000 UTC m=+0.582387387 container cleanup 0809ad9b5ec45012eb937b11f1b3003ca18bd02884a956db0d2e187377516c3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:02:49.777 INFO:journalctl@ceph.mon.a.smithi046.stdout:May 09 05:02:49 smithi046 bash[117593]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-a 2024-05-09T05:02:49.934 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mon.a.service' 2024-05-09T05:02:49.978 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:02:49.979 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-09T05:02:49.979 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-09T05:02:49.979 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mon.b 2024-05-09T05:02:50.377 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:50 smithi072 systemd[1]: Stopping Ceph mon.b for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:02:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:50 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-b[34084]: 2024-05-09T05:02:50.376+0000 7f0cb454f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:02:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:50 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-b[34084]: 2024-05-09T05:02:50.376+0000 7f0cb454f700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-09T05:02:50.630 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:50 smithi072 podman[70123]: 2024-05-09 05:02:50.500287655 +0000 UTC m=+0.321357967 container died bb655ac8ed64d62855b455f6a135794e92384a629f6c92e4627f00afd8526b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-b, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:02:51.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:50 smithi072 podman[70123]: 2024-05-09 05:02:50.897891483 +0000 UTC m=+0.718961795 container cleanup bb655ac8ed64d62855b455f6a135794e92384a629f6c92e4627f00afd8526b4d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:02:51.130 INFO:journalctl@ceph.mon.b.smithi072.stdout:May 09 05:02:50 smithi072 bash[70123]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-b 2024-05-09T05:02:51.267 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mon.b.service' 2024-05-09T05:02:51.308 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:02:51.308 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-09T05:02:51.308 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-09T05:02:51.308 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mon.c 2024-05-09T05:02:51.622 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:51 smithi110 systemd[1]: Stopping Ceph mon.c for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:02:51.973 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:51 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-c[34900]: 2024-05-09T05:02:51.621+0000 7f02c6c5f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:02:51.973 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:51 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-c[34900]: 2024-05-09T05:02:51.621+0000 7f02c6c5f700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-09T05:02:51.973 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:51 smithi110 podman[71835]: 2024-05-09 05:02:51.662818352 +0000 UTC m=+0.163536326 container died f5239ba2c3bb771e0f04a1e1b500fac11908b55343b85b904b0afbf21250f737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-c, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-09T05:02:52.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:52 smithi110 podman[71835]: 2024-05-09 05:02:52.081070561 +0000 UTC m=+0.581788541 container cleanup f5239ba2c3bb771e0f04a1e1b500fac11908b55343b85b904b0afbf21250f737 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-05-09T05:02:52.307 INFO:journalctl@ceph.mon.c.smithi110.stdout:May 09 05:02:52 smithi110 bash[71835]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mon-c 2024-05-09T05:02:52.547 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mon.c.service' 2024-05-09T05:02:52.593 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:02:52.593 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-09T05:02:52.593 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-09T05:02:52.593 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.x 2024-05-09T05:02:52.922 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 05:02:52 smithi046 systemd[1]: Stopping Ceph mgr.x for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:02:53.224 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 05:02:52 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T05:02:52.921+0000 7f88ffdde700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:02:53.224 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 05:02:52 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x[30367]: 2024-05-09T05:02:52.921+0000 7f88ffdde700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-05-09T05:02:53.224 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 05:02:52 smithi046 podman[117714]: 2024-05-09 05:02:52.964271714 +0000 UTC m=+0.165800378 container died a3cb336f79308678ad9c1bbd97f083feea349e6b49314c83601697bea93399a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-05-09T05:02:53.581 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 05:02:53 smithi046 podman[117714]: 2024-05-09 05:02:53.325177378 +0000 UTC m=+0.526706050 container cleanup a3cb336f79308678ad9c1bbd97f083feea349e6b49314c83601697bea93399a3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:c9da4049f638c0f84bdd9b4e33c42e86046ac554, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-05-09T05:02:53.581 INFO:journalctl@ceph.mgr.x.smithi046.stdout:May 09 05:02:53 smithi046 bash[117714]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-x 2024-05-09T05:02:53.749 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-09T05:02:53.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:02:53.787 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-09T05:02:53.787 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-09T05:02:53.787 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.y 2024-05-09T05:02:54.130 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 05:02:53 smithi072 systemd[1]: Stopping Ceph mgr.y for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:02:54.380 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 05:02:54 smithi072 podman[70244]: 2024-05-09 05:02:54.130406891 +0000 UTC m=+0.155389596 container died feb402c9a46772cd2eb68eff3aa711638840d0a71042bbee1cbd70e4d1913eb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-09T05:02:54.766 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 05:02:54 smithi072 podman[70244]: 2024-05-09 05:02:54.537785424 +0000 UTC m=+0.562768142 container cleanup feb402c9a46772cd2eb68eff3aa711638840d0a71042bbee1cbd70e4d1913eb8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-09T05:02:54.766 INFO:journalctl@ceph.mgr.y.smithi072.stdout:May 09 05:02:54 smithi072 bash[70244]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-y 2024-05-09T05:02:54.906 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-09T05:02:54.939 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:02:54.939 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-09T05:02:54.939 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-09T05:02:54.939 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.z 2024-05-09T05:02:55.282 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 05:02:54 smithi110 systemd[1]: Stopping Ceph mgr.z for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:02:55.557 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 05:02:55 smithi110 podman[71957]: 2024-05-09 05:02:55.28331574 +0000 UTC m=+0.153790730 container died e9f541a521cfda53d9d64308914d0e55dcccff919a5523fcba58072b63ac962d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:02:55.946 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 05:02:55 smithi110 podman[71957]: 2024-05-09 05:02:55.712052829 +0000 UTC m=+0.582527811 container cleanup e9f541a521cfda53d9d64308914d0e55dcccff919a5523fcba58072b63ac962d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-05-09T05:02:55.947 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 05:02:55 smithi110 bash[71957]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z 2024-05-09T05:02:56.184 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-09T05:02:56.204 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 05:02:55 smithi110 podman[71971]: 2024-05-09 05:02:55.947086482 +0000 UTC m=+0.667065347 container remove e9f541a521cfda53d9d64308914d0e55dcccff919a5523fcba58072b63ac962d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mgr-z, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-09T05:02:56.204 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 05:02:55 smithi110 systemd[1]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.z.service: Main process exited, code=exited, status=143/n/a 2024-05-09T05:02:56.205 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 05:02:56 smithi110 systemd[1]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.z.service: Failed with result 'exit-code'. 2024-05-09T05:02:56.205 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 05:02:56 smithi110 systemd[1]: Stopped Ceph mgr.z for ecf9ae2c-0dbd-11ef-bc97-c7b262605968. 2024-05-09T05:02:56.205 INFO:journalctl@ceph.mgr.z.smithi110.stdout:May 09 05:02:56 smithi110 systemd[1]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mgr.z.service: Consumed 21.409s CPU time. 2024-05-09T05:02:56.218 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:02:56.218 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-09T05:02:56.218 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-09T05:02:56.219 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.0 2024-05-09T05:02:56.530 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 05:02:56 smithi046 systemd[1]: Stopping Ceph osd.0 for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:02:56.830 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 05:02:56 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-0[45186]: 2024-05-09T05:02:56.528+0000 7f7625537700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:02:56.943 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 05:02:56 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-0[45186]: 2024-05-09T05:02:56.528+0000 7f7625537700 -1 osd.0 98 *** Got signal Terminated *** 2024-05-09T05:02:56.943 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 05:02:56 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-0[45186]: 2024-05-09T05:02:56.528+0000 7f7625537700 -1 osd.0 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:03:02.224 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 05:03:01 smithi046 podman[117836]: 2024-05-09 05:03:01.92213733 +0000 UTC m=+5.506665175 container died de012b99267cf13e843439dd73f64ea5a05f63eee4d07e5badad5bf4869b239a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:03:02.580 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 05:03:02 smithi046 podman[117836]: 2024-05-09 05:03:02.415241377 +0000 UTC m=+5.999769221 container cleanup de012b99267cf13e843439dd73f64ea5a05f63eee4d07e5badad5bf4869b239a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:03:02.580 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 05:03:02 smithi046 bash[117836]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-0 2024-05-09T05:03:03.048 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 05:03:02 smithi046 podman[117853]: 2024-05-09 05:03:02.717054913 +0000 UTC m=+0.783997351 container remove de012b99267cf13e843439dd73f64ea5a05f63eee4d07e5badad5bf4869b239a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True) 2024-05-09T05:03:03.330 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 05:03:03 smithi046 podman[117940]: 2024-05-09 05:03:02.951415819 +0000 UTC m=+0.018728782 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:03:03.330 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 05:03:03 smithi046 podman[117940]: 2024-05-09 05:03:03.076586345 +0000 UTC m=+0.143899340 container create a98cbf2119183306a94afdaa8623b26da43b13ca6855121e0125907500bdb7f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-09T05:03:03.693 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 05:03:03 smithi046 podman[117940]: 2024-05-09 05:03:03.386288919 +0000 UTC m=+0.453601875 container init a98cbf2119183306a94afdaa8623b26da43b13ca6855121e0125907500bdb7f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True) 2024-05-09T05:03:03.693 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 05:03:03 smithi046 podman[117940]: 2024-05-09 05:03:03.390146858 +0000 UTC m=+0.457459812 container start a98cbf2119183306a94afdaa8623b26da43b13ca6855121e0125907500bdb7f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-05-09T05:03:03.693 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 05:03:03 smithi046 podman[117940]: 2024-05-09 05:03:03.490391307 +0000 UTC m=+0.557704266 container attach a98cbf2119183306a94afdaa8623b26da43b13ca6855121e0125907500bdb7f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-09T05:03:04.052 INFO:journalctl@ceph.osd.0.smithi046.stdout:May 09 05:03:03 smithi046 podman[117940]: 2024-05-09 05:03:03.693368248 +0000 UTC m=+0.760681201 container died a98cbf2119183306a94afdaa8623b26da43b13ca6855121e0125907500bdb7f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-09T05:03:04.464 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.0.service' 2024-05-09T05:03:04.513 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:03:04.514 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-09T05:03:04.514 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-09T05:03:04.514 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.1 2024-05-09T05:03:04.831 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 05:03:04 smithi072 systemd[1]: Stopping Ceph osd.1 for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:03:05.130 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 05:03:04 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-1[38706]: 2024-05-09T05:03:04.830+0000 7f26fc6c0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:03:05.131 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 05:03:04 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-1[38706]: 2024-05-09T05:03:04.830+0000 7f26fc6c0700 -1 osd.1 98 *** Got signal Terminated *** 2024-05-09T05:03:05.131 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 05:03:04 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-1[38706]: 2024-05-09T05:03:04.830+0000 7f26fc6c0700 -1 osd.1 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:03:10.517 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 05:03:10 smithi072 podman[70369]: 2024-05-09 05:03:10.234523078 +0000 UTC m=+5.534796811 container died 1e8bf7d9484a0b1ad63009e673dab1ae652c644040cc05c795202a6cac3acfa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=) 2024-05-09T05:03:10.863 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 05:03:10 smithi072 podman[70369]: 2024-05-09 05:03:10.617928952 +0000 UTC m=+5.918202663 container cleanup 1e8bf7d9484a0b1ad63009e673dab1ae652c644040cc05c795202a6cac3acfa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-09T05:03:10.863 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 05:03:10 smithi072 bash[70369]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-1 2024-05-09T05:03:11.130 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 05:03:10 smithi072 podman[70387]: 2024-05-09 05:03:10.863936224 +0000 UTC m=+0.631745574 container remove 1e8bf7d9484a0b1ad63009e673dab1ae652c644040cc05c795202a6cac3acfa6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-1, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-09T05:03:11.473 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 05:03:11 smithi072 podman[70479]: 2024-05-09 05:03:11.096850822 +0000 UTC m=+0.018051389 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:03:11.474 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 05:03:11 smithi072 podman[70479]: 2024-05-09 05:03:11.2158507 +0000 UTC m=+0.137051293 container create 1e57ec5f6152c14a13af284eb8d52e9df9e3ddf593c07e45de8b70bbc00ef3d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-1-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:03:11.772 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 05:03:11 smithi072 podman[70479]: 2024-05-09 05:03:11.474154307 +0000 UTC m=+0.395354870 container init 1e57ec5f6152c14a13af284eb8d52e9df9e3ddf593c07e45de8b70bbc00ef3d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:03:11.772 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 05:03:11 smithi072 podman[70479]: 2024-05-09 05:03:11.477964374 +0000 UTC m=+0.399164933 container start 1e57ec5f6152c14a13af284eb8d52e9df9e3ddf593c07e45de8b70bbc00ef3d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True) 2024-05-09T05:03:11.772 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 05:03:11 smithi072 podman[70479]: 2024-05-09 05:03:11.591311619 +0000 UTC m=+0.512512176 container attach 1e57ec5f6152c14a13af284eb8d52e9df9e3ddf593c07e45de8b70bbc00ef3d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T05:03:12.129 INFO:journalctl@ceph.osd.1.smithi072.stdout:May 09 05:03:11 smithi072 podman[70479]: 2024-05-09 05:03:11.772773694 +0000 UTC m=+0.693974248 container died 1e57ec5f6152c14a13af284eb8d52e9df9e3ddf593c07e45de8b70bbc00ef3d7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True) 2024-05-09T05:03:12.481 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.1.service' 2024-05-09T05:03:12.521 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:03:12.521 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-09T05:03:12.521 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-09T05:03:12.521 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.2 2024-05-09T05:03:12.807 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 05:03:12 smithi110 systemd[1]: Stopping Ceph osd.2 for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:03:13.307 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 05:03:12 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-2[40098]: 2024-05-09T05:03:12.942+0000 7f9d76af2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:03:13.307 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 05:03:12 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-2[40098]: 2024-05-09T05:03:12.942+0000 7f9d76af2700 -1 osd.2 98 *** Got signal Terminated *** 2024-05-09T05:03:13.307 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 05:03:12 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-2[40098]: 2024-05-09T05:03:12.942+0000 7f9d76af2700 -1 osd.2 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:03:18.768 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 05:03:18 smithi110 podman[72083]: 2024-05-09 05:03:18.467609106 +0000 UTC m=+5.755052603 container died 34b2bea055dee2f58eac6545a11435c709e1afbfc4b2f500234362902dde95a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:03:19.057 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 05:03:18 smithi110 podman[72083]: 2024-05-09 05:03:18.876179481 +0000 UTC m=+6.163622988 container cleanup 34b2bea055dee2f58eac6545a11435c709e1afbfc4b2f500234362902dde95a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True) 2024-05-09T05:03:19.058 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 05:03:18 smithi110 bash[72083]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-2 2024-05-09T05:03:19.554 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 05:03:19 smithi110 podman[72100]: 2024-05-09 05:03:19.12791631 +0000 UTC m=+0.663499320 container remove 34b2bea055dee2f58eac6545a11435c709e1afbfc4b2f500234362902dde95a2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:03:19.807 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 05:03:19 smithi110 podman[72190]: 2024-05-09 05:03:19.457185802 +0000 UTC m=+0.017919540 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:03:19.807 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 05:03:19 smithi110 podman[72190]: 2024-05-09 05:03:19.578889251 +0000 UTC m=+0.139623052 container create afdf08d3c1576d499ee696e5105c385eb8c5f2718bbc0f97946250f77358d7f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:03:20.157 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 05:03:19 smithi110 podman[72190]: 2024-05-09 05:03:19.864867328 +0000 UTC m=+0.425601068 container init afdf08d3c1576d499ee696e5105c385eb8c5f2718bbc0f97946250f77358d7f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5) 2024-05-09T05:03:20.157 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 05:03:19 smithi110 podman[72190]: 2024-05-09 05:03:19.867960871 +0000 UTC m=+0.428694622 container start afdf08d3c1576d499ee696e5105c385eb8c5f2718bbc0f97946250f77358d7f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-2-deactivate, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0) 2024-05-09T05:03:20.157 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 05:03:19 smithi110 podman[72190]: 2024-05-09 05:03:19.965423474 +0000 UTC m=+0.526157238 container attach afdf08d3c1576d499ee696e5105c385eb8c5f2718bbc0f97946250f77358d7f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-09T05:03:20.440 INFO:journalctl@ceph.osd.2.smithi110.stdout:May 09 05:03:20 smithi110 podman[72190]: 2024-05-09 05:03:20.157565703 +0000 UTC m=+0.718299447 container died afdf08d3c1576d499ee696e5105c385eb8c5f2718bbc0f97946250f77358d7f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2) 2024-05-09T05:03:20.811 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.2.service' 2024-05-09T05:03:20.849 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:03:20.849 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-09T05:03:20.850 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-09T05:03:20.850 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.3 2024-05-09T05:03:21.182 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:20 smithi046 systemd[1]: Stopping Ceph osd.3 for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:03:21.580 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:21 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3[51847]: 2024-05-09T05:03:21.181+0000 7fe115313700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:03:21.580 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:21 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3[51847]: 2024-05-09T05:03:21.181+0000 7fe115313700 -1 osd.3 98 *** Got signal Terminated *** 2024-05-09T05:03:21.580 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:21 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3[51847]: 2024-05-09T05:03:21.181+0000 7fe115313700 -1 osd.3 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:03:23.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:23 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:23.198+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:23.580 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:23 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:23.181+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:24.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:24 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:24.196+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:24.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:24 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:24.158+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:24.558 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:24 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:24.154+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:24.830 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:24 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3[51847]: 2024-05-09T05:03:24.558+0000 7fe10d8c1700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.212727+0000 front 2024-05-09T05:03:01.212666+0000 (oldest deadline 2024-05-09T05:03:24.112882+0000) 2024-05-09T05:03:25.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:24 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:24.916+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:25.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:25 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:25.129+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:25.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:25 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:25.204+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:25.568 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:25 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:25.141+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:25.830 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:25 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3[51847]: 2024-05-09T05:03:25.568+0000 7fe10d8c1700 -1 osd.3 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.212727+0000 front 2024-05-09T05:03:01.212666+0000 (oldest deadline 2024-05-09T05:03:24.112882+0000) 2024-05-09T05:03:26.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:25 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:25.914+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:26.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:26 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:26.221+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:26.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:26 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:26.140+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:26.580 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:26 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:26.167+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:27.057 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:26 smithi046 podman[118060]: 2024-05-09 05:03:26.75800319 +0000 UTC m=+5.702425282 container died 96fd5a286dd62af710f5386eac0876e1117df988863ab3506245cfc1072e774f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:03:27.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:27 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:27.165+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:27.330 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:27 smithi046 podman[118060]: 2024-05-09 05:03:27.265115523 +0000 UTC m=+6.209537665 container cleanup 96fd5a286dd62af710f5386eac0876e1117df988863ab3506245cfc1072e774f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:03:27.330 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:27 smithi046 bash[118060]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3 2024-05-09T05:03:27.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:26 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:26.903+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:27.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:27 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:27.247+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:27.591 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:27 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:27.097+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:27.809 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:27 smithi046 podman[118077]: 2024-05-09 05:03:27.491884176 +0000 UTC m=+0.735802691 container remove 96fd5a286dd62af710f5386eac0876e1117df988863ab3506245cfc1072e774f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-09T05:03:28.080 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:27 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:27.938+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:28.081 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:27 smithi046 podman[118167]: 2024-05-09 05:03:27.809328884 +0000 UTC m=+0.106638402 container create 60cfe6de7b8890a2dbdbff2e12adaf8d506f21521eb2df8ff3ffe15d61f13512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2) 2024-05-09T05:03:28.081 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:27 smithi046 podman[118167]: 2024-05-09 05:03:27.72105385 +0000 UTC m=+0.018363385 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:03:28.081 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:28 smithi046 podman[118167]: 2024-05-09 05:03:28.045251219 +0000 UTC m=+0.342560737 container init 60cfe6de7b8890a2dbdbff2e12adaf8d506f21521eb2df8ff3ffe15d61f13512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-09T05:03:28.081 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:28 smithi046 podman[118167]: 2024-05-09 05:03:28.04877154 +0000 UTC m=+0.346081067 container start 60cfe6de7b8890a2dbdbff2e12adaf8d506f21521eb2df8ff3ffe15d61f13512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T05:03:28.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:27 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:27.872+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:28.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:28 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:28.136+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:28.558 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:28 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:28.224+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:28.580 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:28 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:28.192+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:28.580 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:28 smithi046 podman[118167]: 2024-05-09 05:03:28.128629707 +0000 UTC m=+0.425939230 container attach 60cfe6de7b8890a2dbdbff2e12adaf8d506f21521eb2df8ff3ffe15d61f13512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:03:28.580 INFO:journalctl@ceph.osd.3.smithi046.stdout:May 09 05:03:28 smithi046 podman[118167]: 2024-05-09 05:03:28.354128598 +0000 UTC m=+0.651438121 container died 60cfe6de7b8890a2dbdbff2e12adaf8d506f21521eb2df8ff3ffe15d61f13512 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:03:29.034 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.3.service' 2024-05-09T05:03:29.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:03:29.085 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-09T05:03:29.085 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-09T05:03:29.085 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.4 2024-05-09T05:03:29.135 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:28 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:28.884+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:29.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:28 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:28.986+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:29.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:29 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:29.185+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:29.412 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:29 smithi072 systemd[1]: Stopping Ceph osd.4 for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:03:29.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:29 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:29.265+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:29.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:29 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:29.125+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:29.880 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:29 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4[45207]: 2024-05-09T05:03:29.411+0000 7f9beb810700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:03:29.880 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:29 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4[45207]: 2024-05-09T05:03:29.411+0000 7f9beb810700 -1 osd.4 98 *** Got signal Terminated *** 2024-05-09T05:03:29.880 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:29 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4[45207]: 2024-05-09T05:03:29.411+0000 7f9beb810700 -1 osd.4 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:03:30.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:30 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:30.153+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:30.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:29 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:29.955+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:30.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:29 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:29.929+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:30.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:30 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:30.124+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:30.558 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:30 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:30.241+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:31.217 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:31 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:31.105+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:31.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:31 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:31.183+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:31.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:30 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:30.965+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:31.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:30 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:30.965+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:31.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:30 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:30.946+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:31.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:31 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:31.217+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:31.880 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:31 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4[45207]: 2024-05-09T05:03:31.437+0000 7f9be7dc6700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:06.735612+0000 front 2024-05-09T05:03:06.735580+0000 (oldest deadline 2024-05-09T05:03:30.835738+0000) 2024-05-09T05:03:31.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:31 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:31.395+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:32.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:31 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:31.637+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:32.231 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:32 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:32.156+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:32.232 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:31 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:31.968+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:32.232 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:31 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:31.968+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:32.350 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:31 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:31.974+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:32.350 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:31 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:31.974+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:32.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:32 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:32.174+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:32.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:32 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:32.086+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:32.630 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:32 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:32.349+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:32.631 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:32 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4[45207]: 2024-05-09T05:03:32.402+0000 7f9be7dc6700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:06.735612+0000 front 2024-05-09T05:03:06.735580+0000 (oldest deadline 2024-05-09T05:03:30.835738+0000) 2024-05-09T05:03:33.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:32 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:32.682+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:33.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:33 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:33.120+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:33.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:32 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:32.975+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:33.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:32 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:32.975+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:33.355 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:32 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:32.926+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:33.356 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:32 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:32.926+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:33.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:33 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:33.152+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:33.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:33 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:33.072+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:33.630 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:33 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4[45207]: 2024-05-09T05:03:33.355+0000 7f9be7dc6700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:06.735612+0000 front 2024-05-09T05:03:06.735580+0000 (oldest deadline 2024-05-09T05:03:30.835738+0000) 2024-05-09T05:03:33.631 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:33 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:33.375+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:34.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:33 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:33.692+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:34.308 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:33 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:33.958+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:34.308 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:33 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:33.958+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:34.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:33 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:33.964+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:34.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:33 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:33.964+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:34.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:34 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:34.159+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:34.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:34 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:34.159+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:34.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:34 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:34.106+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:34.558 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:34 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:34.107+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:34.558 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:34 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:34.107+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.107087+0000 front 2024-05-09T05:03:09.107255+0000 (oldest deadline 2024-05-09T05:03:33.207024+0000) 2024-05-09T05:03:34.630 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:34 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4[45207]: 2024-05-09T05:03:34.307+0000 7f9be7dc6700 -1 osd.4 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:06.735612+0000 front 2024-05-09T05:03:06.735580+0000 (oldest deadline 2024-05-09T05:03:30.835738+0000) 2024-05-09T05:03:34.630 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:34 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:34.409+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:35.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:34 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:34.740+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:35.067 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:34 smithi072 podman[70602]: 2024-05-09 05:03:34.754640917 +0000 UTC m=+5.472089462 container died 5e6569d86fd31262016b6d7d973750f513a0088b1eefbe97c37680d020912e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-09T05:03:35.067 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:34 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:34.970+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:35.068 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:34 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:34.970+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:35.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:35 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:35.118+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:35.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:35 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:35.118+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:35.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:34 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:34.941+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:35.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:34 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:34.941+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:35.380 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:35 smithi072 podman[70602]: 2024-05-09 05:03:35.17660449 +0000 UTC m=+5.894053029 container cleanup 5e6569d86fd31262016b6d7d973750f513a0088b1eefbe97c37680d020912e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-09T05:03:35.380 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:35 smithi072 bash[70602]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4 2024-05-09T05:03:35.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:35 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:35.134+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:35.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:35 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:35.134+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:35.558 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:35 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:35.087+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:35.558 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:35 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:35.087+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.107087+0000 front 2024-05-09T05:03:09.107255+0000 (oldest deadline 2024-05-09T05:03:33.207024+0000) 2024-05-09T05:03:35.796 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:35 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:35.433+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:35.797 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:35 smithi072 podman[70621]: 2024-05-09 05:03:35.397633531 +0000 UTC m=+0.637390205 container remove 5e6569d86fd31262016b6d7d973750f513a0088b1eefbe97c37680d020912e67 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:03:36.050 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:35 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:35.984+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:36.050 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:35 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:35.984+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:36.050 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:35 smithi072 podman[70714]: 2024-05-09 05:03:35.69958208 +0000 UTC m=+0.017769985 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:03:36.050 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:35 smithi072 podman[70714]: 2024-05-09 05:03:35.817741281 +0000 UTC m=+0.135929228 container create 94d9ed01d87f0dfaa3632e79273d82b393fd4fcc737e57703386c673e1f4da5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4-deactivate, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-05-09T05:03:36.050 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:36 smithi072 podman[70714]: 2024-05-09 05:03:36.047208682 +0000 UTC m=+0.365396589 container init 94d9ed01d87f0dfaa3632e79273d82b393fd4fcc737e57703386c673e1f4da5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS) 2024-05-09T05:03:36.050 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:36 smithi072 podman[70714]: 2024-05-09 05:03:36.050245564 +0000 UTC m=+0.368433469 container start 94d9ed01d87f0dfaa3632e79273d82b393fd4fcc737e57703386c673e1f4da5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-09T05:03:36.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:35 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:35.733+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:36.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:36 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:36.092+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:36.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:36 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:36.092+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:36.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:35 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:35.975+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:36.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:35 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:35.975+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:36.343 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:36 smithi072 podman[70714]: 2024-05-09 05:03:36.156305627 +0000 UTC m=+0.474493541 container attach 94d9ed01d87f0dfaa3632e79273d82b393fd4fcc737e57703386c673e1f4da5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-09T05:03:36.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:36 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:36.128+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:36.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:36 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:36.128+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:36.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:36 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:36.109+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:36.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:36 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:36.109+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.107087+0000 front 2024-05-09T05:03:09.107255+0000 (oldest deadline 2024-05-09T05:03:33.207024+0000) 2024-05-09T05:03:36.630 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:36 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:36.440+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:36.630 INFO:journalctl@ceph.osd.4.smithi072.stdout:May 09 05:03:36 smithi072 podman[70714]: 2024-05-09 05:03:36.343587855 +0000 UTC m=+0.661775765 container died 94d9ed01d87f0dfaa3632e79273d82b393fd4fcc737e57703386c673e1f4da5b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=) 2024-05-09T05:03:37.035 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.4.service' 2024-05-09T05:03:37.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:36 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:36.775+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:37.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:03:37.079 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-09T05:03:37.080 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-09T05:03:37.080 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.5 2024-05-09T05:03:37.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:37 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:37.123+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:37.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:37 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:37.123+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:37.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:37 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:37.009+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:37.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:37 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:37.009+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:37.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:37 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:37.033+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:37.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:37 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:37.033+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:37.381 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:37 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:37.117+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:37.381 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:37 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:37.117+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:37.382 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:37 smithi110 systemd[1]: Stopping Ceph osd.5 for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:03:37.382 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:37 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:37.136+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:37.382 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:37 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:37.136+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.107087+0000 front 2024-05-09T05:03:09.107255+0000 (oldest deadline 2024-05-09T05:03:33.207024+0000) 2024-05-09T05:03:37.382 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:37 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:37.380+0000 7fe8f2d22700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:03:37.382 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:37 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:37.380+0000 7fe8f2d22700 -1 osd.5 98 *** Got signal Terminated *** 2024-05-09T05:03:37.733 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:37 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:37.380+0000 7fe8f2d22700 -1 osd.5 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:03:37.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:37 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:37.443+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:38.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:37 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:37.733+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:38.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:38 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:38.113+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:38.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:38 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:38.113+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:38.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:38 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:38.009+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:38.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:38 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:38.009+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:38.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:38 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:38.056+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:38.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:38 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:38.056+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:38.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:38.161+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:38.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:38.161+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:38.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:38.102+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:38.558 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:38.102+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.107087+0000 front 2024-05-09T05:03:09.107255+0000 (oldest deadline 2024-05-09T05:03:33.207024+0000) 2024-05-09T05:03:38.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:38 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:38.472+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:39.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:38 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:38.761+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:39.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:39 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:39.114+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:39.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:39 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:39.114+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:39.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:39 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:39.012+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:39.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:39 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:39.012+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:39.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:39 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:39.081+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:39.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:39 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:39.081+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:39.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:39 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:39.116+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:39.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:39 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:39.116+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.107087+0000 front 2024-05-09T05:03:09.107255+0000 (oldest deadline 2024-05-09T05:03:33.207024+0000) 2024-05-09T05:03:39.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:39 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:39.186+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:39.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:39 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:39.186+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:39.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:39 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:39.508+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:40.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:39 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:39.770+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:40.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:39 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:39.770+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:40.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:40 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:40.025+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:40.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:40 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:40.025+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:40.331 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:40 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:40.079+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:40.331 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:40 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:40.079+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:40.331 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:40 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:40.079+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:13.866272+0000 front 2024-05-09T05:03:13.866402+0000 (oldest deadline 2024-05-09T05:03:39.766433+0000) 2024-05-09T05:03:40.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:40 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:40.040+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:40.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:40 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:40.040+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:40.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:40 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:40.116+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:40.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:40 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:40.116+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.107087+0000 front 2024-05-09T05:03:09.107255+0000 (oldest deadline 2024-05-09T05:03:33.207024+0000) 2024-05-09T05:03:40.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:40 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:40.116+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.008702+0000 front 2024-05-09T05:03:16.008749+0000 (oldest deadline 2024-05-09T05:03:40.108819+0000) 2024-05-09T05:03:40.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:40 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:40.173+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:40.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:40 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:40.173+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:40.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:40 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:40.173+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:40.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:40 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:40.508+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:41.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:40 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:40.749+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:41.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:40 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:40.749+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:41.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:40 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:40.976+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:41.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:40 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:40.976+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:41.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:41 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:41.081+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:41.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:41 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:41.081+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:41.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:41 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:41.081+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:13.866272+0000 front 2024-05-09T05:03:13.866402+0000 (oldest deadline 2024-05-09T05:03:39.766433+0000) 2024-05-09T05:03:41.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:41 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:41.022+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:41.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:41 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:41.022+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:41.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:41 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:41.022+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:41.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:41 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:41.136+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:41.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:41 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:41.136+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.107087+0000 front 2024-05-09T05:03:09.107255+0000 (oldest deadline 2024-05-09T05:03:33.207024+0000) 2024-05-09T05:03:41.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:41 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:41.136+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.008702+0000 front 2024-05-09T05:03:16.008749+0000 (oldest deadline 2024-05-09T05:03:40.108819+0000) 2024-05-09T05:03:41.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:41 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:41.201+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:41.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:41 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:41.201+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:41.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:41 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:41.201+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:41.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:41 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:41.528+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:41.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:41 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:41.528+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:42.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:41 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:41.771+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:42.058 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:41 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:41.771+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:42.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:41 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:41.976+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:42.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:41 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:41.976+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:42.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:42 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:42.088+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:42.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:42 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:42.088+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:42.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:42 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:42.088+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:13.866272+0000 front 2024-05-09T05:03:13.866402+0000 (oldest deadline 2024-05-09T05:03:39.766433+0000) 2024-05-09T05:03:42.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:41 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:41.977+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:42.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:41 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:41.977+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:42.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:41 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:41.977+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:42.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:42 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:42.155+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:02:57.604389+0000 front 2024-05-09T05:02:57.604346+0000 (oldest deadline 2024-05-09T05:03:22.304373+0000) 2024-05-09T05:03:42.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:42 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:42.155+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.107087+0000 front 2024-05-09T05:03:09.107255+0000 (oldest deadline 2024-05-09T05:03:33.207024+0000) 2024-05-09T05:03:42.557 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:42 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5[46760]: 2024-05-09T05:03:42.155+0000 7fe8eb2d0700 -1 osd.5 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.008702+0000 front 2024-05-09T05:03:16.008749+0000 (oldest deadline 2024-05-09T05:03:40.108819+0000) 2024-05-09T05:03:42.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:42 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:42.167+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:42.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:42 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:42.167+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:42.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:42 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:42.167+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:42.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:42 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:42.571+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:42.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:42 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:42.571+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:43.057 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:42 smithi110 podman[72306]: 2024-05-09 05:03:42.949017868 +0000 UTC m=+5.669626089 container died b7022f42db85afe675c26ee9c7d14944ea27be39ef6c5b2f278591b399f3499f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-09T05:03:43.058 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:42 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:42.789+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:43.058 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:42 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:42.789+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:43.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:42 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:42.953+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:43.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:42 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:42.953+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:43.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:42 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:42.953+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:43.331 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:43 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:43.070+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:43.331 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:43 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:43.070+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:43.331 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:43 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:43.070+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:13.866272+0000 front 2024-05-09T05:03:13.866402+0000 (oldest deadline 2024-05-09T05:03:39.766433+0000) 2024-05-09T05:03:43.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:42 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:42.947+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:43.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:42 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:42.947+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:43.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:42 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:42.947+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:43.549 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:43 smithi110 podman[72306]: 2024-05-09 05:03:43.348487161 +0000 UTC m=+6.069095374 container cleanup b7022f42db85afe675c26ee9c7d14944ea27be39ef6c5b2f278591b399f3499f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-09T05:03:43.550 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:43 smithi110 bash[72306]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5 2024-05-09T05:03:43.550 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:43 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:43.155+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:43.550 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:43 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:43.155+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:43.550 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:43 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:43.155+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:43.807 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:43 smithi110 podman[72323]: 2024-05-09 05:03:43.549930415 +0000 UTC m=+0.597451764 container remove b7022f42db85afe675c26ee9c7d14944ea27be39ef6c5b2f278591b399f3499f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-05-09T05:03:43.807 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:43 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:43.764+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:43.807 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:43 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:43.764+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:43.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:43 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:43.592+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:43.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:43 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:43.592+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:44.307 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:44 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:44.107+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:44.308 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:44 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:44.107+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:44.308 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:44 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:44.107+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:44.308 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:43 smithi110 podman[72417]: 2024-05-09 05:03:43.87141468 +0000 UTC m=+0.018332572 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:03:44.308 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:44 smithi110 podman[72417]: 2024-05-09 05:03:44.00103743 +0000 UTC m=+0.147955346 container create 1a02df2a4e8a94534a2e2c3e7bc602abe9562bd40e9d6d7b5a2c0c0b040793fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-09T05:03:44.308 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:44 smithi110 podman[72417]: 2024-05-09 05:03:44.211826281 +0000 UTC m=+0.358744155 container init 1a02df2a4e8a94534a2e2c3e7bc602abe9562bd40e9d6d7b5a2c0c0b040793fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-09T05:03:44.308 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:44 smithi110 podman[72417]: 2024-05-09 05:03:44.215181457 +0000 UTC m=+0.362099332 container start 1a02df2a4e8a94534a2e2c3e7bc602abe9562bd40e9d6d7b5a2c0c0b040793fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-09T05:03:44.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:44 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:44.087+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:44.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:44 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:44.087+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:44.331 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:44 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:44.087+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:13.866272+0000 front 2024-05-09T05:03:13.866402+0000 (oldest deadline 2024-05-09T05:03:39.766433+0000) 2024-05-09T05:03:44.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:43 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:43.971+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:44.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:43 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:43.971+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:44.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:43 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:43.971+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:44.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:43 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:43.919+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:44.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:43 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:43.919+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:44.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:43 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:43.919+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:44.780 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:44 smithi110 podman[72417]: 2024-05-09 05:03:44.311881852 +0000 UTC m=+0.458799738 container attach 1a02df2a4e8a94534a2e2c3e7bc602abe9562bd40e9d6d7b5a2c0c0b040793fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T05:03:44.780 INFO:journalctl@ceph.osd.5.smithi110.stdout:May 09 05:03:44 smithi110 podman[72417]: 2024-05-09 05:03:44.507033098 +0000 UTC m=+0.653950973 container died 1a02df2a4e8a94534a2e2c3e7bc602abe9562bd40e9d6d7b5a2c0c0b040793fb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:03:44.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:44 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:44.581+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:44.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:44 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:44.581+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:45.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:44 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:44.779+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:45.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:44 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:44.779+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:45.191 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.5.service' 2024-05-09T05:03:45.232 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:03:45.233 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-09T05:03:45.233 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-09T05:03:45.233 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.6 2024-05-09T05:03:45.262 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:44 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:44.993+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:45.262 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:44 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:44.993+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:45.262 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:44 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:44.993+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:45.262 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:45 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:45.100+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:45.262 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:45 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:45.100+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:45.262 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:45 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:45.100+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:13.866272+0000 front 2024-05-09T05:03:13.866402+0000 (oldest deadline 2024-05-09T05:03:39.766433+0000) 2024-05-09T05:03:45.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:44 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:44.880+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:45.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:44 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:44.880+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:45.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:44 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:44.880+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:45.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:45 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:45.101+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:45.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:45 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:45.101+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:45.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:45 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:45.101+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:45.569 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:45 smithi046 systemd[1]: Stopping Ceph osd.6 for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:03:45.830 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:45 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:45.569+0000 7f4e287f4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:03:45.830 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:45 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:45.569+0000 7f4e287f4700 -1 osd.6 98 *** Got signal Terminated *** 2024-05-09T05:03:45.830 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:45 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:45.569+0000 7f4e287f4700 -1 osd.6 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:03:45.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:45 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:45.605+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:45.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:45 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:45.605+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:46.058 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:45 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:45.774+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:46.059 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:45 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:45.774+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:46.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:45 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:45.975+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:46.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:45 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:45.975+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:46.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:45 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:45.975+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:46.331 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:46 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:46.111+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:46.331 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:46 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:46.111+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:46.331 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:46 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:46.111+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:13.866272+0000 front 2024-05-09T05:03:13.866402+0000 (oldest deadline 2024-05-09T05:03:39.766433+0000) 2024-05-09T05:03:46.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:45 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:45.907+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:46.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:45 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:45.907+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:46.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:45 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:45.907+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:46.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:46 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:46.110+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:46.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:46 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:46.110+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:46.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:46 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:46.110+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:46.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:46 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:46.589+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:46.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:46 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:46.589+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:47.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:46 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:46.795+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:47.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:46 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:46.795+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:47.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:47 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:47.067+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:47.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:47 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:47.067+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:47.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:47 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:47.067+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:13.866272+0000 front 2024-05-09T05:03:13.866402+0000 (oldest deadline 2024-05-09T05:03:39.766433+0000) 2024-05-09T05:03:47.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:47 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:47.067+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:25.667141+0000 front 2024-05-09T05:03:25.667223+0000 (oldest deadline 2024-05-09T05:03:46.767267+0000) 2024-05-09T05:03:47.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:46 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:46.986+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:47.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:46 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:46.986+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:47.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:46 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:46.986+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:47.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:46 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:46.986+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:03:47.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:46 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:46.917+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:47.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:46 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:46.917+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:47.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:46 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:46.917+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:47.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:47 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:47.100+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:47.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:47 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:47.100+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:47.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:47 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:47.100+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:47.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:47 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:47.629+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:47.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:47 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:47.629+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:47.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:47 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:47.629+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.195423+0000 front 2024-05-09T05:03:22.195495+0000 (oldest deadline 2024-05-09T05:03:46.895436+0000) 2024-05-09T05:03:48.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:47 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:47.779+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:48.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:47 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:47.779+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:48.058 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:47 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:47.779+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:03:48.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:48 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:48.080+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:48.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:48 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:48.080+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:48.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:48 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:48.080+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:13.866272+0000 front 2024-05-09T05:03:13.866402+0000 (oldest deadline 2024-05-09T05:03:39.766433+0000) 2024-05-09T05:03:48.331 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:48 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:48.080+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:25.667141+0000 front 2024-05-09T05:03:25.667223+0000 (oldest deadline 2024-05-09T05:03:46.767267+0000) 2024-05-09T05:03:48.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:48 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:48.014+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:48.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:48 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:48.014+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:48.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:48 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:48.014+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:48.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:48 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:48.014+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:03:48.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:47 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:47.894+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:48.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:47 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:47.894+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:48.381 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:47 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:47.894+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:48.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:48 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:48.122+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:48.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:48 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:48.122+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:48.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:48 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:48.122+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:48.875 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:48 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:48.580+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:48.875 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:48 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:48.580+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:48.875 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:48 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:48.580+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.195423+0000 front 2024-05-09T05:03:22.195495+0000 (oldest deadline 2024-05-09T05:03:46.895436+0000) 2024-05-09T05:03:49.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:48 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:48.766+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:49.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:48 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:48.766+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:49.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:48 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:48.766+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:03:49.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:48 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:48.874+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:49.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:48 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:48.874+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:49.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:48 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:48.874+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:49.329 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:49 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:49.164+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:49.329 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:49 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:49.164+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:49.329 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:49 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:49.164+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:49.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:49 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:49.048+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:49.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:49 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:49.048+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:49.331 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:49 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:49.048+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:13.866272+0000 front 2024-05-09T05:03:13.866402+0000 (oldest deadline 2024-05-09T05:03:39.766433+0000) 2024-05-09T05:03:49.331 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:49 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:49.048+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:25.667141+0000 front 2024-05-09T05:03:25.667223+0000 (oldest deadline 2024-05-09T05:03:46.767267+0000) 2024-05-09T05:03:49.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:49 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:49.053+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:49.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:49 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:49.053+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:49.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:49 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:49.053+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:49.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:49 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:49.053+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:03:49.877 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:49 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:49.589+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:49.877 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:49 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:49.589+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:49.877 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:49 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:49.589+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.195423+0000 front 2024-05-09T05:03:22.195495+0000 (oldest deadline 2024-05-09T05:03:46.895436+0000) 2024-05-09T05:03:50.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:49 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:49.774+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:50.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:49 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:49.774+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:50.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:49 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:49.774+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:03:50.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:49 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:49.876+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:50.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:49 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:49.876+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:50.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:49 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:49.876+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:50.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:49 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:49.876+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:03:50.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:50 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:50.061+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:50.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:50 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:50.061+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:50.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:50 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:50.061+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:50.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:50 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:50.061+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:03:50.331 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:50 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:50.025+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.563299+0000 front 2024-05-09T05:03:00.563375+0000 (oldest deadline 2024-05-09T05:03:22.863450+0000) 2024-05-09T05:03:50.331 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:50 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:50.025+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.166038+0000 front 2024-05-09T05:03:09.165848+0000 (oldest deadline 2024-05-09T05:03:33.865812+0000) 2024-05-09T05:03:50.332 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:50 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:50.025+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:13.866272+0000 front 2024-05-09T05:03:13.866402+0000 (oldest deadline 2024-05-09T05:03:39.766433+0000) 2024-05-09T05:03:50.332 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:50 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6[58649]: 2024-05-09T05:03:50.025+0000 7f4e24daa700 -1 osd.6 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:25.667141+0000 front 2024-05-09T05:03:25.667223+0000 (oldest deadline 2024-05-09T05:03:46.767267+0000) 2024-05-09T05:03:50.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:50 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:50.204+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:50.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:50 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:50.204+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:50.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:50 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:50.204+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:50.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:50 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:50.604+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:50.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:50 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:50.604+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:50.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:50 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:50.604+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.195423+0000 front 2024-05-09T05:03:22.195495+0000 (oldest deadline 2024-05-09T05:03:46.895436+0000) 2024-05-09T05:03:51.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:50 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:50.766+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:51.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:50 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:50.766+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:51.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:50 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:50.766+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:03:51.330 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:51 smithi046 podman[118284]: 2024-05-09 05:03:51.048220617 +0000 UTC m=+5.610393583 container died 25b84c76eb263b31d79a3e85b4dc317054cf6bbbdd931c1f4b7de97a8fcb3314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-09T05:03:51.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:51 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:51.070+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:51.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:51 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:51.070+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:51.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:51 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:51.070+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:51.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:51 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:51.070+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:03:51.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:50 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:50.882+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:51.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:50 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:50.882+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:51.381 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:50 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:50.882+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:51.381 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:50 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:50.882+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:03:51.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:51 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:51.253+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:51.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:51 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:51.253+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:51.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:51 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:51.253+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:51.761 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:51 smithi046 podman[118284]: 2024-05-09 05:03:51.544019631 +0000 UTC m=+6.106192664 container cleanup 25b84c76eb263b31d79a3e85b4dc317054cf6bbbdd931c1f4b7de97a8fcb3314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:03:51.761 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:51 smithi046 bash[118284]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6 2024-05-09T05:03:51.852 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:51 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:51.585+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:51.852 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:51 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:51.585+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:51.852 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:51 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:51.585+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.195423+0000 front 2024-05-09T05:03:22.195495+0000 (oldest deadline 2024-05-09T05:03:46.895436+0000) 2024-05-09T05:03:52.026 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:51 smithi046 podman[118301]: 2024-05-09 05:03:51.761912898 +0000 UTC m=+0.711925820 container remove 25b84c76eb263b31d79a3e85b4dc317054cf6bbbdd931c1f4b7de97a8fcb3314 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True) 2024-05-09T05:03:52.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:51 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:51.785+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:52.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:51 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:51.785+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:52.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:51 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:51.785+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:03:52.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:51 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:51.851+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:52.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:51 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:51.851+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:52.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:51 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:51.851+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:52.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:51 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:51.851+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:03:52.324 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:52 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:52.026+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:52.324 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:52 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:52.026+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:52.324 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:52 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:52.026+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:52.324 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:52 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:52.026+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:03:52.325 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:52 smithi046 podman[118390]: 2024-05-09 05:03:52.071298576 +0000 UTC m=+0.109268773 container create 306926c1ea938286aa4898309eed7dd574f6b1839c92d84625ac8e14f6c39718 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:03:52.325 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:52 smithi046 podman[118390]: 2024-05-09 05:03:51.980620447 +0000 UTC m=+0.018590659 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:03:52.325 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:52 smithi046 podman[118390]: 2024-05-09 05:03:52.323893063 +0000 UTC m=+0.361863261 container init 306926c1ea938286aa4898309eed7dd574f6b1839c92d84625ac8e14f6c39718 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-05-09T05:03:52.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:52 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:52.229+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:52.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:52 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:52.229+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:52.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:52 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:52.229+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:52.580 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:52 smithi046 podman[118390]: 2024-05-09 05:03:52.327662733 +0000 UTC m=+0.365632933 container start 306926c1ea938286aa4898309eed7dd574f6b1839c92d84625ac8e14f6c39718 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-09T05:03:52.580 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:52 smithi046 podman[118390]: 2024-05-09 05:03:52.41564422 +0000 UTC m=+0.453614419 container attach 306926c1ea938286aa4898309eed7dd574f6b1839c92d84625ac8e14f6c39718 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:03:52.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:52 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:52.571+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:52.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:52 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:52.571+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:52.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:52 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:52.571+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.195423+0000 front 2024-05-09T05:03:22.195495+0000 (oldest deadline 2024-05-09T05:03:46.895436+0000) 2024-05-09T05:03:52.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:52 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:52.810+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:52.881 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:52 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:52.810+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:52.881 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:52 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:52.810+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:52.881 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:52 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:52.810+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:03:52.926 INFO:journalctl@ceph.osd.6.smithi046.stdout:May 09 05:03:52 smithi046 podman[118390]: 2024-05-09 05:03:52.632081138 +0000 UTC m=+0.670051339 container died 306926c1ea938286aa4898309eed7dd574f6b1839c92d84625ac8e14f6c39718 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-6-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-09T05:03:53.204 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:52 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:52.813+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:53.205 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:52 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:52.813+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:53.205 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:52 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:52.813+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:03:53.227 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:53 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:53.011+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:53.227 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:53 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:53.011+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:53.227 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:53 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:53.011+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:53.227 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:53 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:53.011+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:03:53.252 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.6.service' 2024-05-09T05:03:53.335 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:03:53.335 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-09T05:03:53.335 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-09T05:03:53.335 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.7 2024-05-09T05:03:53.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:53 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:53.204+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:53.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:53 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:53.204+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:53.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:53 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:53.204+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:53.630 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:53 smithi072 systemd[1]: Stopping Ceph osd.7 for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:03:53.630 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:53 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:53.533+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:53.630 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:53 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:53.533+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:53.630 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:53 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:53.533+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.195423+0000 front 2024-05-09T05:03:22.195495+0000 (oldest deadline 2024-05-09T05:03:46.895436+0000) 2024-05-09T05:03:53.896 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:53 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:53.664+0000 7f931cb84700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:03:53.897 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:53 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:53.664+0000 7f931cb84700 -1 osd.7 98 *** Got signal Terminated *** 2024-05-09T05:03:53.897 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:53 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:53.664+0000 7f931cb84700 -1 osd.7 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:03:53.897 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:53 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:53.833+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:53.897 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:53 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:53.833+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:53.897 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:53 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:53.833+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:53.897 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:53 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:53.833+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:03:54.175 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:53 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:53.837+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:54.176 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:53 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:53.837+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:54.176 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:53 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:53.837+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:03:54.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:54 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:54.042+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:54.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:54 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:54.042+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:54.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:54 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:54.042+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:54.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:54 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:54.042+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:03:54.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:54 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:54.175+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:54.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:54 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:54.175+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:54.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:54 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:54.175+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:54.834 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:54 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:54.532+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:54.834 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:54 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:54.532+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:54.834 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:54 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:54.532+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.195423+0000 front 2024-05-09T05:03:22.195495+0000 (oldest deadline 2024-05-09T05:03:46.895436+0000) 2024-05-09T05:03:55.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:54 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:54.852+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:55.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:54 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:54.852+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:55.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:54 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:54.852+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:55.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:54 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:54.852+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:03:55.160 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:54 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:54.859+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:55.160 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:54 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:54.859+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:55.160 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:54 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:54.859+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:03:55.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:55 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:55.025+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:55.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:55 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:55.025+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:55.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:55 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:55.025+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:55.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:55 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:55.025+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:03:55.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:55 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:55.159+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:55.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:55 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:55.159+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:55.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:55 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:55.159+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:55.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:55 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:55.525+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:55.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:55 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:55.525+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:55.880 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:55 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:55.525+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.195423+0000 front 2024-05-09T05:03:22.195495+0000 (oldest deadline 2024-05-09T05:03:46.895436+0000) 2024-05-09T05:03:56.200 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:55 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:55.844+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:56.200 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:55 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:55.844+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:56.200 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:55 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:55.844+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:03:56.200 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:55 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:55.844+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.600331+0000 front 2024-05-09T05:03:33.600302+0000 (oldest deadline 2024-05-09T05:03:55.800722+0000) 2024-05-09T05:03:56.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:56 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:56.025+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:56.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:56 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:56.025+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:56.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:56 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:56.025+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:56.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:56 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:56.025+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:03:56.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:55 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:55.886+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:56.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:55 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:55.886+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:56.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:55 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:55.886+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:56.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:55 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:55.886+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:03:56.380 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:55 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:55.886+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:03:56.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:56 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:56.200+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:56.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:56 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:56.200+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:56.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:56 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:56.200+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:56.846 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:56 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:56.482+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:56.846 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:56 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:56.482+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:56.846 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:56 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:56.482+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.195423+0000 front 2024-05-09T05:03:22.195495+0000 (oldest deadline 2024-05-09T05:03:46.895436+0000) 2024-05-09T05:03:57.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:56 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:56.845+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:57.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:56 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:56.845+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:57.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:56 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:56.845+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:57.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:56 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:56.845+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:03:57.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:56 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:56.845+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:03:57.206 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:56 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:56.846+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:57.206 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:56 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:56.846+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:57.206 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:56 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:56.846+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:03:57.206 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:56 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:56.846+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.600331+0000 front 2024-05-09T05:03:33.600302+0000 (oldest deadline 2024-05-09T05:03:55.800722+0000) 2024-05-09T05:03:57.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:57.024+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:57.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:57.024+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:57.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:57.024+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:57.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:57.024+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:03:57.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:57.024+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:03:57.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:57 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:57.206+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:57.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:57 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:57.206+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:57.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:57 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:57.206+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:57.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:57 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:57.206+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:03:57.804 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:57 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:57.482+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:57.804 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:57 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:57.482+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:57.804 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:57 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:57.482+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.195423+0000 front 2024-05-09T05:03:22.195495+0000 (oldest deadline 2024-05-09T05:03:46.895436+0000) 2024-05-09T05:03:58.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:57 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:57.804+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:58.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:57 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:57.804+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:58.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:57 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:57.804+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:58.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:57 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:57.804+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:03:58.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:57 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:57.804+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:03:58.219 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:57 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:57.850+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:58.219 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:57 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:57.850+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:58.219 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:57 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:57.850+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:03:58.219 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:57 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:57.850+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.600331+0000 front 2024-05-09T05:03:33.600302+0000 (oldest deadline 2024-05-09T05:03:55.800722+0000) 2024-05-09T05:03:58.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:57.990+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:58.347 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:57.990+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:58.347 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:57.990+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:58.347 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:57.990+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:03:58.347 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:57 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:57.990+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:03:58.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:58 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:58.218+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:58.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:58 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:58.218+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:58.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:58 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:58.218+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:58.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:58 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:58.218+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:03:58.840 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:58 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:58.468+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:05.292641+0000 front 2024-05-09T05:03:05.292670+0000 (oldest deadline 2024-05-09T05:03:31.192837+0000) 2024-05-09T05:03:58.840 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:58 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:58.468+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:17.093722+0000 front 2024-05-09T05:03:17.093864+0000 (oldest deadline 2024-05-09T05:03:41.094272+0000) 2024-05-09T05:03:58.840 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:58 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7[51744]: 2024-05-09T05:03:58.468+0000 7f931913a700 -1 osd.7 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.195423+0000 front 2024-05-09T05:03:22.195495+0000 (oldest deadline 2024-05-09T05:03:46.895436+0000) 2024-05-09T05:03:59.123 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:58 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:58.839+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:03:59.124 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:58 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:58.839+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:03:59.124 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:58 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:58.839+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:03:59.124 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:58 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:58.839+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:03:59.124 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:58 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:58.839+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:03:59.220 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:58 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:58.895+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:03:59.221 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:58 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:58.895+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:03:59.221 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:58 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:58.895+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:03:59.221 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:58 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:58.895+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.600331+0000 front 2024-05-09T05:03:33.600302+0000 (oldest deadline 2024-05-09T05:03:55.800722+0000) 2024-05-09T05:03:59.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:58 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:58.997+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:03:59.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:58 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:58.997+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:03:59.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:58 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:58.997+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:03:59.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:58 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:58.997+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:03:59.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:03:58 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:03:58.997+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:03:59.380 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:59 smithi072 podman[70817]: 2024-05-09 05:03:59.124089537 +0000 UTC m=+5.592709073 container died 35164b5d7257898b2c6f358ecdf3a648bda749d26250e5f41acf535c0fce0483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_CLEAN=True) 2024-05-09T05:03:59.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:59 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:59.220+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:03:59.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:59 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:59.220+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:03:59.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:59 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:59.220+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:03:59.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:03:59 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:03:59.220+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:03:59.789 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:59 smithi072 podman[70817]: 2024-05-09 05:03:59.569320489 +0000 UTC m=+6.037940024 container cleanup 35164b5d7257898b2c6f358ecdf3a648bda749d26250e5f41acf535c0fce0483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:03:59.789 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:59 smithi072 bash[70817]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7 2024-05-09T05:04:00.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:59 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:59.866+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:04:00.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:59 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:59.866+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:04:00.058 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:59 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:59.866+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:04:00.058 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:03:59 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:03:59.866+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.600331+0000 front 2024-05-09T05:03:33.600302+0000 (oldest deadline 2024-05-09T05:03:55.800722+0000) 2024-05-09T05:04:00.130 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:03:59 smithi072 podman[70847]: 2024-05-09 05:03:59.788734302 +0000 UTC m=+0.660848299 container remove 35164b5d7257898b2c6f358ecdf3a648bda749d26250e5f41acf535c0fce0483 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507) 2024-05-09T05:04:00.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:59 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:59.840+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:00.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:59 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:59.840+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:00.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:59 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:59.840+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:00.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:59 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:59.840+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:00.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:03:59 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:03:59.840+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:00.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:00 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:00.012+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:04:00.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:00 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:00.012+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:04:00.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:00 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:00.012+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:04:00.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:00 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:00.012+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:04:00.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:00 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:00.012+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:04:00.441 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:04:00 smithi072 podman[70941]: 2024-05-09 05:04:00.09308499 +0000 UTC m=+0.017760337 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:04:00.441 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:04:00 smithi072 podman[70941]: 2024-05-09 05:04:00.190621645 +0000 UTC m=+0.115296957 container create 8e274f2def3db53a301b9bb8de9a2ecb22a35444f1897daafa218065d52092d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-05-09T05:04:00.441 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:04:00 smithi072 podman[70941]: 2024-05-09 05:04:00.438100411 +0000 UTC m=+0.362775736 container init 8e274f2def3db53a301b9bb8de9a2ecb22a35444f1897daafa218065d52092d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-09T05:04:00.441 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:04:00 smithi072 podman[70941]: 2024-05-09 05:04:00.441539591 +0000 UTC m=+0.366214903 container start 8e274f2def3db53a301b9bb8de9a2ecb22a35444f1897daafa218065d52092d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7-deactivate, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-05-09T05:04:00.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:00 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:00.218+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:00.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:00 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:00.218+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:00.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:00 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:00.218+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:00.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:00 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:00.218+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:00.733 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:04:00 smithi072 podman[70941]: 2024-05-09 05:04:00.555747897 +0000 UTC m=+0.480423217 container attach 8e274f2def3db53a301b9bb8de9a2ecb22a35444f1897daafa218065d52092d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:04:01.009 INFO:journalctl@ceph.osd.7.smithi072.stdout:May 09 05:04:00 smithi072 podman[70941]: 2024-05-09 05:04:00.734622861 +0000 UTC m=+0.659298181 container died 8e274f2def3db53a301b9bb8de9a2ecb22a35444f1897daafa218065d52092d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-09T05:04:01.010 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:00 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:00.868+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:01.010 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:00 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:00.868+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:01.010 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:00 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:00.868+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:01.010 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:00 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:00.868+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:01.010 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:00 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:00.868+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:01.258 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:00 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:00.866+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:04:01.258 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:00 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:00.866+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:04:01.258 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:00 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:00.866+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:04:01.258 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:00 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:00.866+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.600331+0000 front 2024-05-09T05:03:33.600302+0000 (oldest deadline 2024-05-09T05:03:55.800722+0000) 2024-05-09T05:04:01.317 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.7.service' 2024-05-09T05:04:01.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:01 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:01.051+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:04:01.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:01 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:01.051+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:04:01.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:01 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:01.051+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:04:01.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:01 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:01.051+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:04:01.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:01 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:01.051+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:04:01.357 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:04:01.357 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-09T05:04:01.357 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-09T05:04:01.358 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.8 2024-05-09T05:04:01.557 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:01 smithi110 systemd[1]: Stopping Ceph osd.8 for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:04:01.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:01 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:01.258+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:01.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:01 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:01.258+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:01.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:01 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:01.258+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:01.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:01 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:01.258+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:02.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:01 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:01.676+0000 7f9d22afe700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:04:02.058 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:01 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:01.676+0000 7f9d22afe700 -1 osd.8 98 *** Got signal Terminated *** 2024-05-09T05:04:02.058 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:01 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:01.676+0000 7f9d22afe700 -1 osd.8 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:04:02.058 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:01 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:01.842+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:04:02.058 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:01 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:01.842+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:04:02.058 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:01 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:01.842+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:04:02.058 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:01 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:01.842+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.600331+0000 front 2024-05-09T05:03:33.600302+0000 (oldest deadline 2024-05-09T05:03:55.800722+0000) 2024-05-09T05:04:02.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:01 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:01.830+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:02.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:01 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:01.830+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:02.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:01 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:01.830+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:02.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:01 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:01.830+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:02.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:01 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:01.830+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:02.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:02 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:02.071+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:04:02.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:02 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:02.071+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:04:02.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:02 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:02.071+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:04:02.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:02 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:02.072+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:04:02.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:02 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:02.072+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:04:02.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:02 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:02.305+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:02.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:02 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:02.305+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:02.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:02 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:02.305+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:02.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:02 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:02.305+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:03.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:02 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:02.798+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:03.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:02 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:02.798+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:03.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:02 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:02.798+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:03.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:02 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:02.798+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:03.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:02 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:02.798+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:03.307 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:02 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:02.863+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:04:03.307 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:02 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:02.863+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:04:03.307 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:02 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:02.863+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:04:03.308 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:02 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:02.863+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.600331+0000 front 2024-05-09T05:03:33.600302+0000 (oldest deadline 2024-05-09T05:03:55.800722+0000) 2024-05-09T05:04:03.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:03 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:03.064+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:04:03.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:03 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:03.064+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:04:03.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:03 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:03.064+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:04:03.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:03 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:03.064+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:04:03.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:03 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:03.064+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:04:03.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:03 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:03.339+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:03.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:03 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:03.339+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:03.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:03 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:03.339+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:03.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:03 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:03.339+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:04.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:03 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:03.780+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:04.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:03 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:03.780+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:04.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:03 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:03.780+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:04.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:03 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:03.780+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:04.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:03 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:03.780+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:04.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:03 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:03.780+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:04.307 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:03 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:03.816+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:04:04.307 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:03 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:03.816+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:04:04.307 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:03 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:03.816+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:04:04.307 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:03 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:03.816+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.600331+0000 front 2024-05-09T05:03:33.600302+0000 (oldest deadline 2024-05-09T05:03:55.800722+0000) 2024-05-09T05:04:04.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:04.057+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:04:04.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:04.057+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:04:04.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:04.057+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:04:04.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:04.057+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:04:04.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:04 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:04.057+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:04:04.793 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:04 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:04.320+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:04.793 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:04 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:04.320+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:04.793 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:04 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:04.320+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:04.793 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:04 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:04.320+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:04.793 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:04 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:04.320+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:05.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:04 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:04.793+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:04:05.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:04 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:04.793+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:04:05.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:04 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:04.793+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:04:05.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:04 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:04.793+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.600331+0000 front 2024-05-09T05:03:33.600302+0000 (oldest deadline 2024-05-09T05:03:55.800722+0000) 2024-05-09T05:04:05.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:04 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:04.776+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:05.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:04 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:04.776+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:05.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:04 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:04.776+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:05.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:04 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:04.776+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:05.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:04 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:04.776+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:05.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:04 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:04.776+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:05.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:05 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:05.305+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:05.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:05 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:05.305+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:05.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:05 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:05.305+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:05.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:05 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:05.305+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:05.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:05 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:05.305+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:05.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:05.100+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:04:05.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:05.100+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:04:05.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:05.100+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:04:05.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:05.100+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:04:05.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:05.100+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:04:05.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:05 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:05.100+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.707206+0000 front 2024-05-09T05:03:39.707261+0000 (oldest deadline 2024-05-09T05:04:05.007169+0000) 2024-05-09T05:04:06.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:05 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:05.733+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:06.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:05 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:05.733+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:06.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:05 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:05.733+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:06.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:05 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:05.733+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:06.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:05 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:05.733+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:06.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:05 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:05.733+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:06.295 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:05 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:05.834+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.195331+0000 front 2024-05-09T05:03:09.195451+0000 (oldest deadline 2024-05-09T05:03:30.895325+0000) 2024-05-09T05:04:06.295 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:05 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:05.834+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:15.596441+0000 front 2024-05-09T05:03:15.596522+0000 (oldest deadline 2024-05-09T05:03:39.096634+0000) 2024-05-09T05:04:06.296 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:05 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:05.834+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:24.898614+0000 front 2024-05-09T05:03:24.898785+0000 (oldest deadline 2024-05-09T05:03:47.198546+0000) 2024-05-09T05:04:06.296 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:05 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8[53313]: 2024-05-09T05:04:05.834+0000 7f9d1f0b4700 -1 osd.8 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.600331+0000 front 2024-05-09T05:03:33.600302+0000 (oldest deadline 2024-05-09T05:03:55.800722+0000) 2024-05-09T05:04:06.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:06 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:06.295+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:06.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:06 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:06.295+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:06.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:06 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:06.295+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:06.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:06 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:06.295+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:06.558 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:06 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:06.295+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:06.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:06 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:06.124+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:04:06.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:06 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:06.124+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:04:06.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:06 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:06.124+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:04:06.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:06 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:06.124+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:04:06.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:06 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:06.124+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:04:06.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:06 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:06.124+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.707206+0000 front 2024-05-09T05:03:39.707261+0000 (oldest deadline 2024-05-09T05:04:05.007169+0000) 2024-05-09T05:04:07.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:06 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:06.758+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:07.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:06 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:06.758+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:07.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:06 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:06.758+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:07.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:06 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:06.758+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:07.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:06 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:06.758+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:07.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:06 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:06.758+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:07.315 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:07 smithi110 podman[72534]: 2024-05-09 05:04:07.06073763 +0000 UTC m=+5.507617671 container died 490af2b2aeb1a5dcfb104fa136a99a3d73d740fc2d3def8f8b911e9aad948b96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:04:07.581 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:07 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:07.133+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:04:07.581 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:07 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:07.133+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:04:07.582 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:07 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:07.133+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:04:07.582 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:07 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:07.133+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:04:07.582 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:07 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:07.133+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:04:07.582 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:07 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:07.133+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.707206+0000 front 2024-05-09T05:03:39.707261+0000 (oldest deadline 2024-05-09T05:04:05.007169+0000) 2024-05-09T05:04:07.745 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:07 smithi110 podman[72534]: 2024-05-09 05:04:07.469364319 +0000 UTC m=+5.916244349 container cleanup 490af2b2aeb1a5dcfb104fa136a99a3d73d740fc2d3def8f8b911e9aad948b96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-09T05:04:07.746 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:07 smithi110 bash[72534]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8 2024-05-09T05:04:07.746 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:07 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:07.315+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:07.746 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:07 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:07.315+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:07.746 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:07 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:07.315+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:07.746 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:07 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:07.315+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:07.746 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:07 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:07.315+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:08.057 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:07 smithi110 podman[72551]: 2024-05-09 05:04:07.745802751 +0000 UTC m=+0.679609394 container remove 490af2b2aeb1a5dcfb104fa136a99a3d73d740fc2d3def8f8b911e9aad948b96 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-09T05:04:08.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:07 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:07.713+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:08.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:07 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:07.713+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:08.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:07 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:07.713+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:08.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:07 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:07.713+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:08.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:07 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:07.713+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:08.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:07 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:07.713+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:08.424 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:08 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:08.307+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:08.424 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:08 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:08.307+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:08.424 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:08 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:08.307+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:08.424 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:08 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:08.307+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:08.424 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:08 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:08.307+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:08.424 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:08 smithi110 podman[72645]: 2024-05-09 05:04:07.988579935 +0000 UTC m=+0.018913070 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:04:08.424 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:08 smithi110 podman[72645]: 2024-05-09 05:04:08.105162949 +0000 UTC m=+0.135496057 container create 4a7d0da6addf39b355c542da72ae105edce75f34ad44e26e17d6a535afe783af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS) 2024-05-09T05:04:08.424 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:08 smithi110 podman[72645]: 2024-05-09 05:04:08.324247747 +0000 UTC m=+0.354580854 container init 4a7d0da6addf39b355c542da72ae105edce75f34ad44e26e17d6a535afe783af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507) 2024-05-09T05:04:08.424 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:08 smithi110 podman[72645]: 2024-05-09 05:04:08.327253693 +0000 UTC m=+0.357586804 container start 4a7d0da6addf39b355c542da72ae105edce75f34ad44e26e17d6a535afe783af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-05-09T05:04:08.425 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:08 smithi110 podman[72645]: 2024-05-09 05:04:08.424356455 +0000 UTC m=+0.454689581 container attach 4a7d0da6addf39b355c542da72ae105edce75f34ad44e26e17d6a535afe783af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-09T05:04:08.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:08 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:08.108+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:04:08.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:08 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:08.108+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:04:08.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:08 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:08.108+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:04:08.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:08 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:08.108+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:04:08.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:08 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:08.108+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:04:08.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:08 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:08.108+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.707206+0000 front 2024-05-09T05:03:39.707261+0000 (oldest deadline 2024-05-09T05:04:05.007169+0000) 2024-05-09T05:04:08.807 INFO:journalctl@ceph.osd.8.smithi110.stdout:May 09 05:04:08 smithi110 podman[72645]: 2024-05-09 05:04:08.626421291 +0000 UTC m=+0.656754409 container died 4a7d0da6addf39b355c542da72ae105edce75f34ad44e26e17d6a535afe783af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:04:09.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:08 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:08.672+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:09.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:08 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:08.672+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:09.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:08 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:08.672+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:09.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:08 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:08.672+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:09.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:08 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:08.672+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:09.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:08 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:08.672+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:09.258 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.8.service' 2024-05-09T05:04:09.298 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:04:09.298 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-09T05:04:09.298 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-09T05:04:09.298 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.9 2024-05-09T05:04:09.327 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:09 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:09.074+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:04:09.327 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:09 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:09.074+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:04:09.328 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:09 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:09.074+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:04:09.328 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:09 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:09.074+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:04:09.328 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:09 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:09.074+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:04:09.328 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:09 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:09.074+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.707206+0000 front 2024-05-09T05:03:39.707261+0000 (oldest deadline 2024-05-09T05:04:05.007169+0000) 2024-05-09T05:04:09.510 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:09 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:09.353+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:09.510 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:09 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:09.353+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:09.510 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:09 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:09.353+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:09.510 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:09 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:09.353+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:09.510 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:09 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:09.353+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:09.580 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:09 smithi046 systemd[1]: Stopping Ceph osd.9 for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:04:10.050 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:09 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:09.622+0000 7fa11bbca700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:04:10.050 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:09 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:09.622+0000 7fa11bbca700 -1 osd.9 98 *** Got signal Terminated *** 2024-05-09T05:04:10.051 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:09 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:09.622+0000 7fa11bbca700 -1 osd.9 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:04:10.051 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:10 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:10.050+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:04:10.051 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:10 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:10.050+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:04:10.051 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:10 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:10.050+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:04:10.051 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:10 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:10.050+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:04:10.051 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:10 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:10.050+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:04:10.051 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:10 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:10.050+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.707206+0000 front 2024-05-09T05:03:39.707261+0000 (oldest deadline 2024-05-09T05:04:05.007169+0000) 2024-05-09T05:04:10.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:09 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:09.634+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:10.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:09 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:09.634+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:10.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:09 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:09.634+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:10.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:09 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:09.634+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:10.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:09 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:09.634+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:10.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:09 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:09.634+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:10.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:10 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:10.356+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:10.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:10 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:10.356+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:10.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:10 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:10.356+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:10.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:10 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:10.356+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:10.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:10 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:10.356+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:10.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:10 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:10.594+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:10.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:10 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:10.594+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:10.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:10 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:10.594+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:10.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:10 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:10.594+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:10.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:10 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:10.594+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:10.881 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:10 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:10.594+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:11.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:11 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:11.013+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:04:11.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:11 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:11.013+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:04:11.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:11 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:11.013+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:04:11.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:11 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:11.013+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:04:11.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:11 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:11.013+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:04:11.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:11 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:11.013+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.707206+0000 front 2024-05-09T05:03:39.707261+0000 (oldest deadline 2024-05-09T05:04:05.007169+0000) 2024-05-09T05:04:11.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:11 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:11.372+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:11.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:11 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:11.372+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:11.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:11 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:11.372+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:11.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:11 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:11.372+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:11.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:11 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:11.372+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:11.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:11 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:11.624+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:11.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:11 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:11.624+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:11.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:11 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:11.624+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:11.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:11 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:11.624+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:11.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:11 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:11.624+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:11.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:11 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:11.624+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:12.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:12 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:12.015+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:04:12.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:12 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:12.015+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:04:12.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:12 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:12.015+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:04:12.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:12 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:12.015+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:04:12.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:12 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:12.015+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:04:12.295 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:12 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:12.015+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.707206+0000 front 2024-05-09T05:03:39.707261+0000 (oldest deadline 2024-05-09T05:04:05.007169+0000) 2024-05-09T05:04:12.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:12 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:12.377+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:12.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:12 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:12.377+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:12.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:12 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:12.377+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:12.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:12 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:12.377+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:12.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:12 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:12.377+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:12.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:12 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:12.589+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:12.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:12 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:12.589+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:12.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:12 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:12.589+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:12.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:12 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:12.589+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:12.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:12 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:12.589+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:12.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:12 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:12.589+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:13.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:12 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:12.972+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:04:13.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:12 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:12.972+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:04:13.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:12 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:12.972+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:04:13.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:12 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:12.972+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:04:13.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:12 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:12.972+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:04:13.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:12 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:12.972+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.707206+0000 front 2024-05-09T05:03:39.707261+0000 (oldest deadline 2024-05-09T05:04:05.007169+0000) 2024-05-09T05:04:13.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:13 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:13.382+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:13.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:13 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:13.382+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:13.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:13 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:13.382+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:13.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:13 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:13.382+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:13.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:13 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:13.382+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:13.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:13 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:13.602+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:13.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:13 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:13.602+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:13.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:13 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:13.602+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:13.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:13 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:13.602+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:13.880 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:13 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:13.602+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:13.881 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:13 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:13.602+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:14.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:13 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:13.943+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.100909+0000 front 2024-05-09T05:03:01.101083+0000 (oldest deadline 2024-05-09T05:03:27.001094+0000) 2024-05-09T05:04:14.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:13 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:13.943+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:07.502607+0000 front 2024-05-09T05:03:07.502589+0000 (oldest deadline 2024-05-09T05:03:30.402407+0000) 2024-05-09T05:04:14.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:13 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:13.943+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.804477+0000 front 2024-05-09T05:03:16.804497+0000 (oldest deadline 2024-05-09T05:03:42.104265+0000) 2024-05-09T05:04:14.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:13 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:13.943+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:22.104820+0000 front 2024-05-09T05:03:22.104775+0000 (oldest deadline 2024-05-09T05:03:46.804836+0000) 2024-05-09T05:04:14.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:13 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:13.943+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.506011+0000 front 2024-05-09T05:03:31.506133+0000 (oldest deadline 2024-05-09T05:03:56.205929+0000) 2024-05-09T05:04:14.331 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:13 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9[65668]: 2024-05-09T05:04:13.943+0000 7fa118180700 -1 osd.9 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.707206+0000 front 2024-05-09T05:03:39.707261+0000 (oldest deadline 2024-05-09T05:04:05.007169+0000) 2024-05-09T05:04:14.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:14 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:14.374+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:14.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:14 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:14.374+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:14.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:14 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:14.374+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:14.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:14 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:14.374+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:14.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:14 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:14.374+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:14.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:14 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:14.374+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:15.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:14 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:14.641+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:15.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:14 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:14.641+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:15.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:14 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:14.641+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:15.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:14 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:14.641+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:15.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:14 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:14.641+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:15.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:14 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:14.641+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:15.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:14 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:14.641+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.238138+0000 front 2024-05-09T05:03:49.238179+0000 (oldest deadline 2024-05-09T05:04:13.938017+0000) 2024-05-09T05:04:15.512 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:15 smithi046 podman[118506]: 2024-05-09 05:04:15.104311972 +0000 UTC m=+5.604073872 container died 04f7fd9bf78f4ec7feab2328b0beafb0de6d1a7e5c9d4b1e30b488d44f82fb89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:04:15.513 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:15 smithi046 podman[118506]: 2024-05-09 05:04:15.513019425 +0000 UTC m=+6.012781323 container cleanup 04f7fd9bf78f4ec7feab2328b0beafb0de6d1a7e5c9d4b1e30b488d44f82fb89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:04:15.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:15 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:15.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:15.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:15 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:15.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:15.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:15 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:15.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:15.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:15 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:15.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:15.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:15 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:15.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:15.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:15 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:15.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:15.830 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:15 smithi046 bash[118506]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9 2024-05-09T05:04:15.830 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:15 smithi046 podman[118523]: 2024-05-09 05:04:15.730984996 +0000 UTC m=+0.619454277 container remove 04f7fd9bf78f4ec7feab2328b0beafb0de6d1a7e5c9d4b1e30b488d44f82fb89 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-09T05:04:16.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:15 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:15.683+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:16.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:15 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:15.683+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:16.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:15 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:15.683+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:16.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:15 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:15.683+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:16.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:15 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:15.683+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:16.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:15 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:15.683+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:16.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:15 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:15.683+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.238138+0000 front 2024-05-09T05:03:49.238179+0000 (oldest deadline 2024-05-09T05:04:13.938017+0000) 2024-05-09T05:04:16.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:16 smithi046 podman[118612]: 2024-05-09 05:04:15.951057941 +0000 UTC m=+0.018629904 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:04:16.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:16 smithi046 podman[118612]: 2024-05-09 05:04:16.048296778 +0000 UTC m=+0.115868783 container create ce2b8706c2c0a6b0139a5fb8711312b5e4cf8caaa95c4bbce675a6dffad8131a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-09T05:04:16.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:16 smithi046 podman[118612]: 2024-05-09 05:04:16.267731794 +0000 UTC m=+0.335303760 container init ce2b8706c2c0a6b0139a5fb8711312b5e4cf8caaa95c4bbce675a6dffad8131a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-09T05:04:16.330 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:16 smithi046 podman[118612]: 2024-05-09 05:04:16.271638238 +0000 UTC m=+0.339210209 container start ce2b8706c2c0a6b0139a5fb8711312b5e4cf8caaa95c4bbce675a6dffad8131a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507) 2024-05-09T05:04:16.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:16 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:16.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:16.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:16 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:16.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:16.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:16 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:16.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:16.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:16 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:16.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:16.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:16 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:16.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:16.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:16 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:16.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:16.830 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:16 smithi046 podman[118612]: 2024-05-09 05:04:16.475342138 +0000 UTC m=+0.542914102 container attach ce2b8706c2c0a6b0139a5fb8711312b5e4cf8caaa95c4bbce675a6dffad8131a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5) 2024-05-09T05:04:16.830 INFO:journalctl@ceph.osd.9.smithi046.stdout:May 09 05:04:16 smithi046 podman[118612]: 2024-05-09 05:04:16.581354806 +0000 UTC m=+0.648926775 container died ce2b8706c2c0a6b0139a5fb8711312b5e4cf8caaa95c4bbce675a6dffad8131a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-05-09T05:04:17.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:16 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:16.686+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:17.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:16 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:16.686+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:17.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:16 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:16.686+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:17.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:16 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:16.686+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:17.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:16 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:16.686+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:17.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:16 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:16.686+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:17.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:16 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:16.686+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.238138+0000 front 2024-05-09T05:03:49.238179+0000 (oldest deadline 2024-05-09T05:04:13.938017+0000) 2024-05-09T05:04:17.280 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.9.service' 2024-05-09T05:04:17.362 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:04:17.363 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-09T05:04:17.363 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-09T05:04:17.363 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.10 2024-05-09T05:04:17.677 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:17 smithi072 systemd[1]: Stopping Ceph osd.10 for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:04:17.677 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:17 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:17.676+0000 7f189d992700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:04:17.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:17 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:17.366+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:17.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:17 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:17.366+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:17.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:17 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:17.366+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:17.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:17 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:17.366+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:17.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:17 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:17.366+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:17.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:17 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:17.366+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:18.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:17 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:17.676+0000 7f189d992700 -1 osd.10 98 *** Got signal Terminated *** 2024-05-09T05:04:18.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:17 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:17.676+0000 7f189d992700 -1 osd.10 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:04:18.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:17 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:17.690+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:18.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:17 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:17.690+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:18.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:17 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:17.690+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:18.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:17 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:17.690+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:18.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:17 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:17.690+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:18.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:17 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:17.690+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:18.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:17 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:17.690+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.238138+0000 front 2024-05-09T05:03:49.238179+0000 (oldest deadline 2024-05-09T05:04:13.938017+0000) 2024-05-09T05:04:18.708 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:18 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:18.389+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:18.709 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:18 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:18.389+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:18.709 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:18 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:18.389+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:18.709 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:18 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:18.389+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:18.709 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:18 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:18.389+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:18.709 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:18 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:18.389+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:19.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:18 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:18.672+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:19.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:18 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:18.672+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:19.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:18 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:18.672+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:19.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:18 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:18.672+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:19.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:18 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:18.672+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:19.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:18 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:18.672+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:19.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:18 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:18.672+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.238138+0000 front 2024-05-09T05:03:49.238179+0000 (oldest deadline 2024-05-09T05:04:13.938017+0000) 2024-05-09T05:04:19.720 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:19 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:19.361+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:19.720 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:19 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:19.361+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:19.720 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:19 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:19.361+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:19.720 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:19 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:19.361+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:19.721 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:19 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:19.361+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:19.721 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:19 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:19.361+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:20.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:19 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:19.657+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:20.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:19 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:19.657+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:20.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:19 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:19.657+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:20.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:19 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:19.657+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:20.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:19 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:19.657+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:20.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:19 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:19.657+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:20.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:19 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:19.657+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.238138+0000 front 2024-05-09T05:03:49.238179+0000 (oldest deadline 2024-05-09T05:04:13.938017+0000) 2024-05-09T05:04:20.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:20 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:20.395+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:20.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:20 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:20.395+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:20.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:20 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:20.395+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:20.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:20 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:20.395+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:20.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:20 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:20.395+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:20.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:20 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:20.395+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:20.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:20 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:20.395+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-09T05:03:54.077539+0000 front 2024-05-09T05:03:54.077561+0000 (oldest deadline 2024-05-09T05:04:19.977568+0000) 2024-05-09T05:04:21.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:20 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:20.681+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:21.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:20 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:20.681+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:21.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:20 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:20.681+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:21.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:20 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:20.681+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:21.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:20 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:20.681+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:21.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:20 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:20.681+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:21.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:20 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:20.681+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.238138+0000 front 2024-05-09T05:03:49.238179+0000 (oldest deadline 2024-05-09T05:04:13.938017+0000) 2024-05-09T05:04:21.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:21 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:21.348+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:21.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:21 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:21.348+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:21.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:21 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:21.348+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:21.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:21 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:21.348+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:21.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:21 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:21.348+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:21.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:21 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:21.348+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:21.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:21 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:21.348+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-09T05:03:54.077539+0000 front 2024-05-09T05:03:54.077561+0000 (oldest deadline 2024-05-09T05:04:19.977568+0000) 2024-05-09T05:04:22.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:21 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:21.726+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:01.028630+0000 front 2024-05-09T05:03:01.028786+0000 (oldest deadline 2024-05-09T05:03:23.928812+0000) 2024-05-09T05:04:22.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:21 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:21.726+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:08.630238+0000 front 2024-05-09T05:03:08.630194+0000 (oldest deadline 2024-05-09T05:03:31.530344+0000) 2024-05-09T05:04:22.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:21 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:21.726+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:16.732215+0000 front 2024-05-09T05:03:16.732121+0000 (oldest deadline 2024-05-09T05:03:40.832161+0000) 2024-05-09T05:04:22.130 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:21 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:21.726+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6814 osd.3 since back 2024-05-09T05:03:23.133459+0000 front 2024-05-09T05:03:23.133311+0000 (oldest deadline 2024-05-09T05:03:49.533664+0000) 2024-05-09T05:04:22.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:21 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:21.726+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:31.834698+0000 front 2024-05-09T05:03:31.834754+0000 (oldest deadline 2024-05-09T05:03:55.334852+0000) 2024-05-09T05:04:22.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:21 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:21.726+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:41.736796+0000 front 2024-05-09T05:03:41.736685+0000 (oldest deadline 2024-05-09T05:04:03.436475+0000) 2024-05-09T05:04:22.131 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:21 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10[58489]: 2024-05-09T05:04:21.726+0000 7f1899f48700 -1 osd.10 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.238138+0000 front 2024-05-09T05:03:49.238179+0000 (oldest deadline 2024-05-09T05:04:13.938017+0000) 2024-05-09T05:04:22.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:22 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:22.394+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:22.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:22 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:22.394+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:22.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:22 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:22.394+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:22.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:22 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:22.394+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:22.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:22 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:22.394+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:22.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:22 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:22.394+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:22.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:22 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:22.394+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-09T05:03:54.077539+0000 front 2024-05-09T05:03:54.077561+0000 (oldest deadline 2024-05-09T05:04:19.977568+0000) 2024-05-09T05:04:23.289 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:23 smithi072 podman[71049]: 2024-05-09 05:04:23.007892438 +0000 UTC m=+5.447079815 container died b6a877dba76d180c93f9965909a2269b9a17acecd45bfcb130c8bfc4697af4ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-09T05:04:23.630 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:23 smithi072 podman[71049]: 2024-05-09 05:04:23.397819897 +0000 UTC m=+5.837007274 container cleanup b6a877dba76d180c93f9965909a2269b9a17acecd45bfcb130c8bfc4697af4ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-09T05:04:23.630 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:23 smithi072 bash[71049]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10 2024-05-09T05:04:23.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:23 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:23.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:23.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:23 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:23.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:23.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:23 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:23.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:23.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:23 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:23.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:23.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:23 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:23.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:23.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:23 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:23.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:23.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:23 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:23.365+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-09T05:03:54.077539+0000 front 2024-05-09T05:03:54.077561+0000 (oldest deadline 2024-05-09T05:04:19.977568+0000) 2024-05-09T05:04:24.009 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:23 smithi072 podman[71068]: 2024-05-09 05:04:23.658970319 +0000 UTC m=+0.647336044 container remove b6a877dba76d180c93f9965909a2269b9a17acecd45bfcb130c8bfc4697af4ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True) 2024-05-09T05:04:24.316 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:24 smithi072 podman[71167]: 2024-05-09 05:04:23.914404536 +0000 UTC m=+0.020158559 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:04:24.316 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:24 smithi072 podman[71167]: 2024-05-09 05:04:24.018971859 +0000 UTC m=+0.124725882 container create 1c1250b9983f7ccf67ec425734553e1c13a94faaa5705b55ec0729c2374d8d23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-09T05:04:24.316 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:24 smithi072 podman[71167]: 2024-05-09 05:04:24.224869253 +0000 UTC m=+0.330623278 container init 1c1250b9983f7ccf67ec425734553e1c13a94faaa5705b55ec0729c2374d8d23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T05:04:24.316 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:24 smithi072 podman[71167]: 2024-05-09 05:04:24.228242134 +0000 UTC m=+0.333996150 container start 1c1250b9983f7ccf67ec425734553e1c13a94faaa5705b55ec0729c2374d8d23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:04:24.317 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:24 smithi072 podman[71167]: 2024-05-09 05:04:24.316586389 +0000 UTC m=+0.422340407 container attach 1c1250b9983f7ccf67ec425734553e1c13a94faaa5705b55ec0729c2374d8d23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:04:24.630 INFO:journalctl@ceph.osd.10.smithi072.stdout:May 09 05:04:24 smithi072 podman[71196]: 2024-05-09 05:04:24.554800174 +0000 UTC m=+0.019228129 container died 1c1250b9983f7ccf67ec425734553e1c13a94faaa5705b55ec0729c2374d8d23 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-10-deactivate, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-09T05:04:24.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:24 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:24.410+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:24.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:24 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:24.410+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:24.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:24 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:24.410+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:24.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:24 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:24.410+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:24.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:24 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:24.410+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:24.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:24 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:24.410+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:24.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:24 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:24.410+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-09T05:03:54.077539+0000 front 2024-05-09T05:03:54.077561+0000 (oldest deadline 2024-05-09T05:04:19.977568+0000) 2024-05-09T05:04:25.087 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.10.service' 2024-05-09T05:04:25.125 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:04:25.125 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-09T05:04:25.125 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-09T05:04:25.125 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.11 2024-05-09T05:04:25.423 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:25 smithi110 systemd[1]: Stopping Ceph osd.11 for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:04:25.423 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:25 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:25.390+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:25.423 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:25 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:25.390+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:25.423 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:25 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:25.390+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:25.423 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:25 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:25.390+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:25.423 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:25 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:25.390+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:25.423 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:25 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:25.390+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:25.423 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:25 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:25.390+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-09T05:03:54.077539+0000 front 2024-05-09T05:03:54.077561+0000 (oldest deadline 2024-05-09T05:04:19.977568+0000) 2024-05-09T05:04:25.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:25 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:25.422+0000 7f28ff98e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:04:25.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:25 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:25.422+0000 7f28ff98e700 -1 osd.11 98 *** Got signal Terminated *** 2024-05-09T05:04:25.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:25 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:25.422+0000 7f28ff98e700 -1 osd.11 98 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-09T05:04:26.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:26 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:26.418+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:26.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:26 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:26.418+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:26.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:26 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:26.418+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:26.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:26 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:26.418+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:26.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:26 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:26.418+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:26.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:26 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:26.418+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:26.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:26 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:26.418+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-09T05:03:54.077539+0000 front 2024-05-09T05:03:54.077561+0000 (oldest deadline 2024-05-09T05:04:19.977568+0000) 2024-05-09T05:04:27.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:27 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:27.409+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:27.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:27 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:27.409+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:27.807 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:27 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:27.409+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:27.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:27 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:27.409+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:27.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:27 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:27.409+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:27.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:27 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:27.409+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:27.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:27 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:27.409+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-09T05:03:54.077539+0000 front 2024-05-09T05:03:54.077561+0000 (oldest deadline 2024-05-09T05:04:19.977568+0000) 2024-05-09T05:04:28.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:28 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:28.400+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:28.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:28 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:28.400+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:28.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:28 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:28.400+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:28.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:28 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:28.400+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:28.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:28 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:28.400+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:28.808 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:28 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:28.400+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:28.809 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:28 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:28.400+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-09T05:03:54.077539+0000 front 2024-05-09T05:03:54.077561+0000 (oldest deadline 2024-05-09T05:04:19.977568+0000) 2024-05-09T05:04:29.762 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:29 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:29.426+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6806 osd.0 since back 2024-05-09T05:03:00.369788+0000 front 2024-05-09T05:03:00.369587+0000 (oldest deadline 2024-05-09T05:03:23.869781+0000) 2024-05-09T05:04:29.762 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:29 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:29.426+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6804 osd.1 since back 2024-05-09T05:03:09.572060+0000 front 2024-05-09T05:03:09.571962+0000 (oldest deadline 2024-05-09T05:03:34.272018+0000) 2024-05-09T05:04:29.762 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:29 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:29.426+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6804 osd.2 since back 2024-05-09T05:03:14.272442+0000 front 2024-05-09T05:03:14.272476+0000 (oldest deadline 2024-05-09T05:03:39.572586+0000) 2024-05-09T05:04:29.762 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:29 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:29.426+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6812 osd.4 since back 2024-05-09T05:03:33.674485+0000 front 2024-05-09T05:03:33.674371+0000 (oldest deadline 2024-05-09T05:03:56.574531+0000) 2024-05-09T05:04:29.762 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:29 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:29.426+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6812 osd.5 since back 2024-05-09T05:03:39.475614+0000 front 2024-05-09T05:03:39.475707+0000 (oldest deadline 2024-05-09T05:04:04.175643+0000) 2024-05-09T05:04:29.762 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:29 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:29.426+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.46:6822 osd.6 since back 2024-05-09T05:03:49.977359+0000 front 2024-05-09T05:03:49.977419+0000 (oldest deadline 2024-05-09T05:04:14.077092+0000) 2024-05-09T05:04:29.762 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:29 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:29.426+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.72:6820 osd.7 since back 2024-05-09T05:03:54.077539+0000 front 2024-05-09T05:03:54.077561+0000 (oldest deadline 2024-05-09T05:04:19.977568+0000) 2024-05-09T05:04:29.762 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:29 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11[59924]: 2024-05-09T05:04:29.426+0000 7f28f7f3c700 -1 osd.11 98 heartbeat_check: no reply from 172.21.15.110:6820 osd.8 since back 2024-05-09T05:04:05.278302+0000 front 2024-05-09T05:04:05.278336+0000 (oldest deadline 2024-05-09T05:04:28.778338+0000) 2024-05-09T05:04:31.181 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:30 smithi110 podman[72762]: 2024-05-09 05:04:30.919492735 +0000 UTC m=+5.603177348 container died e32e5e7e98f4e0c575f4c76d93ba976e618aba0201358c9b2e801a687e9b653c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-09T05:04:31.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:31 smithi110 podman[72762]: 2024-05-09 05:04:31.263942041 +0000 UTC m=+5.947626664 container cleanup e32e5e7e98f4e0c575f4c76d93ba976e618aba0201358c9b2e801a687e9b653c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-09T05:04:31.557 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:31 smithi110 bash[72762]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11 2024-05-09T05:04:31.929 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:31 smithi110 podman[72778]: 2024-05-09 05:04:31.590460281 +0000 UTC m=+0.672620173 container remove e32e5e7e98f4e0c575f4c76d93ba976e618aba0201358c9b2e801a687e9b653c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD) 2024-05-09T05:04:32.179 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:31 smithi110 podman[72867]: 2024-05-09 05:04:31.832671194 +0000 UTC m=+0.018182083 image pull db794d706f492c144ccc8f2c5bbc1870e116f5c4f3309dfab4416b39d099f8d7 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584 2024-05-09T05:04:32.180 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:31 smithi110 podman[72867]: 2024-05-09 05:04:31.941307845 +0000 UTC m=+0.126818721 container create b52f00ca4ccf8b76084075b3a5fc6c862df9e88986ad481643096755693aca36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11-deactivate, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-09T05:04:32.180 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:32 smithi110 podman[72867]: 2024-05-09 05:04:32.176992259 +0000 UTC m=+0.362503135 container init b52f00ca4ccf8b76084075b3a5fc6c862df9e88986ad481643096755693aca36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:04:32.180 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:32 smithi110 podman[72867]: 2024-05-09 05:04:32.180123756 +0000 UTC m=+0.365634635 container start b52f00ca4ccf8b76084075b3a5fc6c862df9e88986ad481643096755693aca36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-05-09T05:04:32.465 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:32 smithi110 podman[72867]: 2024-05-09 05:04:32.285261856 +0000 UTC m=+0.470772739 container attach b52f00ca4ccf8b76084075b3a5fc6c862df9e88986ad481643096755693aca36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:04:32.754 INFO:journalctl@ceph.osd.11.smithi110.stdout:May 09 05:04:32 smithi110 podman[72867]: 2024-05-09 05:04:32.466174011 +0000 UTC m=+0.651684886 container died b52f00ca4ccf8b76084075b3a5fc6c862df9e88986ad481643096755693aca36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:04:33.028 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@osd.11.service' 2024-05-09T05:04:33.066 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:04:33.066 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-09T05:04:33.066 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-09T05:04:33.066 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.a 2024-05-09T05:04:33.481 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 05:04:33 smithi046 systemd[1]: Stopping Ceph mds.a for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:04:33.760 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 05:04:33 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-a[72682]: 2024-05-09T05:04:33.480+0000 7f2ddac82700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:04:33.760 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 05:04:33 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-a[72682]: 2024-05-09T05:04:33.480+0000 7f2ddac82700 -1 mds.a *** got signal Terminated *** 2024-05-09T05:04:37.330 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 05:04:37 smithi046 podman[118718]: 2024-05-09 05:04:37.073057013 +0000 UTC m=+3.807716055 container died 883e51440f9174047758cf9c37997128ef3db67c28c3b8a463fc23e7f0aacf02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-a, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-09T05:04:37.830 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 05:04:37 smithi046 podman[118718]: 2024-05-09 05:04:37.630376152 +0000 UTC m=+4.365035199 container cleanup 883e51440f9174047758cf9c37997128ef3db67c28c3b8a463fc23e7f0aacf02 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:04:37.830 INFO:journalctl@ceph.mds.a.smithi046.stdout:May 09 05:04:37 smithi046 bash[118718]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-a 2024-05-09T05:04:38.069 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.a.service' 2024-05-09T05:04:38.106 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:04:38.107 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-09T05:04:38.107 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-09T05:04:38.107 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.d 2024-05-09T05:04:38.451 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 05:04:38 smithi046 systemd[1]: Stopping Ceph mds.d for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:04:38.830 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 05:04:38 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-d[71387]: 2024-05-09T05:04:38.450+0000 7ff4e3711700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:04:38.830 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 05:04:38 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-d[71387]: 2024-05-09T05:04:38.450+0000 7ff4e3711700 -1 mds.d *** got signal Terminated *** 2024-05-09T05:04:42.830 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 05:04:42 smithi046 podman[118851]: 2024-05-09 05:04:42.466959426 +0000 UTC m=+4.123621901 container died eeecbdc5532bfe295b8ece6481dd3af60589bc2b70025f8f29668cdcae0aa8f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-d, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-09T05:04:43.228 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 05:04:42 smithi046 podman[118851]: 2024-05-09 05:04:42.867332554 +0000 UTC m=+4.523995012 container cleanup eeecbdc5532bfe295b8ece6481dd3af60589bc2b70025f8f29668cdcae0aa8f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-d, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:04:43.229 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 05:04:42 smithi046 bash[118851]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-d 2024-05-09T05:04:43.229 INFO:journalctl@ceph.mds.d.smithi046.stdout:May 09 05:04:43 smithi046 podman[118865]: 2024-05-09 05:04:43.101616185 +0000 UTC m=+0.632468768 container remove eeecbdc5532bfe295b8ece6481dd3af60589bc2b70025f8f29668cdcae0aa8f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-d, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5) 2024-05-09T05:04:43.251 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.d.service' 2024-05-09T05:04:43.286 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:04:43.286 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-09T05:04:43.286 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-09T05:04:43.287 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.g 2024-05-09T05:04:43.580 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 05:04:43 smithi046 systemd[1]: Stopping Ceph mds.g for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:04:44.080 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 05:04:43 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-g[72239]: 2024-05-09T05:04:43.601+0000 7f1062a7a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:04:44.080 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 05:04:43 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-g[72239]: 2024-05-09T05:04:43.601+0000 7f1062a7a700 -1 mds.g *** got signal Terminated *** 2024-05-09T05:04:45.080 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 05:04:44 smithi046 podman[118973]: 2024-05-09 05:04:44.994961105 +0000 UTC m=+1.485935852 container died 89c1baac619925ee579facfd139178ebbe669ae9827ec0d144a4a833c97a6f15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-g, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-09T05:04:45.580 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 05:04:45 smithi046 podman[118973]: 2024-05-09 05:04:45.354243825 +0000 UTC m=+1.845218570 container cleanup 89c1baac619925ee579facfd139178ebbe669ae9827ec0d144a4a833c97a6f15 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-g, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2) 2024-05-09T05:04:45.580 INFO:journalctl@ceph.mds.g.smithi046.stdout:May 09 05:04:45 smithi046 bash[118973]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-g 2024-05-09T05:04:45.793 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.g.service' 2024-05-09T05:04:45.828 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:04:45.828 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-09T05:04:45.828 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-09T05:04:45.828 DEBUG:teuthology.orchestra.run.smithi046:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.j 2024-05-09T05:04:46.142 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 05:04:45 smithi046 systemd[1]: Stopping Ceph mds.j for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:04:46.580 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 05:04:46 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-j[70963]: 2024-05-09T05:04:46.141+0000 7f77fa901700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:04:46.580 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 05:04:46 smithi046 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-j[70963]: 2024-05-09T05:04:46.141+0000 7f77fa901700 -1 mds.j *** got signal Terminated *** 2024-05-09T05:04:50.533 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 05:04:50 smithi046 podman[119094]: 2024-05-09 05:04:50.155794295 +0000 UTC m=+4.107933432 container died 03a8ee9e792ca3ffec1c2215cef6a618db7e46383d918a32d842592079a8f5b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:04:50.830 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 05:04:50 smithi046 podman[119094]: 2024-05-09 05:04:50.615944859 +0000 UTC m=+4.568083995 container cleanup 03a8ee9e792ca3ffec1c2215cef6a618db7e46383d918a32d842592079a8f5b9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-j, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD) 2024-05-09T05:04:50.830 INFO:journalctl@ceph.mds.j.smithi046.stdout:May 09 05:04:50 smithi046 bash[119094]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-j 2024-05-09T05:04:51.045 DEBUG:teuthology.orchestra.run.smithi046:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.j.service' 2024-05-09T05:04:51.079 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:04:51.079 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-09T05:04:51.079 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-09T05:04:51.079 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.b 2024-05-09T05:04:51.380 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 05:04:51 smithi072 systemd[1]: Stopping Ceph mds.b for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:04:51.880 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 05:04:51 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-b[63750]: 2024-05-09T05:04:51.402+0000 7f790f1ec700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:04:51.880 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 05:04:51 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-b[63750]: 2024-05-09T05:04:51.402+0000 7f790f1ec700 -1 mds.b *** got signal Terminated *** 2024-05-09T05:04:54.522 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 05:04:54 smithi072 podman[71274]: 2024-05-09 05:04:54.261230743 +0000 UTC m=+2.988169795 container died b35740bd40ea853d6842dde24de84482982de9133964f3c9520b48194ed60ee5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-b, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-09T05:04:54.880 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 05:04:54 smithi072 podman[71274]: 2024-05-09 05:04:54.630999788 +0000 UTC m=+3.357938840 container cleanup b35740bd40ea853d6842dde24de84482982de9133964f3c9520b48194ed60ee5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-b, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:04:54.880 INFO:journalctl@ceph.mds.b.smithi072.stdout:May 09 05:04:54 smithi072 bash[71274]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-b 2024-05-09T05:04:55.045 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.b.service' 2024-05-09T05:04:55.078 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:04:55.078 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-09T05:04:55.079 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-09T05:04:55.079 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.e 2024-05-09T05:04:55.630 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 05:04:55 smithi072 systemd[1]: Stopping Ceph mds.e for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:04:55.630 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 05:04:55 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-e[63126]: 2024-05-09T05:04:55.385+0000 7f4c1ad71700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:04:55.630 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 05:04:55 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-e[63126]: 2024-05-09T05:04:55.385+0000 7f4c1ad71700 -1 mds.e *** got signal Terminated *** 2024-05-09T05:04:58.557 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 05:04:58 smithi072 podman[71392]: 2024-05-09 05:04:58.263971205 +0000 UTC m=+2.974544661 container died 169e3dba0d2916e5797d495978f5001611bb599bbb3e201155c2272ef7300ee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-09T05:04:58.880 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 05:04:58 smithi072 podman[71392]: 2024-05-09 05:04:58.648681063 +0000 UTC m=+3.359254508 container cleanup 169e3dba0d2916e5797d495978f5001611bb599bbb3e201155c2272ef7300ee7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-05-09T05:04:58.880 INFO:journalctl@ceph.mds.e.smithi072.stdout:May 09 05:04:58 smithi072 bash[71392]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-e 2024-05-09T05:04:59.067 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.e.service' 2024-05-09T05:04:59.100 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:04:59.101 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-09T05:04:59.101 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-09T05:04:59.101 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.h 2024-05-09T05:04:59.421 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 05:04:59 smithi072 systemd[1]: Stopping Ceph mds.h for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:04:59.880 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 05:04:59 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-h[63443]: 2024-05-09T05:04:59.420+0000 7fe06e2e4700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:04:59.880 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 05:04:59 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-h[63443]: 2024-05-09T05:04:59.420+0000 7fe06e2e4700 -1 mds.h *** got signal Terminated *** 2024-05-09T05:05:02.566 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 05:05:02 smithi072 podman[71524]: 2024-05-09 05:05:02.259586167 +0000 UTC m=+2.944241389 container died 9e50b6586128f8ca3a64904545d6a5ae8301a395b8864801d4400e224dfe17ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-h, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:05:02.880 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 05:05:02 smithi072 podman[71524]: 2024-05-09 05:05:02.674201751 +0000 UTC m=+3.358856978 container cleanup 9e50b6586128f8ca3a64904545d6a5ae8301a395b8864801d4400e224dfe17ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-h, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-05-09T05:05:02.880 INFO:journalctl@ceph.mds.h.smithi072.stdout:May 09 05:05:02 smithi072 bash[71524]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-h 2024-05-09T05:05:03.026 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.h.service' 2024-05-09T05:05:03.058 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:05:03.058 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-09T05:05:03.058 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-09T05:05:03.058 DEBUG:teuthology.orchestra.run.smithi072:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.k 2024-05-09T05:05:03.371 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 09 05:05:03 smithi072 systemd[1]: Stopping Ceph mds.k for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:05:03.630 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 09 05:05:03 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-k[62806]: 2024-05-09T05:05:03.370+0000 7f9aa2c77700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:05:03.630 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 09 05:05:03 smithi072 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-k[62806]: 2024-05-09T05:05:03.370+0000 7f9aa2c77700 -1 mds.k *** got signal Terminated *** 2024-05-09T05:05:06.698 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 09 05:05:06 smithi072 podman[71645]: 2024-05-09 05:05:06.426019075 +0000 UTC m=+3.148222962 container died 83a0c56c6f0a5346eaeb8d34939fc3d7e15cb91a7c4abdb3e773c4184f3b83d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-09T05:05:07.034 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 09 05:05:06 smithi072 podman[71645]: 2024-05-09 05:05:06.791049401 +0000 UTC m=+3.513253296 container cleanup 83a0c56c6f0a5346eaeb8d34939fc3d7e15cb91a7c4abdb3e773c4184f3b83d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS) 2024-05-09T05:05:07.034 INFO:journalctl@ceph.mds.k.smithi072.stdout:May 09 05:05:06 smithi072 bash[71645]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-k 2024-05-09T05:05:07.171 DEBUG:teuthology.orchestra.run.smithi072:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.k.service' 2024-05-09T05:05:07.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:05:07.203 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-09T05:05:07.203 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-09T05:05:07.203 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.c 2024-05-09T05:05:07.497 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 05:05:07 smithi110 systemd[1]: Stopping Ceph mds.c for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:05:07.762 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 05:05:07 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-c[65218]: 2024-05-09T05:05:07.496+0000 7ff2af04f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:05:07.762 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 05:05:07 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-c[65218]: 2024-05-09T05:05:07.496+0000 7ff2af04f700 -1 mds.c *** got signal Terminated *** 2024-05-09T05:05:10.557 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 05:05:10 smithi110 podman[72972]: 2024-05-09 05:05:10.275057782 +0000 UTC m=+2.880799553 container died f31847814bf1ad0722d8b0e6b3f95f313e6ad1f0825926285999d7eeb332b6f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-c, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-09T05:05:10.947 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 05:05:10 smithi110 podman[72972]: 2024-05-09 05:05:10.688185366 +0000 UTC m=+3.293927135 container cleanup f31847814bf1ad0722d8b0e6b3f95f313e6ad1f0825926285999d7eeb332b6f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-c, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-05-09T05:05:10.947 INFO:journalctl@ceph.mds.c.smithi110.stdout:May 09 05:05:10 smithi110 bash[72972]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-c 2024-05-09T05:05:11.159 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.c.service' 2024-05-09T05:05:11.195 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:05:11.195 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-09T05:05:11.195 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-09T05:05:11.195 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.f 2024-05-09T05:05:11.508 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 05:05:11 smithi110 systemd[1]: Stopping Ceph mds.f for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:05:11.807 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 05:05:11 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-f[64908]: 2024-05-09T05:05:11.508+0000 7f0744b96700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:05:11.807 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 05:05:11 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-f[64908]: 2024-05-09T05:05:11.508+0000 7f0744b96700 -1 mds.f *** got signal Terminated *** 2024-05-09T05:05:14.542 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 05:05:14 smithi110 podman[73104]: 2024-05-09 05:05:14.266667887 +0000 UTC m=+2.853596328 container died 6c87c4863332aacc042950c0fcd796fad9c54d1c1bcc878a3cf9c8d816ac7d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-09T05:05:14.807 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 05:05:14 smithi110 podman[73104]: 2024-05-09 05:05:14.625313811 +0000 UTC m=+3.212242242 container cleanup 6c87c4863332aacc042950c0fcd796fad9c54d1c1bcc878a3cf9c8d816ac7d52 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-09T05:05:14.807 INFO:journalctl@ceph.mds.f.smithi110.stdout:May 09 05:05:14 smithi110 bash[73104]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-f 2024-05-09T05:05:15.060 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.f.service' 2024-05-09T05:05:15.091 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:05:15.092 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-09T05:05:15.092 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-09T05:05:15.092 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.i 2024-05-09T05:05:15.557 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 05:05:15 smithi110 systemd[1]: Stopping Ceph mds.i for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:05:15.557 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 05:05:15 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-i[65526]: 2024-05-09T05:05:15.413+0000 7f5661e0d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:05:15.557 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 05:05:15 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-i[65526]: 2024-05-09T05:05:15.413+0000 7f5661e0d700 -1 mds.i *** got signal Terminated *** 2024-05-09T05:05:18.557 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 05:05:18 smithi110 podman[73225]: 2024-05-09 05:05:18.264328003 +0000 UTC m=+2.952697857 container died d7005ada0039d91e4762487439d792142af3fdb3b24ae6e8b499003bc860375d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-09T05:05:18.996 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 05:05:18 smithi110 podman[73225]: 2024-05-09 05:05:18.721163486 +0000 UTC m=+3.409533342 container cleanup d7005ada0039d91e4762487439d792142af3fdb3b24ae6e8b499003bc860375d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-i, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=) 2024-05-09T05:05:18.997 INFO:journalctl@ceph.mds.i.smithi110.stdout:May 09 05:05:18 smithi110 bash[73225]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-i 2024-05-09T05:05:19.146 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.i.service' 2024-05-09T05:05:19.177 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:05:19.177 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-09T05:05:19.177 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-09T05:05:19.177 DEBUG:teuthology.orchestra.run.smithi110:> sudo systemctl stop ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.l 2024-05-09T05:05:19.558 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 09 05:05:19 smithi110 systemd[1]: Stopping Ceph mds.l for ecf9ae2c-0dbd-11ef-bc97-c7b262605968... 2024-05-09T05:05:19.558 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 09 05:05:19 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-l[64591]: 2024-05-09T05:05:19.474+0000 7f3a3dbba700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-09T05:05:19.558 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 09 05:05:19 smithi110 ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-l[64591]: 2024-05-09T05:05:19.474+0000 7f3a3dbba700 -1 mds.l *** got signal Terminated *** 2024-05-09T05:05:23.800 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 09 05:05:23 smithi110 podman[73344]: 2024-05-09 05:05:23.405147055 +0000 UTC m=+4.010640880 container died 7f7cc7082be409806ee37dd6e4406ff6a44d4504c724de12fd5a17ac6754b672 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-l, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404) 2024-05-09T05:05:24.057 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 09 05:05:23 smithi110 podman[73344]: 2024-05-09 05:05:23.899496365 +0000 UTC m=+4.504990188 container cleanup 7f7cc7082be409806ee37dd6e4406ff6a44d4504c724de12fd5a17ac6754b672 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:ef3f584d871dd6119fe1ccb29d0b8b624ec6e3462fcf3f49b8c928dd6ba67584, name=ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-l, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240508.183908-debug-c9da404, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-09T05:05:24.057 INFO:journalctl@ceph.mds.l.smithi110.stdout:May 09 05:05:23 smithi110 bash[73344]: ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968-mds-l 2024-05-09T05:05:24.257 DEBUG:teuthology.orchestra.run.smithi110:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ecf9ae2c-0dbd-11ef-bc97-c7b262605968@mds.l.service' 2024-05-09T05:05:24.288 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-09T05:05:24.288 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-09T05:05:24.288 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T05:05:24.490 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T05:05:26.991 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T05:05:27.181 INFO:teuthology.orchestra.run.smithi072.stdout:Deleting cluster with fsid: ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T05:05:29.595 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-09T05:05:29.785 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T05:05:32.129 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T05:05:32.164 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T05:05:32.195 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-09T05:05:32.221 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-09T05:05:32.223 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699071/remote/smithi046/crash 2024-05-09T05:05:32.224 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/crash -- . 2024-05-09T05:05:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:tar: /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T05:05:32.264 INFO:teuthology.orchestra.run.smithi046.stderr:tar: Error is not recoverable: exiting now 2024-05-09T05:05:32.266 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699071/remote/smithi072/crash 2024-05-09T05:05:32.267 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/crash -- . 2024-05-09T05:05:32.310 INFO:teuthology.orchestra.run.smithi072.stderr:tar: /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T05:05:32.310 INFO:teuthology.orchestra.run.smithi072.stderr:tar: Error is not recoverable: exiting now 2024-05-09T05:05:32.312 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699071/remote/smithi110/crash 2024-05-09T05:05:32.312 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/crash -- . 2024-05-09T05:05:32.355 INFO:teuthology.orchestra.run.smithi110.stderr:tar: /var/lib/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-09T05:05:32.355 INFO:teuthology.orchestra.run.smithi110.stderr:tar: Error is not recoverable: exiting now 2024-05-09T05:05:32.356 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-09T05:05:32.356 DEBUG:teuthology.orchestra.run.smithi046:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-05-09T05:05:32.396 INFO:tasks.cephadm:Compressing logs... 2024-05-09T05:05:32.396 DEBUG:teuthology.orchestra.run.smithi046:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T05:05:32.439 DEBUG:teuthology.orchestra.run.smithi072:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T05:05:32.441 DEBUG:teuthology.orchestra.run.smithi110:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-09T05:05:32.465 INFO:teuthology.orchestra.run.smithi110.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.log 2024-05-09T05:05:32.465 INFO:teuthology.orchestra.run.smithi110.stderr:: No such file or directory 2024-05-09T05:05:32.465 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-09T05:05:32.465 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-09T05:05:32.465 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T05:05:32.466 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-09T05:05:32.466 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-09T05:05:32.466 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-09T05:05:32.469 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-09T05:05:32.470 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-09T05:05:32.470 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.z.log: 87.6% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T05:05:32.470 INFO:teuthology.orchestra.run.smithi110.stderr: 90.4% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T05:05:32.474 INFO:teuthology.orchestra.run.smithi072.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.log 2024-05-09T05:05:32.474 INFO:teuthology.orchestra.run.smithi072.stderr:: No such file or directory 2024-05-09T05:05:32.475 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T05:05:32.475 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose --/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-09T05:05:32.475 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-09T05:05:32.475 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-09T05:05:32.475 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-09T05:05:32.475 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log: /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.7.log: /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.b.log: /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-09T05:05:32.475 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-09T05:05:32.475 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.y.log: /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.h.log: 87.5% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T05:05:32.475 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-09T05:05:32.475 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T05:05:32.475 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.10.log: 90.4% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T05:05:32.480 INFO:teuthology.orchestra.run.smithi046.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2024-05-09T05:05:32.480 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.log 2024-05-09T05:05:32.480 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-09T05:05:32.480 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-09T05:05:32.480 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-09T05:05:32.480 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-09T05:05:32.480 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-09T05:05:32.481 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-09T05:05:32.481 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-09T05:05:32.481 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-09T05:05:32.481 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mon.a.log: 87.5% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-09T05:05:32.481 INFO:teuthology.orchestra.run.smithi046.stderr: 90.3% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-09T05:05:32.481 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T05:05:32.481 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-09T05:05:32.481 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log: 84.0% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T05:05:32.481 INFO:teuthology.orchestra.run.smithi046.stderr: 94.5% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-09T05:05:32.481 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T05:05:32.481 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.a.log: 94.7%gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-09T05:05:32.481 INFO:teuthology.orchestra.run.smithi046.stderr: -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-09T05:05:32.481 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log: 94.5% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-09T05:05:32.481 INFO:teuthology.orchestra.run.smithi046.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-09T05:05:32.481 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T05:05:32.482 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-09T05:05:32.483 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T05:05:32.483 INFO:teuthology.orchestra.run.smithi072.stderr: 92.0% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-09T05:05:32.483 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T05:05:32.483 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-09T05:05:32.484 INFO:teuthology.orchestra.run.smithi110.stderr: 92.1% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-09T05:05:32.484 INFO:teuthology.orchestra.run.smithi110.stderr:gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-09T05:05:32.484 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-09T05:05:32.484 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-09T05:05:32.484 INFO:teuthology.orchestra.run.smithi110.stderr: 83.7% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-09T05:05:32.485 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.9.log: 94.6% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-09T05:05:32.487 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-09T05:05:32.487 INFO:teuthology.orchestra.run.smithi046.stderr:/var/log/ceph/cephadm.log: 91.1% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T05:05:32.491 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.e.log: /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-09T05:05:32.492 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T05:05:32.493 INFO:teuthology.orchestra.run.smithi072.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T05:05:32.494 INFO:teuthology.orchestra.run.smithi046.stderr: 94.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T05:05:32.499 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.k.log: 94.4% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-09T05:05:32.499 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-09T05:05:32.501 INFO:teuthology.orchestra.run.smithi072.stderr:/var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T05:05:32.512 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-09T05:05:32.518 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.l.log: 94.6% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-09T05:05:32.526 INFO:teuthology.orchestra.run.smithi110.stderr: 91.0% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-09T05:05:32.528 INFO:teuthology.orchestra.run.smithi110.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-09T05:05:32.534 INFO:teuthology.orchestra.run.smithi110.stderr: 94.6% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-09T05:05:32.534 INFO:teuthology.orchestra.run.smithi110.stderr: 94.5% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-09T05:05:32.542 INFO:teuthology.orchestra.run.smithi072.stderr: 92.6% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-09T05:05:32.553 INFO:teuthology.orchestra.run.smithi110.stderr: 92.6% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-09T05:05:32.778 INFO:teuthology.orchestra.run.smithi072.stderr: 96.2% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-09T05:05:32.863 INFO:teuthology.orchestra.run.smithi046.stderr: 90.1% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-09T05:05:32.959 INFO:teuthology.orchestra.run.smithi110.stderr: 91.9% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-09T05:05:32.992 INFO:teuthology.orchestra.run.smithi072.stderr: 91.8% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-09T05:05:33.389 INFO:teuthology.orchestra.run.smithi046.stderr: 90.3% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-09T05:05:34.985 INFO:teuthology.orchestra.run.smithi072.stderr: 94.7% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-09T05:05:36.809 INFO:teuthology.orchestra.run.smithi046.stderr: 93.4% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-09T05:05:37.716 INFO:teuthology.orchestra.run.smithi110.stderr: 93.4% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-09T05:05:38.062 INFO:teuthology.orchestra.run.smithi072.stderr: 93.5% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-09T05:05:38.063 INFO:teuthology.orchestra.run.smithi110.stderr: 93.5% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-09T05:05:38.161 INFO:teuthology.orchestra.run.smithi110.stderr: 93.7% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-09T05:05:38.263 INFO:teuthology.orchestra.run.smithi046.stderr: 93.5% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-09T05:05:38.432 INFO:teuthology.orchestra.run.smithi046.stderr: 93.5% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-09T05:05:38.631 INFO:teuthology.orchestra.run.smithi072.stderr: 93.3% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-09T05:05:39.062 INFO:teuthology.orchestra.run.smithi072.stderr: 93.3% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-09T05:05:39.273 INFO:teuthology.orchestra.run.smithi046.stderr: 93.5% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-09T05:05:39.275 INFO:teuthology.orchestra.run.smithi046.stderr: 2024-05-09T05:05:39.275 INFO:teuthology.orchestra.run.smithi046.stderr:real 0m6.822s 2024-05-09T05:05:39.275 INFO:teuthology.orchestra.run.smithi046.stderr:user 0m23.462s 2024-05-09T05:05:39.275 INFO:teuthology.orchestra.run.smithi046.stderr:sys 0m0.797s 2024-05-09T05:05:39.360 INFO:teuthology.orchestra.run.smithi110.stderr: 93.3% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-09T05:05:39.362 INFO:teuthology.orchestra.run.smithi110.stderr: 2024-05-09T05:05:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:real 0m6.907s 2024-05-09T05:05:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:user 0m23.274s 2024-05-09T05:05:39.362 INFO:teuthology.orchestra.run.smithi110.stderr:sys 0m0.837s 2024-05-09T05:05:39.571 INFO:teuthology.orchestra.run.smithi072.stderr: 93.4% -- replaced with /var/log/ceph/ecf9ae2c-0dbd-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-09T05:05:39.573 INFO:teuthology.orchestra.run.smithi072.stderr: 2024-05-09T05:05:39.573 INFO:teuthology.orchestra.run.smithi072.stderr:real 0m7.118s 2024-05-09T05:05:39.573 INFO:teuthology.orchestra.run.smithi072.stderr:user 0m27.828s 2024-05-09T05:05:39.573 INFO:teuthology.orchestra.run.smithi072.stderr:sys 0m0.992s 2024-05-09T05:05:39.573 INFO:tasks.cephadm:Archiving logs... 2024-05-09T05:05:39.574 DEBUG:teuthology.misc:Transferring archived files from smithi046:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699071/remote/smithi046/log 2024-05-09T05:05:39.575 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T05:05:43.162 DEBUG:teuthology.misc:Transferring archived files from smithi072:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699071/remote/smithi072/log 2024-05-09T05:05:43.163 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T05:05:47.206 DEBUG:teuthology.misc:Transferring archived files from smithi110:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699071/remote/smithi110/log 2024-05-09T05:05:47.207 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-09T05:05:50.843 INFO:tasks.cephadm:Removing cluster... 2024-05-09T05:05:50.843 DEBUG:teuthology.orchestra.run.smithi046:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 --force 2024-05-09T05:05:51.046 INFO:teuthology.orchestra.run.smithi046.stdout:Deleting cluster with fsid: ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T05:05:51.337 DEBUG:teuthology.orchestra.run.smithi072:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 --force 2024-05-09T05:05:51.526 INFO:teuthology.orchestra.run.smithi072.stdout:Deleting cluster with fsid: ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T05:05:51.817 DEBUG:teuthology.orchestra.run.smithi110:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ecf9ae2c-0dbd-11ef-bc97-c7b262605968 --force 2024-05-09T05:05:52.007 INFO:teuthology.orchestra.run.smithi110.stdout:Deleting cluster with fsid: ecf9ae2c-0dbd-11ef-bc97-c7b262605968 2024-05-09T05:05:52.286 INFO:tasks.cephadm:Removing cephadm ... 2024-05-09T05:05:52.286 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T05:05:52.306 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T05:05:52.323 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-09T05:05:52.338 INFO:tasks.cephadm:Teardown complete 2024-05-09T05:05:52.338 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-09T05:05:52.350 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-09T05:05:52.351 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T05:05:52.353 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T05:05:52.365 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-09T05:05:52.476 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-09T05:05:52.488 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-09T05:05:52.488 DEBUG:teuthology.orchestra.run.smithi046:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T05:05:52.490 DEBUG:teuthology.orchestra.run.smithi072:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T05:05:52.493 DEBUG:teuthology.orchestra.run.smithi110:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-09T05:05:52.505 INFO:teuthology.orchestra.run.smithi072.stderr:bash: line 1: ntpq: command not found 2024-05-09T05:05:52.505 INFO:teuthology.orchestra.run.smithi046.stderr:bash: line 1: ntpq: command not found 2024-05-09T05:05:52.506 INFO:teuthology.orchestra.run.smithi110.stderr:bash: line 1: ntpq: command not found 2024-05-09T05:05:52.510 INFO:teuthology.orchestra.run.smithi072.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T05:05:52.510 INFO:teuthology.orchestra.run.smithi072.stdout:=============================================================================== 2024-05-09T05:05:52.510 INFO:teuthology.orchestra.run.smithi072.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 +2907us[+2709us] +/- 44ms 2024-05-09T05:05:52.510 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 54 -2159us[-2159us] +/- 50ms 2024-05-09T05:05:52.510 INFO:teuthology.orchestra.run.smithi072.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -1941us[-1941us] +/- 70ms 2024-05-09T05:05:52.510 INFO:teuthology.orchestra.run.smithi072.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:05:52.511 INFO:teuthology.orchestra.run.smithi046.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T05:05:52.511 INFO:teuthology.orchestra.run.smithi046.stdout:=============================================================================== 2024-05-09T05:05:52.511 INFO:teuthology.orchestra.run.smithi046.stdout:^* hv01.front.sepia.ceph.com 3 6 377 56 +2909us[+2641us] +/- 44ms 2024-05-09T05:05:52.511 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 56 -2162us[-2162us] +/- 50ms 2024-05-09T05:05:52.511 INFO:teuthology.orchestra.run.smithi046.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 55 -1917us[-1917us] +/- 70ms 2024-05-09T05:05:52.511 INFO:teuthology.orchestra.run.smithi046.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:05:52.512 INFO:teuthology.orchestra.run.smithi110.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-09T05:05:52.512 INFO:teuthology.orchestra.run.smithi110.stdout:=============================================================================== 2024-05-09T05:05:52.512 INFO:teuthology.orchestra.run.smithi110.stdout:^* hv01.front.sepia.ceph.com 3 6 377 59 +2885us[+2628us] +/- 44ms 2024-05-09T05:05:52.512 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 55 -2167us[-2167us] +/- 50ms 2024-05-09T05:05:52.512 INFO:teuthology.orchestra.run.smithi110.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -1950us[-1950us] +/- 70ms 2024-05-09T05:05:52.512 INFO:teuthology.orchestra.run.smithi110.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-09T05:05:52.513 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-09T05:05:52.524 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-09T05:05:52.525 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-09T05:05:52.562 DEBUG:teuthology.orchestra.run.smithi046:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T05:05:52.653 DEBUG:teuthology.orchestra.run.smithi072:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T05:05:52.718 DEBUG:teuthology.orchestra.run.smithi110:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-09T05:05:52.783 DEBUG:teuthology.orchestra.run.smithi046:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T05:05:52.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:05:52.815 DEBUG:teuthology.orchestra.run.smithi072:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T05:05:52.842 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:05:52.842 DEBUG:teuthology.orchestra.run.smithi110:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-09T05:05:52.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:05:52.869 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-09T05:05:52.880 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-09T05:05:52.912 INFO:teuthology.task.internal:Duration was 2070.238575 seconds 2024-05-09T05:05:52.912 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-09T05:05:52.961 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-09T05:05:52.962 DEBUG:teuthology.orchestra.run.smithi046:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T05:05:52.964 DEBUG:teuthology.orchestra.run.smithi072:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T05:05:52.967 DEBUG:teuthology.orchestra.run.smithi110:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-09T05:05:53.000 INFO:teuthology.orchestra.run.smithi072.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T05:05:53.003 INFO:teuthology.orchestra.run.smithi110.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T05:05:53.005 INFO:teuthology.orchestra.run.smithi046.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-09T05:05:53.416 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-09T05:05:53.417 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi046.front.sepia.ceph.com 2024-05-09T05:05:53.417 DEBUG:teuthology.orchestra.run.smithi046:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T05:05:53.442 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi072.front.sepia.ceph.com 2024-05-09T05:05:53.442 DEBUG:teuthology.orchestra.run.smithi072:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T05:05:53.482 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi110.front.sepia.ceph.com 2024-05-09T05:05:53.482 DEBUG:teuthology.orchestra.run.smithi110:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-09T05:05:53.507 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-09T05:05:53.507 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T05:05:53.510 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T05:05:53.525 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-09T05:05:53.572 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-09T05:05:53.573 DEBUG:teuthology.orchestra.run.smithi046:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T05:05:53.582 DEBUG:teuthology.orchestra.run.smithi072:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T05:05:53.589 DEBUG:teuthology.orchestra.run.smithi110:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-09T05:05:54.716 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-09T05:05:54.729 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-09T05:05:54.729 DEBUG:teuthology.orchestra.run.smithi046:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T05:05:54.756 DEBUG:teuthology.orchestra.run.smithi072:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T05:05:54.783 DEBUG:teuthology.orchestra.run.smithi110:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-09T05:05:54.811 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-09T05:05:54.824 DEBUG:teuthology.orchestra.run.smithi046:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T05:05:54.826 DEBUG:teuthology.orchestra.run.smithi072:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T05:05:54.829 DEBUG:teuthology.orchestra.run.smithi110:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-09T05:05:54.848 INFO:teuthology.orchestra.run.smithi046.stdout:kernel.core_pattern = core 2024-05-09T05:05:54.851 INFO:teuthology.orchestra.run.smithi072.stdout:kernel.core_pattern = core 2024-05-09T05:05:54.875 INFO:teuthology.orchestra.run.smithi110.stdout:kernel.core_pattern = core 2024-05-09T05:05:54.905 DEBUG:teuthology.orchestra.run.smithi046:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T05:05:54.920 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:05:54.920 DEBUG:teuthology.orchestra.run.smithi072:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T05:05:54.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:05:54.946 DEBUG:teuthology.orchestra.run.smithi110:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-09T05:05:54.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-09T05:05:54.961 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-09T05:05:54.973 INFO:teuthology.task.internal:Transferring archived files... 2024-05-09T05:05:54.974 DEBUG:teuthology.misc:Transferring archived files from smithi046:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699071/remote/smithi046 2024-05-09T05:05:54.974 DEBUG:teuthology.orchestra.run.smithi046:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T05:05:55.037 DEBUG:teuthology.misc:Transferring archived files from smithi072:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699071/remote/smithi072 2024-05-09T05:05:55.038 DEBUG:teuthology.orchestra.run.smithi072:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T05:05:55.074 DEBUG:teuthology.misc:Transferring archived files from smithi110:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-05-08_22:06:20-fs-wip-pdonnell-testing-20240508.183908-debug-distro-default-smithi/7699071/remote/smithi110 2024-05-09T05:05:55.074 DEBUG:teuthology.orchestra.run.smithi110:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-09T05:05:55.114 INFO:teuthology.task.internal:Removing archive directory... 2024-05-09T05:05:55.114 DEBUG:teuthology.orchestra.run.smithi046:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T05:05:55.117 DEBUG:teuthology.orchestra.run.smithi072:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T05:05:55.119 DEBUG:teuthology.orchestra.run.smithi110:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-09T05:05:55.165 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-09T05:05:55.180 INFO:teuthology.task.internal:Not uploading archives. 2024-05-09T05:05:55.180 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-09T05:05:55.206 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-09T05:05:55.206 DEBUG:teuthology.orchestra.run.smithi046:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T05:05:55.209 DEBUG:teuthology.orchestra.run.smithi072:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T05:05:55.211 DEBUG:teuthology.orchestra.run.smithi110:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-09T05:05:55.222 INFO:teuthology.orchestra.run.smithi046.stdout: 265299 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 05:05 /home/ubuntu/cephtest 2024-05-09T05:05:55.224 INFO:teuthology.orchestra.run.smithi072.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 05:05 /home/ubuntu/cephtest 2024-05-09T05:05:55.226 INFO:teuthology.orchestra.run.smithi110.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 9 05:05 /home/ubuntu/cephtest 2024-05-09T05:05:55.227 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-09T05:05:55.256 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-09T05:05:55.358 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/no 6-workunit/suites/iozone}} duration: 2070.2385749816895 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-05-09T05:05:55.358 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-09T05:05:55.451 INFO:teuthology.run:pass